No heading-only chunks, agent-message skip, refusals reach the agent (#4784, #4785, #4794) #198

Merged

bvandeusen merged 3 commits from dev into main 2026-10-03 21:47:03 -04:00

Conversation 0 Commits 3 Files Changed 7

+243 -22

bvandeusen commented 2026-10-03 21:47:00 -04:00

Owner

Copy Link

Copy Source

Three defects found by the first judged auto-inject review (#4772):

• #4784 "A chunk holding only a heading ranks first on generic queries" — no chunk under 160 chars beyond its headings; CHUNKER_VERSION 2→3, so the startup backfill re-embeds the corpus.
• #4785 "Subagent hand-backs are scored as operator prompts" — <agent-message> joins the skip list; plugin 2026.10.04.0125.
• #4794 "Every refusal a Scribe tool writes reaches the agent as a bare 'Error executing tool X'" — a ValueError refusal is re-raised as a ToolError carrying its message; crashes stay masked.

Dev CI runs 7979 and 7980 green.

🤖 Generated with Claude Code

Three defects found by the first judged auto-inject review (#4772): - #4784 "A chunk holding only a heading ranks first on generic queries" — no chunk under 160 chars beyond its headings; CHUNKER_VERSION 2→3, so the startup backfill re-embeds the corpus. - #4785 "Subagent hand-backs are scored as operator prompts" — `<agent-message>` joins the skip list; plugin 2026.10.04.0125. - #4794 "Every refusal a Scribe tool writes reaches the agent as a bare 'Error executing tool X'" — a ValueError refusal is re-raised as a ToolError carrying its message; crashes stay masked. Dev CI runs 7979 and 7980 green. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 3 commits 2026-10-03 21:47:00 -04:00

fix(embeddings): no chunk is a heading alone (#4784) ... 78fa01b025

Judging live auto-inject menus (#4772) found "## Work log — 2026-08-21",
"## Findings worth carrying forward" and "Two dev→main PRs this session."
each embedded as a whole chunk. Text about nothing embeds close to
everything, so they took ranks 1–3 on vague queries and pushed real records
down.

Three ways the chunker made them, each closed:
- _split_paragraphs flushed a heading on its own when the paragraph under it
  was long. A thin lead is now carried into the paragraph that follows, and
  a hard cut never lands in the first quarter of the budget, where the
  newline it finds closes a heading.
- A heading with no body (above its ### parts) became a section of its own
  when the section before it was full. A thin section now takes the next.
- A short lead-in or a short last section stood alone. The lead-in joins
  what follows; a thin tail joins what precedes it.

A continuation piece now repeats the LAST heading before it, since one
section can now hold several. CHUNKER_VERSION 2 → 3, so the startup backfill
re-embeds; tuned dials will report their shape as changed, which is true.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

fix(plugin): a subagent hand-back is not an operator prompt (#4785) ... df4b37673f

`<agent-message …>` turns were scored by auto-inject and the prompt rule
arm as if the operator had typed them, spending a retrieval and writing a
retrieval_logs row that reads as a real message (log 54355, found by the
#4772 review). Same class as #4142. Matched up to the tag name, since the
tag carries attributes; `<agent-messages>` and the tag named mid-sentence
are still retrieved against.

Plugin 2026.10.04.0125.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

fix(mcp): a tool's refusal reaches the agent with its reason (#4794) ... 5c6d9a7b55

SDK 2.x passes on only a ToolError's text. Any other exception becomes
UnexpectedToolError("Error executing tool X") and its message stays on the
server. ValueError is how every Scribe tool refuses — what was refused, why,
what to do instead — so every refusal reached the agent bare, and it could
only retry blind. Seen live: tune_retrieval(actor="operator") and
judge_menu(verdicts=[]) both answered "Error executing tool …" and nothing
else.

StrictArgsMCPServer.call_tool re-raises an UnexpectedToolError caused by a
ValueError as a ToolError carrying the message, in the SDK's own
"Error executing tool X: <reason>" shape. Other exceptions are crashes and
stay masked. The stale comment claiming the SDK returns ValueError text is
corrected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

bvandeusen merged commit a19c344770 into main 2026-10-03 21:47:03 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

renovate-bot bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledScribe#198