feat(plugin): a recognized retrieval miss has a route, and the record comes first (#4133)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 50s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / Python tests (push) Successful in 1m34s
CI & Build / Build & push image (push) Successful in 43s

The tuning loop shipped in steps 4 and 6 and logged zero events in its
lifetime. `tune_retrieval`, `retrieval_telemetry` and `retrieval_surfaces`
appeared on no instruction surface at all — not the skills, not the hooks,
not the MCP instructions — so the decision that "the model should be the
thing handling it 9 times out of 10" could not begin to happen.

What was missing was not an auditor but a route. `using-scribe` now carries
it, ordered: read the refused records, fix the trigger, and only then
consider the dial. The order is the content. A rule's `when_to_apply` IS
the text its score is computed against, so a miss is evidence about that
text first; rewording one trigger changes one rule's reach, while moving a
floor changes what every record on the surface does and cannot tell a
badly-worded trigger from a genuinely distant one.

Measured, and the reason the order is asserted rather than suggested: rule
1 scored 0.6515 and ranked 5th for the moment it governed, behind three
rules that restrained the same act. Every percentile said "lower the
floor"; at 0.60 the arm delivered those three restraints and still not rule
1. Rewriting the trigger to lead with the symptom put it 1st at 0.7130.

Also: the create path gets a precondition. A new record is itself a
retrieval-affecting act, so before writing one, what_might_apply asks what
already covers that moment — fifty candidates and no bar, because a bar is
what lets the existing record hide.

`_INSTRUCTIONS` gets one index line, not the route: 1,986 of 2,000
characters, since Claude Code cuts the rest mid-word (#2562).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
This commit is contained in:
2026-09-18 00:37:04 -04:00
co-authored by Claude Opus 5
parent a7d736860f
commit 104c1d6f37
5 changed files with 226 additions and 1 deletions
+56
View File
@@ -318,6 +318,19 @@ with no trigger is not a quiet rule, it is an unreachable one. Write the moment
in the words a session actually produces — the command, the error, the
half-formed ask — not the category it belongs to.
**Before writing one, ask what already covers that moment.**
`what_might_apply("the moment you are about to write a record for")` — fifty
candidates and no bar, so an existing record cannot hide under a threshold the
way it can from `search`. When something already covers the moment, the reach
is that record: improve its trigger or its statement rather than standing a
second one beside it. Two records describing the same moment compete in one
ranked list against one budget, and the slot they take from each other is the
third candidate that would have said something different. Two records may
legitimately share a moment and say *different* things — a rule for what must
happen, a preference for how to report it. What this catches is the same thing
said twice at two strengths, which is worse than either alone: a session that
retrieves the softer copy has been told that binding guidance is optional.
**First ask whether it's a rule at all.** A rule is prose you have to remember
and apply; Scribe's other entities are structure a tool can resolve and check.
Visual standards belong in a **design system**, not a rulebook — a token can be
@@ -326,6 +339,49 @@ and none of that survives being written as a rule. A repeatable procedure is a
**process**; reusable code is a **snippet**. Reach for a rule when the thing
really is a standing instruction about how to work.
## When a record doesn't reach the moment it should
Retrieval misjudging is ordinary, and it is fixable — but only by whoever
notices. **Either direction counts:** a rule that should have governed a moment
and never arrived, and a rule that arrives on every turn and never applies. So
does **either noticer**: the operator saying *"that should have fired"*, and you
noticing it yourself — you reached for a rule nobody offered you, or you were
handed the same rule five times and set it aside five times.
**Take it to the record first and the dial second.** A rule's `when_to_apply`
IS the text its similarity score is computed against, so when a rule misses a
moment it governs, the overwhelmingly likely cause is that its trigger does not
describe that moment in the words a session actually produces. Rewording one
trigger changes one rule's reach. Moving a floor changes what every record on
that surface does, and a floor cannot tell a badly-worded trigger from a
genuinely distant record — so one lowered to rescue a single rule admits
everything else that was sitting in the same band.
1. **Read the refused records.** `retrieval_telemetry(days=N,
near_miss_samples=5)` names by id what each surface refused and by how much.
Open them with `get_rule` / `get_note`. This is the step that carries the
answer: the statistic says a record was close, and only the record says
whether it was *right*.
2. **Fix the trigger.** `update_rule(when_to_apply=...)`, written as the
symptom — what the session was doing or saying at the moment it needed this
rule — not the situation the rule belongs to. Then check that it worked:
`what_might_apply("the moment, in the operator's own words")` and read where
the rule now ranks. The change is measurable, so measure it, and say the
before and after when you report it.
3. **Then consider the dial.** `retrieval_surfaces` shows what is in force per
arm and whether the number is still calibrated; `tune_retrieval` moves it.
`reason` is required and has to say what you read, because it is what lets
the operator disagree with a number they did not choose.
Reaching for `tune_retrieval` before opening a single record is the wrong move,
and it is the one that feels efficient. Worked example, measured on this
install: a rule granting a routine push scored 0.6515 and ranked 5th for the
moment it governed, behind three rules that *restrained* the same act. Every
percentile said "lower the floor" — and lowering it would have delivered those
three restraints and still not the rule. Rewriting the trigger to lead with the
symptom moved the same rule to 1st at 0.7130, ahead of all three. Only then was
the floor worth touching.
## Building UI: the project's design system binds
`enter_project` names the project's `design_system` when it has one. Before