docs(6209): wide net, startup, stamps, slippage, miss route and parity tests
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Build & push image (push) Canceled after 0s
CI & Build / Python tests (push) Canceled after 1m23s

Eight more test files state the property they pin as a present-tense
fact. Incident narratives (a dated storage stall, a week of zero tuning
events, a two-project poisoned ledger, a twenty-site owner audit) become
the failure shape each test defends; one-install figures that name a rule
become a general worked example with the calibration numbers kept. The
family-grouping regression guard is renamed for what it checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-21 21:27:17 -04:00
co-authored by Claude Opus 5.5
parent d41e72d26c
commit ff08500128
8 changed files with 154 additions and 196 deletions
+20 -27
View File
@@ -1,18 +1,13 @@
"""A recognized retrieval miss has a route, and the route is ordered (#4133).
"""A recognized retrieval miss has a route, and the route is ordered.
WHY THIS EXISTS
Milestone 416 built the tuning loop across steps 4 and 6 — `retrieval_surfaces`
to read a dial, `tune_retrieval` to move one with its argument attached,
`migrate_retrieval_floor` to carry it across a model change — and then measured,
on 2026-09-17, that `retrieval_tuning_history` returned **zero events**. Not one
call, ever.
The cause was not a tool being wrong. The three tools appeared on NO instruction
surface: not the skills, not the hooks, not the MCP `_INSTRUCTIONS`. They were
classified in `server.py`'s read/write lists and named nowhere a session would
read them. The operator's decision for #4102 — *"the model should be the thing
handling it 9 times out of 10"* — could not begin to happen.
The tuning loop is three tools — `retrieval_surfaces` to read a dial,
`tune_retrieval` to move one with its argument attached,
`migrate_retrieval_floor` to carry it across a model change — and the model is
meant to be the one using them most of the time. A tool classified in
`server.py`'s read/write lists and named on no instruction surface is never
called: nothing a session reads tells it the tool exists.
So this file guards a ROUTE, not a behaviour: that a session which notices a
rule missed the moment it governed has somewhere written down to take it.
@@ -27,13 +22,12 @@ computed against, so a miss is evidence about that text first; a floor is one
number governing every record on the surface and cannot tell a badly-worded
trigger from a genuinely distant one.
Measured on this install, which is why the order is asserted rather than
suggested: rule 1 "`dev` is home" scored 0.6515 and ranked 5th for the moment it
governed, behind three rules that RESTRAINED the same act. Every percentile said
"lower the floor" — and at 0.60 the arm delivered those three restraints and
still not rule 1. Rewriting the trigger to lead with the symptom put the same
rule 1st at 0.7130. The dial-first path was available, statistically supported,
and wrong.
Which is why the order is asserted rather than suggested: a rule can score
0.65 and rank fifth for the moment it governs, behind rules that RESTRAIN the
same act. Every percentile says "lower the floor" — and at 0.60 the arm
delivers those restraints and still not that rule, while rewriting its trigger
to lead with the symptom puts it first at 0.71. The dial-first path can be
available, statistically supported, and wrong.
"""
from __future__ import annotations
@@ -50,8 +44,8 @@ TUNING_TOOLS = ("retrieval_telemetry", "update_rule", "retrieval_surfaces",
def _skill() -> str:
# The route lives in missed-retrieval.md, a reference file of using-scribe
# (#4398); the skill is SKILL.md and its references read together.
# The route lives in missed-retrieval.md, a reference file of using-scribe;
# the skill is SKILL.md and its references read together.
from tests.helpers import skill_text
return skill_text("using-scribe")
@@ -142,17 +136,16 @@ def test_writing_a_new_record_asks_what_already_covers_the_moment():
def test_the_route_stays_off_the_index():
"""`_INSTRUCTIONS` is the only surface every MCP client gets (decision #4027).
"""`_INSTRUCTIONS` is the only surface every MCP client gets.
It carried a one-line MISSED pointer to this route until #4389, which found
the index was being used as a rulebook: a missed rule is noticed mid-work,
not at session start, and the spec asks server instructions for the
It carries no pointer to this route, because the index is not a rulebook:
a missed rule is noticed mid-work, not at session start, and the spec asks server instructions for the
workflow across tools, not stance. using-scribe states the route (the tests
above pin it); the index keeps what_might_apply, the tool that opens it.
Whitespace-flattened before matching: the block is hard-wrapped, so a
phrase straddles a line break the moment anything before it changes length
(#4103).
phrase straddles a line break the moment anything before it changes
length.
"""
text = " ".join(_instructions().split()).lower()
assert "what_might_apply" in text