docs(6209): wide net, startup, stamps, slippage, miss route and parity tests
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Build & push image (push) Canceled after 0s
CI & Build / Python tests (push) Canceled after 1m23s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Build & push image (push) Canceled after 0s
CI & Build / Python tests (push) Canceled after 1m23s
Eight more test files state the property they pin as a present-tense fact. Incident narratives (a dated storage stall, a week of zero tuning events, a two-project poisoned ledger, a twenty-site owner audit) become the failure shape each test defends; one-install figures that name a rule become a general worked example with the calibration numbers kept. The family-grouping regression guard is renamed for what it checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,18 +1,13 @@
|
||||
"""A recognized retrieval miss has a route, and the route is ordered (#4133).
|
||||
"""A recognized retrieval miss has a route, and the route is ordered.
|
||||
|
||||
WHY THIS EXISTS
|
||||
|
||||
Milestone 416 built the tuning loop across steps 4 and 6 — `retrieval_surfaces`
|
||||
to read a dial, `tune_retrieval` to move one with its argument attached,
|
||||
`migrate_retrieval_floor` to carry it across a model change — and then measured,
|
||||
on 2026-09-17, that `retrieval_tuning_history` returned **zero events**. Not one
|
||||
call, ever.
|
||||
|
||||
The cause was not a tool being wrong. The three tools appeared on NO instruction
|
||||
surface: not the skills, not the hooks, not the MCP `_INSTRUCTIONS`. They were
|
||||
classified in `server.py`'s read/write lists and named nowhere a session would
|
||||
read them. The operator's decision for #4102 — *"the model should be the thing
|
||||
handling it 9 times out of 10"* — could not begin to happen.
|
||||
The tuning loop is three tools — `retrieval_surfaces` to read a dial,
|
||||
`tune_retrieval` to move one with its argument attached,
|
||||
`migrate_retrieval_floor` to carry it across a model change — and the model is
|
||||
meant to be the one using them most of the time. A tool classified in
|
||||
`server.py`'s read/write lists and named on no instruction surface is never
|
||||
called: nothing a session reads tells it the tool exists.
|
||||
|
||||
So this file guards a ROUTE, not a behaviour: that a session which notices a
|
||||
rule missed the moment it governed has somewhere written down to take it.
|
||||
@@ -27,13 +22,12 @@ computed against, so a miss is evidence about that text first; a floor is one
|
||||
number governing every record on the surface and cannot tell a badly-worded
|
||||
trigger from a genuinely distant one.
|
||||
|
||||
Measured on this install, which is why the order is asserted rather than
|
||||
suggested: rule 1 "`dev` is home" scored 0.6515 and ranked 5th for the moment it
|
||||
governed, behind three rules that RESTRAINED the same act. Every percentile said
|
||||
"lower the floor" — and at 0.60 the arm delivered those three restraints and
|
||||
still not rule 1. Rewriting the trigger to lead with the symptom put the same
|
||||
rule 1st at 0.7130. The dial-first path was available, statistically supported,
|
||||
and wrong.
|
||||
Which is why the order is asserted rather than suggested: a rule can score
|
||||
0.65 and rank fifth for the moment it governs, behind rules that RESTRAIN the
|
||||
same act. Every percentile says "lower the floor" — and at 0.60 the arm
|
||||
delivers those restraints and still not that rule, while rewriting its trigger
|
||||
to lead with the symptom puts it first at 0.71. The dial-first path can be
|
||||
available, statistically supported, and wrong.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -50,8 +44,8 @@ TUNING_TOOLS = ("retrieval_telemetry", "update_rule", "retrieval_surfaces",
|
||||
|
||||
|
||||
def _skill() -> str:
|
||||
# The route lives in missed-retrieval.md, a reference file of using-scribe
|
||||
# (#4398); the skill is SKILL.md and its references read together.
|
||||
# The route lives in missed-retrieval.md, a reference file of using-scribe;
|
||||
# the skill is SKILL.md and its references read together.
|
||||
from tests.helpers import skill_text
|
||||
return skill_text("using-scribe")
|
||||
|
||||
@@ -142,17 +136,16 @@ def test_writing_a_new_record_asks_what_already_covers_the_moment():
|
||||
|
||||
|
||||
def test_the_route_stays_off_the_index():
|
||||
"""`_INSTRUCTIONS` is the only surface every MCP client gets (decision #4027).
|
||||
"""`_INSTRUCTIONS` is the only surface every MCP client gets.
|
||||
|
||||
It carried a one-line MISSED pointer to this route until #4389, which found
|
||||
the index was being used as a rulebook: a missed rule is noticed mid-work,
|
||||
not at session start, and the spec asks server instructions for the
|
||||
It carries no pointer to this route, because the index is not a rulebook:
|
||||
a missed rule is noticed mid-work, not at session start, and the spec asks server instructions for the
|
||||
workflow across tools, not stance. using-scribe states the route (the tests
|
||||
above pin it); the index keeps what_might_apply, the tool that opens it.
|
||||
|
||||
Whitespace-flattened before matching: the block is hard-wrapped, so a
|
||||
phrase straddles a line break the moment anything before it changes length
|
||||
(#4103).
|
||||
phrase straddles a line break the moment anything before it changes
|
||||
length.
|
||||
"""
|
||||
text = " ".join(_instructions().split()).lower()
|
||||
assert "what_might_apply" in text
|
||||
|
||||
Reference in New Issue
Block a user