Merge pull request #165 from bvandeusen/dev

A recognized retrieval miss has a route, and the record comes first (#4133)

Closes milestone 416 at 9/9. CI 6999 green on 104c1d6.

plugin/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "scribe",
   "description": "Scribe for Claude Code: connects the scribe MCP server, adds the hooks that deliver live project state and relevant records at the right moment, ships the shared client-neutral Scribe skills (using-scribe, writing-plans, reporting-back, systematic-debugging, verification, brainstorming, reusing-code, shape-accounting), and syncs your saved Scribe Processes as skills (/scribe:sync).",
-  "version": "2026.09.17.1618",
+  "version": "2026.09.18.0436",
   "author": {
     "name": "Bryan Van Deusen"
   },

plugin/skills/using-scribe/SKILL.md

@@ -318,6 +318,19 @@ with no trigger is not a quiet rule, it is an unreachable one. Write the moment
 in the words a session actually produces — the command, the error, the
 half-formed ask — not the category it belongs to.
 
+**Before writing one, ask what already covers that moment.**
+`what_might_apply("the moment you are about to write a record for")` — fifty
+candidates and no bar, so an existing record cannot hide under a threshold the
+way it can from `search`. When something already covers the moment, the reach
+is that record: improve its trigger or its statement rather than standing a
+second one beside it. Two records describing the same moment compete in one
+ranked list against one budget, and the slot they take from each other is the
+third candidate that would have said something different. Two records may
+legitimately share a moment and say *different* things — a rule for what must
+happen, a preference for how to report it. What this catches is the same thing
+said twice at two strengths, which is worse than either alone: a session that
+retrieves the softer copy has been told that binding guidance is optional.
+
 **First ask whether it's a rule at all.** A rule is prose you have to remember
 and apply; Scribe's other entities are structure a tool can resolve and check.
 Visual standards belong in a **design system**, not a rulebook — a token can be
@@ -326,6 +339,49 @@ and none of that survives being written as a rule. A repeatable procedure is a
 **process**; reusable code is a **snippet**. Reach for a rule when the thing
 really is a standing instruction about how to work.
 
+## When a record doesn't reach the moment it should
+
+Retrieval misjudging is ordinary, and it is fixable — but only by whoever
+notices. **Either direction counts:** a rule that should have governed a moment
+and never arrived, and a rule that arrives on every turn and never applies. So
+does **either noticer**: the operator saying *"that should have fired"*, and you
+noticing it yourself — you reached for a rule nobody offered you, or you were
+handed the same rule five times and set it aside five times.
+
+**Take it to the record first and the dial second.** A rule's `when_to_apply`
+IS the text its similarity score is computed against, so when a rule misses a
+moment it governs, the overwhelmingly likely cause is that its trigger does not
+describe that moment in the words a session actually produces. Rewording one
+trigger changes one rule's reach. Moving a floor changes what every record on
+that surface does, and a floor cannot tell a badly-worded trigger from a
+genuinely distant record — so one lowered to rescue a single rule admits
+everything else that was sitting in the same band.
+
+1. **Read the refused records.** `retrieval_telemetry(days=N,
+   near_miss_samples=5)` names by id what each surface refused and by how much.
+   Open them with `get_rule` / `get_note`. This is the step that carries the
+   answer: the statistic says a record was close, and only the record says
+   whether it was *right*.
+2. **Fix the trigger.** `update_rule(when_to_apply=...)`, written as the
+   symptom — what the session was doing or saying at the moment it needed this
+   rule — not the situation the rule belongs to. Then check that it worked:
+   `what_might_apply("the moment, in the operator's own words")` and read where
+   the rule now ranks. The change is measurable, so measure it, and say the
+   before and after when you report it.
+3. **Then consider the dial.** `retrieval_surfaces` shows what is in force per
+   arm and whether the number is still calibrated; `tune_retrieval` moves it.
+   `reason` is required and has to say what you read, because it is what lets
+   the operator disagree with a number they did not choose.
+
+Reaching for `tune_retrieval` before opening a single record is the wrong move,
+and it is the one that feels efficient. Worked example, measured on this
+install: a rule granting a routine push scored 0.6515 and ranked 5th for the
+moment it governed, behind three rules that *restrained* the same act. Every
+percentile said "lower the floor" — and lowering it would have delivered those
+three restraints and still not the rule. Rewriting the trigger to lead with the
+symptom moved the same rule to 1st at 0.7130, ahead of all three. Only then was
+the floor worth touching.
+
 ## Building UI: the project's design system binds
 
 `enter_project` names the project's `design_system` when it has one. Before

src/scribe/mcp/server.py

@@ -49,6 +49,8 @@ client reads Agent Skills) and in each tool's description. The index:
   candidates, no bar. search(content_type="rule") reads one you already
   suspect. Silence means nothing matched, not none. Rules bind; preferences
   guide.
+- MISSED: a rule that should have reached you is a trigger to fix, not a
+  floor to move (retrieval_telemetry).
 - RECALL: search before acting, scoped with the active project_id.
 - RECORD: create_task; a fix is kind="issue". add_task_log as you go; status
   in_progress on start, done on finish. Tag system_ids as you write.

tests/test_guidance_ownership.py

@@ -142,6 +142,15 @@ TOPICS: tuple[Topic, ...] = (
     Topic("where a new rule goes, and its trigger", U, ("create_project_rule", "when_to_apply"),
           "whichever home it gets"),
     Topic("a rule vs the other entities", U, ("standing instruction",), "first ask whether it's a rule at all"),
+    # Milestone 416 step 9: the tuning tools shipped in #4102/#4104 and were
+    # named on NO instruction surface — measured, `retrieval_tuning_history`
+    # returned zero events. Machinery with no route to it.
+    Topic("a missed rule is a trigger to fix before a floor to move", U,
+          ("retrieval_telemetry", "tune_retrieval", "retrieval_surfaces"),
+          "take it to the record first and the dial second",
+          index=("retrieval_telemetry",)),
+    Topic("ask what already covers a moment before writing a record", U,
+          ("what_might_apply",), "ask what already covers that moment"),
     Topic("reference notes update in place; dev-logs don't", U, ("reference note",),
           "state updates in place; chronicles don't"),
     # ── process arcs — owned by their skills ──

tests/test_retrieval_miss_route.py

@@ -0,0 +1,158 @@
+"""A recognized retrieval miss has a route, and the route is ordered (#4133).
+
+WHY THIS EXISTS
+
+Milestone 416 built the tuning loop across steps 4 and 6 — `retrieval_surfaces`
+to read a dial, `tune_retrieval` to move one with its argument attached,
+`migrate_retrieval_floor` to carry it across a model change — and then measured,
+on 2026-09-17, that `retrieval_tuning_history` returned **zero events**. Not one
+call, ever.
+
+The cause was not a tool being wrong. The three tools appeared on NO instruction
+surface: not the skills, not the hooks, not the MCP `_INSTRUCTIONS`. They were
+classified in `server.py`'s read/write lists and named nowhere a session would
+read them. The operator's decision for #4102 — *"the model should be the thing
+handling it 9 times out of 10"* — could not begin to happen.
+
+So this file guards a ROUTE, not a behaviour: that a session which notices a
+rule missed the moment it governed has somewhere written down to take it.
+
+WHY THE ORDER IS THE LOAD-BEARING PART
+
+`test_the_record_comes_before_the_dial` is the test that matters here, and it is
+positional on purpose. Both halves of the route are individually reasonable —
+read the telemetry, move the floor — and stated in either order they read as a
+menu. They are not a menu. A rule's `when_to_apply` IS the text its score is
+computed against, so a miss is evidence about that text first; a floor is one
+number governing every record on the surface and cannot tell a badly-worded
+trigger from a genuinely distant one.
+
+Measured on this install, which is why the order is asserted rather than
+suggested: rule 1 "`dev` is home" scored 0.6515 and ranked 5th for the moment it
+governed, behind three rules that RESTRAINED the same act. Every percentile said
+"lower the floor" — and at 0.60 the arm delivered those three restraints and
+still not rule 1. Rewriting the trigger to lead with the symptom put the same
+rule 1st at 0.7130. The dial-first path was available, statistically supported,
+and wrong.
+"""
+from __future__ import annotations
+
+import pathlib
+import re
+
+ROOT = pathlib.Path(__file__).resolve().parents[1]
+SKILL = ROOT / "plugin/skills/using-scribe/SKILL.md"
+
+# The three tools that shipped with no route to them. Named together because
+# the gap was all three at once, and a partial fix would leave the loop broken
+# at whichever step lost its pointer.
+TUNING_TOOLS = ("retrieval_telemetry", "update_rule", "retrieval_surfaces",
+                "tune_retrieval")
+
+
+def _skill() -> str:
+    return SKILL.read_text()
+
+
+def _instructions() -> str:
+    src = (ROOT / "src/scribe/mcp/server.py").read_text()
+    match = re.search(r'_INSTRUCTIONS = """(.*?)"""', src, re.S)
+    assert match, "server.py no longer defines _INSTRUCTIONS as a literal"
+    return match.group(1)
+
+
+def test_the_route_names_every_tool_it_depends_on():
+    """The measured gap: machinery with no instruction surface pointing at it."""
+    text = _skill()
+    missing = [t for t in TUNING_TOOLS if t not in text]
+    assert not missing, (
+        f"using-scribe no longer names {missing}. These tools reach a session "
+        f"through this skill and nowhere else — when they were named on no "
+        f"surface at all, the tuning loop logged zero events in its lifetime."
+    )
+
+
+def test_the_record_comes_before_the_dial():
+    """THE GUARD. Stated in the wrong order, the route becomes a menu.
+
+    Positional rather than a presence check, because presence is exactly what
+    a reordering preserves. Asserted on first occurrence: what a reader meets
+    first is what they act on.
+    """
+    text = _skill()
+    read_records = text.index("retrieval_telemetry")
+    fix_trigger = text.index("update_rule(when_to_apply")
+    move_dial = text.index("tune_retrieval")
+
+    assert read_records < move_dial, (
+        "the route reaches tune_retrieval before retrieval_telemetry. Reading "
+        "the refused records is the step that separates a real miss from a bar "
+        "doing its job — a percentile cannot, and has been measured pointing "
+        "the wrong way."
+    )
+    assert fix_trigger < move_dial, (
+        "the route reaches tune_retrieval before update_rule. A miss is "
+        "evidence about the trigger first: rewording one trigger changes one "
+        "rule's reach, while moving a floor changes every record on the surface."
+    )
+
+
+def test_the_route_says_what_a_floor_costs_that_a_trigger_does_not():
+    """The REASON for the order, not just the order.
+
+    Without it the ordering is arbitrary and the first session under deadline
+    will invert it. The asymmetry is the whole argument: one trigger against
+    every record on the surface.
+    """
+    text = " ".join(_skill().split()).lower()
+    assert "take it to the record first and the dial second" in text
+    assert "changes one rule's reach" in text
+    assert "changes what every record on" in text
+
+
+def test_a_session_can_notice_its_own_miss():
+    """Both noticers, or the route only ever fires when the operator complains.
+
+    The self-noticed case is the one that scales — it does not need somebody
+    watching — and it is the one a surface written only around operator
+    feedback silently omits.
+    """
+    text = " ".join(_skill().split()).lower()
+    assert "either noticer" in text
+    assert "reached for a rule nobody offered you" in text
+    # Both directions too: a rule that never arrives and one that always does
+    # are the same misjudgment, and they point at different dials.
+    assert "either direction counts" in text
+
+
+def test_writing_a_new_record_asks_what_already_covers_the_moment():
+    """The create path is retrieval-affecting, and nothing said so.
+
+    A second record on one moment splits a single budget between two claims,
+    and when the copies differ in force the softer one teaches a session that
+    binding guidance is optional. `what_might_apply` rather than `search`
+    because a bar is exactly what lets the existing record hide.
+    """
+    text = " ".join(_skill().split()).lower()
+    assert "ask what already covers that moment" in text
+    assert "what_might_apply" in text
+    assert "no bar" in text
+
+
+def test_the_index_points_at_the_route_without_restating_it():
+    """`_INSTRUCTIONS` is the only surface every MCP client gets (decision #4027).
+
+    It indexes; using-scribe states. A client with no Agent Skills support still
+    learns the ordering exists and which tool opens it.
+
+    Whitespace-flattened before matching. The block is hard-wrapped to fit a
+    2,000-character budget, so a phrase straddles a line break the moment
+    anything before it changes length — #4103 shipped a guard that broke
+    exactly that way, on text nobody had touched.
+    """
+    text = " ".join(_instructions().split())
+    assert "retrieval_telemetry" in text
+    assert "not a floor to move" in text
+    # The route itself must NOT be here — 2,000 characters is the whole budget
+    # and Claude Code cuts the rest mid-word (#2562).
+    assert "take it to the record first" not in text.lower()