feat(retrieval): the standing-rule arm gets its own bar, and asks for one rule not two (#3318)

Milestone 333 step 4 — the split #2223 made one surface down, now made for the
third corpus. The arm inherited WRITEPATH_DEFAULT_THRESHOLD = 0.68, a number
measured against code-vs-note-PROSE and never re-derived for code-vs-RULE-TEXT.

THE DEFAULT IS ARGUED STRUCTURALLY, NOT READ OFF A HISTOGRAM (rule 115). Two
facts hold on any install, including one with six rules and no telemetry:

- The eligible corpus is tiny — conditional rules only, a handful to a few
  dozen against thousands of notes. A top-k over forty candidates always
  returns something, so "the best match cleared the bar" stops meaning "a good
  match exists". A bar calibrated for best-of-thousands is cleared by
  best-of-forty as arithmetic, not relevance.
- Rules are short imperative technical English, far more homogeneous than note
  prose. #2223 put the code-vs-prose floor at 0.55-0.63 and set 0.68 above it;
  a more homogeneous corpus has a HIGHER floor, so 0.68 is not merely
  inherited, it sits below where this corpus's noise lives.

0.72 errs deliberately toward silence on an asymmetry that is also structural:
this hint fires on EVERY write. A missed rule is recoverable — it is still in
Scribe and the agent can search it. A hint that cries wolf is not: it teaches
the reader to skip the whole block, and the true positives go with it. The
arm's own comment already said "noise on a hint that fires on every write is
how a hint gets ignored".

Pinned as an INEQUALITY, not a value: test_the_rule_bar_defaults_above_the_code_bar
asserts RULEHINT > WRITEPATH, so tuning the number stays free while inverting
the relationship — which would silently reinstate #3311 — does not.

RULEHINT_LIMIT = 1, and deliberately not a knob. With a corpus this small, k=2
means the second line is almost always the second-best noise wearing the same
confident framing as the first; halving k halves that regardless of the bar.
It stays a constant because it is a decision about how loud one hint may be,
not a per-install tuning question — and a knob nobody turns only adds a way to
misconfigure the surface.

Reachable from Settings, no restart (rule 25), with copy that says which way to
move it and points at retrieval_telemetry's rule pull-through — which step 3
made readable — to tell "arriving unread" from "never arrived".

Every config stand-in in the suite gained the key, not just the one that
noticed. The arm reads `rule_threshold` while BUILDING its search arguments, so
a missing key raises inside its fail-open except and turns the arm into a
silent no-op — indistinguishable from it running and finding nothing. That is
the same vacuous-pass shape that bit step 2, one layer down (rule 33).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN

frontend/src/views/SettingsView.vue

@@ -86,6 +86,7 @@ const kbWritePathEnabled = ref(true);
 // code embeddings sit on a much higher similarity floor than prose, so 0.55 let
 // unrelated code through (#2223). Shares top-k, not the threshold.
 const kbWritePathThreshold = ref("0.68");
+const kbRuleHintThreshold = ref("0.72");
 // Near-duplicate report floors, one per record kind (services/dedup.py).
 // Snippets are single-chunk, so their floor sits below the 0.90 write-time
 // gate and catches what it lets through. Notes/tasks are scored at chunk
@@ -148,12 +149,17 @@ async function saveKbInject() {
   // Same `|| default` reasoning: falling back to 0 would surface every
   // snippet in the corpus on every edit, which is the failure this knob fixes.
   const wpT = Math.min(1, Math.max(0, Number(kbWritePathThreshold.value) || 0.68));
+  // Same `|| default` reasoning again, and it bites harder here: a rule hint
+  // fires on every write, so a fallback of 0 would attach a standing rule to
+  // every edit in the session.
+  const rhT = Math.min(1, Math.max(0, Number(kbRuleHintThreshold.value) || 0.72));
   kbInjectThreshold.value = String(t);
   kbInjectTopK.value = String(k);
   kbDupThresholdSnippet.value = String(dupSnip);
   kbDupThresholdNote.value = String(dupNote);
   kbDupThresholdTask.value = String(dupTask);
   kbWritePathThreshold.value = String(wpT);
+  kbRuleHintThreshold.value = String(rhT);
   savingKbInject.value = true;
   kbInjectSaved.value = false;
   try {
@@ -166,6 +172,10 @@ async function saveKbInject() {
       // measurements that split them.
       kb_writepath_enabled: kbWritePathEnabled.value ? 'true' : 'false',
       kb_writepath_threshold: String(wpT),
+      // A THIRD corpus with a third bar — see RULEHINT_DEFAULT_THRESHOLD
+      // in services/plugin_context.py for why rules cannot share the
+      // code threshold any more than code could share the prose one.
+      kb_rulehint_threshold: String(rhT),
       kb_duplicate_threshold_snippet: String(dupSnip),
       kb_duplicate_threshold_note: String(dupNote),
       kb_duplicate_threshold_task: String(dupTask),
@@ -611,6 +621,9 @@ onMounted(async () => {
     kbInjectTopK.value = allSettings.kb_autoinject_top_k;
   }
   kbWritePathEnabled.value = allSettings.kb_writepath_enabled !== "false";
+  if (allSettings.kb_rulehint_threshold !== undefined) {
+    kbRuleHintThreshold.value = allSettings.kb_rulehint_threshold;
+  }
   if (allSettings.kb_writepath_threshold !== undefined) {
     kbWritePathThreshold.value = allSettings.kb_writepath_threshold;
   }
@@ -1456,6 +1469,29 @@ async function deleteUser(userId: number) {
             location, not by resemblance.
           </p>
         </div>
+        <div class="field">
+          <label for="kb-rulehint-threshold">Standing-rule confidence threshold (0–1)</label>
+          <input
+            id="kb-rulehint-threshold"
+            v-model="kbRuleHintThreshold"
+            type="number"
+            min="0"
+            max="1"
+            step="0.01"
+            class="fs-input input"
+            style="max-width: 8rem"
+          />
+          <p class="field-hint">
+            The same hint can mention a standing rule whose trigger resembles what's
+            being written — only rules marked <em>conditional</em>, since always-on
+            ones are already loaded. Stricter again than the threshold above, because
+            there are far fewer rules than snippets: with a small set, something
+            always ranks first, so the bar has to carry more of the judgement.
+            Raise it if rules keep arriving unread; lower it if a rule you needed
+            never showed up. Settings → check the pull-through in
+            <code>retrieval_telemetry</code> to see which is happening.
+          </p>
+        </div>
         <!-- A design system belongs to a PROJECT, and the picker for it lives on
              the project. There was a setting here that designated the system
              this install's own interface was built from; it only ever described

src/scribe/services/plugin_context.py

@@ -86,6 +86,60 @@ WRITEPATH_THRESHOLD_KEY = "kb_writepath_threshold"
 WRITEPATH_DEFAULT_ENABLED = True
 WRITEPATH_DEFAULT_THRESHOLD = 0.68
 
+# The standing-rule arm (milestone 307) gets its own bar — the split #2223 made
+# one surface down, now made for the THIRD corpus. It inherited 0.68 above, and
+# that number was measured against code-vs-note-PROSE. It was never re-derived
+# for code-vs-RULE-TEXT.
+#
+# THE STRUCTURAL ARGUMENT, which is the only kind admissible here (rule 115).
+# Two facts hold on any install, including one with six rules and no telemetry:
+#
+#   1. The eligible corpus is TINY. The arm searches `tier="conditional"`
+#      rules only — a handful to a few dozen documents against thousands of
+#      notes. A top-k over forty candidates always returns something, so
+#      "the best match cleared the bar" stops meaning "a good match exists"
+#      and starts meaning "forty things were ranked". A bar calibrated for
+#      best-of-thousands is cleared by best-of-forty as arithmetic, not
+#      relevance.
+#   2. Rules are short imperative technical English — a far more HOMOGENEOUS
+#      corpus than note prose. #2223 measured the floor for code against prose
+#      at 0.55-0.63 and set 0.68 above it. A more homogeneous corpus has a
+#      HIGHER floor, so 0.68 is not merely inherited, it is below where this
+#      corpus's noise sits.
+#
+# WHY 0.72 AND NOT A NUMBER OFF A HISTOGRAM. The exact offset between prose's
+# floor and rule-text's is not derivable in general — it depends on how an
+# install writes its rules — so the default errs deliberately toward SILENCE
+# rather than toward recall, on an asymmetry that is itself structural: this
+# hint fires on EVERY write. A missed rule is recoverable, because the rule is
+# still in Scribe and the agent can search it. A hint that cries wolf is not:
+# it teaches the reader to skip the whole block, and the surface is lost along
+# with the true positives it would have carried. The arm's own comment already
+# says "noise on a hint that fires on every write is how a hint gets ignored".
+#
+# TUNE IT FROM YOUR OWN INSTANCE, which is now possible: `retrieval_telemetry`
+# reports `rule_usage.pull_through` (milestone 333 step 3). Raise this if rules
+# arrive unread; lower it if rules you needed never arrived. What would RETIRE
+# it: a cross-encoder rerank (#1038), which would make a similarity bar the
+# wrong control entirely.
+RULEHINT_THRESHOLD_KEY = "kb_rulehint_threshold"
+RULEHINT_DEFAULT_THRESHOLD = 0.72
+
+# ONE rule per write, not two — and this is deliberately NOT a knob.
+#
+# With a corpus this small, top-k does as much damage as the threshold: k=2
+# over forty candidates means the second line is almost always the second-best
+# noise, arriving with the same confident framing as the first. Halving k
+# halves that regardless of where the bar sits.
+#
+# It stays a constant because it is a decision about how LOUD one hint may be,
+# not a per-install tuning question. The hint already carries prior art, shape
+# signals and staleness; rules are the fourth voice in it, and a fourth voice
+# that speaks twice is where a reader stops reading. Nothing suggests an
+# operator wants this different, and a knob nobody turns is a knob that only
+# adds a way to misconfigure the surface (rule 25 cuts both ways).
+RULEHINT_LIMIT = 1
+
 # Minimum SUBSTANCE (non-whitespace chars) a payload must carry before the
 # semantic arm will run at all — the cheap half of the operator's #89 idea
 # ("a sliding scale between number of characters and semantic threshold").
@@ -691,10 +745,19 @@ async def get_writepath_config(user_id: int) -> dict:
         threshold = WRITEPATH_DEFAULT_THRESHOLD
     threshold = min(1.0, max(0.0, threshold))
 
+    try:
+        rule_threshold = float(await get_setting(
+            user_id, RULEHINT_THRESHOLD_KEY, str(RULEHINT_DEFAULT_THRESHOLD)))
+    except (TypeError, ValueError):
+        rule_threshold = RULEHINT_DEFAULT_THRESHOLD
+    rule_threshold = min(1.0, max(0.0, rule_threshold))
+
     return {
         **cfg,
         "enabled": enabled_raw.strip().lower() in ("true", "1", "yes", "on"),
         "threshold": threshold,
+        # Its own bar, for a third corpus — see RULEHINT_DEFAULT_THRESHOLD.
+        "rule_threshold": rule_threshold,
     }
 
 
@@ -1132,8 +1195,8 @@ async def build_write_path_hint(
         # than "nobody passed the number".
         rule_t0 = time.perf_counter()
         hits = await semantic_search_rules(
-            user_id, code or path, limit=2,
-            threshold=cfg["threshold"], tier="conditional",
+            user_id, code or path, limit=RULEHINT_LIMIT,
+            threshold=cfg["rule_threshold"], tier="conditional",
         )
         rule_ms = (time.perf_counter() - rule_t0) * 1000.0
         fresh = [(score, rule) for score, rule in hits if rule.id not in already]
@@ -1160,7 +1223,8 @@ async def build_write_path_hint(
             # is its own (milestone 333 step 1). The gap it described is closed.
             record_retrieval(
                 user_id=user_id, source="write_path_rule", query=code or path,
-                threshold=cfg["threshold"], limit=2, project_id=project_id,
+                threshold=cfg["rule_threshold"], limit=RULEHINT_LIMIT,
+                project_id=project_id,
                 is_task=None, results=fresh, duration_ms=rule_ms,
             )
             # `rule_ids` is `fresh`, i.e. AFTER exclude_rule_ids. A rule the

tests/test_note_usage.py

@@ -154,7 +154,8 @@ async def test_unscored_location_arms_are_recorded(lookups, expected_source):
         patch.object(
             plugin_context,
             "get_writepath_config",
-            AsyncMock(return_value={"enabled": True, "threshold": 0.55, "top_k": 3}),
+            AsyncMock(return_value={"enabled": True, "threshold": 0.55, "top_k": 3,
+                                    "rule_threshold": 0.72}),
         ),
         patch.object(
             plugin_context.snippets_svc,

tests/test_rule_usage_wiring.py

@@ -47,17 +47,25 @@ _PRIOR_ART = [(0.72, fake_note(id=9, title="debounce helper", user_id=1,
                                note_type="snippet"))]
 
 
-def _arm_patches(pc, hits, recorder, prior_art=None):
-    """The minimum stubbing that lets the rule arm run and nothing else."""
+def _arm_patches(pc, hits, recorder, prior_art=None, cfg=None, rule_search=None):
+    """The minimum stubbing that lets the rule arm run and nothing else.
+
+    `cfg` and `rule_search` are overridable so a caller can inspect what the
+    arm ASKED for rather than only what it did with the answer — patching them
+    a second time on top would work, but reads as an accident.
+    """
     return (
         patch.object(pc, "get_writepath_config",
-                     AsyncMock(return_value={"enabled": True, "threshold": 0.6,
-                                             "top_k": 3})),
+                     AsyncMock(return_value=cfg or {
+                         "enabled": True, "threshold": 0.6,
+                         "top_k": 3, "rule_threshold": 0.6,
+                     })),
         patch.object(pc.snippets_svc, "list_snippets", AsyncMock(return_value=([], 0))),
         patch.object(pc, "semantic_search_notes",
                      AsyncMock(return_value=_PRIOR_ART if prior_art is None
                                else prior_art)),
-        patch.object(pc, "semantic_search_rules", AsyncMock(return_value=hits)),
+        patch.object(pc, "semantic_search_rules",
+                     rule_search or AsyncMock(return_value=hits)),
         patch.object(pc, "record_retrieval", MagicMock()),
         patch.object(pc, "record_surfaced", MagicMock()),
         patch.object(pc, "record_rule_surfaced", recorder),
@@ -122,6 +130,36 @@ async def test_nothing_is_recorded_when_every_hit_was_already_held():
     assert rec.call_count == 0
 
 
+@pytest.mark.asyncio
+async def test_the_arm_searches_on_its_OWN_bar_not_the_code_one():
+    """The consuming half of step 4. `get_writepath_config` assembling a
+    separate `rule_threshold` means nothing if the arm still passes
+    `cfg["threshold"]` to its search — the split would exist in the config and
+    not in the behaviour, and #3311 would be exactly where it was.
+
+    The two values are deliberately different here so the assertion can tell
+    them apart.
+    """
+    from scribe.services import plugin_context as pc
+
+    search = AsyncMock(return_value=[])
+    with ExitStack() as stack:
+        for ctx in _arm_patches(
+            pc, [], MagicMock(), rule_search=search,
+            cfg={"enabled": True, "threshold": 0.60,
+                 "top_k": 3, "rule_threshold": 0.81},
+        ):
+            stack.enter_context(ctx)
+        await pc.build_write_path_hint(
+            1, "frontend/src/api/client.ts", code="x" * 400,
+        )
+
+    kw = search.await_args.kwargs
+    assert kw["threshold"] == 0.81, "the arm is still using the code threshold"
+    assert kw["limit"] == pc.RULEHINT_LIMIT
+    assert kw["tier"] == "conditional"
+
+
 @pytest.mark.asyncio
 async def test_the_arm_does_not_fire_on_a_write_that_matched_nothing():
     """The gate, pinned — because the fixture above now depends on it and a

tests/test_services_plugin_context.py

@@ -419,7 +419,8 @@ async def test_write_path_semantic_arm_asks_for_experience_not_just_snippets():
     rec = MagicMock()
     with patch.object(pc, "get_writepath_config",
                       AsyncMock(return_value={"enabled": True, "threshold": 0.6,
-                                              "top_k": 3})), \
+                                              "top_k": 3,
+                                              "rule_threshold": 0.72})), \
          patch.object(pc.snippets_svc, "list_snippets",
                       AsyncMock(return_value=([], 0))), \
          patch.object(pc, "semantic_search_notes", search), \
@@ -451,7 +452,8 @@ async def test_write_path_labels_a_non_snippet_hit_with_its_kind():
             (0.71, fake_note(id=7, title="Debounce dropped the trailing call", user_id=1, is_task=True, task_kind="issue"))]
     with patch.object(pc, "get_writepath_config",
                       AsyncMock(return_value={"enabled": True, "threshold": 0.6,
-                                              "top_k": 3})), \
+                                              "top_k": 3,
+                                              "rule_threshold": 0.72})), \
          patch.object(pc.snippets_svc, "list_snippets",
                       AsyncMock(return_value=([], 0))), \
          patch.object(pc, "semantic_search_notes", AsyncMock(return_value=hits)), \

tests/test_write_path_trigger.py

@@ -25,7 +25,13 @@ def _snippet_item(nid, title, user_id=1):
 
 
 def _cfg(**over):
-    base = {"enabled": True, "threshold": 0.68, "top_k": 3}
+    # `rule_threshold` is the standing-rule arm's own bar (milestone 333 step
+    # 4). It belongs in the stand-in even though most tests here never reach
+    # that arm: the arm reads it while BUILDING its search arguments, so a
+    # missing key raises inside its fail-open except and turns the arm into a
+    # silent no-op — which is indistinguishable from it working and finding
+    # nothing.
+    base = {"enabled": True, "threshold": 0.68, "top_k": 3, "rule_threshold": 0.72}
     base.update(over)
     return base
 
@@ -426,6 +432,74 @@ async def test_writepath_threshold_is_operator_tunable_and_clamped():
     assert (await _cfg_with("banana"))["threshold"] == pc.WRITEPATH_DEFAULT_THRESHOLD
 
 
+@pytest.mark.asyncio
+async def test_the_rule_arm_has_its_own_tunable_bar():
+    """Rule #25 again, for the THIRD corpus (milestone 333 step 4).
+
+    Separate from the code threshold above and separately settable, because the
+    two are measured against different things: 0.68 was derived from code
+    against note PROSE (#2223), and rules are short imperative technical
+    English — a more homogeneous corpus whose noise floor sits higher.
+    """
+    from scribe.services import plugin_context as pc
+
+    async def _cfg_with(raw):
+        stored = {pc.RULEHINT_THRESHOLD_KEY: raw}
+        with patch.object(pc, "get_setting",
+                          AsyncMock(side_effect=lambda uid, k, d: stored.get(k, d))):
+            return await pc.get_writepath_config(1)
+
+    assert (await _cfg_with("0.8"))["rule_threshold"] == 0.8
+    assert (await _cfg_with("5"))["rule_threshold"] == 1.0
+    assert (await _cfg_with("-3"))["rule_threshold"] == 0.0
+    # Garbage falls back to the default, not to 0.0 — which on THIS arm would
+    # attach a standing rule to every write in the session.
+    assert (await _cfg_with("banana"))["rule_threshold"] == pc.RULEHINT_DEFAULT_THRESHOLD
+
+
+@pytest.mark.asyncio
+async def test_the_two_write_path_bars_are_independent():
+    """The split, asserted. Setting one must not move the other — the failure
+    that would silently undo this step is a config assembler that reads one key
+    into both fields."""
+    from scribe.services import plugin_context as pc
+
+    stored = {pc.WRITEPATH_THRESHOLD_KEY: "0.90", pc.RULEHINT_THRESHOLD_KEY: "0.61"}
+    with patch.object(pc, "get_setting",
+                      AsyncMock(side_effect=lambda uid, k, d: stored.get(k, d))):
+        cfg = await pc.get_writepath_config(1)
+
+    assert cfg["threshold"] == 0.90
+    assert cfg["rule_threshold"] == 0.61
+
+
+def test_the_rule_bar_defaults_above_the_code_bar():
+    """Not a number check — a DIRECTION check, and the only part of the default
+    that is defensible without one instance's histogram (rule 115).
+
+    The eligible rule corpus is orders of magnitude smaller than the note
+    corpus, so a top-k over it always returns something and a bar calibrated
+    for best-of-thousands is cleared by best-of-forty as arithmetic. Rules are
+    also more homogeneous than note prose, so their noise floor is higher. Both
+    facts point the same way: this bar must sit ABOVE the one it inherited.
+
+    Pinned as an inequality so tuning the value stays free while inverting the
+    relationship — which would silently reinstate #3311 — does not.
+    """
+    from scribe.services import plugin_context as pc
+
+    assert pc.RULEHINT_DEFAULT_THRESHOLD > pc.WRITEPATH_DEFAULT_THRESHOLD
+
+
+def test_the_rule_arm_asks_for_one_rule_not_two():
+    """With a corpus this small, top-k does as much damage as the threshold:
+    k=2 over a few dozen candidates means the second line is almost always the
+    second-best noise, carrying the same confident framing as the first."""
+    from scribe.services import plugin_context as pc
+
+    assert pc.RULEHINT_LIMIT == 1
+
+
 # --- the minimum-substance floor on the semantic arm (#2223) ------------------
 
 @pytest.mark.asyncio