refactor(retrieval): one registry for every surface's floor and budget (#4102)

Groundwork for the step's real change. The operator's decision is that the
floor is chosen and adjusted by the model using it, not shipped as a value
somebody has to defend:

  "we need a model consistent surface for the adjustment of these floor values.
   the user should be able to touch it but the model should be the thing
   handling it 9 times out of 10."

A tuning surface cannot be consistent across six arms that each spell their
configuration differently, so the arms stop owning their numbers.
`retrieval_surfaces.SURFACES` names each one, its floor key and default, its
budget key and default, and — because they are rendered by the tuning tool and
the Settings UI — what it asks, over what corpus, and how often it fires. A
floor cannot be moved responsibly by anyone who does not know those three.

Three things fall out:

- **`k` becomes a real budget everywhere.** Only auto-inject had a configurable
  one; `RULEHINT_LIMIT`, `PROMPTRULE_LIMIT` and `reply_preferences.LIMIT` were
  constants. `k` is what binds under a low floor, so it has to be settable per
  surface — and per surface is the point, since `pre_tool_rule` fires before
  every Bash call while `prompt_rule` fires once a turn.
- **`write_path` gets its own budget, inherited not reset.** It shared
  auto-inject's outright on the argument that "how many titles at once" means
  the same thing on both. It does not, for the same reason. Unset, it still
  reads auto-inject's key, so an install that tuned the shared knob does not
  silently drop to a new default.
- **The duplicated read-and-clamp goes.** That shape is canon #2860 across 295
  of 372 judged siblings. Survivable while the numbers were constants; not once
  they are meant to move.

The long measurement comments stay exactly where they are — #2223's noise-floor
probe, #3853's command-vs-code split, #3851's band measurement. The constants
they annotate now alias the registry, so there is one value and the reasoning
still sits beside it.

Tests build the write-path config from the registry (`helpers.writepath_cfg`)
instead of from hand-written dicts. That is not tidiness: the rule arms read
their numbers inside a fail-open `except`, so a dict missing one key does not
raise where a reader would see it — the arm silently becomes a no-op that reads
exactly like "fired and found nothing". Ten hand-written dicts each looked
complete on the day they were typed.

tests/test_retrieval_surfaces.py pins the identity everything rests on: a
surface's name IS its telemetry source. Nothing in the type system says so —
`record_retrieval(source="pre_tool_rule")` is a literal in another file — and
renaming one without the other yields an arm that can be tuned and not
measured, or measured and not tuned, with no symptom either way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy

src/scribe/services/retrieval_surfaces.py

@@ -0,0 +1,243 @@
+"""One registry of the retrieval surfaces, and the two numbers each one has (#4102).
+
+WHY THIS EXISTS
+
+Six push arms each carried their own loose copy of the same shape: a settings
+key, a default, and a limit that was usually a module constant nobody could
+change. `_threshold()` was duplicated so widely that the shape ledger counts it
+as canon across 295 of 372 judged siblings (snippet #2860). That was survivable
+while the numbers were shipped constants an operator occasionally edited.
+
+It stops being survivable once the numbers are meant to MOVE. The operator's
+decision for this step:
+
+    "the floor should be chosen and adjusted by the model using it. we've come
+    back to something either fails or has to be looked at by the user we need a
+    model consistent surface for the adjustment of these floor values. the user
+    should be able to touch it but the model should be the thing handling it 9
+    times out of 10."
+
+A tuning surface cannot be "model consistent" if every arm spells its own
+configuration differently. So the arms stop owning their numbers and read them
+from here instead, and the tuning tool, the routes and the Settings UI all
+enumerate THIS table rather than hard-coding six special cases.
+
+WHAT A FLOOR IS NOW, AND WHAT IT IS NOT
+
+It is not a relevance judgement. Relevance is decided by the reader, which is
+the only participant that can read a trigger against a situation — that is the
+milestone's whole argument, and the injected line has always said so out loud
+("read it before deciding it does not apply").
+
+A floor answers the cheaper question: *is this worth ranking at all*. `k` is
+what binds, and `k` is a BUDGET — how much of this surface's attention a
+candidate list may spend. An arm that fires before every Bash call cannot
+afford what an arm that fires once a turn can.
+
+WHY THE DEFAULTS BELOW ARE STARTING POINTS AND NOT ANSWERS
+
+A cosine score is a distance in BAAI/bge-small-en-v1.5's vector space, measured
+against THIS corpus. It cannot transfer to an install with different records,
+and rule 115 forbids defending a shipped default from this instance's
+telemetry — which is what made the old design unbuildable: every number was a
+guess everywhere except here, and there was no mechanism that could ever
+improve it.
+
+The mechanism is the fix. Scribe ships a starting point and the means to
+correct it, so the values below carry the measurement that motivated them (see
+the long comments in `plugin_context.py`, which are kept where they are because
+they record how each number was first arrived at) without claiming to be right
+for anybody else.
+
+HOW A FLOOR SHOULD ACTUALLY BE MOVED
+
+By reading the records the floor refused — `retrieval_telemetry(
+near_miss_samples=N)` returns them by id — and never by the percentile alone.
+That is not a style preference; it is the one case where the two disagreed and
+was checked. `report_preference` logged 69 consecutive declines with the
+refused score 0.0006 under the bar, and every percentile said "lower it".
+Reading the refused record showed it was rule 77 "Extract intent from loose
+phrasing", a false positive, so lowering the bar would have delivered that rule
+on every completion report ever written. The statistic and the correct action
+pointed in opposite directions, and only opening the record could tell.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from scribe.services.settings import get_setting
+
+# A budget nobody should be able to set past. Not a tuning value — a guard on
+# the worst case, so a mistyped setting cannot turn a menu into a wall of text.
+# Shared by every surface because it bounds the same thing everywhere: how many
+# lines a single unsolicited injection may occupy.
+MAX_BUDGET = 10
+
+
+@dataclass(frozen=True)
+class Surface:
+    """One push arm's tunable pair, plus enough prose to tune it responsibly.
+
+    `asks` / `over` / `fires` are not documentation for this file — they are
+    rendered by the tuning tool and the Settings UI. A floor cannot be moved
+    sensibly by anyone, model or human, who does not know what the query is, what
+    corpus it runs against, or how often it costs something. Those three facts
+    are exactly what separates these arms from each other, and they were
+    previously recoverable only by reading `plugin_context.py`.
+    """
+
+    name: str
+    """The telemetry `source` value, and the join key.
+
+    MUST equal the string this arm passes to `record_retrieval`. Everything
+    useful about tuning depends on that identity: the tool that moves a floor
+    and the table that says what the floor did have to be talking about the same
+    arm. A test asserts it rather than a comment asking nicely.
+    """
+
+    floor_key: str
+    floor_default: float
+    budget_key: str
+    budget_default: int
+    asks: str
+    over: str
+    fires: str
+    budget_falls_back_to: str = ""
+    """A budget key to inherit when this surface has none of its own set.
+
+    Only `write_path` uses it, and only because it USED to share auto-inject's
+    `top_k` outright. Giving it a key without this would silently reset the
+    budget of every install that had tuned the shared one — a behaviour change
+    delivered as a default, which is the shape of regression nobody reports
+    because nothing looks broken.
+    """
+
+
+SURFACES: dict[str, Surface] = {
+    "auto_inject": Surface(
+        name="auto_inject",
+        floor_key="kb_autoinject_threshold",
+        floor_default=0.55,
+        budget_key="kb_autoinject_top_k",
+        budget_default=3,
+        asks="the operator's message, as they typed it",
+        over="notes, snippets, processes and issues",
+        fires="once per operator turn",
+    ),
+    "write_path": Surface(
+        name="write_path",
+        floor_key="kb_writepath_threshold",
+        floor_default=0.68,
+        budget_key="kb_writepath_top_k",
+        budget_default=3,
+        budget_falls_back_to="kb_autoinject_top_k",
+        asks="the code being written, rewritten as a concept query",
+        over="snippets and recorded issues",
+        fires="before every Write and Edit",
+    ),
+    "write_path_rule": Surface(
+        name="write_path_rule",
+        floor_key="kb_rulehint_threshold",
+        floor_default=0.72,
+        budget_key="kb_rulehint_top_k",
+        budget_default=5,
+        asks="the code being written, against rule triggers",
+        over="global rules plus the bound project's own",
+        fires="before every Write and Edit",
+    ),
+    "pre_tool_rule": Surface(
+        name="pre_tool_rule",
+        floor_key="kb_toolrule_threshold",
+        floor_default=0.68,
+        budget_key="kb_toolrule_top_k",
+        budget_default=5,
+        asks="the command about to run, against rule triggers",
+        over="global rules plus the bound project's own",
+        fires="before every Bash call — the busiest arm there is",
+    ),
+    "prompt_rule": Surface(
+        name="prompt_rule",
+        floor_key="kb_promptrule_threshold",
+        floor_default=0.72,
+        budget_key="kb_promptrule_top_k",
+        budget_default=3,
+        asks="the operator's message, against rule triggers",
+        over="global rules plus the bound project's own",
+        fires="once per operator turn",
+    ),
+    "report_preference": Surface(
+        name="report_preference",
+        floor_key="kb_reportpref_threshold",
+        floor_default=0.72,
+        budget_key="kb_reportpref_top_k",
+        budget_default=3,
+        # THE ONE FIXED QUERY, and the reason this arm behaves unlike the rest.
+        # The others score something that varies per call; this one scores a
+        # constant string, so its top score for a given corpus is also a
+        # constant. A floor a hair above that constant is not a quiet arm, it is
+        # a dead one, and no amount of traffic will ever reveal it — which is
+        # precisely how this arm spent 69 calls declining the same record.
+        asks="a fixed question about how to lay out a completion report",
+        over="preferences",
+        fires="when a task finishes",
+    ),
+}
+
+# Reserved slots are deliberately absent. `preference_slot` and `reuse_slot`
+# borrow their parent arm's floor and are hard-limited to one hit each, because
+# their entire purpose is to guarantee a single line to a kind of record that
+# keeps losing a general score contest (#2246, #3894). A budget of "1" is the
+# feature; exposing it as tunable would invite setting it to 0 and silently
+# removing the guarantee.
+
+
+def surface_names() -> list[str]:
+    """Every tunable surface, in a stable order for menus and listings."""
+    return list(SURFACES)
+
+
+def get_surface(name: str) -> Surface:
+    """Look one up, refusing an unknown name loudly.
+
+    A typo'd surface must not be writable. Settings keys are free-form strings
+    in a generic table, so a tuning call naming `pretool_rule` would otherwise
+    write a key nothing ever reads — a change that appears to succeed, reports a
+    new value, and alters nothing.
+    """
+    try:
+        return SURFACES[name]
+    except KeyError:
+        raise ValueError(
+            f"unknown retrieval surface {name!r}. Tunable surfaces are: "
+            + ", ".join(surface_names())
+        ) from None
+
+
+async def floor_for(user_id: int, name: str) -> float:
+    """This install's current floor for a surface, clamped to [0, 1]."""
+    s = get_surface(name)
+    try:
+        value = float(await get_setting(user_id, s.floor_key, str(s.floor_default)))
+    except (TypeError, ValueError):
+        value = s.floor_default
+    return min(1.0, max(0.0, value))
+
+
+async def budget_for(user_id: int, name: str) -> int:
+    """This install's current budget for a surface, clamped to [1, MAX_BUDGET].
+
+    The lower clamp is 1, never 0: a surface turned off is turned off by its
+    `enabled` switch, which says so. A budget of zero would be an arm that runs
+    a search, logs a retrieval, and renders nothing — indistinguishable in the
+    telemetry from a bar nothing cleared, which is the exact confusion this
+    milestone exists to remove.
+    """
+    s = get_surface(name)
+    raw = await get_setting(user_id, s.budget_key, "")
+    if not raw and s.budget_falls_back_to:
+        raw = await get_setting(user_id, s.budget_falls_back_to, "")
+    try:
+        value = int(float(raw)) if raw else s.budget_default
+    except (TypeError, ValueError):
+        value = s.budget_default
+    return min(MAX_BUDGET, max(1, value))