"""One registry of the retrieval surfaces, and the two numbers each one has (#4102). WHY THIS EXISTS Six push arms each carried their own loose copy of the same shape: a settings key, a default, and a limit that was usually a module constant nobody could change. The read-and-clamp was written out separately in `plugin_context` (twice over, for auto-inject and the write path), in three rule arms, and again as `reply_preferences._threshold`. That was survivable while the numbers were shipped constants an operator occasionally edited. It stops being survivable once the numbers are meant to MOVE. The operator's decision for this step: "the floor should be chosen and adjusted by the model using it. we've come back to something either fails or has to be looked at by the user we need a model consistent surface for the adjustment of these floor values. the user should be able to touch it but the model should be the thing handling it 9 times out of 10." A tuning surface cannot be "model consistent" if every arm spells its own configuration differently. So the arms stop owning their numbers and read them from here instead, and the tuning tool, the routes and the Settings UI all enumerate THIS table rather than hard-coding six special cases. WHAT A FLOOR IS NOW, AND WHAT IT IS NOT It is not a relevance judgement. Relevance is decided by the reader, which is the only participant that can read a trigger against a situation — that is the milestone's whole argument, and the injected line has always said so out loud ("read it before deciding it does not apply"). A floor answers the cheaper question: *is this worth ranking at all*. `k` is what binds, and `k` is a BUDGET — how much of this surface's attention a candidate list may spend. An arm that fires before every Bash call cannot afford what an arm that fires once a turn can. WHY THE DEFAULTS BELOW ARE STARTING POINTS AND NOT ANSWERS A cosine score is a distance in BAAI/bge-small-en-v1.5's vector space, measured against THIS corpus. It cannot transfer to an install with different records, and rule 115 forbids defending a shipped default from this instance's telemetry — which is what made the old design unbuildable: every number was a guess everywhere except here, and there was no mechanism that could ever improve it. The mechanism is the fix. Scribe ships a starting point and the means to correct it, so the values below carry the measurement that motivated them (see the long comments in `plugin_context.py`, which are kept where they are because they record how each number was first arrived at) without claiming to be right for anybody else. HOW A FLOOR SHOULD ACTUALLY BE MOVED By reading the records the floor refused — `retrieval_telemetry( near_miss_samples=N)` returns them by id — and never by the percentile alone. That is not a style preference; it is the one case where the two disagreed and was checked. `report_preference` logged 69 consecutive declines with the refused score 0.0006 under the bar, and every percentile said "lower it". Reading the refused record showed it was rule 77 "Extract intent from loose phrasing", a false positive, so lowering the bar would have delivered that rule on every completion report ever written. The statistic and the correct action pointed in opposite directions, and only opening the record could tell. """ from __future__ import annotations from dataclasses import dataclass from scribe.services.settings import get_setting # A budget nobody should be able to set past. Not a tuning value — a guard on # the worst case, so a mistyped setting cannot turn a menu into a wall of text. # Shared by every surface because it bounds the same thing everywhere: how many # lines a single unsolicited injection may occupy. MAX_BUDGET = 10 @dataclass(frozen=True) class Surface: """One push arm's tunable pair, plus enough prose to tune it responsibly. `asks` / `over` / `fires` are not documentation for this file — they are rendered by the tuning tool and the Settings UI. A floor cannot be moved sensibly by anyone, model or human, who does not know what the query is, what corpus it runs against, or how often it costs something. Those three facts are exactly what separates these arms from each other, and they were previously recoverable only by reading `plugin_context.py`. """ name: str """The telemetry `source` value, and the join key. MUST equal the string this arm passes to `record_retrieval`. Everything useful about tuning depends on that identity: the tool that moves a floor and the table that says what the floor did have to be talking about the same arm. A test asserts it rather than a comment asking nicely. """ floor_key: str floor_default: float budget_key: str budget_default: int asks: str over: str fires: str measured_model: str = "BAAI/bge-small-en-v1.5" measured_shape: int = 1 """What the SHIPPED defaults above were measured against (#4104). A floor is a distance in one embedding model's geometry, over documents cut one particular way. Either can change, and when one does every number in this table describes something that no longer exists. TWO FIELDS, NEVER ONE FUSED STRING (rule 149). A mismatch has to be able to say WHICH half moved: a new embedding model and a re-cut document shape invalidate the same numbers for different reasons and call for different responses. `"@"` could only report that something changed, which is the answer nobody can act on. Same reason `calibration_stamp()` returns a dict and the event table gives each half its own column. Recorded per surface rather than once for the module because they need not move together: a surface retuned after a model change carries the new stamp while its untouched siblings still carry the old one, and telling those apart is the whole job. LITERALS, deliberately, rather than an import of the live values — a stamp says what was true when the number was chosen, so one that tracked the current model would always agree with it and could never report staleness. """ budget_falls_back_to: str = "" """A budget key to inherit when this surface has none of its own set. Only `write_path` uses it, and only because it USED to share auto-inject's `top_k` outright. Giving it a key without this would silently reset the budget of every install that had tuned the shared one — a behaviour change delivered as a default, which is the shape of regression nobody reports because nothing looks broken. """ SURFACES: dict[str, Surface] = { "auto_inject": Surface( name="auto_inject", floor_key="kb_autoinject_threshold", floor_default=0.55, budget_key="kb_autoinject_top_k", budget_default=3, asks="the operator's message, as they typed it", over="notes, snippets, processes and issues", fires="once per operator turn", ), "write_path": Surface( name="write_path", floor_key="kb_writepath_threshold", floor_default=0.68, budget_key="kb_writepath_top_k", budget_default=3, budget_falls_back_to="kb_autoinject_top_k", asks="the code being written, rewritten as a concept query", over="snippets and recorded issues", fires="before every Write and Edit", ), "write_path_rule": Surface( name="write_path_rule", floor_key="kb_rulehint_threshold", floor_default=0.72, budget_key="kb_rulehint_top_k", budget_default=5, asks="the code being written, against rule triggers", over="global rules plus the bound project's own", fires="before every Write and Edit", ), "pre_tool_rule": Surface( name="pre_tool_rule", floor_key="kb_toolrule_threshold", floor_default=0.68, budget_key="kb_toolrule_top_k", budget_default=5, asks="the command about to run, against rule triggers", over="global rules plus the bound project's own", fires="before every Bash call — the busiest arm there is", ), "prompt_rule": Surface( name="prompt_rule", floor_key="kb_promptrule_threshold", floor_default=0.72, budget_key="kb_promptrule_top_k", budget_default=3, asks="the operator's message, against rule triggers", over="global rules plus the bound project's own", fires="once per operator turn", ), "report_preference": Surface( name="report_preference", floor_key="kb_reportpref_threshold", floor_default=0.72, budget_key="kb_reportpref_top_k", budget_default=3, # THE ONE FIXED QUERY, and the reason this arm behaves unlike the rest. # The others score something that varies per call; this one scores a # constant string, so its top score for a given corpus is also a # constant. A floor a hair above that constant is not a quiet arm, it is # a dead one, and no amount of traffic will ever reveal it — which is # precisely how this arm spent 69 calls declining the same record. asks="a fixed question about how to lay out a completion report", over="preferences", fires="when a task finishes", ), } # Reserved slots are deliberately absent. `preference_slot`, `reuse_slot` and # `lesson_slot` borrow their parent arm's floor and are hard-limited to one hit # each, because their entire purpose is to guarantee a single line to a kind of # record that keeps losing a general score contest (#2246, #3894) — or, for # `lesson_slot`, to a kind whose loss is total rather than merely a lost # convenience, since a lesson has no act arm to fall back on and nobody browses # lessons looking for one (milestone 385). A budget of "1" is the feature; # exposing it as tunable would invite setting it to 0 and silently removing the # guarantee. # # They are still logged under their own `source`, which is what keeps them # judgeable without being tunable: `best_available_id` names the record each # bar refused, so a slot that never places, or one that places weak hits, shows # up as evidence rather than as an argument. def surface_names() -> list[str]: """Every tunable surface, in a stable order for menus and listings.""" return list(SURFACES) def get_surface(name: str) -> Surface: """Look one up, refusing an unknown name loudly. A typo'd surface must not be writable. Settings keys are free-form strings in a generic table, so a tuning call naming `pretool_rule` would otherwise write a key nothing ever reads — a change that appears to succeed, reports a new value, and alters nothing. """ try: return SURFACES[name] except KeyError: raise ValueError( f"unknown retrieval surface {name!r}. Tunable surfaces are: " + ", ".join(surface_names()) ) from None def dial_for_key(key: str) -> tuple[str, str] | None: """Which `(surface, dial)` a settings key belongs to, or None. The registry read backwards, and it exists for one caller: the generic `/api/settings` endpoint, which accepts any key at all. Without this, a floor written through that endpoint moves with no event recorded, and the tuning history says nothing happened — a trail with holes in it, which is worse than no trail because it reads as complete. Derived rather than listed so a seventh surface is covered the moment it is added here, which is the only way this stays true. """ for surface in SURFACES.values(): if key == surface.floor_key: return (surface.name, "floor") if key == surface.budget_key: return (surface.name, "budget") return None async def floor_for(user_id: int, name: str) -> float: """This install's current floor for a surface, clamped to [0, 1].""" s = get_surface(name) try: value = float(await get_setting(user_id, s.floor_key, str(s.floor_default))) except (TypeError, ValueError): value = s.floor_default return min(1.0, max(0.0, value)) async def budget_for(user_id: int, name: str) -> int: """This install's current budget for a surface, clamped to [1, MAX_BUDGET]. The lower clamp is 1, never 0: a surface turned off is turned off by its `enabled` switch, which says so. A budget of zero would be an arm that runs a search, logs a retrieval, and renders nothing — indistinguishable in the telemetry from a bar nothing cleared, which is the exact confusion this milestone exists to remove. """ s = get_surface(name) raw = await get_setting(user_id, s.budget_key, "") if not raw and s.budget_falls_back_to: raw = await get_setting(user_id, s.budget_falls_back_to, "") try: value = int(float(raw)) if raw else s.budget_default except (TypeError, ValueError): value = s.budget_default return min(MAX_BUDGET, max(1, value))