"""Reading and moving a retrieval surface's floor and budget (#4102). The write half of the loop `retrieval_telemetry` opens. That tool says what a bar did; these say what the bar IS, and let it be changed with the argument attached. """ from scribe.mcp._context import current_user_id from scribe.services import retrieval_migration as migration_svc from scribe.services import retrieval_tuning as tuning_svc async def retrieval_surfaces() -> dict: """Every retrieval surface Scribe pushes on, with its floor and its budget. Read this before changing either, and read it beside `retrieval_telemetry(days=…, near_miss_samples=5)` — this tool says what is in force, that one says what it did. Each surface carries what it ASKS, what corpus it asks OVER, and how often it FIRES, because a floor cannot be moved sensibly without all three. An arm firing before every Bash call is spending attention on a scale an arm firing once a turn is not, and the same number means different things to a query that is a shell command, a code payload, or an operator's sentence. `floor` is a COST floor: is this worth ranking at all. It is not a relevance judgement — relevance is decided by the reader, which is the only participant that can read a record's trigger against the actual situation, and every injected line says so out loud ("read it before deciding it does not apply"). `budget` is what actually binds. It is how many lines this surface may spend on one injection, and it is the control to reach for when a surface feels noisy — lowering a budget removes the weakest candidates, while raising a floor removes whichever candidates happen to sit under a number. `last_change` carries the reason last given for each dial, or is absent when the surface is still on its shipped starting point. Those starting points are starting points: they were measured against one corpus with one embedding model and cannot be right for another install by construction. That is why this tool exists rather than a better set of defaults. `calibration` says what SPACE each number was chosen in, per dial, and whether that space has moved. A floor is a cosine distance in one embedding model's geometry over documents cut one particular way; swap the model and every score shifts at once, re-cut the documents and the same record embeds different text. Either way the number is a measurement of something that no longer exists, and no amount of telemetry will say so — the scores simply come out different and the bar keeps applying. Read the three fields apart: - `stale: true` with `model_changed` — the geometry is new. Every floor on this install needs re-measuring, not adjusting; a number that meant "quite similar" in the old space means nothing particular in this one. - `stale: true` with `shape_changed` — the documents are cut differently. The scale still holds, but what a record embeds has changed, so which records clear a bar has. - `stale: null`, `source: "unstamped"` — the dial was moved before Scribe recorded this. It was measured under SOMETHING and there is no way to say what, which is a different answer from "it is fine". Treat it as worth re-measuring, and the next tuning call stamps it. NOTHING IS RETUNED AUTOMATICALLY on the strength of this, here or anywhere. A stale stamp says a number is no longer a measurement; it does not say what the number should be. That judgement wants the same procedure as any other tuning change — `retrieval_telemetry(near_miss_samples=5)`, open the records it names, then `tune_retrieval` with what you read in the reason. """ return {"surfaces": await tuning_svc.current_settings(current_user_id())} async def tune_retrieval( surface: str, dial: str, value: float, reason: str, actor: str = "model", ) -> dict: """Move one surface's floor or budget, recording why. DO NOT CALL THIS FROM A PERCENTILE ALONE. `retrieval_telemetry` gives `near_misses.p90` — the mass sitting just under the bar — and that number says nothing about whether the mass is RELEVANT. The two have been measured disagreeing: one surface logged 69 consecutive declines with the refused record 0.0006 under its bar, and every percentile said "lower it". The refused record turned out to be a rule about interpreting a REQUEST, matched against a query about report layout — a false positive. Lowering would have attached that rule to every completion report ever written. The statistic and the correct action pointed in opposite directions, and only opening the record could tell. So the procedure is: `retrieval_telemetry(near_miss_samples=5)`, then `get_rule` / `get_note` the `record_id`s it names, and decide whether those records SHOULD have surfaced for those queries. Then move the dial, and say in `reason` what you read and what it showed. `reason` is required and must be substantive. It is the guardrail, not bookkeeping: it is what lets the operator review a change they did not make, disagree with it, and revert it. A number that moved with no stated basis is one nobody can audit — including you, next week. WHICH DIAL. Reach for `budget` when the complaint is volume, and `floor` when the complaint is quality. Raising a floor to quieten a surface throws away its best candidates along with its worst, because a floor cannot tell rank from relevance; lowering a budget keeps the top of the ranking and drops the tail, which is usually what was wanted. Args: surface: the surface name from `retrieval_surfaces` — also its `retrieval_telemetry` source, so the two always name the same arm. dial: "floor" or "budget". value: the new value. A floor is clamped to [0, 1], a budget to a whole number in [1, 10]. The result says whether a clamp bit, which matters: believing you set a value the registry corrected means reading the next telemetry as evidence about a bar never in force. reason: what you read and what it showed. Required. actor: "model" (default) or "human" — pass "human" only when relaying a value the operator chose themselves, so the audit trail can tell a change they made from one made on their behalf. """ return await tuning_svc.set_dial( current_user_id(), surface, dial, value, reason=reason, actor=actor, ) async def retrieval_tuning_history(surface: str = "", limit: int = 20) -> dict: """What has been changed about retrieval on this install, newest first. Reach for it before moving a dial that has been moved before — the previous reason is the argument the next change has to answer, and a surface that has been walked up and down repeatedly is evidence the floor is not the problem. Args: surface: restrict to one surface; omit for everything. limit: how many events, default 20, capped at 200. """ return { "events": await tuning_svc.tuning_history( current_user_id(), surface=surface or None, limit=limit, ) } async def migrate_retrieval_floor( surface: str, sample: int = 200, apply: bool = False, ) -> dict: """Carry one surface's floor across an embedding-model or chunker change. Reach for this when `retrieval_surfaces` reports a dial `stale` — and only then. A floor is a cosine similarity, so a new embedding model moves every score on the install at once and the stored number silently stops describing anything. Re-deriving six floors by reading telemetry is the work this avoids. WHAT IT TRANSFERS. Not the number — the SELECTIVITY. A floor's real content is a decision about how much of what an arm sees is worth spending attention on, and that decision survives a change of units. This measures what fraction of the surface's recent calls the old floor admitted, re-scores those same queries under the current model, and proposes the value that admits the same fraction. DRY RUN BY DEFAULT. With `apply=False` it returns the arithmetic and writes nothing. Read it before applying: the sample sizes, both score ranges, and whether the shift looks like a change of scale or like a corpus that has not finished re-embedding. A model change is the moment every number is uncertain at once, which is the worst moment to let a statistic move dials unattended. AND IT IS STILL A STARTING POINT. Percentile-preserving means the new floor is as good as the old one was, not better — if the old floor was wrong, this faithfully carries the wrongness onto the new scale. It is the number to begin from while the surface collects enough calls to judge properly, which is `retrieval_telemetry(near_miss_samples=5)` and `tune_retrieval` as usual. Args: surface: the surface name from `retrieval_surfaces`. sample: how many recent logged calls to re-score, default 200. Each one is an embedding plus a scan, so this is real work; a surface with only a handful of logged calls gives a percentile made of noise. apply: False (default) returns the proposal. True writes it, as an ordinary tuning event with the arithmetic in its reason — so a migrated floor is reviewable and revertible like any other. """ return await migration_svc.migrate_floor( current_user_id(), surface, sample=sample, apply=apply, ) def register(mcp) -> None: mcp.tool(name="retrieval_surfaces")(retrieval_surfaces) mcp.tool(name="migrate_retrieval_floor")(migrate_retrieval_floor) mcp.tool(name="tune_retrieval")(tune_retrieval) mcp.tool(name="retrieval_tuning_history")(retrieval_tuning_history)