FabledScribe/src/scribe/mcp/tools/retrieval_tuning.py

"""Reading and moving a retrieval surface's floor and budget (#4102).

The write half of the loop `retrieval_telemetry` opens. That tool says what a
bar did; these say what the bar IS, and let it be changed with the argument
attached.
"""
from scribe.mcp._context import current_user_id
from scribe.services import retrieval_migration as migration_svc
from scribe.services import retrieval_tuning as tuning_svc


async def retrieval_surfaces() -> dict:
    """Every retrieval surface Scribe pushes on, with its floor and its budget.

    Read this before changing either, and read it beside
    `retrieval_telemetry(days=…, near_miss_samples=5)` — this tool says what is
    in force, that one says what it did.

    Each surface carries what it ASKS, what corpus it asks OVER, and how often
    it FIRES, because a floor cannot be moved sensibly without all three. An arm
    firing before every Bash call is spending attention on a scale an arm firing
    once a turn is not, and the same number means different things to a query
    that is a shell command, a code payload, or an operator's sentence.

    `floor` is a COST floor: is this worth ranking at all. It is not a relevance
    judgement — relevance is decided by the reader, which is the only
    participant that can read a record's trigger against the actual situation,
    and every injected line says so out loud ("read it before deciding it does
    not apply").

    `budget` is what actually binds. It is how many lines this surface may spend
    on one injection, and it is the control to reach for when a surface feels
    noisy — lowering a budget removes the weakest candidates, while raising a
    floor removes whichever candidates happen to sit under a number.

    `last_change` carries the reason last given for each dial, or is absent when
    the surface is still on its shipped starting point. Those starting points
    are starting points: they were measured against one corpus with one
    embedding model and cannot be right for another install by construction.
    That is why this tool exists rather than a better set of defaults.

    `calibration` says what SPACE each number was chosen in, per dial, and
    whether that space has moved. A floor is a cosine distance in one embedding
    model's geometry over documents cut one particular way; swap the model and
    every score shifts at once, re-cut the documents and the same record embeds
    different text. Either way the number is a measurement of something that no
    longer exists, and no amount of telemetry will say so — the scores simply
    come out different and the bar keeps applying.

    Read the three fields apart:

    - `stale: true` with `model_changed` — the geometry is new. Every floor on
      this install needs re-measuring, not adjusting; a number that meant "quite
      similar" in the old space means nothing particular in this one.
    - `stale: true` with `shape_changed` — the documents are cut differently.
      The scale still holds, but what a record embeds has changed, so which
      records clear a bar has.
    - `stale: null`, `source: "unstamped"` — the dial was moved before Scribe
      recorded this. It was measured under SOMETHING and there is no way to say
      what, which is a different answer from "it is fine". Treat it as worth
      re-measuring, and the next tuning call stamps it.

    NOTHING IS RETUNED AUTOMATICALLY on the strength of this, here or anywhere.
    A stale stamp says a number is no longer a measurement; it does not say what
    the number should be. That judgement wants the same procedure as any other
    tuning change — `retrieval_telemetry(near_miss_samples=5)`, open the records
    it names, then `tune_retrieval` with what you read in the reason.
    """
    return {"surfaces": await tuning_svc.current_settings(current_user_id())}


async def tune_retrieval(
    surface: str, dial: str, value: float, reason: str, actor: str = "model",
) -> dict:
    """Move one surface's floor or budget, recording why.

    DO NOT CALL THIS FROM A PERCENTILE ALONE. `retrieval_telemetry` gives
    `near_misses.p90` — the mass sitting just under the bar — and that number
    says nothing about whether the mass is RELEVANT. The two have been measured
    disagreeing: one surface logged 69 consecutive declines with the refused
    record 0.0006 under its bar, and every percentile said "lower it". The
    refused record turned out to be a rule about interpreting a REQUEST, matched
    against a query about report layout — a false positive. Lowering would have
    attached that rule to every completion report ever written. The statistic
    and the correct action pointed in opposite directions, and only opening the
    record could tell.

    So the procedure is: `retrieval_telemetry(near_miss_samples=5)`, then
    `get_rule` / `get_note` the `record_id`s it names, and decide whether those
    records SHOULD have surfaced for those queries. Then move the dial, and say
    in `reason` what you read and what it showed.

    `reason` is required and must be substantive. It is the guardrail, not
    bookkeeping: it is what lets the operator review a change they did not make,
    disagree with it, and revert it. A number that moved with no stated basis is
    one nobody can audit — including you, next week.

    WHICH DIAL. Reach for `budget` when the complaint is volume, and `floor`
    when the complaint is quality. Raising a floor to quieten a surface throws
    away its best candidates along with its worst, because a floor cannot tell
    rank from relevance; lowering a budget keeps the top of the ranking and
    drops the tail, which is usually what was wanted.

    Args:
        surface: the surface name from `retrieval_surfaces` — also its
            `retrieval_telemetry` source, so the two always name the same arm.
        dial: "floor" or "budget".
        value: the new value. A floor is clamped to [0, 1], a budget to a whole
            number in [1, 10]. The result says whether a clamp bit, which
            matters: believing you set a value the registry corrected means
            reading the next telemetry as evidence about a bar never in force.
        reason: what you read and what it showed. Required.
        actor: "model" (default) or "human" — pass "human" only when relaying a
            value the operator chose themselves, so the audit trail can tell
            a change they made from one made on their behalf.
    """
    return await tuning_svc.set_dial(
        current_user_id(), surface, dial, value, reason=reason, actor=actor,
    )


async def retrieval_tuning_history(surface: str = "", limit: int = 20) -> dict:
    """What has been changed about retrieval on this install, newest first.

    Reach for it before moving a dial that has been moved before — the previous
    reason is the argument the next change has to answer, and a surface that has
    been walked up and down repeatedly is evidence the floor is not the problem.

    Args:
        surface: restrict to one surface; omit for everything.
        limit: how many events, default 20, capped at 200.
    """
    return {
        "events": await tuning_svc.tuning_history(
            current_user_id(), surface=surface or None, limit=limit,
        )
    }


async def migrate_retrieval_floor(
    surface: str, sample: int = 200, apply: bool = False,
) -> dict:
    """Carry one surface's floor across an embedding-model or chunker change.

    Reach for this when `retrieval_surfaces` reports a dial `stale` — and only
    then. A floor is a cosine similarity, so a new embedding model moves every
    score on the install at once and the stored number silently stops describing
    anything. Re-deriving six floors by reading telemetry is the work this
    avoids.

    WHAT IT TRANSFERS. Not the number — the SELECTIVITY. A floor's real content
    is a decision about how much of what an arm sees is worth spending attention
    on, and that decision survives a change of units. This measures what fraction
    of the surface's recent calls the old floor admitted, re-scores those same
    queries under the current model, and proposes the value that admits the same
    fraction.

    DRY RUN BY DEFAULT. With `apply=False` it returns the arithmetic and writes
    nothing. Read it before applying: the sample sizes, both score ranges, and
    whether the shift looks like a change of scale or like a corpus that has not
    finished re-embedding. A model change is the moment every number is
    uncertain at once, which is the worst moment to let a statistic move dials
    unattended.

    AND IT IS STILL A STARTING POINT. Percentile-preserving means the new floor
    is as good as the old one was, not better — if the old floor was wrong, this
    faithfully carries the wrongness onto the new scale. It is the number to
    begin from while the surface collects enough calls to judge properly, which
    is `retrieval_telemetry(near_miss_samples=5)` and `tune_retrieval` as usual.

    Args:
        surface: the surface name from `retrieval_surfaces`.
        sample: how many recent logged calls to re-score, default 200. Each one
            is an embedding plus a scan, so this is real work; a surface with
            only a handful of logged calls gives a percentile made of noise.
        apply: False (default) returns the proposal. True writes it, as an
            ordinary tuning event with the arithmetic in its reason — so a
            migrated floor is reviewable and revertible like any other.
    """
    return await migration_svc.migrate_floor(
        current_user_id(), surface, sample=sample, apply=apply,
    )


def register(mcp) -> None:
    mcp.tool(name="retrieval_surfaces")(retrieval_surfaces)
    mcp.tool(name="migrate_retrieval_floor")(migrate_retrieval_floor)
    mcp.tool(name="tune_retrieval")(tune_retrieval)
    mcp.tool(name="retrieval_tuning_history")(retrieval_tuning_history)