feat(retrieval): a tuned number carries the space it was measured in (#4104)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 48s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Failing after 1m4s
CI & Build / Build & push image (push) Skipped

Milestone 416 step 6. A retrieval floor is a cosine similarity, which only
means something inside one embedding model's geometry over documents cut one
particular way. Change either and every floor on the install keeps applying
while describing nothing — and nothing anywhere says so, because the scores
simply come out different and the bar goes on cutting.

`CHUNKER_VERSION` already solved this for documents: stamped per row, so the
backfill re-embeds precisely what is stale. The same idea, applied to the
numbers:

- `calibration_stamp()` — embedding model + document shape, one definition.
  TWO fields, never a fused string (rule 149): a mismatch has to say WHICH half
  moved, because they call for different responses.
- `retrieval_tuning_events` gains `embedding_model` / `shape_version`
  (migration 0104), stamped on every write. Nullable and NOT backfilled —
  "unstamped" is the honest answer for a row written before this existed, and
  it reports as `stale: null`, never as fine.
- `current_settings` reports calibration per dial: tuned rows from their event,
  untouched dials from the registry default's own stamp.
- `retrieval_surfaces` and the Settings panel show the mismatch. The panel
  renders ONLY when something is stale, so seeing it at all is the signal.
- `migrate_floor` / `migrate_retrieval_floor` answers "a path for thresholds to
  be inherited by the next model so that they don't have to recalibrate a lot":
  the raw cosine cannot cross models, but the PERCENTILE it represented can.
  Measure what fraction of a surface's logged calls the old floor admitted,
  re-score those queries under the current model, take the value admitting the
  same fraction. Dry run by default; applying writes an ordinary tuning event
  with the arithmetic in its reason.

Nothing auto-retunes. A stale stamp says a number is no longer a measurement;
it does not say what the number should be, and #4102 measured the one case
where the statistic and the correct action pointed opposite ways.

The load-bearing test is an ABSENCE: no chat-model identifier may appear
anywhere in the calibration path. Claude produces none of these scores, so a
Claude upgrade must trigger nothing — a false alarm here teaches the operator
to ignore the real one on the day bge-small becomes bge-base.

Backup v17 carries both columns, unfilled on the way out and on the way back:
a round trip must not turn "we don't know" into a stated fact.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
This commit is contained in:
2026-09-17 12:45:48 -04:00
co-authored by Claude Opus 5
parent dcf800ed65
commit aee24c9c1c
13 changed files with 1002 additions and 2 deletions
+25
View File
@@ -103,6 +103,31 @@ class Surface:
asks: str
over: str
fires: str
measured_model: str = "BAAI/bge-small-en-v1.5"
measured_shape: int = 1
"""What the SHIPPED defaults above were measured against (#4104).
A floor is a distance in one embedding model's geometry, over documents cut
one particular way. Either can change, and when one does every number in
this table describes something that no longer exists.
TWO FIELDS, NEVER ONE FUSED STRING (rule 149). A mismatch has to be able to
say WHICH half moved: a new embedding model and a re-cut document shape
invalidate the same numbers for different reasons and call for different
responses. `"<model>@<n>"` could only report that something changed, which
is the answer nobody can act on. Same reason `calibration_stamp()` returns
a dict and the event table gives each half its own column.
Recorded per surface rather than once for the module because they need not
move together: a surface retuned after a model change carries the new stamp
while its untouched siblings still carry the old one, and telling those
apart is the whole job.
LITERALS, deliberately, rather than an import of the live values — a stamp
says what was true when the number was chosen, so one that tracked the
current model would always agree with it and could never report staleness.
"""
budget_falls_back_to: str = ""
"""A budget key to inherit when this surface has none of its own set.