2 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	aee24c9c1c	feat(retrieval): a tuned number carries the space it was measured in (#4104) ... Milestone 416 step 6. A retrieval floor is a cosine similarity, which only means something inside one embedding model's geometry over documents cut one particular way. Change either and every floor on the install keeps applying while describing nothing — and nothing anywhere says so, because the scores simply come out different and the bar goes on cutting. `CHUNKER_VERSION` already solved this for documents: stamped per row, so the backfill re-embeds precisely what is stale. The same idea, applied to the numbers: - `calibration_stamp()` — embedding model + document shape, one definition. TWO fields, never a fused string (rule 149): a mismatch has to say WHICH half moved, because they call for different responses. - `retrieval_tuning_events` gains `embedding_model` / `shape_version` (migration 0104), stamped on every write. Nullable and NOT backfilled — "unstamped" is the honest answer for a row written before this existed, and it reports as `stale: null`, never as fine. - `current_settings` reports calibration per dial: tuned rows from their event, untouched dials from the registry default's own stamp. - `retrieval_surfaces` and the Settings panel show the mismatch. The panel renders ONLY when something is stale, so seeing it at all is the signal. - `migrate_floor` / `migrate_retrieval_floor` answers "a path for thresholds to be inherited by the next model so that they don't have to recalibrate a lot": the raw cosine cannot cross models, but the PERCENTILE it represented can. Measure what fraction of a surface's logged calls the old floor admitted, re-score those queries under the current model, take the value admitting the same fraction. Dry run by default; applying writes an ordinary tuning event with the arithmetic in its reason. Nothing auto-retunes. A stale stamp says a number is no longer a measurement; it does not say what the number should be, and #4102 measured the one case where the statistic and the correct action pointed opposite ways. The load-bearing test is an ABSENCE: no chat-model identifier may appear anywhere in the calibration path. Claude produces none of these scores, so a Claude upgrade must trigger nothing — a false alarm here teaches the operator to ignore the real one on the day bge-small becomes bge-base. Backup v17 carries both columns, unfilled on the way out and on the way back: a round trip must not turn "we don't know" into a stated fact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-17 12:45:48 -04:00
bvandeusenandClaude Opus 5	003bfd7a0a	fix(tests): two readers moved, and the settings guard now checks the registry (#4102) ... CI on 09b4845 caught both, and the second is an improvement rather than a repair. `test_services_reply_preferences` patched `reply_preferences.get_setting`, which the registry refactor removed — its floor and budget are resolved through `retrieval_surfaces` now. Its key-independence test also asserted the arm asks for exactly one key; it asks for two, because both of its numbers are its own since this step, so the assertion names both and adds a check that the registry and the module constant still agree about the floor key. They writing different keys is the failure where the Settings form saves one string and the arm reads another. `test_settings_defaults_agree` parsed `plugin_context.py` for a bare module-level float, and those constants now alias the registry. Rather than teach the regex about aliases, the six retrieval floors are keyed on their SURFACE NAME and read from the registry directly — which is strictly better for this guard: a surface name is also its telemetry source, so a row names the same arm the readout does, and a floor cannot be checked against a stale constant that happened to keep its old value. `PLAN_MATCH_DEFAULT_THRESHOLD` is not a push surface and keeps the older shape, with a note saying why. Added while there: `test_every_tunable_surface_has_a_control`, derived from the registry, so a seventh surface arrives as a failing test rather than as a number only the model can reach (rules 25, 27). Also lands the audit trail this step needs — `retrieval_tuning_events` (model + migration 0103) and `services/retrieval_tuning.py`. The tool layer on top is the next commit; the table is here because `reason` being REQUIRED is the whole guardrail, and the schema is where that starts. A number moved silently leaves nothing for the operator to review or disagree with, and the operator's decision is that the model moves these "9 times out of 10". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-17 11:20:47 -04:00