docs(6205): retrieval_tuning comments state the reason, not the instance that taught it
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Build & push image (push) Canceled after 0s
CI & Build / Python tests (push) Canceled after 1m3s
CI & Build / integration (push) Canceled after 1m1s

The module docstring states why the model moves dials and why a reason is
required, without the decision quote or the declined-rule story. Bare
record citations go; the release-default and ledger-gap docstrings describe
the condition they guard instead of the measurement that found it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-21 20:25:15 -04:00
co-authored by Claude Opus 5.5
parent 5605b863b2
commit 502ecf6743
+45 -59
View File
@@ -1,32 +1,23 @@
"""Moving a retrieval surface's floor or budget, with the argument attached (#4102). """Moving a retrieval surface's floor or budget, with the argument attached.
WHY THIS EXISTS WHY THIS EXISTS
The operator's decision for milestone 416 step 4: A floor is chosen and adjusted by the model that uses it; the operator can
touch it, but the model handles it most of the time. `retrieval_surfaces`
"the floor should be chosen and adjusted by the model using it. we've come makes the arms describe their numbers the same way. This module is the write
back to something either fails or has to be looked at by the user we need a half: one call that moves one dial on one surface, records what it was, what
model consistent surface for the adjustment of these floor values. the user it became, who moved it and why, and refuses to do any of that without a
should be able to touch it but the model should be the thing handling it 9 reason.
times out of 10."
`retrieval_surfaces` made the six arms describe their numbers the same way.
This module is the write half: one call that moves one dial on one surface,
records what it was, what it became, who moved it and why, and refuses to do
any of that without a reason.
WHY A REASON IS REQUIRED WHY A REASON IS REQUIRED
Because the alternative was tried and measured. The milestone originally listed Because a statistic and the correct action can point in opposite directions.
self-tuning as a non-goal on the strength of one case: `report_preference` An arm can log dozens of consecutive declines with the refused record a hair
logged 69 consecutive declines with the refused record 0.0006 under the bar, and under the bar, and every percentile in the readout says "lower it" — while
every percentile in the readout said "lower it". Reading the refused record the refused record, once read, turns out to be a false positive that a lower
showed it was rule 77 "Extract intent from loose phrasing" — a false positive — bar would attach to every matching act.
so lowering the bar would have attached that rule to every completion report
ever written. The statistic and the correct action pointed in opposite
directions.
What separated them was opening the record. A required `reason` is the cheapest What separates them is opening the record. A required `reason` is the cheapest
mechanism that makes that step happen: a caller who must write down why has to mechanism that makes that step happen: a caller who must write down why has to
have looked, and a caller who writes down something wrong has left the operator have looked, and a caller who writes down something wrong has left the operator
a sentence to disagree with. A number moved silently leaves nothing. a sentence to disagree with. A number moved silently leaves nothing.
@@ -62,7 +53,7 @@ logger = logging.getLogger(__name__)
DIALS = ("floor", "budget") DIALS = ("floor", "budget")
# The actor for a change nobody made by hand: the shipped default moved # The actor for a change nobody made by hand: the shipped default moved
# between releases. Beside "model" and "human" (#4225). # between releases. Beside "model" and "human".
RELEASE_ACTOR = "release" RELEASE_ACTOR = "release"
# Long enough to say what was read and what it showed; short enough that nobody # Long enough to say what was read and what it showed; short enough that nobody
@@ -96,17 +87,17 @@ def _calibration(row, s, live: dict) -> dict:
Three sources, and they are not interchangeable: Three sources, and they are not interchangeable:
- `tuned` — the dial was moved after #4104 and the event carries its - `tuned` — the dial was moved and the event carries its stamp.
stamp. The only case where the answer is known. The only case where the answer is known.
- `shipped` — the dial has never been moved, so the number in force is - `shipped` — the dial has never been moved, so the number in force is
the registry default, and the registry records what THAT the registry default, and the registry records what THAT
was measured against (`Surface.measured_model`). was measured against (`Surface.measured_model`).
- `unstamped` — the dial was moved before this step existed. It was - `unstamped` — the dial was moved before events carried a stamp. It was
measured under something; naming it would be inventing a measured under something; naming it would be inventing a
fact, so `stale` is None rather than True or False. fact, so `stale` is None rather than True or False.
"Unknown" and "fine" must not render the same. "Unknown" and "fine" must not render the same.
`model_changed` and `shape_changed` are reported apart (rule 149) because `model_changed` and `shape_changed` are reported apart because
they call for different responses: a new embedding model means every number they call for different responses: a new embedding model means every number
is a distance in a geometry that no longer exists, while a re-cut document is a distance in a geometry that no longer exists, while a re-cut document
shape means the same records now embed different text. A caller that only shape means the same records now embed different text. A caller that only
@@ -144,12 +135,12 @@ async def current_settings(user_id: int) -> list[dict]:
the reason last given, so the next change argues with the last one instead the reason last given, so the next change argues with the last one instead
of overwriting it blind. of overwriting it blind.
From #4104 it also carries `calibration` per dial: the embedding model and It also carries `calibration` per dial: the embedding model and
document shape the number in force was measured in, and whether either has document shape the number in force was measured in, and whether either has
moved since. NOTHING AUTO-RETUNES on the strength of it. A stale stamp says moved since. NOTHING AUTO-RETUNES on the strength of it. A stale stamp says
a number is a measurement of a space that no longer exists — it does not a number is a measurement of a space that no longer exists — it does not
say what the number should be now, and the one time a statistic was allowed say what the number should be now, and a statistic is the wrong thing to
to answer that question it was wrong (see the module docstring). The stamp answer that question (see the module docstring). The stamp
is here so a reader knows which floors to go and re-measure. is here so a reader knows which floors to go and re-measure.
""" """
live = calibration_stamp() live = calibration_stamp()
@@ -161,10 +152,9 @@ async def current_settings(user_id: int) -> list[dict]:
# "The newest two rows" is not "the newest row of each kind": a # "The newest two rows" is not "the newest row of each kind": a
# surface whose floor was moved three times and whose budget was # surface whose floor was moved three times and whose budget was
# moved once returns two floor rows, and the budget change # moved once returns two floor rows, and the budget change
# disappears. That was a missing reason when this only fed # disappears — a missing reason in `last_change` and a WRONG
# `last_change`; since #4104 it is also a WRONG calibration answer — # calibration answer: a tuned dial reporting as "still on the
# a tuned dial reporting as "still on the shipped default", which is # shipped default", the one state a reader would not think to check.
# the one state a reader would not think to check.
last = {} last = {}
for dial in DIALS: for dial in DIALS:
row = ( row = (
@@ -172,17 +162,16 @@ async def current_settings(user_id: int) -> list[dict]:
select(RetrievalTuningEvent) select(RetrievalTuningEvent)
.where( .where(
RetrievalTuningEvent.surface == name, RetrievalTuningEvent.surface == name,
# THIS USER'S CHANGES *OR* A RELEASE'S (#4225). # THIS USER'S CHANGES *OR* A RELEASE'S.
# A release's rows carry no user id because no user # A release's rows carry no user id because no user
# made them, and taking the newest of the two is # made them, and taking the newest of the two is
# what makes the answer true: a dial the operator # what makes the answer true: a dial the operator
# has tuned is explained by their change, and an # has tuned is explained by their change, and an
# untouched one by the release that last shipped # untouched one by the release that last shipped
# its default. Filtering to the user alone reported # its default. Filtering to the user alone would
# "still on the shipped starting point" for a # report "still on the shipped starting point" for
# default that had in fact moved — the one state a # a default that had in fact moved — the one state
# reader would not think to check, which is the # a reader would not think to check.
# same sentence #4104 wrote about the bug above.
or_( or_(
RetrievalTuningEvent.user_id == user_id, RetrievalTuningEvent.user_id == user_id,
RetrievalTuningEvent.user_id.is_(None), RetrievalTuningEvent.user_id.is_(None),
@@ -270,7 +259,7 @@ async def set_dial(
await set_setting(user_id, key, stored) await set_setting(user_id, key, stored)
async with async_session() as session: async with async_session() as session:
# Stamped with the space this number was chosen in (#4104). Read at # Stamped with the space this number was chosen in. Read at
# write time rather than passed in: the caller measuring a floor and # write time rather than passed in: the caller measuring a floor and
# the caller recording it are the same call, so there is no window in # the caller recording it are the same call, so there is no window in
# which they could disagree. # which they could disagree.
@@ -301,7 +290,7 @@ async def set_dial(
async def record_release_defaults() -> list[dict]: async def record_release_defaults() -> list[dict]:
"""Write down that a RELEASE moved a shipped default (#4225). """Write down that a RELEASE moved a shipped default.
THE HOLE THIS FILLS. `retrieval_tuning_events` records dial turns — a THE HOLE THIS FILLS. `retrieval_tuning_events` records dial turns — a
person or a model choosing a number. It is silent about the other way a person or a model choosing a number. It is silent about the other way a
@@ -310,12 +299,10 @@ async def record_release_defaults() -> list[dict]:
one of those consumers is `band_hugs_floor`, which compares a percentile one of those consumers is `band_hugs_floor`, which compares a percentile
against a floor and has no way to notice they come from different regimes. against a floor and has no way to notice they come from different regimes.
Measured on the instance that found this: `write_path_rule` went 0.68 -> A window that straddles a shipped change holds calls made under two bars,
0.72 on 2026-09-02 as a shipped default, and a 30-day window opening and the warning can report a band sitting "-0.0216 above" its floor — a
2026-08-22 therefore held six days of calls made under the old bar. The negative distance above something, which is what a two-population
warning reported the band sitting "-0.0216 above" its floor — a negative comparison looks like when it finally says so out loud.
distance above something, which is what a two-population comparison looks
like when it finally says so out loud.
`user_id IS NULL`, because no user did this. A release acts on every `user_id IS NULL`, because no user did this. A release acts on every
account that has not overridden the dial, and writing one row per user account that has not overridden the dial, and writing one row per user
@@ -423,18 +410,17 @@ async def floor_moves_since(since) -> dict[str, str]:
async def floor_history_gaps(since) -> dict[str, str | None]: async def floor_history_gaps(since) -> dict[str, str | None]:
"""Surfaces whose floor history does NOT cover a window starting at `since`. """Surfaces whose floor history does NOT cover a window starting at `since`.
The companion `floor_moves_since` needs, and the hole #4225's fix left. The companion `floor_moves_since` needs. That function answers "did this
That function answers "did this arm's floor move inside the window", and an arm's floor move inside the window", and an empty answer must not be read
empty answer was being read as "no, it held steady". Those are the same as "no, it held steady". Those are the same answer only while the LEDGER
answer only while the LEDGER covers the window. Before the ledger existed covers the window. Before an arm's first ledger row there are no rows, so
there are no rows for any arm, so every arm reported "no move" for a period it would report "no move" for a period nobody recorded — absence of
nobody had recorded — absence of evidence arriving as evidence of absence. evidence arriving as evidence of absence.
It is not hypothetical, and it is not a one-off: every install passes Every install passes through this. The ledger's first row for an arm is
through it. The ledger's first row for an arm is written when that install written when that install first records baselines, so for a window longer
first boots the release that records baselines, so for a window longer
than the install is old — or than the arm is old, for an arm added than the install is old — or than the arm is old, for an arm added
later — there is no history to consult, and the band check was reading later — there is no history to consult, and the band check must not read
that silence as a clean bill of health. that silence as a clean bill of health.
Returns ONLY the arms with a gap, mapped to the ISO timestamp from which Returns ONLY the arms with a gap, mapped to the ISO timestamp from which
@@ -488,7 +474,7 @@ async def tuning_history(
RetrievalTuningEvent.user_id == user_id, RetrievalTuningEvent.user_id == user_id,
# Shipped-default changes, which belong to no account and # Shipped-default changes, which belong to no account and
# would otherwise be missing from the one surface built # would otherwise be missing from the one surface built
# for asking why a number is where it is (#4225). # for asking why a number is where it is.
RetrievalTuningEvent.user_id.is_(None), RetrievalTuningEvent.user_id.is_(None),
)) ))
.order_by(RetrievalTuningEvent.created_at.desc()) .order_by(RetrievalTuningEvent.created_at.desc())