fix(telemetry): silence is only an answer where the ledger was watching (#4268)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 59s
CI & Build / Python tests (push) Successful in 1m44s
CI & Build / Build & push image (push) Successful in 31s

`band_hugs_floor` compares an arm's weakest tenth against its floor, and
#4225 made it SUSPEND rather than soften when the floor moved inside the
window — because across a change the scores and the bar come from two
different populations. It reads `floor_moves_since`, which answers "did
this arm's floor move", and it read an empty answer as "no, it held
steady".

Those are the same answer only where the ledger was watching. Before an
arm's first floor row there is nothing to move, nothing to report, and no
way to tell a steady floor from an unrecorded one. The suspension was
reading absence of evidence as evidence of absence, and the symptom is the
one #4225 documented: a band "-0.0208 above its floor" — an impossible
negative distance, printed with the suspension silent.

Every install passes through this. The ledger's first row for an arm is
written when that install first boots the release that records baselines,
so any window longer than the install is old reaches back past it. The
lowered-floor direction hides: the gap comes out comfortably positive and
reads as a clean bill of health.

`floor_history_gaps(since)` answers the question its companion cannot:
which arms' floor history does not REACH the start of the window. Arms
come from `surface_names()`, not from the ledger — an arm the ledger has
never heard of is exactly the one at risk, so it cannot be the ledger that
decides which arms get asked about.

Baselines count here, which is the one place the two deliberately
disagree. A baseline records a default without changing it, so it is not a
move and `floor_moves_since` filters it out. It IS the ledger beginning to
observe the arm, and from that moment silence genuinely means the floor
held — filtering it out here would suspend the band check forever on every
install that has never tuned.

`floor_history_unknown` sits between the known move and the band, so an
arm with a date to give gives it. Two sentences, because the remedies
differ: a date says ask again with a smaller `days`; no history at all
says there is nothing to wait for, and names what starts the record.

Why now: #4261 measures the work-log change by reading these warnings, and
every window for the next month opens before this ledger's first row.
Measuring against an instrument that prints a number it cannot support is
the #4225 trap one level up.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
This commit is contained in:
2026-09-21 14:38:16 -04:00
co-authored by Claude Opus 5
parent a58a225d7b
commit 42360f616c
4 changed files with 367 additions and 6 deletions
+58 -3
View File
@@ -40,7 +40,7 @@ from scribe.services.retrieval_registry import (
POINTS, UNBIDDEN, get_point, is_registered, sources_expected_to_emit,
)
from scribe.services.retrieval_surfaces import SURFACES, floor_for
from scribe.services.retrieval_tuning import floor_moves_since
from scribe.services.retrieval_tuning import floor_history_gaps, floor_moves_since
from scribe.services.settings import get_setting
logger = logging.getLogger(__name__)
@@ -500,7 +500,8 @@ def _warn(code, detail, source=None, **numbers) -> dict:
def _compute_warnings(sources: dict, usage: dict, rule_usage: dict,
floors: dict, min_calls: int, epsilon: float,
floor_moves: dict | None = None) -> list[dict]:
floor_moves: dict | None = None,
floor_gaps: dict | None = None) -> list[dict]:
"""The four checks, over whatever sources the window actually contains.
DELIBERATELY NOT KEYED ON A HARD-CODED SOURCE LIST. An arm added next
@@ -634,6 +635,7 @@ def _compute_warnings(sources: dict, usage: dict, rule_usage: dict,
floor = floors.get(name)
p10 = (b.get("top_score") or {}).get("p10")
moved = (floor_moves or {}).get(name)
gaps = floor_gaps or {}
if calls >= min_calls and floor is not None and p10 is not None:
# ── The floor moved inside the window ────────────────────────
#
@@ -666,6 +668,48 @@ def _compute_warnings(sources: dict, usage: dict, rule_usage: dict,
f"`days` that starts after it, or wait.",
source=name, floor=floor, p10=p10, moved_at=moved,
))
# ── The ledger does not reach back this far ──────────────────
#
# The same suspension, for the case the one above cannot see. The
# check overhead asks `floor_moves_since` whether this arm's floor
# moved, and reads an empty answer as "it held steady". That
# reading is only sound where the ledger was WATCHING; before its
# first row for this arm, "no move recorded" and "no move" are
# different statements, and the first was standing in for the
# second.
#
# Not theoretical, and not a one-off: every install passes
# through it. A window longer than the install is old reaches back
# past the ledger's first row, and the symptom is the same
# impossible negative the comment above describes — printed with
# the suspension silent, because there was nothing on record for
# it to notice.
#
# Deliberately ordered after the known move: when both apply, the
# arm that has a date to give should give it.
elif name in gaps:
known_from = gaps[name]
out.append(_warn(
"floor_history_unknown",
(
f"this arm's floor has been recorded since "
f"{known_from}, which is after this window opens, so "
f"whether it moved earlier in the window is not known "
f"— and an unrecorded move is exactly what makes a "
f"band reading meaningless. The band check is "
f"suspended for this arm until the window starts at "
f"or after that date: ask again with a smaller `days`, "
f"or wait."
if known_from else
f"this arm has no floor history at all, so there is "
f"nothing to say whether its floor of {floor} is the "
f"one its calls were made under. The band check is "
f"suspended for this arm until a floor is recorded for "
f"it — moving it once with `tune_retrieval`, or the "
f"next release baseline, starts the record."
),
source=name, floor=floor, p10=p10, known_from=known_from,
))
elif p10 - floor < epsilon:
gap = p10 - floor
out.append(_warn(
@@ -1445,9 +1489,20 @@ async def retrieval_summary(
except Exception: # pragma: no cover - telemetry never raises
logger.warning("could not read floor changes", exc_info=True)
# And the arms whose floor history does not REACH the start of the window.
# `floor_moves` answers "did it move"; for a window that opens before this
# arm's first ledger row, that answer is unavailable rather than no — and
# the band check was reading the two as the same. Read the same way and for
# the same reason: the ledger belongs to the install, not to an account.
floor_gaps: dict[str, str | None] = {}
try:
floor_gaps = await floor_history_gaps(since)
except Exception: # pragma: no cover - telemetry never raises
logger.warning("could not read floor history coverage", exc_info=True)
out["warnings"] = _compute_warnings(
out["sources"], usage, rule_usage, floors, min_calls, epsilon,
floor_moves,
floor_moves, floor_gaps,
)
# "Active" means this window saw real traffic SOMEWHERE. Without that