fix(telemetry): a search that never ran is not a decline (#3765)
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / Python lint (push) Successful in 4s
CI & Build / TypeScript typecheck (push) Successful in 46s
CI & Build / integration (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m21s
CI & Build / Build & push image (push) Successful in 36s

`best_available_score` was added by #3670 so a bar could be judged from
what it rejected, and it arrived null on four unrelated causes: the corpus
offered nothing, the query was empty, the embedder was down, or the
DATABASE QUERY FAILED. Only the first is a measurement. The fourth is the
#2663 shape — a swallowed failure rendering as a clean zero — inside the
field added to fix an instance of the #2663 shape.

Found while trying to explain why reuse_slot returned nothing on 45 of 45
calls, and auto_inject on 153 of 161. That investigation is still open;
what it established first is that the readout could not answer it.

THE FIX IS NOT A NEW COLUMN. A call that never searched writes no row, so
every remaining null means one thing: searched, and nothing came close.
That is the convention the pre-tool arm already follows for a blank
command — "a row here would report a call that never happened and drag the
clear-rate down with phantom declines" — extended from the case a caller
can see in advance to the ones only the search knows about.

Both searches stamp `report["searched"]` FALSE before anything can return
and True only where a real result set exists, so every early return leaves
it false. It has to be the first thing done to the dict: a return added
above that line would leave the key absent.

ABSENT IS A THIRD STATE AND IT DEFAULTS TO TRUE. A caller that passes no
report cannot know, and the safe reading there is the old behaviour. Only
a real search can report False, so absent means "nobody asked" and never
"it failed" — which is also why 66 existing mocked searches across twelve
test files keep working unchanged rather than being rewritten to simulate
a flag they do not care about.

A FAILURE IS NOT MADE INVISIBLE. semantic_search_notes already logs a
WARNING on a query failure, which is where a broken search belongs: a
counter cannot say "I am broken" without a reader already trusting it.

Three tests, and the middle one is what makes them discriminate — a
blanket `return` passes the first and fails the second, because a call
that searched and came back empty is the only evidence a threshold is too
high (#3497).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
This commit is contained in:
2026-09-09 12:47:46 -04:00
co-authored by Claude Opus 5
parent 277f5df515
commit 623464323e
6 changed files with 159 additions and 0 deletions
@@ -135,6 +135,7 @@ def record_retrieval(
duration_ms: float | None = None,
suppressed: int | None = None,
best_available: float | None = None,
searched: bool = True,
) -> None:
"""Fire-and-forget: record one retrieval call.
@@ -146,9 +147,35 @@ def record_retrieval(
provide. retrieval_logs is not restored at all, so it has no such hazard,
and `source` already distinguishes the surfaces.
`searched=False` WRITES NO ROW, and that is the point rather than an
optimisation. A semantic search has three ways to return nothing without
having run — an empty query, an unavailable embedder, and the broad
`except` around the query itself — and each one currently arrives here
looking exactly like a ranker that declined. Logging it would report a
decline nobody made, drag `zero_result_calls` down with phantom evidence
about a threshold, and leave `best_available_score` null for a reason that
has nothing to do with the corpus. That last ambiguity is #3765: the field
added to judge a bar was null on four unrelated causes, one of them a
swallowed failure, and no reader could tell them apart.
Dropping the row is what makes the remaining nulls mean ONE thing —
"searched, and there was nothing".
The same convention already governs the pre-tool arm: a blank command costs
no embedding query, so it writes no row, because "a row here would report a
call that never happened and drag the clear-rate down with phantom
declines". This extends it from a case the caller could see in advance to
the ones only the search knows about.
A FAILURE IS NOT MADE INVISIBLE BY THIS. `semantic_search_notes` logs a
WARNING on a query failure, which is where a broken search belongs — a
counter cannot say "I am broken" without a reader already trusting it.
Builds the payload inline (synchronously) then schedules the insert so the
caller returns immediately. Never raises — telemetry must not affect search.
"""
if not searched:
return
try:
payload = _build_payload(
user_id=user_id,