fix(telemetry): a search that never ran is not a decline (#3765)
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / Python lint (push) Successful in 4s
CI & Build / TypeScript typecheck (push) Successful in 46s
CI & Build / integration (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m21s
CI & Build / Build & push image (push) Successful in 36s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / Python lint (push) Successful in 4s
CI & Build / TypeScript typecheck (push) Successful in 46s
CI & Build / integration (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m21s
CI & Build / Build & push image (push) Successful in 36s
`best_available_score` was added by #3670 so a bar could be judged from what it rejected, and it arrived null on four unrelated causes: the corpus offered nothing, the query was empty, the embedder was down, or the DATABASE QUERY FAILED. Only the first is a measurement. The fourth is the #2663 shape — a swallowed failure rendering as a clean zero — inside the field added to fix an instance of the #2663 shape. Found while trying to explain why reuse_slot returned nothing on 45 of 45 calls, and auto_inject on 153 of 161. That investigation is still open; what it established first is that the readout could not answer it. THE FIX IS NOT A NEW COLUMN. A call that never searched writes no row, so every remaining null means one thing: searched, and nothing came close. That is the convention the pre-tool arm already follows for a blank command — "a row here would report a call that never happened and drag the clear-rate down with phantom declines" — extended from the case a caller can see in advance to the ones only the search knows about. Both searches stamp `report["searched"]` FALSE before anything can return and True only where a real result set exists, so every early return leaves it false. It has to be the first thing done to the dict: a return added above that line would leave the key absent. ABSENT IS A THIRD STATE AND IT DEFAULTS TO TRUE. A caller that passes no report cannot know, and the safe reading there is the old behaviour. Only a real search can report False, so absent means "nobody asked" and never "it failed" — which is also why 66 existing mocked searches across twelve test files keep working unchanged rather than being rewritten to simulate a flag they do not care about. A FAILURE IS NOT MADE INVISIBLE. semantic_search_notes already logs a WARNING on a query failure, which is where a broken search belongs: a counter cannot say "I am broken" without a reader already trusting it. Three tests, and the middle one is what makes them discriminate — a blanket `return` passes the first and fails the second, because a call that searched and came back empty is the only evidence a threshold is too high (#3497). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
This commit is contained in:
@@ -484,6 +484,7 @@ async def _reserve_slot_for_reuse(
|
||||
threshold=cfg["threshold"], limit=1, project_id=project_id,
|
||||
is_task=None, results=reuse,
|
||||
best_available=_rep.get("best_available_score"),
|
||||
searched=bool(_rep.get("searched", True)),
|
||||
duration_ms=(time.perf_counter() - _t0) * 1000.0,
|
||||
)
|
||||
# Verify the kind rather than trusting the query that asked for it, and
|
||||
@@ -552,6 +553,7 @@ async def build_autoinject_hint(
|
||||
threshold=cfg["threshold"], limit=cfg["top_k"],
|
||||
project_id=(project_id or None), is_task=None, results=hits,
|
||||
best_available=_rep_ai.get("best_available_score"),
|
||||
searched=bool(_rep_ai.get("searched", True)),
|
||||
duration_ms=(time.perf_counter() - t0) * 1000.0,
|
||||
)
|
||||
if not hits:
|
||||
@@ -1046,6 +1048,7 @@ async def build_write_path_hint(
|
||||
best_available=(
|
||||
None if withheld_here else _rep_wp.get("best_available_score")
|
||||
),
|
||||
searched=bool(_rep_wp.get("searched", True)),
|
||||
duration_ms=(time.perf_counter() - t0) * 1000.0,
|
||||
)
|
||||
if hits:
|
||||
@@ -1342,6 +1345,7 @@ async def build_write_path_hint(
|
||||
project_id=project_id,
|
||||
is_task=None, results=fresh, duration_ms=rule_ms,
|
||||
best_available=_rep_wpr.get("best_available_score"),
|
||||
searched=bool(_rep_wpr.get("searched", True)),
|
||||
# What the ranker found and this session had already been told.
|
||||
# Without it a zero row cannot say whether the bar was too high or
|
||||
# the reader was simply ahead of it — and only the first is a
|
||||
@@ -1449,6 +1453,7 @@ async def build_tool_rule_hint(
|
||||
project_id=project_id,
|
||||
is_task=None, results=fresh, duration_ms=duration_ms,
|
||||
best_available=_rep_ptr.get("best_available_score"),
|
||||
searched=bool(_rep_ptr.get("searched", True)),
|
||||
# See the sibling arm. It matters more here: this arm fires on every
|
||||
# Bash call, so a long session excludes its way to an all-zero row
|
||||
# and the threshold looks wrong when nothing about it is.
|
||||
|
||||
Reference in New Issue
Block a user