feat(telemetry): tell a ranker decline from a repeat before the observation window opens (#3497) #139

Merged
bvandeusen merged 1 commits from dev into main 2026-09-03 21:23:01 -04:00
Owner

Making the rule arms log every call exposed a second ambiguity in the same row. result_count == 0 is two unrelated events wearing one number:

  • the ranker found nothing above the bar — the only evidence a threshold is set too high; and
  • the ranker found only what this session had already been shown — which says nothing whatever about the bar.

A long session excludes its way into the second, so the arm reads worse the longer it runs correctly.

Why it merges now rather than after a watch period

Rows written from here on carry the ambiguity permanently. The fix only helps rows written after it deploys, so holding it on dev while sessions run spends exactly the thing it exists to protect.

What changed

retrieval_logs.suppressed_count (migration 0095, nullable) holds what the caller dropped as already-shown. Both rule arms report it — they filter in Python and always know. The note arms pass exclusions into semantic_search_notes and never see what was dropped, so they store NULL.

The null is load-bearing. It means "not measured here", and the readout renders suppression: null rather than a zeroed dict. Defaulting to 0 would let an unmeasured surface read as a perfectly clean one — the same substitution of an artifact for a measurement that #3311 made. No backfill, for the same reason: existing rows genuinely do not know.

retrieval_telemetry's sources gains suppression with measured_calls, calls_with_suppression and zero_because_already_shown. Subtract the last from zero_result_calls for the true ranker declines. The MCP tool docstring now says to read the two together and warns against reading the null as a zero.

Deploy

Runs migration 0095 — a single nullable ADD COLUMN, no backfill, no table rewrite. No plugin change.

CI green on dev @ 8be555d (run 5481, all six jobs; the integration lane ran the migration against pg17).

🤖 Generated with Claude Code

https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ

Making the rule arms log every call exposed a second ambiguity in the same row. `result_count == 0` is two unrelated events wearing one number: - the ranker found nothing above the bar — the only evidence a threshold is set too high; and - the ranker found only what this session had already been shown — which says nothing whatever about the bar. A long session excludes its way into the second, so the arm reads worse the longer it runs correctly. ## Why it merges now rather than after a watch period Rows written from here on carry the ambiguity permanently. The fix only helps rows written after it deploys, so holding it on `dev` while sessions run spends exactly the thing it exists to protect. ## What changed `retrieval_logs.suppressed_count` (migration 0095, nullable) holds what the caller dropped as already-shown. Both rule arms report it — they filter in Python and always know. The note arms pass exclusions **into** `semantic_search_notes` and never see what was dropped, so they store NULL. **The null is load-bearing.** It means "not measured here", and the readout renders `suppression: null` rather than a zeroed dict. Defaulting to 0 would let an unmeasured surface read as a perfectly clean one — the same substitution of an artifact for a measurement that #3311 made. No backfill, for the same reason: existing rows genuinely do not know. `retrieval_telemetry`'s `sources` gains `suppression` with `measured_calls`, `calls_with_suppression` and `zero_because_already_shown`. Subtract the last from `zero_result_calls` for the true ranker declines. The MCP tool docstring now says to read the two together and warns against reading the null as a zero. ## Deploy Runs migration 0095 — a single nullable `ADD COLUMN`, no backfill, no table rewrite. No plugin change. CI green on `dev` @ `8be555d` (run 5481, all six jobs; the integration lane ran the migration against pg17). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
bvandeusen added 1 commit 2026-09-03 21:22:57 -04:00
feat(telemetry): tell a ranker decline from a repeat before the observation window opens (#3497)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 31s
CI & Build / TypeScript typecheck (push) Successful in 37s
CI & Build / Python tests (push) Successful in 1m4s
CI & Build / Build & push image (push) Successful in 28s
8be555d6dd
Making the rule arms log every call exposed a second ambiguity in the same
row. `result_count == 0` is two unrelated events wearing one number:

  - the ranker found nothing above the bar — the only evidence a threshold is
    set too high; and
  - the ranker found only what this session had already been shown — which
    says nothing whatever about the bar.

A long session excludes its way into the second, so the arm reads worse the
longer it runs correctly. Rows written now carry the ambiguity permanently,
which is why this lands before any watch period rather than after.

`retrieval_logs.suppressed_count` (0095, nullable) holds what the caller
dropped as already-shown. Both rule arms report it; they filter in Python and
always know. The note arms pass exclusions INTO semantic_search_notes and
never see what was dropped, so they store NULL.

THE NULL IS LOAD-BEARING. It means "not measured here", and the readout
renders it as `suppression: null` rather than a zeroed dict. Defaulting to 0
would let an unmeasured surface read as a perfectly clean one — the same
substitution of an artifact for a measurement that #3311 made. No backfill,
for the same reason: existing rows genuinely do not know.

`retrieval_telemetry`'s `sources` gains `suppression` with `measured_calls`,
`calls_with_suppression` and `zero_because_already_shown`; subtract the last
from `zero_result_calls` for the true ranker declines. The MCP tool docstring
says to read the two together and warns against reading the null as a zero.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
bvandeusen merged commit 0915c48bb0 into main 2026-09-03 21:23:01 -04:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: bvandeusen/FabledScribe#139