bvandeusenandClaude Opus 5
8be555d6dd
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 31s
CI & Build / TypeScript typecheck (push) Successful in 37s
CI & Build / Python tests (push) Successful in 1m4s
CI & Build / Build & push image (push) Successful in 28s
Making the rule arms log every call exposed a second ambiguity in the same
row. `result_count == 0` is two unrelated events wearing one number:
- the ranker found nothing above the bar — the only evidence a threshold is
set too high; and
- the ranker found only what this session had already been shown — which
says nothing whatever about the bar.
A long session excludes its way into the second, so the arm reads worse the
longer it runs correctly. Rows written now carry the ambiguity permanently,
which is why this lands before any watch period rather than after.
`retrieval_logs.suppressed_count` (0095, nullable) holds what the caller
dropped as already-shown. Both rule arms report it; they filter in Python and
always know. The note arms pass exclusions INTO semantic_search_notes and
never see what was dropped, so they store NULL.
THE NULL IS LOAD-BEARING. It means "not measured here", and the readout
renders it as `suppression: null` rather than a zeroed dict. Defaulting to 0
would let an unmeasured surface read as a perfectly clean one — the same
substitution of an artifact for a measurement that #3311 made. No backfill,
for the same reason: existing rows genuinely do not know.
`retrieval_telemetry`'s `sources` gains `suppression` with `measured_calls`,
`calls_with_suppression` and `zero_because_already_shown`; subtract the last
from `zero_result_calls` for the true ranker declines. The MCP tool docstring
says to read the two together and warns against reading the null as a zero.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
53 lines
1.8 KiB
Python
53 lines
1.8 KiB
Python
"""add retrieval_logs.suppressed_count — tell a ranker decline from a repeat (#3497)
|
|
|
|
Revision ID: 0095
|
|
Revises: 0094
|
|
Create Date: 2026-09-03
|
|
|
|
`result_count == 0` has always meant "this surface said nothing", which is the
|
|
right number for "was the hint any use" and the wrong one for tuning a
|
|
threshold. It folds together two unrelated events:
|
|
|
|
- the ranker found nothing above the bar — the ONLY evidence a threshold is
|
|
set too high; and
|
|
- the ranker found something the session had already been shown — a decline
|
|
that says nothing whatever about the bar.
|
|
|
|
The rule arms filter in Python after the search, so they can count the second
|
|
kind exactly. The note arms pass `exclude_ids` INTO semantic_search_notes, so
|
|
the dropped rows never come back and there is nothing to count.
|
|
|
|
NULLABLE, AND THE NULL IS THE POINT. A surface that does not measure
|
|
suppression stores NULL, not 0, and the readout renders it as "not measured"
|
|
rather than "none". Defaulting to 0 would make an unmeasured surface look like
|
|
a perfectly clean one — the exact substitution of an artifact for a
|
|
measurement that #3311 made and that #3497 exists to correct. Doing it again,
|
|
in the migration that fixes it, would be its own small joke.
|
|
|
|
No backfill for the same reason: existing rows genuinely do not know, and
|
|
saying so is the honest state. `retrieval_logs` is not restored from backup,
|
|
so no importer changes.
|
|
|
|
Downgrade drops the column. Purely observational — nothing reads it for
|
|
correctness.
|
|
"""
|
|
from alembic import op
|
|
import sqlalchemy as sa
|
|
|
|
|
|
revision = "0095"
|
|
down_revision = "0094"
|
|
branch_labels = None
|
|
depends_on = None
|
|
|
|
|
|
def upgrade() -> None:
|
|
op.add_column(
|
|
"retrieval_logs",
|
|
sa.Column("suppressed_count", sa.Integer(), nullable=True),
|
|
)
|
|
|
|
|
|
def downgrade() -> None:
|
|
op.drop_column("retrieval_logs", "suppressed_count")
|