Commit Graph
3 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5 8be555d6dd feat(telemetry): tell a ranker decline from a repeat before the observation window opens (#3497)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 31s
CI & Build / TypeScript typecheck (push) Successful in 37s
CI & Build / Python tests (push) Successful in 1m4s
CI & Build / Build & push image (push) Successful in 28s
Making the rule arms log every call exposed a second ambiguity in the same
row. `result_count == 0` is two unrelated events wearing one number:

  - the ranker found nothing above the bar — the only evidence a threshold is
    set too high; and
  - the ranker found only what this session had already been shown — which
    says nothing whatever about the bar.

A long session excludes its way into the second, so the arm reads worse the
longer it runs correctly. Rows written now carry the ambiguity permanently,
which is why this lands before any watch period rather than after.

`retrieval_logs.suppressed_count` (0095, nullable) holds what the caller
dropped as already-shown. Both rule arms report it; they filter in Python and
always know. The note arms pass exclusions INTO semantic_search_notes and
never see what was dropped, so they store NULL.

THE NULL IS LOAD-BEARING. It means "not measured here", and the readout
renders it as `suppression: null` rather than a zeroed dict. Defaulting to 0
would let an unmeasured surface read as a perfectly clean one — the same
substitution of an artifact for a measurement that #3311 made. No backfill,
for the same reason: existing rows genuinely do not know.

`retrieval_telemetry`'s `sources` gains `suppression` with `measured_calls`,
`calls_with_suppression` and `zero_because_already_shown`; subtract the last
from `zero_result_calls` for the true ranker declines. The MCP tool docstring
says to read the two together and warns against reading the null as a zero.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-03 21:12:38 -04:00
bvandeusenandClaude Fable 5 b0eda32575 refactor(models): one iso() for every to_dict timestamp; mixins replace hand-rolled created_at/updated_at (#2827, milestone 296 area 3)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / integration (push) Successful in 25s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 57s
CI & Build / Build & push image (push) Successful in 26s
Reading all 28 models against each other: 54 `x.isoformat() if x else None` /
`x.isoformat()` expressions in 23 to_dict methods, in two guarded/unguarded
wordings, become iso() from models/base.py — uniform, and a row read before
flush serialises as null instead of raising. Rulebook / RulebookTopic / Rule
carried byte-identical copies of TimestampMixin's two columns;
InvitationToken / PasswordResetToken / NoteUsageEvent carried CreatedAtMixin's —
all six now use the mixin. AppLog and RetrievalLog keep their explicit
created_at, commented: their composite index orders on `created_at.desc()`,
which needs the column object in the class body. Schema-neutral (same column
definitions) — no migration.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 11:17:42 -04:00
bvandeusenandClaude Opus 4.8 807f478cac feat(search): retrieval telemetry — log every semantic retrieval
CI & Build / Python lint (push) Successful in 2s
CI & Build / integration (push) Successful in 16s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 44s
CI & Build / Build & push image (push) Successful in 58s
Add retrieval_logs (migration 0068) + services/retrieval_telemetry with a
fire-and-forget record_retrieval(), wired into the MCP search tool
(source=mcp_search) and the REST search route (source=rest_search). Captures
query, effective params, and the per-result score distribution so KB-injection
thresholds can be tuned from data rather than guessed.

Scribe: project 2, milestone 93, task 1032.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E
2026-06-22 20:10:15 -04:00