Commit Graph
3 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5 8b9b3a1d9b feat(telemetry): the preload emits, and the always-on set stops being unfalsifiable (#3473)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 23s
CI & Build / integration (push) Successful in 31s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 27s
The ranked rule arm became measurable in M333. The preload did not — and
that is the surface whose value is actually in question. `list_always_on_rules`,
the SessionStart block and every `rules_payload` caller handed rules over
wholesale and emitted nothing, so the resident set's token cost was certain
and its usefulness could not be tested even in principle.

Bulk deliveries now record as AMBIENT, beside the ranked count and never
inside pull-through. Folding them in would mean growing the always-on set
depressed the arm's measured precision and trimming it flattered the arm,
neither for any reason to do with the arm.

`RANKED_SOURCES` inverts the note twin's `AMBIENT_SOURCES` deliberately: there
is one ranked rule source and this change adds seven bulk ones, so naming the
rare half makes a forgotten surface default to ambient — under-counting it —
rather than padding the denominator with surfacings nobody chose.

Two lookalike call sites are deliberately left silent, with a test to keep
them that way: the write-path etag arm and `rules_etag_for` read the rules to
build or compare a MARKER and show nobody anything.

No migration — `event` and `source` are plain Text with no CHECK (rule 36
does not apply). Snippet #2858 updated to the new `rules_payload` contract.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-02 23:06:40 -04:00
bvandeusenandClaude Opus 5 6627cfc2f0 feat(rules): a usage badge on the rule list, and the badge becomes canon instead of a second copy (#3319)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 32s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Successful in 33s
Milestone 333 step 5, and rule 27 — the counter had a tuning point from step
4 and no operator-facing one until now.

The task said to reuse the snippet badge's classes rather than mint a parallel
set, citing the eight duplicated CSS families the ledger already carries
(#3207). `.usage-tag` lived in SnippetListView's SCOPED block, so "reuse" was
not available: copying it into the rule pane would have been the ninth family,
and importing it is not a thing a scoped block permits. So it was promoted
rather than copied.

Three pieces, each of which existed once and now exists once:

- `components.css` gains `.usage-tag` / `.usage-dead`, geometry and colour
  only, with the scoped original deleted rather than left behind.
- `UsageBadge.vue` holds the logic the two lists would otherwise duplicate —
  the >=3 dead-weight threshold, the empty-string-renders-nothing rule, the
  tooltip.
- `types/usage.ts` holds `RecordUsage`, one client type over two tables.
  `SnippetUsage` becomes an alias, so no existing consumer changes.

THE ADVICE IS A PROP, and that is the substance rather than the plumbing. The
counts read identically for every kind; the remedy does not. A snippet offered
and never opened should probably be rewritten or deleted — one action. A rule
in the same position has TWO possible causes and the operator has to pick:
its trigger may fire on the wrong work, in which case `when_to_apply` wants
rewording, or it may genuinely not be wanted. Baking "delete it" into the
component would give the wrong nudge half the time on the surface where being
wrong is most expensive, since a deleted rule stops binding behaviour.

The route zero-fills every row through `usage_for_rules`, one aggregate per
page — per-row would be N+1 by construction. That matters more here than for
snippets: every rule on every existing install predates `rule_usage_events`,
so the zero-filled shape IS the common case for a while, and a route that
attached the key only where it found events would leave the badge reading
undefined on almost every row.

`usage_for_rules` had no test at all — step 1 covered the write path and the
zero shape and left the aggregate uncovered, which only became load-bearing
when a list started rendering it. It now has an integration test over real
Postgres, including that a rule with no events comes back zero-filled rather
than absent.

Recorded as snippet #3460, per the design system's own instruction that the
component layer lives as snippets rather than as prose.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
2026-09-02 18:37:55 -04:00
bvandeusenandClaude Opus 5 8826be7a91 feat(telemetry): rule_usage_events — the table, the service, and a restore that maps rule ids through the rule map (#3315)
CI & Build / Python lint (push) Failing after 3s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 30s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Skipped
Milestone 333 step 1. The write-path standing-rule arm is the only retrieval
surface in Scribe whose usefulness cannot be observed — and, not
coincidentally, the only one that has never declined to fire. 296 calls, zero
zero-result, 100% clearing its threshold, while every other surface declines
most of the time (#3311, and re-measured in note #3430). `retrieval_logs`
gives it scores; scores say what the ranker thought, never whether the hint
landed.

WHY A SIBLING TABLE AND NOT A COLUMN ON note_usage_events. The row carries no
note-specific field and the readout is the same shape, which is the strongest
case for sharing that note #3163 admits. What decides against it is identity at
RESTORE: the note importer maps note_id through note_id_map, so a rule id
parked in that column comes back attached to whatever note holds that number in
the target database. Not dropped — reattached. The restore reports success, the
counters are populated, and every one is about the wrong record, with no other
field to disagree with. rule_versions made the same call for the same reason;
this is the third rule-side sibling and it reads like the first two.

FK-free on rule_id and user_id, matching note_usage_events / retrieval_logs /
app_logs, and deliberately unlike rule_versions. A version belongs to a rule's
history and dies with it; telemetry outlives what it describes. Deleting a rule
must not erase the evidence that it was surfaced forty times and opened never,
because that evidence is the case for having deleted it.

The service uses `background.spawn` rather than a third copy of the
strong-reference dance — that module's own docstring says new callers should,
and a fourth copy is how one of them drifts. The AppLog canary #2663 demands is
kept, and since `rule_usage` needed exactly `note_usage`'s semantics, that
canary moved into `background.report_telemetry_failure` and note_usage now
calls it. `retrieval_telemetry` deliberately keeps its own: its canary is a
different shape (one process-wide flag, no AppLog row), so repointing it would
change behaviour rather than consolidate it.

No ambient bucket, and that is a decision. The note twin splits ranked from
ambient surfacings because enter_project and the skill sync deliver records
without choosing them (#2477). Rules have the same problem waiting —
list_always_on_rules loads them wholesale — but nothing emits here yet, so an
empty AMBIENT_SOURCES would be machinery pretending to a distinction the data
does not contain. `source` stays granular, so the split stays a readout-level
change needing no migration.

Backup carries it (v14). The round-trip test seeds a NOTE alongside the rule so
the target database has a note id to collide with — without that decoy, a
restore running rule ids through the wrong map would merely drop them and the
test would pass by absence, rather than failing on the populated-and-wrong
result that is the actual hazard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
2026-09-02 16:55:22 -04:00