feat(telemetry): the preload emits, and the always-on set stops being unfalsifiable (#3473)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 23s
CI & Build / integration (push) Successful in 31s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 27s

The ranked rule arm became measurable in M333. The preload did not — and
that is the surface whose value is actually in question. `list_always_on_rules`,
the SessionStart block and every `rules_payload` caller handed rules over
wholesale and emitted nothing, so the resident set's token cost was certain
and its usefulness could not be tested even in principle.

Bulk deliveries now record as AMBIENT, beside the ranked count and never
inside pull-through. Folding them in would mean growing the always-on set
depressed the arm's measured precision and trimming it flattered the arm,
neither for any reason to do with the arm.

`RANKED_SOURCES` inverts the note twin's `AMBIENT_SOURCES` deliberately: there
is one ranked rule source and this change adds seven bulk ones, so naming the
rare half makes a forgotten surface default to ambient — under-counting it —
rather than padding the denominator with surfacings nobody chose.

Two lookalike call sites are deliberately left silent, with a test to keep
them that way: the write-path etag arm and `rules_etag_for` read the rules to
build or compare a MARKER and show nobody anything.

No migration — `event` and `source` are plain Text with no CHECK (rule 36
does not apply). Snippet #2858 updated to the new `rules_payload` contract.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
This commit is contained in:
2026-09-02 23:06:40 -04:00
co-authored by Claude Opus 5
parent 6627cfc2f0
commit 8b9b3a1d9b
14 changed files with 505 additions and 48 deletions
+10 -1
View File
@@ -18,7 +18,7 @@ from scribe.mcp._context import current_user_id
from scribe.services import dedup as dedup_svc
from scribe.services import rulebooks as rulebooks_svc
from scribe.services import trash as trash_svc
from scribe.services.rule_usage import record_rule_pulled
from scribe.services.rule_usage import record_rule_pulled, record_rule_surfaced
# ── Rulebook CRUD ───────────────────────────────────────────────────────
@@ -265,6 +265,15 @@ async def list_always_on_rules(project_id: int = 0) -> dict:
"""
uid = current_user_id()
rules = await rulebooks_svc.list_always_on_rules(uid, project_id=project_id)
# AMBIENT source: the resident set, handed over whole. No ranker chose
# these, so they must not land in the pull-through numerator's denominator
# — but they must land SOMEWHERE, or the largest rule surface in the
# product stays the one surface its own scoreboard cannot see (#3473).
record_rule_surfaced(
user_id=uid,
rule_ids=[r.id for r in rules],
source="list_always_on_rules",
)
return {
"rules": [_rule_summary(r) for r in rules],
"total": len(rules),