feat(telemetry): the preload emits, and the always-on set stops being unfalsifiable (#3473)

The ranked rule arm became measurable in M333. The preload did not — and
that is the surface whose value is actually in question. `list_always_on_rules`,
the SessionStart block and every `rules_payload` caller handed rules over
wholesale and emitted nothing, so the resident set's token cost was certain
and its usefulness could not be tested even in principle.

Bulk deliveries now record as AMBIENT, beside the ranked count and never
inside pull-through. Folding them in would mean growing the always-on set
depressed the arm's measured precision and trimming it flattered the arm,
neither for any reason to do with the arm.

`RANKED_SOURCES` inverts the note twin's `AMBIENT_SOURCES` deliberately: there
is one ranked rule source and this change adds seven bulk ones, so naming the
rare half makes a forgotten surface default to ambient — under-counting it —
rather than padding the denominator with surfacings nobody chose.

Two lookalike call sites are deliberately left silent, with a test to keep
them that way: the write-path etag arm and `rules_etag_for` read the rules to
build or compare a MARKER and show nobody anything.

No migration — `event` and `source` are plain Text with no CHECK (rule 36
does not apply). Snippet #2858 updated to the new `rules_payload` contract.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ

src/scribe/mcp/tools/milestones.py

@@ -57,7 +57,7 @@ async def get_milestone(milestone_id: int) -> dict:
     return {
         "milestone": out,
         "steps": [t.to_dict() for t in steps],
-        **rulebooks_svc.rules_payload(applicable),
+        **rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_milestone"),
     }
 
 

src/scribe/mcp/tools/projects.py

@@ -207,7 +207,7 @@ async def enter_project(project_id: int) -> dict:
         ],
         "design_system": design_system,
         "milestone_summary": milestone_summary,
-        **rulebooks_svc.rules_payload(applicable),
+        **rulebooks_svc.rules_payload(applicable, user_id=uid, source="enter_project"),
         "open_tasks": [
             {
                 "id": t.id, "title": t.title, "status": t.status,
@@ -251,7 +251,7 @@ async def get_project(project_id: int) -> dict:
     applicable = await rulebooks_svc.get_applicable_rules(
         project_id=project_id, user_id=uid,
     )
-    data.update(rulebooks_svc.rules_payload(applicable))
+    data.update(rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_project"))
     return data
 
 

src/scribe/mcp/tools/rulebooks.py

@@ -18,7 +18,7 @@ from scribe.mcp._context import current_user_id
 from scribe.services import dedup as dedup_svc
 from scribe.services import rulebooks as rulebooks_svc
 from scribe.services import trash as trash_svc
-from scribe.services.rule_usage import record_rule_pulled
+from scribe.services.rule_usage import record_rule_pulled, record_rule_surfaced
 
 
 # ── Rulebook CRUD ───────────────────────────────────────────────────────
@@ -265,6 +265,15 @@ async def list_always_on_rules(project_id: int = 0) -> dict:
     """
     uid = current_user_id()
     rules = await rulebooks_svc.list_always_on_rules(uid, project_id=project_id)
+    # AMBIENT source: the resident set, handed over whole. No ranker chose
+    # these, so they must not land in the pull-through numerator's denominator
+    # — but they must land SOMEWHERE, or the largest rule surface in the
+    # product stays the one surface its own scoreboard cannot see (#3473).
+    record_rule_surfaced(
+        user_id=uid,
+        rule_ids=[r.id for r in rules],
+        source="list_always_on_rules",
+    )
     return {
         "rules": [_rule_summary(r) for r in rules],
         "total": len(rules),

src/scribe/mcp/tools/search.py

@@ -197,17 +197,31 @@ It is an UPPER BOUND per surface: a pull records the door it came
     query failed while the rest of the readout stood.
 
     `rule_usage` — the same question for RULES, from `rule_usage_events`:
-    `surfaced`, `pulled` split into `pulled_by_agent` / `pulled_by_human`, the
-    distinct-rule counts, and `pull_through` on the same definition (agent
-    pulls over surfacings).
+    `surfaced` and `ambient`, `pulled` split into `pulled_by_agent` /
+    `pulled_by_human`, the distinct-rule counts, and `pull_through` on the same
+    definition (agent pulls over RANKED surfacings).
 
     A SEPARATE BLOCK, not folded into `usage`, and reading it as one number
     with that is the mistake to avoid. The corpora differ by orders of
     magnitude — a few dozen eligible rules against thousands of notes — so a
     blended ratio would be the note ratio with noise on it and would hide the
-    rule arm entirely. It also has no `ambient` key, because nothing surfaces a
-    rule un-ranked: `list_always_on_rules` and `enter_project` hand over rules
-    wholesale but emit no event, so there is no ambient class to separate.
+    rule arm entirely.
+
+    `surfaced` VS `ambient` IS THE READING THAT MATTERS HERE. `surfaced` counts
+    rules a ranker chose — today only the write-path arm — and those are claims
+    a pull can settle. `ambient` counts BULK DELIVERIES: the SessionStart
+    preload, `list_always_on_rules`, and the `rules_payload` surfaces
+    (`enter_project`, `get_project`, `get_milestone`, `start_planning`,
+    `get_task`), which hand over the whole applicable set at once with nobody
+    choosing anything. A large `ambient` says the resident set is big and
+    arrives often — never that it is useful, and never that it is read.
+
+    `pull_through` therefore divides by `surfaced` alone. Fold the preload in
+    and growing the always-on set would depress the arm's measured precision
+    while trimming it would flatter it, for reasons having nothing to do with
+    the arm. To judge the PRELOAD instead, compare `ambient` against pulls of
+    those same rules over time: a resident set surfaced thousands of times and
+    opened never is the dead-weight signal, one tier up.
 
     Read it against `sources["write_path_rule"]`. That surface has never once
     declined to fire, and until this block existed there was no way to tell a

src/scribe/mcp/tools/tasks.py

@@ -103,7 +103,7 @@ async def get_task(task_id: int) -> dict:
         applicable = await rulebooks_svc.get_applicable_rules(
             project_id=note.project_id, user_id=uid,
         )
-        data.update(rulebooks_svc.rules_payload(applicable))
+        data.update(rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_task"))
     data.update(await access_svc.describe_provenance(uid, note))
     # Same reasoning as get_note's record_pulled, and this is the tool where it
     # matters MOST: auto-inject ranks kind-blind over a corpus that is