bvandeusenandClaude Opus 5
8826be7a91
CI & Build / Python lint (push) Failing after 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 30s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Skipped
Milestone 333 step 1. The write-path standing-rule arm is the only retrieval surface in Scribe whose usefulness cannot be observed — and, not coincidentally, the only one that has never declined to fire. 296 calls, zero zero-result, 100% clearing its threshold, while every other surface declines most of the time (#3311, and re-measured in note #3430). `retrieval_logs` gives it scores; scores say what the ranker thought, never whether the hint landed. WHY A SIBLING TABLE AND NOT A COLUMN ON note_usage_events. The row carries no note-specific field and the readout is the same shape, which is the strongest case for sharing that note #3163 admits. What decides against it is identity at RESTORE: the note importer maps note_id through note_id_map, so a rule id parked in that column comes back attached to whatever note holds that number in the target database. Not dropped — reattached. The restore reports success, the counters are populated, and every one is about the wrong record, with no other field to disagree with. rule_versions made the same call for the same reason; this is the third rule-side sibling and it reads like the first two. FK-free on rule_id and user_id, matching note_usage_events / retrieval_logs / app_logs, and deliberately unlike rule_versions. A version belongs to a rule's history and dies with it; telemetry outlives what it describes. Deleting a rule must not erase the evidence that it was surfaced forty times and opened never, because that evidence is the case for having deleted it. The service uses `background.spawn` rather than a third copy of the strong-reference dance — that module's own docstring says new callers should, and a fourth copy is how one of them drifts. The AppLog canary #2663 demands is kept, and since `rule_usage` needed exactly `note_usage`'s semantics, that canary moved into `background.report_telemetry_failure` and note_usage now calls it. `retrieval_telemetry` deliberately keeps its own: its canary is a different shape (one process-wide flag, no AppLog row), so repointing it would change behaviour rather than consolidate it. No ambient bucket, and that is a decision. The note twin splits ranked from ambient surfacings because enter_project and the skill sync deliver records without choosing them (#2477). Rules have the same problem waiting — list_always_on_rules loads them wholesale — but nothing emits here yet, so an empty AMBIENT_SOURCES would be machinery pretending to a distinction the data does not contain. `source` stays granular, so the split stays a readout-level change needing no migration. Backup carries it (v14). The round-trip test seeds a NOTE alongside the rule so the target database has a note id to collide with — without that decoy, a restore running rule ids through the wrong map would merely drop them and the test would pass by absence, rather than failing on the populated-and-wrong result that is the actual hazard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
87 lines
4.0 KiB
Python
87 lines
4.0 KiB
Python
"""add rule_usage_events — was a surfaced rule ever read? (milestone 333 step 1)
|
|
|
|
Revision ID: 0094
|
|
Revises: 0093
|
|
Create Date: 2026-09-02
|
|
|
|
The sibling `note_usage_events` has had since 0071, and the third rule-side
|
|
table to arrive after `rule_embeddings` and `rule_versions` — each one added
|
|
because the rule side kept inheriting machinery built for notes and getting
|
|
the weaker version of it.
|
|
|
|
WHAT IT MEASURES. The write-path standing-rule arm is the only retrieval
|
|
surface in Scribe whose usefulness cannot be observed, and — not coincidentally
|
|
— the only one that has never declined to fire. Over 30 days it took 296 calls,
|
|
returned something on every one, and cleared its threshold 100% of the time,
|
|
while every other surface declines most of the time (#3311). That is either a
|
|
perfectly tuned surface or a bar it cannot fail to clear, and `retrieval_logs`
|
|
cannot tell them apart: it records what the ranker scored, never whether the
|
|
hint was any use.
|
|
|
|
WHY NOT A rule_id COLUMN ON note_usage_events. The row shares no note-specific
|
|
fields and the aggregate readout is the same shape, which is the strongest case
|
|
for sharing that note #3163 admits. What decides against it is identity at
|
|
RESTORE: `note_usage_events`'s importer maps `note_id` through `note_id_map`
|
|
and drops what does not resolve. A rule id parked in that column would come
|
|
back from a backup silently reattached to whatever note took that number —
|
|
telemetry not merely lost but wrong, and wrong in a way nothing downstream
|
|
could detect. `rule_versions` made the same call for the same reason.
|
|
|
|
FK-free on `rule_id` and `user_id`, matching note_usage_events, retrieval_logs
|
|
and app_logs — and deliberately unlike `rule_versions`, which does carry FKs.
|
|
The difference is what the row is for: a version belongs to a rule's history
|
|
and dies with it; telemetry outlives the row it describes. Deleting a rule must
|
|
not erase the evidence that it was surfaced forty times and opened never, since
|
|
that evidence is exactly the case for having deleted it.
|
|
|
|
No CHECK on `event`, matching the note twin. Rule 36 governs adding a value to
|
|
a column that is already gated; it does not require gating one that never was,
|
|
and a two-member enum whose members are written by two functions in one module
|
|
is not where that discipline earns its cost.
|
|
|
|
Downgrade drops the table outright. The data is purely observational — nothing
|
|
reads it for correctness, so losing it costs history and no behaviour.
|
|
"""
|
|
from alembic import op
|
|
import sqlalchemy as sa
|
|
|
|
|
|
revision = "0094"
|
|
down_revision = "0093"
|
|
branch_labels = None
|
|
depends_on = None
|
|
|
|
|
|
def upgrade() -> None:
|
|
op.create_table(
|
|
"rule_usage_events",
|
|
# BigInteger throughout where the note twin uses Integer: rules.id is
|
|
# BigInteger, so rule_id must be, and a high-churn append-only table is
|
|
# a poor place to discover an id ceiling.
|
|
sa.Column("id", sa.BigInteger(), primary_key=True),
|
|
sa.Column(
|
|
"created_at",
|
|
sa.DateTime(timezone=True),
|
|
nullable=False,
|
|
server_default=sa.text("now()"),
|
|
),
|
|
sa.Column("user_id", sa.BigInteger(), nullable=True),
|
|
sa.Column("rule_id", sa.BigInteger(), nullable=False),
|
|
sa.Column("event", sa.Text(), nullable=False),
|
|
sa.Column("source", sa.Text(), nullable=False),
|
|
)
|
|
# Every readout is "these rule ids, split by event", so the composite is the
|
|
# one that actually gets used; the others serve pruning and per-user views.
|
|
op.create_index(
|
|
"ix_rule_usage_rule_event", "rule_usage_events", ["rule_id", "event"]
|
|
)
|
|
op.create_index("ix_rule_usage_created_at", "rule_usage_events", ["created_at"])
|
|
op.create_index("ix_rule_usage_user_id", "rule_usage_events", ["user_id"])
|
|
|
|
|
|
def downgrade() -> None:
|
|
op.drop_index("ix_rule_usage_user_id", table_name="rule_usage_events")
|
|
op.drop_index("ix_rule_usage_created_at", table_name="rule_usage_events")
|
|
op.drop_index("ix_rule_usage_rule_event", table_name="rule_usage_events")
|
|
op.drop_table("rule_usage_events")
|