feat(rules): a surfaced rule gets an outcome, not just a read (#4212)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m3s
CI & Build / Python tests (push) Failing after 1m6s
CI & Build / Build & push image (push) Skipped

Milestone 419 step 1. `rule_usage_events` could say a rule was SURFACED and
that it was PULLED. It could not say what happened next, so a rule that fires
constantly and is always obeyed and a rule that fires constantly and is never
obeyed left byte-identical telemetry. The second is far the more urgent and
was the one the readout could not name — measured on a session where three of
seven misses were caught by the operator and none by the system.

Two new events, `applied` and `departed`, and a `detail` column carrying the
why of a departure. No CHECK migration: `event` was created in 0094 as plain
Text with no constraint, verified in the migration rather than assumed from
the model, so rule 36 does not bite here — said in both places because the
next person adding a value will reach for it.

THE THIRD STATE IS DERIVED, AND THAT IS THE DESIGN. Read-and-silently-
unchanged is the failure this milestone was opened on, and it cannot be
reported: an agent that knew it was ignoring a rule would not be ignoring it.
So nothing here asks. `applied` and `departed` are reported; the third state
is a rule that was opened and left no trace. An `ignored` enum member would
collect nothing while reading as though it had measured something, which is
#3311's failure — a statistic that could not vary being taken for a finding.

`detail` is a column rather than two more bare event strings because a
departure stripped of its reason reads back as a miss, so the two states this
exists to separate would collapse again one layer down, in the readout, where
nobody would see it happen. Nullable: following a rule needs no argument, and
an expensive event is one that stops being recorded.

`outcome_state` is the single reading of the four states, taking the aggregate
`usage_for_rules` already returns, so the badge, the readout and any later
session summary cannot disagree about what "followed" means — the drift #3246
found across the rules system. A departure outranks an application: a rule
both applied and argued with is a rule someone argued with, and the argument
is the half worth surfacing.

`rule_outcome` is the MCP door, classed as a WRITE. The read-only set
tolerates getters that call record_pulled, but those are reads that leave a
trace; this tool's entire effect is the row, and the row carries prose the
agent authored. A read-scoped key that can put text in the operator's
database is not read-scoped, whatever table it lands in.

Backup carries `detail` on both sides. It is the one field here a fresh
install cannot re-earn — counts come back by being used again, a stated
reason exists once — and #4197 records that the column guard watches the
export side only, so the round-trip test is the thing that would catch a
one-sided add.

Delivery is deliberately not settled here: how an agent gets prompted to
record an outcome is step 3's subject, and the same record serves whichever
answer that step reaches.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
This commit is contained in:
2026-09-20 23:56:05 -04:00
co-authored by Claude Opus 5
parent 0fe19a8440
commit dfcb000719
9 changed files with 615 additions and 11 deletions
@@ -0,0 +1,65 @@
"""rule_usage_events carry an outcome, not just a read (#4212, milestone 419)
Revision ID: 0106
Revises: 0105
Create Date: 2026-09-20
Milestone 419's first step. `rule_usage_events` can say a rule was SURFACED
and that it was PULLED. It cannot say what happened next, so these two
sessions leave identical telemetry:
- a rule surfaced, opened, and followed;
- a rule surfaced, opened, and silently ignored.
The second is the more urgent by a distance, and it is the one the readout
cannot name. That is the whole of what this milestone is about, measured on a
session where three of seven misses were caught by the operator and none by
the system.
TWO NEW EVENT VALUES, AND NO CHECK MIGRATION. `event` was created in 0094 as
plain `sa.Text()` with no constraint — checked in the migration itself, not
assumed from the model — so `applied` and `departed` join `surfaced` and
`pulled` without a DROP/ADD pair. Rule 36 governs CHECK-whitelisted columns
and this is not one; noted explicitly because the next reader will reach for
rule 36 here, and should be able to see in one place why it does not bite.
THE THIRD STATE IS DERIVED, AND THAT IS NOT A SHORTCUT. Read-and-silently-
unchanged is the absence of an outcome, and it has to be: an agent that knew
it was ignoring a rule would not be ignoring it. There is no honest way to ask
for that event, so nothing here tries. `applied` and `departed` are reported;
the third state is what is left over when a rule was pulled and neither
arrived. A schema that offered an `ignored` value would collect nothing and
read as though it had measured something, which is the #3311 failure — a
statistic that cannot vary being mistaken for a finding.
`detail` CARRIES THE WHY OF A DEPARTURE, and is the reason this is a column
rather than two more bare event strings. A departure without its reason is
indistinguishable from a miss when someone reads the table back, so the two
states the milestone wants to tell apart would collapse again one layer down.
Nullable because `applied` needs no argument — following a rule is the
unremarkable case, and demanding prose for it would make the cheap event
expensive and stop it being recorded at all.
NO NEW INDEX. Every outcome readout starts from a set of rule ids and narrows
by event, which is exactly `ix_rule_usage_rule_event` (rule_id, event) from
0094. Adding a `detail` index would serve no query anyone has — the column is
read, never filtered on.
"""
import sqlalchemy as sa
from alembic import op
revision = "0106"
down_revision = "0105"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"rule_usage_events",
sa.Column("detail", sa.Text(), nullable=True),
)
def downgrade() -> None:
op.drop_column("rule_usage_events", "detail")