feat(rules): a surfaced rule gets an outcome, not just a read (#4212)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m3s
CI & Build / Python tests (push) Failing after 1m6s
CI & Build / Build & push image (push) Skipped

Milestone 419 step 1. `rule_usage_events` could say a rule was SURFACED and
that it was PULLED. It could not say what happened next, so a rule that fires
constantly and is always obeyed and a rule that fires constantly and is never
obeyed left byte-identical telemetry. The second is far the more urgent and
was the one the readout could not name — measured on a session where three of
seven misses were caught by the operator and none by the system.

Two new events, `applied` and `departed`, and a `detail` column carrying the
why of a departure. No CHECK migration: `event` was created in 0094 as plain
Text with no constraint, verified in the migration rather than assumed from
the model, so rule 36 does not bite here — said in both places because the
next person adding a value will reach for it.

THE THIRD STATE IS DERIVED, AND THAT IS THE DESIGN. Read-and-silently-
unchanged is the failure this milestone was opened on, and it cannot be
reported: an agent that knew it was ignoring a rule would not be ignoring it.
So nothing here asks. `applied` and `departed` are reported; the third state
is a rule that was opened and left no trace. An `ignored` enum member would
collect nothing while reading as though it had measured something, which is
#3311's failure — a statistic that could not vary being taken for a finding.

`detail` is a column rather than two more bare event strings because a
departure stripped of its reason reads back as a miss, so the two states this
exists to separate would collapse again one layer down, in the readout, where
nobody would see it happen. Nullable: following a rule needs no argument, and
an expensive event is one that stops being recorded.

`outcome_state` is the single reading of the four states, taking the aggregate
`usage_for_rules` already returns, so the badge, the readout and any later
session summary cannot disagree about what "followed" means — the drift #3246
found across the rules system. A departure outranks an application: a rule
both applied and argued with is a rule someone argued with, and the argument
is the half worth surfacing.

`rule_outcome` is the MCP door, classed as a WRITE. The read-only set
tolerates getters that call record_pulled, but those are reads that leave a
trace; this tool's entire effect is the row, and the row carries prose the
agent authored. A read-scoped key that can put text in the operator's
database is not read-scoped, whatever table it lands in.

Backup carries `detail` on both sides. It is the one field here a fresh
install cannot re-earn — counts come back by being used again, a stated
reason exists once — and #4197 records that the column guard watches the
export side only, so the round-trip test is the thing that would catch a
one-sided add.

Delivery is deliberately not settled here: how an agent gets prompted to
record an outcome is step 3's subject, and the same record serves whichever
answer that step reaches.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
This commit is contained in:
2026-09-20 23:56:05 -04:00
co-authored by Claude Opus 5
parent 0fe19a8440
commit dfcb000719
9 changed files with 615 additions and 11 deletions
+64 -2
View File
@@ -18,7 +18,9 @@ from scribe.mcp._context import current_user_id
from scribe.services import dedup as dedup_svc
from scribe.services import rulebooks as rulebooks_svc
from scribe.services import trash as trash_svc
from scribe.services.rule_usage import record_rule_pulled
from scribe.services.rule_usage import (
record_rule_outcome, record_rule_pulled,
)
# ── Rulebook CRUD ───────────────────────────────────────────────────────
@@ -245,6 +247,66 @@ async def get_rule(rule_id: int) -> dict:
return await rulebooks_svc.rule_detail(uid, rule)
async def rule_outcome(rule_id: int, outcome: str, why: str = "") -> dict:
"""Record what a rule you read ACTUALLY CHANGED — applied, or departed from.
Call this after a rule has been surfaced to you and you have acted. It is
the only way the system can tell a rule that is working from a rule that
is being read and ignored: `surfaced` says it was offered, `get_rule` says
it was opened, and until this exists neither says whether it made any
difference. A rule obeyed every time and a rule ignored every time leave
identical telemetry, and the second is the one worth knowing about.
`outcome` is one of:
"applied" — it changed what you did, or it confirmed the approach you
were already taking. `why` is optional; following a rule is
the ordinary case and does not need an argument.
"departed" — you read it and deliberately did not follow it. `why` is
REQUIRED and is the whole value of the call: a departure
without its reason is indistinguishable from a miss when
somebody reads this back, and "somebody" is usually you, in
a later session, with none of today's context.
There is deliberately NO value for "read it and ignored it". That state is
real, and it is the one this measurement exists to expose — but it is not
something you can report, because noticing it is the same act as not doing
it. It is derived instead: a rule you opened and never came back to. The
honest way to keep yourself out of that bucket is to call this, not to
reach for a word that describes it.
A departure is a legitimate answer and is not a confession. Rules are
written for the common case; recording the edge you found is how the rule
gets better, and a corpus where nothing is ever departed from is a corpus
nobody is really reading.
"""
uid = current_user_id()
rule = await rulebooks_svc.get_rule(rule_id, uid)
if rule is None:
raise ValueError(f"rule {rule_id} not found")
choice = (outcome or "").strip().lower()
if choice not in ("applied", "departed"):
raise ValueError(
f"outcome must be 'applied' or 'departed', got {outcome!r}"
)
if choice == "departed" and not (why or "").strip():
raise ValueError(
"a departure needs its reason — pass `why`. Without it the record "
"cannot be told from a rule that was simply missed."
)
record_rule_outcome(
user_id=uid, rule_id=int(rule.id), outcome=choice,
source="mcp_rule_outcome", detail=why,
)
return {
"rule_id": int(rule.id),
"title": rule.title,
"outcome": choice,
"why": (why or "").strip() or None,
"recorded": True,
}
async def create_rule(
topic_id: int, title: str, statement: str, when_to_apply: str,
why: str = "", how_to_apply: str = "", order_index: int = 0,
@@ -1086,7 +1148,7 @@ def register(mcp) -> None:
for fn in (
list_rulebooks, get_rulebook, create_rulebook, update_rulebook, delete_rulebook,
list_topics, create_topic, update_topic, delete_topic,
list_rules, get_rule,
list_rules, get_rule, rule_outcome,
create_rule, create_project_rule, update_rule, move_rule, delete_rule,
create_preference, update_preference,
relate_rules, unrelate_rules,