feat(rules): a surfaced rule gets an outcome, not just a read (#4212)

Milestone 419 step 1. `rule_usage_events` could say a rule was SURFACED and
that it was PULLED. It could not say what happened next, so a rule that fires
constantly and is always obeyed and a rule that fires constantly and is never
obeyed left byte-identical telemetry. The second is far the more urgent and
was the one the readout could not name — measured on a session where three of
seven misses were caught by the operator and none by the system.

Two new events, `applied` and `departed`, and a `detail` column carrying the
why of a departure. No CHECK migration: `event` was created in 0094 as plain
Text with no constraint, verified in the migration rather than assumed from
the model, so rule 36 does not bite here — said in both places because the
next person adding a value will reach for it.

THE THIRD STATE IS DERIVED, AND THAT IS THE DESIGN. Read-and-silently-
unchanged is the failure this milestone was opened on, and it cannot be
reported: an agent that knew it was ignoring a rule would not be ignoring it.
So nothing here asks. `applied` and `departed` are reported; the third state
is a rule that was opened and left no trace. An `ignored` enum member would
collect nothing while reading as though it had measured something, which is
#3311's failure — a statistic that could not vary being taken for a finding.

`detail` is a column rather than two more bare event strings because a
departure stripped of its reason reads back as a miss, so the two states this
exists to separate would collapse again one layer down, in the readout, where
nobody would see it happen. Nullable: following a rule needs no argument, and
an expensive event is one that stops being recorded.

`outcome_state` is the single reading of the four states, taking the aggregate
`usage_for_rules` already returns, so the badge, the readout and any later
session summary cannot disagree about what "followed" means — the drift #3246
found across the rules system. A departure outranks an application: a rule
both applied and argued with is a rule someone argued with, and the argument
is the half worth surfacing.

`rule_outcome` is the MCP door, classed as a WRITE. The read-only set
tolerates getters that call record_pulled, but those are reads that leave a
trace; this tool's entire effect is the row, and the row carries prose the
agent authored. A read-scoped key that can put text in the operator's
database is not read-scoped, whatever table it lands in.

Backup carries `detail` on both sides. It is the one field here a fresh
install cannot re-earn — counts come back by being used again, a stated
reason exists once — and #4197 records that the column guard watches the
export side only, so the round-trip test is the thing that would catch a
one-sided add.

Delivery is deliberately not settled here: how an agent gets prompted to
record an outcome is step 3's subject, and the same record serves whichever
answer that step reaches.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy

src/scribe/mcp/server.py

@@ -185,6 +185,14 @@ _WRITE_TOOLS = frozenset({
     "create_rule", "create_project_rule", "update_rule", "move_rule", "delete_rule",
     "create_preference", "update_preference",
     "relate_rules", "unrelate_rules", "mark_rule_verified",
+    # rule_outcome writes only telemetry, which is the case _READ_ONLY_TOOLS
+    # above explicitly tolerates for getters that call record_pulled. It is
+    # classed as a WRITE anyway, on the difference that matters: those are
+    # reads that happen to leave a trace, while this tool's entire effect is
+    # the row — and the row carries `detail`, free prose the agent authored.
+    # A read-scoped key that can put text into the operator's database is not
+    # read-scoped, whatever table it lands in (#4212).
+    "rule_outcome",
     # retrieval tuning — a write in both senses: it moves the number the arm
     # reads, and it appends the reason to the audit trail (#4102).
     "tune_retrieval",

src/scribe/mcp/tools/rulebooks.py

@@ -18,7 +18,9 @@ from scribe.mcp._context import current_user_id
 from scribe.services import dedup as dedup_svc
 from scribe.services import rulebooks as rulebooks_svc
 from scribe.services import trash as trash_svc
-from scribe.services.rule_usage import record_rule_pulled
+from scribe.services.rule_usage import (
+    record_rule_outcome, record_rule_pulled,
+)
 
 
 # ── Rulebook CRUD ───────────────────────────────────────────────────────
@@ -245,6 +247,66 @@ async def get_rule(rule_id: int) -> dict:
     return await rulebooks_svc.rule_detail(uid, rule)
 
 
+async def rule_outcome(rule_id: int, outcome: str, why: str = "") -> dict:
+    """Record what a rule you read ACTUALLY CHANGED — applied, or departed from.
+
+    Call this after a rule has been surfaced to you and you have acted. It is
+    the only way the system can tell a rule that is working from a rule that
+    is being read and ignored: `surfaced` says it was offered, `get_rule` says
+    it was opened, and until this exists neither says whether it made any
+    difference. A rule obeyed every time and a rule ignored every time leave
+    identical telemetry, and the second is the one worth knowing about.
+
+    `outcome` is one of:
+
+      "applied"  — it changed what you did, or it confirmed the approach you
+                   were already taking. `why` is optional; following a rule is
+                   the ordinary case and does not need an argument.
+      "departed" — you read it and deliberately did not follow it. `why` is
+                   REQUIRED and is the whole value of the call: a departure
+                   without its reason is indistinguishable from a miss when
+                   somebody reads this back, and "somebody" is usually you, in
+                   a later session, with none of today's context.
+
+    There is deliberately NO value for "read it and ignored it". That state is
+    real, and it is the one this measurement exists to expose — but it is not
+    something you can report, because noticing it is the same act as not doing
+    it. It is derived instead: a rule you opened and never came back to. The
+    honest way to keep yourself out of that bucket is to call this, not to
+    reach for a word that describes it.
+
+    A departure is a legitimate answer and is not a confession. Rules are
+    written for the common case; recording the edge you found is how the rule
+    gets better, and a corpus where nothing is ever departed from is a corpus
+    nobody is really reading.
+    """
+    uid = current_user_id()
+    rule = await rulebooks_svc.get_rule(rule_id, uid)
+    if rule is None:
+        raise ValueError(f"rule {rule_id} not found")
+    choice = (outcome or "").strip().lower()
+    if choice not in ("applied", "departed"):
+        raise ValueError(
+            f"outcome must be 'applied' or 'departed', got {outcome!r}"
+        )
+    if choice == "departed" and not (why or "").strip():
+        raise ValueError(
+            "a departure needs its reason — pass `why`. Without it the record "
+            "cannot be told from a rule that was simply missed."
+        )
+    record_rule_outcome(
+        user_id=uid, rule_id=int(rule.id), outcome=choice,
+        source="mcp_rule_outcome", detail=why,
+    )
+    return {
+        "rule_id": int(rule.id),
+        "title": rule.title,
+        "outcome": choice,
+        "why": (why or "").strip() or None,
+        "recorded": True,
+    }
+
+
 async def create_rule(
     topic_id: int, title: str, statement: str, when_to_apply: str,
     why: str = "", how_to_apply: str = "", order_index: int = 0,
@@ -1086,7 +1148,7 @@ def register(mcp) -> None:
     for fn in (
         list_rulebooks, get_rulebook, create_rulebook, update_rulebook, delete_rulebook,
         list_topics, create_topic, update_topic, delete_topic,
-        list_rules, get_rule,
+        list_rules, get_rule, rule_outcome,
         create_rule, create_project_rule, update_rule, move_rule, delete_rule,
         create_preference, update_preference,
         relate_rules, unrelate_rules,

src/scribe/models/rule_usage.py

@@ -7,9 +7,24 @@ from scribe.models.base import CreatedAtMixin, iso
 SURFACED = "surfaced"
 PULLED = "pulled"
 
+# The outcome half (#4212, milestone 419). A rule that was read and then
+# ignored has always been indistinguishable from one that was read and
+# obeyed; these are the two events that can tell them apart.
+#
+# There is deliberately NO third value for "read and ignored". That state is
+# real and is the whole point of the milestone, but it cannot be reported:
+# an agent that knew it was ignoring a rule would not be ignoring it. It is
+# DERIVED — a pull with no outcome — and an enum member for it would collect
+# nothing while reading as though it had measured something, which is #3311's
+# failure exactly.
+APPLIED = "applied"
+DEPARTED = "departed"
+OUTCOMES = (APPLIED, DEPARTED)
+
 
 class RuleUsageEvent(Base, CreatedAtMixin):
-    """One row per time a rule was SURFACED to the agent, or PULLED in full.
+    """One row per time a rule was SURFACED to the agent, PULLED in full, or
+    ACTED ON — applied, or departed from with a stated reason.
 
     The sibling `note_usage_events` has had since 2026-07, third in the line
     after `rule_embeddings` and `rule_versions` — and, like those, it exists
@@ -58,7 +73,13 @@ class RuleUsageEvent(Base, CreatedAtMixin):
     user_id: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
     rule_id: Mapped[int] = mapped_column(BigInteger, nullable=False)
 
-    # 'surfaced' | 'pulled'
+    # 'surfaced' | 'pulled' | 'applied' | 'departed'
+    #
+    # Plain Text with no CHECK, as created in 0094 — which is why 0106 added
+    # the outcome pair without a DROP/ADD migration. Rule 36 governs
+    # CHECK-whitelisted columns and this is not one. Said here as well as in
+    # the migration because this is where the next person adding a value will
+    # look first.
     event: Mapped[str] = mapped_column(Text, nullable=False)
 
     # Which surface produced it. A CONVENTION, not a fixed vocabulary, and the
@@ -76,6 +97,18 @@ class RuleUsageEvent(Base, CreatedAtMixin):
     # without saying why.
     source: Mapped[str] = mapped_column(Text, nullable=False)
 
+    # The WHY of a departure, and the reason the outcome pair is not simply
+    # two more bare event strings. A departure stripped of its reason reads
+    # back as a miss, so the two states this table exists to separate would
+    # collapse again one layer down — in the readout, where nobody would see
+    # it happen.
+    #
+    # Nullable because `applied` needs no argument. Following a rule is the
+    # unremarkable case; demanding prose for it would make the cheap event
+    # expensive, and an expensive event is one that stops being recorded.
+    # Empty on a `surfaced` or `pulled` row, which nobody asks a reason of.
+    detail: Mapped[str | None] = mapped_column(Text, nullable=True)
+
     __table_args__ = (
         # Every readout is "these rule ids, split by event" — a covering
         # composite beats separate single-column indexes for it.
@@ -92,4 +125,5 @@ class RuleUsageEvent(Base, CreatedAtMixin):
             "rule_id": self.rule_id,
             "event": self.event,
             "source": self.source,
+            "detail": self.detail,
         }

src/scribe/services/backup.py

@@ -345,6 +345,12 @@ def _rule_usage_event_rows(rows) -> list[dict]:
         {
             "user_id": r.user_id, "rule_id": r.rule_id, "event": r.event,
             "source": r.source,
+            # The reason a rule was departed from (#4212). Exported because
+            # it is the only field on this table that cannot be recomputed:
+            # counts can be re-derived from a fresh install's own use, a
+            # stated reason cannot, and a `departed` row that comes back
+            # without one is indistinguishable from a rule that was missed.
+            "detail": r.detail,
             "created_at": r.created_at.isoformat() if r.created_at else None,
         }
         for r in rows
@@ -1573,6 +1579,12 @@ async def _restore_v2(data: dict) -> dict:
                 rule_id=mapped_rid,
                 event=ev.get("event", ""),
                 source=ev.get("source", ""),
+                # `.get(...) or None` rather than a bare default: an archive
+                # written before 0106 has no key at all, and one written
+                # after may carry "" for a non-departure row. Both mean "no
+                # reason", and both must land as NULL so the readout does not
+                # have to tell an empty string from an absent one.
+                detail=(ev.get("detail") or None),
                 created_at=_dt(ev.get("created_at")),
             ))
             stats["rule_usage_events"] += 1

src/scribe/services/rule_usage.py

@@ -85,7 +85,9 @@ from sqlalchemy import case, func, select
 
 from scribe.models import async_session
 from scribe.models.base import iso
-from scribe.models.rule_usage import PULLED, SURFACED, RuleUsageEvent
+from scribe.models.rule_usage import (
+    APPLIED, DEPARTED, OUTCOMES, PULLED, SURFACED, RuleUsageEvent,
+)
 from scribe.services.background import report_telemetry_failure, spawn
 
 logger = logging.getLogger(__name__)
@@ -207,6 +209,107 @@ def record_rule_pulled(*, user_id: int | None, rule_id: int, source: str) -> Non
     _schedule(rows)
 
 
+def record_rule_outcome(
+    *,
+    user_id: int | None,
+    rule_id: int,
+    outcome: str,
+    source: str,
+    detail: str = "",
+) -> None:
+    """Fire-and-forget: record what a rule ACTUALLY CHANGED (#4212).
+
+    The third stream, and the one milestone 419 exists for. `surfaced` says
+    the system offered a rule; `pulled` says somebody opened it. Neither says
+    whether it made any difference, so a rule that fires constantly and is
+    always obeyed and a rule that fires constantly and is never obeyed have,
+    until now, produced identical telemetry. The second is far the more
+    urgent and is precisely the one the readout could not name.
+
+    Two outcomes, because there are only two a judge can honestly report:
+
+      APPLIED  — the rule changed what was done, or confirmed it. No `detail`
+                 required: following a rule is the unremarkable case and
+                 charging prose for it is how an event stops being recorded.
+      DEPARTED — read, and deliberately not followed. `detail` is REQUIRED
+                 and is the entire value of the event. A departure without
+                 its reason reads back as a miss, which collapses the two
+                 states this exists to separate.
+
+    THERE IS NO THIRD CALL, and the absence is the design. Read-and-silently-
+    unchanged is real — it is the failure this milestone was opened on — but
+    it cannot be reported, because an agent that knew it was ignoring a rule
+    would not be ignoring it. It is derived: a rule pulled, with no outcome
+    behind it. See `outcome_state`.
+
+    Guarded rather than trusting: a bad outcome or a reasonless departure is
+    dropped and REPORTED, never written. Telemetry that lies is worse than
+    telemetry that is missing (#2663), and a `departed` row with an empty
+    reason is a lie the readout cannot detect.
+    """
+    if outcome not in OUTCOMES:
+        logger.warning("rule outcome rejected: unknown outcome %r", outcome)
+        spawn(_report_failure("outcome_unknown"), site="rule_usage_outcome")
+        return
+    if outcome == DEPARTED and not (detail or "").strip():
+        logger.warning("rule outcome rejected: departure with no reason")
+        spawn(_report_failure("outcome_no_reason"), site="rule_usage_outcome")
+        return
+    try:
+        rows = [
+            {
+                "user_id": user_id,
+                "rule_id": int(rule_id),
+                "event": outcome,
+                "source": source,
+                "detail": (detail or "").strip() or None,
+            }
+        ]
+    except Exception:
+        logger.debug("rule usage payload build failed", exc_info=True)
+        return
+    _schedule(rows)
+
+
+# What a rule's usage says happened to it, in one word. The four states are
+# ordered by how much the system actually knows, and only the last two are
+# new — the point of the milestone is that UNACTED used to be invisible
+# inside APPLIED.
+UNREAD = "unread"        # surfaced, never opened
+UNACTED = "unacted"      # opened, and nothing recorded after — the blind spot
+FOLLOWED = "followed"    # opened and applied
+DEPARTED_FROM = "departed"   # opened and deliberately not followed, with a why
+
+
+def outcome_state(usage: dict) -> str:
+    """The three states milestone 419 asked to be able to tell apart, plus
+    the one that already existed.
+
+    Pure, and reading only the aggregate `usage_for_rules` already returns —
+    so the readout, the badge and any later session summary all answer this
+    question the same way. Two callers computing "was this followed" from raw
+    counts is the drift #3246 found across the rules system, arriving again.
+
+    PRECEDENCE, and it is deliberate: a departure outranks an application.
+    A rule both applied and departed from in the same window is a rule
+    someone argued with, and the argument is the interesting half — reporting
+    it as plain compliance would hide the one row a reader most wants.
+
+    UNACTED is the derived state and the reason this function exists. It is
+    not "no data"; it is a rule that was surfaced, deliberately OPENED, and
+    then left no trace of having mattered. That is a much stronger signal
+    than never having been opened at all, and it is the signal that was
+    previously indistinguishable from compliance.
+    """
+    if int(usage.get("departed_count") or 0):
+        return DEPARTED_FROM
+    if int(usage.get("applied_count") or 0):
+        return FOLLOWED
+    if int(usage.get("pull_count") or 0):
+        return UNACTED
+    return UNREAD
+
+
 def empty_rule_usage() -> dict:
     """The zero readout — what a rule with no recorded events looks like.
 
@@ -227,6 +330,21 @@ def empty_rule_usage() -> dict:
         "surfaced_count": 0,
         "ambient_count": 0,
         "pull_count": 0,
+        # The outcome half (#4212). Zero here means "nothing recorded", which
+        # for a rule that was also never pulled is simply silence — and for
+        # one that WAS pulled is the blind spot this milestone is named for.
+        # `outcome_state` is what tells those apart; no caller should be
+        # reading these counts raw to decide it.
+        #
+        # The REASON for a departure is on the row (`detail`), not here. One
+        # GROUP BY cannot carry the text of the latest departure without a
+        # DISTINCT ON alongside it, and a key that the aggregate could never
+        # fill would read as "no reason given" on every rule that has one —
+        # a permanently-null field that lies. The readout that needs the
+        # prose reads the rows (#4213).
+        "applied_count": 0,
+        "departed_count": 0,
+        "last_outcome_at": None,
         "last_surfaced_at": None,
         "last_pulled_at": None,
     }
@@ -288,7 +406,19 @@ async def usage_for_rules(rule_ids: list[int]) -> dict[int, dict]:
         slot = out.get(int(rule_id))
         if slot is None:
             continue
-        if event == SURFACED and is_amb:
+        # Outcomes first, and never split by ambient. `ambient` asks whether
+        # a RANKER chose to show the rule; an outcome is reported by a judge
+        # after the fact and has no ranker behind it, so the flag is noise
+        # here. Branching on it would silently drop every outcome row into a
+        # bucket nothing reads.
+        if event in OUTCOMES:
+            key = "applied_count" if event == APPLIED else "departed_count"
+            slot[key] = slot[key] + int(n)
+            prev = slot["last_outcome_at"]
+            now = iso(last_at)
+            if now and (prev is None or now > prev):
+                slot["last_outcome_at"] = now
+        elif event == SURFACED and is_amb:
             slot["ambient_count"] = int(n)
         elif event == SURFACED:
             slot["surfaced_count"] = int(n)