feat(rules): a surfaced rule gets an outcome, not just a read (#4212)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m3s
CI & Build / Python tests (push) Failing after 1m6s
CI & Build / Build & push image (push) Skipped
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m3s
CI & Build / Python tests (push) Failing after 1m6s
CI & Build / Build & push image (push) Skipped
Milestone 419 step 1. `rule_usage_events` could say a rule was SURFACED and that it was PULLED. It could not say what happened next, so a rule that fires constantly and is always obeyed and a rule that fires constantly and is never obeyed left byte-identical telemetry. The second is far the more urgent and was the one the readout could not name — measured on a session where three of seven misses were caught by the operator and none by the system. Two new events, `applied` and `departed`, and a `detail` column carrying the why of a departure. No CHECK migration: `event` was created in 0094 as plain Text with no constraint, verified in the migration rather than assumed from the model, so rule 36 does not bite here — said in both places because the next person adding a value will reach for it. THE THIRD STATE IS DERIVED, AND THAT IS THE DESIGN. Read-and-silently- unchanged is the failure this milestone was opened on, and it cannot be reported: an agent that knew it was ignoring a rule would not be ignoring it. So nothing here asks. `applied` and `departed` are reported; the third state is a rule that was opened and left no trace. An `ignored` enum member would collect nothing while reading as though it had measured something, which is #3311's failure — a statistic that could not vary being taken for a finding. `detail` is a column rather than two more bare event strings because a departure stripped of its reason reads back as a miss, so the two states this exists to separate would collapse again one layer down, in the readout, where nobody would see it happen. Nullable: following a rule needs no argument, and an expensive event is one that stops being recorded. `outcome_state` is the single reading of the four states, taking the aggregate `usage_for_rules` already returns, so the badge, the readout and any later session summary cannot disagree about what "followed" means — the drift #3246 found across the rules system. A departure outranks an application: a rule both applied and argued with is a rule someone argued with, and the argument is the half worth surfacing. `rule_outcome` is the MCP door, classed as a WRITE. The read-only set tolerates getters that call record_pulled, but those are reads that leave a trace; this tool's entire effect is the row, and the row carries prose the agent authored. A read-scoped key that can put text in the operator's database is not read-scoped, whatever table it lands in. Backup carries `detail` on both sides. It is the one field here a fresh install cannot re-earn — counts come back by being used again, a stated reason exists once — and #4197 records that the column guard watches the export side only, so the round-trip test is the thing that would catch a one-sided add. Delivery is deliberately not settled here: how an agent gets prompted to record an outcome is step 3's subject, and the same record serves whichever answer that step reaches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
This commit is contained in:
@@ -18,7 +18,9 @@ from scribe.mcp._context import current_user_id
|
||||
from scribe.services import dedup as dedup_svc
|
||||
from scribe.services import rulebooks as rulebooks_svc
|
||||
from scribe.services import trash as trash_svc
|
||||
from scribe.services.rule_usage import record_rule_pulled
|
||||
from scribe.services.rule_usage import (
|
||||
record_rule_outcome, record_rule_pulled,
|
||||
)
|
||||
|
||||
|
||||
# ── Rulebook CRUD ───────────────────────────────────────────────────────
|
||||
@@ -245,6 +247,66 @@ async def get_rule(rule_id: int) -> dict:
|
||||
return await rulebooks_svc.rule_detail(uid, rule)
|
||||
|
||||
|
||||
async def rule_outcome(rule_id: int, outcome: str, why: str = "") -> dict:
|
||||
"""Record what a rule you read ACTUALLY CHANGED — applied, or departed from.
|
||||
|
||||
Call this after a rule has been surfaced to you and you have acted. It is
|
||||
the only way the system can tell a rule that is working from a rule that
|
||||
is being read and ignored: `surfaced` says it was offered, `get_rule` says
|
||||
it was opened, and until this exists neither says whether it made any
|
||||
difference. A rule obeyed every time and a rule ignored every time leave
|
||||
identical telemetry, and the second is the one worth knowing about.
|
||||
|
||||
`outcome` is one of:
|
||||
|
||||
"applied" — it changed what you did, or it confirmed the approach you
|
||||
were already taking. `why` is optional; following a rule is
|
||||
the ordinary case and does not need an argument.
|
||||
"departed" — you read it and deliberately did not follow it. `why` is
|
||||
REQUIRED and is the whole value of the call: a departure
|
||||
without its reason is indistinguishable from a miss when
|
||||
somebody reads this back, and "somebody" is usually you, in
|
||||
a later session, with none of today's context.
|
||||
|
||||
There is deliberately NO value for "read it and ignored it". That state is
|
||||
real, and it is the one this measurement exists to expose — but it is not
|
||||
something you can report, because noticing it is the same act as not doing
|
||||
it. It is derived instead: a rule you opened and never came back to. The
|
||||
honest way to keep yourself out of that bucket is to call this, not to
|
||||
reach for a word that describes it.
|
||||
|
||||
A departure is a legitimate answer and is not a confession. Rules are
|
||||
written for the common case; recording the edge you found is how the rule
|
||||
gets better, and a corpus where nothing is ever departed from is a corpus
|
||||
nobody is really reading.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rule = await rulebooks_svc.get_rule(rule_id, uid)
|
||||
if rule is None:
|
||||
raise ValueError(f"rule {rule_id} not found")
|
||||
choice = (outcome or "").strip().lower()
|
||||
if choice not in ("applied", "departed"):
|
||||
raise ValueError(
|
||||
f"outcome must be 'applied' or 'departed', got {outcome!r}"
|
||||
)
|
||||
if choice == "departed" and not (why or "").strip():
|
||||
raise ValueError(
|
||||
"a departure needs its reason — pass `why`. Without it the record "
|
||||
"cannot be told from a rule that was simply missed."
|
||||
)
|
||||
record_rule_outcome(
|
||||
user_id=uid, rule_id=int(rule.id), outcome=choice,
|
||||
source="mcp_rule_outcome", detail=why,
|
||||
)
|
||||
return {
|
||||
"rule_id": int(rule.id),
|
||||
"title": rule.title,
|
||||
"outcome": choice,
|
||||
"why": (why or "").strip() or None,
|
||||
"recorded": True,
|
||||
}
|
||||
|
||||
|
||||
async def create_rule(
|
||||
topic_id: int, title: str, statement: str, when_to_apply: str,
|
||||
why: str = "", how_to_apply: str = "", order_index: int = 0,
|
||||
@@ -1086,7 +1148,7 @@ def register(mcp) -> None:
|
||||
for fn in (
|
||||
list_rulebooks, get_rulebook, create_rulebook, update_rulebook, delete_rulebook,
|
||||
list_topics, create_topic, update_topic, delete_topic,
|
||||
list_rules, get_rule,
|
||||
list_rules, get_rule, rule_outcome,
|
||||
create_rule, create_project_rule, update_rule, move_rule, delete_rule,
|
||||
create_preference, update_preference,
|
||||
relate_rules, unrelate_rules,
|
||||
|
||||
Reference in New Issue
Block a user