Rule outcomes, the contract hint, and four extractor/backup fixes #174

Merged

bvandeusen merged 18 commits from dev into main 2026-09-21 06:31:10 -04:00

Conversation 0 Commits 18 Files Changed 51

+5802 -587

18 Commits

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	029692945e	fix(hooks): the by-name duplicate arm confirms its grep hits against the real extractor (#4227) ... #4222 fixed one end of this defect — the extractors that decide what a payload DEFINES now blank comment and string spans before any line matcher runs. This is the other end: `scribe_local_dups`, which decides which OTHER files already define that name, and which was still a plain `git grep`. A grep sees lines, not spans, so the sentence class with only modifier rules is a deletion that went half-way. — real prose from a module docstring in this repo — matched the arm's pattern for `name=with`. Writing a genuine `change`, `beside` or `wrapped` would be told it already existed, and pointed at a docstring. EVERY HIT IS NOW CONFIRMED by running `scribe_defs` over the candidate file and keeping only names it actually reports. That is the only check that cannot disagree with the other end of the pipe, which is the whole point. MEASURED, NOT ASSUMED — both numbers the task reasoned from turned out wrong. - WHAT IT REMOVES, across 141 payload files of this repo: 15 of 210 report lines. Every one a string literal, a comment, a TypeScript `import { type Foo }`, or Vue's `const emit = defineEmits()` boilerplate. No real definition was lost. Where a name had both — `create_note` — the phantom in a test's `shape_form("async def create_note(...)")` argument dropped out and the definition in services/notes.py stayed. `with` went from four files to none, which is the correct answer: nothing here defines it, and it is a keyword in several of these languages. - WHAT IT COSTS: mean 193ms -> 222ms, worst 569ms -> 572ms. The task feared "over a second added to a PreToolUse hook" from 48 confirmations. It is about 15%, because the arm was already dominated by its twelve `git grep` calls, and because confirmation runs once per DISTINCT candidate file rather than once per (name, file) pair. A deliberately pathological payload — nine names that are ordinary English words — reaches 28 distinct files and 947KB; `scribe_defs` runs at ~33ms per 250KB. THE CANDIDATE CAP IS RAISED FROM FOUR TO TWELVE, and that is load-bearing. Confirmation REMOVES hits, so capping before it runs lets phantom matches crowd a real definition out of the window — hits dropped before anyone looked at them, which is #4042's bug in a new place. The display cap stays at four and now applies to CONFIRMED hits, which is where a cap belongs. #4042's own `|| true` inside the substitution is untouched, and its regression case still passes. The task's own advice not to fix this by tightening the grep pattern is followed and written down: requiring `(` or `{` or `:` after the name rejects `class with only…` and also `class Foo extends Bar {`, `class Foo : Base()` and `type Foo struct {`. This arm exists because it works with no server, no index and no binding (#2280, #2682), which makes a miss here invisible — a visible false positive is the better failure. Guards: tests/test_hook_duplicate_confirmation.py, against real git repos because what is pinned is the interaction between `git grep`, `head` and the extractor. Seven of the eight fail against the previous implementation; the eighth is #4042's regression case, whose job is to keep passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 02:49:07 -04:00
bvandeusenandClaude Opus 5	edbc31f8ca	feat(lessons): the kind whose whole question is "is this trigger right" was the one kind that could not see its own counts (#4196) ... The lesson slot has recorded surfaced-vs-opened since it shipped. Nothing showed it. `get_lesson`'s REST door attached `usage` to the payload and no view rendered it; the listing did not attach it at all, and neither MCP door did. #4196 asks when a lesson that keeps getting followed should become a rule, and names the trap in the same breath: raw frequency cannot separate "this should bind" from "this trigger is too broad", and the second is the commoner reading by a wide margin. Surfaced-AND-opened can separate them. Neither question is answerable by a reader who cannot see the numbers, which is why this is the first step and not the threshold. NO THRESHOLD IS PROPOSED HERE, deliberately. The corpus today is 10 lessons with 7 recorded surfacings and 3 opens, over about fifteen hours of usage data. A promotion rule fitted to that would be fitting noise — #3311's failure, and the warning lesson #4228 was written to carry. `UsageBadge` already declines to render a verdict under three surfacings for the same reason. So #4196 stays open: its subject, the promotion path, is still unbuilt. What lands is the evidence it needs. - REST `GET /api/lessons` and MCP `list_lessons` attach `usage` to every row, from one aggregate per page rather than a per-row read, which would be N+1 by construction. Every row carries the key zero-filled, so "never surfaced" is a state a reader can see rather than a missing field they have to interpret. - MCP `get_lesson` attaches it too, and reads it BEFORE recording its own pull. That door records a pull on every open — it has to, or the kind sits permanently at zero — which makes the order load-bearing in a way it is not for a kind that only counts. The REST detail door already ordered it this way; the two now agree about what the number means. - `LessonDetailView` renders `UsageBadge` (snippet #3460) rather than re-spelling the chip, with the advice keyed to this kind: a lesson that is repeatedly offered and never opened is usually keyed to a situation nobody is in, so it points at re-keying `when_to_apply`, not at deleting the claim. Guards, in the two styles this pair of doors already uses: the MCP side driven behaviourally through mocks, including the call ORDER for `get_lesson`; the REST side on structure like its siblings in test_lesson_rest_door.py, because the route is decorated and returns a Quart response. Rule 167's falsifier is included. KNOWN GAP, not fixed here: `KnowledgeView` is the only lesson LIST in the UI and it reads `/knowledge`, not `/lessons` — so the REST listing change reaches `frontend/src/api/lessons.ts::listLessons`, which currently has no consumer. The agent-facing listing does reach a reader today. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 02:33:33 -04:00
bvandeusenandClaude Opus 5	5d06b74599	test(backup): prove the three restored code_shapes columns survive real Postgres (#4197) ... The column guard is a claim about kwargs; "the value comes back" is a different claim, and only a round trip settles it. This drives the real `restore_full_backup` over a judged shape carrying all three of the columns that were going missing. `diverges_from` gets the harder assertion. It is a FK to notes.id, so the tempting fix — carry the exported id across — produces a row pointing at whatever note holds that number in the target database: not dropped, REATTACHED, with the restore reporting success and the divergence about the wrong snippet. So the test asserts WHOSE note the pointer landed on rather than which integer it holds, and it refuses to run at all if the restore happened to reuse the source id, which would let it pass without proving anything. Same shape as the assertion in the rule_usage round trip, for the same seam. Also corrects this module's own docstring, which said the round-trip module "is not written yet". Three existed; this is the fourth. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 02:18:51 -04:00
bvandeusenandClaude Opus 5	33346da381	fix(backup): a restore could lose a column and still report success — code_shapes lost three (#4197) ... The column guard (#3182) makes a dropped column unexpressible on the way OUT. Nothing watched the way back IN, and that is the worse half: an export gap leaves an obviously thin backup, an import gap means holding a complete, correct file and restoring an incomplete database from it, with a success message. WHAT IT FOUND, the first time it ran. `code_shapes` was exporting `reason_code`, `recheck_at` and `diverges_from` and importing none of them. A restored ledger would have carried every judgment's verdict and not the code for WHY — the column the accounting reads to tell a scoped-css exemption from convention-plumbing — with every recheck flag cleared and every divergence pointer gone. `diverges_from` is a FK to notes.id, so it is re-mapped rather than carried: the raw source id would point at whatever snippet took that number in the destination, which is wrong rather than missing and is the milestone 333 trap one table over. WHY THE GUARD WAS ONE-SIDED. Not an oversight — the code was. Export goes through per-table pure helpers, so a test can hand one a stand-in and read which keys came out. `_restore_v2` built all 26 models inline in one 690-line procedural function, and there was no per-table unit to hand anything to. So the construction moved out, into a `_build_*` helper per table taking the exported row plus the id maps built so far. What deliberately did NOT move is the loops, the flushes and the id-map bookkeeping: that is the order-dependent part, where a mistake is a restore that half-works, and it gains nothing from being split. Returning None is "skip" and a None field is "degrade" — which one a table wants stays the table's own call, because both are right somewhere: a shape event without its project says nothing, while a usage event without one is still a real pull and dropping it would deflate the pull-through the table exists to report. The refactor was checked to be behaviour-preserving before the guard went in: all 26 constructor kwarg sets identical to HEAD, and all 28 skip conditions accounted for — 25 now in builders, 3 in loops that construct nothing (the rule_systems raw insert and the two in the final project patch). The guard then composes the two halves end to end: export a stand-in row, feed THAT dict to the builder, read which columns the model actually received. `test_the_usage_importer_restores_the_reading_project` read the source of `_restore_v2` with `inspect.getsource` because there was nothing to call; it is replaced by tests that call the builders and assert the skip/degrade behaviour directly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 02:15:10 -04:00
bvandeusenandClaude Opus 5	d309fd7f0f	test(drafter): pin the rest of #2990's verify list to the span scan (#4222) ... #2990 asked for three cases. The wrapped-comment-continuation one shipped with #4222; these are the other two — a comment holding a whole rule defines nothing, and an unterminated comment does not swallow the file. Both already pass on both sides; the vectors are what stops a later change to the scan quietly taking them back. The phantom #2990 was filed for is among the twenty-two #4222 removes: `.editor-body` in TaskEditorView.vue, a persisted `code_shapes` row whose signature ends in `*/` and whose `used_by` count is zero — so it has been sitting in the unused_css number the operator is meant to act on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 02:04:06 -04:00
bvandeusenandClaude Opus 5	84476d7ecf	fix(drafter): a wrapped docstring line beginning "class AND the" defines a shape called AND (#4222) ... The definition extractor is line-oriented and knows nothing about what a line is INSIDE. A docstring that wraps onto a line starting with a keyword announces a definition: `AND` reached a live session as a divergence prompt asking it to justify a symbol that does not exist, and `is` reached it as a repo-wide duplicate of four files that define nothing of the kind. Measured, not assumed: running the extractor over every scannable file with and without the scan differs by twenty-two phantoms. Two of them — `with` and `nobody`, both out of the module docstring in check_dangling_styles.py — are persisted `code_shapes` rows that have been judged. Those need no migration: sync_shapes marks a row it no longer extracts as vanished. `ast` would be the honest tool for .py and is not what this uses, because the extractor is mirrored rule for rule by an awk program in the hook, awk cannot parse Python, and a fix only one of the pair can run is the drift the mirror exists to prevent. Both sides now run the same left-to-right scan and blank comment and string spans to their own newlines before any matcher sees a line. Three things the scan has to get right, each of which cost real definitions while it was being written: - a string that HOLDS a marker is not a marker. `"red /* "` in test_design_stylesheet.py and a triple quote inside a single-quoted regex in plugin_context.py each ate every definition below them. - `#` is a colour in CSS and a comment in Python, and the extractor is handed no path. An alphanumeric straight after it settles it. - an unterminated opener blanks NOTHING. The scan rewinds past it and continues, so a stray marker costs one span rather than the rest of the file. The comment claiming the two extractors agree has been the only thing holding them together, and a comment cannot fail. The mirror test now RUNS the hook's awk over the same vectors: with the old program it reports the phantoms, which is what a guard that can fail looks like. Across all 631 scannable files in this repo the two now agree line for line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 02:01:56 -04:00
bvandeusenandClaude Opus 5	04775c3496	fix(drafter): the base rate counted a dataclass and an async service unit as comparable things (#4208) ... A CORRECTION TO THIS ISSUE'S OWN PLAN, first, because the plan was wrong in a way that would have cost the acceptance case. #4208 recommended widening `kind` past `css | sym` to carry the form, calling the denominator the deeper win and a migration the price. Two things: 1. THE MIGRATION IS NOT NEEDED. The issue says density "is computed from stored rows and `shape_form` only runs on read". `canon_density` does `select(CodeShape)...scalars().all()` and filters to the directory in PYTHON; `signature` is a stored column; `canon_form(siblings, ...)` on the next line already derives form from those rows. Bucketing the denominator costs a list comprehension. 2. WIDENING `kind` WOULD BREAK MILESTONE #2793's ACCEPTANCE CASE. If `kind` separated `fn` from `async-fn`, density would bucket them apart — which does silence #4204's four `def` helpers, as claimed. Apply it to a hand-rolled SYNC `confirmDanger` in a directory where an async confirm helper is canon: the canon leaves the candidate's denominator, nothing dominates, no flag. That is the flag the milestone exists to produce, and it is the same inversion the first form gate made, one layer down. So option 1 is not expensive-but-right, it is wrong; and option 2 (compare meaning) is not the interim, it is the only lever. `confirmDanger` beside an async confirm helper and `is_registered` beside a service unit are identical at every structural level — same family, same form contradiction, same density. They differ only in whether the candidate does the canon's JOB, and no signature carries that. WHAT THIS SHIPS is the part that is right and provable: an honest denominator. `comparable_siblings` narrows the count to rows whose family does not contradict the candidate's, using `families_conflict` — the same predicate the gate uses, so the count and the verdict cannot drift into disagreeing about what comparable means. "372 judged siblings" stops counting a dataclass, a constant and an async service unit as three comparable things. NARROWED BY FAMILY, NOT FORM, for the reason above: `fn` beside `async-fn` stays a fair question. An unreadable sibling STAYS COUNTED — dropping it would shrink `judged`, raise the share, and fire the check more on the directories it can read least. Every unknown-form decision in this module goes that way. Density is now per candidate rather than per kind, cached on (kind, form). TWO BUGS THIS CHANGE HAD, both caught before CI and both pinned: - I passed a FAMILY where `families_conflict` reads a FORM, so `shape_family("callable")` returned "" and the narrowing was a silent no-op that still read as applied. The regression guard deliberately uses a callable: `type` is both a form and a family name, so testing with it proves nothing and the bug hides. - My new `_Row` in the test file shadowed the one already there — same fields, different `snippet_id` default — silently breaking three passing `canon_form` tests. Reused the existing class. A duplicate definition quietly changing a neighbour's meaning is this file's own subject. The residue test's docstring said separating the four needs "widen `kind` or a comparison of meaning". Corrected: they are not alternatives. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 01:33:45 -04:00
bvandeusenandClaude Opus 5	512d0326a0	fix(telemetry): a floor that moved inside the window makes the band check a comparison of two populations (#4225) ... `retrieval_telemetry(days=30)` reported, for write_path_rule: "the weakest tenth of what this arm returns scores 0.6984, only -0.0216 above its floor of 0.72" A negative distance above something. The tenth percentile of what an arm RETURNED cannot sit below the floor that gates what it may return — not inside one population. MEASURED CAUSE. write_path_rule's floor was 0.68 until 2026-09-02, when 2385100 (#3318) raised the shipped default to 0.72. The window opened 2026-08-22, so six days of it are calls made under the old bar; top_score.min for the surface is exactly 0.68, the old bar still in the sample. AND THE CHANGE LEFT NO TRACE THE READOUT COULD SEE. retrieval_tuning_events records dial turns — a person or a model choosing a number. It was silent about the other way a floor moves: somebody edits floor_default and ships it. retrieval_tuning_history returned {"events": []} and retrieval_surfaces said last_change: {}, source: "shipped". All true, and all of it silent about a floor that had in fact moved. THE RAISE ANNOUNCED ITSELF. A LOWERED FLOOR WOULD NOT: the gap comes out comfortably positive and reads as a clean bill of health on a sample that half predates the bar being judged. Both directions are now pinned. So the check is SUSPENDED, not softened. band_hugs_floor asks whether the scores are piled on the bar; that needs the scores and the bar to come from the same regime. Where they do not, the honest answer is that this sample cannot say, plus the date after which one can — floor_moved_mid_window replaces band_hugs_floor for that arm and never accompanies it. A reader told a number is unavailable goes and gets one; a reader handed a qualified number uses it. NO MIGRATION. `actor` is Text with no CHECK precisely so a new kind of actor is not one — the model's own comment says so, and this is the case it anticipated. "release" joins "model" and "human". user_id is already nullable, which is right: no user did this, a release acts on every account that has not overridden the dial, and a row per user would both multiply and misattribute it. Both readers now take the newest of (this user's change, the release's). THE FIRST SIGHTING IS A BASELINE, written with old_value NULL. Nothing moved; the row exists so the next release has a predecessor. That null is load-bearing: floor_moves_since asks for old_value IS NOT NULL, so a fresh install's baseline does not silently retire the check on every new install. UI: the tuning history rendered actor as `human ? 'you' : 'Claude'`, so a release row would have told the operator that Claude moved a floor it never touched — the one failure the actor column exists to prevent. Three-way now, with an unknown value printing itself rather than guessing. Recorded at startup, inline and awaited. What #4181 cost three hours was concurrency — a background task racing the hook for the same pool. Sequential creates no contention, and this is twelve single-row reads. It must finish before serving because a readout served before the change was recorded is the exact answer this exists to stop giving. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 01:17:37 -04:00
bvandeusenandClaude Opus 5	36b54bff1f	fix(plugin): the ledger that proves a rule was read was being deleted by the compaction that asked about it (#4217) ... Milestone 419's acceptance, and it failed the first time it was run — which is the only reason this commit exists. THE MEASUREMENT. Ran the step-5 readout against this session's real traffic instead of a fixture. It reported 19 rules named by an arm and none opened. That is false: the session had called `get_rule` 45 times. Across six real sessions on this instance: 208 opens, 3 surviving ledger entries. 1.4%. THE CAUSE. `.opened.ids` was doing two jobs with opposite lifetimes. - "this context HOLDS rule 156" — false after a compaction, and three hooks read it to decide whether to stay quiet. Clearing it is correct. - "rule 156 WAS OPENED" — which no compaction makes untrue, and which the session-end readout is built on. `scribe_clear_session_ledgers` sweeps every `<sid>*.ids` on SessionStart source=compact. Right for the first claim, and it was deleting the second. The TTL did the same thing more quietly: `scribe_rules_live` ages an exclusion ledger, which is right, and would have eaten the early part of any long session's evidence too. So the readout was reporting only the stretch since the last compaction while reading as though it had reported the session — a statistic that cannot vary being mistaken for a finding (#3311), which is the shape this whole milestone exists to stop producing. It ran AT the seam it was blind to. THE SPLIT. `scribe_rules_append` now writes both: the exclusion ledger it always wrote, and `<kind>.keep.ids`, an evidence twin that is never aged and never swept. The readout reads twins; the three `held` readers are untouched. DERIVED, NOT LISTED, because a list is what broke this before — the comment above the sweep says so about its own history. Every ledger written through the appender gets a twin, including the next one somebody adds; a new ledger is born on the swept side unless its name opts out. `scribe_checkpoint_allowed` writes its twin explicitly since it bypasses the appender, and there the split lands right on both sides: the cap counts the swept file, so a compaction honestly restores the budget to stop an act the context can no longer justify, while the record that a stop happened stays. Removed `scribe_ledger_ids`, orphaned by the change — a dead helper beside a live one is a thing the next reader trusts. test_session_ledger_clear.py asserted every ledger dies and could not have caught this: its fixture never created a twin, so the sweep was one glob away from either mistake with only one of them guarded. Both sides now asserted. The slippage tests build their ledgers through the real writers rather than by hand, for the same reason — a fixture that writes the bytes itself keeps passing after the writer stops. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 01:04:17 -04:00
bvandeusenandClaude Opus 5	ae773740b4	feat(plugin): the seam that erases the evidence is where the unresolved rules get named (#4216) ... Step 5 of milestone 419. The milestone's subject is that a rule read and ignored is arithmetically identical to a rule read and followed, and the compaction is where that identity becomes permanent — the turns holding the evidence are summarised away, and the unjudged thing survives as nothing. WHY THIS IS ASSEMBLED IN THE HOOK. `rule_usage_events` has no session column; it is per user over a window. A session-scoped answer therefore cannot be asked of the server, and has to be built where a session is a thing that exists. Four ledgers four hooks already write: .rules.ids an arm NAMED the rule .opened.ids the session called get_rule (#4100) .acted.ids the session called rule_outcome (new here) .checkpoint.ids the rule HELD an act (#4214) Every one is an observed tool call. Nothing asks the model what it followed — milestone 386 ruled that out, because a model asked "did you apply rule 156?" says yes. Two subtractions: named-minus-opened is the arm talking to nobody, opened-minus-acted is the milestone's whole subject. PreCompact stdout is the compaction's custom instructions (#3680), not a message to the model, so the readout does not say "you slipped" — it says which ids must be carried through, which is the one thing a summary can do about an unjudged finding. SILENT WHEN NOTHING HAPPENED, and the accusations are conditional on having members. "0 rules unresolved" on every compaction is how a readout teaches its reader to skip it. Traffic is still reported, because the static instructions already ask for it in prose; these lines are the measured version. scribe_record_outcome.sh is the third ledger's writer, matched on mcp__.*__rule_outcome and mirroring scribe_record_opened.sh: TMPDIR only, silent, exit 0 on every path. A PostToolUse hook that spoke would put a line after every rule_outcome call and give recording an outcome a cost. Also: check_plugin.py skipped the new hook for want of a smoke event, which would have left the newest of the three ledgers as the only one the plugin lane never runs. Added, mirroring its sibling. tests/test_precompact_hook.py now isolates TMPDIR — the hook reads session ledgers from there, so without isolation a test would see whatever this real session had accumulated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 00:53:29 -04:00
bvandeusenandClaude Opus 5	fdfb2d94ac	feat(plugin): you altered the shape of something — here is everything that reads it (#4215) ... Milestone 419 step 4. Five of the milestone's seven misses were the same move: acting on the thing in hand without reading the contract around it. Lesson #4207 says so in words, and was written by its author hours before a structurally identical mistake, having been surfaced twice in the turns between. Text delivered at the moment of acting is too weak a carrier for a reflex that has to change what the act IS. This looks it up instead. Rule 33 one scope down: its checks are between layers, and the same question exists between a definition and its callers. THREE KINDS OF EXPOSED NAME, because a contract breaks three ways that look nothing alike in source — the defined symbol (a rename or removal), its parameter names (arity), and the quoted keys of its dict literals (the shape of what it returns). THE THIRD IS THE ONE A SIGNATURE-WATCHER MISSES, and it is in because of the miss that produced this step. Two commits ago `get_writepath_config` gained one dict key; three arms read that dict inside a fail-open `except`, every one silently became a no-op, and ten tests went red with nothing pointing at the cause. No signature changed. Run against that exact edit, the check now names tests/helpers.py — the actual root cause — among six files, before the write. TWO GATES, AND THE SECOND IS WHAT MAKES IT USABLE. A change to the exposed set is necessary but not sufficient: a definition nothing else references has no contract to break, so the readers lookup runs second and an empty result ends it silently. Body-only edits say nothing, a subject is named once per session, and the ledger lives in the swept directory under the `.ids` convention, so the existing compaction-clear guards cover it — checked against test_session_ledger_clear's own parsers rather than assumed. LOCAL AND SERVERLESS, like the duplicate-name arm beside it. It needs the working tree and nothing else; the server has no checkout, so this is the only place the question can be asked. It is a NUDGE: scribe_prior_art.sh still returns no permissionDecision, which is the operator's recorded decision that a recall aid may not stand in the way of a write. A test asserts that here as well as in test_write_path_trigger.py, because this is the arm most likely to tempt someone into making it a gate — it reports something that may already be broken. The `sym` half delegates to `scribe_defs` rather than repeating its patterns: those cover nine languages and have been corrected several times, and a second copy would inherit today's version and quietly stop agreeing with it (#3497). Verified by lifting the test file's own helpers and driving all 19 cases against the real shell over a fixture git repo. Two of my own errors were caught that way and are fixed: the fixtures were arriving as single lines because Python `repr` inside bash single quotes leaves `\n` as two characters (the extractor is line-oriented, so the tests would have gone green against input no editor can produce), and the no-readers case put its subject in a file that was not the excluded one, so it had a reader and tested the opposite of its name. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 00:43:29 -04:00
bvandeusenandClaude Opus 5	028d5218fc	fix(tests): the prompt arm has no act to hold, so it carries no checkpoint (#4214) ... CI 7126: 1 failed, 2227 passed — down from 14. The last one was mine, and a different mistake from the batch before it: I widened four assertions on the literal `{"context": "", "rule_ids": []}` by replacing the string, without checking which arm each test was calling. Three are the tool arm. The fourth, `test_the_prompt_arm_says_nothing_when_asked_nothing`, is not. The distinction is real rather than an omission, so the assertion goes back with it written down. The two ACT arms can hold a call because there is a composed act to hold. The prompt arm fires on the operator's message, before anything has been decided — there is nothing to put a rule in front of, and a checkpoint there would have to guess at an act that does not exist yet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 00:32:23 -04:00
bvandeusenandClaude Opus 5	4e59af380a	fix(tests): the config stand-in fell behind the real one, and ten arms silently no-opped (#4214) ... CI 7124 Python tests: 14 failed. One root cause behind ten of them, and the failure was the exact one `tests/helpers.writepath_cfg`'s docstring already warns about in prose — while being unable to prevent this instance of it. Three arms read their numbers out of that config dict inside a fail-open `except`. A missing key raises where nobody sees it, so the arm becomes a silent no-op, indistinguishable from the arm working and finding nothing. The helper derives its keys from `retrieval_surfaces.SURFACES` precisely to stop that — and `checkpoint_threshold` is deliberately NOT a surface, because everything in that table is a floor/budget pair belonging to one query and the checkpoint runs none. The derivation therefore could not see it, the write-path rule arm died before `record_retrieval`, and ten tests went red at once. Fixed at the helper, from the module constant, so there is still exactly one literal and it lives in the product. And the guard the docstring claimed now exists: `test_the_config_stand_in_carries_every_key_the_real_one_does` compares the stand-in's key set against the real `get_writepath_config`, so the next key that is not a surface fails loudly here instead of quietly disabling an arm under test. `test_retrieval_surfaces`'s hand-written key list gains it for the same reason, spelled out in place. The other four were the contract widening itself: `checkpoint` is present on every return of the tool arm, including its early ones, so four assertions comparing the whole dict needed it. That key is deliberately always present — the two arms feed one shell reader where an absent key and an empty one are read the same, so the difference is invisible exactly where it would bite. Verified statically: every cfg in the suite now routes through `writepath_cfg` (test_write_path_trigger's local `_cfg` delegates to it), no hand-written config dict survives, and the real config's eight keys match the stand-in's. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 00:30:20 -04:00
bvandeusenandClaude Opus 5	91bc0fb01e	feat(plugin): a high-confidence rule is put in front of a command, not beside its result (#4214) ... Milestone 419 step 3, the pre-act checkpoint. Every rule surface in this plugin returns `additionalContext`, which Claude Code delivers alongside the tool RESULT — so the rule is read after the call is written and lands as commentary on a decision already made. That is the milestone's central finding, measured over a session with seven misses, three caught by the operator and none by this system. The action arm can now return a `deny` instead. The act does not run, the rule's text can be read before the call exists, and the remedy is one `get_rule` call after which the act may be re-submitted unchanged. Nothing reaches the operator: a deny is a message to the model. "CONSEQUENTIAL" IS DERIVED, NOT ENUMERATED. The obvious implementation lists act kinds — a write to product code, a schema change, a bulk classification, a merge. Every one of those is consequential because THIS operator wrote rules about it, and shipping that list is this instance's corpus hard-coded into the product (rule 115). So the corpus decides: an act is consequential when the install's own rules speak to it above the checkpoint bar. A fresh install with no rules never stops anything. FOUR CONDITIONS, EACH PREVENTING A DIFFERENT WRONG. Above the bar; a rule and never a preference (which claims no such force); the band's top hit only (the ranker's confidence claim attaches to its first element); and only a rule the session has NOT opened — `held` is observable from the get_rule PostToolUse hook (#4100), not self-report. WHY "NOT OPENED" RATHER THAN "NO OUTCOME RECORDED". An outcome can be satisfied with one cheap call asserting compliance without producing any, and a checkpoint dismissible that way manufactures exactly the compliance data step 2 was built to measure. Reading a rule cannot be faked in that direction: after `get_rule` the statement is in context, which is the whole of what was wanted. THE BAR IS MEASURED. `retrieval_telemetry(days=30)`: write_path_rule p90 0.7628 max 0.8817; pre_tool_rule p90 0.7373 max 0.8293. 0.80 is above p90 on both and below max on both, so it selects from the top decile of an already selective arm and is still reachable. It ships as a setting with a Settings card, because a cosine distance in one model's geometry over one corpus cannot transfer. TWO GUARDS ON THE WORST CASE: at most one hold per rule and five per session, so a mis-set floor degrades to a noisy session rather than one that cannot proceed. The ledger lives in the swept directory and is named `.ids`, so the existing compaction-clear guards cover it. WRITES ARE NOT HELD, AND THAT IS THE OPERATOR'S DECISION RATHER THAN MINE. `scribe_prior_art.sh` carries a tested property that it never returns a permissionDecision — a recall aid may not stand in the way of a write. Three of the milestone's seven misses were file edits and none are reachable from the command side, so there is a live argument for extending this; that argument is exactly why the boundary is now asserted by a test rather than left to memory. The write-path arm computes and returns the same block so the decision can be revisited with evidence; the hook ignores it, and a change of mind is a hook edit rather than a feature. Verified by lifting `checkpoint_for` and `_rule_band` out of source with `ast` and exercising the shipped functions over 17 populations, by running the ledger and deny envelope in bash (10 cases, including that a refused hold is not written and that a garbled rule id fails closed), and by scripts/check_plugin.py — which caught the unminted plugin version, 0300 -> 0426. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 00:26:54 -04:00
bvandeusenandClaude Opus 5	bb8013928f	feat(telemetry): the readout names rules that were opened and changed nothing (#4213) ... Milestone 419 step 2. Step 1 made an outcome recordable; this makes it readable. `retrieval_summary`'s rule block gains `applied`, `departed` and `distinct_rules_acted`, and `_compute_warnings` gains two codes. TWO CODES, NOT ONE WITH A ZERO IN IT. `read_and_unacted` reports rules that were opened and left no outcome, against the ones that did. It only fires once outcomes exist anywhere in the window, because a window with none cannot tell "every rule was ignored" from "nothing calls `rule_outcome` yet" — and on every install the day this ships, the truth is the second. Claiming the first there would be #3311's failure exactly: a statistic that could not vary being read as a fact about the corpus. The cold case gets its own code, `outcomes_never_recorded`, whose prose says in as many words that it does NOT mean the rules were ignored. `applied` AND `departed` ARE NOT SUMMED. A departure carries the reason the agent gave and is evidence about the RULE; an application is evidence about the agent. Folded together they would say only "an outcome exists", which is true of both and useful about neither. `distinct_rules_acted` counts either, because for the unacted arithmetic the distinction does not matter. An outcome is not a pull. The fold branches on OUTCOMES first and never routes an outcome through the surfaced/ambient split: `source` on an outcome row names the door the outcome came through, not a ranker, so the ambient distinction has nothing to say about it. An integration test holds that line — if an outcome leaked into the pull counters the silently-unchanged rule would vanish into a compliant-looking total, which is the confusion #4212 was opened to end. Verified by lifting the shipped `_compute_warnings` out of source with `ast` and exercising it against the six populations the new tests assert: cold instrument, warm instrument, departures-only, full compliance, nothing opened, and a failed read. The integration tests for the new counts run against real Postgres in CI — count(distinct) with an IN over an unconstrained column is a SQL shape a mock would agree with whatever it did, which is what #2663 was. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 00:13:45 -04:00
bvandeusenandClaude Opus 5	4ebf478575	fix(rules): the registration guard counts 24 now that rule_outcome exists ... Milestone 419's own subject, committed while building it. The module header of mcp/tools/rulebooks.py says the tool count "lives in the registration test, which fails when it drifts" — I read that line while adding the tool and did not act on it, which is a rule surfaced, read, and silently unchanged. The only reason it was caught is that the guard exists and CI ran it; nothing about my process would have found it. Worth stating plainly because the milestone is about exactly this class of miss, and step 1 shipped the record that would let a future session SEE it: a rule read and not followed is invisible unless something independent notices. Here the something was a test written by whoever last changed this count. Integration was already green on the previous run, so the migration, the backup round trip and the new `detail` column were never in question — this was one integer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-20 23:58:27 -04:00
bvandeusenandClaude Opus 5	dfcb000719	feat(rules): a surfaced rule gets an outcome, not just a read (#4212) ... Milestone 419 step 1. `rule_usage_events` could say a rule was SURFACED and that it was PULLED. It could not say what happened next, so a rule that fires constantly and is always obeyed and a rule that fires constantly and is never obeyed left byte-identical telemetry. The second is far the more urgent and was the one the readout could not name — measured on a session where three of seven misses were caught by the operator and none by the system. Two new events, `applied` and `departed`, and a `detail` column carrying the why of a departure. No CHECK migration: `event` was created in 0094 as plain Text with no constraint, verified in the migration rather than assumed from the model, so rule 36 does not bite here — said in both places because the next person adding a value will reach for it. THE THIRD STATE IS DERIVED, AND THAT IS THE DESIGN. Read-and-silently- unchanged is the failure this milestone was opened on, and it cannot be reported: an agent that knew it was ignoring a rule would not be ignoring it. So nothing here asks. `applied` and `departed` are reported; the third state is a rule that was opened and left no trace. An `ignored` enum member would collect nothing while reading as though it had measured something, which is #3311's failure — a statistic that could not vary being taken for a finding. `detail` is a column rather than two more bare event strings because a departure stripped of its reason reads back as a miss, so the two states this exists to separate would collapse again one layer down, in the readout, where nobody would see it happen. Nullable: following a rule needs no argument, and an expensive event is one that stops being recorded. `outcome_state` is the single reading of the four states, taking the aggregate `usage_for_rules` already returns, so the badge, the readout and any later session summary cannot disagree about what "followed" means — the drift #3246 found across the rules system. A departure outranks an application: a rule both applied and argued with is a rule someone argued with, and the argument is the half worth surfacing. `rule_outcome` is the MCP door, classed as a WRITE. The read-only set tolerates getters that call record_pulled, but those are reads that leave a trace; this tool's entire effect is the row, and the row carries prose the agent authored. A read-scoped key that can put text in the operator's database is not read-scoped, whatever table it lands in. Backup carries `detail` on both sides. It is the one field here a fresh install cannot re-earn — counts come back by being used again, a stated reason exists once — and #4197 records that the column guard watches the export side only, so the round-trip test is the thing that would catch a one-sided add. Delivery is deliberately not settled here: how an agent gets prompted to record an outcome is step 3's subject, and the same record serves whichever answer that step reaches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-20 23:56:05 -04:00
bvandeusenandClaude Opus 5	0fe19a8440	fix(ledger): a canon may hold a class and the to_dict beside it (#4220) ... The review surface shipped yesterday reported two canons on its first live day and both were sound. Coherence was "do all judged rows share a form", which #2844 failed at 37/62 = 0.597 for containing a model class and the to_dict the canon's own text says the class must carry, and #2849 failed at 4/7 for pairing sync loop-starters with the async ticks they schedule. A review surface whose whole output is noise is one that stops being read. The obvious repair is a trap, and there is now a test standing in front of it. Grouping by family and keeping a majority test makes the check BLIND: before #2844 was cleaned by hand it held 37 classes and 56 callables, which as families is 56/93 = 0.602 — a clean pass, and the 31 rows that had no business being there (Vue functions, route handlers, a dozen tests) would never have been reported at all. A looser bar in the same shape is worse than the bug. So the verdict is inverted. Instead of asking whether most rows agree, it asks how many rows the canon CANNOT ACCOUNT FOR: a row in the majority family is accounted for; a callable defined in a file that also holds a majority-family `type` row is a method of a member, not a foreign body; and strangers above a fifth of the readable rows make the canon incoherent. The majority vote abstains those methods, so a class's own serialisers cannot outvote the classes and turn the members into the strangers. Measured on the real ledger before it was written, which is why it is this rule and not a nudge to the share: clean #2844 has 0 strangers in 62, #2849 has 0 in 7, and polluted #2844 had 31 in 93 — the same 31 withdrawn by hand this morning, named exactly. The entry now carries `families`, the majority `family`, `attached`, `stranger_count`, `unattended`, and `strangers` — THE ROWS THAT DO NOT FIT, replacing a sample of the first twelve members. The reader's question is which rows are wrong, and a sample of the agreeing majority cannot answer it. `unattended` is the discriminator between a check that is too strict and a ledger full of junk: both canons flagged on day one were entirely audit-judged, and nothing showed that without opening each one. Scoped to the review surface. `canon_form` still answers at the precise form level for stamping and divergence, where a sync helper beside an async canon is a fair question; nothing here changes what the ledger writes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-20 23:39:01 -04:00