A rule must say when it applies; the ledger says what was read, not what was shown #162

Merged

bvandeusen merged 6 commits from dev into main 2026-09-16 17:50:08 -04:00

Conversation 0 Commits 6 Files Changed 29

+766 -48

6 Commits

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	957a72c501	fix(tests): the compact band pins three holding states, not two (#4100) ... CI run 6945: 11 failures down to 1. The survivor is test_shortening_a_line_does_not_decide_what_it_says_about_holding, which asserted "no longer hold it" appears in the compact line for seen=True — the phrase that now belongs to the OPENED state, not the named one. Its subject is a property, not a string: `compact` and the ledger are independent axes, and shortening a line must not change what it claims about holding. So the fix follows the axis rather than swapping the phrase. The axis grew from two states to three, and the test now checks all three are distinct under compact — pinning only two would let the compact branch collapse the new middle state into either neighbour, which is the same regression it was written for with one more place to hide. Added the assertion that matters most when room is short: a line the session never opened must not imply it did. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-16 17:35:08 -04:00
bvandeusenandClaude Opus 5	ebd6cb203c	fix(tests): the repeat tail is the middle state now, not the opened one (#4100) ... CI run 6943: 11 failures, two causes, both existing guards correctly catching the behaviour change rather than defects in it. TEN were `_SEEN_TAIL = "You saw it earlier this session"`. Every one of those tests drives an arm with `exclude_rule_ids` alone — the NAMING ledger — which since this change is the middle state, not the opened one. That is the whole point of the step: `seen` stopped meaning "you saw it". The constant now holds the middle tail, so each test keeps asserting exactly what it was written to assert (a repeat gets a line distinct from a first surfacing) against the wording that is now true. Added `_HELD_TAIL` and a negative assertion with it: neither call names an opened ledger, so neither may claim the session read anything. That catches `held` defaulting true, which would have every repeat assert the strongest of the three claims on no evidence — and none of the ten existing tests would have noticed. THE ELEVENTH was my own test edit being wrong, not the code. I asserted `held_rule_ids` would appear in `re.findall(r"printf '([a-z_]+)=", defs)`, but `scribe_held_query` spells it `printf '&held_rule_ids=`, and the leading `&` means that regex never matches it. The regex enumerates SCOPE keys — the alternatives that open a query — and the ledger key is appended to a query that already has one. Two different contracts that happened to share a file, so they are now asserted separately and the difference is written down. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-16 17:19:19 -04:00
bvandeusenandClaude Opus 5	ad26b3f458	feat(retrieval): the ledger records what was OPENED, not merely what was shown (#4100) ... Milestone 386 made a repeat REFERENCED rather than withheld, and the line it chose says "You saw it earlier this session". Nothing ever checked that. The arms emit a TEASER — title, trigger, get_rule(N) — so a session can be shown a rule twenty times and never read a word of it, and a compaction summarises the teaser away leaving nothing behind. The server was asserting something about the reader's context it had no way to know. Three states now, where there were two: never surfaced "it is not in this session's loaded set" named, unopened "Mentioned earlier this session but not opened — read it…" opened "You opened it earlier this session; pull it… again" The middle one is the honest one and the one that was missing. It keeps the full invitation, because a session that skipped a teaser is in nearly the position of one never shown it. HOW "OPENED" BECOMES OBSERVABLE. A new PostToolUse hook watches the get_rule call itself and appends to `<sid>.opened.ids`. PostToolUse does fire for MCP tools — the event's own output schema carries `updatedMCPToolOutput`, which would be meaningless otherwise — and the matcher is `mcp__.*__get_rule` so the server segment, which varies by install, is not pinned. This is NOT the self-report 386 rejected. That objection was to ASKING a model whether it holds a rule, which is unverifiable. A tool call is an event the harness reports whether anyone asks. Recording what a session DID and believing what it SAYS about itself are different kinds of evidence. Both ledgers clear together on compact/clear. Keeping `.opened.ids` across a compaction would have the arms telling a freshly-summarised session "you opened it earlier" about a rule now nowhere in its context — a more confident version of the bug being removed. Same reader (scribe_rules_live) for both, so ageing, last-entry-wins and the bare-id format are defined once. Also closes two smoke-coverage holes the checker was reporting as SKIP: the new recorder, and scribe_precompact_preserve.sh from #3680. The latter needed STATIC_FLOOR to become a set — PreCompact's contract is inverted, its stdout BECOMES the summarizer's instructions, so silence is its failure mode and a generic read of it looks like a leak. Step 2 of milestone 416, and a hard prerequisite for step 4: while suppression keys on shown, widening k marks records "seen" faster than they are read, and the ledger would degrade in proportion to the improvement. Plugin minted 2026.09.16.1232 -> 2026.09.16.2102. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-16 17:03:39 -04:00
bvandeusenandClaude Opus 5	bb6ab0f2a8	fix(tests): the trigger backfill assumed keyword calls; three fixtures pass positionally (#4099) ... CI run 6937 red — 3 collection errors, `positional argument follows keyword argument`, in the three integration fixtures that call the service positionally (`create_rule(topic.id, uid, "title", "statement")`). The script that added `when_to_apply=` to 15 fixtures inserted it as the FIRST argument, which is valid only where every other argument is already a keyword. Moved to the last argument in every call, which is legal in both styles, and the continuation indent now matches the surrounding arguments. Also repairs self-inflicted damage: the same script added a trigger to the two tests in test_rule_trigger_required.py whose whole purpose is to call the creators WITHOUT one. They would have stopped raising and the guard's own proof would have inverted — a test that passes for the opposite reason than the one it names, which is worse than a failing one. Caught by ast.parse across tests/ rather than by the next CI round trip. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-16 14:49:05 -04:00
bvandeusenandClaude Opus 5	07bdbf1647	feat(rules)!: a rule cannot be created, or edited into, having no trigger (#4099) ... `when_to_apply` is not metadata. `rule_document` embeds a rule as `{title} — {trigger}` / `When to apply: {trigger}\n\n{statement}`, the trigger appearing twice so purpose dominates a short vector — the shape note 2485 measured on snippets (a 0.153 top-to-second gap against 0.010–0.023 for everything else). Without one the document silently becomes title + statement: a DIFFERENT shape, ranked against a corpus it does not match, with nothing to report it. Every bar and every rank in the system assumes one shape. `create_preference` has refused an empty trigger since it shipped. The two rule creators defaulted it to "" — so the shape was enforced for the record kind that guides and optional for the kind that binds. The guard lives in the SERVICE, because both doors reach it: the MCP tools and the frontend's fast path in routes/rulebooks.py. Written in either alone, the other could still create a rule that never fires. The route keeps a matching check for the STATUS CODE only (400, not the 404 it maps ValueError to). update_rule refuses to EMPTY an existing trigger, checked after the mutation so it covers `clear=[...]`, an emptied form input, and any route added later. Deliberately asked as "did this edit remove one" rather than "does one exist": a rule predating the guard has none, and refusing to save it would freeze precisely the unreachable records that most need fixing. Deliberately not following arose_from_id, which the human door exempts itself from because provenance is about auditing what the AGENT changed. That reasoning does not reach this field — a missing trigger is not a missing explanation, it is a rule that does not work, and it fails an operator as badly as a session. 15 test fixtures across 6 files were creating rules with no trigger. They now pass one; that they did not is the point — curation is not a guarantee. Step 1 of milestone 416 "Retrieval stops guessing a bar". First because every later step assumes one document shape, and it is much cheaper to guarantee before a corpus grows than to backfill after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-16 14:45:32 -04:00
bvandeusenandClaude Opus 5	5ae60734bb	fix(retrieval): the completion-report arm gets its own bar, not the prompt arm's (#3860) ... `report_preference` shipped reading PROMPTRULE_THRESHOLD_KEY, so the two arms were one dial: tuning the bar for an operator's prose silently retuned the lookup that runs when a task closes. That coupling is worse on this arm than it would be anywhere else. Every other retrieval arm scores a query that varies per call, so a mis-set bar shows up as a changed clear-rate. COMPLETION_QUERY is a fixed string, so this arm's best score for a given corpus is a CONSTANT — and a constant sitting under the bar is a dead arm rather than a quiet one. No volume of traffic reveals it. Found by the first live read for milestone 394 step 9: 69 calls, 69 declines, every one naming the same record at the same score (0.7194 against a 0.72 bar). Reading `best_available_id` (#3807) showed the record was about interpreting a REQUEST, not about report shape — so the declines were correct and the arm is healthy. The percentile alone would have said "lower the bar", which would have delivered a false positive on every completion report ever written. The bar does not move; the key does. Both defaults stay 0.72, so this changes no behaviour on any install — it makes "leave this one where it is" expressible, which it was not before. Settings grows the control (rules 25, 27), and the default-agreement check grows a row. Deliberately NOT included: a change to PROMPTRULE_DEFAULT_THRESHOLD. The evidence for moving it is this install's near-miss table, and rule 115 keeps a shipped default from being justified by one instance's corpus. That bar is a per-user setting and belongs in the operator's Settings, not in the product. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-16 13:47:34 -04:00