fix(plugin): the ledger that proves a rule was read was being deleted by the compaction that asked about it (#4217)

Milestone 419's acceptance, and it failed the first time it was run — which
is the only reason this commit exists.

THE MEASUREMENT. Ran the step-5 readout against this session's real traffic
instead of a fixture. It reported 19 rules named by an arm and none opened.
That is false: the session had called `get_rule` 45 times. Across six real
sessions on this instance: 208 opens, 3 surviving ledger entries. 1.4%.

THE CAUSE. `.opened.ids` was doing two jobs with opposite lifetimes.

  - "this context HOLDS rule 156" — false after a compaction, and three hooks
    read it to decide whether to stay quiet. Clearing it is correct.
  - "rule 156 WAS OPENED" — which no compaction makes untrue, and which the
    session-end readout is built on.

`scribe_clear_session_ledgers` sweeps every `<sid>*.ids` on SessionStart
source=compact. Right for the first claim, and it was deleting the second.
The TTL did the same thing more quietly: `scribe_rules_live` ages an
exclusion ledger, which is right, and would have eaten the early part of any
long session's evidence too.

So the readout was reporting only the stretch since the last compaction while
reading as though it had reported the session — a statistic that cannot vary
being mistaken for a finding (#3311), which is the shape this whole milestone
exists to stop producing. It ran AT the seam it was blind to.

THE SPLIT. `scribe_rules_append` now writes both: the exclusion ledger it
always wrote, and `<kind>.keep.ids`, an evidence twin that is never aged and
never swept. The readout reads twins; the three `held` readers are untouched.

DERIVED, NOT LISTED, because a list is what broke this before — the comment
above the sweep says so about its own history. Every ledger written through
the appender gets a twin, including the next one somebody adds; a new ledger
is born on the swept side unless its name opts out. `scribe_checkpoint_allowed`
writes its twin explicitly since it bypasses the appender, and there the split
lands right on both sides: the cap counts the swept file, so a compaction
honestly restores the budget to stop an act the context can no longer justify,
while the record that a stop happened stays.

Removed `scribe_ledger_ids`, orphaned by the change — a dead helper beside a
live one is a thing the next reader trusts.

test_session_ledger_clear.py asserted every ledger dies and could not have
caught this: its fixture never created a twin, so the sweep was one glob away
from either mistake with only one of them guarded. Both sides now asserted.
The slippage tests build their ledgers through the real writers rather than
by hand, for the same reason — a fixture that writes the bytes itself keeps
passing after the writer stops.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy

plugin/hooks/scribe_defs.sh

@@ -525,37 +525,6 @@ scribe_contract_block() {
   return 0
 }
 
-# ── The session slippage readout (#4216, milestone 419) ───────────────────
-#
-# WHAT IT ANSWERS, AND WHY IT CANNOT BE ASKED OF THE SERVER. Which rules fired
-# this session, which changed an action, which did not. `rule_usage_events`
-# carries no session column — it is per user over a window — so a
-# session-scoped answer has to be assembled where a session is a thing that
-# exists. That is here, from ledgers four hooks already write:
-#
-#   .rules.ids      an arm NAMED the rule           (a teaser was shown)
-#   .opened.ids     the session called get_rule     (#4100, an observed event)
-#   .acted.ids      the session called rule_outcome (#4216)
-#   .checkpoint.ids the rule HELD an act            (#4214, the strongest)
-#
-# EVERY LINE IS AN OBSERVED TOOL CALL. Nothing here asks the model what it
-# followed — milestone 386 ruled that out, and rightly: a model asked "did you
-# apply rule 156?" will say yes. These four files record what HAPPENED.
-#
-# THE SUBTRACTIONS ARE THE POINT. Named-minus-opened is the arm talking to
-# nobody; opened-minus-acted is the milestone's whole subject, a rule read and
-# then indistinguishable from one that worked. Neither is an accusation — a
-# rule may be read and correctly judged not to apply — which is why the lines
-# below ask for the leftovers to be CARRIED, not explained.
-scribe_ledger_ids() {
-  # Live ids from one ledger as space-separated words, for set arithmetic.
-  # Aged through scribe_rules_live so a rule named two hours ago does not read
-  # as something this context still holds; a bare-id ledger (the checkpoint
-  # one) simply has no stamps and survives the ageing unchanged.
-  local f="$1"
-  [ -n "$f" ] && [ -f "$f" ] || return 0
-  scribe_rules_live "$f" | tr ',' ' '
-}
 
 scribe_slippage_lines() {
   # $1 state dir, $2 sanitised session id.
@@ -573,10 +542,16 @@ scribe_slippage_lines() {
   # skip it.
   local dir="$1" sid="$2" named opened acted held
   [ -n "$dir" ] && [ -n "$sid" ] || return 0
-  named=$(scribe_ledger_ids "$dir/${sid}.rules.ids")
-  opened=$(scribe_ledger_ids "$dir/${sid}.opened.ids")
-  acted=$(scribe_ledger_ids "$dir/${sid}.acted.ids")
-  held=$(scribe_ledger_ids "$dir/${sid}.checkpoint.ids")
+  # THE TWINS, not the exclusion ledgers. This runs at the compaction, which
+  # is the moment the exclusion ledgers are about to be cleared and the moment
+  # their TTL has usually already eaten the early part of a long session. A
+  # readout built on them would report only the last stretch of the session
+  # and read as though it had reported all of it — the #3311 shape, and the
+  # one this milestone exists to stop producing.
+  named=$(scribe_ledger_kept "$dir/${sid}.rules.keep.ids")
+  opened=$(scribe_ledger_kept "$dir/${sid}.opened.keep.ids")
+  acted=$(scribe_ledger_kept "$dir/${sid}.acted.keep.ids")
+  held=$(scribe_ledger_kept "$dir/${sid}.checkpoint.keep.ids")
   [ -n "$named$opened" ] || return 0
 
   local unread unresolved
@@ -790,10 +765,42 @@ scribe_rules_live() {
 # Append surfaced ids, stamped. Reads ids on stdin, one per line — the shape
 # `scribe_json_list "$flat" '.rule_ids'` already produces at both call sites.
 scribe_rules_append() {
+  # Writes TWO files, and the second one is the point (#4217).
+  #
+  # `$f` is an EXCLUSION ledger: it answers "does this context already hold
+  # this?", so it is aged by TTL and swept at every compaction — after a
+  # compaction the agent genuinely does not hold what it was shown, and
+  # forgetting is correct. Three hooks depend on exactly that.
+  #
+  # The twin is an EVIDENCE ledger: it answers "did this happen?", which no
+  # compaction can make untrue. It is never aged and never swept.
+  #
+  # These are opposite lifetimes and `.opened.ids` was serving both, which is
+  # how a session with 45 `get_rule` calls came to report none: the compaction
+  # cleared the ledger that was also the record. Measured across six sessions
+  # on this instance — 208 opens, 3 surviving — before the split.
+  #
+  # DERIVED HERE rather than listed at the call sites, because a list is what
+  # broke this before (see `scribe_clear_session_ledgers`). Every ledger
+  # written through this function gets its twin, including the next one
+  # somebody adds. The cost is a second small file per ledger per session.
   local f="$1" now
   [ -n "$f" ] || return 0
   now=$(date +%s 2>/dev/null) || now=0
-  awk -v ts="$now" 'NF { print $1 "\t" ts }' >> "$f" 2>/dev/null || true
+  awk -v ts="$now" -v keep="${f%.ids}.keep.ids" '
+    NF { line = $1 "\t" ts; print line; print line >> keep }
+  ' >> "$f" 2>/dev/null || true
+}
+
+scribe_ledger_kept() {
+  # Ids from an EVIDENCE twin: deduped, order preserved, NOT aged. A rule
+  # opened two hours ago was still opened, so the TTL that keeps an exclusion
+  # ledger honest would here delete the finding.
+  local f="$1"
+  [ -n "$f" ] && [ -f "$f" ] || return 0
+  awk -F'\t' '$1 != "" && !seen[$1]++ {
+    out = out (out == "" ? "" : " ") $1
+  } END { if (out != "") print out }' "$f" 2>/dev/null || true
 }
 
 # The OPENED ledger's contribution to a rule arm's query string (#4100).
@@ -853,6 +860,11 @@ scribe_checkpoint_allowed() {
     [ "$n" -ge "$_SCRIBE_CHECKPOINT_CAP" ] && return 1
   fi
   printf '%s\n' "$id" >> "$f" 2>/dev/null || return 1
+  # The evidence twin, beside the cap rather than instead of it. `$f` is swept
+  # at a compaction and SHOULD be: after one, this context has not read the
+  # rule, so the budget to stop an act on it is honestly fresh. That a stop
+  # already happened is a different claim, and it stays true.
+  printf '%s\n' "$id" >> "${f%.ids}.keep.ids" 2>/dev/null || true
   return 0
 }
 
@@ -914,10 +926,19 @@ scribe_held_query() {
 SCRIBE_LEDGER_DIRS="scribe-priorart scribe-autoinject"
 
 scribe_clear_session_ledgers() {
-  local sid="$1" dir
+  # SPARES `*.keep.ids`, which are evidence rather than exclusions — see
+  # `scribe_rules_append`. Everything else still goes: the convention is
+  # unchanged and still covers a ledger added tomorrow, which is what the
+  # block above insists on. The twin opts OUT by its name, so a new ledger is
+  # born on the swept side unless somebody says otherwise.
+  local sid="$1" dir f
   [ -n "$sid" ] || return 0
   for dir in $SCRIBE_LEDGER_DIRS; do
-    rm -f "${TMPDIR:-/tmp}/$dir/$sid"*.ids 2>/dev/null || true
+    for f in "${TMPDIR:-/tmp}/$dir/$sid"*.ids; do
+      [ -f "$f" ] || continue
+      case "$f" in *.keep.ids) continue ;; esac
+      rm -f "$f" 2>/dev/null || true
+    done
   done
   return 0
 }