feat(plugin): a high-confidence rule is put in front of a command, not beside its result (#4214)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / integration (push) Successful in 59s
CI & Build / Python tests (push) Failing after 1m13s
CI & Build / Build & push image (push) Skipped
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / integration (push) Successful in 59s
CI & Build / Python tests (push) Failing after 1m13s
CI & Build / Build & push image (push) Skipped
Milestone 419 step 3, the pre-act checkpoint. Every rule surface in this plugin returns `additionalContext`, which Claude Code delivers alongside the tool RESULT — so the rule is read after the call is written and lands as commentary on a decision already made. That is the milestone's central finding, measured over a session with seven misses, three caught by the operator and none by this system. The action arm can now return a `deny` instead. The act does not run, the rule's text can be read before the call exists, and the remedy is one `get_rule` call after which the act may be re-submitted unchanged. Nothing reaches the operator: a deny is a message to the model. "CONSEQUENTIAL" IS DERIVED, NOT ENUMERATED. The obvious implementation lists act kinds — a write to product code, a schema change, a bulk classification, a merge. Every one of those is consequential because THIS operator wrote rules about it, and shipping that list is this instance's corpus hard-coded into the product (rule 115). So the corpus decides: an act is consequential when the install's own rules speak to it above the checkpoint bar. A fresh install with no rules never stops anything. FOUR CONDITIONS, EACH PREVENTING A DIFFERENT WRONG. Above the bar; a rule and never a preference (which claims no such force); the band's top hit only (the ranker's confidence claim attaches to its first element); and only a rule the session has NOT opened — `held` is observable from the get_rule PostToolUse hook (#4100), not self-report. WHY "NOT OPENED" RATHER THAN "NO OUTCOME RECORDED". An outcome can be satisfied with one cheap call asserting compliance without producing any, and a checkpoint dismissible that way manufactures exactly the compliance data step 2 was built to measure. Reading a rule cannot be faked in that direction: after `get_rule` the statement is in context, which is the whole of what was wanted. THE BAR IS MEASURED. `retrieval_telemetry(days=30)`: write_path_rule p90 0.7628 max 0.8817; pre_tool_rule p90 0.7373 max 0.8293. 0.80 is above p90 on both and below max on both, so it selects from the top decile of an already selective arm and is still reachable. It ships as a setting with a Settings card, because a cosine distance in one model's geometry over one corpus cannot transfer. TWO GUARDS ON THE WORST CASE: at most one hold per rule and five per session, so a mis-set floor degrades to a noisy session rather than one that cannot proceed. The ledger lives in the swept directory and is named `.ids`, so the existing compaction-clear guards cover it. WRITES ARE NOT HELD, AND THAT IS THE OPERATOR'S DECISION RATHER THAN MINE. `scribe_prior_art.sh` carries a tested property that it never returns a permissionDecision — a recall aid may not stand in the way of a write. Three of the milestone's seven misses were file edits and none are reachable from the command side, so there is a live argument for extending this; that argument is exactly why the boundary is now asserted by a test rather than left to memory. The write-path arm computes and returns the same block so the decision can be revisited with evidence; the hook ignores it, and a change of mind is a hook edit rather than a feature. Verified by lifting `checkpoint_for` and `_rule_band` out of source with `ast` and exercising the shipped functions over 17 populations, by running the ledger and deny envelope in bash (10 cases, including that a refused hold is not written and that a garbled rule id fails closed), and by scripts/check_plugin.py — which caught the unminted plugin version, 0300 -> 0426. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
This commit is contained in:
@@ -558,6 +558,69 @@ scribe_rules_append() {
|
||||
#
|
||||
# Same reader as the naming ledger on purpose, so ageing, the last-entry-wins
|
||||
# rule and the bare-id format are defined once and cannot drift apart.
|
||||
# ── The pre-act checkpoint's session ledger (#4214, milestone 419) ─────────
|
||||
#
|
||||
# A checkpoint STOPS an act rather than annotating it, so unlike every other
|
||||
# ledger here its job is to make sure the same stop cannot happen twice. Two
|
||||
# guards, and they fail in different directions:
|
||||
#
|
||||
# per rule — a rule may hold at most ONE act per session. Once it has, the
|
||||
# remedy has been offered; repeating it on the next call would
|
||||
# turn a reader who decided the rule does not apply into a
|
||||
# reader who cannot proceed.
|
||||
# per session — at most `_SCRIBE_CHECKPOINT_CAP` stops in total, whatever the
|
||||
# corpus scores. A mis-set floor or a corpus that suddenly
|
||||
# resembles everything must degrade to a noisy session, never
|
||||
# to one that cannot make progress. This is the guard on the
|
||||
# worst case, not a tuning value.
|
||||
#
|
||||
# NOT AGED, unlike the naming ledgers. Those age because they describe what a
|
||||
# session is still HOLDING, and a context stops holding things. This one
|
||||
# describes what already HAPPENED — a stop was raised and its remedy offered —
|
||||
# and that does not become untrue an hour later. Ageing it would let a long
|
||||
# session be stopped by the same rule repeatedly, which is the one outcome
|
||||
# both guards exist to prevent.
|
||||
_SCRIBE_CHECKPOINT_CAP=5
|
||||
|
||||
scribe_checkpoint_allowed() {
|
||||
# $1 ledger file, $2 rule id. Returns 0 (and RECORDS the stop) when this act
|
||||
# may be held; non-zero otherwise. Records on the way out rather than asking
|
||||
# the caller to, because a caller that forgets is a session that can be
|
||||
# stopped forever and the failure is invisible until it happens.
|
||||
local f="$1" id="$2" n
|
||||
[ -n "$f" ] || return 1
|
||||
id=$(printf '%s' "$id" | tr -cd '0-9')
|
||||
[ -n "$id" ] || return 1
|
||||
if [ -f "$f" ]; then
|
||||
# Already held an act for this rule — the remedy has been offered once.
|
||||
grep -qx "$id" "$f" 2>/dev/null && return 1
|
||||
n=$(grep -c '^[0-9][0-9]*$' "$f" 2>/dev/null || printf '0')
|
||||
# `grep -c` over a missing file can print nothing; a bare arithmetic test
|
||||
# on an empty string is a syntax error in some shells and silently true in
|
||||
# others, which is how a cap comes to cap nothing (#3191's shape).
|
||||
n=$(printf '%s' "$n" | tr -cd '0-9')
|
||||
[ -n "$n" ] || n=0
|
||||
[ "$n" -ge "$_SCRIBE_CHECKPOINT_CAP" ] && return 1
|
||||
fi
|
||||
printf '%s\n' "$id" >> "$f" 2>/dev/null || return 1
|
||||
return 0
|
||||
}
|
||||
|
||||
scribe_json_deny() {
|
||||
# $1 hook event name, $2 the reason the agent reads INSTEAD of the result.
|
||||
#
|
||||
# The one place this plugin emits a decision on a tool call. `deny` returns
|
||||
# the reason to the MODEL and the call does not run — it is not a prompt to
|
||||
# the operator, costs them nothing, and is undone by the model simply
|
||||
# submitting the call again. That is the whole difference from `ask`, which
|
||||
# would hand a judgement that is the agent's to make to the person who asked
|
||||
# for the work.
|
||||
local esc
|
||||
esc=$(printf '%s' "$2" | scribe_json_escape) || return 0
|
||||
printf '{"hookSpecificOutput":{"hookEventName":"%s","permissionDecision":"deny","permissionDecisionReason":"%s"}}\n' \
|
||||
"$1" "$esc"
|
||||
}
|
||||
|
||||
scribe_held_query() {
|
||||
local ids
|
||||
ids=$(scribe_rules_live "$1")
|
||||
|
||||
Reference in New Issue
Block a user