The hooks need no jq, and the prompt boundary retrieves against prompts #171

Merged

bvandeusen merged 4 commits from dev into main 2026-09-20 18:50:03 -04:00

Conversation 0 Commits 4 Files Changed 25

+1388 -235

4 Commits

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	1f7ff7b215	fix(plugin): the prompt boundary retrieves against prompts, not plumbing (#4200) ... Claude Code submits more than typed words through UserPromptSubmit. A task notification, a slash-command echo and the caveat banner a local command prints all arrive as user turns, reaching `.prompt` indistinguishable from something the operator wrote. scribe_autoinject.sh believed all of them. The cost that matters is not the wasted embedding — it is the log row. Every such call counts in the denominator of every prompt-boundary surface, so delivery rate reads low for a reason unrelated to retrieval; and each refusal lands in `near_misses`, where a later tuning decision reads it as demand. Measured while taking milestone 399's acceptance (#3898): 15 of the top 20 `preference_slot` near-misses were `<task-notification>` blocks, all matching ONE record — #140 "Let each action land before starting the next" — all within thousandths of the 0.70 floor. A notification that an action finished really does resemble a preference about letting actions land. Lowering the floor to serve that apparent demand would have injected that record into every notification: the instrument arguing for the wrong fix, which is #379 again. scribe_skip_prompt is a PREFIX test, not a substring one, and that is the whole safety argument. A real prompt may contain one of these tags — an operator pasting a transcript, or a `<system-reminder>` after typed words — and must still be retrieved against. Nothing an operator types begins with a client envelope. The compaction-resume injection is deliberately NOT filtered: it is machine-written, but it summarises real work, and a resumed session is where recalling a rule earns its keep. Client-side only, so no server contract moves and lagging plugin caches keep working. The tags are Claude Code protocol constructs, identical on every install — instance-agnostic under rule 115. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-20 18:45:13 -04:00
bvandeusenandClaude Opus 5	7e653e16dc	docs(ci): jq is not a CI requirement and should not be promoted (#4107) ... ci-requirements.md is the document that drives promoting a per-job dep into the ci-python image, and it still recorded jq as installed in the plugin job and "load-bearing for the smoke test specifically", on the grounds that every hook opened `command -v jq || exit 0` and the test would otherwise pass while exercising nothing. That was the tail wagging the dog. The hooks ship to users; jq is absent by default on macOS, the Debian/Ubuntu slim images, Alpine and most CI containers, and a machine without it got no context, no rules, no prior art and no process sync in silence. The answer was to remove the dependency, not to install it harder. Recorded as a promotion the entry now argues AGAINST rather than deleted: the next person to read this file should find out why jq is absent, not merely that it is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-20 12:24:37 -04:00
bvandeusenandClaude Opus 5	4bfa0cbbc0	fix(plugin): stop CI installing the jq the hooks no longer use (#4107) ... Three loose ends from a49e7ed, all found by CI or by re-reading it. TWO TESTS PINNED THE jq SYNTAX, not the word `jq`, so grepping the suite for the tool name missed them: test_write_path_trigger asserted the literal filter strings `(.note_ids // []) - (.sync_note_ids // [])` and `(.rule_ids // [])[]?` appear in the hook. Re-spelled with the readers that replaced them. The claim each makes is unchanged — that the reuse and sync channels keep their own state file, their own query parameter and their own write-back. CI STILL INSTALLED jq, TWICE. The "Install jq for hook execution tests" step existed because those tests skipped rather than failed without it; the Plugin hooks job installed it alongside shellcheck for the same reason. Removing a dependency from the product and leaving CI to install it anyway means the smoke tests never run under the condition they exist to assert. Both gone — the hook tests now run on a bare image. `awk 'END { print $0 }'` IS UNDEFINED IN POSIX. gawk retains the last record in END and mawk does too, but it is not guaranteed, and reaching for a non-portable idiom inside the change whose entire purpose is portability is the same mistake one layer down. Now `{ last = $0 } END { if (NR) print last }`. Verified it still selects the same start line on a real transcript. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-20 12:23:42 -04:00
bvandeusenandClaude Opus 5	a49e7ed2af	fix(plugin): the hooks need no jq and no tac (#4107) ... Every hook opened `command -v jq >/dev/null 2>&1 || exit 0`, so on a machine without jq the operator got no session context, no rules, no prior art and no process sync — and not one word saying why, because `exit 0` is indistinguishable from "ran fine, nothing to say". jq is absent by default on macOS, on the Debian/Ubuntu slim images, on Alpine and in most CI containers. That is not a prerequisite to document; it is the plugin handing its own packaging problem to whoever installs it. `tac` was worse: GNU-only, so the prior-art hook's enclosing-definition arm did nothing at all on every Mac, silently, from the day it shipped. It is not replaced but removed — scribe_defs judges each line independently, so extracting forward and taking `tail -1` is the same answer as reversing and taking the head, and it drops the early-exit `head` that #4042 was filed for. No server contract changed, so a lagging plugin cache keeps working. scribe_json.awk JSON -> IDX<TAB>PATH<TAB>VALUE. Two modes: `whole` for an event or a response body, `lines` for a transcript, where an unparseable record is dropped and the rest still read — the `map(try fromjson catch empty)` the jq program opened with. Arrays also report their LENGTH at `[#]`, which is what keeps "zero notes" distinct from "no answer" (#2932). scribe_turn.awk the turn-bounding program, replacing the thirty lines of jq in the Stop hook. scribe_defs.sh scribe_json_flat / _pick / _list / _len / _list_minus read, scribe_json_out writes the envelope (five copies of one shape, gone), scribe_urlenc replaces `jq -sRr '@uri'`. Percent-encoding goes through `od -tu1` rather than an awk character loop on purpose: awk's idea of a character follows the locale, so gawk reads an accented letter as one and mawk as two, and an encoder built on substr() would emit a different URL depending on which awk is installed. Encoding is defined on bytes. Verified byte-identical to `jq -sRr '@uri'`. Measured, not assumed. The per-event path costs 8ms against jq's 3ms. The transcript path was 70x slower until two fixes: the Stop hook now finds where the turn starts with a fixed-string grep before parsing (a needle carrying unescaped quotes cannot occur inside a JSON string, so it matches only at a record's top level — checked against a full JSON parse of a 27MB transcript: 152 prompt records, 152 matches, no misses, no extras), and the parser reads each token out of a 1024-byte window instead of copying the rest of the buffer per token, which was quadratic in line length on the 400KB tool results a transcript carries. Differential-tested against the jq program it replaces over 724 windows cut from three real transcripts — 724 identical, 0 mismatched, 45 of them exercising a real task close and a real reply. That sweep is what caught `scribe_turn.awk` never setting FS, which truncated every multi-word reply at its first space and was invisible to a test whose replies were all empty. check_plugin.py's `jq -R` lint becomes a guard against either binary coming back, and three smoke checks lose their `shutil.which("jq")` skip. jq is not in `ci-python` either, so those three announced a skip on every CI run and had never once run there: removing the dependency from the product also closed a permanent hole in its verification. They pass now across all ten hooks. tests/test_hook_json_reader.py is a differential against Python's `json` over nested objects, arrays, unicode, escapes, control characters, empty cases and a value longer than the token window, plus the envelope, the encoder and the turn analyzer. 139 cases. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-20 12:20:45 -04:00