fix(plugin): the hooks need no jq and no tac (#4107)

Every hook opened `command -v jq >/dev/null 2>&1 || exit 0`, so on a machine
without jq the operator got no session context, no rules, no prior art and no
process sync — and not one word saying why, because `exit 0` is
indistinguishable from "ran fine, nothing to say". jq is absent by default on
macOS, on the Debian/Ubuntu slim images, on Alpine and in most CI containers.
That is not a prerequisite to document; it is the plugin handing its own
packaging problem to whoever installs it.

`tac` was worse: GNU-only, so the prior-art hook's enclosing-definition arm
did nothing at all on every Mac, silently, from the day it shipped. It is not
replaced but removed — scribe_defs judges each line independently, so
extracting forward and taking `tail -1` is the same answer as reversing and
taking the head, and it drops the early-exit `head` that #4042 was filed for.

No server contract changed, so a lagging plugin cache keeps working.

  scribe_json.awk   JSON -> IDX<TAB>PATH<TAB>VALUE. Two modes: `whole` for an
                    event or a response body, `lines` for a transcript, where
                    an unparseable record is dropped and the rest still read —
                    the `map(try fromjson catch empty)` the jq program opened
                    with. Arrays also report their LENGTH at `[#]`, which is
                    what keeps "zero notes" distinct from "no answer" (#2932).
  scribe_turn.awk   the turn-bounding program, replacing the thirty lines of
                    jq in the Stop hook.
  scribe_defs.sh    scribe_json_flat / _pick / _list / _len / _list_minus read,
                    scribe_json_out writes the envelope (five copies of one
                    shape, gone), scribe_urlenc replaces `jq -sRr '@uri'`.

Percent-encoding goes through `od -tu1` rather than an awk character loop on
purpose: awk's idea of a character follows the locale, so gawk reads an
accented letter as one and mawk as two, and an encoder built on substr() would
emit a different URL depending on which awk is installed. Encoding is defined
on bytes. Verified byte-identical to `jq -sRr '@uri'`.

Measured, not assumed. The per-event path costs 8ms against jq's 3ms. The
transcript path was 70x slower until two fixes: the Stop hook now finds where
the turn starts with a fixed-string grep before parsing (a needle carrying
unescaped quotes cannot occur inside a JSON string, so it matches only at a
record's top level — checked against a full JSON parse of a 27MB transcript:
152 prompt records, 152 matches, no misses, no extras), and the parser reads
each token out of a 1024-byte window instead of copying the rest of the buffer
per token, which was quadratic in line length on the 400KB tool results a
transcript carries.

Differential-tested against the jq program it replaces over 724 windows cut
from three real transcripts — 724 identical, 0 mismatched, 45 of them
exercising a real task close and a real reply. That sweep is what caught
`scribe_turn.awk` never setting FS, which truncated every multi-word reply at
its first space and was invisible to a test whose replies were all empty.

check_plugin.py's `jq -R` lint becomes a guard against either binary coming
back, and three smoke checks lose their `shutil.which("jq")` skip. jq is not
in `ci-python` either, so those three announced a skip on every CI run and had
never once run there: removing the dependency from the product also closed a
permanent hole in its verification. They pass now across all ten hooks.

tests/test_hook_json_reader.py is a differential against Python's `json` over
nested objects, arrays, unicode, escapes, control characters, empty cases and
a value longer than the token window, plus the envelope, the encoder and the
turn analyzer. 139 cases.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy

plugin/hooks/scribe_defs.sh

@@ -19,6 +19,10 @@
 #   scribe_rules_live FILE     live rule ids from the exclusion ledger,
 #                              comma-joined; entries age out (#3751)
 #   scribe_rules_append FILE   stdin ids -> the ledger, timestamped
+#   scribe_json_flat           stdin JSON -> IDX<TAB>PATH<TAB>VALUE lines,
+#                              queried with scribe_json_pick / _list / _len.
+#                              scribe_json_out writes the hook envelope,
+#                              scribe_urlenc percent-encodes. No jq (#4107).
 #   scribe_scope_query DIR     `project_id=N` or `repo=<enc>` for DIR — the
 #                              project-scope key EVERY hook sends (#4085).
 #                              Helpers: scribe_marker_file, scribe_url_host,
@@ -39,6 +43,216 @@ scribe_skip_path() {
   return 1
 }
 
+# ---------------------------------------------------------------------------
+# JSON AND URL-ENCODING, WITHOUT jq (#4107).
+#
+# Until this section every hook opened `command -v jq >/dev/null 2>&1 || exit 0`
+# and a machine without jq got no session context, no rules, no prior art and
+# no process sync — silently, because `exit 0` is indistinguishable from "ran
+# fine, nothing to say". That is the plugin pushing its own packaging problem
+# onto whoever installs it, and it is the same silence #4085 existed to remove.
+# jq is absent by default on macOS, on the Debian/Ubuntu slim images, on Alpine
+# and in most CI containers. awk, sed, tr and od are POSIX; every one of them
+# is already required by code above this line.
+#
+# scribe_json.awk does the parsing and carries the format. These are the four
+# jobs the hooks actually had jq for:
+#
+#   READ    scribe_json_flat / _flat_lines turn stdin into `IDX<TAB>PATH<TAB>
+#           VALUE` lines ONCE, and scribe_json_pick / _list / _len query that
+#           text. Parse once, query many: a hook reads four fields off one
+#           event, and re-parsing per field is four passes over a payload that
+#           may be an entire source file.
+#   WRITE   scribe_json_out emits the hookSpecificOutput envelope, which is the
+#           one piece of JSON these hooks produce.
+#   ENCODE  scribe_urlenc replaces `jq -sRr '@uri'`, byte-exact for UTF-8.
+#   DECODE  scribe_json_unescape turns a raw JSON string body into text.
+
+_scribe_self=${BASH_SOURCE[0]:-$0}
+# shellcheck disable=SC1007  # `CDPATH= cd` scopes one variable to one command
+SCRIBE_HOOK_DIR=$(CDPATH= cd -- "$(dirname -- "$_scribe_self")" 2>/dev/null && pwd) \
+  || SCRIBE_HOOK_DIR=$(dirname -- "$_scribe_self")
+
+# One JSON value on stdin → flat lines. Empty output means it did not parse.
+scribe_json_flat() {
+  awk -f "$SCRIBE_HOOK_DIR/scribe_json.awk" 2>/dev/null || true
+}
+
+# JSONL on stdin → flat lines, IDX counting the records that PARSED. A line
+# that does not parse is dropped and the rest are still read.
+scribe_json_flat_lines() {
+  awk -v mode=lines -f "$SCRIBE_HOOK_DIR/scribe_json.awk" 2>/dev/null || true
+}
+
+# $1 flat text, $2 exact path → the decoded scalar, or "" if absent.
+#
+# `null` READS AS ABSENT, matching the `// empty` every call site used to carry.
+# A field the server left null and one it never sent mean the same thing to
+# every caller here, and the shells downstream test `[ -n "$x" ]`.
+scribe_json_pick() {
+  printf '%s\n' "${1:-}" \
+    | awk -F'\t' -v p="$2" '$2 == p { if ($3 != "null") print $3; exit }' 2>/dev/null \
+    | scribe_json_unescape
+}
+
+# $1 flat text, $2 array path → each element decoded, one per line. Elements
+# are ids and keys here, so a value containing a newline is not a case that
+# arises; one would split into two lines.
+scribe_json_list() {
+  printf '%s\n' "${1:-}" \
+    | awk -F'\t' -v p="$2" '
+        BEGIN { plen = length(p) }
+        substr($2, 1, plen) == p {
+          rest = substr($2, plen + 1)
+          if (rest ~ /^\[[0-9]+\]$/ && $3 != "null") print $3
+        }' 2>/dev/null \
+    | scribe_json_unescape
+}
+
+# $1 flat text, $2 array path, $3 array path to SUBTRACT → the elements of $2
+# that are not in $3, in order. The reuse and sync channels are two classes of
+# one answer and each has its own session ledger, so the ids that go in the
+# reuse file are precisely `note_ids` minus `sync_note_ids` — jq wrote that as
+# `(.note_ids // []) - (.sync_note_ids // [])`.
+scribe_json_list_minus() {
+  local keep
+  keep=$(scribe_json_list "$1" "$3")
+  scribe_json_list "$1" "$2" \
+    | awk -v drop="$keep" '
+        BEGIN { n = split(drop, a, "\n"); for (i = 1; i <= n; i++) if (a[i] != "") s[a[i]] = 1 }
+        NF && !($0 in s)' 2>/dev/null || true
+}
+
+# $1 flat text, $2 array path → its length. "" when the path is not an array,
+# which is NOT the same as 0 — see the `[#]` note in scribe_json.awk.
+scribe_json_len() {
+  printf '%s\n' "${1:-}" \
+    | awk -F'\t' -v p="$2" '$2 == p "[#]" { print $3; exit }' 2>/dev/null
+}
+
+# Raw JSON string bodies on stdin → text. One line in, one value out.
+scribe_json_unescape() {
+  awk '
+    function hex4(h,   i, c, d, v) {
+      v = 0
+      for (i = 1; i <= 4; i++) {
+        c = tolower(substr(h, i, 1))
+        d = index("0123456789abcdef", c) - 1
+        if (d < 0) return -1
+        v = v * 16 + d
+      }
+      return v
+    }
+    {
+      v = $0
+      if (index(v, "\\") == 0) { print v; next }
+      o = ""
+      i = 1
+      L = length(v)
+      while (i <= L) {
+        c = substr(v, i, 1)
+        if (c != "\\") { o = o c; i++; continue }
+        d = substr(v, i + 1, 1)
+        i += 2
+        if (d == "n") o = o "\n"
+        else if (d == "t") o = o "\t"
+        else if (d == "r") o = o "\r"
+        else if (d == "b") o = o sprintf("%c", 8)
+        else if (d == "f") o = o sprintf("%c", 12)
+        else if (d == "u") {
+          hi = hex4(substr(v, i, 4))
+          if (hi < 0) { o = o "\\u"; continue }
+          i += 4
+          # A surrogate PAIR is one character written as two escapes; decoding
+          # the halves separately yields two replacement characters instead.
+          if (hi >= 55296 && hi <= 56319 && substr(v, i, 2) == "\\u") {
+            lo = hex4(substr(v, i + 2, 4))
+            if (lo >= 56320 && lo <= 57343) {
+              hi = 65536 + (hi - 55296) * 1024 + (lo - 56320)
+              i += 6
+            }
+          }
+          o = o sprintf("%c", hi)
+        }
+        else o = o d
+      }
+      print o
+    }' 2>/dev/null || true
+}
+
+# Text on stdin → a JSON string BODY (escaped, no surrounding quotes, one line).
+#
+# split/join rather than gsub: gsub reads `\` and `&` in its replacement as
+# metacharacters, so emitting a literal backslash through it takes a
+# quadruple-escape whose meaning varies between awks. Plain concatenation has
+# no such reading, and getting this wrong corrupts every injected context that
+# happens to contain a Windows path or a regex.
+scribe_json_escape() {
+  awk '
+    function rep(str, sep, with,   parts, cnt, i, o) {
+      cnt = split(str, parts, sep)
+      o = parts[1]
+      for (i = 2; i <= cnt; i++) o = o with parts[i]
+      return o
+    }
+    {
+      line = $0
+      line = rep(line, "\\\\", "\\\\")
+      line = rep(line, "\"", "\\\"")
+      line = rep(line, "\t", "\\t")
+      line = rep(line, "\r", "\\r")
+      # EVERY OTHER CONTROL CHARACTER TOO, or the envelope is not JSON. A raw
+      # byte below 0x20 inside a string is invalid, and Claude Code discards
+      # the whole hook output rather than the one field — so a single stray
+      # character anywhere in an injected context costs the entire injection.
+      # jq escaped these; a writer that knew only about tab and carriage
+      # return would have regressed quietly, on the rare input nobody thinks
+      # to test by hand.
+      for (c = 1; c < 32; c++) {
+        if (c == 9 || c == 10 || c == 13) continue
+        ctl = sprintf("%c", c)
+        if (index(line, ctl)) line = rep(line, ctl, sprintf("\\u%04x", c))
+      }
+      if (NR > 1) printf "\\n"
+      printf "%s", line
+    }' 2>/dev/null || true
+}
+
+# The one JSON document these hooks WRITE: $1 hook event name, $2 the context.
+# Every hook emitted this through `jq -n --arg c`, five copies of one shape.
+scribe_json_out() {
+  local esc
+  esc=$(printf '%s' "$2" | scribe_json_escape) || return 0
+  printf '{"hookSpecificOutput":{"hookEventName":"%s","additionalContext":"%s"}}\n' \
+    "$1" "$esc"
+}
+
+# stdin → percent-encoded, replacing `jq -sRr '@uri'`.
+#
+# THROUGH `od`, NOT A CHARACTER LOOP IN awk, and the reason is the bug this
+# would otherwise reintroduce. awk's idea of a "character" follows the locale:
+# in a UTF-8 locale gawk reads é as one character and mawk as two bytes, so any
+# encoder built on substr() produces a different URL on the same input
+# depending on which awk is installed. Percent-encoding is defined on BYTES. od
+# -tu1 gives bytes, on every platform, and the only characters passed through
+# unencoded are the unreserved ASCII set — which every awk agrees about.
+#
+# Slightly more aggressive than jq's `@uri`, which leaves `!~*'()` alone. Those
+# are legal either way and the server decodes both identically.
+scribe_urlenc() {
+  od -An -v -tu1 2>/dev/null \
+    | awk '{
+        for (i = 1; i <= NF; i++) {
+          b = $i + 0
+          if ((b >= 48 && b <= 57) || (b >= 65 && b <= 90) ||
+              (b >= 97 && b <= 122) || b == 45 || b == 46 || b == 95 || b == 126)
+            printf "%s", sprintf("%c", b)
+          else
+            printf "%%%02X", b
+        }
+      }' 2>/dev/null || true
+}
+
 # ---------------------------------------------------------------------------
 # kind<TAB>name for each thing a piece of code DEFINES, in source order. One
 # program, two consumers: the local duplicate arm (every definition in the
@@ -281,7 +495,7 @@ scribe_rules_live() {
 }
 
 # Append surfaced ids, stamped. Reads ids on stdin, one per line — the shape
-# `jq -r '(.rule_ids // [])[]?'` already produces at both call sites.
+# `scribe_json_list "$flat" '.rule_ids'` already produces at both call sites.
 scribe_rules_append() {
   local f="$1" now
   [ -n "$f" ] || return 0
@@ -425,19 +639,20 @@ scribe_url_host() {
 # `set -u` these hooks all run with aborts the hook and costs the whole
 # session's context — a failure far larger than the message it was fetching.
 scribe_marker_read() {
-  local f="$1" id inst want
+  local f="$1" flat id inst want
   [ -n "$f" ] && [ -f "$f" ] || { printf '\t'; return 0; }
-  command -v jq >/dev/null 2>&1 || { printf '\t'; return 0; }
-  # A bare integer is valid JSON, so one filter reads both forms.
-  id=$(jq -r 'if type=="number" then (.|floor|tostring)
-              elif type=="object" then (.project_id // empty | tostring)
-              else empty end' "$f" 2>/dev/null) || id=""
+  flat=$(scribe_json_flat < "$f") || flat=""
+  # Both accepted forms, read off one parse: the object's `project_id`, or —
+  # for the bare integer someone writes by hand — the root scalar, whose path
+  # is the empty string because it sits under no key.
+  id=$(scribe_json_pick "$flat" '.project_id')
+  [ -n "$id" ] || id=$(scribe_json_pick "$flat" '')
   case "$id" in ''|*[!0-9]*) id="" ;; esac
   if [ -z "$id" ] || [ "$id" = "0" ]; then
     printf '\tnames no project_id'
     return 0
   fi
-  inst=$(jq -r 'if type=="object" then (.instance // empty) else empty end' "$f" 2>/dev/null) || inst=""
+  inst=$(scribe_json_pick "$flat" '.instance')
   if [ -n "$inst" ]; then
     want=$(scribe_url_host "${url:-}")
     inst=$(scribe_url_host "$inst")
@@ -468,7 +683,6 @@ scribe_scope_query() {
   fi
   repo=$(git -C "$dir" remote get-url origin 2>/dev/null || true)
   [ -n "$repo" ] || return 0
-  command -v jq >/dev/null 2>&1 || return 0
-  enc=$(printf '%s' "$repo" | jq -sRr '@uri' 2>/dev/null) || enc=""
+  enc=$(printf '%s' "$repo" | scribe_urlenc) || enc=""
   [ -n "$enc" ] && printf 'repo=%s' "$enc"
 }