bvandeusenandClaude Opus 5
a49e7ed2af
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 49s
CI & Build / TypeScript typecheck (push) Successful in 57s
CI & Build / Python tests (push) Failing after 1m7s
CI & Build / Build & push image (push) Skipped
Every hook opened `command -v jq >/dev/null 2>&1 || exit 0`, so on a machine without jq the operator got no session context, no rules, no prior art and no process sync — and not one word saying why, because `exit 0` is indistinguishable from "ran fine, nothing to say". jq is absent by default on macOS, on the Debian/Ubuntu slim images, on Alpine and in most CI containers. That is not a prerequisite to document; it is the plugin handing its own packaging problem to whoever installs it. `tac` was worse: GNU-only, so the prior-art hook's enclosing-definition arm did nothing at all on every Mac, silently, from the day it shipped. It is not replaced but removed — scribe_defs judges each line independently, so extracting forward and taking `tail -1` is the same answer as reversing and taking the head, and it drops the early-exit `head` that #4042 was filed for. No server contract changed, so a lagging plugin cache keeps working. scribe_json.awk JSON -> IDX<TAB>PATH<TAB>VALUE. Two modes: `whole` for an event or a response body, `lines` for a transcript, where an unparseable record is dropped and the rest still read — the `map(try fromjson catch empty)` the jq program opened with. Arrays also report their LENGTH at `[#]`, which is what keeps "zero notes" distinct from "no answer" (#2932). scribe_turn.awk the turn-bounding program, replacing the thirty lines of jq in the Stop hook. scribe_defs.sh scribe_json_flat / _pick / _list / _len / _list_minus read, scribe_json_out writes the envelope (five copies of one shape, gone), scribe_urlenc replaces `jq -sRr '@uri'`. Percent-encoding goes through `od -tu1` rather than an awk character loop on purpose: awk's idea of a character follows the locale, so gawk reads an accented letter as one and mawk as two, and an encoder built on substr() would emit a different URL depending on which awk is installed. Encoding is defined on bytes. Verified byte-identical to `jq -sRr '@uri'`. Measured, not assumed. The per-event path costs 8ms against jq's 3ms. The transcript path was 70x slower until two fixes: the Stop hook now finds where the turn starts with a fixed-string grep before parsing (a needle carrying unescaped quotes cannot occur inside a JSON string, so it matches only at a record's top level — checked against a full JSON parse of a 27MB transcript: 152 prompt records, 152 matches, no misses, no extras), and the parser reads each token out of a 1024-byte window instead of copying the rest of the buffer per token, which was quadratic in line length on the 400KB tool results a transcript carries. Differential-tested against the jq program it replaces over 724 windows cut from three real transcripts — 724 identical, 0 mismatched, 45 of them exercising a real task close and a real reply. That sweep is what caught `scribe_turn.awk` never setting FS, which truncated every multi-word reply at its first space and was invisible to a test whose replies were all empty. check_plugin.py's `jq -R` lint becomes a guard against either binary coming back, and three smoke checks lose their `shutil.which("jq")` skip. jq is not in `ci-python` either, so those three announced a skip on every CI run and had never once run there: removing the dependency from the product also closed a permanent hole in its verification. They pass now across all ten hooks. tests/test_hook_json_reader.py is a differential against Python's `json` over nested objects, arrays, unicode, escapes, control characters, empty cases and a value longer than the token window, plus the envelope, the encoder and the turn analyzer. 139 cases. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
689 lines
32 KiB
Bash
689 lines
32 KiB
Bash
#!/usr/bin/env bash
|
|
# shellcheck shell=bash
|
|
# Scribe plugin — the pieces the hooks share (#2901, #2278).
|
|
#
|
|
# scribe_prior_art.sh fires BEFORE a Write/Edit tool call; scribe_after_write.sh
|
|
# fires AFTER a Bash tool call and diffs the working tree, so code written by
|
|
# sed/heredocs/scripts gets the same prior-art and ledger checks. Both need the
|
|
# same three things, kept here so they cannot drift apart:
|
|
#
|
|
# scribe_skip_path PATH formats that hold prose or data, not shapes
|
|
# scribe_defs stdin code → "kind<TAB>name" per definition
|
|
# scribe_local_dups ROOT REL "kind<TAB>name" lines on stdin → the by-name
|
|
# local-duplicate lines (ARM 1, #2280)
|
|
# scribe_unreached STATE SID SECS REL the "Scribe didn't answer" line, once
|
|
# per outage (#2932) — or nothing, if said lately
|
|
# scribe_reached STATE SID the server answered: the next outage speaks again
|
|
# scribe_config sets `url` + `token` from the env, returns 0
|
|
# only if BOTH are usable (#2278)
|
|
# scribe_rules_live FILE live rule ids from the exclusion ledger,
|
|
# comma-joined; entries age out (#3751)
|
|
# scribe_rules_append FILE stdin ids -> the ledger, timestamped
|
|
# scribe_json_flat stdin JSON -> IDX<TAB>PATH<TAB>VALUE lines,
|
|
# queried with scribe_json_pick / _list / _len.
|
|
# scribe_json_out writes the hook envelope,
|
|
# scribe_urlenc percent-encodes. No jq (#4107).
|
|
# scribe_scope_query DIR `project_id=N` or `repo=<enc>` for DIR — the
|
|
# project-scope key EVERY hook sends (#4085).
|
|
# Helpers: scribe_marker_file, scribe_url_host,
|
|
# scribe_marker_read (id<TAB>why-not),
|
|
# scribe_marker_project (the id alone)
|
|
#
|
|
# Sourced, not executed: `. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"`.
|
|
|
|
# Skip formats that hold prose or data rather than reusable code. Purely to
|
|
# avoid a pointless round-trip — the server would return nothing for these
|
|
# anyway. Config formats are NOT skipped: a CI workflow or a compose file is
|
|
# often exactly the thing worth reusing.
|
|
scribe_skip_path() {
|
|
case "$1" in
|
|
*.md|*.mdx|*.txt|*.rst|*.json|*.lock|*.log|*.csv|*.tsv|*.svg|*.png|*.jpg|*.jpeg|*.gif|*.ico|*.pdf)
|
|
return 0 ;;
|
|
esac
|
|
return 1
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# JSON AND URL-ENCODING, WITHOUT jq (#4107).
|
|
#
|
|
# Until this section every hook opened `command -v jq >/dev/null 2>&1 || exit 0`
|
|
# and a machine without jq got no session context, no rules, no prior art and
|
|
# no process sync — silently, because `exit 0` is indistinguishable from "ran
|
|
# fine, nothing to say". That is the plugin pushing its own packaging problem
|
|
# onto whoever installs it, and it is the same silence #4085 existed to remove.
|
|
# jq is absent by default on macOS, on the Debian/Ubuntu slim images, on Alpine
|
|
# and in most CI containers. awk, sed, tr and od are POSIX; every one of them
|
|
# is already required by code above this line.
|
|
#
|
|
# scribe_json.awk does the parsing and carries the format. These are the four
|
|
# jobs the hooks actually had jq for:
|
|
#
|
|
# READ scribe_json_flat / _flat_lines turn stdin into `IDX<TAB>PATH<TAB>
|
|
# VALUE` lines ONCE, and scribe_json_pick / _list / _len query that
|
|
# text. Parse once, query many: a hook reads four fields off one
|
|
# event, and re-parsing per field is four passes over a payload that
|
|
# may be an entire source file.
|
|
# WRITE scribe_json_out emits the hookSpecificOutput envelope, which is the
|
|
# one piece of JSON these hooks produce.
|
|
# ENCODE scribe_urlenc replaces `jq -sRr '@uri'`, byte-exact for UTF-8.
|
|
# DECODE scribe_json_unescape turns a raw JSON string body into text.
|
|
|
|
_scribe_self=${BASH_SOURCE[0]:-$0}
|
|
# shellcheck disable=SC1007 # `CDPATH= cd` scopes one variable to one command
|
|
SCRIBE_HOOK_DIR=$(CDPATH= cd -- "$(dirname -- "$_scribe_self")" 2>/dev/null && pwd) \
|
|
|| SCRIBE_HOOK_DIR=$(dirname -- "$_scribe_self")
|
|
|
|
# One JSON value on stdin → flat lines. Empty output means it did not parse.
|
|
scribe_json_flat() {
|
|
awk -f "$SCRIBE_HOOK_DIR/scribe_json.awk" 2>/dev/null || true
|
|
}
|
|
|
|
# JSONL on stdin → flat lines, IDX counting the records that PARSED. A line
|
|
# that does not parse is dropped and the rest are still read.
|
|
scribe_json_flat_lines() {
|
|
awk -v mode=lines -f "$SCRIBE_HOOK_DIR/scribe_json.awk" 2>/dev/null || true
|
|
}
|
|
|
|
# $1 flat text, $2 exact path → the decoded scalar, or "" if absent.
|
|
#
|
|
# `null` READS AS ABSENT, matching the `// empty` every call site used to carry.
|
|
# A field the server left null and one it never sent mean the same thing to
|
|
# every caller here, and the shells downstream test `[ -n "$x" ]`.
|
|
scribe_json_pick() {
|
|
printf '%s\n' "${1:-}" \
|
|
| awk -F'\t' -v p="$2" '$2 == p { if ($3 != "null") print $3; exit }' 2>/dev/null \
|
|
| scribe_json_unescape
|
|
}
|
|
|
|
# $1 flat text, $2 array path → each element decoded, one per line. Elements
|
|
# are ids and keys here, so a value containing a newline is not a case that
|
|
# arises; one would split into two lines.
|
|
scribe_json_list() {
|
|
printf '%s\n' "${1:-}" \
|
|
| awk -F'\t' -v p="$2" '
|
|
BEGIN { plen = length(p) }
|
|
substr($2, 1, plen) == p {
|
|
rest = substr($2, plen + 1)
|
|
if (rest ~ /^\[[0-9]+\]$/ && $3 != "null") print $3
|
|
}' 2>/dev/null \
|
|
| scribe_json_unescape
|
|
}
|
|
|
|
# $1 flat text, $2 array path, $3 array path to SUBTRACT → the elements of $2
|
|
# that are not in $3, in order. The reuse and sync channels are two classes of
|
|
# one answer and each has its own session ledger, so the ids that go in the
|
|
# reuse file are precisely `note_ids` minus `sync_note_ids` — jq wrote that as
|
|
# `(.note_ids // []) - (.sync_note_ids // [])`.
|
|
scribe_json_list_minus() {
|
|
local keep
|
|
keep=$(scribe_json_list "$1" "$3")
|
|
scribe_json_list "$1" "$2" \
|
|
| awk -v drop="$keep" '
|
|
BEGIN { n = split(drop, a, "\n"); for (i = 1; i <= n; i++) if (a[i] != "") s[a[i]] = 1 }
|
|
NF && !($0 in s)' 2>/dev/null || true
|
|
}
|
|
|
|
# $1 flat text, $2 array path → its length. "" when the path is not an array,
|
|
# which is NOT the same as 0 — see the `[#]` note in scribe_json.awk.
|
|
scribe_json_len() {
|
|
printf '%s\n' "${1:-}" \
|
|
| awk -F'\t' -v p="$2" '$2 == p "[#]" { print $3; exit }' 2>/dev/null
|
|
}
|
|
|
|
# Raw JSON string bodies on stdin → text. One line in, one value out.
|
|
scribe_json_unescape() {
|
|
awk '
|
|
function hex4(h, i, c, d, v) {
|
|
v = 0
|
|
for (i = 1; i <= 4; i++) {
|
|
c = tolower(substr(h, i, 1))
|
|
d = index("0123456789abcdef", c) - 1
|
|
if (d < 0) return -1
|
|
v = v * 16 + d
|
|
}
|
|
return v
|
|
}
|
|
{
|
|
v = $0
|
|
if (index(v, "\\") == 0) { print v; next }
|
|
o = ""
|
|
i = 1
|
|
L = length(v)
|
|
while (i <= L) {
|
|
c = substr(v, i, 1)
|
|
if (c != "\\") { o = o c; i++; continue }
|
|
d = substr(v, i + 1, 1)
|
|
i += 2
|
|
if (d == "n") o = o "\n"
|
|
else if (d == "t") o = o "\t"
|
|
else if (d == "r") o = o "\r"
|
|
else if (d == "b") o = o sprintf("%c", 8)
|
|
else if (d == "f") o = o sprintf("%c", 12)
|
|
else if (d == "u") {
|
|
hi = hex4(substr(v, i, 4))
|
|
if (hi < 0) { o = o "\\u"; continue }
|
|
i += 4
|
|
# A surrogate PAIR is one character written as two escapes; decoding
|
|
# the halves separately yields two replacement characters instead.
|
|
if (hi >= 55296 && hi <= 56319 && substr(v, i, 2) == "\\u") {
|
|
lo = hex4(substr(v, i + 2, 4))
|
|
if (lo >= 56320 && lo <= 57343) {
|
|
hi = 65536 + (hi - 55296) * 1024 + (lo - 56320)
|
|
i += 6
|
|
}
|
|
}
|
|
o = o sprintf("%c", hi)
|
|
}
|
|
else o = o d
|
|
}
|
|
print o
|
|
}' 2>/dev/null || true
|
|
}
|
|
|
|
# Text on stdin → a JSON string BODY (escaped, no surrounding quotes, one line).
|
|
#
|
|
# split/join rather than gsub: gsub reads `\` and `&` in its replacement as
|
|
# metacharacters, so emitting a literal backslash through it takes a
|
|
# quadruple-escape whose meaning varies between awks. Plain concatenation has
|
|
# no such reading, and getting this wrong corrupts every injected context that
|
|
# happens to contain a Windows path or a regex.
|
|
scribe_json_escape() {
|
|
awk '
|
|
function rep(str, sep, with, parts, cnt, i, o) {
|
|
cnt = split(str, parts, sep)
|
|
o = parts[1]
|
|
for (i = 2; i <= cnt; i++) o = o with parts[i]
|
|
return o
|
|
}
|
|
{
|
|
line = $0
|
|
line = rep(line, "\\\\", "\\\\")
|
|
line = rep(line, "\"", "\\\"")
|
|
line = rep(line, "\t", "\\t")
|
|
line = rep(line, "\r", "\\r")
|
|
# EVERY OTHER CONTROL CHARACTER TOO, or the envelope is not JSON. A raw
|
|
# byte below 0x20 inside a string is invalid, and Claude Code discards
|
|
# the whole hook output rather than the one field — so a single stray
|
|
# character anywhere in an injected context costs the entire injection.
|
|
# jq escaped these; a writer that knew only about tab and carriage
|
|
# return would have regressed quietly, on the rare input nobody thinks
|
|
# to test by hand.
|
|
for (c = 1; c < 32; c++) {
|
|
if (c == 9 || c == 10 || c == 13) continue
|
|
ctl = sprintf("%c", c)
|
|
if (index(line, ctl)) line = rep(line, ctl, sprintf("\\u%04x", c))
|
|
}
|
|
if (NR > 1) printf "\\n"
|
|
printf "%s", line
|
|
}' 2>/dev/null || true
|
|
}
|
|
|
|
# The one JSON document these hooks WRITE: $1 hook event name, $2 the context.
|
|
# Every hook emitted this through `jq -n --arg c`, five copies of one shape.
|
|
scribe_json_out() {
|
|
local esc
|
|
esc=$(printf '%s' "$2" | scribe_json_escape) || return 0
|
|
printf '{"hookSpecificOutput":{"hookEventName":"%s","additionalContext":"%s"}}\n' \
|
|
"$1" "$esc"
|
|
}
|
|
|
|
# stdin → percent-encoded, replacing `jq -sRr '@uri'`.
|
|
#
|
|
# THROUGH `od`, NOT A CHARACTER LOOP IN awk, and the reason is the bug this
|
|
# would otherwise reintroduce. awk's idea of a "character" follows the locale:
|
|
# in a UTF-8 locale gawk reads é as one character and mawk as two bytes, so any
|
|
# encoder built on substr() produces a different URL on the same input
|
|
# depending on which awk is installed. Percent-encoding is defined on BYTES. od
|
|
# -tu1 gives bytes, on every platform, and the only characters passed through
|
|
# unencoded are the unreserved ASCII set — which every awk agrees about.
|
|
#
|
|
# Slightly more aggressive than jq's `@uri`, which leaves `!~*'()` alone. Those
|
|
# are legal either way and the server decodes both identically.
|
|
scribe_urlenc() {
|
|
od -An -v -tu1 2>/dev/null \
|
|
| awk '{
|
|
for (i = 1; i <= NF; i++) {
|
|
b = $i + 0
|
|
if ((b >= 48 && b <= 57) || (b >= 65 && b <= 90) ||
|
|
(b >= 97 && b <= 122) || b == 45 || b == 46 || b == 95 || b == 126)
|
|
printf "%s", sprintf("%c", b)
|
|
else
|
|
printf "%%%02X", b
|
|
}
|
|
}' 2>/dev/null || true
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# kind<TAB>name for each thing a piece of code DEFINES, in source order. One
|
|
# program, two consumers: the local duplicate arm (every definition in the
|
|
# payload) and the ledger feed (#2791, below: the definitions being written,
|
|
# or the one enclosing an Edit). Rule-for-rule mirrored by the server's
|
|
# services/coverage.py extract_shapes — ledger rows are keyed by what THAT
|
|
# sees, so the two must agree on what counts as a definition.
|
|
scribe_defs() {
|
|
awk '
|
|
{
|
|
# CSS class definition: .name { or .name,
|
|
if (match($0, /^[[:space:]]*\.[A-Za-z][A-Za-z0-9_-]*[[:space:]]*[,{]/)) {
|
|
t = $0; sub(/^[[:space:]]*\./, "", t); sub(/[[:space:]]*[,{].*$/, "", t)
|
|
if (t != "") print "css\t" t; next
|
|
}
|
|
line = $0; sub(/^[[:space:]]+/, "", line)
|
|
# Strip leading declaration modifiers so the definition keyword is the
|
|
# first word regardless of language (export/pub/private/suspend/...).
|
|
sub(/^((pub(\([a-z]+\))?|export|default|private|internal|protected|public|static|suspend|async|open|sealed|data|abstract|final|inline|unsafe|extern|override)[[:space:]]+)*/, "", line)
|
|
# Go method with receiver: func (r *T) Name(
|
|
if (match(line, /^func[[:space:]]*\([^)]*\)[[:space:]]*[A-Za-z_]/)) {
|
|
t = line; sub(/^func[[:space:]]*\([^)]*\)[[:space:]]*/, "", t)
|
|
sub(/[^A-Za-z0-9_].*$/, "", t)
|
|
if (t != "") print "sym\t" t; next
|
|
}
|
|
# Keyword-announced definitions, functions and named types alike.
|
|
# Dunders are skipped: every class defines __init__, so "already defined
|
|
# in N other files" is guaranteed noise for them — and noise is what
|
|
# teaches sessions to skip the hint.
|
|
if (match(line, /^(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+[A-Za-z_$]/)) {
|
|
t = line; sub(/^[a-z]+[[:space:]]+/, "", t)
|
|
sub(/[^A-Za-z0-9_$].*$/, "", t)
|
|
# `type` defines only when something follows the name (= or {); an
|
|
# import specifier `type Foo,` is the same two words and defines
|
|
# nothing (mirror of coverage.py, #2904).
|
|
if (line ~ /^type[[:space:]]/) {
|
|
rest = line; sub(/^type[[:space:]]+[A-Za-z_$][A-Za-z0-9_$]*/, "", rest)
|
|
if (rest !~ /[={]/) next
|
|
}
|
|
if (t != "" && t !~ /^__.*__$/) print "sym\t" t; next
|
|
}
|
|
# Arrow/expression assignment: const name = (…) / let name = async (
|
|
if (match(line, /^(const|let)[[:space:]]+[A-Za-z_$][A-Za-z0-9_$]*[[:space:]]*=[[:space:]]*(async[[:space:]]*)?[(<]/)) {
|
|
t = line; sub(/^(const|let)[[:space:]]+/, "", t)
|
|
sub(/[^A-Za-z0-9_$].*$/, "", t)
|
|
if (t != "") print "sym\t" t; next
|
|
}
|
|
}
|
|
' 2>/dev/null
|
|
}
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# ARM 1 — BY NAME, LOCALLY (#2280). Does a definition of this already exist?
|
|
#
|
|
# The recorded arms ask Scribe what was RECORDED; the ledger arm (#2900) asks
|
|
# what a BOUND repo's ledger knows. A helper nobody recorded, in a repo nobody
|
|
# bound, is invisible to both — which is how `.btn-primary` came to be defined
|
|
# four times, already diverged. This arm asks the one question only the
|
|
# developer's machine can answer, inside the repo, holding the code about to
|
|
# be written: no index, no storage, no server — it runs even on an install
|
|
# that has never configured Scribe.
|
|
#
|
|
# Definition-shaped patterns only. Grepping for bare occurrences would match
|
|
# every CALL site and drown the real finding — and a hint that is mostly noise
|
|
# is one people learn to skip, which is worse than none. ALL code, not a
|
|
# language shortlist (#2682): the same keyword family scribe_defs announces.
|
|
#
|
|
# $1 repo root, $2 repo-relative path of the file being written (excluded from
|
|
# the grep — it would always match itself on an Edit). Definitions on stdin.
|
|
# Prints one "> - `name` is already defined in N other file(s): …" per hit.
|
|
scribe_local_dups() {
|
|
local root="$1" rel="$2" kind name pat hits count label files
|
|
while IFS=$'\t' read -r kind name; do
|
|
[ -n "${name:-}" ] || continue
|
|
case "$kind" in
|
|
css) pat="^[[:space:]]*\.${name}[[:space:]]*[,{]" ;;
|
|
*) pat="(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+${name}[^A-Za-z0-9_]|func[[:space:]]*\([^)]*\)[[:space:]]*${name}[[:space:]]*\(|(const|let)[[:space:]]+${name}[[:space:]]*=" ;;
|
|
esac
|
|
# -I skips binaries; :(exclude) drops the file being written.
|
|
# `|| true` INSIDE the substitution, not `|| hits=""` outside it (#4042):
|
|
# under the hooks' `pipefail`, `head` exiting after four lines kills a
|
|
# git grep that is still writing, the pipeline reports SIGPIPE, and an
|
|
# outer fallback then wipes the four hits head already printed. The name
|
|
# most duplicated — the one this arm exists for — was the one it dropped.
|
|
hits=$(git -C "$root" grep -I -l -E -e "$pat" -- . ":(exclude)${rel}" 2>/dev/null | head -4 || true)
|
|
[ -n "$hits" ] || continue
|
|
count=$(printf '%s\n' "$hits" | grep -c . 2>/dev/null || echo 0)
|
|
label=$([ "$kind" = css ] && printf '.%s' "$name" || printf '%s' "$name")
|
|
files=$(printf '%s' "$hits" | tr '\n' ' ' | sed 's/ $//')
|
|
printf '> - `%s` is already defined in %s other file(s): %s\n' "$label" "$count" "$files"
|
|
done
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# The blind spot made visible (#2932). Both write-path hooks fail OPEN when the
|
|
# instance is slow or down — right for noise, wrong for silence: a session
|
|
# cannot tell "the ledger checked and found nothing" from "the ledger never
|
|
# answered", and a self-surfacing system cannot afford an invisible miss (the
|
|
# first write after a redeploy lost its derive line to a 4s cold start and
|
|
# nobody knew). So a failed call says so — ONCE per outage: the marker holds
|
|
# the time it last spoke; within ten minutes of that it stays quiet, and a
|
|
# successful call clears it so the next outage announces itself afresh.
|
|
# Unconfigured installs never reach this: no URL/token means no call was owed.
|
|
# Where every hook gets its endpoint and credential. Four lines, and each of
|
|
# the five hooks carried its own copy until #2278 — which is exactly the
|
|
# missing-sibling shape: the `${...}` guard below is a correctness detail a
|
|
# sixth hook would have forgotten, and nothing would have failed loudly.
|
|
#
|
|
# Sets `url` and `token` as globals rather than echoing them: a token must not
|
|
# pass through a subshell's output, where it could land in a log or an `xtrace`
|
|
# line. Returns 0 only when both are usable, so a caller can either bail
|
|
# (`scribe_config || exit 0`) or carry on degraded — the session-context hook
|
|
# still owes its static floor when Scribe is unconfigured.
|
|
# Declared here, not just assigned inside the function: `scribe_defs.sh` owns
|
|
# these two names, and a sourcing hook should have them defined the moment it
|
|
# sources — before any code path that might reference them. It also lets
|
|
# the linter see the assignment, which it cannot follow into a function in
|
|
# another file without -x (SC2154).
|
|
url=""
|
|
token=""
|
|
|
|
scribe_config() {
|
|
url=${SCRIBE_URL:-${CLAUDE_PLUGIN_OPTION_API_ENDPOINT:-}}
|
|
token=${SCRIBE_TOKEN:-${CLAUDE_PLUGIN_OPTION_API_TOKEN:-}}
|
|
# An unexpanded `${...}` placeholder arriving as a literal would be sent as a
|
|
# garbage Bearer token and 401. Treat it as unset.
|
|
case "$url" in *'${'*) url="" ;; esac
|
|
case "$token" in *'${'*) token="" ;; esac
|
|
[ -n "$url" ] && [ -n "$token" ]
|
|
}
|
|
|
|
_SCRIBE_UNREACHED_QUIET=600
|
|
|
|
scribe_unreached() {
|
|
local marker="$1/$2.unreached" now last
|
|
now=$(date +%s 2>/dev/null) || now=0
|
|
if [ -f "$marker" ]; then
|
|
last=$(cat "$marker" 2>/dev/null) || last=0
|
|
case "$last" in ''|*[!0-9]*) last=0 ;; esac
|
|
[ $((now - last)) -lt "$_SCRIBE_UNREACHED_QUIET" ] && return 0
|
|
fi
|
|
printf '%s' "$now" > "$marker" 2>/dev/null || true
|
|
printf '> Scribe did not answer the prior-art check for `%s` within %ss — this write went UNCHECKED against the record and the shape ledger (the local by-name arm, if it spoke above, needed no server). If the name matters, check it yourself: `search` for the concept, `list_shapes(project_id, path=…)` for the ledger. Said once per outage; if it keeps happening the instance is slow or down.' "$4" "$3"
|
|
}
|
|
|
|
scribe_reached() {
|
|
rm -f "$1/$2.unreached" 2>/dev/null || true
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# THE RULE EXCLUSION LEDGER, and how entries in it AGE (#3751).
|
|
#
|
|
# Two hooks write this file and two read it, which is the whole reason the
|
|
# parsing lives here: `scribe_prior_art.sh` and `scribe_tool_rules.sh` share
|
|
# one ledger so a rule named by one arm is not re-offered by the other, and a
|
|
# format only one of them understood would break that on the first read.
|
|
#
|
|
# WHAT PROBLEM AGEING SOLVES. #3749 clears the ledger when an EVENT destroys
|
|
# context — a compaction or a /clear. This is the case with no event at all: a
|
|
# long session where the rule was named two hundred turns ago and has simply
|
|
# fallen out of attention. It is #3702's argument at the tier level (present in
|
|
# context and salient at the moment are different properties) applied to time
|
|
# instead of to tier.
|
|
#
|
|
# PER-ENTRY TIMESTAMPS, NOT A FILE MTIME. Clearing the whole ledger when the
|
|
# file is old is one line of shell and wrong in exactly the session that needs
|
|
# it: a single recent write keeps every stale id alive, and the ids that go
|
|
# stale first are the ones from the rules that fire most.
|
|
#
|
|
# WALL TIME, NOT A TURN COUNT, and the trade is real rather than dismissed. A
|
|
# turn count is a truer model of salience — an idle session does not forget —
|
|
# but a hook has no turn number without keeping its own counter, which is a
|
|
# second piece of session state to write, read, clear on compaction and get
|
|
# wrong. Wall time is available from `date` and costs nothing. The failure mode
|
|
# it accepts is a session left idle over lunch treating its rules as forgotten,
|
|
# which produces one extra full line per rule and no other harm.
|
|
_SCRIBE_RULE_TTL=2700
|
|
|
|
# 45 MINUTES, and the reasoning rather than the number (rule 32).
|
|
#
|
|
# There is no data on this yet, so it is a judgement made to be revised — the
|
|
# telemetry that would settle it is the one #3807 just built, and a reading of
|
|
# how often an aged-out rule gets PULLED after it returns is what should move
|
|
# this.
|
|
#
|
|
# Too short and the exclusion stops existing and the repetition it prevents
|
|
# comes back. Too long and it never fires at all in a session short enough to
|
|
# matter. 45 minutes is about one working stretch on a single task: long enough
|
|
# that a rule does not re-announce itself while you are still doing the thing
|
|
# it governs, short enough that a multi-hour session gets a genuine refresh
|
|
# rather than one 9am mention.
|
|
#
|
|
# Being wrong on the short side is now the cheaper error, which is why this
|
|
# leans short. Since #3750 an excluded rule is REFERENCED rather than withheld,
|
|
# so the ledger is no longer the only thing standing between a session and a
|
|
# rule it has forgotten — an expired entry costs one full line instead of one
|
|
# short one, and the exclusion re-arms the moment it is spent.
|
|
|
|
# Live ids from a ledger, comma-joined for `exclude_rule_ids`. Empty output for
|
|
# a missing, empty or fully-aged file — the callers already treat "" as "send
|
|
# no exclusions".
|
|
#
|
|
# THE LAST ENTRY FOR AN ID WINS, and this is what stops a rule ping-ponging.
|
|
# The file is append-only, so a rule that ages out, gets surfaced fresh and is
|
|
# appended again has TWO lines. Reading the first would leave it permanently
|
|
# expired and it would re-announce itself on every single call from then on —
|
|
# the loudest possible failure, from the mechanism meant to quieten things.
|
|
# Appends are chronological, so the last line for an id is its most recent.
|
|
#
|
|
# A BARE ID — no tab, no timestamp — IS LIVE. That is the pre-#3751 format, and
|
|
# a session in flight when this ships has a ledger full of them. Treating
|
|
# unknown as expired would make every one of those sessions re-announce every
|
|
# rule it had already been told, all at once, which is precisely the noise this
|
|
# exists to prevent. Unknown means "not measured" and never "old" — the same
|
|
# null discipline the retrieval_logs columns use. Those entries simply never
|
|
# age, which is bounded: the session ends.
|
|
scribe_rules_live() {
|
|
local f="$1" now
|
|
[ -n "$f" ] && [ -f "$f" ] || return 0
|
|
now=$(date +%s 2>/dev/null) || now=0
|
|
awk -F'\t' -v now="$now" -v ttl="$_SCRIBE_RULE_TTL" '
|
|
{
|
|
id = $1
|
|
gsub(/[^0-9]/, "", id)
|
|
if (id == "") next
|
|
if (!(id in seen)) { seen[id] = 1; seq[++n] = id }
|
|
stamp[id] = ($2 ~ /^[0-9]+$/) ? $2 : ""
|
|
}
|
|
END {
|
|
out = ""
|
|
for (i = 1; i <= n; i++) {
|
|
id = seq[i]
|
|
if (stamp[id] != "" && now > 0 && (now - stamp[id]) > ttl) continue
|
|
out = out (out == "" ? "" : ",") id
|
|
}
|
|
print out
|
|
}
|
|
' "$f" 2>/dev/null || true
|
|
}
|
|
|
|
# Append surfaced ids, stamped. Reads ids on stdin, one per line — the shape
|
|
# `scribe_json_list "$flat" '.rule_ids'` already produces at both call sites.
|
|
scribe_rules_append() {
|
|
local f="$1" now
|
|
[ -n "$f" ] || return 0
|
|
now=$(date +%s 2>/dev/null) || now=0
|
|
awk -v ts="$now" 'NF { print $1 "\t" ts }' >> "$f" 2>/dev/null || true
|
|
}
|
|
|
|
# The OPENED ledger's contribution to a rule arm's query string (#4100).
|
|
#
|
|
# TWO LEDGERS, BECAUSE THEY RECORD TWO DIFFERENT FACTS. `.rules.ids` holds
|
|
# every id an arm has NAMED; `.opened.ids` holds the ids the session actually
|
|
# read, written by scribe_record_opened.sh from the `get_rule` call itself.
|
|
# Named is not read: the injected line is a teaser, and one skimmed past
|
|
# leaves nothing behind — least of all across a compaction. Sending both lets
|
|
# the server tell a reader who opened a rule from one who was only shown it,
|
|
# instead of telling them both the same untrue thing.
|
|
#
|
|
# Same reader as the naming ledger on purpose, so ageing, the last-entry-wins
|
|
# rule and the bare-id format are defined once and cannot drift apart.
|
|
scribe_held_query() {
|
|
local ids
|
|
ids=$(scribe_rules_live "$1")
|
|
[ -n "$ids" ] && printf '&held_rule_ids=%s' "$ids"
|
|
return 0
|
|
}
|
|
|
|
# Drop EVERY per-session ledger, matched by convention rather than listed (#4101).
|
|
#
|
|
# A LIST IS THE BUG. Until now the compact/clear branch named its files one at
|
|
# a time, and it named two of the five: `.rules.ids` and `.opened.ids` were
|
|
# cleared while `.ids` (notes), `.sync.ids` (shape signals) and `.derive.ids`
|
|
# survived. So milestone 386's defect — "a compaction destroys the context but
|
|
# not the ledger, so the most applicable records become permanently
|
|
# unreachable mid-session" — was fixed for rules and left standing on the note
|
|
# and snippet surfaces, which are the ones that fire most often.
|
|
#
|
|
# Nobody decided that. The list was written when rules were the only ledger
|
|
# that mattered and was never revisited when the others arrived, which is what
|
|
# a hand-maintained list of "things to remember to clean up" does. Adding three
|
|
# more `rm` lines would rebuild the same trap for the sixth ledger.
|
|
#
|
|
# So the rule is the NAME: a per-session ledger is `<sid>[.<kind>].ids`, and
|
|
# everything matching that goes. A new ledger following the convention is
|
|
# covered the day it is written, by nobody. One that does not follow it is a
|
|
# deliberate exception and has to say so.
|
|
#
|
|
# Scoped to `.ids` rather than `<sid>.*` so a marker that is REWRITTEN on
|
|
# compact rather than discarded can still live in these directories without
|
|
# being swept away by a glob that was never told about it. `<sid>.unreached`
|
|
# is the live example: it records that the instance could not be reached, not
|
|
# what the session holds, and #2932 needs it to outlive a compaction.
|
|
#
|
|
# TWO DIRECTORIES, WHICH IS ITS OWN LESSON. The first cut of this swept only
|
|
# `scribe-priorart` — every ledger named in the hooks was there, so the list
|
|
# looked complete. `scribe_autoinject.sh` keeps its note ledger in
|
|
# `scribe-autoinject`, so the arm that fires most (598 calls in five days) was
|
|
# the one the clear could not reach, and the fix read as finished. Hence the
|
|
# roster here rather than a path at the call site: one place to add to, and
|
|
# the tests read THIS string rather than a copy of it.
|
|
SCRIBE_LEDGER_DIRS="scribe-priorart scribe-autoinject"
|
|
|
|
scribe_clear_session_ledgers() {
|
|
local sid="$1" dir
|
|
[ -n "$sid" ] || return 0
|
|
for dir in $SCRIBE_LEDGER_DIRS; do
|
|
rm -f "${TMPDIR:-/tmp}/$dir/$sid"*.ids 2>/dev/null || true
|
|
done
|
|
return 0
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# WHICH PROJECT IS THIS DIRECTORY'S? (#4085)
|
|
#
|
|
# Every hook here scopes its request to a project, and until this existed all
|
|
# six did it the same single way: `git remote get-url origin`, resolved
|
|
# server-side through the repo bindings. That works well inside a bound repo
|
|
# and not at all outside one — a session in a plain directory got no project
|
|
# scope from ANY hook, silently, because the one key the whole chain turns on
|
|
# only exists in a git repo.
|
|
#
|
|
# The second key is a `.scribe` file in the directory (or above it, the way
|
|
# git finds its root), naming the project the work belongs to. It is a
|
|
# POINTER, not a copy: an id and enough to check the id means what it says.
|
|
# Nothing Scribe should be holding goes in it.
|
|
#
|
|
# {"instance": "https://scribe.example.com", "project_id": 2,
|
|
# "project": "FabledScribe"}
|
|
#
|
|
# instance WHICH Scribe the id belongs to, and the reason this file is
|
|
# not just a number. A project id means nothing on its own: id 2
|
|
# is a different project on every instance, so a marker that
|
|
# travels — a copied directory, a shared machine, a repo someone
|
|
# else clones — would silently scope a session to the wrong
|
|
# project. Compared HOST-ONLY against the configured endpoint, so
|
|
# http/https and a trailing slash don't cause a false mismatch.
|
|
# A mismatch drops the id: no project beats the wrong project.
|
|
# project_id the pointer itself. Required.
|
|
# project a human label. NOTHING READS IT. It is there so the file
|
|
# answers "what is this?" when opened, and so a stale one is
|
|
# visible rather than inert.
|
|
#
|
|
# A bare integer is also accepted (`echo 2 > .scribe`), because it is what a
|
|
# person writes by hand and it parses as JSON already. It skips the instance
|
|
# check by having nothing to check — deliberate, and the reason the written
|
|
# form carries `instance`.
|
|
#
|
|
# THE MARKER WINS over a git remote. Someone put the file there on purpose;
|
|
# a remote is just where the code happens to be pushed. That also makes the
|
|
# marker the way to override a binding for one directory.
|
|
|
|
# Nearest `.scribe` at or above DIR. Walks up to the filesystem root, capped so
|
|
# a pathological path cannot spin.
|
|
scribe_marker_file() {
|
|
local dir="$1" depth=0
|
|
[ -n "$dir" ] || return 0
|
|
while [ "$depth" -lt 40 ]; do
|
|
[ -f "$dir/.scribe" ] && { printf '%s' "$dir/.scribe"; return 0; }
|
|
case "$dir" in ""|"/") return 0 ;; esac
|
|
dir=$(dirname -- "$dir" 2>/dev/null) || return 0
|
|
depth=$((depth + 1))
|
|
done
|
|
return 0
|
|
}
|
|
|
|
# Host of a URL, lowercased, port kept. "" for empty input.
|
|
scribe_url_host() {
|
|
printf '%s' "${1:-}" \
|
|
| sed -e 's#^[A-Za-z][A-Za-z0-9+.-]*://##' -e 's#^[^/@]*@##' -e 's#[/?].*$##' \
|
|
| tr 'A-Z' 'a-z'
|
|
}
|
|
|
|
# Read a marker file: prints "ID<TAB>REASON", at most one of them non-empty.
|
|
#
|
|
# "7\t" use project 7
|
|
# "\tnames no …" a file is there and deliberately NOT used; say why
|
|
# "\t" no marker file at all — the ordinary case, say nothing
|
|
#
|
|
# One line rather than an id plus a global, because every caller reads this
|
|
# through `$( )` and a global set inside a command substitution dies with the
|
|
# subshell. The caller would then read an unset variable, which under the
|
|
# `set -u` these hooks all run with aborts the hook and costs the whole
|
|
# session's context — a failure far larger than the message it was fetching.
|
|
scribe_marker_read() {
|
|
local f="$1" flat id inst want
|
|
[ -n "$f" ] && [ -f "$f" ] || { printf '\t'; return 0; }
|
|
flat=$(scribe_json_flat < "$f") || flat=""
|
|
# Both accepted forms, read off one parse: the object's `project_id`, or —
|
|
# for the bare integer someone writes by hand — the root scalar, whose path
|
|
# is the empty string because it sits under no key.
|
|
id=$(scribe_json_pick "$flat" '.project_id')
|
|
[ -n "$id" ] || id=$(scribe_json_pick "$flat" '')
|
|
case "$id" in ''|*[!0-9]*) id="" ;; esac
|
|
if [ -z "$id" ] || [ "$id" = "0" ]; then
|
|
printf '\tnames no project_id'
|
|
return 0
|
|
fi
|
|
inst=$(scribe_json_pick "$flat" '.instance')
|
|
if [ -n "$inst" ]; then
|
|
want=$(scribe_url_host "${url:-}")
|
|
inst=$(scribe_url_host "$inst")
|
|
if [ -n "$want" ] && [ "$inst" != "$want" ]; then
|
|
printf '\tpoints at %s, but this session is configured for %s' "$inst" "$want"
|
|
return 0
|
|
fi
|
|
fi
|
|
printf '%s\t' "$id"
|
|
}
|
|
|
|
# Just the id a marker names, or "" — the half scribe_scope_query needs.
|
|
scribe_marker_project() {
|
|
scribe_marker_read "$1" | cut -f1
|
|
}
|
|
|
|
# The query args identifying DIR's project — `project_id=N` or `repo=<enc>` —
|
|
# with NO leading `?` or `&`, so each caller keeps its own separator. Empty
|
|
# when neither key is available. Requires `url` to be set (scribe_config) for
|
|
# the marker's instance check; without it a marker is still honoured, since an
|
|
# unconfigured hook is not going to send the request anyway.
|
|
scribe_scope_query() {
|
|
local dir="$1" id repo enc
|
|
id=$(scribe_marker_project "$(scribe_marker_file "$dir")")
|
|
if [ -n "$id" ]; then
|
|
printf 'project_id=%s' "$id"
|
|
return 0
|
|
fi
|
|
repo=$(git -C "$dir" remote get-url origin 2>/dev/null || true)
|
|
[ -n "$repo" ] || return 0
|
|
enc=$(printf '%s' "$repo" | scribe_urlenc) || enc=""
|
|
[ -n "$enc" ] && printf 'repo=%s' "$enc"
|
|
}
|