bvandeusenandClaude Opus 5
84476d7ecf
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m18s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Canceled after 8s
The definition extractor is line-oriented and knows nothing about what a
line is INSIDE. A docstring that wraps onto a line starting with a keyword
announces a definition: `AND` reached a live session as a divergence prompt
asking it to justify a symbol that does not exist, and `is` reached it as a
repo-wide duplicate of four files that define nothing of the kind.
Measured, not assumed: running the extractor over every scannable file with
and without the scan differs by twenty-two phantoms. Two of them — `with`
and `nobody`, both out of the module docstring in check_dangling_styles.py —
are persisted `code_shapes` rows that have been judged. Those need no
migration: sync_shapes marks a row it no longer extracts as vanished.
`ast` would be the honest tool for .py and is not what this uses, because
the extractor is mirrored rule for rule by an awk program in the hook, awk
cannot parse Python, and a fix only one of the pair can run is the drift the
mirror exists to prevent. Both sides now run the same left-to-right scan and
blank comment and string spans to their own newlines before any matcher sees
a line. Three things the scan has to get right, each of which cost real
definitions while it was being written:
- a string that HOLDS a marker is not a marker. `"red /* "` in
test_design_stylesheet.py and a triple quote inside a single-quoted
regex in plugin_context.py each ate every definition below them.
- `#` is a colour in CSS and a comment in Python, and the extractor is
handed no path. An alphanumeric straight after it settles it.
- an unterminated opener blanks NOTHING. The scan rewinds past it and
continues, so a stray marker costs one span rather than the rest of the
file.
The comment claiming the two extractors agree has been the only thing
holding them together, and a comment cannot fail. The mirror test now RUNS
the hook's awk over the same vectors: with the old program it reports the
phantoms, which is what a guard that can fail looks like. Across all 631
scannable files in this repo the two now agree line for line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
1176 lines
55 KiB
Bash
1176 lines
55 KiB
Bash
#!/usr/bin/env bash
|
|
# shellcheck shell=bash
|
|
# Scribe plugin — the pieces the hooks share (#2901, #2278).
|
|
#
|
|
# scribe_prior_art.sh fires BEFORE a Write/Edit tool call; scribe_after_write.sh
|
|
# fires AFTER a Bash tool call and diffs the working tree, so code written by
|
|
# sed/heredocs/scripts gets the same prior-art and ledger checks. Both need the
|
|
# same three things, kept here so they cannot drift apart:
|
|
#
|
|
# scribe_skip_path PATH formats that hold prose or data, not shapes
|
|
# scribe_skip_prompt TEXT the turn was written by the CLIENT, not the
|
|
# operator — skip it entirely (#4200)
|
|
# scribe_defs stdin code → "kind<TAB>name" per definition
|
|
# scribe_local_dups ROOT REL "kind<TAB>name" lines on stdin → the by-name
|
|
# local-duplicate lines (ARM 1, #2280)
|
|
# scribe_unreached STATE SID SECS REL the "Scribe didn't answer" line, once
|
|
# per outage (#2932) — or nothing, if said lately
|
|
# scribe_reached STATE SID the server answered: the next outage speaks again
|
|
# scribe_config sets `url` + `token` from the env, returns 0
|
|
# only if BOTH are usable (#2278)
|
|
# scribe_rules_live FILE live rule ids from the exclusion ledger,
|
|
# comma-joined; entries age out (#3751)
|
|
# scribe_rules_append FILE stdin ids -> the ledger, timestamped
|
|
# scribe_json_flat stdin JSON -> IDX<TAB>PATH<TAB>VALUE lines,
|
|
# queried with scribe_json_pick / _list / _len.
|
|
# scribe_json_out writes the hook envelope,
|
|
# scribe_urlenc percent-encodes. No jq (#4107).
|
|
# scribe_scope_query DIR `project_id=N` or `repo=<enc>` for DIR — the
|
|
# project-scope key EVERY hook sends (#4085).
|
|
# Helpers: scribe_marker_file, scribe_url_host,
|
|
# scribe_marker_read (id<TAB>why-not),
|
|
# scribe_marker_project (the id alone)
|
|
#
|
|
# Sourced, not executed: `. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"`.
|
|
|
|
# Skip formats that hold prose or data rather than reusable code. Purely to
|
|
# avoid a pointless round-trip — the server would return nothing for these
|
|
# anyway. Config formats are NOT skipped: a CI workflow or a compose file is
|
|
# often exactly the thing worth reusing.
|
|
scribe_skip_path() {
|
|
case "$1" in
|
|
*.md|*.mdx|*.txt|*.rst|*.json|*.lock|*.log|*.csv|*.tsv|*.svg|*.png|*.jpg|*.jpeg|*.gif|*.ico|*.pdf)
|
|
return 0 ;;
|
|
esac
|
|
return 1
|
|
}
|
|
|
|
# Skip a turn the CLIENT wrote rather than the operator (#4200). Claude Code
|
|
# submits several kinds of machine-written text through UserPromptSubmit — a
|
|
# task notification, the echo of a slash command, the caveat banner a local
|
|
# command prints — and they reach `.prompt` indistinguishable from typed words
|
|
# unless something looks.
|
|
#
|
|
# THIS IS A TELEMETRY FIX FIRST AND A NOISE FIX SECOND, which is the part
|
|
# worth keeping straight: retrieving against a notification usually returns
|
|
# nothing, so the visible cost looks like one wasted embedding. The real cost
|
|
# is that the call is LOGGED. It inflates the denominator of every
|
|
# prompt-boundary surface, so delivery rate reads low for a reason that has
|
|
# nothing to do with retrieval; and the refusal lands in `near_misses`, where
|
|
# a later tuning decision reads it as unmet demand. Measured on #3898: 15 of
|
|
# the top 20 preference near-misses were `<task-notification>` blocks, ALL
|
|
# matching one record, ALL within thousandths of the floor. A floor lowered to
|
|
# serve that apparent demand would inject that record into every notification
|
|
# — the instrument arguing for the wrong fix, which is #379's lesson again.
|
|
#
|
|
# A PREFIX TEST, NOT A SUBSTRING ONE, and that distinction is the whole safety
|
|
# argument. A real prompt may well CONTAIN one of these tags — an operator
|
|
# pasting a transcript, or a `<system-reminder>` appended after typed words —
|
|
# and must still be retrieved against. Nothing an operator types BEGINS with a
|
|
# client envelope.
|
|
#
|
|
# Leading whitespace goes first: the tag is what identifies the turn, and one
|
|
# stray newline ahead of it would otherwise walk past the entire filter.
|
|
#
|
|
# NOT LISTED, ON PURPOSE: the compaction-resume injection. It is machine
|
|
# written too, but it is a summary of real work rather than plumbing, and a
|
|
# resumed session is exactly where recalling a rule earns its keep.
|
|
scribe_skip_prompt() {
|
|
_scribe_prompt=${1#"${1%%[![:space:]]*}"}
|
|
case "$_scribe_prompt" in
|
|
"<task-notification>"*|\
|
|
"<command-name>"*|"<command-message>"*|"<command-args>"*|\
|
|
"<local-command-stdout>"*|"<local-command-stderr>"*|"<local-command-caveat>"*)
|
|
return 0 ;;
|
|
esac
|
|
return 1
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# JSON AND URL-ENCODING, WITHOUT jq (#4107).
|
|
#
|
|
# Until this section every hook opened `command -v jq >/dev/null 2>&1 || exit 0`
|
|
# and a machine without jq got no session context, no rules, no prior art and
|
|
# no process sync — silently, because `exit 0` is indistinguishable from "ran
|
|
# fine, nothing to say". That is the plugin pushing its own packaging problem
|
|
# onto whoever installs it, and it is the same silence #4085 existed to remove.
|
|
# jq is absent by default on macOS, on the Debian/Ubuntu slim images, on Alpine
|
|
# and in most CI containers. awk, sed, tr and od are POSIX; every one of them
|
|
# is already required by code above this line.
|
|
#
|
|
# scribe_json.awk does the parsing and carries the format. These are the four
|
|
# jobs the hooks actually had jq for:
|
|
#
|
|
# READ scribe_json_flat / _flat_lines turn stdin into `IDX<TAB>PATH<TAB>
|
|
# VALUE` lines ONCE, and scribe_json_pick / _list / _len query that
|
|
# text. Parse once, query many: a hook reads four fields off one
|
|
# event, and re-parsing per field is four passes over a payload that
|
|
# may be an entire source file.
|
|
# WRITE scribe_json_out emits the hookSpecificOutput envelope, which is the
|
|
# one piece of JSON these hooks produce.
|
|
# ENCODE scribe_urlenc replaces `jq -sRr '@uri'`, byte-exact for UTF-8.
|
|
# DECODE scribe_json_unescape turns a raw JSON string body into text.
|
|
|
|
_scribe_self=${BASH_SOURCE[0]:-$0}
|
|
# shellcheck disable=SC1007 # `CDPATH= cd` scopes one variable to one command
|
|
SCRIBE_HOOK_DIR=$(CDPATH= cd -- "$(dirname -- "$_scribe_self")" 2>/dev/null && pwd) \
|
|
|| SCRIBE_HOOK_DIR=$(dirname -- "$_scribe_self")
|
|
|
|
# One JSON value on stdin → flat lines. Empty output means it did not parse.
|
|
scribe_json_flat() {
|
|
awk -f "$SCRIBE_HOOK_DIR/scribe_json.awk" 2>/dev/null || true
|
|
}
|
|
|
|
# JSONL on stdin → flat lines, IDX counting the records that PARSED. A line
|
|
# that does not parse is dropped and the rest are still read.
|
|
scribe_json_flat_lines() {
|
|
awk -v mode=lines -f "$SCRIBE_HOOK_DIR/scribe_json.awk" 2>/dev/null || true
|
|
}
|
|
|
|
# $1 flat text, $2 exact path → the decoded scalar, or "" if absent.
|
|
#
|
|
# `null` READS AS ABSENT, matching the `// empty` every call site used to carry.
|
|
# A field the server left null and one it never sent mean the same thing to
|
|
# every caller here, and the shells downstream test `[ -n "$x" ]`.
|
|
scribe_json_pick() {
|
|
printf '%s\n' "${1:-}" \
|
|
| awk -F'\t' -v p="$2" '$2 == p { if ($3 != "null") print $3; exit }' 2>/dev/null \
|
|
| scribe_json_unescape
|
|
}
|
|
|
|
# $1 flat text, $2 array path → each element decoded, one per line. Elements
|
|
# are ids and keys here, so a value containing a newline is not a case that
|
|
# arises; one would split into two lines.
|
|
scribe_json_list() {
|
|
printf '%s\n' "${1:-}" \
|
|
| awk -F'\t' -v p="$2" '
|
|
BEGIN { plen = length(p) }
|
|
substr($2, 1, plen) == p {
|
|
rest = substr($2, plen + 1)
|
|
if (rest ~ /^\[[0-9]+\]$/ && $3 != "null") print $3
|
|
}' 2>/dev/null \
|
|
| scribe_json_unescape
|
|
}
|
|
|
|
# $1 flat text, $2 array path, $3 array path to SUBTRACT → the elements of $2
|
|
# that are not in $3, in order. The reuse and sync channels are two classes of
|
|
# one answer and each has its own session ledger, so the ids that go in the
|
|
# reuse file are precisely `note_ids` minus `sync_note_ids` — jq wrote that as
|
|
# `(.note_ids // []) - (.sync_note_ids // [])`.
|
|
scribe_json_list_minus() {
|
|
local keep
|
|
keep=$(scribe_json_list "$1" "$3")
|
|
scribe_json_list "$1" "$2" \
|
|
| awk -v drop="$keep" '
|
|
BEGIN { n = split(drop, a, "\n"); for (i = 1; i <= n; i++) if (a[i] != "") s[a[i]] = 1 }
|
|
NF && !($0 in s)' 2>/dev/null || true
|
|
}
|
|
|
|
# $1 flat text, $2 array path → its length. "" when the path is not an array,
|
|
# which is NOT the same as 0 — see the `[#]` note in scribe_json.awk.
|
|
scribe_json_len() {
|
|
printf '%s\n' "${1:-}" \
|
|
| awk -F'\t' -v p="$2" '$2 == p "[#]" { print $3; exit }' 2>/dev/null
|
|
}
|
|
|
|
# Raw JSON string bodies on stdin → text. One line in, one value out.
|
|
scribe_json_unescape() {
|
|
awk '
|
|
function hex4(h, i, c, d, v) {
|
|
v = 0
|
|
for (i = 1; i <= 4; i++) {
|
|
c = tolower(substr(h, i, 1))
|
|
d = index("0123456789abcdef", c) - 1
|
|
if (d < 0) return -1
|
|
v = v * 16 + d
|
|
}
|
|
return v
|
|
}
|
|
{
|
|
v = $0
|
|
if (index(v, "\\") == 0) { print v; next }
|
|
o = ""
|
|
i = 1
|
|
L = length(v)
|
|
while (i <= L) {
|
|
c = substr(v, i, 1)
|
|
if (c != "\\") { o = o c; i++; continue }
|
|
d = substr(v, i + 1, 1)
|
|
i += 2
|
|
if (d == "n") o = o "\n"
|
|
else if (d == "t") o = o "\t"
|
|
else if (d == "r") o = o "\r"
|
|
else if (d == "b") o = o sprintf("%c", 8)
|
|
else if (d == "f") o = o sprintf("%c", 12)
|
|
else if (d == "u") {
|
|
hi = hex4(substr(v, i, 4))
|
|
if (hi < 0) { o = o "\\u"; continue }
|
|
i += 4
|
|
# A surrogate PAIR is one character written as two escapes; decoding
|
|
# the halves separately yields two replacement characters instead.
|
|
if (hi >= 55296 && hi <= 56319 && substr(v, i, 2) == "\\u") {
|
|
lo = hex4(substr(v, i + 2, 4))
|
|
if (lo >= 56320 && lo <= 57343) {
|
|
hi = 65536 + (hi - 55296) * 1024 + (lo - 56320)
|
|
i += 6
|
|
}
|
|
}
|
|
o = o sprintf("%c", hi)
|
|
}
|
|
else o = o d
|
|
}
|
|
print o
|
|
}' 2>/dev/null || true
|
|
}
|
|
|
|
# Text on stdin → a JSON string BODY (escaped, no surrounding quotes, one line).
|
|
#
|
|
# split/join rather than gsub: gsub reads `\` and `&` in its replacement as
|
|
# metacharacters, so emitting a literal backslash through it takes a
|
|
# quadruple-escape whose meaning varies between awks. Plain concatenation has
|
|
# no such reading, and getting this wrong corrupts every injected context that
|
|
# happens to contain a Windows path or a regex.
|
|
scribe_json_escape() {
|
|
awk '
|
|
function rep(str, sep, with, parts, cnt, i, o) {
|
|
cnt = split(str, parts, sep)
|
|
o = parts[1]
|
|
for (i = 2; i <= cnt; i++) o = o with parts[i]
|
|
return o
|
|
}
|
|
{
|
|
line = $0
|
|
line = rep(line, "\\\\", "\\\\")
|
|
line = rep(line, "\"", "\\\"")
|
|
line = rep(line, "\t", "\\t")
|
|
line = rep(line, "\r", "\\r")
|
|
# EVERY OTHER CONTROL CHARACTER TOO, or the envelope is not JSON. A raw
|
|
# byte below 0x20 inside a string is invalid, and Claude Code discards
|
|
# the whole hook output rather than the one field — so a single stray
|
|
# character anywhere in an injected context costs the entire injection.
|
|
# jq escaped these; a writer that knew only about tab and carriage
|
|
# return would have regressed quietly, on the rare input nobody thinks
|
|
# to test by hand.
|
|
for (c = 1; c < 32; c++) {
|
|
if (c == 9 || c == 10 || c == 13) continue
|
|
ctl = sprintf("%c", c)
|
|
if (index(line, ctl)) line = rep(line, ctl, sprintf("\\u%04x", c))
|
|
}
|
|
if (NR > 1) printf "\\n"
|
|
printf "%s", line
|
|
}' 2>/dev/null || true
|
|
}
|
|
|
|
# The one JSON document these hooks WRITE: $1 hook event name, $2 the context.
|
|
# Every hook emitted this through `jq -n --arg c`, five copies of one shape.
|
|
scribe_json_out() {
|
|
local esc
|
|
esc=$(printf '%s' "$2" | scribe_json_escape) || return 0
|
|
printf '{"hookSpecificOutput":{"hookEventName":"%s","additionalContext":"%s"}}\n' \
|
|
"$1" "$esc"
|
|
}
|
|
|
|
# stdin → percent-encoded, replacing `jq -sRr '@uri'`.
|
|
#
|
|
# THROUGH `od`, NOT A CHARACTER LOOP IN awk, and the reason is the bug this
|
|
# would otherwise reintroduce. awk's idea of a "character" follows the locale:
|
|
# in a UTF-8 locale gawk reads é as one character and mawk as two bytes, so any
|
|
# encoder built on substr() produces a different URL on the same input
|
|
# depending on which awk is installed. Percent-encoding is defined on BYTES. od
|
|
# -tu1 gives bytes, on every platform, and the only characters passed through
|
|
# unencoded are the unreserved ASCII set — which every awk agrees about.
|
|
#
|
|
# Slightly more aggressive than jq's `@uri`, which leaves `!~*'()` alone. Those
|
|
# are legal either way and the server decodes both identically.
|
|
scribe_urlenc() {
|
|
od -An -v -tu1 2>/dev/null \
|
|
| awk '{
|
|
for (i = 1; i <= NF; i++) {
|
|
b = $i + 0
|
|
if ((b >= 48 && b <= 57) || (b >= 65 && b <= 90) ||
|
|
(b >= 97 && b <= 122) || b == 45 || b == 46 || b == 95 || b == 126)
|
|
printf "%s", sprintf("%c", b)
|
|
else
|
|
printf "%%%02X", b
|
|
}
|
|
}' 2>/dev/null || true
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# kind<TAB>name for each thing a piece of code DEFINES, in source order. One
|
|
# program, two consumers: the local duplicate arm (every definition in the
|
|
# payload) and the ledger feed (#2791, below: the definitions being written,
|
|
# or the one enclosing an Edit). Rule-for-rule mirrored by the server's
|
|
# services/coverage.py extract_shapes — ledger rows are keyed by what THAT
|
|
# sees, so the two must agree on what counts as a definition.
|
|
#
|
|
# THE WHOLE INPUT IS BUFFERED (#4222) so the span scan below can look ahead.
|
|
# The matchers are line-oriented and know nothing about what a line is INSIDE:
|
|
# a wrapped docstring beginning "class AND the …" announces a shape called
|
|
# `AND`, which reaches the session mid-edit as a divergence prompt about a
|
|
# symbol that does not exist. blank_spans() replaces every comment and string
|
|
# span with its own newlines before a matcher sees a line — the same scan, in
|
|
# the same order, as coverage.py::_blank_spans. Change one, change both.
|
|
scribe_defs() {
|
|
awk '
|
|
BEGIN {
|
|
SQ = sprintf("%c", 39)
|
|
SQ3 = SQ SQ SQ
|
|
DQ = "\""
|
|
DQ3 = DQ DQ DQ
|
|
# The only characters that can begin a span, a line comment or a
|
|
# string. Everything between two of them is copied in one go rather
|
|
# than a character at a time.
|
|
MARKERS = "[" DQ SQ "/#]"
|
|
}
|
|
|
|
# Offset just past the one-line string opening at c, or c+1 when it does
|
|
# not close before the end of the line — so an apostrophe in prose costs
|
|
# one character rather than everything up to the next quote.
|
|
function string_end(L, c, q, i, n, ch) {
|
|
n = length(L); i = c + 1
|
|
while (i <= n) {
|
|
ch = substr(L, i, 1)
|
|
if (ch == "\\") { i = i + 2; continue }
|
|
if (ch == q) return i + 1
|
|
i++
|
|
}
|
|
return c + 1
|
|
}
|
|
|
|
# Does the "#" at c open a comment, or is it a CSS colour or id? An
|
|
# alphanumeric straight after it is #fff or #app; anything else is a
|
|
# comment in every language that has one.
|
|
function hash_comment(L, c) {
|
|
return substr(L, c + 1, 1) !~ /^[A-Za-z0-9]$/
|
|
}
|
|
|
|
# raw[1..n] -> msk[1..n] with comment and string spans emptied. Line
|
|
# COUNT is preserved and column positions are not; the matchers lstrip.
|
|
# ONLY CLOSED SPANS ARE BLANKED: an opener with no closer is rewound past
|
|
# and scanning resumes, so a stray marker costs one span rather than
|
|
# every definition below it.
|
|
function blank_spans(raw, n, msk,
|
|
i, c, L, len, state, closer, oplen, sl, sc, sprefix,
|
|
t3, t2, ch, e, k, rest, m) {
|
|
for (i = 1; i <= n; i++) msk[i] = ""
|
|
i = 1; c = 1; state = 0; closer = ""
|
|
while (1) {
|
|
while (i <= n) {
|
|
L = raw[i]; len = length(L)
|
|
if (c > len) { i++; c = 1; continue }
|
|
if (state) {
|
|
e = index(substr(L, c), closer)
|
|
if (e == 0) { i++; c = 1; continue }
|
|
c = c + e - 1 + length(closer)
|
|
state = 0; closer = ""
|
|
continue
|
|
}
|
|
rest = substr(L, c)
|
|
m = match(rest, MARKERS)
|
|
if (m == 0) { msk[i] = msk[i] rest; i++; c = 1; continue }
|
|
if (m > 1) {
|
|
msk[i] = msk[i] substr(rest, 1, m - 1)
|
|
c = c + m - 1
|
|
continue
|
|
}
|
|
t3 = substr(L, c, 3); t2 = substr(L, c, 2); ch = substr(L, c, 1)
|
|
if (t3 == DQ3 || t3 == SQ3) {
|
|
sl = i; sc = c; sprefix = msk[i]
|
|
state = 1; closer = t3; oplen = 3; c = c + 3
|
|
continue
|
|
}
|
|
if (t2 == "/*") {
|
|
sl = i; sc = c; sprefix = msk[i]
|
|
state = 1; closer = "*/"; oplen = 2; c = c + 2
|
|
continue
|
|
}
|
|
if (t2 == "//" || (ch == "#" && hash_comment(L, c))) {
|
|
# A line comment is COPIED, not blanked: its continuation lines
|
|
# carry their own marker, so none can read as a definition alone.
|
|
msk[i] = msk[i] substr(L, c)
|
|
i++; c = 1
|
|
continue
|
|
}
|
|
if (ch == DQ || ch == SQ) {
|
|
k = string_end(L, c, ch)
|
|
msk[i] = msk[i] substr(L, c, k - c)
|
|
c = k
|
|
continue
|
|
}
|
|
msk[i] = msk[i] ch
|
|
c++
|
|
}
|
|
if (!state) return
|
|
for (k = sl; k <= n; k++) msk[k] = ""
|
|
msk[sl] = sprefix
|
|
i = sl; c = sc + oplen; state = 0; closer = ""
|
|
}
|
|
}
|
|
|
|
function emit(line, t, rest) {
|
|
# CSS class definition: .name { or .name,
|
|
if (match(line, /^[[:space:]]*\.[A-Za-z][A-Za-z0-9_-]*[[:space:]]*[,{]/)) {
|
|
t = line; sub(/^[[:space:]]*\./, "", t); sub(/[[:space:]]*[,{].*$/, "", t)
|
|
if (t != "") print "css\t" t; return
|
|
}
|
|
sub(/^[[:space:]]+/, "", line)
|
|
# Strip leading declaration modifiers so the definition keyword is the
|
|
# first word regardless of language (export/pub/private/suspend/...).
|
|
sub(/^((pub(\([a-z]+\))?|export|default|private|internal|protected|public|static|suspend|async|open|sealed|data|abstract|final|inline|unsafe|extern|override)[[:space:]]+)*/, "", line)
|
|
# Go method with receiver: func (r *T) Name(
|
|
if (match(line, /^func[[:space:]]*\([^)]*\)[[:space:]]*[A-Za-z_]/)) {
|
|
t = line; sub(/^func[[:space:]]*\([^)]*\)[[:space:]]*/, "", t)
|
|
sub(/[^A-Za-z0-9_].*$/, "", t)
|
|
if (t != "") print "sym\t" t; return
|
|
}
|
|
# Keyword-announced definitions, functions and named types alike.
|
|
# Dunders are skipped: every class defines __init__, so "already defined
|
|
# in N other files" is guaranteed noise for them — and noise is what
|
|
# teaches sessions to skip the hint.
|
|
if (match(line, /^(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+[A-Za-z_$]/)) {
|
|
t = line; sub(/^[a-z]+[[:space:]]+/, "", t)
|
|
sub(/[^A-Za-z0-9_$].*$/, "", t)
|
|
# `type` defines only when something follows the name (= or {); an
|
|
# import specifier `type Foo,` is the same two words and defines
|
|
# nothing (mirror of coverage.py, #2904).
|
|
if (line ~ /^type[[:space:]]/) {
|
|
rest = line; sub(/^type[[:space:]]+[A-Za-z_$][A-Za-z0-9_$]*/, "", rest)
|
|
if (rest !~ /[={]/) return
|
|
}
|
|
if (t != "" && t !~ /^__.*__$/) print "sym\t" t; return
|
|
}
|
|
# Arrow/expression assignment: const name = (…) / let name = async (
|
|
if (match(line, /^(const|let)[[:space:]]+[A-Za-z_$][A-Za-z0-9_$]*[[:space:]]*=[[:space:]]*(async[[:space:]]*)?[(<]/)) {
|
|
t = line; sub(/^(const|let)[[:space:]]+/, "", t)
|
|
sub(/[^A-Za-z0-9_$].*$/, "", t)
|
|
if (t != "") print "sym\t" t; return
|
|
}
|
|
}
|
|
|
|
{ raw[NR] = $0 }
|
|
|
|
END {
|
|
blank_spans(raw, NR, msk)
|
|
for (r = 1; r <= NR; r++) emit(msk[r])
|
|
}
|
|
' 2>/dev/null
|
|
}
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# ARM 1 — BY NAME, LOCALLY (#2280). Does a definition of this already exist?
|
|
#
|
|
# The recorded arms ask Scribe what was RECORDED; the ledger arm (#2900) asks
|
|
# what a BOUND repo's ledger knows. A helper nobody recorded, in a repo nobody
|
|
# bound, is invisible to both — which is how `.btn-primary` came to be defined
|
|
# four times, already diverged. This arm asks the one question only the
|
|
# developer's machine can answer, inside the repo, holding the code about to
|
|
# be written: no index, no storage, no server — it runs even on an install
|
|
# that has never configured Scribe.
|
|
#
|
|
# Definition-shaped patterns only. Grepping for bare occurrences would match
|
|
# every CALL site and drown the real finding — and a hint that is mostly noise
|
|
# is one people learn to skip, which is worse than none. ALL code, not a
|
|
# language shortlist (#2682): the same keyword family scribe_defs announces.
|
|
#
|
|
# $1 repo root, $2 repo-relative path of the file being written (excluded from
|
|
# the grep — it would always match itself on an Edit). Definitions on stdin.
|
|
# Prints one "> - `name` is already defined in N other file(s): …" per hit.
|
|
# ── The contract around a change (#4215, milestone 419) ───────────────────
|
|
#
|
|
# WHAT THIS ANSWERS. "You altered the arity, name or shape of something — here
|
|
# is everything that reads it." Rule 33's interface-contract check, one scope
|
|
# down: not between layers but between a definition and its callers.
|
|
#
|
|
# WHY IT IS A CHECK AND NOT A LESSON. #4207 was written — "widening a tuple is
|
|
# an interface change to every unpack site, and the compiler will not tell
|
|
# you" — hours before a structurally identical mistake was made by its author,
|
|
# and it was surfaced twice in the turns before. Text delivered at the moment
|
|
# of acting is too weak a carrier for a reflex that has to change what the act
|
|
# IS. This looks it up instead.
|
|
#
|
|
# `scribe_exposed` — the names a CALLER can depend on, from a blob of code.
|
|
# Three kinds, because a contract breaks three ways and they look nothing
|
|
# alike in the source:
|
|
#
|
|
# sym what is defined rename / removal
|
|
# arg its parameter names arity and order
|
|
# key quoted keys of dict literals the shape of what it RETURNS
|
|
#
|
|
# The third is here because of the miss that produced this step. A config
|
|
# function gained one dict key; three arms read that dict inside a fail-open
|
|
# `except`, so every one of them silently became a no-op and ten tests went
|
|
# red at once with nothing pointing at the cause. No signature changed. A
|
|
# check that only watched signatures would have watched the wrong thing.
|
|
scribe_exposed() {
|
|
# The `sym` half DELEGATES to scribe_defs rather than repeating its patterns.
|
|
# Those patterns cover nine languages and have been corrected several times
|
|
# (the Go receiver form, the `type` import-specifier false positive, the
|
|
# dunder skip); a second copy here would inherit today's version and then
|
|
# quietly stop agreeing with it, which is #3497's history for the two rule
|
|
# arms. One reader, called twice.
|
|
local blob
|
|
blob=$(cat)
|
|
{
|
|
printf '%s' "$blob" | scribe_defs
|
|
printf '%s' "$blob" | awk '
|
|
function emit(kind, name) {
|
|
if (name != "" && name !~ /^__.*__$/) print kind "\t" name
|
|
}
|
|
{
|
|
line = $0; sub(/^[[:space:]]+/, "", line)
|
|
sub(/^((pub(\([a-z]+\))?|export|default|private|internal|protected|public|static|suspend|async|open|sealed|data|abstract|final|inline|unsafe|extern|override)[[:space:]]+)*/, "", line)
|
|
|
|
# Parameter names, from whatever announces a definition. Taken from the
|
|
# FIRST parenthesis only: a default value can itself contain parens and
|
|
# a greedy match would swallow the body of a one-liner.
|
|
if (match(line, /^(function|def|func|fun|fn|sub)[[:space:]]+[A-Za-z_$][A-Za-z0-9_$]*[[:space:]]*\(/) \
|
|
|| match(line, /^(const|let)[[:space:]]+[A-Za-z_$][A-Za-z0-9_$]*[[:space:]]*=[[:space:]]*(async[[:space:]]*)?\(/)) {
|
|
args = line
|
|
sub(/^[^(]*\(/, "", args)
|
|
sub(/\).*$/, "", args)
|
|
n = split(args, parts, ",")
|
|
for (i = 1; i <= n; i++) {
|
|
a = parts[i]
|
|
gsub(/^[[:space:]]+|[[:space:]]+$/, "", a)
|
|
# Strip a type annotation, a default, and the * / ** / & markers.
|
|
sub(/[:=].*$/, "", a)
|
|
gsub(/^[*&]+/, "", a)
|
|
gsub(/[[:space:]]/, "", a)
|
|
# `self` and `cls` are not part of anything a caller passes.
|
|
if (a != "" && a != "self" && a != "cls" && a ~ /^[A-Za-z_$][A-Za-z0-9_$]*$/)
|
|
emit("arg", a)
|
|
}
|
|
}
|
|
|
|
# Quoted keys of a dict / object literal. Anchored on the quote so a
|
|
# dictionary ACCESS (`cfg["k"]`) does not read as a definition of one —
|
|
# only `"k":` counts, which is the writing position.
|
|
rest = $0
|
|
while (match(rest, /["'\''][A-Za-z_][A-Za-z0-9_]*["'\''][[:space:]]*:/)) {
|
|
tok = substr(rest, RSTART, RLENGTH)
|
|
rest = substr(rest, RSTART + RLENGTH)
|
|
gsub(/["'\'']/, "", tok); sub(/[[:space:]]*:$/, "", tok)
|
|
emit("key", tok)
|
|
}
|
|
}
|
|
' 2>/dev/null
|
|
} | sort -u
|
|
}
|
|
|
|
# Who references `name` anywhere else in the repo. Word-bounded, so `cfg` does
|
|
# not match `cfg_path`, and the defining file is excluded — a definition is
|
|
# always its own first mention and listing it says nothing.
|
|
#
|
|
# `|| true` INSIDE the substitution for the reason #4042 records at
|
|
# scribe_local_dups: under `pipefail` a `head` that exits early kills the
|
|
# still-writing git grep, and an outer fallback then wipes the hits head had
|
|
# already printed.
|
|
scribe_contract_readers() {
|
|
local root="$1" rel="$2" name="$3"
|
|
[ -n "$root" ] && [ -n "$name" ] || return 0
|
|
git -C "$root" grep -I -l -w -e "$name" -- . ":(exclude)${rel}" 2>/dev/null \
|
|
| head -6 || true
|
|
}
|
|
|
|
# The whole check, rendered. Kept here rather than inline in the hook so it
|
|
# can be exercised against a pair of blobs with no event, no server and no
|
|
# session — which is how every case in test_contract_around_the_change.py is
|
|
# written.
|
|
#
|
|
# Arguments: root, repo-relative path, the subject definition, the old text,
|
|
# the new text, and the session ledger (may be empty).
|
|
#
|
|
# TWO GATES, AND THE SECOND IS WHAT KEEPS THIS QUIET. A change to the exposed
|
|
# set is necessary but not sufficient: a definition NOTHING else references
|
|
# has no contract to break, so the readers lookup runs second and an empty
|
|
# result ends it silently. On a repo of any size most edits touch something
|
|
# local, so most edits say nothing here — and a hint that fires on everything
|
|
# is one that gets skipped.
|
|
scribe_contract_block() {
|
|
local root="$1" rel="$2" subject="$3" old_text="$4" new_text="$5" ledger="$6"
|
|
local changed gained lost readers count
|
|
[ -n "$root" ] && [ -n "$subject" ] || return 0
|
|
[ -n "$old_text" ] && [ -n "$new_text" ] || return 0
|
|
|
|
# Named once per session per subject. A second edit to the same definition
|
|
# is the SAME contract question, and answering it again would punish the
|
|
# ordinary rhythm of getting a change right over several passes.
|
|
if [ -n "$ledger" ] && [ -f "$ledger" ]; then
|
|
grep -qxF "$subject" "$ledger" 2>/dev/null && return 0
|
|
fi
|
|
|
|
# One pass, no process substitution: `comm` would need /dev/fd, and this
|
|
# also keeps the two sides' extraction visibly identical.
|
|
changed=$(
|
|
{
|
|
printf '%s' "$old_text" | scribe_exposed | sed 's/^/O\t/'
|
|
printf '%s' "$new_text" | scribe_exposed | sed 's/^/N\t/'
|
|
} | awk -F'\t' '
|
|
NF >= 3 { k = $2 "\t" $3; side[k] = side[k] $1 }
|
|
END {
|
|
for (k in side)
|
|
if (side[k] == "O") print "lost\t" k
|
|
else if (side[k] == "N") print "gained\t" k
|
|
}
|
|
' 2>/dev/null
|
|
)
|
|
[ -n "$changed" ] || return 0
|
|
|
|
readers=$(scribe_contract_readers "$root" "$rel" "$subject")
|
|
[ -n "$readers" ] || return 0
|
|
count=$(printf '%s\n' "$readers" | grep -c . 2>/dev/null || printf '0')
|
|
|
|
gained=$(printf '%s\n' "$changed" | awk -F'\t' '$1=="gained" {printf "%s%s %s", (n++?", ":""), $2, $3}')
|
|
lost=$(printf '%s\n' "$changed" | awk -F'\t' '$1=="lost" {printf "%s%s %s", (n++?", ":""), $2, $3}')
|
|
|
|
printf '> The contract around `%s` changed, and %s other file(s) reference it (`git grep -w`; a nudge, not a gate):\n' \
|
|
"$subject" "$count"
|
|
[ -n "$gained" ] && printf '> gained: %s\n' "$gained"
|
|
[ -n "$lost" ] && printf '> lost: %s\n' "$lost"
|
|
printf '> read by: %s\n' "$(printf '%s' "$readers" | tr '\n' ' ' | sed 's/ $//')"
|
|
printf '> A caller that passes or reads the old shape keeps compiling and fails only when that line runs (lesson #4207). Read them before moving on.\n'
|
|
[ -n "$ledger" ] && printf '%s\n' "$subject" >> "$ledger" 2>/dev/null
|
|
return 0
|
|
}
|
|
|
|
|
|
scribe_slippage_lines() {
|
|
# $1 state dir, $2 sanitised session id.
|
|
#
|
|
# SILENT ONLY WHEN NO RULE TOUCHED THE SESSION AT ALL. Traffic that did
|
|
# happen is always reported, because "which rules governed this work" is
|
|
# what the static instructions above already ask the summariser to preserve
|
|
# in prose — these lines are the measured version of that, and they are
|
|
# three short lines.
|
|
#
|
|
# What is conditional is the ACCUSATION. Each subtraction prints only when
|
|
# it has members, so a session that opened everything it was shown and
|
|
# resolved everything it opened gets the traffic and no more. "0 rules
|
|
# unresolved" on every compaction is how a readout teaches its reader to
|
|
# skip it.
|
|
local dir="$1" sid="$2" named opened acted held
|
|
[ -n "$dir" ] && [ -n "$sid" ] || return 0
|
|
# THE TWINS, not the exclusion ledgers. This runs at the compaction, which
|
|
# is the moment the exclusion ledgers are about to be cleared and the moment
|
|
# their TTL has usually already eaten the early part of a long session. A
|
|
# readout built on them would report only the last stretch of the session
|
|
# and read as though it had reported all of it — the #3311 shape, and the
|
|
# one this milestone exists to stop producing.
|
|
named=$(scribe_ledger_kept "$dir/${sid}.rules.keep.ids")
|
|
opened=$(scribe_ledger_kept "$dir/${sid}.opened.keep.ids")
|
|
acted=$(scribe_ledger_kept "$dir/${sid}.acted.keep.ids")
|
|
held=$(scribe_ledger_kept "$dir/${sid}.checkpoint.keep.ids")
|
|
[ -n "$named$opened" ] || return 0
|
|
|
|
local unread unresolved
|
|
unread=$(scribe_ids_minus "$named" "$opened")
|
|
unresolved=$(scribe_ids_minus "$opened" "$acted")
|
|
|
|
printf -- '- This session'"'"'s rule traffic, from what actually happened rather than from recollection:\n'
|
|
[ -n "$opened" ] && printf -- ' read: %s\n' "$opened"
|
|
[ -n "$held" ] && printf -- ' held an act before it ran: %s\n' "$held"
|
|
[ -n "$unread" ] && printf -- ' named by an arm and never opened: %s\n' "$unread"
|
|
if [ -n "$unresolved" ]; then
|
|
printf -- ' READ WITH NO OUTCOME RECORDED: %s. Carry these over as outstanding. A rule read and left unresolved looks exactly like one that worked, and the summary is where that difference is lost for good — say `rule_outcome(id, "applied")`, or `rule_outcome(id, "departed", why=...)` where you deliberately went another way.\n' "$unresolved"
|
|
fi
|
|
return 0
|
|
}
|
|
|
|
scribe_ids_minus() {
|
|
# Set difference over two space-separated id lists, order preserved. Written
|
|
# as one awk pass rather than a nested shell loop because the ledgers can
|
|
# hold a few dozen ids by the end of a long session and this runs inside the
|
|
# compaction path, where a slow hook delays the thing it is decorating.
|
|
local a="$1" b="$2"
|
|
[ -n "$a" ] || return 0
|
|
awk -v a="$a" -v b="$b" '
|
|
BEGIN {
|
|
n = split(b, drop, " ")
|
|
for (i = 1; i <= n; i++) if (drop[i] != "") skip[drop[i]] = 1
|
|
m = split(a, keep, " ")
|
|
out = ""
|
|
for (i = 1; i <= m; i++) {
|
|
id = keep[i]
|
|
if (id == "" || (id in skip) || (id in done)) continue
|
|
done[id] = 1
|
|
out = out (out == "" ? "" : " ") id
|
|
}
|
|
if (out != "") print out
|
|
}
|
|
' </dev/null 2>/dev/null
|
|
}
|
|
|
|
scribe_local_dups() {
|
|
local root="$1" rel="$2" kind name pat hits count label files
|
|
while IFS=$'\t' read -r kind name; do
|
|
[ -n "${name:-}" ] || continue
|
|
case "$kind" in
|
|
css) pat="^[[:space:]]*\.${name}[[:space:]]*[,{]" ;;
|
|
*) pat="(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+${name}[^A-Za-z0-9_]|func[[:space:]]*\([^)]*\)[[:space:]]*${name}[[:space:]]*\(|(const|let)[[:space:]]+${name}[[:space:]]*=" ;;
|
|
esac
|
|
# -I skips binaries; :(exclude) drops the file being written.
|
|
# `|| true` INSIDE the substitution, not `|| hits=""` outside it (#4042):
|
|
# under the hooks' `pipefail`, `head` exiting after four lines kills a
|
|
# git grep that is still writing, the pipeline reports SIGPIPE, and an
|
|
# outer fallback then wipes the four hits head already printed. The name
|
|
# most duplicated — the one this arm exists for — was the one it dropped.
|
|
hits=$(git -C "$root" grep -I -l -E -e "$pat" -- . ":(exclude)${rel}" 2>/dev/null | head -4 || true)
|
|
[ -n "$hits" ] || continue
|
|
count=$(printf '%s\n' "$hits" | grep -c . 2>/dev/null || echo 0)
|
|
label=$([ "$kind" = css ] && printf '.%s' "$name" || printf '%s' "$name")
|
|
files=$(printf '%s' "$hits" | tr '\n' ' ' | sed 's/ $//')
|
|
printf '> - `%s` is already defined in %s other file(s): %s\n' "$label" "$count" "$files"
|
|
done
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# The blind spot made visible (#2932). Both write-path hooks fail OPEN when the
|
|
# instance is slow or down — right for noise, wrong for silence: a session
|
|
# cannot tell "the ledger checked and found nothing" from "the ledger never
|
|
# answered", and a self-surfacing system cannot afford an invisible miss (the
|
|
# first write after a redeploy lost its derive line to a 4s cold start and
|
|
# nobody knew). So a failed call says so — ONCE per outage: the marker holds
|
|
# the time it last spoke; within ten minutes of that it stays quiet, and a
|
|
# successful call clears it so the next outage announces itself afresh.
|
|
# Unconfigured installs never reach this: no URL/token means no call was owed.
|
|
# Where every hook gets its endpoint and credential. Four lines, and each of
|
|
# the five hooks carried its own copy until #2278 — which is exactly the
|
|
# missing-sibling shape: the `${...}` guard below is a correctness detail a
|
|
# sixth hook would have forgotten, and nothing would have failed loudly.
|
|
#
|
|
# Sets `url` and `token` as globals rather than echoing them: a token must not
|
|
# pass through a subshell's output, where it could land in a log or an `xtrace`
|
|
# line. Returns 0 only when both are usable, so a caller can either bail
|
|
# (`scribe_config || exit 0`) or carry on degraded — the session-context hook
|
|
# still owes its static floor when Scribe is unconfigured.
|
|
# Declared here, not just assigned inside the function: `scribe_defs.sh` owns
|
|
# these two names, and a sourcing hook should have them defined the moment it
|
|
# sources — before any code path that might reference them. It also lets
|
|
# the linter see the assignment, which it cannot follow into a function in
|
|
# another file without -x (SC2154).
|
|
url=""
|
|
token=""
|
|
|
|
scribe_config() {
|
|
url=${SCRIBE_URL:-${CLAUDE_PLUGIN_OPTION_API_ENDPOINT:-}}
|
|
token=${SCRIBE_TOKEN:-${CLAUDE_PLUGIN_OPTION_API_TOKEN:-}}
|
|
# An unexpanded `${...}` placeholder arriving as a literal would be sent as a
|
|
# garbage Bearer token and 401. Treat it as unset.
|
|
case "$url" in *'${'*) url="" ;; esac
|
|
case "$token" in *'${'*) token="" ;; esac
|
|
[ -n "$url" ] && [ -n "$token" ]
|
|
}
|
|
|
|
_SCRIBE_UNREACHED_QUIET=600
|
|
|
|
scribe_unreached() {
|
|
local marker="$1/$2.unreached" now last
|
|
now=$(date +%s 2>/dev/null) || now=0
|
|
if [ -f "$marker" ]; then
|
|
last=$(cat "$marker" 2>/dev/null) || last=0
|
|
case "$last" in ''|*[!0-9]*) last=0 ;; esac
|
|
[ $((now - last)) -lt "$_SCRIBE_UNREACHED_QUIET" ] && return 0
|
|
fi
|
|
printf '%s' "$now" > "$marker" 2>/dev/null || true
|
|
printf '> Scribe did not answer the prior-art check for `%s` within %ss — this write went UNCHECKED against the record and the shape ledger (the local by-name arm, if it spoke above, needed no server). If the name matters, check it yourself: `search` for the concept, `list_shapes(project_id, path=…)` for the ledger. Said once per outage; if it keeps happening the instance is slow or down.' "$4" "$3"
|
|
}
|
|
|
|
scribe_reached() {
|
|
rm -f "$1/$2.unreached" 2>/dev/null || true
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# THE RULE EXCLUSION LEDGER, and how entries in it AGE (#3751).
|
|
#
|
|
# Two hooks write this file and two read it, which is the whole reason the
|
|
# parsing lives here: `scribe_prior_art.sh` and `scribe_tool_rules.sh` share
|
|
# one ledger so a rule named by one arm is not re-offered by the other, and a
|
|
# format only one of them understood would break that on the first read.
|
|
#
|
|
# WHAT PROBLEM AGEING SOLVES. #3749 clears the ledger when an EVENT destroys
|
|
# context — a compaction or a /clear. This is the case with no event at all: a
|
|
# long session where the rule was named two hundred turns ago and has simply
|
|
# fallen out of attention. It is #3702's argument at the tier level (present in
|
|
# context and salient at the moment are different properties) applied to time
|
|
# instead of to tier.
|
|
#
|
|
# PER-ENTRY TIMESTAMPS, NOT A FILE MTIME. Clearing the whole ledger when the
|
|
# file is old is one line of shell and wrong in exactly the session that needs
|
|
# it: a single recent write keeps every stale id alive, and the ids that go
|
|
# stale first are the ones from the rules that fire most.
|
|
#
|
|
# WALL TIME, NOT A TURN COUNT, and the trade is real rather than dismissed. A
|
|
# turn count is a truer model of salience — an idle session does not forget —
|
|
# but a hook has no turn number without keeping its own counter, which is a
|
|
# second piece of session state to write, read, clear on compaction and get
|
|
# wrong. Wall time is available from `date` and costs nothing. The failure mode
|
|
# it accepts is a session left idle over lunch treating its rules as forgotten,
|
|
# which produces one extra full line per rule and no other harm.
|
|
_SCRIBE_RULE_TTL=2700
|
|
|
|
# 45 MINUTES, and the reasoning rather than the number (rule 32).
|
|
#
|
|
# There is no data on this yet, so it is a judgement made to be revised — the
|
|
# telemetry that would settle it is the one #3807 just built, and a reading of
|
|
# how often an aged-out rule gets PULLED after it returns is what should move
|
|
# this.
|
|
#
|
|
# Too short and the exclusion stops existing and the repetition it prevents
|
|
# comes back. Too long and it never fires at all in a session short enough to
|
|
# matter. 45 minutes is about one working stretch on a single task: long enough
|
|
# that a rule does not re-announce itself while you are still doing the thing
|
|
# it governs, short enough that a multi-hour session gets a genuine refresh
|
|
# rather than one 9am mention.
|
|
#
|
|
# Being wrong on the short side is now the cheaper error, which is why this
|
|
# leans short. Since #3750 an excluded rule is REFERENCED rather than withheld,
|
|
# so the ledger is no longer the only thing standing between a session and a
|
|
# rule it has forgotten — an expired entry costs one full line instead of one
|
|
# short one, and the exclusion re-arms the moment it is spent.
|
|
|
|
# Live ids from a ledger, comma-joined for `exclude_rule_ids`. Empty output for
|
|
# a missing, empty or fully-aged file — the callers already treat "" as "send
|
|
# no exclusions".
|
|
#
|
|
# THE LAST ENTRY FOR AN ID WINS, and this is what stops a rule ping-ponging.
|
|
# The file is append-only, so a rule that ages out, gets surfaced fresh and is
|
|
# appended again has TWO lines. Reading the first would leave it permanently
|
|
# expired and it would re-announce itself on every single call from then on —
|
|
# the loudest possible failure, from the mechanism meant to quieten things.
|
|
# Appends are chronological, so the last line for an id is its most recent.
|
|
#
|
|
# A BARE ID — no tab, no timestamp — IS LIVE. That is the pre-#3751 format, and
|
|
# a session in flight when this ships has a ledger full of them. Treating
|
|
# unknown as expired would make every one of those sessions re-announce every
|
|
# rule it had already been told, all at once, which is precisely the noise this
|
|
# exists to prevent. Unknown means "not measured" and never "old" — the same
|
|
# null discipline the retrieval_logs columns use. Those entries simply never
|
|
# age, which is bounded: the session ends.
|
|
scribe_rules_live() {
|
|
local f="$1" now
|
|
[ -n "$f" ] && [ -f "$f" ] || return 0
|
|
now=$(date +%s 2>/dev/null) || now=0
|
|
awk -F'\t' -v now="$now" -v ttl="$_SCRIBE_RULE_TTL" '
|
|
{
|
|
id = $1
|
|
gsub(/[^0-9]/, "", id)
|
|
if (id == "") next
|
|
if (!(id in seen)) { seen[id] = 1; seq[++n] = id }
|
|
stamp[id] = ($2 ~ /^[0-9]+$/) ? $2 : ""
|
|
}
|
|
END {
|
|
out = ""
|
|
for (i = 1; i <= n; i++) {
|
|
id = seq[i]
|
|
if (stamp[id] != "" && now > 0 && (now - stamp[id]) > ttl) continue
|
|
out = out (out == "" ? "" : ",") id
|
|
}
|
|
print out
|
|
}
|
|
' "$f" 2>/dev/null || true
|
|
}
|
|
|
|
# Append surfaced ids, stamped. Reads ids on stdin, one per line — the shape
|
|
# `scribe_json_list "$flat" '.rule_ids'` already produces at both call sites.
|
|
scribe_rules_append() {
|
|
# Writes TWO files, and the second one is the point (#4217).
|
|
#
|
|
# `$f` is an EXCLUSION ledger: it answers "does this context already hold
|
|
# this?", so it is aged by TTL and swept at every compaction — after a
|
|
# compaction the agent genuinely does not hold what it was shown, and
|
|
# forgetting is correct. Three hooks depend on exactly that.
|
|
#
|
|
# The twin is an EVIDENCE ledger: it answers "did this happen?", which no
|
|
# compaction can make untrue. It is never aged and never swept.
|
|
#
|
|
# These are opposite lifetimes and `.opened.ids` was serving both, which is
|
|
# how a session with 45 `get_rule` calls came to report none: the compaction
|
|
# cleared the ledger that was also the record. Measured across six sessions
|
|
# on this instance — 208 opens, 3 surviving — before the split.
|
|
#
|
|
# DERIVED HERE rather than listed at the call sites, because a list is what
|
|
# broke this before (see `scribe_clear_session_ledgers`). Every ledger
|
|
# written through this function gets its twin, including the next one
|
|
# somebody adds. The cost is a second small file per ledger per session.
|
|
local f="$1" now
|
|
[ -n "$f" ] || return 0
|
|
now=$(date +%s 2>/dev/null) || now=0
|
|
awk -v ts="$now" -v keep="${f%.ids}.keep.ids" '
|
|
NF { line = $1 "\t" ts; print line; print line >> keep }
|
|
' >> "$f" 2>/dev/null || true
|
|
}
|
|
|
|
scribe_ledger_kept() {
|
|
# Ids from an EVIDENCE twin: deduped, order preserved, NOT aged. A rule
|
|
# opened two hours ago was still opened, so the TTL that keeps an exclusion
|
|
# ledger honest would here delete the finding.
|
|
local f="$1"
|
|
[ -n "$f" ] && [ -f "$f" ] || return 0
|
|
awk -F'\t' '$1 != "" && !seen[$1]++ {
|
|
out = out (out == "" ? "" : " ") $1
|
|
} END { if (out != "") print out }' "$f" 2>/dev/null || true
|
|
}
|
|
|
|
# The OPENED ledger's contribution to a rule arm's query string (#4100).
|
|
#
|
|
# TWO LEDGERS, BECAUSE THEY RECORD TWO DIFFERENT FACTS. `.rules.ids` holds
|
|
# every id an arm has NAMED; `.opened.ids` holds the ids the session actually
|
|
# read, written by scribe_record_opened.sh from the `get_rule` call itself.
|
|
# Named is not read: the injected line is a teaser, and one skimmed past
|
|
# leaves nothing behind — least of all across a compaction. Sending both lets
|
|
# the server tell a reader who opened a rule from one who was only shown it,
|
|
# instead of telling them both the same untrue thing.
|
|
#
|
|
# Same reader as the naming ledger on purpose, so ageing, the last-entry-wins
|
|
# rule and the bare-id format are defined once and cannot drift apart.
|
|
# ── The pre-act checkpoint's session ledger (#4214, milestone 419) ─────────
|
|
#
|
|
# A checkpoint STOPS an act rather than annotating it, so unlike every other
|
|
# ledger here its job is to make sure the same stop cannot happen twice. Two
|
|
# guards, and they fail in different directions:
|
|
#
|
|
# per rule — a rule may hold at most ONE act per session. Once it has, the
|
|
# remedy has been offered; repeating it on the next call would
|
|
# turn a reader who decided the rule does not apply into a
|
|
# reader who cannot proceed.
|
|
# per session — at most `_SCRIBE_CHECKPOINT_CAP` stops in total, whatever the
|
|
# corpus scores. A mis-set floor or a corpus that suddenly
|
|
# resembles everything must degrade to a noisy session, never
|
|
# to one that cannot make progress. This is the guard on the
|
|
# worst case, not a tuning value.
|
|
#
|
|
# NOT AGED, unlike the naming ledgers. Those age because they describe what a
|
|
# session is still HOLDING, and a context stops holding things. This one
|
|
# describes what already HAPPENED — a stop was raised and its remedy offered —
|
|
# and that does not become untrue an hour later. Ageing it would let a long
|
|
# session be stopped by the same rule repeatedly, which is the one outcome
|
|
# both guards exist to prevent.
|
|
_SCRIBE_CHECKPOINT_CAP=5
|
|
|
|
scribe_checkpoint_allowed() {
|
|
# $1 ledger file, $2 rule id. Returns 0 (and RECORDS the stop) when this act
|
|
# may be held; non-zero otherwise. Records on the way out rather than asking
|
|
# the caller to, because a caller that forgets is a session that can be
|
|
# stopped forever and the failure is invisible until it happens.
|
|
local f="$1" id="$2" n
|
|
[ -n "$f" ] || return 1
|
|
id=$(printf '%s' "$id" | tr -cd '0-9')
|
|
[ -n "$id" ] || return 1
|
|
if [ -f "$f" ]; then
|
|
# Already held an act for this rule — the remedy has been offered once.
|
|
grep -qx "$id" "$f" 2>/dev/null && return 1
|
|
n=$(grep -c '^[0-9][0-9]*$' "$f" 2>/dev/null || printf '0')
|
|
# `grep -c` over a missing file can print nothing; a bare arithmetic test
|
|
# on an empty string is a syntax error in some shells and silently true in
|
|
# others, which is how a cap comes to cap nothing (#3191's shape).
|
|
n=$(printf '%s' "$n" | tr -cd '0-9')
|
|
[ -n "$n" ] || n=0
|
|
[ "$n" -ge "$_SCRIBE_CHECKPOINT_CAP" ] && return 1
|
|
fi
|
|
printf '%s\n' "$id" >> "$f" 2>/dev/null || return 1
|
|
# The evidence twin, beside the cap rather than instead of it. `$f` is swept
|
|
# at a compaction and SHOULD be: after one, this context has not read the
|
|
# rule, so the budget to stop an act on it is honestly fresh. That a stop
|
|
# already happened is a different claim, and it stays true.
|
|
printf '%s\n' "$id" >> "${f%.ids}.keep.ids" 2>/dev/null || true
|
|
return 0
|
|
}
|
|
|
|
scribe_json_deny() {
|
|
# $1 hook event name, $2 the reason the agent reads INSTEAD of the result.
|
|
#
|
|
# The one place this plugin emits a decision on a tool call. `deny` returns
|
|
# the reason to the MODEL and the call does not run — it is not a prompt to
|
|
# the operator, costs them nothing, and is undone by the model simply
|
|
# submitting the call again. That is the whole difference from `ask`, which
|
|
# would hand a judgement that is the agent's to make to the person who asked
|
|
# for the work.
|
|
local esc
|
|
esc=$(printf '%s' "$2" | scribe_json_escape) || return 0
|
|
printf '{"hookSpecificOutput":{"hookEventName":"%s","permissionDecision":"deny","permissionDecisionReason":"%s"}}\n' \
|
|
"$1" "$esc"
|
|
}
|
|
|
|
scribe_held_query() {
|
|
local ids
|
|
ids=$(scribe_rules_live "$1")
|
|
[ -n "$ids" ] && printf '&held_rule_ids=%s' "$ids"
|
|
return 0
|
|
}
|
|
|
|
# Drop EVERY per-session ledger, matched by convention rather than listed (#4101).
|
|
#
|
|
# A LIST IS THE BUG. Until now the compact/clear branch named its files one at
|
|
# a time, and it named two of the five: `.rules.ids` and `.opened.ids` were
|
|
# cleared while `.ids` (notes), `.sync.ids` (shape signals) and `.derive.ids`
|
|
# survived. So milestone 386's defect — "a compaction destroys the context but
|
|
# not the ledger, so the most applicable records become permanently
|
|
# unreachable mid-session" — was fixed for rules and left standing on the note
|
|
# and snippet surfaces, which are the ones that fire most often.
|
|
#
|
|
# Nobody decided that. The list was written when rules were the only ledger
|
|
# that mattered and was never revisited when the others arrived, which is what
|
|
# a hand-maintained list of "things to remember to clean up" does. Adding three
|
|
# more `rm` lines would rebuild the same trap for the sixth ledger.
|
|
#
|
|
# So the rule is the NAME: a per-session ledger is `<sid>[.<kind>].ids`, and
|
|
# everything matching that goes. A new ledger following the convention is
|
|
# covered the day it is written, by nobody. One that does not follow it is a
|
|
# deliberate exception and has to say so.
|
|
#
|
|
# Scoped to `.ids` rather than `<sid>.*` so a marker that is REWRITTEN on
|
|
# compact rather than discarded can still live in these directories without
|
|
# being swept away by a glob that was never told about it. `<sid>.unreached`
|
|
# is the live example: it records that the instance could not be reached, not
|
|
# what the session holds, and #2932 needs it to outlive a compaction.
|
|
#
|
|
# TWO DIRECTORIES, WHICH IS ITS OWN LESSON. The first cut of this swept only
|
|
# `scribe-priorart` — every ledger named in the hooks was there, so the list
|
|
# looked complete. `scribe_autoinject.sh` keeps its note ledger in
|
|
# `scribe-autoinject`, so the arm that fires most (598 calls in five days) was
|
|
# the one the clear could not reach, and the fix read as finished. Hence the
|
|
# roster here rather than a path at the call site: one place to add to, and
|
|
# the tests read THIS string rather than a copy of it.
|
|
SCRIBE_LEDGER_DIRS="scribe-priorart scribe-autoinject"
|
|
|
|
scribe_clear_session_ledgers() {
|
|
# SPARES `*.keep.ids`, which are evidence rather than exclusions — see
|
|
# `scribe_rules_append`. Everything else still goes: the convention is
|
|
# unchanged and still covers a ledger added tomorrow, which is what the
|
|
# block above insists on. The twin opts OUT by its name, so a new ledger is
|
|
# born on the swept side unless somebody says otherwise.
|
|
local sid="$1" dir f
|
|
[ -n "$sid" ] || return 0
|
|
for dir in $SCRIBE_LEDGER_DIRS; do
|
|
for f in "${TMPDIR:-/tmp}/$dir/$sid"*.ids; do
|
|
[ -f "$f" ] || continue
|
|
case "$f" in *.keep.ids) continue ;; esac
|
|
rm -f "$f" 2>/dev/null || true
|
|
done
|
|
done
|
|
return 0
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# WHICH PROJECT IS THIS DIRECTORY'S? (#4085)
|
|
#
|
|
# Every hook here scopes its request to a project, and until this existed all
|
|
# six did it the same single way: `git remote get-url origin`, resolved
|
|
# server-side through the repo bindings. That works well inside a bound repo
|
|
# and not at all outside one — a session in a plain directory got no project
|
|
# scope from ANY hook, silently, because the one key the whole chain turns on
|
|
# only exists in a git repo.
|
|
#
|
|
# The second key is a `.scribe` file in the directory (or above it, the way
|
|
# git finds its root), naming the project the work belongs to. It is a
|
|
# POINTER, not a copy: an id and enough to check the id means what it says.
|
|
# Nothing Scribe should be holding goes in it.
|
|
#
|
|
# {"instance": "https://scribe.example.com", "project_id": 2,
|
|
# "project": "FabledScribe"}
|
|
#
|
|
# instance WHICH Scribe the id belongs to, and the reason this file is
|
|
# not just a number. A project id means nothing on its own: id 2
|
|
# is a different project on every instance, so a marker that
|
|
# travels — a copied directory, a shared machine, a repo someone
|
|
# else clones — would silently scope a session to the wrong
|
|
# project. Compared HOST-ONLY against the configured endpoint, so
|
|
# http/https and a trailing slash don't cause a false mismatch.
|
|
# A mismatch drops the id: no project beats the wrong project.
|
|
# project_id the pointer itself. Required.
|
|
# project a human label. NOTHING READS IT. It is there so the file
|
|
# answers "what is this?" when opened, and so a stale one is
|
|
# visible rather than inert.
|
|
#
|
|
# A bare integer is also accepted (`echo 2 > .scribe`), because it is what a
|
|
# person writes by hand and it parses as JSON already. It skips the instance
|
|
# check by having nothing to check — deliberate, and the reason the written
|
|
# form carries `instance`.
|
|
#
|
|
# THE MARKER WINS over a git remote. Someone put the file there on purpose;
|
|
# a remote is just where the code happens to be pushed. That also makes the
|
|
# marker the way to override a binding for one directory.
|
|
|
|
# Nearest `.scribe` at or above DIR. Walks up to the filesystem root, capped so
|
|
# a pathological path cannot spin.
|
|
scribe_marker_file() {
|
|
local dir="$1" depth=0
|
|
[ -n "$dir" ] || return 0
|
|
while [ "$depth" -lt 40 ]; do
|
|
[ -f "$dir/.scribe" ] && { printf '%s' "$dir/.scribe"; return 0; }
|
|
case "$dir" in ""|"/") return 0 ;; esac
|
|
dir=$(dirname -- "$dir" 2>/dev/null) || return 0
|
|
depth=$((depth + 1))
|
|
done
|
|
return 0
|
|
}
|
|
|
|
# Host of a URL, lowercased, port kept. "" for empty input.
|
|
scribe_url_host() {
|
|
printf '%s' "${1:-}" \
|
|
| sed -e 's#^[A-Za-z][A-Za-z0-9+.-]*://##' -e 's#^[^/@]*@##' -e 's#[/?].*$##' \
|
|
| tr 'A-Z' 'a-z'
|
|
}
|
|
|
|
# Read a marker file: prints "ID<TAB>REASON", at most one of them non-empty.
|
|
#
|
|
# "7\t" use project 7
|
|
# "\tnames no …" a file is there and deliberately NOT used; say why
|
|
# "\t" no marker file at all — the ordinary case, say nothing
|
|
#
|
|
# One line rather than an id plus a global, because every caller reads this
|
|
# through `$( )` and a global set inside a command substitution dies with the
|
|
# subshell. The caller would then read an unset variable, which under the
|
|
# `set -u` these hooks all run with aborts the hook and costs the whole
|
|
# session's context — a failure far larger than the message it was fetching.
|
|
scribe_marker_read() {
|
|
local f="$1" flat id inst want
|
|
[ -n "$f" ] && [ -f "$f" ] || { printf '\t'; return 0; }
|
|
flat=$(scribe_json_flat < "$f") || flat=""
|
|
# Both accepted forms, read off one parse: the object's `project_id`, or —
|
|
# for the bare integer someone writes by hand — the root scalar, whose path
|
|
# is the empty string because it sits under no key.
|
|
id=$(scribe_json_pick "$flat" '.project_id')
|
|
[ -n "$id" ] || id=$(scribe_json_pick "$flat" '')
|
|
case "$id" in ''|*[!0-9]*) id="" ;; esac
|
|
if [ -z "$id" ] || [ "$id" = "0" ]; then
|
|
printf '\tnames no project_id'
|
|
return 0
|
|
fi
|
|
inst=$(scribe_json_pick "$flat" '.instance')
|
|
if [ -n "$inst" ]; then
|
|
want=$(scribe_url_host "${url:-}")
|
|
inst=$(scribe_url_host "$inst")
|
|
if [ -n "$want" ] && [ "$inst" != "$want" ]; then
|
|
printf '\tpoints at %s, but this session is configured for %s' "$inst" "$want"
|
|
return 0
|
|
fi
|
|
fi
|
|
printf '%s\t' "$id"
|
|
}
|
|
|
|
# Just the id a marker names, or "" — the half scribe_scope_query needs.
|
|
scribe_marker_project() {
|
|
scribe_marker_read "$1" | cut -f1
|
|
}
|
|
|
|
# The query args identifying DIR's project — `project_id=N` or `repo=<enc>` —
|
|
# with NO leading `?` or `&`, so each caller keeps its own separator. Empty
|
|
# when neither key is available. Requires `url` to be set (scribe_config) for
|
|
# the marker's instance check; without it a marker is still honoured, since an
|
|
# unconfigured hook is not going to send the request anyway.
|
|
scribe_scope_query() {
|
|
local dir="$1" id repo enc
|
|
id=$(scribe_marker_project "$(scribe_marker_file "$dir")")
|
|
if [ -n "$id" ]; then
|
|
printf 'project_id=%s' "$id"
|
|
return 0
|
|
fi
|
|
repo=$(git -C "$dir" remote get-url origin 2>/dev/null || true)
|
|
[ -n "$repo" ] || return 0
|
|
enc=$(printf '%s' "$repo" | scribe_urlenc) || enc=""
|
|
[ -n "$enc" ] && printf 'repo=%s' "$enc"
|
|
}
|