docs(6204): shipped instruction text describes the product, not this instance's records
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 39s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 39s
MCP tool docstrings and comments, the server's orientation notes, the plugin skills, every hook script and the plugin README/PACKAGING drop record ids, milestone numbers, dates and incident stories, and state each non-obvious why as a present-tense fact. History stays in the task logs. Corrections found on the way: - plugin README/PACKAGING said every hook is a GET; moment, rule-opened and reply-rules POST, so the plugin's key needs write scope. Both now say so, name Settings -> MCP Access, and list every wired hook and endpoint. - the contract-change hint printed "(lesson #4207)", this instance's record, to every install. It no longer cites a record. - tasks.py claimed there is no delete_task; there is. - rulebooks.py cited delete_event and a design spec that no longer exist. - scribe_after_write.sh carried an always-empty etag_q; removed. - test_write_path_trigger pinned "| tac | scribe_defs | head -1", text that existed only in a comment about the old code; it now pins the real tail -1 pipeline. - docs/development.md: CI runs on Gitea Actions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+143
-192
@@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
# shellcheck shell=bash
|
||||
# Scribe plugin — the pieces the hooks share (#2901, #2278).
|
||||
# Scribe plugin — the pieces the hooks share.
|
||||
#
|
||||
# scribe_prior_art.sh fires BEFORE a Write/Edit tool call; scribe_after_write.sh
|
||||
# fires AFTER a Bash tool call and diffs the working tree, so code written by
|
||||
@@ -9,24 +9,24 @@
|
||||
#
|
||||
# scribe_skip_path PATH formats that hold prose or data, not shapes
|
||||
# scribe_skip_prompt TEXT the turn was written by the CLIENT, not the
|
||||
# operator — skip it entirely (#4200)
|
||||
# operator — skip it entirely
|
||||
# scribe_defs stdin code → "kind<TAB>name" per definition
|
||||
# scribe_local_dups ROOT REL "kind<TAB>name" lines on stdin → the by-name
|
||||
# local-duplicate lines (ARM 1, #2280)
|
||||
# local-duplicate lines (ARM 1)
|
||||
# scribe_unreached STATE SID SECS REL the "Scribe didn't answer" line, once
|
||||
# per outage (#2932) — or nothing, if said lately
|
||||
# per outage — or nothing, if said lately
|
||||
# scribe_reached STATE SID the server answered: the next outage speaks again
|
||||
# scribe_config sets `url` + `token` from the env, returns 0
|
||||
# only if BOTH are usable (#2278)
|
||||
# only if BOTH are usable
|
||||
# scribe_rules_live FILE live rule ids from the exclusion ledger,
|
||||
# comma-joined; entries age out (#3751)
|
||||
# comma-joined; entries age out
|
||||
# scribe_rules_append FILE stdin ids -> the ledger, timestamped
|
||||
# scribe_json_flat stdin JSON -> IDX<TAB>PATH<TAB>VALUE lines,
|
||||
# queried with scribe_json_pick / _list / _len.
|
||||
# scribe_json_out writes the hook envelope,
|
||||
# scribe_urlenc percent-encodes. No jq (#4107).
|
||||
# scribe_urlenc percent-encodes. No jq.
|
||||
# scribe_scope_query DIR `project_id=N` or `repo=<enc>` for DIR — the
|
||||
# project-scope key EVERY hook sends (#4085).
|
||||
# project-scope key EVERY hook sends.
|
||||
# Helpers: scribe_marker_file, scribe_url_host,
|
||||
# scribe_marker_read (id<TAB>why-not),
|
||||
# scribe_marker_project (the id alone)
|
||||
@@ -45,23 +45,21 @@ scribe_skip_path() {
|
||||
return 1
|
||||
}
|
||||
|
||||
# Skip a turn the CLIENT wrote rather than the operator (#4200). Claude Code
|
||||
# Skip a turn the CLIENT wrote rather than the operator. Claude Code
|
||||
# submits several kinds of machine-written text through UserPromptSubmit — a
|
||||
# task notification, the echo of a slash command, the caveat banner a local
|
||||
# command prints — and they reach `.prompt` indistinguishable from typed words
|
||||
# unless something looks.
|
||||
#
|
||||
# THIS IS A TELEMETRY FIX FIRST AND A NOISE FIX SECOND, which is the part
|
||||
# worth keeping straight: retrieving against a notification usually returns
|
||||
# nothing, so the visible cost looks like one wasted embedding. The real cost
|
||||
# is that the call is LOGGED. It inflates the denominator of every
|
||||
# prompt-boundary surface, so delivery rate reads low for a reason that has
|
||||
# nothing to do with retrieval; and the refusal lands in `near_misses`, where
|
||||
# a later tuning decision reads it as unmet demand. Measured on #3898: 15 of
|
||||
# the top 20 preference near-misses were `<task-notification>` blocks, ALL
|
||||
# matching one record, ALL within thousandths of the floor. A floor lowered to
|
||||
# serve that apparent demand would inject that record into every notification
|
||||
# — the instrument arguing for the wrong fix, which is #379's lesson again.
|
||||
# THIS PROTECTS TELEMETRY FIRST AND CUTS NOISE SECOND. Retrieving against a
|
||||
# notification usually returns nothing, so the visible cost looks like one
|
||||
# wasted embedding. The real cost is that the call is LOGGED. It inflates the
|
||||
# denominator of every prompt-boundary surface, so delivery rate reads low for
|
||||
# a reason that has nothing to do with retrieval; and the refusal lands in
|
||||
# `near_misses`, where a tuning decision reads it as unmet demand. Notification
|
||||
# blocks resemble each other, so they pile up just under a floor, and a floor
|
||||
# lowered to serve that apparent demand would inject the same record into
|
||||
# every notification.
|
||||
#
|
||||
# A PREFIX TEST, NOT A SUBSTRING ONE, and that distinction is the whole safety
|
||||
# argument. A real prompt may well CONTAIN one of these tags — an operator
|
||||
@@ -88,16 +86,12 @@ scribe_skip_prompt() {
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# JSON AND URL-ENCODING, WITHOUT jq (#4107).
|
||||
# JSON AND URL-ENCODING, WITHOUT jq.
|
||||
#
|
||||
# Until this section every hook opened `command -v jq >/dev/null 2>&1 || exit 0`
|
||||
# and a machine without jq got no session context, no rules, no prior art and
|
||||
# no process sync — silently, because `exit 0` is indistinguishable from "ran
|
||||
# fine, nothing to say". That is the plugin pushing its own packaging problem
|
||||
# onto whoever installs it, and it is the same silence #4085 existed to remove.
|
||||
# jq is absent by default on macOS, on the Debian/Ubuntu slim images, on Alpine
|
||||
# and in most CI containers. awk, sed, tr and od are POSIX; every one of them
|
||||
# is already required by code above this line.
|
||||
# and in most CI containers, and a hook that exits quietly without it looks
|
||||
# exactly like one that ran and had nothing to say. awk, sed, tr and od are
|
||||
# POSIX; every one of them is already required by code above this line.
|
||||
#
|
||||
# scribe_json.awk does the parsing and carries the format. These are the four
|
||||
# jobs the hooks actually had jq for:
|
||||
@@ -129,13 +123,13 @@ scribe_json_flat_lines() {
|
||||
}
|
||||
|
||||
# $1 transcript path → the tail of the last assistant REPLY text, decoded, on
|
||||
# one line; "" when there is none (#4364). The auto-inject query pairs it with
|
||||
# one line; "" when there is none. The auto-inject query pairs it with
|
||||
# a short prompt, so "yes do that" still says what it is about.
|
||||
#
|
||||
# Cheap by construction, because it runs on every prompt: the last 512 KB only,
|
||||
# and grep narrows to assistant records carrying a text block BEFORE anything is
|
||||
# parsed — a transcript is mostly tool results, and parsing those in awk is the
|
||||
# multi-second cost the completion-report check measured (#4107). Fixed strings
|
||||
# parsed — a transcript is mostly tool results, and parsing those in awk costs
|
||||
# seconds. Fixed strings
|
||||
# with UNESCAPED quotes can only match at a record's own top level.
|
||||
# A sidechain is a subagent talking, not this session, so it is skipped.
|
||||
scribe_recent_context() {
|
||||
@@ -160,9 +154,8 @@ scribe_recent_context() {
|
||||
|
||||
# $1 flat text, $2 exact path → the decoded scalar, or "" if absent.
|
||||
#
|
||||
# `null` READS AS ABSENT, matching the `// empty` every call site used to carry.
|
||||
# A field the server left null and one it never sent mean the same thing to
|
||||
# every caller here, and the shells downstream test `[ -n "$x" ]`.
|
||||
# `null` READS AS ABSENT. A field the server left null and one it never sent
|
||||
# mean the same thing to every caller here, and the shells downstream test `[ -n "$x" ]`.
|
||||
scribe_json_pick() {
|
||||
printf '%s\n' "${1:-}" \
|
||||
| awk -F'\t' -v p="$2" '$2 == p { if ($3 != "null") print $3; exit }' 2>/dev/null \
|
||||
@@ -186,8 +179,7 @@ scribe_json_list() {
|
||||
# $1 flat text, $2 array path, $3 array path to SUBTRACT → the elements of $2
|
||||
# that are not in $3, in order. The reuse and sync channels are two classes of
|
||||
# one answer and each has its own session ledger, so the ids that go in the
|
||||
# reuse file are precisely `note_ids` minus `sync_note_ids` — jq wrote that as
|
||||
# `(.note_ids // []) - (.sync_note_ids // [])`.
|
||||
# reuse file are precisely `note_ids` minus `sync_note_ids`.
|
||||
scribe_json_list_minus() {
|
||||
local keep
|
||||
keep=$(scribe_json_list "$1" "$3")
|
||||
@@ -210,10 +202,8 @@ scribe_json_len() {
|
||||
# The server escapes every non-ASCII character this way (the JSON default),
|
||||
# and `sprintf("%c", n)` for n > 127 means a different thing in every awk and
|
||||
# locale: gawk in a UTF-8 locale writes the character, gawk in the C locale
|
||||
# and mawk write the single byte n % 256. So "·" (U+00B7) arrived as a lone
|
||||
# 0xB7 and "—" (U+2014) as 0x14 wherever the hook ran without a UTF-8 locale —
|
||||
# found by a test that ran the hook with a bare environment (milestone 500).
|
||||
# Under LC_ALL=C every awk writes exactly the byte asked for, so the encoding
|
||||
# and mawk write the single byte n % 256, so "·" (U+00B7) would arrive as a
|
||||
# lone 0xB7 wherever the hook runs without a UTF-8 locale. Under LC_ALL=C every awk writes exactly the byte asked for, so the encoding
|
||||
# below is the only one that happens.
|
||||
scribe_json_unescape() {
|
||||
LC_ALL=C awk '
|
||||
@@ -348,12 +338,12 @@ scribe_urlenc() {
|
||||
# ---------------------------------------------------------------------------
|
||||
# kind<TAB>name for each thing a piece of code DEFINES, in source order. One
|
||||
# program, two consumers: the local duplicate arm (every definition in the
|
||||
# payload) and the ledger feed (#2791, below: the definitions being written,
|
||||
# payload) and the ledger feed (below: the definitions being written,
|
||||
# or the one enclosing an Edit). Rule-for-rule mirrored by the server's
|
||||
# services/coverage.py extract_shapes — ledger rows are keyed by what THAT
|
||||
# sees, so the two must agree on what counts as a definition.
|
||||
#
|
||||
# THE WHOLE INPUT IS BUFFERED (#4222) so the span scan below can look ahead.
|
||||
# THE WHOLE INPUT IS BUFFERED so the span scan below can look ahead.
|
||||
# The matchers are line-oriented and know nothing about what a line is INSIDE:
|
||||
# a wrapped docstring beginning "class AND the …" announces a shape called
|
||||
# `AND`, which reaches the session mid-edit as a divergence prompt about a
|
||||
@@ -482,7 +472,7 @@ scribe_defs() {
|
||||
sub(/[^A-Za-z0-9_$].*$/, "", t)
|
||||
# `type` defines only when something follows the name (= or {); an
|
||||
# import specifier `type Foo,` is the same two words and defines
|
||||
# nothing (mirror of coverage.py, #2904).
|
||||
# nothing (mirror of coverage.py).
|
||||
if (line ~ /^type[[:space:]]/) {
|
||||
rest = line; sub(/^type[[:space:]]+[A-Za-z_$][A-Za-z0-9_$]*/, "", rest)
|
||||
if (rest !~ /[={]/) return
|
||||
@@ -508,12 +498,11 @@ scribe_defs() {
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ARM 1 — BY NAME, LOCALLY (#2280). Does a definition of this already exist?
|
||||
# ARM 1 — BY NAME, LOCALLY. Does a definition of this already exist?
|
||||
#
|
||||
# The recorded arms ask Scribe what was RECORDED; the ledger arm (#2900) asks
|
||||
# what a BOUND repo's ledger knows. A helper nobody recorded, in a repo nobody
|
||||
# bound, is invisible to both — which is how `.btn-primary` came to be defined
|
||||
# four times, already diverged. This arm asks the one question only the
|
||||
# The recorded arms ask Scribe what was RECORDED; the ledger arm asks what a
|
||||
# BOUND repo's ledger knows. A helper nobody recorded, in a repo nobody bound,
|
||||
# is invisible to both, so it gets defined again in the next file. This arm asks the one question only the
|
||||
# developer's machine can answer, inside the repo, holding the code about to
|
||||
# be written: no index, no storage, no server — it runs even on an install
|
||||
# that has never configured Scribe.
|
||||
@@ -521,23 +510,22 @@ scribe_defs() {
|
||||
# Definition-shaped patterns only. Grepping for bare occurrences would match
|
||||
# every CALL site and drown the real finding — and a hint that is mostly noise
|
||||
# is one people learn to skip, which is worse than none. ALL code, not a
|
||||
# language shortlist (#2682): the same keyword family scribe_defs announces.
|
||||
# language shortlist: the same keyword family scribe_defs announces.
|
||||
#
|
||||
# $1 repo root, $2 repo-relative path of the file being written (excluded from
|
||||
# the grep — it would always match itself on an Edit). Definitions on stdin.
|
||||
# Prints one "> - `name` is already defined in N other file(s): …" per hit.
|
||||
# ── The contract around a change (#4215, milestone 419) ───────────────────
|
||||
# ── The contract around a change ───────────────────────────────────────
|
||||
#
|
||||
# WHAT THIS ANSWERS. "You altered the arity, name or shape of something — here
|
||||
# is everything that reads it." Rule 33's interface-contract check, one scope
|
||||
# down: not between layers but between a definition and its callers.
|
||||
# is everything that reads it." An interface-contract check between a
|
||||
# definition and its callers.
|
||||
#
|
||||
# WHY IT IS A CHECK AND NOT A LESSON. #4207 was written — "widening a tuple is
|
||||
# an interface change to every unpack site, and the compiler will not tell
|
||||
# you" — hours before a structurally identical mistake was made by its author,
|
||||
# and it was surfaced twice in the turns before. Text delivered at the moment
|
||||
# of acting is too weak a carrier for a reflex that has to change what the act
|
||||
# IS. This looks it up instead.
|
||||
# WHY IT IS A CHECK AND NOT ADVICE. "Widening a tuple is an interface change to
|
||||
# every unpack site, and the compiler will not tell you" is easy to know and
|
||||
# easy to forget mid-edit. Text delivered at the moment of acting is too weak a
|
||||
# carrier for a reflex that has to change what the act IS, so this looks the
|
||||
# callers up instead.
|
||||
#
|
||||
# `scribe_exposed` — the names a CALLER can depend on, from a blob of code.
|
||||
# Three kinds, because a contract breaks three ways and they look nothing
|
||||
@@ -547,18 +535,14 @@ scribe_defs() {
|
||||
# arg its parameter names arity and order
|
||||
# key quoted keys of dict literals the shape of what it RETURNS
|
||||
#
|
||||
# The third is here because of the miss that produced this step. A config
|
||||
# function gained one dict key; three arms read that dict inside a fail-open
|
||||
# `except`, so every one of them silently became a no-op and ten tests went
|
||||
# red at once with nothing pointing at the cause. No signature changed. A
|
||||
# check that only watched signatures would have watched the wrong thing.
|
||||
# The third matters because a returned dict can change shape with no
|
||||
# signature changing: callers that read it inside a fail-open `except` go
|
||||
# silently inert, with nothing pointing at the cause.
|
||||
scribe_exposed() {
|
||||
# The `sym` half DELEGATES to scribe_defs rather than repeating its patterns.
|
||||
# Those patterns cover nine languages and have been corrected several times
|
||||
# (the Go receiver form, the `type` import-specifier false positive, the
|
||||
# dunder skip); a second copy here would inherit today's version and then
|
||||
# quietly stop agreeing with it, which is #3497's history for the two rule
|
||||
# arms. One reader, called twice.
|
||||
# Those patterns cover nine languages and carry their own edge cases (the Go
|
||||
# receiver form, the `type` import specifier, the dunder skip); a second copy
|
||||
# here would drift from them. One reader, called twice.
|
||||
local blob
|
||||
blob=$(cat)
|
||||
{
|
||||
@@ -612,7 +596,7 @@ scribe_exposed() {
|
||||
# not match `cfg_path`, and the defining file is excluded — a definition is
|
||||
# always its own first mention and listing it says nothing.
|
||||
#
|
||||
# `|| true` INSIDE the substitution for the reason #4042 records at
|
||||
# `|| true` INSIDE the substitution, for the reason given at
|
||||
# scribe_local_dups: under `pipefail` a `head` that exits early kills the
|
||||
# still-writing git grep, and an outer fallback then wipes the hits head had
|
||||
# already printed.
|
||||
@@ -679,7 +663,7 @@ scribe_contract_block() {
|
||||
[ -n "$gained" ] && printf '> gained: %s\n' "$gained"
|
||||
[ -n "$lost" ] && printf '> lost: %s\n' "$lost"
|
||||
printf '> read by: %s\n' "$(printf '%s' "$readers" | tr '\n' ' ' | sed 's/ $//')"
|
||||
printf '> A caller that passes or reads the old shape keeps compiling and fails only when that line runs (lesson #4207). Read them before moving on.\n'
|
||||
printf '> A caller that passes or reads the old shape keeps compiling and fails only when that line runs. Read them before moving on.\n'
|
||||
[ -n "$ledger" ] && printf '%s\n' "$subject" >> "$ledger" 2>/dev/null
|
||||
return 0
|
||||
}
|
||||
@@ -705,8 +689,7 @@ scribe_slippage_lines() {
|
||||
# is the moment the exclusion ledgers are about to be cleared and the moment
|
||||
# their TTL has usually already eaten the early part of a long session. A
|
||||
# readout built on them would report only the last stretch of the session
|
||||
# and read as though it had reported all of it — the #3311 shape, and the
|
||||
# one this milestone exists to stop producing.
|
||||
# and read as though it had reported all of it.
|
||||
named=$(scribe_ledger_kept "$dir/${sid}.rules.keep.ids")
|
||||
opened=$(scribe_ledger_kept "$dir/${sid}.opened.keep.ids")
|
||||
acted=$(scribe_ledger_kept "$dir/${sid}.acted.keep.ids")
|
||||
@@ -751,38 +734,33 @@ scribe_ids_minus() {
|
||||
' </dev/null 2>/dev/null
|
||||
}
|
||||
|
||||
# ARM 1, BY NAME (#2280) — and the confirmation pass that makes it mean
|
||||
# something (#4227).
|
||||
# ARM 1, BY NAME — and the confirmation pass that makes it mean something.
|
||||
#
|
||||
# WHY A GREP IS NOT ENOUGH. The pattern below looks for a definition keyword
|
||||
# followed by the name. A grep sees LINES, not spans, so the sentence
|
||||
# followed by the name. A grep sees LINES, not spans, so a docstring sentence
|
||||
# like
|
||||
#
|
||||
# class with only modifier rules is a deletion that went half-way.
|
||||
#
|
||||
# — real prose, from the module docstring of scripts/check_dangling_styles.py —
|
||||
# matches it for `name=with`. #4222 fixed the other end of this same defect, in
|
||||
# the extractors that decide what a payload DEFINES; this is the end that
|
||||
# decides which other files already define it, and it was still a plain grep.
|
||||
# matches it for `name=with`.
|
||||
#
|
||||
# SO EVERY HIT IS CONFIRMED by running the real extractor over the candidate
|
||||
# file and keeping only names it actually reports. That is the honest check and
|
||||
# the only one that cannot disagree with the other end of the pipe.
|
||||
#
|
||||
# TIGHTENING THE PATTERN WOULD HAVE BEEN CHEAPER AND WRONG. Requiring `(` or
|
||||
# TIGHTENING THE PATTERN WOULD BE CHEAPER AND WRONG. Requiring `(` or
|
||||
# `{` or `:` after the name rejects `class with only…` — and also rejects
|
||||
# `class Foo extends Bar {`, `class Foo : Base()` and `type Foo struct {`. This
|
||||
# arm's whole justification (#2280, #2682) is that it works with no server, no
|
||||
# arm's whole justification is that it works with no server, no
|
||||
# index and no binding, which makes a miss here invisible. Trading a visible
|
||||
# false positive for an invisible false negative is a bad trade.
|
||||
#
|
||||
# ONCE PER DISTINCT FILE, not once per (name, file) pair: the same file is
|
||||
# usually a candidate for several names at once, and the extractor reads the
|
||||
# whole file either way. Measured on this repo, a deliberately pathological
|
||||
# payload — nine names that are ordinary English words — produced 28 distinct
|
||||
# candidate files totalling 947KB, and `scribe_defs` runs at roughly 33ms per
|
||||
# 250KB, so the confirmation costs about 200ms in the worst case anyone has
|
||||
# been able to construct here. The per-(name, file) shape would have paid that
|
||||
# several times over for the same answer.
|
||||
# whole file either way. `scribe_defs` runs at roughly 33ms per 250KB, so
|
||||
# even a pathological payload of ordinary-word names costs a few hundred ms to
|
||||
# confirm; the per-(name, file) shape would pay that several times over for
|
||||
# the same answer.
|
||||
_SCRIBE_DUP_CANDIDATES=12
|
||||
_SCRIBE_DUP_SHOWN=4
|
||||
_SCRIBE_DUP_BUDGET=48
|
||||
@@ -794,12 +772,10 @@ scribe_local_dups() {
|
||||
|
||||
# PASS 1 — candidates. One grep per name, unchanged except for the cap.
|
||||
#
|
||||
# THE CAP IS RAISED FROM FOUR, and that is not a detail. Confirmation REMOVES
|
||||
# hits, so capping before it runs lets three phantom matches crowd out a real
|
||||
# definition in the fourth file — hits dropped before anyone looked at them,
|
||||
# which is #4042's bug wearing a different hat. The display cap stays at four
|
||||
# (_SCRIBE_DUP_SHOWN); it now applies to CONFIRMED hits, which is where a cap
|
||||
# belongs.
|
||||
# THE CANDIDATE CAP IS WIDER THAN THE DISPLAY CAP. Confirmation REMOVES
|
||||
# hits, so capping before it runs would let phantom matches crowd out a real
|
||||
# definition. The display cap (_SCRIBE_DUP_SHOWN) applies to CONFIRMED hits,
|
||||
# which is where a cap belongs.
|
||||
while IFS=$'\t' read -r kind name; do
|
||||
[ -n "${name:-}" ] || continue
|
||||
case "$kind" in
|
||||
@@ -807,11 +783,11 @@ scribe_local_dups() {
|
||||
*) pat="(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+${name}[^A-Za-z0-9_]|func[[:space:]]*\([^)]*\)[[:space:]]*${name}[[:space:]]*\(|(const|let)[[:space:]]+${name}[[:space:]]*=" ;;
|
||||
esac
|
||||
# -I skips binaries; :(exclude) drops the file being written.
|
||||
# `|| true` INSIDE the substitution, not `|| hits=""` outside it (#4042):
|
||||
# under the hooks' `pipefail`, `head` exiting early kills a git grep that
|
||||
# is still writing, the pipeline reports SIGPIPE, and an outer fallback
|
||||
# then wipes the hits head had already printed. The name most duplicated —
|
||||
# the one this arm exists for — was the one it dropped.
|
||||
# `|| true` INSIDE the substitution, not `|| hits=""` outside it: under
|
||||
# the hooks' `pipefail`, `head` exiting early kills a git grep that is
|
||||
# still writing, the pipeline reports SIGPIPE, and an outer fallback would
|
||||
# wipe the hits head had already printed — dropping exactly the most
|
||||
# duplicated name.
|
||||
hits=$(git -C "$root" grep -I -l -E -e "$pat" -- . ":(exclude)${rel}" 2>/dev/null \
|
||||
| head -"$_SCRIBE_DUP_CANDIDATES" || true)
|
||||
[ -n "$hits" ] || continue
|
||||
@@ -868,19 +844,9 @@ scribe_local_dups() {
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The blind spot made visible (#2932). Both write-path hooks fail OPEN when the
|
||||
# instance is slow or down — right for noise, wrong for silence: a session
|
||||
# cannot tell "the ledger checked and found nothing" from "the ledger never
|
||||
# answered", and a self-surfacing system cannot afford an invisible miss (the
|
||||
# first write after a redeploy lost its derive line to a 4s cold start and
|
||||
# nobody knew). So a failed call says so — ONCE per outage: the marker holds
|
||||
# the time it last spoke; within ten minutes of that it stays quiet, and a
|
||||
# successful call clears it so the next outage announces itself afresh.
|
||||
# Unconfigured installs never reach this: no URL/token means no call was owed.
|
||||
# Where every hook gets its endpoint and credential. Four lines, and each of
|
||||
# the five hooks carried its own copy until #2278 — which is exactly the
|
||||
# missing-sibling shape: the `${...}` guard below is a correctness detail a
|
||||
# sixth hook would have forgotten, and nothing would have failed loudly.
|
||||
# Where every hook gets its endpoint and credential. One copy, because the
|
||||
# `${...}` guard below is a correctness detail a hook with its own copy would
|
||||
# forget, and nothing would fail loudly.
|
||||
#
|
||||
# Sets `url` and `token` as globals rather than echoing them: a token must not
|
||||
# pass through a subshell's output, where it could land in a log or an `xtrace`
|
||||
@@ -905,6 +871,14 @@ scribe_config() {
|
||||
[ -n "$url" ] && [ -n "$token" ]
|
||||
}
|
||||
|
||||
# The blind spot made visible. Both write-path hooks fail OPEN when the
|
||||
# instance is slow or down — right for noise, wrong for silence: a session
|
||||
# cannot tell "the ledger checked and found nothing" from "the ledger never
|
||||
# answered" (a cold start after a redeploy is enough to miss the budget). So a
|
||||
# failed call says so — ONCE per outage: the marker holds the time it last
|
||||
# spoke; within ten minutes of that it stays quiet, and a successful call
|
||||
# clears it so the next outage announces itself afresh. Unconfigured installs
|
||||
# never reach this: no URL/token means no call was owed.
|
||||
_SCRIBE_UNREACHED_QUIET=600
|
||||
|
||||
scribe_unreached() {
|
||||
@@ -924,19 +898,18 @@ scribe_reached() {
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# THE RULE EXCLUSION LEDGER, and how entries in it AGE (#3751).
|
||||
# THE RULE EXCLUSION LEDGER, and how entries in it AGE.
|
||||
#
|
||||
# Two hooks write this file and two read it, which is the whole reason the
|
||||
# parsing lives here: `scribe_prior_art.sh` and `scribe_tool_rules.sh` share
|
||||
# one ledger so a rule named by one arm is not re-offered by the other, and a
|
||||
# format only one of them understood would break that on the first read.
|
||||
#
|
||||
# WHAT PROBLEM AGEING SOLVES. #3749 clears the ledger when an EVENT destroys
|
||||
# WHAT PROBLEM AGEING SOLVES. The ledger is cleared when an EVENT destroys
|
||||
# context — a compaction or a /clear. This is the case with no event at all: a
|
||||
# long session where the rule was named two hundred turns ago and has simply
|
||||
# fallen out of attention. It is #3702's argument at the tier level (present in
|
||||
# context and salient at the moment are different properties) applied to time
|
||||
# instead of to tier.
|
||||
# fallen out of attention. Present in context and salient at the moment are
|
||||
# different properties.
|
||||
#
|
||||
# PER-ENTRY TIMESTAMPS, NOT A FILE MTIME. Clearing the whole ledger when the
|
||||
# file is old is one line of shell and wrong in exactly the session that needs
|
||||
@@ -952,12 +925,10 @@ scribe_reached() {
|
||||
# which produces one extra full line per rule and no other harm.
|
||||
_SCRIBE_RULE_TTL=2700
|
||||
|
||||
# 45 MINUTES, and the reasoning rather than the number (rule 32).
|
||||
# 45 MINUTES, and the reasoning rather than the number.
|
||||
#
|
||||
# There is no data on this yet, so it is a judgement made to be revised — the
|
||||
# telemetry that would settle it is the one #3807 just built, and a reading of
|
||||
# how often an aged-out rule gets PULLED after it returns is what should move
|
||||
# this.
|
||||
# It is a judgement made to be revised: how often an aged-out rule gets PULLED
|
||||
# after it returns is the reading that should move it.
|
||||
#
|
||||
# Too short and the exclusion stops existing and the repetition it prevents
|
||||
# comes back. Too long and it never fires at all in a session short enough to
|
||||
@@ -966,11 +937,10 @@ _SCRIBE_RULE_TTL=2700
|
||||
# it governs, short enough that a multi-hour session gets a genuine refresh
|
||||
# rather than one 9am mention.
|
||||
#
|
||||
# Being wrong on the short side is now the cheaper error, which is why this
|
||||
# leans short. Since #3750 an excluded rule is REFERENCED rather than withheld,
|
||||
# so the ledger is no longer the only thing standing between a session and a
|
||||
# rule it has forgotten — an expired entry costs one full line instead of one
|
||||
# short one, and the exclusion re-arms the moment it is spent.
|
||||
# Being wrong on the short side is the cheaper error, which is why this leans
|
||||
# short. An excluded rule is REFERENCED rather than withheld, so an expired
|
||||
# entry costs one full line instead of one short one, and the exclusion
|
||||
# re-arms the moment it is spent.
|
||||
|
||||
# Live ids from a ledger, comma-joined for `exclude_rule_ids`. Empty output for
|
||||
# a missing, empty or fully-aged file — the callers already treat "" as "send
|
||||
@@ -983,13 +953,10 @@ _SCRIBE_RULE_TTL=2700
|
||||
# the loudest possible failure, from the mechanism meant to quieten things.
|
||||
# Appends are chronological, so the last line for an id is its most recent.
|
||||
#
|
||||
# A BARE ID — no tab, no timestamp — IS LIVE. That is the pre-#3751 format, and
|
||||
# a session in flight when this ships has a ledger full of them. Treating
|
||||
# unknown as expired would make every one of those sessions re-announce every
|
||||
# rule it had already been told, all at once, which is precisely the noise this
|
||||
# exists to prevent. Unknown means "not measured" and never "old" — the same
|
||||
# null discipline the retrieval_logs columns use. Those entries simply never
|
||||
# age, which is bounded: the session ends.
|
||||
# A BARE ID — no tab, no timestamp — IS LIVE. Unknown means "not measured" and
|
||||
# never "old": treating it as expired would re-announce every such rule at
|
||||
# once, which is precisely the noise this exists to prevent. Those entries
|
||||
# simply never age, which is bounded: the session ends.
|
||||
scribe_rules_live() {
|
||||
local f="$1" now
|
||||
[ -n "$f" ] && [ -f "$f" ] || return 0
|
||||
@@ -1017,7 +984,7 @@ scribe_rules_live() {
|
||||
# Append surfaced ids, stamped. Reads ids on stdin, one per line — the shape
|
||||
# `scribe_json_list "$flat" '.rule_ids'` already produces at both call sites.
|
||||
scribe_rules_append() {
|
||||
# Writes TWO files, and the second one is the point (#4217).
|
||||
# Writes TWO files, and the second one is the point.
|
||||
#
|
||||
# `$f` is an EXCLUSION ledger: it answers "does this context already hold
|
||||
# this?", so it is aged by TTL and swept at every compaction — after a
|
||||
@@ -1027,13 +994,11 @@ scribe_rules_append() {
|
||||
# The twin is an EVIDENCE ledger: it answers "did this happen?", which no
|
||||
# compaction can make untrue. It is never aged and never swept.
|
||||
#
|
||||
# These are opposite lifetimes and `.opened.ids` was serving both, which is
|
||||
# how a session with 45 `get_rule` calls came to report none: the compaction
|
||||
# cleared the ledger that was also the record. Measured across six sessions
|
||||
# on this instance — 208 opens, 3 surviving — before the split.
|
||||
# These are opposite lifetimes, so one file cannot serve both: a compaction
|
||||
# would clear the ledger that is also the record.
|
||||
#
|
||||
# DERIVED HERE rather than listed at the call sites, because a list is what
|
||||
# broke this before (see `scribe_clear_session_ledgers`). Every ledger
|
||||
# DERIVED HERE rather than listed at the call sites, because a list goes
|
||||
# stale (see `scribe_clear_session_ledgers`). Every ledger
|
||||
# written through this function gets its twin, including the next one
|
||||
# somebody adds. The cost is a second small file per ledger per session.
|
||||
local f="$1" now
|
||||
@@ -1055,19 +1020,7 @@ scribe_ledger_kept() {
|
||||
} END { if (out != "") print out }' "$f" 2>/dev/null || true
|
||||
}
|
||||
|
||||
# The OPENED ledger's contribution to a rule arm's query string (#4100).
|
||||
#
|
||||
# TWO LEDGERS, BECAUSE THEY RECORD TWO DIFFERENT FACTS. `.rules.ids` holds
|
||||
# every id an arm has NAMED; `.opened.ids` holds the ids the session actually
|
||||
# read, written by scribe_record_opened.sh from the `get_rule` call itself.
|
||||
# Named is not read: the injected line is a teaser, and one skimmed past
|
||||
# leaves nothing behind — least of all across a compaction. Sending both lets
|
||||
# the server tell a reader who opened a rule from one who was only shown it,
|
||||
# instead of telling them both the same untrue thing.
|
||||
#
|
||||
# Same reader as the naming ledger on purpose, so ageing, the last-entry-wins
|
||||
# rule and the bare-id format are defined once and cannot drift apart.
|
||||
# ── The pre-act checkpoint's session ledger (#4214, milestone 419) ─────────
|
||||
# ── The pre-act checkpoint's session ledger ──────────────────────────────
|
||||
#
|
||||
# A checkpoint STOPS an act rather than annotating it, so unlike every other
|
||||
# ledger here its job is to make sure the same stop cannot happen twice. Two
|
||||
@@ -1106,7 +1059,7 @@ scribe_checkpoint_allowed() {
|
||||
n=$(grep -c '^[0-9][0-9]*$' "$f" 2>/dev/null || printf '0')
|
||||
# `grep -c` over a missing file can print nothing; a bare arithmetic test
|
||||
# on an empty string is a syntax error in some shells and silently true in
|
||||
# others, which is how a cap comes to cap nothing (#3191's shape).
|
||||
# others, which is how a cap comes to cap nothing.
|
||||
n=$(printf '%s' "$n" | tr -cd '0-9')
|
||||
[ -n "$n" ] || n=0
|
||||
[ "$n" -ge "$_SCRIBE_CHECKPOINT_CAP" ] && return 1
|
||||
@@ -1135,6 +1088,18 @@ scribe_json_deny() {
|
||||
"$1" "$esc"
|
||||
}
|
||||
|
||||
# The OPENED ledger's contribution to a rule arm's query string.
|
||||
#
|
||||
# TWO LEDGERS, BECAUSE THEY RECORD TWO DIFFERENT FACTS. `.rules.ids` holds
|
||||
# every id an arm has NAMED; `.opened.ids` holds the ids the session actually
|
||||
# read, written by scribe_record_opened.sh from the `get_rule` call itself.
|
||||
# Named is not read: the injected line is a teaser, and one skimmed past
|
||||
# leaves nothing behind — least of all across a compaction. Sending both lets
|
||||
# the server tell a reader who opened a rule from one who was only shown it,
|
||||
# instead of telling them both the same untrue thing.
|
||||
#
|
||||
# Same reader as the naming ledger on purpose, so ageing, the last-entry-wins
|
||||
# rule and the bare-id format are defined once and cannot drift apart.
|
||||
scribe_held_query() {
|
||||
local ids
|
||||
ids=$(scribe_rules_live "$1")
|
||||
@@ -1142,8 +1107,8 @@ scribe_held_query() {
|
||||
return 0
|
||||
}
|
||||
|
||||
# The Systems whose rulings this session was already shown in full (milestone
|
||||
# 444), as the `seen_ruling_systems` query both PreToolUse hooks send. A flat
|
||||
# The Systems whose rulings this session was already shown in full, as the
|
||||
# `seen_ruling_systems` query both PreToolUse hooks send. A flat
|
||||
# read, not aged like the rule file: a ruling binds for the whole session, and
|
||||
# the server already answers a repeat with one short reference line. The file
|
||||
# follows the ledger name convention, so a compaction clears it and the
|
||||
@@ -1156,20 +1121,11 @@ scribe_rulings_query() {
|
||||
return 0
|
||||
}
|
||||
|
||||
# Drop EVERY per-session ledger, matched by convention rather than listed (#4101).
|
||||
# Drop EVERY per-session ledger, matched by convention rather than listed.
|
||||
#
|
||||
# A LIST IS THE BUG. Until now the compact/clear branch named its files one at
|
||||
# a time, and it named two of the five: `.rules.ids` and `.opened.ids` were
|
||||
# cleared while `.ids` (notes), `.sync.ids` (shape signals) and `.derive.ids`
|
||||
# survived. So milestone 386's defect — "a compaction destroys the context but
|
||||
# not the ledger, so the most applicable records become permanently
|
||||
# unreachable mid-session" — was fixed for rules and left standing on the note
|
||||
# and snippet surfaces, which are the ones that fire most often.
|
||||
#
|
||||
# Nobody decided that. The list was written when rules were the only ledger
|
||||
# that mattered and was never revisited when the others arrived, which is what
|
||||
# a hand-maintained list of "things to remember to clean up" does. Adding three
|
||||
# more `rm` lines would rebuild the same trap for the sixth ledger.
|
||||
# A compaction destroys the context, so a ledger that survives it makes the
|
||||
# records it lists unreachable for the rest of the session. A hand-kept list
|
||||
# of files to clear misses whichever ledger was added after it was written.
|
||||
#
|
||||
# So the rule is the NAME: a per-session ledger is `<sid>[.<kind>].ids`, and
|
||||
# everything matching that goes. A new ledger following the convention is
|
||||
@@ -1180,21 +1136,19 @@ scribe_rulings_query() {
|
||||
# compact rather than discarded can still live in these directories without
|
||||
# being swept away by a glob that was never told about it. `<sid>.unreached`
|
||||
# is the live example: it records that the instance could not be reached, not
|
||||
# what the session holds, and #2932 needs it to outlive a compaction.
|
||||
# what the session holds, and the once-per-outage line needs it to outlive a
|
||||
# compaction.
|
||||
#
|
||||
# TWO DIRECTORIES, WHICH IS ITS OWN LESSON. The first cut of this swept only
|
||||
# `scribe-priorart` — every ledger named in the hooks was there, so the list
|
||||
# looked complete. `scribe_autoinject.sh` keeps its note ledger in
|
||||
# `scribe-autoinject`, so the arm that fires most (598 calls in five days) was
|
||||
# the one the clear could not reach, and the fix read as finished. Hence the
|
||||
# roster here rather than a path at the call site: one place to add to, and
|
||||
# the tests read THIS string rather than a copy of it.
|
||||
# TWO DIRECTORIES: `scribe_autoinject.sh` keeps its note ledger in
|
||||
# `scribe-autoinject`, apart from the prior-art hooks'. The roster lives here
|
||||
# rather than as a path at each call site: one place to add to, and the tests
|
||||
# read THIS string rather than a copy of it.
|
||||
SCRIBE_LEDGER_DIRS="scribe-priorart scribe-autoinject"
|
||||
|
||||
# THE WRITTEN-SHAPES LEDGER (milestone 439). Every definition a write names,
|
||||
# THE WRITTEN-SHAPES LEDGER. Every definition a write names,
|
||||
# appended as `path<TAB>kind<TAB>name`, for the Stop hook to ask about at the
|
||||
# end of the turn (scribe_shape_check.sh). Its own file — one file, one
|
||||
# question (lesson #4226) — under the prior-art directory, so the session
|
||||
# question — under the prior-art directory, so the session
|
||||
# clear above already sweeps it. Read `kind<TAB>name` lines on stdin (the
|
||||
# scribe_defs format).
|
||||
#
|
||||
@@ -1226,7 +1180,7 @@ scribe_written_append() {
|
||||
return 0
|
||||
}
|
||||
|
||||
# THE REPLY-SHAPE LEDGER (milestone 500). Which default reply shapes this
|
||||
# THE REPLY-SHAPE LEDGER. Which default reply shapes this
|
||||
# session has been shown IN FULL — `core`, `completion`, `asks`, `plan` — one
|
||||
# key per line in <sid>.shapes.ids under the prior-art directory. The server
|
||||
# sends a shape in full when its key is absent and as a one-line reminder when
|
||||
@@ -1274,10 +1228,10 @@ scribe_clear_session_ledgers() {
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# WHAT HAPPENED IN THE LAST TURN of a transcript (#4107): scribe_turn.awk's
|
||||
# WHAT HAPPENED IN THE LAST TURN of a transcript: scribe_turn.awk's
|
||||
# facts — bounded, closed, task_ids, reply — one `KEY<TAB>VALUE` per line.
|
||||
# Shared by the two Stop hooks that read a turn: the report check and the
|
||||
# reply check (milestone 458). Empty output when the turn cannot be bounded.
|
||||
# reply check. Empty output when the turn cannot be bounded.
|
||||
#
|
||||
# The turn is parsed once. A window of recent lines, flattened record by
|
||||
# record and then read by scribe_turn.awk, which carries the turn-bounding
|
||||
@@ -1289,9 +1243,8 @@ scribe_clear_session_ledgers() {
|
||||
# WHERE THE TURN STARTS, FOUND BEFORE PARSING RATHER THAN AFTER. The window is
|
||||
# 3000 lines and routinely 7MB, of which a turn is the last few hundred lines
|
||||
# and about a sixth of the bytes — the rest is tool results these checks never
|
||||
# look at. The predecessor parsed all of it and threw most away, which jq
|
||||
# could afford and a parser written in awk cannot: measured at 6.5s for a 7MB
|
||||
# window against 94ms, on a hook that runs at the end of every turn.
|
||||
# look at. Parsing all of it in awk takes seconds (6.5s for a 7MB window,
|
||||
# against 94ms narrowed first), on a hook that runs at the end of every turn.
|
||||
#
|
||||
# So grep — C, and reading a FIXED string — narrows first. A prompt record is
|
||||
# `"type":"user"` whose `content` is a STRING; a tool result is the same type
|
||||
@@ -1342,14 +1295,12 @@ scribe_turn_fact() {
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# WHICH PROJECT IS THIS DIRECTORY'S? (#4085)
|
||||
# WHICH PROJECT IS THIS DIRECTORY'S?
|
||||
#
|
||||
# Every hook here scopes its request to a project, and until this existed all
|
||||
# six did it the same single way: `git remote get-url origin`, resolved
|
||||
# server-side through the repo bindings. That works well inside a bound repo
|
||||
# and not at all outside one — a session in a plain directory got no project
|
||||
# scope from ANY hook, silently, because the one key the whole chain turns on
|
||||
# only exists in a git repo.
|
||||
# Every hook here scopes its request to a project. The first key is `git
|
||||
# remote get-url origin`, resolved server-side through the repo bindings. That
|
||||
# works inside a bound repo and not at all outside one, where a session would
|
||||
# get no project scope from any hook.
|
||||
#
|
||||
# The second key is a `.scribe` file in the directory (or above it, the way
|
||||
# git finds its root), naming the project the work belongs to. It is a
|
||||
|
||||
Reference in New Issue
Block a user