Merge pull request 'Self-surfacing DRY — duplicate families named at the write and on arrival, tool-agnostic (milestone 299 steps 1–4)' (#125) from dev into main
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 23s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 58s
CI & Build / Build & push image (push) Successful in 14s

This commit was merged in pull request #125.
This commit is contained in:
2026-08-22 13:41:24 -04:00
21 changed files with 1046 additions and 127 deletions
+2
View File
@@ -172,6 +172,8 @@ endpoint at `/mcp`, not these REST routes.
| GET | `/api/plugin/context` | SessionStart context payload (rules + active-project) | | GET | `/api/plugin/context` | SessionStart context payload (rules + active-project) |
| GET | `/api/plugin/retrieve` | Title-first knowledge-injection candidates | | GET | `/api/plugin/retrieve` | Title-first knowledge-injection candidates |
| GET | `/api/plugin/processes` | Stored Processes for skill-stub sync | | GET | `/api/plugin/processes` | Stored Processes for skill-stub sync |
| GET | `/api/plugin/prior-art` | Write-path hint for the plugin hooks (params: `path`, `code`, `repo`, `shapes`, `exclude_ids`, `exclude_sync_ids`, `exclude_derive`); returns `context`, `note_ids`, `sync_note_ids`, `stamped`, `divergence`, `derive`, `derive_keys` |
| GET / POST | `/api/projects/<id>/coverage`, `…/coverage/refresh` | Shape-ledger accounting (`pattern_coverage` line, counts, `derive_groups`, `derive_new`, `divergence`, `recheck`) |
| GET / PUT | `/api/plugin/marketplace-url` | Read / set the plugin marketplace URL | | GET / PUT | `/api/plugin/marketplace-url` | Read / set the plugin marketplace URL |
## Dashboard, Export, Trash, Users ## Dashboard, Export, Trash, Users
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "scribe", "name": "scribe",
"description": "Scribe system-of-record for Claude Code: MCP tools over your notes/tasks/projects/rules, a session-start push channel that surfaces your always-on rules + active-project context, process-skills (writing-plans, systematic-debugging, verification, brainstorming, reusing-code), and your saved Scribe Processes auto-surfaced as skills (/scribe:sync). Replaces superpowers + file-memory with one app-backed plugin.", "description": "Scribe system-of-record for Claude Code: MCP tools over your notes/tasks/projects/rules, a session-start push channel that surfaces your always-on rules + active-project context, process-skills (writing-plans, systematic-debugging, verification, brainstorming, reusing-code), and your saved Scribe Processes auto-surfaced as skills (/scribe:sync). Replaces superpowers + file-memory with one app-backed plugin.",
"version": "0.1.38", "version": "0.1.39",
"author": { "name": "Bryan Van Deusen" }, "author": { "name": "Bryan Van Deusen" },
"mcpServers": { "mcpServers": {
"scribe": { "scribe": {
+11 -1
View File
@@ -52,8 +52,18 @@ On install you'll be asked for:
but never stop it; silent when nothing is recorded, which is most of the time. but never stop it; silent when nothing is recorded, which is most of the time.
Two framings: a REUSE menu (similar/nearby records), and a SYNC nudge when a Two framings: a REUSE menu (similar/nearby records), and a SYNC nudge when a
snippet records the exact file being edited — "updating the record is part of snippet records the exact file being edited — "updating the record is part of
the edit" — each with its own once-per-session dedup. the edit" — each with its own once-per-session dedup. A third, ledger-fed
line names a duplicate family (no canon) or a canon recorded elsewhere for
the names being written (its own dedup channel, `exclude_derive`).
Toggle in **Settings → Knowledge auto-inject**. Toggle in **Settings → Knowledge auto-inject**.
- `hooks/hooks.json` → PostToolUse hook on `Bash`
(`hooks/scribe_after_write.sh`): code written through sed/heredocs/scripts
never reaches the PreToolUse hook, so this one diffs the working tree after
every Bash call (per-session path+blob snapshot; one `git status` when
nothing changed) and runs the same arms on the definitions just written,
through the same endpoint and the same dedup channels. `additionalContext`
only; silent on any failure. The extractor, the prose/data skip list and the
local by-name duplicate arm are shared in `hooks/scribe_defs.sh`.
- `skills/` → the universal process-skills, surfaced by description match. - `skills/` → the universal process-skills, surfaced by description match.
- `hooks/scribe_sync_processes.sh` (a 2nd SessionStart hook) + the `/scribe:sync` - `hooks/scribe_sync_processes.sh` (a 2nd SessionStart hook) + the `/scribe:sync`
command → generate `~/.claude/skills/scribe-proc-*` stubs from your Scribe command → generate `~/.claude/skills/scribe-proc-*` stubs from your Scribe
+11
View File
@@ -34,6 +34,17 @@
} }
] ]
} }
],
"PostToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/scribe_after_write.sh\""
}
]
}
] ]
} }
} }
+221
View File
@@ -0,0 +1,221 @@
#!/usr/bin/env bash
# Scribe plugin — PostToolUse write-path trigger on Bash (#2901).
#
# scribe_prior_art.sh fires before a Write/Edit TOOL CALL. Code written any
# other way — sed, heredocs, python edit scripts, `cat > file` — never reached
# it, so a whole class of edits (the ones a long session makes most) got no
# prior-art hint, no ledger feed and no duplicate-family warning. This hook
# closes that: after EVERY Bash call it asks git what changed in the working
# tree since it last looked, and runs the same arms on the definitions that
# were just written — the local by-name duplicate arm, the recorded prior-art
# arms and the ledger's derive/divergence checks (#2900/#2793), via the same
# /api/plugin/prior-art endpoint the pre-write hook uses.
#
# Post-hoc by a few seconds, in the same moment and the same session: "the
# copy just landed; here is its family" — not "an audit found it later".
#
# Cheap when nothing changed: one `git status`. State per session, beside the
# pre-write hook's (its three dedup channels are SHARED, so a family named by
# one hook is not named again by the other):
# ${TMPDIR:-/tmp}/scribe-afterwrite/<sid>.snap path<TAB>blob-hash of every
# dirty/untracked file last seen
# ${TMPDIR:-/tmp}/scribe-priorart/<sid>.* the dedup channels
#
# NEVER BLOCKS. It returns `additionalContext` only (no decision — there is
# nothing left to decide, the write already happened). Any failure —
# unconfigured, unreachable, not a git repo, malformed — exits 0 in silence.
#
# Config (same as the other hooks):
# CLAUDE_PLUGIN_OPTION_API_ENDPOINT base URL, no trailing slash
# CLAUDE_PLUGIN_OPTION_API_TOKEN fmcp_ API key (sensitive)
# SCRIBE_URL / SCRIBE_TOKEN override for the settings.json dogfooding path.
set -uo pipefail
command -v jq >/dev/null 2>&1 || exit 0
command -v git >/dev/null 2>&1 || exit 0
# shellcheck source=plugin/hooks/scribe_defs.sh
. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"
# PostToolUse delivers { session_id, cwd, tool_name, tool_input, tool_response }.
event=$(cat 2>/dev/null || true)
tool_name=$(printf '%s' "$event" | jq -r '.tool_name // empty' 2>/dev/null) || exit 0
[ "$tool_name" = "Bash" ] || exit 0
session_id=$(printf '%s' "$event" | jq -r '.session_id // empty' 2>/dev/null) || session_id=""
event_cwd=$(printf '%s' "$event" | jq -r '.cwd // empty' 2>/dev/null) || event_cwd=""
work_dir=${event_cwd:-${CLAUDE_PROJECT_DIR:-$PWD}}
repo_root=$(git -C "$work_dir" rev-parse --show-toplevel 2>/dev/null) || exit 0
[ -n "$repo_root" ] || exit 0
safe_sid=$(printf '%s' "${session_id:-nosession}" | tr -c 'A-Za-z0-9._-' '_')
snap_dir="${TMPDIR:-/tmp}/scribe-afterwrite"
mkdir -p "$snap_dir" 2>/dev/null || true
snap="$snap_dir/${safe_sid}.snap"
# What is dirty now: every modified / added / untracked path, with the blob
# hash of its working-tree content. Hash, not mtime: portable (no stat
# flags), exact (a touch is not a change), and untracked files hash the same
# way tracked ones do.
current=""
while IFS= read -r line; do
[ -n "$line" ] || continue
status=${line:0:2}
path=${line:3}
case "$status" in
D*|*D) continue ;; # a deletion defines nothing
esac
case "$path" in
*" -> "*) path=${path##* -> } ;; # rename: the new name
esac
# Porcelain quotes paths with special characters; those are skipped rather
# than unquoted badly — a filename needing quotes is not where shapes live.
case "$path" in
\"*) continue ;;
esac
[ -f "$repo_root/$path" ] || continue
sha=$(git -C "$repo_root" hash-object -- "$path" 2>/dev/null) || continue
current="${current}${path}"$'\t'"${sha}"$'\n'
done < <(git -C "$repo_root" status --porcelain --untracked-files=all 2>/dev/null)
previous=""
[ -f "$snap" ] && previous=$(cat "$snap" 2>/dev/null || true)
first_run=0
[ -f "$snap" ] || first_run=1
# Write the new snapshot NOW, before anything can fail below — the next call
# must compare against this tree, whatever happens to this one's hint.
printf '%s' "$current" > "$snap" 2>/dev/null || true
# Changed = a (path, hash) pair not in the previous snapshot. On the very
# first call of a session there is no previous snapshot; rather than report
# every pre-existing dirty file as "just written", take only files touched in
# the last minute — the Bash call that just ran is the likely author.
changed=""
while IFS=$'\t' read -r path sha; do
[ -n "${path:-}" ] || continue
if [ "$first_run" = 1 ]; then
[ -n "$(find "$repo_root/$path" -mmin -1 2>/dev/null)" ] || continue
else
case "$previous" in
*"${path}"$'\t'"${sha}"*) continue ;;
esac
fi
scribe_skip_path "$path" && continue
changed="${changed}${path}"$'\n'
done <<< "$current"
[ -n "$changed" ] || exit 0
url=${SCRIBE_URL:-${CLAUDE_PLUGIN_OPTION_API_ENDPOINT:-}}
token=${SCRIBE_TOKEN:-${CLAUDE_PLUGIN_OPTION_API_TOKEN:-}}
case "$url" in *'${'*) url="" ;; esac
case "$token" in *'${'*) token="" ;; esac
repo=$(git -C "$repo_root" remote get-url origin 2>/dev/null || true)
repo_q=""
if [ -n "$repo" ]; then
enc=$(printf '%s' "$repo" | jq -sRr '@uri' 2>/dev/null) || enc=""
[ -n "$enc" ] && repo_q="&repo=${enc}"
fi
# The dedup channels are the PRE-write hook's files, on purpose (see header).
state_dir="${TMPDIR:-/tmp}/scribe-priorart"
mkdir -p "$state_dir" 2>/dev/null || true
idfile="$state_dir/${safe_sid}.ids"
syncfile="$state_dir/${safe_sid}.sync.ids"
derivefile="$state_dir/${safe_sid}.derive.ids"
combined=""
n_files=0
while IFS= read -r rel_path; do
[ -n "${rel_path:-}" ] || continue
# A Bash call that rewrote many files is a refactor or a generator, not a
# shape being instantiated; four is enough to name what matters.
n_files=$((n_files + 1))
[ "$n_files" -le 4 ] || break
file_path="$repo_root/$rel_path"
# The code just written: the ADDED lines of the uncommitted diff for a
# tracked file (sed, not cut: this strips one marker char per line, it is
# not a payload cap), the whole file when untracked.
if git -C "$repo_root" ls-files --error-unmatch -- "$rel_path" >/dev/null 2>&1; then
code=$(git -C "$repo_root" diff -U0 -- "$rel_path" 2>/dev/null | grep '^+' | grep -v '^+++' | sed 's/^+//') || code=""
else
code=$(cat "$file_path" 2>/dev/null) || code=""
fi
[ -n "$code" ] || continue
names=$(printf '%s' "$code" | scribe_defs | sort -u | head -12) || names=""
# Nothing DEFINED in what was written (prose, data, a call-site edit) →
# nothing to say; the arms are about shapes.
[ -n "$names" ] || continue
local_lines=$(scribe_local_dups "$repo_root" "$rel_path" <<< "$names") || local_lines=""
local_context=""
if [ -n "$local_lines" ]; then
local_context="> Already defined elsewhere in this repo — \`${rel_path}\` (just written) adds another copy; check before keeping it (\`git grep\` shown; a nudge, not a gate):"$'\n'"${local_lines}"
fi
context=""
body=""
if [ -n "$url" ] && [ -n "$token" ]; then
q=$(printf '%s' "$code" | head -c 1200)
path_enc=$(printf '%s' "$rel_path" | jq -sRr '@uri' 2>/dev/null) || path_enc=""
code_enc=$(printf '%s' "$q" | jq -sRr '@uri' 2>/dev/null) || code_enc=""
shapes_q=""
enc=$(printf '%s\n' "$names" \
| awk -F'\t' 'NF>=2 {printf "%s%s:%s", (n++?",":""), $1, $2}' \
| jq -sRr '@uri' 2>/dev/null) || enc=""
[ -n "$enc" ] && shapes_q="&shapes=${enc}"
exclude_q=""; sync_exclude_q=""; derive_exclude_q=""
if [ -f "$idfile" ]; then
seen=$(tr '\n' ',' < "$idfile" 2>/dev/null | sed 's/,$//')
[ -n "$seen" ] && exclude_q="&exclude_ids=${seen}"
fi
if [ -f "$syncfile" ]; then
sync_seen=$(tr '\n' ',' < "$syncfile" 2>/dev/null | sed 's/,$//')
[ -n "$sync_seen" ] && sync_exclude_q="&exclude_sync_ids=${sync_seen}"
fi
if [ -f "$derivefile" ]; then
derive_seen=$(tr '\n' ',' < "$derivefile" 2>/dev/null | sed 's/,$//' | jq -sRr '@uri' 2>/dev/null) || derive_seen=""
[ -n "$derive_seen" ] && derive_exclude_q="&exclude_derive=${derive_seen}"
fi
if [ -n "$path_enc" ]; then
body=$(curl -fsS --max-time 4 \
-H "Authorization: Bearer ${token}" \
"${url%/}/api/plugin/prior-art?path=${path_enc}&code=${code_enc}${repo_q}${exclude_q}${sync_exclude_q}${derive_exclude_q}${shapes_q}" 2>/dev/null) || body=""
fi
if [ -n "$body" ]; then
context=$(printf '%s' "$body" | jq -r '.context // empty' 2>/dev/null) || context=""
if [ -n "$context" ]; then
printf '%s' "$body" | jq -r '((.note_ids // []) - (.sync_note_ids // []))[]?' 2>/dev/null >> "$idfile" || true
printf '%s' "$body" | jq -r '(.sync_note_ids // [])[]?' 2>/dev/null >> "$syncfile" || true
printf '%s' "$body" | jq -r '(.derive_keys // [])[]?' 2>/dev/null >> "$derivefile" || true
# Several files in one call may name the same family: keep each
# token once, so the next request's exclude list stays exact.
for f in "$idfile" "$syncfile" "$derivefile"; do
[ -s "$f" ] && { sort -u -o "$f" "$f" 2>/dev/null || true; }
done
fi
fi
fi
# The record nudge (#2664), same gate as the pre-write hook: duplication
# demonstrated locally AND nothing recorded for it.
if [ -n "$local_lines" ]; then
n_recorded=$(printf '%s' "$body" | jq -r '.note_ids | length' 2>/dev/null) || n_recorded=0
if [ "${n_recorded:-0}" = "0" ] || [ "$n_recorded" = "" ]; then
local_context="${local_context}"$'\n'"> None of those existing copies is recorded in Scribe. If the version just written is the canonical one — or this edit is consolidating the copies — record it now with create_snippet so the next session is offered it instead of writing another copy."
fi
fi
part="$local_context"
if [ -n "$context" ]; then
[ -n "$part" ] && part="${part}"$'\n'
part="${part}${context}"
fi
[ -n "$part" ] || continue
[ -n "$combined" ] && combined="${combined}"$'\n'
combined="${combined}${part}"
done <<< "$changed"
[ -n "$combined" ] || exit 0
jq -n --arg c "$combined" \
'{hookSpecificOutput: {hookEventName: "PostToolUse", additionalContext: $c}}'
exit 0
+109
View File
@@ -0,0 +1,109 @@
#!/usr/bin/env bash
# shellcheck shell=bash
# Scribe plugin — the pieces the two write-path hooks share (#2901).
#
# scribe_prior_art.sh fires BEFORE a Write/Edit tool call; scribe_after_write.sh
# fires AFTER a Bash tool call and diffs the working tree, so code written by
# sed/heredocs/scripts gets the same prior-art and ledger checks. Both need the
# same three things, kept here so they cannot drift apart:
#
# scribe_skip_path PATH formats that hold prose or data, not shapes
# scribe_defs stdin code → "kind<TAB>name" per definition
# scribe_local_dups ROOT REL "kind<TAB>name" lines on stdin → the by-name
# local-duplicate lines (ARM 1, #2280)
#
# Sourced, not executed: `. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"`.
# Skip formats that hold prose or data rather than reusable code. Purely to
# avoid a pointless round-trip — the server would return nothing for these
# anyway. Config formats are NOT skipped: a CI workflow or a compose file is
# often exactly the thing worth reusing.
scribe_skip_path() {
case "$1" in
*.md|*.mdx|*.txt|*.rst|*.json|*.lock|*.log|*.csv|*.tsv|*.svg|*.png|*.jpg|*.jpeg|*.gif|*.ico|*.pdf)
return 0 ;;
esac
return 1
}
# ---------------------------------------------------------------------------
# kind<TAB>name for each thing a piece of code DEFINES, in source order. One
# program, two consumers: the local duplicate arm (every definition in the
# payload) and the ledger feed (#2791, below: the definitions being written,
# or the one enclosing an Edit). Rule-for-rule mirrored by the server's
# services/coverage.py extract_shapes — ledger rows are keyed by what THAT
# sees, so the two must agree on what counts as a definition.
scribe_defs() {
awk '
{
# CSS class definition: .name { or .name,
if (match($0, /^[[:space:]]*\.[A-Za-z][A-Za-z0-9_-]*[[:space:]]*[,{]/)) {
t = $0; sub(/^[[:space:]]*\./, "", t); sub(/[[:space:]]*[,{].*$/, "", t)
if (t != "") print "css\t" t; next
}
line = $0; sub(/^[[:space:]]+/, "", line)
# Strip leading declaration modifiers so the definition keyword is the
# first word regardless of language (export/pub/private/suspend/...).
sub(/^((pub(\([a-z]+\))?|export|default|private|internal|protected|public|static|suspend|async|open|sealed|data|abstract|final|inline|unsafe|extern|override)[[:space:]]+)*/, "", line)
# Go method with receiver: func (r *T) Name(
if (match(line, /^func[[:space:]]*\([^)]*\)[[:space:]]*[A-Za-z_]/)) {
t = line; sub(/^func[[:space:]]*\([^)]*\)[[:space:]]*/, "", t)
sub(/[^A-Za-z0-9_].*$/, "", t)
if (t != "") print "sym\t" t; next
}
# Keyword-announced definitions, functions and named types alike.
# Dunders are skipped: every class defines __init__, so "already defined
# in N other files" is guaranteed noise for them — and noise is what
# teaches sessions to skip the hint.
if (match(line, /^(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+[A-Za-z_$]/)) {
t = line; sub(/^[a-z]+[[:space:]]+/, "", t)
sub(/[^A-Za-z0-9_$].*$/, "", t)
if (t != "" && t !~ /^__.*__$/) print "sym\t" t; next
}
# Arrow/expression assignment: const name = (…) / let name = async (
if (match(line, /^(const|let)[[:space:]]+[A-Za-z_$][A-Za-z0-9_$]*[[:space:]]*=[[:space:]]*(async[[:space:]]*)?[(<]/)) {
t = line; sub(/^(const|let)[[:space:]]+/, "", t)
sub(/[^A-Za-z0-9_$].*$/, "", t)
if (t != "") print "sym\t" t; next
}
}
' 2>/dev/null
}
# ---------------------------------------------------------------------------
# ARM 1 — BY NAME, LOCALLY (#2280). Does a definition of this already exist?
#
# The recorded arms ask Scribe what was RECORDED; the ledger arm (#2900) asks
# what a BOUND repo's ledger knows. A helper nobody recorded, in a repo nobody
# bound, is invisible to both — which is how `.btn-primary` came to be defined
# four times, already diverged. This arm asks the one question only the
# developer's machine can answer, inside the repo, holding the code about to
# be written: no index, no storage, no server — it runs even on an install
# that has never configured Scribe.
#
# Definition-shaped patterns only. Grepping for bare occurrences would match
# every CALL site and drown the real finding — and a hint that is mostly noise
# is one people learn to skip, which is worse than none. ALL code, not a
# language shortlist (#2682): the same keyword family scribe_defs announces.
#
# $1 repo root, $2 repo-relative path of the file being written (excluded from
# the grep — it would always match itself on an Edit). Definitions on stdin.
# Prints one "> - `name` is already defined in N other file(s): …" per hit.
scribe_local_dups() {
local root="$1" rel="$2" kind name pat hits count label files
while IFS=$'\t' read -r kind name; do
[ -n "${name:-}" ] || continue
case "$kind" in
css) pat="^[[:space:]]*\.${name}[[:space:]]*[,{]" ;;
*) pat="(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+${name}[^A-Za-z0-9_]|func[[:space:]]*\([^)]*\)[[:space:]]*${name}[[:space:]]*\(|(const|let)[[:space:]]+${name}[[:space:]]*=" ;;
esac
# -I skips binaries; :(exclude) drops the file being written.
hits=$(git -C "$root" grep -I -l -E -e "$pat" -- . ":(exclude)${rel}" 2>/dev/null | head -4) || hits=""
[ -n "$hits" ] || continue
count=$(printf '%s\n' "$hits" | grep -c . 2>/dev/null || echo 0)
label=$([ "$kind" = css ] && printf '.%s' "$name" || printf '%s' "$name")
files=$(printf '%s' "$hits" | tr '\n' ' ' | sed 's/ $//')
printf '> - `%s` is already defined in %s other file(s): %s\n' "$label" "$count" "$files"
done
}
+26 -96
View File
@@ -51,14 +51,12 @@ code=$(printf '%s' "$event" | jq -r '
.tool_input.content // .tool_input.file_content // .tool_input.content // .tool_input.file_content //
.tool_input.new_string // .tool_input.new_str // empty' 2>/dev/null) || code="" .tool_input.new_string // .tool_input.new_str // empty' 2>/dev/null) || code=""
# Skip formats that hold prose or data rather than reusable code. Purely to # Shared with the after-write hook (#2901): the prose/data skip list, the
# avoid a pointless round-trip — the server would return nothing for these # definition extractor and the local by-name duplicate arm live in
# anyway. Config formats are NOT skipped: a CI workflow or a compose file is # scribe_defs.sh so the two hooks cannot drift apart.
# often exactly the thing worth reusing. # shellcheck source=plugin/hooks/scribe_defs.sh
case "$file_path" in . "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"
*.md|*.mdx|*.txt|*.rst|*.json|*.lock|*.log|*.csv|*.tsv|*.svg|*.png|*.jpg|*.jpeg|*.gif|*.ico|*.pdf) scribe_skip_path "$file_path" && exit 0
exit 0 ;;
esac
# Snippet locations are recorded repo-relative, so send a repo-relative path — # Snippet locations are recorded repo-relative, so send a repo-relative path —
# an absolute one would simply match nothing. Resolved BEFORE the config gate # an absolute one would simply match nothing. Resolved BEFORE the config gate
@@ -73,78 +71,8 @@ if [ -n "$repo_root" ]; then
esac esac
fi fi
# --------------------------------------------------------------------------- # ARM 1 — BY NAME, LOCALLY (#2280): does a definition of this already exist
# ARM 1 — BY NAME, LOCALLY (#2280). Does a definition of this already exist? # in the repo? (scribe_local_dups in scribe_defs.sh carries the why.)
#
# The other two arms ask Scribe what was RECORDED. Scribe has never read a line
# of the codebase, so a helper nobody thought to record is invisible to them —
# which is how `.btn-primary` came to be defined four times, in four scoped
# stylesheets, already diverged. It was never a snippet, so no threshold and no
# query rewrite could ever have surfaced it.
#
# This arm closes that by asking the only question the record cannot answer,
# in the only place that can: the hook already runs on the developer's machine,
# inside the repo, holding the code about to be written. No index, no storage,
# no staleness, and no server — it deliberately runs even on an install that
# has never configured Scribe.
#
# Definition-shaped patterns only. Grepping for bare occurrences would match
# every CALL site and drown the real finding — and a hint that is mostly noise
# is one people learn to skip, which is worse than none.
#
# ALL code, not a language shortlist (#2682): the detector was born covering
# only the languages of the repo it was written in, which silently amputated
# this whole arm — and the record nudge gated on it — for every Go/Kotlin/Rust
# project. Definitions are announced by a small keyword family across
# languages (func/fun/fn/function/def/sub · class/struct/trait/interface/
# enum/object/protocol/type), so one modifier-strip + keyword match covers
# them all. Known out of scope: keyword-less declaration syntax (C/Java/Dart
# `ReturnType name(...)`) needs a real parser, and `impl` blocks are excluded
# because several per type is normal Rust, not duplication.
# ---------------------------------------------------------------------------
# kind<TAB>name for each thing a piece of code DEFINES, in source order. One
# program, two consumers: the local duplicate arm (every definition in the
# payload) and the ledger feed (#2791, below: the definitions being written,
# or the one enclosing an Edit). Rule-for-rule mirrored by the server's
# services/coverage.py extract_shapes — ledger rows are keyed by what THAT
# sees, so the two must agree on what counts as a definition.
scribe_defs() {
awk '
{
# CSS class definition: .name { or .name,
if (match($0, /^[[:space:]]*\.[A-Za-z][A-Za-z0-9_-]*[[:space:]]*[,{]/)) {
t = $0; sub(/^[[:space:]]*\./, "", t); sub(/[[:space:]]*[,{].*$/, "", t)
if (t != "") print "css\t" t; next
}
line = $0; sub(/^[[:space:]]+/, "", line)
# Strip leading declaration modifiers so the definition keyword is the
# first word regardless of language (export/pub/private/suspend/...).
sub(/^((pub(\([a-z]+\))?|export|default|private|internal|protected|public|static|suspend|async|open|sealed|data|abstract|final|inline|unsafe|extern|override)[[:space:]]+)*/, "", line)
# Go method with receiver: func (r *T) Name(
if (match(line, /^func[[:space:]]*\([^)]*\)[[:space:]]*[A-Za-z_]/)) {
t = line; sub(/^func[[:space:]]*\([^)]*\)[[:space:]]*/, "", t)
sub(/[^A-Za-z0-9_].*$/, "", t)
if (t != "") print "sym\t" t; next
}
# Keyword-announced definitions, functions and named types alike.
# Dunders are skipped: every class defines __init__, so "already defined
# in N other files" is guaranteed noise for them — and noise is what
# teaches sessions to skip the hint.
if (match(line, /^(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+[A-Za-z_$]/)) {
t = line; sub(/^[a-z]+[[:space:]]+/, "", t)
sub(/[^A-Za-z0-9_$].*$/, "", t)
if (t != "" && t !~ /^__.*__$/) print "sym\t" t; next
}
# Arrow/expression assignment: const name = (…) / let name = async (
if (match(line, /^(const|let)[[:space:]]+[A-Za-z_$][A-Za-z0-9_$]*[[:space:]]*=[[:space:]]*(async[[:space:]]*)?[(<]/)) {
t = line; sub(/^(const|let)[[:space:]]+/, "", t)
sub(/[^A-Za-z0-9_$].*$/, "", t)
if (t != "") print "sym\t" t; next
}
}
' 2>/dev/null
}
names="" names=""
if [ -n "$code" ]; then if [ -n "$code" ]; then
names=$(printf '%s' "$code" | scribe_defs | sort -u | head -12) || names="" names=$(printf '%s' "$code" | scribe_defs | sort -u | head -12) || names=""
@@ -152,21 +80,8 @@ fi
local_lines="" local_lines=""
if [ -n "$repo_root" ] && [ -n "$names" ]; then if [ -n "$repo_root" ] && [ -n "$names" ]; then
while IFS=$'\t' read -r kind name; do local_lines=$(scribe_local_dups "$repo_root" "$rel_path" <<< "$names") || local_lines=""
[ -n "${name:-}" ] || continue [ -n "$local_lines" ] && local_lines="${local_lines}"$'\n'
case "$kind" in
css) pat="^[[:space:]]*\.${name}[[:space:]]*[,{]" ;;
*) pat="(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+${name}[^A-Za-z0-9_]|func[[:space:]]*\([^)]*\)[[:space:]]*${name}[[:space:]]*\(|(const|let)[[:space:]]+${name}[[:space:]]*=" ;;
esac
# -I skips binaries; :(exclude) drops the file being written, which would
# otherwise always match itself on an Edit.
hits=$(git -C "$repo_root" grep -I -l -E -e "$pat" -- . ":(exclude)${rel_path}" 2>/dev/null | head -4) || hits=""
[ -n "$hits" ] || continue
count=$(printf '%s\n' "$hits" | grep -c . 2>/dev/null || echo 0)
label=$([ "$kind" = css ] && printf '.%s' "$name" || printf '%s' "$name")
files=$(printf '%s' "$hits" | tr '\n' ' ' | sed 's/ $//')
local_lines="${local_lines}> - \`${label}\` is already defined in ${count} other file(s): ${files}"$'\n'
done <<< "$names"
fi fi
local_context="" local_context=""
@@ -258,14 +173,22 @@ fi
# the sync nudge when the recorded file itself is edited later. # the sync nudge when the recorded file itself is edited later.
state_dir="${TMPDIR:-/tmp}/scribe-priorart" state_dir="${TMPDIR:-/tmp}/scribe-priorart"
mkdir -p "$state_dir" 2>/dev/null || true mkdir -p "$state_dir" 2>/dev/null || true
#
# A THIRD channel (#2900): the ledger's derive arm names a duplicate family
# (a derive group id) or a canon elsewhere (`canon:<snippet_id>`) for the
# shapes being written. Keyed by that token, not a note id, so it dedups on
# its own file and a family is named once per session, not at every edit.
idfile="" idfile=""
syncfile="" syncfile=""
derivefile=""
exclude_q="" exclude_q=""
sync_exclude_q="" sync_exclude_q=""
derive_exclude_q=""
if [ -n "$session_id" ]; then if [ -n "$session_id" ]; then
safe_sid=$(printf '%s' "$session_id" | tr -c 'A-Za-z0-9._-' '_') safe_sid=$(printf '%s' "$session_id" | tr -c 'A-Za-z0-9._-' '_')
idfile="$state_dir/${safe_sid}.ids" idfile="$state_dir/${safe_sid}.ids"
syncfile="$state_dir/${safe_sid}.sync.ids" syncfile="$state_dir/${safe_sid}.sync.ids"
derivefile="$state_dir/${safe_sid}.derive.ids"
if [ -f "$idfile" ]; then if [ -f "$idfile" ]; then
seen=$(tr '\n' ',' < "$idfile" 2>/dev/null | sed 's/,$//') seen=$(tr '\n' ',' < "$idfile" 2>/dev/null | sed 's/,$//')
[ -n "$seen" ] && exclude_q="&exclude_ids=${seen}" [ -n "$seen" ] && exclude_q="&exclude_ids=${seen}"
@@ -274,13 +197,17 @@ if [ -n "$session_id" ]; then
sync_seen=$(tr '\n' ',' < "$syncfile" 2>/dev/null | sed 's/,$//') sync_seen=$(tr '\n' ',' < "$syncfile" 2>/dev/null | sed 's/,$//')
[ -n "$sync_seen" ] && sync_exclude_q="&exclude_sync_ids=${sync_seen}" [ -n "$sync_seen" ] && sync_exclude_q="&exclude_sync_ids=${sync_seen}"
fi fi
if [ -f "$derivefile" ]; then
derive_seen=$(tr '\n' ',' < "$derivefile" 2>/dev/null | sed 's/,$//' | jq -sRr '@uri' 2>/dev/null) || derive_seen=""
[ -n "$derive_seen" ] && derive_exclude_q="&exclude_derive=${derive_seen}"
fi
fi fi
# `|| true`, not `|| exit 0`: an unreachable instance must not discard a local # `|| true`, not `|| exit 0`: an unreachable instance must not discard a local
# finding that needed no instance to produce. # finding that needed no instance to produce.
body=$(curl -fsS --max-time 5 \ body=$(curl -fsS --max-time 5 \
-H "Authorization: Bearer ${token}" \ -H "Authorization: Bearer ${token}" \
"${url%/}/api/plugin/prior-art?path=${path_enc}&code=${code_enc}${repo_q}${exclude_q}${sync_exclude_q}${shapes_q}" 2>/dev/null) || body="" "${url%/}/api/plugin/prior-art?path=${path_enc}&code=${code_enc}${repo_q}${exclude_q}${sync_exclude_q}${derive_exclude_q}${shapes_q}" 2>/dev/null) || body=""
context="" context=""
if [ -n "$body" ]; then if [ -n "$body" ]; then
@@ -295,6 +222,9 @@ if [ -n "$body" ]; then
if [ -n "$syncfile" ]; then if [ -n "$syncfile" ]; then
printf '%s' "$body" | jq -r '(.sync_note_ids // [])[]?' 2>/dev/null >> "$syncfile" || true printf '%s' "$body" | jq -r '(.sync_note_ids // [])[]?' 2>/dev/null >> "$syncfile" || true
fi fi
if [ -n "$derivefile" ]; then
printf '%s' "$body" | jq -r '(.derive_keys // [])[]?' 2>/dev/null >> "$derivefile" || true
fi
fi fi
fi fi
+4 -1
View File
@@ -66,7 +66,10 @@ for the operator's work, and as your own working memory across sessions.
should read as a map of every shape in it. The backstop still holds: should read as a map of every shape in it. The backstop still holds:
noticing the second copy of anything, or consolidating copies into a shared noticing the second copy of anything, or consolidating copies into a shared
X, means X gets recorded before that work is finished — which is how a X, means X gets recorded before that work is finished — which is how a
codebase is kept from growing four `.btn-primary` definitions. codebase is kept from growing four `.btn-primary` definitions. The write-path
hooks (before a Write/Edit, and after any Bash call that changed the tree)
name a known duplicate family or a canon elsewhere for what was just
written — act on that line at the write, not at the next audit.
- Do **not** keep the operator's rules, plans, or project notes in local - Do **not** keep the operator's rules, plans, or project notes in local
memory / CLAUDE.md in parallel with Scribe — Scribe holds the single copy. memory / CLAUDE.md in parallel with Scribe — Scribe holds the single copy.
- **Compact at clean seams** — because you record as you go, a context - **Compact at clean seams** — because you record as you go, a context
+7
View File
@@ -44,6 +44,13 @@ through recall/auto-inject; this skill is the active reflex around that.
it before you go any further. Either it's the helper you were about to it before you go any further. Either it's the helper you were about to
duplicate — reuse it and drop yours — or it isn't, and the record needs the new duplicate — reuse it and drop yours — or it isn't, and the record needs the new
location adding. Both are cheaper now than after the duplicate settles in. location adding. Both are cheaper now than after the duplicate settles in.
- **A `Shape ledger at …` line is the ledger speaking, not the record.** It
names a duplicate family ("identical body in N other files, no canon") or a
canon elsewhere for a name you just wrote — for edits made through Bash
(sed, heredocs, scripts) as much as through Write/Edit. Derive the family or
reuse the canon *now*; a family that is convention rather than copies is
dismissed with `classify_shapes(..., status="exempt",
reason_code="convention-plumbing")`, never ignored.
- **A `[records this file]` hint is a duty, not a menu.** When the hint says a - **A `[records this file]` hint is a duty, not a menu.** When the hint says a
snippet records the very file you're editing, the record's freshness is now snippet records the very file you're editing, the record's freshness is now
YOUR edit's responsibility: if the edit changes the recorded shape, YOUR edit's responsibility: if the edit changes the recorded shape,
+28
View File
@@ -85,6 +85,34 @@ the dominant form, `create_snippet` it, migrate the outliers, then classify
the rest as instances. Canon is determined from the code; consistency comes the rest as instances. Canon is determined from the code; consistency comes
from the derivation, not from asking permission. from the derivation, not from asking permission.
## Derive groups are drift, not audit material
The catalogue exists so the codebase is DRY **from inception**, not as DRY as
the last sweep left it. Three surfaces say so without anyone running an audit
(milestone 299):
- **At the write** — the prior-art hint (the Write/Edit hook, and since
0.1.39 the after-write hook on Bash, so sed/heredoc/script edits count too)
carries a `Shape ledger at <path>` line when a name just written is a known
**duplicate family** ("identical body in N other files, no canon") or a
**canon elsewhere** ("snippet #N at <path> — reuse, don't redefine"). Act
on it *then*: pull the canon and build from it, or derive the family now —
`create_snippet` the dominant form, repoint the copies, `classify_shapes`
them `instance`. A family is named once per session.
- **On arrival** — the coverage line's `standing:` block (shown even when
nothing is unclassified) and `derive_new` ("+N new copies since last
refresh: .x in <path>") name what drifted since the previous refresh. That
is the todo of the moment, sized to the last batch — not a backlog.
- **A family that is convention, not copies** — component-local `load` /
`toggle` / `save` that happen to share a name — is dismissed, not
consolidated: `classify_shapes(..., status="exempt",
reason_code="convention-plumbing", reason=…)` (or `classify_shapes_by_rule`
for a whole family) removes it from the queue. Dismissal is a judgment and
it is recorded; silence is not.
After the one-time pay-down the derive queue reads empty; anything in it
afterwards is drift of the moment, and the hint already said so at the write.
## The divergence readout — button B where button A is canon ## The divergence readout — button B where button A is canon
Three questions the ledger answers mechanically (#2793): Three questions the ledger answers mechanically (#2793):
+11
View File
@@ -203,6 +203,17 @@ SMOKE_EVENTS: dict[str, str] = {
), ),
"scribe_sync_processes.sh": json.dumps({"source": "startup"}), "scribe_sync_processes.sh": json.dumps({"source": "startup"}),
"scribe_session_context.sh": json.dumps({"source": "startup"}), "scribe_session_context.sh": json.dumps({"source": "startup"}),
# The after-write hook (#2901) diffs the working tree; on CI's clean
# checkout there is nothing to report, so silence is the right assertion.
# (On a dirty local tree with a definition just written it may speak —
# that is the hook working, not a failure of the contract.)
"scribe_after_write.sh": json.dumps(
{"session_id": "smoke", "cwd": ".", "tool_name": "Bash",
"tool_input": {"command": "true"}, "tool_response": {}}
),
# The shared library is sourced, never run; executed bare it defines
# functions and exits — silent by construction.
"scribe_defs.sh": "",
} }
# The one hook that legitimately produces output with no credentials. # The one hook that legitimately produces output with no credentials.
+3 -1
View File
@@ -295,7 +295,9 @@ async def refresh_pattern_coverage(project_id: int) -> dict:
Returns the accounting payload — total, accounted, counts by status, Returns the accounting payload — total, accounted, counts by status,
unclassified, repos, largest_gaps, `proposed` (canon proposals awaiting unclassified, repos, largest_gaps, `proposed` (canon proposals awaiting
confirmation), `derive_groups` (the biggest repeats-with-no-canon confirmation), `derive_groups` (the biggest repeats-with-no-canon
families), `proposer` (what this refresh examined) — plus families), `derive_new` (copies that joined a family since the previous
refresh — the drift to act on now: derive the canon, don't queue an
audit), `proposer` (what this refresh examined) — plus
`pattern_coverage`, the same one-line summary enter_project carries. `pattern_coverage`, the same one-line summary enter_project carries.
""" """
uid = current_user_id() uid = current_user_id()
+8
View File
@@ -129,6 +129,10 @@ async def write_path_prior_art():
surfaced. A separate channel on purpose: a reuse surfaced. A separate channel on purpose: a reuse
hint shown early must not suppress the record-sync hint shown early must not suppress the record-sync
nudge when the recorded file is edited later. nudge when the recorded file is edited later.
exclude_derive (opt) — comma-separated derive keys (a derive group id
or `canon:<snippet_id>`) already named this
session by the ledger arm (#2900); its own
channel, like the two above.
shapes (opt) — comma-separated `kind:name` definitions the hook shapes (opt) — comma-separated `kind:name` definitions the hook
found in (or enclosing) the payload, kind being found in (or enclosing) the payload, kind being
css|sym. The shape ledger's write-path feed css|sym. The shape ledger's write-path feed
@@ -144,6 +148,9 @@ async def write_path_prior_art():
project_id, repo, _unbound = await _project_scope() project_id, repo, _unbound = await _project_scope()
exclude_ids = _int_list(request.args.get("exclude_ids")) exclude_ids = _int_list(request.args.get("exclude_ids"))
exclude_sync_ids = _int_list(request.args.get("exclude_sync_ids")) exclude_sync_ids = _int_list(request.args.get("exclude_sync_ids"))
exclude_derive = [
p.strip() for p in (request.args.get("exclude_derive") or "").split(",") if p.strip()
]
shapes = _parse_shapes(request.args.get("shapes") or "") shapes = _parse_shapes(request.args.get("shapes") or "")
api_key = getattr(g, "api_key", None) api_key = getattr(g, "api_key", None)
may_stamp = api_key is None or getattr(api_key, "scope", "") == "write" may_stamp = api_key is None or getattr(api_key, "scope", "") == "write"
@@ -153,6 +160,7 @@ async def write_path_prior_art():
exclude_ids=exclude_ids, exclude_sync_ids=exclude_sync_ids, exclude_ids=exclude_ids, exclude_sync_ids=exclude_sync_ids,
stamp_shapes=shapes if may_stamp else None, stamp_shapes=shapes if may_stamp else None,
repo_key=repo_bindings_svc.normalize_repo_key(repo) if repo else "", repo_key=repo_bindings_svc.normalize_repo_key(repo) if repo else "",
exclude_derive=exclude_derive,
) )
return jsonify(result) return jsonify(result)
+47 -20
View File
@@ -462,15 +462,20 @@ async def compute_coverage(
await shape_ledger.apply_derive_groups(project_id) await shape_ledger.apply_derive_groups(project_id)
except Exception: except Exception:
logger.warning("derive-first grouping failed", exc_info=True) logger.warning("derive-first grouping failed", exc_info=True)
# The button-B pass (#2793): shapes new since the PREVIOUS computation, # "Since the previous computation" — the cache's stamp. A first seed has
# where a canon dominates. The previous computation's stamp is the cache; # none, so nothing is new then. Read once; two passes use it: the
# a first seed has none, so it flags nothing (everything is new then). # button-B flag (#2793) and the derive-new drift count (#2899).
since = None
try: try:
previous = await get_setting(user_id, f"{_CACHE_KEY_PREFIX}{project_id}") previous = await get_setting(user_id, f"{_CACHE_KEY_PREFIX}{project_id}")
since = None
if previous: if previous:
stamp = (json.loads(previous) or {}).get("computed_at") stamp = (json.loads(previous) or {}).get("computed_at")
since = datetime.fromisoformat(stamp) if stamp else None since = datetime.fromisoformat(stamp) if stamp else None
except Exception:
logger.warning("previous coverage stamp unreadable", exc_info=True)
# The button-B pass (#2793): shapes new since the PREVIOUS computation,
# where a canon dominates.
try:
await shape_ledger.flag_divergence(project_id, since=since) await shape_ledger.flag_divergence(project_id, since=since)
except Exception: except Exception:
logger.warning("divergence pass failed", exc_info=True) logger.warning("divergence pass failed", exc_info=True)
@@ -491,6 +496,7 @@ async def compute_coverage(
unclassified = counts.pop("unclassified") unclassified = counts.pop("unclassified")
proposals = shape_ledger.proposal_summary(rows) proposals = shape_ledger.proposal_summary(rows)
divergence = shape_ledger.divergence_summary(rows) divergence = shape_ledger.divergence_summary(rows)
derive_new = shape_ledger.derive_new_summary(rows, since=since)
return { return {
"total": len(rows), "total": len(rows),
"accounted": len(rows) - unclassified, "accounted": len(rows) - unclassified,
@@ -501,6 +507,10 @@ async def compute_coverage(
"proposed": proposals["proposed"], "proposed": proposals["proposed"],
"derive_groups": proposals["derive_groups"], "derive_groups": proposals["derive_groups"],
"top_canon": proposals.get("top_canon"), "top_canon": proposals.get("top_canon"),
# Drift since the previous refresh (#2899): copies that joined a
# duplicate family — what the arrival line names so drift is noticed
# on entering, not found by an audit.
"derive_new": derive_new,
"proposer": proposer_stats, "proposer": proposer_stats,
# The divergence readout (#2793): button B where button A is canon, # The divergence readout (#2793): button B where button A is canon,
# and judged shapes whose bodies moved since they were judged. # and judged shapes whose bodies moved since they were judged.
@@ -648,29 +658,46 @@ def coverage_line(coverage: dict) -> str:
line += f"{breakdown}" line += f"{breakdown}"
line += f" (estimate{', computed ' + day if day else ''})" line += f" (estimate{', computed ' + day if day else ''})"
unclassified = coverage.get("unclassified", 0) unclassified = coverage.get("unclassified", 0)
# The standing work, built whatever the todo count (#2899). Since the
# scoped bucket (#2869) a ledger can read 100% accounted and still carry
# derive groups, proposals and divergence; gating this block on
# `unclassified > 0` is how 439 derive rows went unmentioned.
standing = []
if coverage.get("proposed"):
standing.append(f"{coverage['proposed']} proposed")
n_groups = len(coverage.get("derive_groups") or [])
if n_groups:
standing.append(f"{n_groups} derive group{'s' if n_groups != 1 else ''}")
# Drift since the previous refresh: copies that joined a family, the
# first one named — the sentence the arrival moment exists to say.
new = coverage.get("derive_new") or {}
if new.get("count"):
n = new["count"]
first_new = (new.get("examples") or [{}])[0]
where = (
f": {first_new['label']} in {first_new['path']}"
if first_new.get("label") and first_new.get("path") else ""
)
standing.append(f"+{n} new cop{'y' if n == 1 else 'ies'} since last refresh{where}")
if coverage.get("divergent"):
standing.append(f"{coverage['divergent']} DIVERGENT")
# The next action, on the line (#2874): the canon with the biggest
# queue to confirm, and the widest body-identical copy to consolidate.
top = coverage.get("top_canon") or {}
if top.get("snippet_id"):
standing.append(f"top canon #{top['snippet_id']} ×{top.get('count', 0)}")
first = (coverage.get("derive_groups") or [{}])[0]
if first.get("label") and first.get("files"):
standing.append(f"top copy {first['label']} ×{first['files']} files")
if unclassified: if unclassified:
line += f"; {unclassified} unclassified" line += f"; {unclassified} unclassified"
standing = []
if coverage.get("proposed"):
standing.append(f"{coverage['proposed']} proposed")
n_groups = len(coverage.get("derive_groups") or [])
if n_groups:
standing.append(f"{n_groups} derive group{'s' if n_groups != 1 else ''}")
if coverage.get("divergent"):
standing.append(f"{coverage['divergent']} DIVERGENT")
# The next action, on the line (#2874): the canon with the biggest
# queue to confirm, and the widest body-identical copy to consolidate.
top = coverage.get("top_canon") or {}
if top.get("snippet_id"):
standing.append(f"top canon #{top['snippet_id']} ×{top.get('count', 0)}")
first = (coverage.get("derive_groups") or [{}])[0]
if first.get("label") and first.get("files"):
standing.append(f"top copy {first['label']} ×{first['files']} files")
if standing: if standing:
line += f" ({', '.join(standing)})" line += f" ({', '.join(standing)})"
gaps = [g["dir"] for g in coverage.get("largest_gaps") or []] gaps = [g["dir"] for g in coverage.get("largest_gaps") or []]
if gaps: if gaps:
line += ", largest: " + ", ".join(gaps) line += ", largest: " + ", ".join(gaps)
elif standing:
line += f"; standing: {', '.join(standing)}"
if coverage.get("recheck"): if coverage.get("recheck"):
line += f"; {coverage['recheck']} judged shape{'s' if coverage['recheck'] != 1 else ''} changed since judged — recheck" line += f"; {coverage['recheck']} judged shape{'s' if coverage['recheck'] != 1 else ''} changed since judged — recheck"
return line return line
+49 -2
View File
@@ -706,6 +706,7 @@ async def build_write_path_hint(
exclude_sync_ids: list[int] | None = None, exclude_sync_ids: list[int] | None = None,
stamp_shapes: list[tuple[str, str]] | None = None, stamp_shapes: list[tuple[str, str]] | None = None,
repo_key: str = "", repo_key: str = "",
exclude_derive: list[str] | None = None,
) -> dict: ) -> dict:
"""Prior-art hint for the plugin's PreToolUse hook on Write/Edit. """Prior-art hint for the plugin's PreToolUse hook on Write/Edit.
@@ -765,7 +766,7 @@ async def build_write_path_hint(
""" """
cfg = await get_writepath_config(user_id) cfg = await get_writepath_config(user_id)
empty = {"context": "", "note_ids": [], "sync_note_ids": [], "config": cfg, empty = {"context": "", "note_ids": [], "sync_note_ids": [], "config": cfg,
"stamped": [], "divergence": []} "stamped": [], "divergence": [], "derive": [], "derive_keys": []}
path = (path or "").strip() path = (path or "").strip()
if not cfg["enabled"] or not path: if not cfg["enabled"] or not path:
return empty return empty
@@ -935,7 +936,20 @@ async def build_write_path_hint(
) )
except Exception: except Exception:
logger.warning("write-time divergence check failed", exc_info=True) logger.warning("write-time divergence check failed", exc_info=True)
if not synced and not menu and not stamped and not divergence: # The in-band DERIVE check (#2900): the ledger's own knowledge of the
# names being written — a duplicate family with no canon, or a canon
# recorded elsewhere. This is the arm the by-name local grep could not
# be: it knows whether the other copies are canon or stray. Keyed per
# session (`exclude_derive`) so a family is named once, not per edit.
derive: list[dict] = []
if stamp_shapes and project_id:
try:
found = await shape_ledger_svc.write_time_derive(project_id, path, stamp_shapes)
skip = set(exclude_derive or [])
derive = [d for d in found if d.get("key") not in skip]
except Exception:
logger.warning("write-time derive check failed", exc_info=True)
if not synced and not menu and not stamped and not divergence and not derive:
return empty return empty
owners = await owner_names_for({ owners = await owner_names_for({
@@ -1003,6 +1017,8 @@ async def build_write_path_hint(
lines.append(_stamp_line(path, stamped)) lines.append(_stamp_line(path, stamped))
if divergence: if divergence:
lines.append(_divergence_line(path, divergence)) lines.append(_divergence_line(path, divergence))
if derive:
lines.append(_derive_line(path, derive))
# Split by arm, which is the whole reason this table exists. The place arm # Split by arm, which is the whole reason this table exists. The place arm
# carries no score and so has no home in retrieval_logs; before #2085 a # carries no score and so has no home in retrieval_logs; before #2085 a
@@ -1027,9 +1043,40 @@ async def build_write_path_hint(
"config": cfg, "config": cfg,
"stamped": stamped, "stamped": stamped,
"divergence": divergence, "divergence": divergence,
"derive": derive,
"derive_keys": [d["key"] for d in derive],
} }
def _derive_line(path: str, derive: list[dict]) -> str:
"""The ledger's word on the names being written (#2900): a duplicate
family to derive, or a canon to reuse — said at the write."""
parts = []
for d in derive:
if d.get("canon"):
c = d["canon"]
parts.append(
f"`{c['label']}` is canon — snippet #{c['snippet_id']} at `{c['path']}`; "
"pull it and reuse, don't redefine"
)
continue
f = d["family"]
how = "identical body" if f.get("identical") else "same name defined"
files = ", ".join(f"`{x}`" for x in f.get("files") or [])
more = f.get("file_count", 0) - len(f.get("files") or [])
if more > 0:
files += f" +{more} more"
parts.append(
f"`{f['label']}` is a duplicate family with no canon — {how} in "
f"{f.get('file_count', 0)} other file(s): {files}; derive it now: "
"record the canon (create_snippet) and make the copies instances "
"(classify_shapes) — or, if these are convention not copies, "
"`classify_shapes(..., status=\"exempt\", reason_code=\"convention-plumbing\")` "
"dismisses the family — rather than adding another copy"
)
return f"> Shape ledger at `{path}`: " + "; ".join(parts) + "."
def _divergence_line(path: str, divergence: list[dict]) -> str: def _divergence_line(path: str, divergence: list[dict]) -> str:
"""Button B where button A is canon — named at the write (#2793).""" """Button B where button A is canon — named at the write (#2793)."""
parts = [ parts = [
+97
View File
@@ -1404,6 +1404,34 @@ def proposal_summary(rows: Iterable[CodeShape], *, top: int = 8) -> dict:
return {"proposed": proposed, "derive_groups": ranked[:top], "top_canon": top_canon} return {"proposed": proposed, "derive_groups": ranked[:top], "top_canon": top_canon}
def derive_new_summary(
rows: Iterable[CodeShape], *, since: datetime | None, top: int = 3
) -> dict:
"""The arrival-moment drift signal (#2899): derive-grouped rows FIRST
SEEN after ``since`` — the previous refresh's stamp, the same one
flag_divergence uses. "Since the last refresh, N more copies joined a
duplicate family" is the sentence that makes the derive queue a thing
you notice on entering, not a thing an audit finds. ``since`` None (a
first seed) means nothing is new. Judged rows never count."""
if since is None:
return {"count": 0, "examples": []}
fresh = [
r for r in rows
if r.proposal_basis == "derive" and r.proposal_group
and r.status in _MECHANICAL_TODO and r.vanished_at is None
and r.created_at is not None and r.created_at > since
]
fresh.sort(key=lambda r: r.created_at, reverse=True)
return {
"count": len(fresh),
"examples": [
{"label": ("." if r.kind == "css" else "") + r.symbol,
"path": r.path, "group": r.proposal_group}
for r in fresh[:top]
],
}
async def confirm_proposals( async def confirm_proposals(
user_id: int, user_id: int,
project_id: int, project_id: int,
@@ -1566,6 +1594,75 @@ async def write_time_divergence(
return out return out
# How many other files a family line names before "…" — enough to go look,
# not a wall.
_DERIVE_FILES_SHOWN = 4
async def write_time_derive(
project_id: int, path: str, shapes: list[tuple[str, str]]
) -> list[dict]:
"""The in-band DERIVE check (#2900): for each (kind, name) the hook
named at ``path``, what the ledger already knows about that name
elsewhere in the project —
family the name sits in a derive-first group (identical body in N
files, or the same name in ≥3): "this is a known duplicate
family with no canon — derive it now, don't add a copy";
canon a `canonical` row of that name at another path: "this is
canon #N at <path> — reuse, don't redefine".
Only for shapes not yet judged at ``path`` (a judged shape is not
re-litigated at every edit), never for the canon's own file. Returns
[{symbol, kind, key, family?|canon?}] — `key` is the dedup token the
hook keeps per session (the group id, or canon:<snippet_id>)."""
wanted = {(k, _norm_symbol(n)): n for k, n in shapes if n}
if not wanted:
return []
async with async_session() as session:
rows = (
await session.execute(
select(CodeShape).where(
CodeShape.project_id == project_id,
CodeShape.vanished_at.is_(None),
CodeShape.symbol.in_({norm for (_k, norm) in wanted}),
)
)
).scalars().all()
out: list[dict] = []
for (kind, norm), name in wanted.items():
same = [r for r in rows if r.kind == kind and _norm_symbol(r.symbol) == norm]
here = next((r for r in same if r.path == path), None)
if here is not None and here.status not in _MECHANICAL_TODO:
continue # judged here (or this IS the canon): nothing to say
others = [r for r in same if r.path != path]
label = ("." if kind == "css" else "") + name
canon = next((r for r in others if r.status == "canonical" and r.snippet_id), None)
if canon is not None:
out.append({"symbol": name, "kind": kind, "key": f"canon:{canon.snippet_id}",
"canon": {"snippet_id": canon.snippet_id, "path": canon.path,
"label": label}})
continue
grouped = [r for r in others if r.proposal_group and r.status in _MECHANICAL_TODO]
if here is not None and here.proposal_group:
grouped = [r for r in grouped if r.proposal_group == here.proposal_group] or grouped
if not grouped:
continue
group = grouped[0].proposal_group
members = [r for r in grouped if r.proposal_group == group]
files = sorted({r.path for r in members})
out.append({
"symbol": name, "kind": kind, "key": group,
"family": {
"group": group, "label": label,
"identical": not group.startswith("name:"),
"files": files[:_DERIVE_FILES_SHOWN], "file_count": len(files),
"size": len(members) + (1 if here is not None else 0),
},
})
return out
async def flag_divergence(project_id: int, *, since: datetime | None) -> int: async def flag_divergence(project_id: int, *, since: datetime | None) -> int:
"""Flag shapes created after ``since`` (the previous refresh) that sit """Flag shapes created after ``since`` (the previous refresh) that sit
where a canon dominates and were not proposed as that canon. With no where a canon dominates and were not proposed as that canon. With no
+145
View File
@@ -0,0 +1,145 @@
"""The PostToolUse after-write hook (#2901): code written through Bash — sed,
heredocs, scripts — gets the same prior-art / ledger checks as a Write/Edit.
Runs the real shell against a temp git repo and a throwaway HTTP sink, like
the pre-write hook's end-to-end tests. Skips where the hook's tools are
missing; asserts on content where they are present."""
from __future__ import annotations
import http.server
import json
import os
import shutil
import subprocess
import threading
import urllib.parse
from pathlib import Path
import pytest
PLUGIN = Path(__file__).resolve().parents[1] / "plugin"
HOOK = PLUGIN / "hooks" / "scribe_after_write.sh"
def _env(tmp_path, url="http://127.0.0.1:9"):
for tool in ("git", "jq", "curl", "bash"):
if shutil.which(tool) is None:
pytest.skip(f"hook runtime tool {tool!r} not installed")
return {"PATH": os.environ["PATH"], "SCRIBE_URL": url, "SCRIBE_TOKEN": "t",
"TMPDIR": str(tmp_path), "HOME": str(tmp_path),
"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@x",
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@x"}
def _repo(tmp_path, env):
repo = tmp_path / "repo"
repo.mkdir()
subprocess.run(["git", "init", "-q"], cwd=repo, check=True, env=env)
(repo / "b.py").write_text("def one():\n return 1\n")
(repo / "c.py").write_text("def slug(t):\n return t.lower()\n")
subprocess.run(["git", "add", "."], cwd=repo, check=True, env=env)
subprocess.run(["git", "commit", "-q", "-m", "base"], cwd=repo, check=True, env=env)
return repo
def _run(repo, env, session="s-after-1", tool="Bash"):
out = subprocess.run(
["bash", str(HOOK)],
input=json.dumps({"session_id": session, "cwd": str(repo), "tool_name": tool,
"tool_input": {"command": "cat > x"}, "tool_response": {}}),
capture_output=True, text=True, env=env,
)
assert out.returncode == 0, out.stderr
return out.stdout
class _Sink(http.server.BaseHTTPRequestHandler):
seen: list[dict] = []
reply = b'{"context":"> family named","note_ids":[],"sync_note_ids":[],"derive_keys":["dup:483a"]}'
def do_GET(self):
type(self).seen.append(urllib.parse.parse_qs(urllib.parse.urlparse(self.path).query))
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.end_headers()
self.wfile.write(type(self).reply)
def log_message(self, *a):
pass
@pytest.fixture
def sink():
_Sink.seen = []
server = http.server.HTTPServer(("127.0.0.1", 0), _Sink)
threading.Thread(target=server.serve_forever, daemon=True).start()
try:
yield server
finally:
server.shutdown()
def test_after_write_names_what_bash_just_wrote_then_stays_quiet_until_the_next_change(tmp_path, sink):
env = _env(tmp_path, url=f"http://127.0.0.1:{sink.server_port}")
repo = _repo(tmp_path, env)
# "A Bash call" wrote an untracked stylesheet and appended to a tracked file.
(repo / "a.css").write_text(".log-empty {\n color: red;\n}\n")
(repo / "b.py").write_text("def one():\n return 1\n\ndef slug(t):\n return t\n")
out = _run(repo, env)
by_path = {q["path"][0]: q for q in _Sink.seen}
assert set(by_path) == {"a.css", "b.py"} # repo-relative, like the pre hook
assert by_path["a.css"]["shapes"] == ["css:log-empty"]
assert by_path["b.py"]["shapes"] == ["sym:slug"]
# Added lines only for the tracked file — the existing def is not "just written".
assert "def slug" in by_path["b.py"]["code"][0] and "def one" not in by_path["b.py"]["code"][0]
ctx = json.loads(out)["hookSpecificOutput"]
assert ctx["hookEventName"] == "PostToolUse"
assert "> family named" in ctx["additionalContext"]
# The local by-name arm rides along: `slug` already lives in c.py.
assert "`slug` is already defined in 1 other file(s): c.py" in ctx["additionalContext"]
# Derive keys landed on the SHARED channel the pre-write hook reads.
state = tmp_path / "scribe-priorart" / "s-after-1.derive.ids"
assert "dup:483a" in state.read_text().split()
# Nothing changed → one git status, no request, no output.
_Sink.seen = []
assert _run(repo, env) == ""
assert _Sink.seen == []
# Another change → only that file, and the dedup channel goes back up.
(repo / "a.css").write_text(".log-empty {\n color: red;\n}\n.other {\n margin: 0;\n}\n")
_run(repo, env)
assert [q["path"][0] for q in _Sink.seen] == ["a.css"]
assert _Sink.seen[0]["exclude_derive"] == ["dup:483a"]
assert set(_Sink.seen[0]["shapes"][0].split(",")) == {"css:log-empty", "css:other"}
def test_after_write_is_silent_where_it_has_nothing_to_say(tmp_path):
env = _env(tmp_path)
repo = _repo(tmp_path, env)
# Not a Bash call → nothing (hooks.json matches Bash, the script re-checks).
(repo / "a.css").write_text(".x {\n color: red;\n}\n")
assert _run(repo, env, tool="Write") == ""
# Not a git repo → nothing.
loose = tmp_path / "loose"
loose.mkdir()
(loose / "a.css").write_text(".x {\n color: red;\n}\n")
assert _run(loose, env, session="s-loose") == ""
# A change that defines nothing (prose, a call-site edit) → nothing, even
# with the server unreachable (port 9 refuses): no definitions, no arms.
(repo / "README.md").write_text("# notes\n")
(repo / "b.py").write_text("def one():\n return one_more()\n")
assert _run(repo, env, session="s-quiet") == ""
def test_after_write_local_arm_and_record_nudge_work_without_a_server(tmp_path):
"""The local by-name arm needs no instance (#2280) and the record nudge
(#2664) fails open with it — a refused connection stands in for the
instance."""
env = _env(tmp_path)
repo = _repo(tmp_path, env)
(repo / "d.py").write_text("def slug(t):\n return t.lower()\n")
out = _run(repo, env, session="s-local")
ctx = json.loads(out)["hookSpecificOutput"]["additionalContext"]
assert "`slug` is already defined in 1 other file(s): c.py" in ctx
assert "create_snippet" in ctx
+76
View File
@@ -597,6 +597,82 @@ async def test_derive_groups_land_on_rows_and_in_the_summary(seeded):
assert {r.symbol for r in rows} == {"slug"} # 2 files < the name floor assert {r.symbol for r in rows} == {"slug"} # 2 files < the name floor
@pytest.mark.integration
async def test_write_time_derive_names_the_family_or_the_canon_for_a_name(seeded):
"""#2900: against real rows — a name in a dup family → the family (other
files, count); a name whose canonical row lives elsewhere → that canon;
a judged row at the path, the canon's own file, or an unknown name →
silence."""
from scribe.services.shape_ledger import apply_derive_groups, write_time_derive
owner, pid, sid = seeded["owner"], seeded["pid"], seeded["snippet"]
defs = _defs(
("v/A.vue", "css", "log-empty", ".log-empty {", ".log-empty { color: red }"),
("v/B.vue", "css", "log-empty", ".log-empty {", ".log-empty { color: red }"),
("v/C.vue", "css", "log-empty", ".log-empty {", ".log-empty { color: red }"),
("src/factory.py", "sym", "factory", "def factory():", "def factory():\n return 1"),
)
await sync_repo_shapes(pid, REPO, defs, seen_marker="m1")
await classify_shapes(owner, pid, [
{"path": "src/factory.py", "symbol": "factory", "status": "canonical", "snippet_id": sid},
])
assert await apply_derive_groups(pid) >= 3
# A 4th copy about to be written → the family, naming the other files.
out = await write_time_derive(pid, "v/D.vue", [("css", "log-empty"), ("css", "unknown")])
assert len(out) == 1 and out[0]["symbol"] == "log-empty" and out[0]["kind"] == "css"
fam = out[0]["family"]
assert fam["identical"] is True and fam["label"] == ".log-empty"
assert fam["files"] == ["v/A.vue", "v/B.vue", "v/C.vue"] and fam["file_count"] == 3
assert out[0]["key"] == fam["group"] and fam["group"].startswith("dup:")
# Editing one existing member still names the OTHER members.
out = await write_time_derive(pid, "v/A.vue", [("css", "log-empty")])
assert out[0]["family"]["files"] == ["v/B.vue", "v/C.vue"] and out[0]["family"]["size"] == 3
# The canon's name elsewhere → the canon; in the canon's own file → silence.
out = await write_time_derive(pid, "src/other.py", [("sym", "factory")])
assert out == [{"symbol": "factory", "kind": "sym", "key": f"canon:{sid}",
"canon": {"snippet_id": sid, "path": "src/factory.py", "label": "factory"}}]
assert await write_time_derive(pid, "src/factory.py", [("sym", "factory")]) == []
# A judged row at the path is not re-litigated.
await classify_shapes(owner, pid, [
{"path": "v/B.vue", "symbol": "log-empty", "status": "exempt", "reason": "print sheet"},
])
assert await write_time_derive(pid, "v/B.vue", [("css", "log-empty")]) == []
@pytest.mark.integration
async def test_derive_new_names_the_copy_that_joined_a_family_since_the_stamp(seeded):
"""#2899: the first sync seeds one `slug`; a later sync adds an identical
copy. Against the stamp between them, derive_new counts ONLY the
newcomer — the drift since the last refresh, not the whole family."""
from datetime import datetime, timezone
from scribe.services.shape_ledger import (
apply_derive_groups, derive_new_summary, live_rows,
)
owner, pid = seeded["owner"], seeded["pid"]
first = _defs(
("a/one.py", "sym", "slug", "def slug(t):", "def slug(t):\n return t.lower()"),
)
await sync_repo_shapes(pid, REPO, first, seen_marker="m1")
stamp = datetime.now(timezone.utc)
second = _defs(
("a/one.py", "sym", "slug", "def slug(t):", "def slug(t):\n return t.lower()"),
("a/two.py", "sym", "slug", "def slug(t):", "def slug(t):\n return t.lower()"),
)
await sync_repo_shapes(pid, REPO, second, seen_marker="m2")
assert await apply_derive_groups(pid) == 2
rows = await live_rows(pid)
out = derive_new_summary(rows, since=stamp)
assert out["count"] == 1
assert out["examples"][0]["path"] == "a/two.py"
assert out["examples"][0]["label"] == "slug"
assert out["examples"][0]["group"].startswith("dup:")
assert derive_new_summary(rows, since=None)["count"] == 0
# --- #2793: the divergence readout against real rows ------------------------- # --- #2793: the divergence readout against real rows -------------------------
+37
View File
@@ -276,6 +276,8 @@ async def test_coverage_measures_the_tree_exactly_and_caches(seeded):
assert coverage["largest_gaps"] == [ assert coverage["largest_gaps"] == [
{"dir": "src", "unclassified": 2, "total": 3} {"dir": "src", "unclassified": 2, "total": 3}
] ]
# #2899: a first computation has no previous stamp — nothing is "new".
assert coverage["derive_new"] == {"count": 0, "examples": []}
# The walk fed the LEDGER (#2788): every extracted shape has a row, the # The walk fed the LEDGER (#2788): every extracted shape has a row, the
# snippet reference locations are mechanically stamped canonical WITH # snippet reference locations are mechanically stamped canonical WITH
@@ -512,6 +514,41 @@ def test_coverage_line_names_the_proposers_standing():
assert "top canon #2844 ×78" in line and "top copy closed-msg (identical body) ×3 files" in line assert "top canon #2844 ×78" in line and "top copy closed-msg (identical body) ×3 files" in line
def test_coverage_line_shows_standing_work_even_with_nothing_unclassified():
"""#2899: since the scoped bucket a ledger can be fully accounted and
still carry derive groups / proposals / divergence — the line names
them as `standing:` instead of hiding them behind the todo count, and
names the drift since the previous refresh first-copy-first."""
from scribe.services.coverage import coverage_line
base = {
"total": 4693, "accounted": 4693, "unclassified": 0,
"counts": {"canonical": 37, "instance": 977, "variant": 73, "exempt": 1797, "scoped": 1809},
"computed_at": "2026-08-22T00:00:00+00:00", "largest_gaps": [],
}
quiet = coverage_line(base)
assert "unclassified" not in quiet and "standing" not in quiet
line = coverage_line({
**base,
"derive_groups": [{"group": "dup:abc", "label": "log-empty (identical body)", "files": 4}],
"derive_new": {"count": 2, "examples": [
{"label": ".error-msg", "path": "frontend/src/components/InceptionCard.vue", "group": "dup:9f0"},
{"label": ".error-msg", "path": "frontend/src/components/Other.vue", "group": "dup:9f0"},
]},
"divergent": 1,
})
assert "; standing: 1 derive group, +2 new copies since last refresh: .error-msg in " \
"frontend/src/components/InceptionCard.vue, 1 DIVERGENT, top copy log-empty (identical body) ×4 files" in line
assert "unclassified" not in line
# One copy reads singular; with a todo the block keeps its old place.
one = coverage_line({**base, "derive_new": {"count": 1, "examples": []}})
assert one.endswith("; standing: +1 new copy since last refresh")
todo = coverage_line({**base, "unclassified": 3, "accounted": 4690, "proposed": 2,
"derive_new": {"count": 1, "examples": [{"label": "x", "path": "a.py"}]},
"largest_gaps": [{"dir": "src", "unclassified": 3, "total": 9}]})
assert "; 3 unclassified (2 proposed, +1 new copy since last refresh: x in a.py), largest: src" in todo
def test_coverage_line_names_divergence_and_recheck(): def test_coverage_line_names_divergence_and_recheck():
from scribe.services.coverage import coverage_line from scribe.services.coverage import coverage_line
+32
View File
@@ -317,6 +317,38 @@ def test_derive_groups_copy_before_name_with_floors():
assert ("i.py", "sym", "one") not in g assert ("i.py", "sym", "one") not in g
def test_derive_new_summary_counts_copies_first_seen_since_the_previous_refresh():
"""#2899: the arrival-moment drift signal — derive-grouped rows created
after the previous refresh's stamp, newest first, judged rows and a
first seed (since=None) never count."""
from datetime import datetime, timedelta, timezone
from scribe.models.code_shape import CodeShape
from scribe.services.shape_ledger import derive_new_summary
t0 = datetime(2026, 8, 22, 12, 0, tzinfo=timezone.utc)
def row(path, symbol, at, kind="css", status="scoped", group="dup:abc", basis="derive"):
r = CodeShape(project_id=2, repo_key="r", path=path, symbol=symbol, kind=kind,
status=status, proposal_basis=basis, proposal_group=group)
r.created_at = at
return r
rows = [
row("v/Old.vue", "error-msg", t0 - timedelta(days=3)), # before the stamp
row("v/InceptionCard.vue", "error-msg", t0 + timedelta(hours=1)), # new copy
row("v/Other.vue", "error-msg", t0 + timedelta(hours=2)), # newer copy
row("v/J.vue", "error-msg", t0 + timedelta(hours=3), status="exempt"), # judged: never
row("s/a.py", "load", t0 + timedelta(hours=1), kind="sym", group=None, basis=None), # no family
]
out = derive_new_summary(rows, since=t0)
assert out["count"] == 2
assert [e["path"] for e in out["examples"]] == ["v/Other.vue", "v/InceptionCard.vue"]
assert out["examples"][0] == {"label": ".error-msg", "path": "v/Other.vue", "group": "dup:abc"}
assert derive_new_summary(rows, since=None) == {"count": 0, "examples": []}
assert derive_new_summary(rows, since=t0, top=1)["examples"] == [
{"label": ".error-msg", "path": "v/Other.vue", "group": "dup:abc"}]
def test_proposal_summary_ranks_body_identical_groups_first_and_sees_scoped_rows(): def test_proposal_summary_ranks_body_identical_groups_first_and_sees_scoped_rows():
"""#2872: dup groups (the real copies) outrank name groups (usually """#2872: dup groups (the real copies) outrank name groups (usually
convention), wider spread first; #2869: scoped rows are in the readout.""" convention), wider spread first; #2869: scoped rows are in the readout."""
+121 -5
View File
@@ -853,14 +853,18 @@ def test_hook_exits_silently_when_unconfigured():
def test_hook_skips_prose_and_data_files(): def test_hook_skips_prose_and_data_files():
"""No round-trip for a markdown edit — the server would return nothing anyway.""" """No round-trip for a markdown edit — the server would return nothing
src = HOOK.read_text() anyway. The list lives in the shared library (#2901) and the hook asks it."""
skip = re.search(r"case \"\$file_path\" in\n(.*?)esac", src, re.S) lib = (PLUGIN / "hooks" / "scribe_defs.sh").read_text()
assert skip, "expected an extension skip list" skip = re.search(r"scribe_skip_path\(\) \{\n case \"\$1\" in\n(.*?)esac", lib, re.S)
assert skip, "expected an extension skip list in scribe_defs.sh"
for ext in ("*.md", "*.json", "*.lock", "*.png"): for ext in ("*.md", "*.json", "*.lock", "*.png"):
assert ext in skip.group(1) assert ext in skip.group(1)
# Config formats are deliberately NOT skipped — a workflow file is reusable. # Config formats are deliberately NOT skipped — a workflow file is reusable.
assert "*.yml" not in skip.group(1) assert "*.yml" not in skip.group(1)
src = HOOK.read_text()
assert 'scribe_skip_path "$file_path" && exit 0' in src
assert '/scribe_defs.sh"' in src # sourced, not copied
def test_plugin_version_bumped_with_the_hook(): def test_plugin_version_bumped_with_the_hook():
@@ -1150,7 +1154,9 @@ def test_hook_names_the_shapes_being_written():
that changes a body, not a signature — the definition enclosing the edit, that changes a body, not a signature — the definition enclosing the edit,
found by walking the target file upward from the edited lines.""" found by walking the target file upward from the edited lines."""
src = HOOK.read_text() src = HOOK.read_text()
assert "scribe_defs()" in src # one extractor, two consumers lib = (PLUGIN / "hooks" / "scribe_defs.sh").read_text()
assert "scribe_defs()" in lib # one extractor, shared (#2901)
assert "scribe_defs()" not in src # ...not a second copy here
assert ".tool_input.old_string" in src # the Edit's anchor assert ".tool_input.old_string" in src # the Edit's anchor
assert "| tac | scribe_defs | head -1" in src # nearest definition above assert "| tac | scribe_defs | head -1" in src # nearest definition above
# The ledger feed sends NAMES, never bodies, and stays on the one GET. # The ledger feed sends NAMES, never bodies, and stays on the one GET.
@@ -1243,6 +1249,116 @@ def test_hook_sends_the_enclosing_definition_for_a_body_edit(tmp_path):
assert seen["shapes"] == ["sym:onTrash"] assert seen["shapes"] == ["sym:onTrash"]
@pytest.mark.asyncio
async def test_the_write_time_derive_check_names_a_family_or_a_canon_in_band():
"""#2900: the ledger's own word on the names being written — a duplicate
family with no canon, or a canon recorded elsewhere — rendered at the
write even when nothing else does; keyed so the session's exclude
channel silences a family already named."""
from scribe.services import plugin_context as pc
found = [
{"symbol": "log-empty", "kind": "css", "key": "dup:483a",
"family": {"group": "dup:483a", "label": ".log-empty", "identical": True,
"files": ["a/TaskLogSection.vue", "a/WorkspaceTaskPanel.vue"],
"file_count": 5, "size": 6}},
{"symbol": "btn-primary", "kind": "css", "key": "canon:2855",
"canon": {"snippet_id": 2855, "path": "frontend/src/assets/components.css",
"label": ".btn-primary"}},
{"symbol": "load", "kind": "sym", "key": "name:sym:load",
"family": {"group": "name:sym:load", "label": "load", "identical": False,
"files": ["a/X.vue", "a/Y.vue", "a/Z.vue"], "file_count": 3, "size": 4}},
]
check = AsyncMock(return_value=found)
patches = dict(
get_writepath_config=AsyncMock(return_value=_cfg()),
semantic_search_notes=AsyncMock(return_value=[]),
record_retrieval=MagicMock(), owner_names_for=AsyncMock(return_value={}),
)
with patch.multiple(pc, **patches), \
patch.object(pc.snippets_svc, "list_snippets", AsyncMock(return_value=([], 0))), \
patch.object(pc.shape_ledger_svc, "recent_pulls", AsyncMock(return_value={})), \
patch.object(pc.shape_ledger_svc, "write_time_divergence", AsyncMock(return_value=[])), \
patch.object(pc.shape_ledger_svc, "write_time_derive", check):
out = await pc.build_write_path_hint(
1, "frontend/src/components/New.vue", code=REAL_CODE, project_id=24,
stamp_shapes=[("css", "log-empty"), ("css", "btn-primary"), ("sym", "load")],
exclude_derive=["name:sym:load"],
)
check.assert_awaited_once_with(24, "frontend/src/components/New.vue",
[("css", "log-empty"), ("css", "btn-primary"), ("sym", "load")])
# The excluded family is gone; the other two render and are keyed.
assert [d["key"] for d in out["derive"]] == ["dup:483a", "canon:2855"]
assert out["derive_keys"] == ["dup:483a", "canon:2855"]
ctx = out["context"]
assert "Shape ledger at `frontend/src/components/New.vue`" in ctx
assert "`.log-empty` is a duplicate family with no canon — identical body in 5 other file(s): " \
"`a/TaskLogSection.vue`, `a/WorkspaceTaskPanel.vue` +3 more; derive it now" in ctx
assert "`.btn-primary` is canon — snippet #2855 at `frontend/src/assets/components.css`" in ctx
assert "convention-plumbing" in ctx
assert "`load`" not in ctx
# No project → no check; a failing check never sinks the hint.
check.reset_mock()
with patch.multiple(pc, **patches), \
patch.object(pc.snippets_svc, "list_snippets", AsyncMock(return_value=([], 0))), \
patch.object(pc.shape_ledger_svc, "recent_pulls", AsyncMock(return_value={})), \
patch.object(pc.shape_ledger_svc, "write_time_divergence", AsyncMock(return_value=[])), \
patch.object(pc.shape_ledger_svc, "write_time_derive", check):
out = await pc.build_write_path_hint(1, "x.py", code=REAL_CODE, stamp_shapes=[("sym", "f")])
check.assert_not_awaited()
assert out["derive"] == [] and out["derive_keys"] == []
boom = AsyncMock(side_effect=RuntimeError("ledger down"))
with patch.multiple(pc, **patches), \
patch.object(pc.snippets_svc, "list_snippets", AsyncMock(return_value=([], 0))), \
patch.object(pc.shape_ledger_svc, "recent_pulls", AsyncMock(return_value={})), \
patch.object(pc.shape_ledger_svc, "write_time_divergence", AsyncMock(return_value=[])), \
patch.object(pc.shape_ledger_svc, "write_time_derive", boom):
out = await pc.build_write_path_hint(1, "x.py", code=REAL_CODE, project_id=24,
stamp_shapes=[("sym", "f")])
assert out["context"] == "" and out["derive"] == []
def test_the_hook_keeps_a_derive_channel_and_sends_it_back(tmp_path):
"""#2900: derive keys the server returns land in the session's own
`.derive.ids` file and go back as `exclude_derive` on the next write —
a family is named once per session, not at every edit."""
import http.server
import threading
import urllib.parse
seen: list[dict] = []
class _Sink(http.server.BaseHTTPRequestHandler):
def do_GET(self):
seen.append(urllib.parse.parse_qs(urllib.parse.urlparse(self.path).query))
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.end_headers()
self.wfile.write(b'{"context":"> family","note_ids":[],"sync_note_ids":[],'
b'"derive_keys":["dup:483a","canon:2855"]}')
def log_message(self, *a):
pass
server = http.server.HTTPServer(("127.0.0.1", 0), _Sink)
threading.Thread(target=server.serve_forever, daemon=True).start()
try:
env = dict(_hook_runtime_env(), SCRIBE_URL=f"http://127.0.0.1:{server.server_port}",
TMPDIR=str(tmp_path))
payload = {"session_id": "s-derive-1", "cwd": str(tmp_path), "tool_name": "Write",
"tool_input": {"file_path": str(tmp_path / "a.css"),
"content": ".log-empty {\n color: red;\n}\n"}}
for _ in range(2):
out = subprocess.run(["bash", str(HOOK)], input=json.dumps(payload),
capture_output=True, text=True, env=env)
assert out.returncode == 0, out.stderr
finally:
server.shutdown()
assert "exclude_derive" not in seen[0]
assert seen[1]["exclude_derive"] == ["dup:483a,canon:2855"]
state = tmp_path / "scribe-priorart" / "s-derive-1.derive.ids"
assert state.read_text().split() == ["dup:483a", "canon:2855", "dup:483a", "canon:2855"]
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_the_write_time_divergence_check_is_named_in_band(): async def test_the_write_time_divergence_check_is_named_in_band():
"""#2793: the hook named a shape at a path whose directory a canon """#2793: the hook named a shape at a path whose directory a canon