Merge pull request 'Self-surfacing DRY — duplicate families named at the write and on arrival, tool-agnostic (milestone 299 steps 1–4)' (#125) from dev into main
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 23s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 58s
CI & Build / Build & push image (push) Successful in 14s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 23s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 58s
CI & Build / Build & push image (push) Successful in 14s
This commit was merged in pull request #125.
This commit is contained in:
@@ -172,6 +172,8 @@ endpoint at `/mcp`, not these REST routes.
|
||||
| GET | `/api/plugin/context` | SessionStart context payload (rules + active-project) |
|
||||
| GET | `/api/plugin/retrieve` | Title-first knowledge-injection candidates |
|
||||
| GET | `/api/plugin/processes` | Stored Processes for skill-stub sync |
|
||||
| GET | `/api/plugin/prior-art` | Write-path hint for the plugin hooks (params: `path`, `code`, `repo`, `shapes`, `exclude_ids`, `exclude_sync_ids`, `exclude_derive`); returns `context`, `note_ids`, `sync_note_ids`, `stamped`, `divergence`, `derive`, `derive_keys` |
|
||||
| GET / POST | `/api/projects/<id>/coverage`, `…/coverage/refresh` | Shape-ledger accounting (`pattern_coverage` line, counts, `derive_groups`, `derive_new`, `divergence`, `recheck`) |
|
||||
| GET / PUT | `/api/plugin/marketplace-url` | Read / set the plugin marketplace URL |
|
||||
|
||||
## Dashboard, Export, Trash, Users
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "scribe",
|
||||
"description": "Scribe system-of-record for Claude Code: MCP tools over your notes/tasks/projects/rules, a session-start push channel that surfaces your always-on rules + active-project context, process-skills (writing-plans, systematic-debugging, verification, brainstorming, reusing-code), and your saved Scribe Processes auto-surfaced as skills (/scribe:sync). Replaces superpowers + file-memory with one app-backed plugin.",
|
||||
"version": "0.1.38",
|
||||
"version": "0.1.39",
|
||||
"author": { "name": "Bryan Van Deusen" },
|
||||
"mcpServers": {
|
||||
"scribe": {
|
||||
|
||||
+11
-1
@@ -52,8 +52,18 @@ On install you'll be asked for:
|
||||
but never stop it; silent when nothing is recorded, which is most of the time.
|
||||
Two framings: a REUSE menu (similar/nearby records), and a SYNC nudge when a
|
||||
snippet records the exact file being edited — "updating the record is part of
|
||||
the edit" — each with its own once-per-session dedup.
|
||||
the edit" — each with its own once-per-session dedup. A third, ledger-fed
|
||||
line names a duplicate family (no canon) or a canon recorded elsewhere for
|
||||
the names being written (its own dedup channel, `exclude_derive`).
|
||||
Toggle in **Settings → Knowledge auto-inject**.
|
||||
- `hooks/hooks.json` → PostToolUse hook on `Bash`
|
||||
(`hooks/scribe_after_write.sh`): code written through sed/heredocs/scripts
|
||||
never reaches the PreToolUse hook, so this one diffs the working tree after
|
||||
every Bash call (per-session path+blob snapshot; one `git status` when
|
||||
nothing changed) and runs the same arms on the definitions just written,
|
||||
through the same endpoint and the same dedup channels. `additionalContext`
|
||||
only; silent on any failure. The extractor, the prose/data skip list and the
|
||||
local by-name duplicate arm are shared in `hooks/scribe_defs.sh`.
|
||||
- `skills/` → the universal process-skills, surfaced by description match.
|
||||
- `hooks/scribe_sync_processes.sh` (a 2nd SessionStart hook) + the `/scribe:sync`
|
||||
command → generate `~/.claude/skills/scribe-proc-*` stubs from your Scribe
|
||||
|
||||
@@ -34,6 +34,17 @@
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"PostToolUse": [
|
||||
{
|
||||
"matcher": "Bash",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/scribe_after_write.sh\""
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,221 @@
|
||||
#!/usr/bin/env bash
|
||||
# Scribe plugin — PostToolUse write-path trigger on Bash (#2901).
|
||||
#
|
||||
# scribe_prior_art.sh fires before a Write/Edit TOOL CALL. Code written any
|
||||
# other way — sed, heredocs, python edit scripts, `cat > file` — never reached
|
||||
# it, so a whole class of edits (the ones a long session makes most) got no
|
||||
# prior-art hint, no ledger feed and no duplicate-family warning. This hook
|
||||
# closes that: after EVERY Bash call it asks git what changed in the working
|
||||
# tree since it last looked, and runs the same arms on the definitions that
|
||||
# were just written — the local by-name duplicate arm, the recorded prior-art
|
||||
# arms and the ledger's derive/divergence checks (#2900/#2793), via the same
|
||||
# /api/plugin/prior-art endpoint the pre-write hook uses.
|
||||
#
|
||||
# Post-hoc by a few seconds, in the same moment and the same session: "the
|
||||
# copy just landed; here is its family" — not "an audit found it later".
|
||||
#
|
||||
# Cheap when nothing changed: one `git status`. State per session, beside the
|
||||
# pre-write hook's (its three dedup channels are SHARED, so a family named by
|
||||
# one hook is not named again by the other):
|
||||
# ${TMPDIR:-/tmp}/scribe-afterwrite/<sid>.snap path<TAB>blob-hash of every
|
||||
# dirty/untracked file last seen
|
||||
# ${TMPDIR:-/tmp}/scribe-priorart/<sid>.* the dedup channels
|
||||
#
|
||||
# NEVER BLOCKS. It returns `additionalContext` only (no decision — there is
|
||||
# nothing left to decide, the write already happened). Any failure —
|
||||
# unconfigured, unreachable, not a git repo, malformed — exits 0 in silence.
|
||||
#
|
||||
# Config (same as the other hooks):
|
||||
# CLAUDE_PLUGIN_OPTION_API_ENDPOINT base URL, no trailing slash
|
||||
# CLAUDE_PLUGIN_OPTION_API_TOKEN fmcp_ API key (sensitive)
|
||||
# SCRIBE_URL / SCRIBE_TOKEN override for the settings.json dogfooding path.
|
||||
set -uo pipefail
|
||||
|
||||
command -v jq >/dev/null 2>&1 || exit 0
|
||||
command -v git >/dev/null 2>&1 || exit 0
|
||||
|
||||
# shellcheck source=plugin/hooks/scribe_defs.sh
|
||||
. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"
|
||||
|
||||
# PostToolUse delivers { session_id, cwd, tool_name, tool_input, tool_response }.
|
||||
event=$(cat 2>/dev/null || true)
|
||||
tool_name=$(printf '%s' "$event" | jq -r '.tool_name // empty' 2>/dev/null) || exit 0
|
||||
[ "$tool_name" = "Bash" ] || exit 0
|
||||
session_id=$(printf '%s' "$event" | jq -r '.session_id // empty' 2>/dev/null) || session_id=""
|
||||
event_cwd=$(printf '%s' "$event" | jq -r '.cwd // empty' 2>/dev/null) || event_cwd=""
|
||||
work_dir=${event_cwd:-${CLAUDE_PROJECT_DIR:-$PWD}}
|
||||
repo_root=$(git -C "$work_dir" rev-parse --show-toplevel 2>/dev/null) || exit 0
|
||||
[ -n "$repo_root" ] || exit 0
|
||||
|
||||
safe_sid=$(printf '%s' "${session_id:-nosession}" | tr -c 'A-Za-z0-9._-' '_')
|
||||
snap_dir="${TMPDIR:-/tmp}/scribe-afterwrite"
|
||||
mkdir -p "$snap_dir" 2>/dev/null || true
|
||||
snap="$snap_dir/${safe_sid}.snap"
|
||||
|
||||
# What is dirty now: every modified / added / untracked path, with the blob
|
||||
# hash of its working-tree content. Hash, not mtime: portable (no stat
|
||||
# flags), exact (a touch is not a change), and untracked files hash the same
|
||||
# way tracked ones do.
|
||||
current=""
|
||||
while IFS= read -r line; do
|
||||
[ -n "$line" ] || continue
|
||||
status=${line:0:2}
|
||||
path=${line:3}
|
||||
case "$status" in
|
||||
D*|*D) continue ;; # a deletion defines nothing
|
||||
esac
|
||||
case "$path" in
|
||||
*" -> "*) path=${path##* -> } ;; # rename: the new name
|
||||
esac
|
||||
# Porcelain quotes paths with special characters; those are skipped rather
|
||||
# than unquoted badly — a filename needing quotes is not where shapes live.
|
||||
case "$path" in
|
||||
\"*) continue ;;
|
||||
esac
|
||||
[ -f "$repo_root/$path" ] || continue
|
||||
sha=$(git -C "$repo_root" hash-object -- "$path" 2>/dev/null) || continue
|
||||
current="${current}${path}"$'\t'"${sha}"$'\n'
|
||||
done < <(git -C "$repo_root" status --porcelain --untracked-files=all 2>/dev/null)
|
||||
|
||||
previous=""
|
||||
[ -f "$snap" ] && previous=$(cat "$snap" 2>/dev/null || true)
|
||||
first_run=0
|
||||
[ -f "$snap" ] || first_run=1
|
||||
# Write the new snapshot NOW, before anything can fail below — the next call
|
||||
# must compare against this tree, whatever happens to this one's hint.
|
||||
printf '%s' "$current" > "$snap" 2>/dev/null || true
|
||||
|
||||
# Changed = a (path, hash) pair not in the previous snapshot. On the very
|
||||
# first call of a session there is no previous snapshot; rather than report
|
||||
# every pre-existing dirty file as "just written", take only files touched in
|
||||
# the last minute — the Bash call that just ran is the likely author.
|
||||
changed=""
|
||||
while IFS=$'\t' read -r path sha; do
|
||||
[ -n "${path:-}" ] || continue
|
||||
if [ "$first_run" = 1 ]; then
|
||||
[ -n "$(find "$repo_root/$path" -mmin -1 2>/dev/null)" ] || continue
|
||||
else
|
||||
case "$previous" in
|
||||
*"${path}"$'\t'"${sha}"*) continue ;;
|
||||
esac
|
||||
fi
|
||||
scribe_skip_path "$path" && continue
|
||||
changed="${changed}${path}"$'\n'
|
||||
done <<< "$current"
|
||||
[ -n "$changed" ] || exit 0
|
||||
|
||||
url=${SCRIBE_URL:-${CLAUDE_PLUGIN_OPTION_API_ENDPOINT:-}}
|
||||
token=${SCRIBE_TOKEN:-${CLAUDE_PLUGIN_OPTION_API_TOKEN:-}}
|
||||
case "$url" in *'${'*) url="" ;; esac
|
||||
case "$token" in *'${'*) token="" ;; esac
|
||||
repo=$(git -C "$repo_root" remote get-url origin 2>/dev/null || true)
|
||||
repo_q=""
|
||||
if [ -n "$repo" ]; then
|
||||
enc=$(printf '%s' "$repo" | jq -sRr '@uri' 2>/dev/null) || enc=""
|
||||
[ -n "$enc" ] && repo_q="&repo=${enc}"
|
||||
fi
|
||||
|
||||
# The dedup channels are the PRE-write hook's files, on purpose (see header).
|
||||
state_dir="${TMPDIR:-/tmp}/scribe-priorart"
|
||||
mkdir -p "$state_dir" 2>/dev/null || true
|
||||
idfile="$state_dir/${safe_sid}.ids"
|
||||
syncfile="$state_dir/${safe_sid}.sync.ids"
|
||||
derivefile="$state_dir/${safe_sid}.derive.ids"
|
||||
|
||||
combined=""
|
||||
n_files=0
|
||||
while IFS= read -r rel_path; do
|
||||
[ -n "${rel_path:-}" ] || continue
|
||||
# A Bash call that rewrote many files is a refactor or a generator, not a
|
||||
# shape being instantiated; four is enough to name what matters.
|
||||
n_files=$((n_files + 1))
|
||||
[ "$n_files" -le 4 ] || break
|
||||
file_path="$repo_root/$rel_path"
|
||||
|
||||
# The code just written: the ADDED lines of the uncommitted diff for a
|
||||
# tracked file (sed, not cut: this strips one marker char per line, it is
|
||||
# not a payload cap), the whole file when untracked.
|
||||
if git -C "$repo_root" ls-files --error-unmatch -- "$rel_path" >/dev/null 2>&1; then
|
||||
code=$(git -C "$repo_root" diff -U0 -- "$rel_path" 2>/dev/null | grep '^+' | grep -v '^+++' | sed 's/^+//') || code=""
|
||||
else
|
||||
code=$(cat "$file_path" 2>/dev/null) || code=""
|
||||
fi
|
||||
[ -n "$code" ] || continue
|
||||
names=$(printf '%s' "$code" | scribe_defs | sort -u | head -12) || names=""
|
||||
# Nothing DEFINED in what was written (prose, data, a call-site edit) →
|
||||
# nothing to say; the arms are about shapes.
|
||||
[ -n "$names" ] || continue
|
||||
|
||||
local_lines=$(scribe_local_dups "$repo_root" "$rel_path" <<< "$names") || local_lines=""
|
||||
local_context=""
|
||||
if [ -n "$local_lines" ]; then
|
||||
local_context="> Already defined elsewhere in this repo — \`${rel_path}\` (just written) adds another copy; check before keeping it (\`git grep\` shown; a nudge, not a gate):"$'\n'"${local_lines}"
|
||||
fi
|
||||
|
||||
context=""
|
||||
body=""
|
||||
if [ -n "$url" ] && [ -n "$token" ]; then
|
||||
q=$(printf '%s' "$code" | head -c 1200)
|
||||
path_enc=$(printf '%s' "$rel_path" | jq -sRr '@uri' 2>/dev/null) || path_enc=""
|
||||
code_enc=$(printf '%s' "$q" | jq -sRr '@uri' 2>/dev/null) || code_enc=""
|
||||
shapes_q=""
|
||||
enc=$(printf '%s\n' "$names" \
|
||||
| awk -F'\t' 'NF>=2 {printf "%s%s:%s", (n++?",":""), $1, $2}' \
|
||||
| jq -sRr '@uri' 2>/dev/null) || enc=""
|
||||
[ -n "$enc" ] && shapes_q="&shapes=${enc}"
|
||||
exclude_q=""; sync_exclude_q=""; derive_exclude_q=""
|
||||
if [ -f "$idfile" ]; then
|
||||
seen=$(tr '\n' ',' < "$idfile" 2>/dev/null | sed 's/,$//')
|
||||
[ -n "$seen" ] && exclude_q="&exclude_ids=${seen}"
|
||||
fi
|
||||
if [ -f "$syncfile" ]; then
|
||||
sync_seen=$(tr '\n' ',' < "$syncfile" 2>/dev/null | sed 's/,$//')
|
||||
[ -n "$sync_seen" ] && sync_exclude_q="&exclude_sync_ids=${sync_seen}"
|
||||
fi
|
||||
if [ -f "$derivefile" ]; then
|
||||
derive_seen=$(tr '\n' ',' < "$derivefile" 2>/dev/null | sed 's/,$//' | jq -sRr '@uri' 2>/dev/null) || derive_seen=""
|
||||
[ -n "$derive_seen" ] && derive_exclude_q="&exclude_derive=${derive_seen}"
|
||||
fi
|
||||
if [ -n "$path_enc" ]; then
|
||||
body=$(curl -fsS --max-time 4 \
|
||||
-H "Authorization: Bearer ${token}" \
|
||||
"${url%/}/api/plugin/prior-art?path=${path_enc}&code=${code_enc}${repo_q}${exclude_q}${sync_exclude_q}${derive_exclude_q}${shapes_q}" 2>/dev/null) || body=""
|
||||
fi
|
||||
if [ -n "$body" ]; then
|
||||
context=$(printf '%s' "$body" | jq -r '.context // empty' 2>/dev/null) || context=""
|
||||
if [ -n "$context" ]; then
|
||||
printf '%s' "$body" | jq -r '((.note_ids // []) - (.sync_note_ids // []))[]?' 2>/dev/null >> "$idfile" || true
|
||||
printf '%s' "$body" | jq -r '(.sync_note_ids // [])[]?' 2>/dev/null >> "$syncfile" || true
|
||||
printf '%s' "$body" | jq -r '(.derive_keys // [])[]?' 2>/dev/null >> "$derivefile" || true
|
||||
# Several files in one call may name the same family: keep each
|
||||
# token once, so the next request's exclude list stays exact.
|
||||
for f in "$idfile" "$syncfile" "$derivefile"; do
|
||||
[ -s "$f" ] && { sort -u -o "$f" "$f" 2>/dev/null || true; }
|
||||
done
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# The record nudge (#2664), same gate as the pre-write hook: duplication
|
||||
# demonstrated locally AND nothing recorded for it.
|
||||
if [ -n "$local_lines" ]; then
|
||||
n_recorded=$(printf '%s' "$body" | jq -r '.note_ids | length' 2>/dev/null) || n_recorded=0
|
||||
if [ "${n_recorded:-0}" = "0" ] || [ "$n_recorded" = "" ]; then
|
||||
local_context="${local_context}"$'\n'"> None of those existing copies is recorded in Scribe. If the version just written is the canonical one — or this edit is consolidating the copies — record it now with create_snippet so the next session is offered it instead of writing another copy."
|
||||
fi
|
||||
fi
|
||||
|
||||
part="$local_context"
|
||||
if [ -n "$context" ]; then
|
||||
[ -n "$part" ] && part="${part}"$'\n'
|
||||
part="${part}${context}"
|
||||
fi
|
||||
[ -n "$part" ] || continue
|
||||
[ -n "$combined" ] && combined="${combined}"$'\n'
|
||||
combined="${combined}${part}"
|
||||
done <<< "$changed"
|
||||
|
||||
[ -n "$combined" ] || exit 0
|
||||
jq -n --arg c "$combined" \
|
||||
'{hookSpecificOutput: {hookEventName: "PostToolUse", additionalContext: $c}}'
|
||||
exit 0
|
||||
@@ -0,0 +1,109 @@
|
||||
#!/usr/bin/env bash
|
||||
# shellcheck shell=bash
|
||||
# Scribe plugin — the pieces the two write-path hooks share (#2901).
|
||||
#
|
||||
# scribe_prior_art.sh fires BEFORE a Write/Edit tool call; scribe_after_write.sh
|
||||
# fires AFTER a Bash tool call and diffs the working tree, so code written by
|
||||
# sed/heredocs/scripts gets the same prior-art and ledger checks. Both need the
|
||||
# same three things, kept here so they cannot drift apart:
|
||||
#
|
||||
# scribe_skip_path PATH formats that hold prose or data, not shapes
|
||||
# scribe_defs stdin code → "kind<TAB>name" per definition
|
||||
# scribe_local_dups ROOT REL "kind<TAB>name" lines on stdin → the by-name
|
||||
# local-duplicate lines (ARM 1, #2280)
|
||||
#
|
||||
# Sourced, not executed: `. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"`.
|
||||
|
||||
# Skip formats that hold prose or data rather than reusable code. Purely to
|
||||
# avoid a pointless round-trip — the server would return nothing for these
|
||||
# anyway. Config formats are NOT skipped: a CI workflow or a compose file is
|
||||
# often exactly the thing worth reusing.
|
||||
scribe_skip_path() {
|
||||
case "$1" in
|
||||
*.md|*.mdx|*.txt|*.rst|*.json|*.lock|*.log|*.csv|*.tsv|*.svg|*.png|*.jpg|*.jpeg|*.gif|*.ico|*.pdf)
|
||||
return 0 ;;
|
||||
esac
|
||||
return 1
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# kind<TAB>name for each thing a piece of code DEFINES, in source order. One
|
||||
# program, two consumers: the local duplicate arm (every definition in the
|
||||
# payload) and the ledger feed (#2791, below: the definitions being written,
|
||||
# or the one enclosing an Edit). Rule-for-rule mirrored by the server's
|
||||
# services/coverage.py extract_shapes — ledger rows are keyed by what THAT
|
||||
# sees, so the two must agree on what counts as a definition.
|
||||
scribe_defs() {
|
||||
awk '
|
||||
{
|
||||
# CSS class definition: .name { or .name,
|
||||
if (match($0, /^[[:space:]]*\.[A-Za-z][A-Za-z0-9_-]*[[:space:]]*[,{]/)) {
|
||||
t = $0; sub(/^[[:space:]]*\./, "", t); sub(/[[:space:]]*[,{].*$/, "", t)
|
||||
if (t != "") print "css\t" t; next
|
||||
}
|
||||
line = $0; sub(/^[[:space:]]+/, "", line)
|
||||
# Strip leading declaration modifiers so the definition keyword is the
|
||||
# first word regardless of language (export/pub/private/suspend/...).
|
||||
sub(/^((pub(\([a-z]+\))?|export|default|private|internal|protected|public|static|suspend|async|open|sealed|data|abstract|final|inline|unsafe|extern|override)[[:space:]]+)*/, "", line)
|
||||
# Go method with receiver: func (r *T) Name(
|
||||
if (match(line, /^func[[:space:]]*\([^)]*\)[[:space:]]*[A-Za-z_]/)) {
|
||||
t = line; sub(/^func[[:space:]]*\([^)]*\)[[:space:]]*/, "", t)
|
||||
sub(/[^A-Za-z0-9_].*$/, "", t)
|
||||
if (t != "") print "sym\t" t; next
|
||||
}
|
||||
# Keyword-announced definitions, functions and named types alike.
|
||||
# Dunders are skipped: every class defines __init__, so "already defined
|
||||
# in N other files" is guaranteed noise for them — and noise is what
|
||||
# teaches sessions to skip the hint.
|
||||
if (match(line, /^(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+[A-Za-z_$]/)) {
|
||||
t = line; sub(/^[a-z]+[[:space:]]+/, "", t)
|
||||
sub(/[^A-Za-z0-9_$].*$/, "", t)
|
||||
if (t != "" && t !~ /^__.*__$/) print "sym\t" t; next
|
||||
}
|
||||
# Arrow/expression assignment: const name = (…) / let name = async (
|
||||
if (match(line, /^(const|let)[[:space:]]+[A-Za-z_$][A-Za-z0-9_$]*[[:space:]]*=[[:space:]]*(async[[:space:]]*)?[(<]/)) {
|
||||
t = line; sub(/^(const|let)[[:space:]]+/, "", t)
|
||||
sub(/[^A-Za-z0-9_$].*$/, "", t)
|
||||
if (t != "") print "sym\t" t; next
|
||||
}
|
||||
}
|
||||
' 2>/dev/null
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ARM 1 — BY NAME, LOCALLY (#2280). Does a definition of this already exist?
|
||||
#
|
||||
# The recorded arms ask Scribe what was RECORDED; the ledger arm (#2900) asks
|
||||
# what a BOUND repo's ledger knows. A helper nobody recorded, in a repo nobody
|
||||
# bound, is invisible to both — which is how `.btn-primary` came to be defined
|
||||
# four times, already diverged. This arm asks the one question only the
|
||||
# developer's machine can answer, inside the repo, holding the code about to
|
||||
# be written: no index, no storage, no server — it runs even on an install
|
||||
# that has never configured Scribe.
|
||||
#
|
||||
# Definition-shaped patterns only. Grepping for bare occurrences would match
|
||||
# every CALL site and drown the real finding — and a hint that is mostly noise
|
||||
# is one people learn to skip, which is worse than none. ALL code, not a
|
||||
# language shortlist (#2682): the same keyword family scribe_defs announces.
|
||||
#
|
||||
# $1 repo root, $2 repo-relative path of the file being written (excluded from
|
||||
# the grep — it would always match itself on an Edit). Definitions on stdin.
|
||||
# Prints one "> - `name` is already defined in N other file(s): …" per hit.
|
||||
scribe_local_dups() {
|
||||
local root="$1" rel="$2" kind name pat hits count label files
|
||||
while IFS=$'\t' read -r kind name; do
|
||||
[ -n "${name:-}" ] || continue
|
||||
case "$kind" in
|
||||
css) pat="^[[:space:]]*\.${name}[[:space:]]*[,{]" ;;
|
||||
*) pat="(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+${name}[^A-Za-z0-9_]|func[[:space:]]*\([^)]*\)[[:space:]]*${name}[[:space:]]*\(|(const|let)[[:space:]]+${name}[[:space:]]*=" ;;
|
||||
esac
|
||||
# -I skips binaries; :(exclude) drops the file being written.
|
||||
hits=$(git -C "$root" grep -I -l -E -e "$pat" -- . ":(exclude)${rel}" 2>/dev/null | head -4) || hits=""
|
||||
[ -n "$hits" ] || continue
|
||||
count=$(printf '%s\n' "$hits" | grep -c . 2>/dev/null || echo 0)
|
||||
label=$([ "$kind" = css ] && printf '.%s' "$name" || printf '%s' "$name")
|
||||
files=$(printf '%s' "$hits" | tr '\n' ' ' | sed 's/ $//')
|
||||
printf '> - `%s` is already defined in %s other file(s): %s\n' "$label" "$count" "$files"
|
||||
done
|
||||
}
|
||||
@@ -51,14 +51,12 @@ code=$(printf '%s' "$event" | jq -r '
|
||||
.tool_input.content // .tool_input.file_content //
|
||||
.tool_input.new_string // .tool_input.new_str // empty' 2>/dev/null) || code=""
|
||||
|
||||
# Skip formats that hold prose or data rather than reusable code. Purely to
|
||||
# avoid a pointless round-trip — the server would return nothing for these
|
||||
# anyway. Config formats are NOT skipped: a CI workflow or a compose file is
|
||||
# often exactly the thing worth reusing.
|
||||
case "$file_path" in
|
||||
*.md|*.mdx|*.txt|*.rst|*.json|*.lock|*.log|*.csv|*.tsv|*.svg|*.png|*.jpg|*.jpeg|*.gif|*.ico|*.pdf)
|
||||
exit 0 ;;
|
||||
esac
|
||||
# Shared with the after-write hook (#2901): the prose/data skip list, the
|
||||
# definition extractor and the local by-name duplicate arm live in
|
||||
# scribe_defs.sh so the two hooks cannot drift apart.
|
||||
# shellcheck source=plugin/hooks/scribe_defs.sh
|
||||
. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"
|
||||
scribe_skip_path "$file_path" && exit 0
|
||||
|
||||
# Snippet locations are recorded repo-relative, so send a repo-relative path —
|
||||
# an absolute one would simply match nothing. Resolved BEFORE the config gate
|
||||
@@ -73,78 +71,8 @@ if [ -n "$repo_root" ]; then
|
||||
esac
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ARM 1 — BY NAME, LOCALLY (#2280). Does a definition of this already exist?
|
||||
#
|
||||
# The other two arms ask Scribe what was RECORDED. Scribe has never read a line
|
||||
# of the codebase, so a helper nobody thought to record is invisible to them —
|
||||
# which is how `.btn-primary` came to be defined four times, in four scoped
|
||||
# stylesheets, already diverged. It was never a snippet, so no threshold and no
|
||||
# query rewrite could ever have surfaced it.
|
||||
#
|
||||
# This arm closes that by asking the only question the record cannot answer,
|
||||
# in the only place that can: the hook already runs on the developer's machine,
|
||||
# inside the repo, holding the code about to be written. No index, no storage,
|
||||
# no staleness, and no server — it deliberately runs even on an install that
|
||||
# has never configured Scribe.
|
||||
#
|
||||
# Definition-shaped patterns only. Grepping for bare occurrences would match
|
||||
# every CALL site and drown the real finding — and a hint that is mostly noise
|
||||
# is one people learn to skip, which is worse than none.
|
||||
#
|
||||
# ALL code, not a language shortlist (#2682): the detector was born covering
|
||||
# only the languages of the repo it was written in, which silently amputated
|
||||
# this whole arm — and the record nudge gated on it — for every Go/Kotlin/Rust
|
||||
# project. Definitions are announced by a small keyword family across
|
||||
# languages (func/fun/fn/function/def/sub · class/struct/trait/interface/
|
||||
# enum/object/protocol/type), so one modifier-strip + keyword match covers
|
||||
# them all. Known out of scope: keyword-less declaration syntax (C/Java/Dart
|
||||
# `ReturnType name(...)`) needs a real parser, and `impl` blocks are excluded
|
||||
# because several per type is normal Rust, not duplication.
|
||||
# ---------------------------------------------------------------------------
|
||||
# kind<TAB>name for each thing a piece of code DEFINES, in source order. One
|
||||
# program, two consumers: the local duplicate arm (every definition in the
|
||||
# payload) and the ledger feed (#2791, below: the definitions being written,
|
||||
# or the one enclosing an Edit). Rule-for-rule mirrored by the server's
|
||||
# services/coverage.py extract_shapes — ledger rows are keyed by what THAT
|
||||
# sees, so the two must agree on what counts as a definition.
|
||||
scribe_defs() {
|
||||
awk '
|
||||
{
|
||||
# CSS class definition: .name { or .name,
|
||||
if (match($0, /^[[:space:]]*\.[A-Za-z][A-Za-z0-9_-]*[[:space:]]*[,{]/)) {
|
||||
t = $0; sub(/^[[:space:]]*\./, "", t); sub(/[[:space:]]*[,{].*$/, "", t)
|
||||
if (t != "") print "css\t" t; next
|
||||
}
|
||||
line = $0; sub(/^[[:space:]]+/, "", line)
|
||||
# Strip leading declaration modifiers so the definition keyword is the
|
||||
# first word regardless of language (export/pub/private/suspend/...).
|
||||
sub(/^((pub(\([a-z]+\))?|export|default|private|internal|protected|public|static|suspend|async|open|sealed|data|abstract|final|inline|unsafe|extern|override)[[:space:]]+)*/, "", line)
|
||||
# Go method with receiver: func (r *T) Name(
|
||||
if (match(line, /^func[[:space:]]*\([^)]*\)[[:space:]]*[A-Za-z_]/)) {
|
||||
t = line; sub(/^func[[:space:]]*\([^)]*\)[[:space:]]*/, "", t)
|
||||
sub(/[^A-Za-z0-9_].*$/, "", t)
|
||||
if (t != "") print "sym\t" t; next
|
||||
}
|
||||
# Keyword-announced definitions, functions and named types alike.
|
||||
# Dunders are skipped: every class defines __init__, so "already defined
|
||||
# in N other files" is guaranteed noise for them — and noise is what
|
||||
# teaches sessions to skip the hint.
|
||||
if (match(line, /^(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+[A-Za-z_$]/)) {
|
||||
t = line; sub(/^[a-z]+[[:space:]]+/, "", t)
|
||||
sub(/[^A-Za-z0-9_$].*$/, "", t)
|
||||
if (t != "" && t !~ /^__.*__$/) print "sym\t" t; next
|
||||
}
|
||||
# Arrow/expression assignment: const name = (…) / let name = async (
|
||||
if (match(line, /^(const|let)[[:space:]]+[A-Za-z_$][A-Za-z0-9_$]*[[:space:]]*=[[:space:]]*(async[[:space:]]*)?[(<]/)) {
|
||||
t = line; sub(/^(const|let)[[:space:]]+/, "", t)
|
||||
sub(/[^A-Za-z0-9_$].*$/, "", t)
|
||||
if (t != "") print "sym\t" t; next
|
||||
}
|
||||
}
|
||||
' 2>/dev/null
|
||||
}
|
||||
|
||||
# ARM 1 — BY NAME, LOCALLY (#2280): does a definition of this already exist
|
||||
# in the repo? (scribe_local_dups in scribe_defs.sh carries the why.)
|
||||
names=""
|
||||
if [ -n "$code" ]; then
|
||||
names=$(printf '%s' "$code" | scribe_defs | sort -u | head -12) || names=""
|
||||
@@ -152,21 +80,8 @@ fi
|
||||
|
||||
local_lines=""
|
||||
if [ -n "$repo_root" ] && [ -n "$names" ]; then
|
||||
while IFS=$'\t' read -r kind name; do
|
||||
[ -n "${name:-}" ] || continue
|
||||
case "$kind" in
|
||||
css) pat="^[[:space:]]*\.${name}[[:space:]]*[,{]" ;;
|
||||
*) pat="(function|def|class|func|fun|fn|sub|struct|trait|interface|enum|object|protocol|type)[[:space:]]+${name}[^A-Za-z0-9_]|func[[:space:]]*\([^)]*\)[[:space:]]*${name}[[:space:]]*\(|(const|let)[[:space:]]+${name}[[:space:]]*=" ;;
|
||||
esac
|
||||
# -I skips binaries; :(exclude) drops the file being written, which would
|
||||
# otherwise always match itself on an Edit.
|
||||
hits=$(git -C "$repo_root" grep -I -l -E -e "$pat" -- . ":(exclude)${rel_path}" 2>/dev/null | head -4) || hits=""
|
||||
[ -n "$hits" ] || continue
|
||||
count=$(printf '%s\n' "$hits" | grep -c . 2>/dev/null || echo 0)
|
||||
label=$([ "$kind" = css ] && printf '.%s' "$name" || printf '%s' "$name")
|
||||
files=$(printf '%s' "$hits" | tr '\n' ' ' | sed 's/ $//')
|
||||
local_lines="${local_lines}> - \`${label}\` is already defined in ${count} other file(s): ${files}"$'\n'
|
||||
done <<< "$names"
|
||||
local_lines=$(scribe_local_dups "$repo_root" "$rel_path" <<< "$names") || local_lines=""
|
||||
[ -n "$local_lines" ] && local_lines="${local_lines}"$'\n'
|
||||
fi
|
||||
|
||||
local_context=""
|
||||
@@ -258,14 +173,22 @@ fi
|
||||
# the sync nudge when the recorded file itself is edited later.
|
||||
state_dir="${TMPDIR:-/tmp}/scribe-priorart"
|
||||
mkdir -p "$state_dir" 2>/dev/null || true
|
||||
#
|
||||
# A THIRD channel (#2900): the ledger's derive arm names a duplicate family
|
||||
# (a derive group id) or a canon elsewhere (`canon:<snippet_id>`) for the
|
||||
# shapes being written. Keyed by that token, not a note id, so it dedups on
|
||||
# its own file and a family is named once per session, not at every edit.
|
||||
idfile=""
|
||||
syncfile=""
|
||||
derivefile=""
|
||||
exclude_q=""
|
||||
sync_exclude_q=""
|
||||
derive_exclude_q=""
|
||||
if [ -n "$session_id" ]; then
|
||||
safe_sid=$(printf '%s' "$session_id" | tr -c 'A-Za-z0-9._-' '_')
|
||||
idfile="$state_dir/${safe_sid}.ids"
|
||||
syncfile="$state_dir/${safe_sid}.sync.ids"
|
||||
derivefile="$state_dir/${safe_sid}.derive.ids"
|
||||
if [ -f "$idfile" ]; then
|
||||
seen=$(tr '\n' ',' < "$idfile" 2>/dev/null | sed 's/,$//')
|
||||
[ -n "$seen" ] && exclude_q="&exclude_ids=${seen}"
|
||||
@@ -274,13 +197,17 @@ if [ -n "$session_id" ]; then
|
||||
sync_seen=$(tr '\n' ',' < "$syncfile" 2>/dev/null | sed 's/,$//')
|
||||
[ -n "$sync_seen" ] && sync_exclude_q="&exclude_sync_ids=${sync_seen}"
|
||||
fi
|
||||
if [ -f "$derivefile" ]; then
|
||||
derive_seen=$(tr '\n' ',' < "$derivefile" 2>/dev/null | sed 's/,$//' | jq -sRr '@uri' 2>/dev/null) || derive_seen=""
|
||||
[ -n "$derive_seen" ] && derive_exclude_q="&exclude_derive=${derive_seen}"
|
||||
fi
|
||||
fi
|
||||
|
||||
# `|| true`, not `|| exit 0`: an unreachable instance must not discard a local
|
||||
# finding that needed no instance to produce.
|
||||
body=$(curl -fsS --max-time 5 \
|
||||
-H "Authorization: Bearer ${token}" \
|
||||
"${url%/}/api/plugin/prior-art?path=${path_enc}&code=${code_enc}${repo_q}${exclude_q}${sync_exclude_q}${shapes_q}" 2>/dev/null) || body=""
|
||||
"${url%/}/api/plugin/prior-art?path=${path_enc}&code=${code_enc}${repo_q}${exclude_q}${sync_exclude_q}${derive_exclude_q}${shapes_q}" 2>/dev/null) || body=""
|
||||
|
||||
context=""
|
||||
if [ -n "$body" ]; then
|
||||
@@ -295,6 +222,9 @@ if [ -n "$body" ]; then
|
||||
if [ -n "$syncfile" ]; then
|
||||
printf '%s' "$body" | jq -r '(.sync_note_ids // [])[]?' 2>/dev/null >> "$syncfile" || true
|
||||
fi
|
||||
if [ -n "$derivefile" ]; then
|
||||
printf '%s' "$body" | jq -r '(.derive_keys // [])[]?' 2>/dev/null >> "$derivefile" || true
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
@@ -66,7 +66,10 @@ for the operator's work, and as your own working memory across sessions.
|
||||
should read as a map of every shape in it. The backstop still holds:
|
||||
noticing the second copy of anything, or consolidating copies into a shared
|
||||
X, means X gets recorded before that work is finished — which is how a
|
||||
codebase is kept from growing four `.btn-primary` definitions.
|
||||
codebase is kept from growing four `.btn-primary` definitions. The write-path
|
||||
hooks (before a Write/Edit, and after any Bash call that changed the tree)
|
||||
name a known duplicate family or a canon elsewhere for what was just
|
||||
written — act on that line at the write, not at the next audit.
|
||||
- Do **not** keep the operator's rules, plans, or project notes in local
|
||||
memory / CLAUDE.md in parallel with Scribe — Scribe holds the single copy.
|
||||
- **Compact at clean seams** — because you record as you go, a context
|
||||
|
||||
@@ -44,6 +44,13 @@ through recall/auto-inject; this skill is the active reflex around that.
|
||||
it before you go any further. Either it's the helper you were about to
|
||||
duplicate — reuse it and drop yours — or it isn't, and the record needs the new
|
||||
location adding. Both are cheaper now than after the duplicate settles in.
|
||||
- **A `Shape ledger at …` line is the ledger speaking, not the record.** It
|
||||
names a duplicate family ("identical body in N other files, no canon") or a
|
||||
canon elsewhere for a name you just wrote — for edits made through Bash
|
||||
(sed, heredocs, scripts) as much as through Write/Edit. Derive the family or
|
||||
reuse the canon *now*; a family that is convention rather than copies is
|
||||
dismissed with `classify_shapes(..., status="exempt",
|
||||
reason_code="convention-plumbing")`, never ignored.
|
||||
- **A `[records this file]` hint is a duty, not a menu.** When the hint says a
|
||||
snippet records the very file you're editing, the record's freshness is now
|
||||
YOUR edit's responsibility: if the edit changes the recorded shape,
|
||||
|
||||
@@ -85,6 +85,34 @@ the dominant form, `create_snippet` it, migrate the outliers, then classify
|
||||
the rest as instances. Canon is determined from the code; consistency comes
|
||||
from the derivation, not from asking permission.
|
||||
|
||||
## Derive groups are drift, not audit material
|
||||
|
||||
The catalogue exists so the codebase is DRY **from inception**, not as DRY as
|
||||
the last sweep left it. Three surfaces say so without anyone running an audit
|
||||
(milestone 299):
|
||||
|
||||
- **At the write** — the prior-art hint (the Write/Edit hook, and since
|
||||
0.1.39 the after-write hook on Bash, so sed/heredoc/script edits count too)
|
||||
carries a `Shape ledger at <path>` line when a name just written is a known
|
||||
**duplicate family** ("identical body in N other files, no canon") or a
|
||||
**canon elsewhere** ("snippet #N at <path> — reuse, don't redefine"). Act
|
||||
on it *then*: pull the canon and build from it, or derive the family now —
|
||||
`create_snippet` the dominant form, repoint the copies, `classify_shapes`
|
||||
them `instance`. A family is named once per session.
|
||||
- **On arrival** — the coverage line's `standing:` block (shown even when
|
||||
nothing is unclassified) and `derive_new` ("+N new copies since last
|
||||
refresh: .x in <path>") name what drifted since the previous refresh. That
|
||||
is the todo of the moment, sized to the last batch — not a backlog.
|
||||
- **A family that is convention, not copies** — component-local `load` /
|
||||
`toggle` / `save` that happen to share a name — is dismissed, not
|
||||
consolidated: `classify_shapes(..., status="exempt",
|
||||
reason_code="convention-plumbing", reason=…)` (or `classify_shapes_by_rule`
|
||||
for a whole family) removes it from the queue. Dismissal is a judgment and
|
||||
it is recorded; silence is not.
|
||||
|
||||
After the one-time pay-down the derive queue reads empty; anything in it
|
||||
afterwards is drift of the moment, and the hint already said so at the write.
|
||||
|
||||
## The divergence readout — button B where button A is canon
|
||||
|
||||
Three questions the ledger answers mechanically (#2793):
|
||||
|
||||
@@ -203,6 +203,17 @@ SMOKE_EVENTS: dict[str, str] = {
|
||||
),
|
||||
"scribe_sync_processes.sh": json.dumps({"source": "startup"}),
|
||||
"scribe_session_context.sh": json.dumps({"source": "startup"}),
|
||||
# The after-write hook (#2901) diffs the working tree; on CI's clean
|
||||
# checkout there is nothing to report, so silence is the right assertion.
|
||||
# (On a dirty local tree with a definition just written it may speak —
|
||||
# that is the hook working, not a failure of the contract.)
|
||||
"scribe_after_write.sh": json.dumps(
|
||||
{"session_id": "smoke", "cwd": ".", "tool_name": "Bash",
|
||||
"tool_input": {"command": "true"}, "tool_response": {}}
|
||||
),
|
||||
# The shared library is sourced, never run; executed bare it defines
|
||||
# functions and exits — silent by construction.
|
||||
"scribe_defs.sh": "",
|
||||
}
|
||||
|
||||
# The one hook that legitimately produces output with no credentials.
|
||||
|
||||
@@ -295,7 +295,9 @@ async def refresh_pattern_coverage(project_id: int) -> dict:
|
||||
Returns the accounting payload — total, accounted, counts by status,
|
||||
unclassified, repos, largest_gaps, `proposed` (canon proposals awaiting
|
||||
confirmation), `derive_groups` (the biggest repeats-with-no-canon
|
||||
families), `proposer` (what this refresh examined) — plus
|
||||
families), `derive_new` (copies that joined a family since the previous
|
||||
refresh — the drift to act on now: derive the canon, don't queue an
|
||||
audit), `proposer` (what this refresh examined) — plus
|
||||
`pattern_coverage`, the same one-line summary enter_project carries.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
|
||||
@@ -129,6 +129,10 @@ async def write_path_prior_art():
|
||||
surfaced. A separate channel on purpose: a reuse
|
||||
hint shown early must not suppress the record-sync
|
||||
nudge when the recorded file is edited later.
|
||||
exclude_derive (opt) — comma-separated derive keys (a derive group id
|
||||
or `canon:<snippet_id>`) already named this
|
||||
session by the ledger arm (#2900); its own
|
||||
channel, like the two above.
|
||||
shapes (opt) — comma-separated `kind:name` definitions the hook
|
||||
found in (or enclosing) the payload, kind being
|
||||
css|sym. The shape ledger's write-path feed
|
||||
@@ -144,6 +148,9 @@ async def write_path_prior_art():
|
||||
project_id, repo, _unbound = await _project_scope()
|
||||
exclude_ids = _int_list(request.args.get("exclude_ids"))
|
||||
exclude_sync_ids = _int_list(request.args.get("exclude_sync_ids"))
|
||||
exclude_derive = [
|
||||
p.strip() for p in (request.args.get("exclude_derive") or "").split(",") if p.strip()
|
||||
]
|
||||
shapes = _parse_shapes(request.args.get("shapes") or "")
|
||||
api_key = getattr(g, "api_key", None)
|
||||
may_stamp = api_key is None or getattr(api_key, "scope", "") == "write"
|
||||
@@ -153,6 +160,7 @@ async def write_path_prior_art():
|
||||
exclude_ids=exclude_ids, exclude_sync_ids=exclude_sync_ids,
|
||||
stamp_shapes=shapes if may_stamp else None,
|
||||
repo_key=repo_bindings_svc.normalize_repo_key(repo) if repo else "",
|
||||
exclude_derive=exclude_derive,
|
||||
)
|
||||
return jsonify(result)
|
||||
|
||||
|
||||
@@ -462,15 +462,20 @@ async def compute_coverage(
|
||||
await shape_ledger.apply_derive_groups(project_id)
|
||||
except Exception:
|
||||
logger.warning("derive-first grouping failed", exc_info=True)
|
||||
# The button-B pass (#2793): shapes new since the PREVIOUS computation,
|
||||
# where a canon dominates. The previous computation's stamp is the cache;
|
||||
# a first seed has none, so it flags nothing (everything is new then).
|
||||
# "Since the previous computation" — the cache's stamp. A first seed has
|
||||
# none, so nothing is new then. Read once; two passes use it: the
|
||||
# button-B flag (#2793) and the derive-new drift count (#2899).
|
||||
since = None
|
||||
try:
|
||||
previous = await get_setting(user_id, f"{_CACHE_KEY_PREFIX}{project_id}")
|
||||
since = None
|
||||
if previous:
|
||||
stamp = (json.loads(previous) or {}).get("computed_at")
|
||||
since = datetime.fromisoformat(stamp) if stamp else None
|
||||
except Exception:
|
||||
logger.warning("previous coverage stamp unreadable", exc_info=True)
|
||||
# The button-B pass (#2793): shapes new since the PREVIOUS computation,
|
||||
# where a canon dominates.
|
||||
try:
|
||||
await shape_ledger.flag_divergence(project_id, since=since)
|
||||
except Exception:
|
||||
logger.warning("divergence pass failed", exc_info=True)
|
||||
@@ -491,6 +496,7 @@ async def compute_coverage(
|
||||
unclassified = counts.pop("unclassified")
|
||||
proposals = shape_ledger.proposal_summary(rows)
|
||||
divergence = shape_ledger.divergence_summary(rows)
|
||||
derive_new = shape_ledger.derive_new_summary(rows, since=since)
|
||||
return {
|
||||
"total": len(rows),
|
||||
"accounted": len(rows) - unclassified,
|
||||
@@ -501,6 +507,10 @@ async def compute_coverage(
|
||||
"proposed": proposals["proposed"],
|
||||
"derive_groups": proposals["derive_groups"],
|
||||
"top_canon": proposals.get("top_canon"),
|
||||
# Drift since the previous refresh (#2899): copies that joined a
|
||||
# duplicate family — what the arrival line names so drift is noticed
|
||||
# on entering, not found by an audit.
|
||||
"derive_new": derive_new,
|
||||
"proposer": proposer_stats,
|
||||
# The divergence readout (#2793): button B where button A is canon,
|
||||
# and judged shapes whose bodies moved since they were judged.
|
||||
@@ -648,29 +658,46 @@ def coverage_line(coverage: dict) -> str:
|
||||
line += f" — {breakdown}"
|
||||
line += f" (estimate{', computed ' + day if day else ''})"
|
||||
unclassified = coverage.get("unclassified", 0)
|
||||
# The standing work, built whatever the todo count (#2899). Since the
|
||||
# scoped bucket (#2869) a ledger can read 100% accounted and still carry
|
||||
# derive groups, proposals and divergence; gating this block on
|
||||
# `unclassified > 0` is how 439 derive rows went unmentioned.
|
||||
standing = []
|
||||
if coverage.get("proposed"):
|
||||
standing.append(f"{coverage['proposed']} proposed")
|
||||
n_groups = len(coverage.get("derive_groups") or [])
|
||||
if n_groups:
|
||||
standing.append(f"{n_groups} derive group{'s' if n_groups != 1 else ''}")
|
||||
# Drift since the previous refresh: copies that joined a family, the
|
||||
# first one named — the sentence the arrival moment exists to say.
|
||||
new = coverage.get("derive_new") or {}
|
||||
if new.get("count"):
|
||||
n = new["count"]
|
||||
first_new = (new.get("examples") or [{}])[0]
|
||||
where = (
|
||||
f": {first_new['label']} in {first_new['path']}"
|
||||
if first_new.get("label") and first_new.get("path") else ""
|
||||
)
|
||||
standing.append(f"+{n} new cop{'y' if n == 1 else 'ies'} since last refresh{where}")
|
||||
if coverage.get("divergent"):
|
||||
standing.append(f"{coverage['divergent']} DIVERGENT")
|
||||
# The next action, on the line (#2874): the canon with the biggest
|
||||
# queue to confirm, and the widest body-identical copy to consolidate.
|
||||
top = coverage.get("top_canon") or {}
|
||||
if top.get("snippet_id"):
|
||||
standing.append(f"top canon #{top['snippet_id']} ×{top.get('count', 0)}")
|
||||
first = (coverage.get("derive_groups") or [{}])[0]
|
||||
if first.get("label") and first.get("files"):
|
||||
standing.append(f"top copy {first['label']} ×{first['files']} files")
|
||||
if unclassified:
|
||||
line += f"; {unclassified} unclassified"
|
||||
standing = []
|
||||
if coverage.get("proposed"):
|
||||
standing.append(f"{coverage['proposed']} proposed")
|
||||
n_groups = len(coverage.get("derive_groups") or [])
|
||||
if n_groups:
|
||||
standing.append(f"{n_groups} derive group{'s' if n_groups != 1 else ''}")
|
||||
if coverage.get("divergent"):
|
||||
standing.append(f"{coverage['divergent']} DIVERGENT")
|
||||
# The next action, on the line (#2874): the canon with the biggest
|
||||
# queue to confirm, and the widest body-identical copy to consolidate.
|
||||
top = coverage.get("top_canon") or {}
|
||||
if top.get("snippet_id"):
|
||||
standing.append(f"top canon #{top['snippet_id']} ×{top.get('count', 0)}")
|
||||
first = (coverage.get("derive_groups") or [{}])[0]
|
||||
if first.get("label") and first.get("files"):
|
||||
standing.append(f"top copy {first['label']} ×{first['files']} files")
|
||||
if standing:
|
||||
line += f" ({', '.join(standing)})"
|
||||
gaps = [g["dir"] for g in coverage.get("largest_gaps") or []]
|
||||
if gaps:
|
||||
line += ", largest: " + ", ".join(gaps)
|
||||
elif standing:
|
||||
line += f"; standing: {', '.join(standing)}"
|
||||
if coverage.get("recheck"):
|
||||
line += f"; {coverage['recheck']} judged shape{'s' if coverage['recheck'] != 1 else ''} changed since judged — recheck"
|
||||
return line
|
||||
|
||||
@@ -706,6 +706,7 @@ async def build_write_path_hint(
|
||||
exclude_sync_ids: list[int] | None = None,
|
||||
stamp_shapes: list[tuple[str, str]] | None = None,
|
||||
repo_key: str = "",
|
||||
exclude_derive: list[str] | None = None,
|
||||
) -> dict:
|
||||
"""Prior-art hint for the plugin's PreToolUse hook on Write/Edit.
|
||||
|
||||
@@ -765,7 +766,7 @@ async def build_write_path_hint(
|
||||
"""
|
||||
cfg = await get_writepath_config(user_id)
|
||||
empty = {"context": "", "note_ids": [], "sync_note_ids": [], "config": cfg,
|
||||
"stamped": [], "divergence": []}
|
||||
"stamped": [], "divergence": [], "derive": [], "derive_keys": []}
|
||||
path = (path or "").strip()
|
||||
if not cfg["enabled"] or not path:
|
||||
return empty
|
||||
@@ -935,7 +936,20 @@ async def build_write_path_hint(
|
||||
)
|
||||
except Exception:
|
||||
logger.warning("write-time divergence check failed", exc_info=True)
|
||||
if not synced and not menu and not stamped and not divergence:
|
||||
# The in-band DERIVE check (#2900): the ledger's own knowledge of the
|
||||
# names being written — a duplicate family with no canon, or a canon
|
||||
# recorded elsewhere. This is the arm the by-name local grep could not
|
||||
# be: it knows whether the other copies are canon or stray. Keyed per
|
||||
# session (`exclude_derive`) so a family is named once, not per edit.
|
||||
derive: list[dict] = []
|
||||
if stamp_shapes and project_id:
|
||||
try:
|
||||
found = await shape_ledger_svc.write_time_derive(project_id, path, stamp_shapes)
|
||||
skip = set(exclude_derive or [])
|
||||
derive = [d for d in found if d.get("key") not in skip]
|
||||
except Exception:
|
||||
logger.warning("write-time derive check failed", exc_info=True)
|
||||
if not synced and not menu and not stamped and not divergence and not derive:
|
||||
return empty
|
||||
|
||||
owners = await owner_names_for({
|
||||
@@ -1003,6 +1017,8 @@ async def build_write_path_hint(
|
||||
lines.append(_stamp_line(path, stamped))
|
||||
if divergence:
|
||||
lines.append(_divergence_line(path, divergence))
|
||||
if derive:
|
||||
lines.append(_derive_line(path, derive))
|
||||
|
||||
# Split by arm, which is the whole reason this table exists. The place arm
|
||||
# carries no score and so has no home in retrieval_logs; before #2085 a
|
||||
@@ -1027,9 +1043,40 @@ async def build_write_path_hint(
|
||||
"config": cfg,
|
||||
"stamped": stamped,
|
||||
"divergence": divergence,
|
||||
"derive": derive,
|
||||
"derive_keys": [d["key"] for d in derive],
|
||||
}
|
||||
|
||||
|
||||
def _derive_line(path: str, derive: list[dict]) -> str:
|
||||
"""The ledger's word on the names being written (#2900): a duplicate
|
||||
family to derive, or a canon to reuse — said at the write."""
|
||||
parts = []
|
||||
for d in derive:
|
||||
if d.get("canon"):
|
||||
c = d["canon"]
|
||||
parts.append(
|
||||
f"`{c['label']}` is canon — snippet #{c['snippet_id']} at `{c['path']}`; "
|
||||
"pull it and reuse, don't redefine"
|
||||
)
|
||||
continue
|
||||
f = d["family"]
|
||||
how = "identical body" if f.get("identical") else "same name defined"
|
||||
files = ", ".join(f"`{x}`" for x in f.get("files") or [])
|
||||
more = f.get("file_count", 0) - len(f.get("files") or [])
|
||||
if more > 0:
|
||||
files += f" +{more} more"
|
||||
parts.append(
|
||||
f"`{f['label']}` is a duplicate family with no canon — {how} in "
|
||||
f"{f.get('file_count', 0)} other file(s): {files}; derive it now: "
|
||||
"record the canon (create_snippet) and make the copies instances "
|
||||
"(classify_shapes) — or, if these are convention not copies, "
|
||||
"`classify_shapes(..., status=\"exempt\", reason_code=\"convention-plumbing\")` "
|
||||
"dismisses the family — rather than adding another copy"
|
||||
)
|
||||
return f"> Shape ledger at `{path}`: " + "; ".join(parts) + "."
|
||||
|
||||
|
||||
def _divergence_line(path: str, divergence: list[dict]) -> str:
|
||||
"""Button B where button A is canon — named at the write (#2793)."""
|
||||
parts = [
|
||||
|
||||
@@ -1404,6 +1404,34 @@ def proposal_summary(rows: Iterable[CodeShape], *, top: int = 8) -> dict:
|
||||
return {"proposed": proposed, "derive_groups": ranked[:top], "top_canon": top_canon}
|
||||
|
||||
|
||||
def derive_new_summary(
|
||||
rows: Iterable[CodeShape], *, since: datetime | None, top: int = 3
|
||||
) -> dict:
|
||||
"""The arrival-moment drift signal (#2899): derive-grouped rows FIRST
|
||||
SEEN after ``since`` — the previous refresh's stamp, the same one
|
||||
flag_divergence uses. "Since the last refresh, N more copies joined a
|
||||
duplicate family" is the sentence that makes the derive queue a thing
|
||||
you notice on entering, not a thing an audit finds. ``since`` None (a
|
||||
first seed) means nothing is new. Judged rows never count."""
|
||||
if since is None:
|
||||
return {"count": 0, "examples": []}
|
||||
fresh = [
|
||||
r for r in rows
|
||||
if r.proposal_basis == "derive" and r.proposal_group
|
||||
and r.status in _MECHANICAL_TODO and r.vanished_at is None
|
||||
and r.created_at is not None and r.created_at > since
|
||||
]
|
||||
fresh.sort(key=lambda r: r.created_at, reverse=True)
|
||||
return {
|
||||
"count": len(fresh),
|
||||
"examples": [
|
||||
{"label": ("." if r.kind == "css" else "") + r.symbol,
|
||||
"path": r.path, "group": r.proposal_group}
|
||||
for r in fresh[:top]
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
async def confirm_proposals(
|
||||
user_id: int,
|
||||
project_id: int,
|
||||
@@ -1566,6 +1594,75 @@ async def write_time_divergence(
|
||||
return out
|
||||
|
||||
|
||||
# How many other files a family line names before "…" — enough to go look,
|
||||
# not a wall.
|
||||
_DERIVE_FILES_SHOWN = 4
|
||||
|
||||
|
||||
async def write_time_derive(
|
||||
project_id: int, path: str, shapes: list[tuple[str, str]]
|
||||
) -> list[dict]:
|
||||
"""The in-band DERIVE check (#2900): for each (kind, name) the hook
|
||||
named at ``path``, what the ledger already knows about that name
|
||||
elsewhere in the project —
|
||||
|
||||
family the name sits in a derive-first group (identical body in N
|
||||
files, or the same name in ≥3): "this is a known duplicate
|
||||
family with no canon — derive it now, don't add a copy";
|
||||
canon a `canonical` row of that name at another path: "this is
|
||||
canon #N at <path> — reuse, don't redefine".
|
||||
|
||||
Only for shapes not yet judged at ``path`` (a judged shape is not
|
||||
re-litigated at every edit), never for the canon's own file. Returns
|
||||
[{symbol, kind, key, family?|canon?}] — `key` is the dedup token the
|
||||
hook keeps per session (the group id, or canon:<snippet_id>)."""
|
||||
wanted = {(k, _norm_symbol(n)): n for k, n in shapes if n}
|
||||
if not wanted:
|
||||
return []
|
||||
async with async_session() as session:
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(CodeShape).where(
|
||||
CodeShape.project_id == project_id,
|
||||
CodeShape.vanished_at.is_(None),
|
||||
CodeShape.symbol.in_({norm for (_k, norm) in wanted}),
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
out: list[dict] = []
|
||||
for (kind, norm), name in wanted.items():
|
||||
same = [r for r in rows if r.kind == kind and _norm_symbol(r.symbol) == norm]
|
||||
here = next((r for r in same if r.path == path), None)
|
||||
if here is not None and here.status not in _MECHANICAL_TODO:
|
||||
continue # judged here (or this IS the canon): nothing to say
|
||||
others = [r for r in same if r.path != path]
|
||||
label = ("." if kind == "css" else "") + name
|
||||
canon = next((r for r in others if r.status == "canonical" and r.snippet_id), None)
|
||||
if canon is not None:
|
||||
out.append({"symbol": name, "kind": kind, "key": f"canon:{canon.snippet_id}",
|
||||
"canon": {"snippet_id": canon.snippet_id, "path": canon.path,
|
||||
"label": label}})
|
||||
continue
|
||||
grouped = [r for r in others if r.proposal_group and r.status in _MECHANICAL_TODO]
|
||||
if here is not None and here.proposal_group:
|
||||
grouped = [r for r in grouped if r.proposal_group == here.proposal_group] or grouped
|
||||
if not grouped:
|
||||
continue
|
||||
group = grouped[0].proposal_group
|
||||
members = [r for r in grouped if r.proposal_group == group]
|
||||
files = sorted({r.path for r in members})
|
||||
out.append({
|
||||
"symbol": name, "kind": kind, "key": group,
|
||||
"family": {
|
||||
"group": group, "label": label,
|
||||
"identical": not group.startswith("name:"),
|
||||
"files": files[:_DERIVE_FILES_SHOWN], "file_count": len(files),
|
||||
"size": len(members) + (1 if here is not None else 0),
|
||||
},
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
async def flag_divergence(project_id: int, *, since: datetime | None) -> int:
|
||||
"""Flag shapes created after ``since`` (the previous refresh) that sit
|
||||
where a canon dominates and were not proposed as that canon. With no
|
||||
|
||||
@@ -0,0 +1,145 @@
|
||||
"""The PostToolUse after-write hook (#2901): code written through Bash — sed,
|
||||
heredocs, scripts — gets the same prior-art / ledger checks as a Write/Edit.
|
||||
|
||||
Runs the real shell against a temp git repo and a throwaway HTTP sink, like
|
||||
the pre-write hook's end-to-end tests. Skips where the hook's tools are
|
||||
missing; asserts on content where they are present."""
|
||||
from __future__ import annotations
|
||||
|
||||
import http.server
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import threading
|
||||
import urllib.parse
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1] / "plugin"
|
||||
HOOK = PLUGIN / "hooks" / "scribe_after_write.sh"
|
||||
|
||||
|
||||
def _env(tmp_path, url="http://127.0.0.1:9"):
|
||||
for tool in ("git", "jq", "curl", "bash"):
|
||||
if shutil.which(tool) is None:
|
||||
pytest.skip(f"hook runtime tool {tool!r} not installed")
|
||||
return {"PATH": os.environ["PATH"], "SCRIBE_URL": url, "SCRIBE_TOKEN": "t",
|
||||
"TMPDIR": str(tmp_path), "HOME": str(tmp_path),
|
||||
"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@x",
|
||||
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@x"}
|
||||
|
||||
|
||||
def _repo(tmp_path, env):
|
||||
repo = tmp_path / "repo"
|
||||
repo.mkdir()
|
||||
subprocess.run(["git", "init", "-q"], cwd=repo, check=True, env=env)
|
||||
(repo / "b.py").write_text("def one():\n return 1\n")
|
||||
(repo / "c.py").write_text("def slug(t):\n return t.lower()\n")
|
||||
subprocess.run(["git", "add", "."], cwd=repo, check=True, env=env)
|
||||
subprocess.run(["git", "commit", "-q", "-m", "base"], cwd=repo, check=True, env=env)
|
||||
return repo
|
||||
|
||||
|
||||
def _run(repo, env, session="s-after-1", tool="Bash"):
|
||||
out = subprocess.run(
|
||||
["bash", str(HOOK)],
|
||||
input=json.dumps({"session_id": session, "cwd": str(repo), "tool_name": tool,
|
||||
"tool_input": {"command": "cat > x"}, "tool_response": {}}),
|
||||
capture_output=True, text=True, env=env,
|
||||
)
|
||||
assert out.returncode == 0, out.stderr
|
||||
return out.stdout
|
||||
|
||||
|
||||
class _Sink(http.server.BaseHTTPRequestHandler):
|
||||
seen: list[dict] = []
|
||||
reply = b'{"context":"> family named","note_ids":[],"sync_note_ids":[],"derive_keys":["dup:483a"]}'
|
||||
|
||||
def do_GET(self):
|
||||
type(self).seen.append(urllib.parse.parse_qs(urllib.parse.urlparse(self.path).query))
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(type(self).reply)
|
||||
|
||||
def log_message(self, *a):
|
||||
pass
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def sink():
|
||||
_Sink.seen = []
|
||||
server = http.server.HTTPServer(("127.0.0.1", 0), _Sink)
|
||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||
try:
|
||||
yield server
|
||||
finally:
|
||||
server.shutdown()
|
||||
|
||||
|
||||
def test_after_write_names_what_bash_just_wrote_then_stays_quiet_until_the_next_change(tmp_path, sink):
|
||||
env = _env(tmp_path, url=f"http://127.0.0.1:{sink.server_port}")
|
||||
repo = _repo(tmp_path, env)
|
||||
# "A Bash call" wrote an untracked stylesheet and appended to a tracked file.
|
||||
(repo / "a.css").write_text(".log-empty {\n color: red;\n}\n")
|
||||
(repo / "b.py").write_text("def one():\n return 1\n\ndef slug(t):\n return t\n")
|
||||
out = _run(repo, env)
|
||||
by_path = {q["path"][0]: q for q in _Sink.seen}
|
||||
assert set(by_path) == {"a.css", "b.py"} # repo-relative, like the pre hook
|
||||
assert by_path["a.css"]["shapes"] == ["css:log-empty"]
|
||||
assert by_path["b.py"]["shapes"] == ["sym:slug"]
|
||||
# Added lines only for the tracked file — the existing def is not "just written".
|
||||
assert "def slug" in by_path["b.py"]["code"][0] and "def one" not in by_path["b.py"]["code"][0]
|
||||
ctx = json.loads(out)["hookSpecificOutput"]
|
||||
assert ctx["hookEventName"] == "PostToolUse"
|
||||
assert "> family named" in ctx["additionalContext"]
|
||||
# The local by-name arm rides along: `slug` already lives in c.py.
|
||||
assert "`slug` is already defined in 1 other file(s): c.py" in ctx["additionalContext"]
|
||||
# Derive keys landed on the SHARED channel the pre-write hook reads.
|
||||
state = tmp_path / "scribe-priorart" / "s-after-1.derive.ids"
|
||||
assert "dup:483a" in state.read_text().split()
|
||||
|
||||
# Nothing changed → one git status, no request, no output.
|
||||
_Sink.seen = []
|
||||
assert _run(repo, env) == ""
|
||||
assert _Sink.seen == []
|
||||
|
||||
# Another change → only that file, and the dedup channel goes back up.
|
||||
(repo / "a.css").write_text(".log-empty {\n color: red;\n}\n.other {\n margin: 0;\n}\n")
|
||||
_run(repo, env)
|
||||
assert [q["path"][0] for q in _Sink.seen] == ["a.css"]
|
||||
assert _Sink.seen[0]["exclude_derive"] == ["dup:483a"]
|
||||
assert set(_Sink.seen[0]["shapes"][0].split(",")) == {"css:log-empty", "css:other"}
|
||||
|
||||
|
||||
def test_after_write_is_silent_where_it_has_nothing_to_say(tmp_path):
|
||||
env = _env(tmp_path)
|
||||
repo = _repo(tmp_path, env)
|
||||
# Not a Bash call → nothing (hooks.json matches Bash, the script re-checks).
|
||||
(repo / "a.css").write_text(".x {\n color: red;\n}\n")
|
||||
assert _run(repo, env, tool="Write") == ""
|
||||
# Not a git repo → nothing.
|
||||
loose = tmp_path / "loose"
|
||||
loose.mkdir()
|
||||
(loose / "a.css").write_text(".x {\n color: red;\n}\n")
|
||||
assert _run(loose, env, session="s-loose") == ""
|
||||
# A change that defines nothing (prose, a call-site edit) → nothing, even
|
||||
# with the server unreachable (port 9 refuses): no definitions, no arms.
|
||||
(repo / "README.md").write_text("# notes\n")
|
||||
(repo / "b.py").write_text("def one():\n return one_more()\n")
|
||||
assert _run(repo, env, session="s-quiet") == ""
|
||||
|
||||
|
||||
def test_after_write_local_arm_and_record_nudge_work_without_a_server(tmp_path):
|
||||
"""The local by-name arm needs no instance (#2280) and the record nudge
|
||||
(#2664) fails open with it — a refused connection stands in for the
|
||||
instance."""
|
||||
env = _env(tmp_path)
|
||||
repo = _repo(tmp_path, env)
|
||||
(repo / "d.py").write_text("def slug(t):\n return t.lower()\n")
|
||||
out = _run(repo, env, session="s-local")
|
||||
ctx = json.loads(out)["hookSpecificOutput"]["additionalContext"]
|
||||
assert "`slug` is already defined in 1 other file(s): c.py" in ctx
|
||||
assert "create_snippet" in ctx
|
||||
@@ -597,6 +597,82 @@ async def test_derive_groups_land_on_rows_and_in_the_summary(seeded):
|
||||
assert {r.symbol for r in rows} == {"slug"} # 2 files < the name floor
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
async def test_write_time_derive_names_the_family_or_the_canon_for_a_name(seeded):
|
||||
"""#2900: against real rows — a name in a dup family → the family (other
|
||||
files, count); a name whose canonical row lives elsewhere → that canon;
|
||||
a judged row at the path, the canon's own file, or an unknown name →
|
||||
silence."""
|
||||
from scribe.services.shape_ledger import apply_derive_groups, write_time_derive
|
||||
|
||||
owner, pid, sid = seeded["owner"], seeded["pid"], seeded["snippet"]
|
||||
defs = _defs(
|
||||
("v/A.vue", "css", "log-empty", ".log-empty {", ".log-empty { color: red }"),
|
||||
("v/B.vue", "css", "log-empty", ".log-empty {", ".log-empty { color: red }"),
|
||||
("v/C.vue", "css", "log-empty", ".log-empty {", ".log-empty { color: red }"),
|
||||
("src/factory.py", "sym", "factory", "def factory():", "def factory():\n return 1"),
|
||||
)
|
||||
await sync_repo_shapes(pid, REPO, defs, seen_marker="m1")
|
||||
await classify_shapes(owner, pid, [
|
||||
{"path": "src/factory.py", "symbol": "factory", "status": "canonical", "snippet_id": sid},
|
||||
])
|
||||
assert await apply_derive_groups(pid) >= 3
|
||||
|
||||
# A 4th copy about to be written → the family, naming the other files.
|
||||
out = await write_time_derive(pid, "v/D.vue", [("css", "log-empty"), ("css", "unknown")])
|
||||
assert len(out) == 1 and out[0]["symbol"] == "log-empty" and out[0]["kind"] == "css"
|
||||
fam = out[0]["family"]
|
||||
assert fam["identical"] is True and fam["label"] == ".log-empty"
|
||||
assert fam["files"] == ["v/A.vue", "v/B.vue", "v/C.vue"] and fam["file_count"] == 3
|
||||
assert out[0]["key"] == fam["group"] and fam["group"].startswith("dup:")
|
||||
# Editing one existing member still names the OTHER members.
|
||||
out = await write_time_derive(pid, "v/A.vue", [("css", "log-empty")])
|
||||
assert out[0]["family"]["files"] == ["v/B.vue", "v/C.vue"] and out[0]["family"]["size"] == 3
|
||||
# The canon's name elsewhere → the canon; in the canon's own file → silence.
|
||||
out = await write_time_derive(pid, "src/other.py", [("sym", "factory")])
|
||||
assert out == [{"symbol": "factory", "kind": "sym", "key": f"canon:{sid}",
|
||||
"canon": {"snippet_id": sid, "path": "src/factory.py", "label": "factory"}}]
|
||||
assert await write_time_derive(pid, "src/factory.py", [("sym", "factory")]) == []
|
||||
# A judged row at the path is not re-litigated.
|
||||
await classify_shapes(owner, pid, [
|
||||
{"path": "v/B.vue", "symbol": "log-empty", "status": "exempt", "reason": "print sheet"},
|
||||
])
|
||||
assert await write_time_derive(pid, "v/B.vue", [("css", "log-empty")]) == []
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
async def test_derive_new_names_the_copy_that_joined_a_family_since_the_stamp(seeded):
|
||||
"""#2899: the first sync seeds one `slug`; a later sync adds an identical
|
||||
copy. Against the stamp between them, derive_new counts ONLY the
|
||||
newcomer — the drift since the last refresh, not the whole family."""
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from scribe.services.shape_ledger import (
|
||||
apply_derive_groups, derive_new_summary, live_rows,
|
||||
)
|
||||
|
||||
owner, pid = seeded["owner"], seeded["pid"]
|
||||
first = _defs(
|
||||
("a/one.py", "sym", "slug", "def slug(t):", "def slug(t):\n return t.lower()"),
|
||||
)
|
||||
await sync_repo_shapes(pid, REPO, first, seen_marker="m1")
|
||||
stamp = datetime.now(timezone.utc)
|
||||
second = _defs(
|
||||
("a/one.py", "sym", "slug", "def slug(t):", "def slug(t):\n return t.lower()"),
|
||||
("a/two.py", "sym", "slug", "def slug(t):", "def slug(t):\n return t.lower()"),
|
||||
)
|
||||
await sync_repo_shapes(pid, REPO, second, seen_marker="m2")
|
||||
assert await apply_derive_groups(pid) == 2
|
||||
|
||||
rows = await live_rows(pid)
|
||||
out = derive_new_summary(rows, since=stamp)
|
||||
assert out["count"] == 1
|
||||
assert out["examples"][0]["path"] == "a/two.py"
|
||||
assert out["examples"][0]["label"] == "slug"
|
||||
assert out["examples"][0]["group"].startswith("dup:")
|
||||
assert derive_new_summary(rows, since=None)["count"] == 0
|
||||
|
||||
|
||||
# --- #2793: the divergence readout against real rows -------------------------
|
||||
|
||||
|
||||
|
||||
@@ -276,6 +276,8 @@ async def test_coverage_measures_the_tree_exactly_and_caches(seeded):
|
||||
assert coverage["largest_gaps"] == [
|
||||
{"dir": "src", "unclassified": 2, "total": 3}
|
||||
]
|
||||
# #2899: a first computation has no previous stamp — nothing is "new".
|
||||
assert coverage["derive_new"] == {"count": 0, "examples": []}
|
||||
|
||||
# The walk fed the LEDGER (#2788): every extracted shape has a row, the
|
||||
# snippet reference locations are mechanically stamped canonical WITH
|
||||
@@ -512,6 +514,41 @@ def test_coverage_line_names_the_proposers_standing():
|
||||
assert "top canon #2844 ×78" in line and "top copy closed-msg (identical body) ×3 files" in line
|
||||
|
||||
|
||||
def test_coverage_line_shows_standing_work_even_with_nothing_unclassified():
|
||||
"""#2899: since the scoped bucket a ledger can be fully accounted and
|
||||
still carry derive groups / proposals / divergence — the line names
|
||||
them as `standing:` instead of hiding them behind the todo count, and
|
||||
names the drift since the previous refresh first-copy-first."""
|
||||
from scribe.services.coverage import coverage_line
|
||||
|
||||
base = {
|
||||
"total": 4693, "accounted": 4693, "unclassified": 0,
|
||||
"counts": {"canonical": 37, "instance": 977, "variant": 73, "exempt": 1797, "scoped": 1809},
|
||||
"computed_at": "2026-08-22T00:00:00+00:00", "largest_gaps": [],
|
||||
}
|
||||
quiet = coverage_line(base)
|
||||
assert "unclassified" not in quiet and "standing" not in quiet
|
||||
line = coverage_line({
|
||||
**base,
|
||||
"derive_groups": [{"group": "dup:abc", "label": "log-empty (identical body)", "files": 4}],
|
||||
"derive_new": {"count": 2, "examples": [
|
||||
{"label": ".error-msg", "path": "frontend/src/components/InceptionCard.vue", "group": "dup:9f0"},
|
||||
{"label": ".error-msg", "path": "frontend/src/components/Other.vue", "group": "dup:9f0"},
|
||||
]},
|
||||
"divergent": 1,
|
||||
})
|
||||
assert "; standing: 1 derive group, +2 new copies since last refresh: .error-msg in " \
|
||||
"frontend/src/components/InceptionCard.vue, 1 DIVERGENT, top copy log-empty (identical body) ×4 files" in line
|
||||
assert "unclassified" not in line
|
||||
# One copy reads singular; with a todo the block keeps its old place.
|
||||
one = coverage_line({**base, "derive_new": {"count": 1, "examples": []}})
|
||||
assert one.endswith("; standing: +1 new copy since last refresh")
|
||||
todo = coverage_line({**base, "unclassified": 3, "accounted": 4690, "proposed": 2,
|
||||
"derive_new": {"count": 1, "examples": [{"label": "x", "path": "a.py"}]},
|
||||
"largest_gaps": [{"dir": "src", "unclassified": 3, "total": 9}]})
|
||||
assert "; 3 unclassified (2 proposed, +1 new copy since last refresh: x in a.py), largest: src" in todo
|
||||
|
||||
|
||||
def test_coverage_line_names_divergence_and_recheck():
|
||||
from scribe.services.coverage import coverage_line
|
||||
|
||||
|
||||
@@ -317,6 +317,38 @@ def test_derive_groups_copy_before_name_with_floors():
|
||||
assert ("i.py", "sym", "one") not in g
|
||||
|
||||
|
||||
def test_derive_new_summary_counts_copies_first_seen_since_the_previous_refresh():
|
||||
"""#2899: the arrival-moment drift signal — derive-grouped rows created
|
||||
after the previous refresh's stamp, newest first, judged rows and a
|
||||
first seed (since=None) never count."""
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from scribe.models.code_shape import CodeShape
|
||||
from scribe.services.shape_ledger import derive_new_summary
|
||||
|
||||
t0 = datetime(2026, 8, 22, 12, 0, tzinfo=timezone.utc)
|
||||
|
||||
def row(path, symbol, at, kind="css", status="scoped", group="dup:abc", basis="derive"):
|
||||
r = CodeShape(project_id=2, repo_key="r", path=path, symbol=symbol, kind=kind,
|
||||
status=status, proposal_basis=basis, proposal_group=group)
|
||||
r.created_at = at
|
||||
return r
|
||||
rows = [
|
||||
row("v/Old.vue", "error-msg", t0 - timedelta(days=3)), # before the stamp
|
||||
row("v/InceptionCard.vue", "error-msg", t0 + timedelta(hours=1)), # new copy
|
||||
row("v/Other.vue", "error-msg", t0 + timedelta(hours=2)), # newer copy
|
||||
row("v/J.vue", "error-msg", t0 + timedelta(hours=3), status="exempt"), # judged: never
|
||||
row("s/a.py", "load", t0 + timedelta(hours=1), kind="sym", group=None, basis=None), # no family
|
||||
]
|
||||
out = derive_new_summary(rows, since=t0)
|
||||
assert out["count"] == 2
|
||||
assert [e["path"] for e in out["examples"]] == ["v/Other.vue", "v/InceptionCard.vue"]
|
||||
assert out["examples"][0] == {"label": ".error-msg", "path": "v/Other.vue", "group": "dup:abc"}
|
||||
assert derive_new_summary(rows, since=None) == {"count": 0, "examples": []}
|
||||
assert derive_new_summary(rows, since=t0, top=1)["examples"] == [
|
||||
{"label": ".error-msg", "path": "v/Other.vue", "group": "dup:abc"}]
|
||||
|
||||
|
||||
def test_proposal_summary_ranks_body_identical_groups_first_and_sees_scoped_rows():
|
||||
"""#2872: dup groups (the real copies) outrank name groups (usually
|
||||
convention), wider spread first; #2869: scoped rows are in the readout."""
|
||||
|
||||
@@ -853,14 +853,18 @@ def test_hook_exits_silently_when_unconfigured():
|
||||
|
||||
|
||||
def test_hook_skips_prose_and_data_files():
|
||||
"""No round-trip for a markdown edit — the server would return nothing anyway."""
|
||||
src = HOOK.read_text()
|
||||
skip = re.search(r"case \"\$file_path\" in\n(.*?)esac", src, re.S)
|
||||
assert skip, "expected an extension skip list"
|
||||
"""No round-trip for a markdown edit — the server would return nothing
|
||||
anyway. The list lives in the shared library (#2901) and the hook asks it."""
|
||||
lib = (PLUGIN / "hooks" / "scribe_defs.sh").read_text()
|
||||
skip = re.search(r"scribe_skip_path\(\) \{\n case \"\$1\" in\n(.*?)esac", lib, re.S)
|
||||
assert skip, "expected an extension skip list in scribe_defs.sh"
|
||||
for ext in ("*.md", "*.json", "*.lock", "*.png"):
|
||||
assert ext in skip.group(1)
|
||||
# Config formats are deliberately NOT skipped — a workflow file is reusable.
|
||||
assert "*.yml" not in skip.group(1)
|
||||
src = HOOK.read_text()
|
||||
assert 'scribe_skip_path "$file_path" && exit 0' in src
|
||||
assert '/scribe_defs.sh"' in src # sourced, not copied
|
||||
|
||||
|
||||
def test_plugin_version_bumped_with_the_hook():
|
||||
@@ -1150,7 +1154,9 @@ def test_hook_names_the_shapes_being_written():
|
||||
that changes a body, not a signature — the definition enclosing the edit,
|
||||
found by walking the target file upward from the edited lines."""
|
||||
src = HOOK.read_text()
|
||||
assert "scribe_defs()" in src # one extractor, two consumers
|
||||
lib = (PLUGIN / "hooks" / "scribe_defs.sh").read_text()
|
||||
assert "scribe_defs()" in lib # one extractor, shared (#2901)
|
||||
assert "scribe_defs()" not in src # ...not a second copy here
|
||||
assert ".tool_input.old_string" in src # the Edit's anchor
|
||||
assert "| tac | scribe_defs | head -1" in src # nearest definition above
|
||||
# The ledger feed sends NAMES, never bodies, and stays on the one GET.
|
||||
@@ -1243,6 +1249,116 @@ def test_hook_sends_the_enclosing_definition_for_a_body_edit(tmp_path):
|
||||
assert seen["shapes"] == ["sym:onTrash"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_write_time_derive_check_names_a_family_or_a_canon_in_band():
|
||||
"""#2900: the ledger's own word on the names being written — a duplicate
|
||||
family with no canon, or a canon recorded elsewhere — rendered at the
|
||||
write even when nothing else does; keyed so the session's exclude
|
||||
channel silences a family already named."""
|
||||
from scribe.services import plugin_context as pc
|
||||
found = [
|
||||
{"symbol": "log-empty", "kind": "css", "key": "dup:483a",
|
||||
"family": {"group": "dup:483a", "label": ".log-empty", "identical": True,
|
||||
"files": ["a/TaskLogSection.vue", "a/WorkspaceTaskPanel.vue"],
|
||||
"file_count": 5, "size": 6}},
|
||||
{"symbol": "btn-primary", "kind": "css", "key": "canon:2855",
|
||||
"canon": {"snippet_id": 2855, "path": "frontend/src/assets/components.css",
|
||||
"label": ".btn-primary"}},
|
||||
{"symbol": "load", "kind": "sym", "key": "name:sym:load",
|
||||
"family": {"group": "name:sym:load", "label": "load", "identical": False,
|
||||
"files": ["a/X.vue", "a/Y.vue", "a/Z.vue"], "file_count": 3, "size": 4}},
|
||||
]
|
||||
check = AsyncMock(return_value=found)
|
||||
patches = dict(
|
||||
get_writepath_config=AsyncMock(return_value=_cfg()),
|
||||
semantic_search_notes=AsyncMock(return_value=[]),
|
||||
record_retrieval=MagicMock(), owner_names_for=AsyncMock(return_value={}),
|
||||
)
|
||||
with patch.multiple(pc, **patches), \
|
||||
patch.object(pc.snippets_svc, "list_snippets", AsyncMock(return_value=([], 0))), \
|
||||
patch.object(pc.shape_ledger_svc, "recent_pulls", AsyncMock(return_value={})), \
|
||||
patch.object(pc.shape_ledger_svc, "write_time_divergence", AsyncMock(return_value=[])), \
|
||||
patch.object(pc.shape_ledger_svc, "write_time_derive", check):
|
||||
out = await pc.build_write_path_hint(
|
||||
1, "frontend/src/components/New.vue", code=REAL_CODE, project_id=24,
|
||||
stamp_shapes=[("css", "log-empty"), ("css", "btn-primary"), ("sym", "load")],
|
||||
exclude_derive=["name:sym:load"],
|
||||
)
|
||||
check.assert_awaited_once_with(24, "frontend/src/components/New.vue",
|
||||
[("css", "log-empty"), ("css", "btn-primary"), ("sym", "load")])
|
||||
# The excluded family is gone; the other two render and are keyed.
|
||||
assert [d["key"] for d in out["derive"]] == ["dup:483a", "canon:2855"]
|
||||
assert out["derive_keys"] == ["dup:483a", "canon:2855"]
|
||||
ctx = out["context"]
|
||||
assert "Shape ledger at `frontend/src/components/New.vue`" in ctx
|
||||
assert "`.log-empty` is a duplicate family with no canon — identical body in 5 other file(s): " \
|
||||
"`a/TaskLogSection.vue`, `a/WorkspaceTaskPanel.vue` +3 more; derive it now" in ctx
|
||||
assert "`.btn-primary` is canon — snippet #2855 at `frontend/src/assets/components.css`" in ctx
|
||||
assert "convention-plumbing" in ctx
|
||||
assert "`load`" not in ctx
|
||||
# No project → no check; a failing check never sinks the hint.
|
||||
check.reset_mock()
|
||||
with patch.multiple(pc, **patches), \
|
||||
patch.object(pc.snippets_svc, "list_snippets", AsyncMock(return_value=([], 0))), \
|
||||
patch.object(pc.shape_ledger_svc, "recent_pulls", AsyncMock(return_value={})), \
|
||||
patch.object(pc.shape_ledger_svc, "write_time_divergence", AsyncMock(return_value=[])), \
|
||||
patch.object(pc.shape_ledger_svc, "write_time_derive", check):
|
||||
out = await pc.build_write_path_hint(1, "x.py", code=REAL_CODE, stamp_shapes=[("sym", "f")])
|
||||
check.assert_not_awaited()
|
||||
assert out["derive"] == [] and out["derive_keys"] == []
|
||||
boom = AsyncMock(side_effect=RuntimeError("ledger down"))
|
||||
with patch.multiple(pc, **patches), \
|
||||
patch.object(pc.snippets_svc, "list_snippets", AsyncMock(return_value=([], 0))), \
|
||||
patch.object(pc.shape_ledger_svc, "recent_pulls", AsyncMock(return_value={})), \
|
||||
patch.object(pc.shape_ledger_svc, "write_time_divergence", AsyncMock(return_value=[])), \
|
||||
patch.object(pc.shape_ledger_svc, "write_time_derive", boom):
|
||||
out = await pc.build_write_path_hint(1, "x.py", code=REAL_CODE, project_id=24,
|
||||
stamp_shapes=[("sym", "f")])
|
||||
assert out["context"] == "" and out["derive"] == []
|
||||
|
||||
|
||||
def test_the_hook_keeps_a_derive_channel_and_sends_it_back(tmp_path):
|
||||
"""#2900: derive keys the server returns land in the session's own
|
||||
`.derive.ids` file and go back as `exclude_derive` on the next write —
|
||||
a family is named once per session, not at every edit."""
|
||||
import http.server
|
||||
import threading
|
||||
import urllib.parse
|
||||
|
||||
seen: list[dict] = []
|
||||
|
||||
class _Sink(http.server.BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
seen.append(urllib.parse.parse_qs(urllib.parse.urlparse(self.path).query))
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(b'{"context":"> family","note_ids":[],"sync_note_ids":[],'
|
||||
b'"derive_keys":["dup:483a","canon:2855"]}')
|
||||
|
||||
def log_message(self, *a):
|
||||
pass
|
||||
|
||||
server = http.server.HTTPServer(("127.0.0.1", 0), _Sink)
|
||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||
try:
|
||||
env = dict(_hook_runtime_env(), SCRIBE_URL=f"http://127.0.0.1:{server.server_port}",
|
||||
TMPDIR=str(tmp_path))
|
||||
payload = {"session_id": "s-derive-1", "cwd": str(tmp_path), "tool_name": "Write",
|
||||
"tool_input": {"file_path": str(tmp_path / "a.css"),
|
||||
"content": ".log-empty {\n color: red;\n}\n"}}
|
||||
for _ in range(2):
|
||||
out = subprocess.run(["bash", str(HOOK)], input=json.dumps(payload),
|
||||
capture_output=True, text=True, env=env)
|
||||
assert out.returncode == 0, out.stderr
|
||||
finally:
|
||||
server.shutdown()
|
||||
assert "exclude_derive" not in seen[0]
|
||||
assert seen[1]["exclude_derive"] == ["dup:483a,canon:2855"]
|
||||
state = tmp_path / "scribe-priorart" / "s-derive-1.derive.ids"
|
||||
assert state.read_text().split() == ["dup:483a", "canon:2855", "dup:483a", "canon:2855"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_write_time_divergence_check_is_named_in_band():
|
||||
"""#2793: the hook named a shape at a path whose directory a canon
|
||||
|
||||
Reference in New Issue
Block a user