feat(shapes): the agent judges what it wrote, at the end of the turn (milestone 439 steps 1-3)
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 18s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m38s
CI & Build / Build & push image (push) Successful in 33s

Recording used to be decided by machinery — the only "record it" prompt
fired when a same-named copy already existed (#2664), so a first instance of
a reusable piece was never asked about, and judgment arrived only through
audits. Now the question is asked where the knowledge is: the end of the
turn that wrote the code, of the agent that wrote it.

- Write hooks keep `<sid>.written.ids` (path, kind, name) for every
  definition a write names; a new file adds a `file` line for its stem — a
  candidate in any language without a framework rule (scribe_written_append).
- Stop hook scribe_shape_check.sh sends the ledger to GET
  /api/plugin/shape-check and blocks once, in the server's words, when
  anything is unjudged. Same discipline as the report check: never twice,
  never without a recorded check, another hook's loop left alone; the ledger
  is kept when the instance cannot be reached.
- shape_ledger.unjudged_shapes: no row, unclassified, scoped and hook stamps
  are unjudged; an agent/audit/import verdict is not. A snippet recorded at
  the shape answers for it until the refresh stamps it canonical.
- services/shape_check owns the reason text and records every outcome in
  app_logs (passed / blocked / judged_after_block / left_after_block).
- classify_shapes(repo=…) judges a shape the ledger has not synced yet via a
  provisional row under a bound repo; the sync confirms it, or vanishes and
  revives it with the verdict intact. An unbound repo is refused.

Plugin version minted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-01 08:39:36 -04:00
co-authored by Claude Opus 5.5
parent eb5cc6d3a7
commit f1fbdf746a
17 changed files with 854 additions and 9 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "scribe",
"description": "Scribe for Claude Code: connects the scribe MCP server, adds the hooks that deliver live project state and relevant records at the right moment, ships the shared client-neutral Scribe skills (using-scribe, writing-plans, reporting-back, systematic-debugging, verification, brainstorming, reusing-code, shape-accounting), and syncs your saved Scribe Processes as skills (/scribe:sync).",
"version": "2026.10.01.0326",
"version": "2026.10.01.1239",
"author": {
"name": "Bryan Van Deusen"
},
+1
View File
@@ -40,6 +40,7 @@ another one means adding files, not moving or rewriting any.
| `hooks/scribe_after_write.sh` | PostToolUse on shell commands: the same check for code written through the shell. |
| `hooks/scribe_tool_rules.sh` | PreToolUse on shell commands: `GET /api/plugin/tool-rules`. |
| `hooks/scribe_report_check.sh` | Stop: when the turn closed a task, checks the reply for the completion sections and reports to `GET /api/plugin/report-check`; blocks once, with the reason the server returns. |
| `hooks/scribe_shape_check.sh` | Stop: sends the definitions the turn wrote (the write hooks' `<sid>.written.ids` ledger) to `GET /api/plugin/shape-check`; blocks once, with the reason the server returns, so the agent judges what it built. |
| `hooks/scribe_sync_processes.sh` + `commands/sync.md` | `GET /api/plugin/processes` → `~/.claude/skills/scribe-proc-*` stubs; `/scribe:sync` on demand. |
| `hooks/scribe_defs.sh` | Shared shell helpers: config, dedup ledgers, outage line. |
| `hooks/scribe_static_context.md` | The adapter static text. |
+9
View File
@@ -91,6 +91,15 @@ On install you'll be asked for:
never blocks twice, and never blocks when the instance did not record the
check (unconfigured or unreachable). Outcomes land in the admin logs under
category `plugin`, action `report_check`.
- `hooks/hooks.json` → a second Stop hook (`hooks/scribe_shape_check.sh`): the
write hooks note every definition a write names (and every new file) in a
session ledger; at the end of the turn this sends them to
`GET /api/plugin/shape-check`, which answers with the ones nobody has judged.
If any are, it blocks once with the server's reason, asking the agent that
wrote them to say what each is — reusable (record a snippet), an instance or
variant of one, or a one-off — in one `classify_shapes` call. Same discipline
as the report check: never twice, never without a recorded check. Outcomes
land under category `plugin`, action `shape_check`.
- `skills/` → the universal process-skills, surfaced by description match.
- `hooks/scribe_sync_processes.sh` (a 2nd SessionStart hook) + the `/scribe:sync`
command → generate `~/.claude/skills/scribe-proc-*` stubs from your Scribe
+4
View File
@@ -98,6 +98,10 @@
{
"type": "command",
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/scribe_report_check.sh\""
},
{
"type": "command",
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/scribe_shape_check.sh\""
}
]
}
+5
View File
@@ -129,14 +129,19 @@ while IFS= read -r rel_path; do
# The code just written: the ADDED lines of the uncommitted diff for a
# tracked file (sed, not cut: this strips one marker char per line, it is
# not a payload cap), the whole file when untracked.
fresh=""
if git -C "$repo_root" ls-files --error-unmatch -- "$rel_path" >/dev/null 2>&1; then
code=$(git -C "$repo_root" diff -U0 -- "$rel_path" 2>/dev/null | grep '^+' | grep -v '^+++' | sed 's/^+//') || code=""
else
code=$(cat "$file_path" 2>/dev/null) || code=""
fresh="new"
fi
[ -n "$code" ] || continue
# `|| true` inside: an early-exiting `head` must not void its own output (#4042).
names=$(printf '%s' "$code" | scribe_defs | sort -u | head -12 || true)
# What this write defined, for the end-of-turn question (milestone 439) —
# recorded before the arms below, which skip a write that defines nothing.
printf '%s\n' "$names" | scribe_written_append "$safe_sid" "$rel_path" "$fresh"
# Nothing DEFINED in what was written (prose, data, a call-site edit) →
# nothing to say; the arms are about shapes.
[ -n "$names" ] || continue
+25
View File
@@ -1158,6 +1158,31 @@ scribe_held_query() {
# the tests read THIS string rather than a copy of it.
SCRIBE_LEDGER_DIRS="scribe-priorart scribe-autoinject"
# THE WRITTEN-SHAPES LEDGER (milestone 439). Every definition a write names,
# appended as `path<TAB>kind<TAB>name`, for the Stop hook to ask about at the
# end of the turn (scribe_shape_check.sh). Its own file — one file, one
# question (lesson #4226) — under the prior-art directory, so the session
# clear above already sweeps it. Read `kind<TAB>name` lines on stdin (the
# scribe_defs format). A NEW file adds one `file` line named by its stem: a
# component, module or package file is a candidate in its own right, in any
# language, without a framework rule saying so.
# $1 session id (already made filename-safe) $2 repo-relative path
# $3 "new" when the file did not exist before this write
scribe_written_append() {
local sid="$1" rel="$2" fresh="${3:-}" dir stem
if [ -z "$sid" ] || [ -z "$rel" ]; then cat >/dev/null; return 0; fi
dir="${TMPDIR:-/tmp}/scribe-priorart"
mkdir -p "$dir" 2>/dev/null || true
{
if [ "$fresh" = "new" ]; then
stem=${rel##*/}; stem=${stem%.*}
[ -n "$stem" ] && printf '%s\tfile\t%s\n' "$rel" "$stem"
fi
awk -F'\t' -v p="$rel" 'NF >= 2 && ($1 == "sym" || $1 == "css") && $2 != "" { printf "%s\t%s\t%s\n", p, $1, $2 }'
} >> "$dir/${sid}.written.ids" 2>/dev/null || true
return 0
}
scribe_clear_session_ledgers() {
# SPARES `*.keep.ids`, which are evidence rather than exclusions — see
# `scribe_rules_append`. Everything else still goes: the convention is
+7
View File
@@ -201,6 +201,13 @@ derive_exclude_q=""
rule_exclude_q=""
if [ -n "$session_id" ]; then
safe_sid=$(printf '%s' "$session_id" | tr -c 'A-Za-z0-9._-' '_')
# What this write defined, for the end-of-turn question (milestone 439). A
# Write onto a path that does not exist yet creates a file — a candidate in
# its own right. Written before any server call: the ask at the end of the
# turn must not depend on this write's hint having been answered.
fresh=""
[ -e "$file_path" ] || fresh="new"
printf '%s\n' "$shapes" | scribe_written_append "$safe_sid" "$rel_path" "$fresh"
idfile="$state_dir/${safe_sid}.ids"
syncfile="$state_dir/${safe_sid}.sync.ids"
derivefile="$state_dir/${safe_sid}.derive.ids"
+105
View File
@@ -0,0 +1,105 @@
#!/usr/bin/env bash
# Scribe plugin — Stop hook: the turn's new shapes are judged by the agent
# that wrote them (milestone 439).
#
# The write hooks (scribe_prior_art.sh, scribe_after_write.sh) append every
# definition a write names to a session ledger, `<sid>.written.ids`
# (`path<TAB>kind<TAB>name`, see scribe_written_append). At the end of the
# turn this hook sends that ledger to the instance, which answers with what
# nobody has judged yet. If anything is, the hook blocks ONCE, in the server's
# words: the agent that built the code is the one participant who knows what
# it is, and the end of the turn is the last moment that is still true.
#
# THE SAME DISCIPLINE AS scribe_report_check.sh, deliberately:
# - it blocks only on a `reason` the instance returned, which it returns
# only for a block it RECORDED — an unconfigured or unreachable instance
# never stops a session, and every intervention is one the numbers see;
# - never twice for one turn: the stop that follows its own block
# (`stop_hook_active` with this hook's marker present) is recorded as
# judged_after_block / left_after_block and let through;
# - another plugin's block loop is left alone (active, no marker).
#
# THE LEDGER IS CONSUMED, NOT TURN-STAMPED. Everything written since the last
# stop this hook answered is "this turn". It is emptied once the instance has
# answered — after a pass, or after the post-block stop — and KEPT when the
# instance could not be reached, so the question is asked at the next stop
# that can be answered rather than silently dropped.
#
# Config (same as the other hooks):
# CLAUDE_PLUGIN_OPTION_API_ENDPOINT base URL, no trailing slash
# CLAUDE_PLUGIN_OPTION_API_TOKEN fmcp_ API key (sensitive)
# SCRIBE_URL / SCRIBE_TOKEN override for the settings.json dogfooding path.
set -uo pipefail
command -v curl >/dev/null 2>&1 || exit 0
# shellcheck source=plugin/hooks/scribe_defs.sh
. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"
# Stop delivers { session_id, transcript_path, cwd, hook_event_name, stop_hook_active }.
event=$(cat 2>/dev/null || true)
event_flat=$(printf '%s' "$event" | scribe_json_flat)
session_id=$(scribe_json_pick "$event_flat" '.session_id')
active=$(scribe_json_pick "$event_flat" '.stop_hook_active')
event_cwd=$(scribe_json_pick "$event_flat" '.cwd')
[ -n "$session_id" ] || exit 0
safe_sid=$(printf '%s' "$session_id" | tr -c 'A-Za-z0-9._-' '_')
ledger="${TMPDIR:-/tmp}/scribe-priorart/${safe_sid}.written.ids"
state_dir="${TMPDIR:-/tmp}/scribe-shapecheck"
mkdir -p "$state_dir" 2>/dev/null || true
marker="$state_dir/${safe_sid}.blocked"
if [ "$active" = "true" ]; then
# A Stop hook already blocked this stop. Another plugin's → nothing to add.
[ -f "$marker" ] || exit 0
phase=after
else
rm -f "$marker" 2>/dev/null || true
phase=check
fi
if [ ! -s "$ledger" ]; then
rm -f "$marker" 2>/dev/null || true
exit 0
fi
scribe_config || exit 0
# Unique lines, oldest first, capped: the request stays a GET (a read-scoped
# key runs the plugin), and a turn that wrote more than this generated code.
written=$(awk '!seen[$0]++' "$ledger" 2>/dev/null | head -n 80)
written_enc=$(printf '%s' "$written" | scribe_urlenc) || exit 0
dir="${event_cwd:-${CLAUDE_PROJECT_DIR:-$PWD}}"
q="phase=${phase}&written=${written_enc}"
scope=$(scribe_scope_query "$dir")
[ -n "$scope" ] && q="${q}&${scope}"
# The repo rides along even when a marker names the project: the instance
# needs it to tell the agent which repo a just-written shape belongs to.
case "$scope" in
repo=*) ;;
*)
remote=$(git -C "$dir" remote get-url origin 2>/dev/null || true)
if [ -n "$remote" ]; then
remote_enc=$(printf '%s' "$remote" | scribe_urlenc) || remote_enc=""
[ -n "$remote_enc" ] && q="${q}&repo=${remote_enc}"
fi
;;
esac
answer=$(curl -fsS --max-time 4 \
-H "Authorization: Bearer ${token}" \
"${url%/}/api/plugin/shape-check?${q}" 2>/dev/null) || exit 0 # unreachable: keep the ledger
if [ "$phase" = "after" ]; then
rm -f "$marker" "$ledger" 2>/dev/null || true
exit 0
fi
reason=$(scribe_json_pick "$(printf '%s' "$answer" | scribe_json_flat)" '.reason')
if [ -z "$reason" ]; then
rm -f "$ledger" 2>/dev/null || true
exit 0
fi
: > "$marker" 2>/dev/null || true
printf '{"decision":"block","reason":"%s"}\n' "$(printf '%s' "$reason" | scribe_json_escape)"
exit 0
+7
View File
@@ -358,6 +358,13 @@ SMOKE_EVENTS: dict[str, str] = {
{"session_id": "smoke", "transcript_path": "/nonexistent/smoke.jsonl",
"cwd": ".", "hook_event_name": "Stop", "stop_hook_active": False}
),
# The end-of-turn shape check (milestone 439). With no ledger for the
# session there is nothing to ask about, and with no instance there is
# nobody to ask: silent, and never a block, configured or not.
"scribe_shape_check.sh": json.dumps(
{"session_id": "smoke", "transcript_path": "/nonexistent/smoke.jsonl",
"cwd": ".", "hook_event_name": "Stop", "stop_hook_active": False}
),
# The PreCompact preserver (#3680). Its whole contract is the inverse of
# every other hook's: it must exit 0 AND print, because stdout is what
# becomes the summarizer's custom instructions. A silent success here is
+15 -7
View File
@@ -4,7 +4,8 @@ The accounting model (note 2786): the snippet library records CANON (small);
the ledger accounts for EVERY extracted shape (total). These tools are how
agents move shapes out of `unclassified` — the todo state — and how they read
what still needs judgment. The ledger rows themselves are fed by the coverage
refresh; these tools only ever judge what the sync has seen.
refresh; a judgment given with `repo` may also land on a shape written this
turn that the sync has not seen yet (milestone 439).
"""
from __future__ import annotations
@@ -14,7 +15,7 @@ from scribe.services import shape_ledger as shape_ledger_svc
async def classify_shapes(
project_id: int, classifications: list[dict], via: str = "agent"
project_id: int, classifications: list[dict], via: str = "agent", repo: str = ""
) -> dict:
"""Record judgments for a project's code shapes — in batch, as rows.
@@ -52,15 +53,22 @@ async def classify_shapes(
and get_snippet's `uses` read them.
via: Who is judging — "agent" (default), "audit" (a sweep), or
"import" (carrying maps recorded elsewhere).
repo: The repo the shapes live in (its remote URL or bound key), for
judging what you wrote THIS turn — shapes the ledger has not
synced yet. With it, a shape no row matches is recorded as a
provisional row under that repo and judged; the next sync
confirms it. Must be bound to the project. Omit it to judge only
synced rows.
All-or-nothing: a structural error, a missing snippet target, or no write
access applies NOTHING. Returns {"classified": N, "unmatched": [...]} —
unmatched names shapes no live ledger row matches (the tree may have
moved since you listed; re-run the project's coverage refresh to re-sync).
All-or-nothing: a structural error, a missing snippet target, an unbound
repo, or no write access applies NOTHING. Returns {"classified": N,
"unmatched": [...], "provisional": N?} — unmatched names shapes no live
ledger row matches (the tree may have moved since you listed; pass `repo`
for a shape you just wrote, or re-run the project's coverage refresh).
"""
uid = current_user_id()
return await shape_ledger_svc.classify_shapes(
uid, project_id, classifications, via=via
uid, project_id, classifications, via=via, repo=repo
)
+66
View File
@@ -15,6 +15,7 @@ from scribe.config import Config
from scribe.services import plugin_context as plugin_ctx_svc
from scribe.services import repo_bindings as repo_bindings_svc
from scribe.services import report_check as report_check_svc
from scribe.services import shape_check as shape_check_svc
from scribe.services import task_claims as task_claims_svc
from scribe.services.settings import get_admin_setting, set_setting
@@ -359,6 +360,71 @@ async def report_check():
return jsonify(body)
@plugin_bp.get("/shape-check")
@login_required
async def shape_check():
"""The end-of-turn question (milestone 439): what did this turn write that
nobody has judged?
The client's write hooks keep a ledger of the definitions each turn wrote;
its Stop hook sends them here. The server decides which carry no judgment
— against the ledger and the project's recorded snippets — records the
outcome, and for a block returns the `reason` the agent is sent back with.
A hook blocks only on a reason it received, so only on a block that was
recorded, and never on the stop that follows one (`phase=after`).
A GET like every plugin endpoint: a read-scoped key runs the plugin, and
this records telemetry the way /report-check does. It changes no ledger
row — the agent's own classify_shapes call does that.
Query:
written (str) — newline-separated `path<TAB>kind<TAB>name` lines,
repo-relative, kind css|sym|file.
phase (str) — check | after. Anything else is a 400.
repo (opt) — working repo remote, resolved like the other arms.
project_id (opt) — explicit scope override.
"""
phase = (request.args.get("phase") or "check").strip()
if phase not in shape_check_svc.PHASES:
return jsonify({"error": f"phase must be one of {list(shape_check_svc.PHASES)}"}), 400
written = shape_check_svc.parse_written(request.args.get("written") or "")
project_id, repo, _unbound = await _project_scope()
body: dict = {"status": "ok", "unjudged": []}
if not project_id or not written:
return jsonify(body)
from scribe.services import access
from scribe.services import coverage as coverage_svc
from scribe.services import shape_ledger as shape_ledger_svc
if not await access.can_read_project(g.user.id, project_id):
return jsonify(body)
repo_key = repo_bindings_svc.normalize_repo_key(repo) if repo else ""
unjudged = await shape_ledger_svc.unjudged_shapes(project_id, written, repo_key=repo_key)
# A snippet recorded AT a shape is the strongest verdict there is — the
# next refresh stamps that row canonical. Until then the ledger cannot
# know, so the recorded locations answer for it: an agent who just
# recorded what it built is not asked again.
if unjudged:
recorded = {
(path, symbol)
for _sid, path, symbol in await coverage_svc._recorded_locations(g.user.id, project_id)
}
unjudged = [u for u in unjudged if (u["path"], u["symbol"]) not in recorded
and not (u["kind"] == "file" and (u["path"], "") in recorded)]
outcome = shape_check_svc.outcome_for(phase, unjudged)
await shape_check_svc.record_shape_check(
g.user.id, outcome, written=len(written), unjudged=len(unjudged),
project_id=project_id,
)
body["outcome"] = outcome
body["unjudged"] = unjudged
if outcome == "blocked":
body["reason"] = shape_check_svc.block_reason(
unjudged, project_id=project_id, repo=repo_key or repo,
)
return jsonify(body)
@plugin_bp.get("/claim-session")
@login_required
async def claim_session():
+18
View File
@@ -122,6 +122,24 @@ async def bindings_for_project(user_id: int, project_id: int) -> list[RepoBindin
return list(rows.scalars().all())
async def is_bound(project_id: int, raw_repo: str) -> str:
"""The normalized key when ``raw_repo`` is bound to ``project_id`` by
anyone, else "". By anyone, not by the caller: a collaborator judging a
shared project's shapes holds no binding of their own, and the question
is whether the repo belongs to the project, not who bound it."""
key = normalize_repo_key(raw_repo or "")
if not key or not project_id:
return ""
async with async_session() as session:
found = await session.execute(
select(RepoBinding.id).where(
RepoBinding.project_id == project_id,
RepoBinding.repo_key == key,
).limit(1)
)
return key if found.first() is not None else ""
async def keys_for_project(user_id: int, project_id: int) -> list[str]:
"""Every repo key bound to a project — the snippet→forge join (#2691).
+147
View File
@@ -0,0 +1,147 @@
"""The end-of-turn shape check: what a turn wrote that nobody has judged, and what the agent is asked.
WHY THIS EXISTS (milestone 439)
The shape ledger used to be filled by machinery: extraction found the
definitions, framework rules decided which were one-offs, and the write-path
hook stamped "instance of #N" with nobody reading. The agent that WROTE the
code — the one participant who knows what it is — was asked only in an audit,
weeks later, if anyone ran one. A first instance of a reusable piece was never
asked about at all, so it was never recorded, and the next session rebuilt it.
So the question moves to the moment the knowledge exists. The client's write
hooks keep a ledger of the definitions each turn wrote; its Stop hook sends
them here at the end of the turn. Two jobs live on this side, as with the
report check (services/report_check.py):
- DECIDING what is unjudged, against the ledger and the project's recorded
snippets — something only the server can see.
- OWNING THE WORDS. A hook carries timing and transport (plugin/PACKAGING.md);
the instruction comes from here, so every client asks the same question
and the wording changes in one place.
And it RECORDS every outcome, so "how much did agents leave unjudged after
being asked" is a number rather than an impression. app_logs, for the reason
report_check gives.
"""
from __future__ import annotations
import json
from scribe.models import async_session
from scribe.models.app_log import AppLog
# check — the turn's first stop: block if anything is unjudged.
# after — the stop that follows a block: never block again, only record
# whether the agent answered.
PHASES = ("check", "after")
OUTCOMES = ("passed", "blocked", "judged_after_block", "left_after_block")
# How many shapes the reason names before "+N more". Enough to judge a normal
# turn in one call; a turn that wrote more than this generated something.
_LISTED = 25
def _evidence_note(evidence: dict) -> str:
"""One parenthesis of what the machinery holds — offered, never asserted."""
parts = []
looks = evidence.get("looks_like") or {}
if looks.get("snippet_id"):
score = looks.get("score")
at = f" {score:.2f}" if isinstance(score, (int, float)) else ""
parts.append(f"looks like #{looks['snippet_id']}{at}")
stamped = evidence.get("stamped") or {}
if stamped.get("snippet_id"):
parts.append(f"the hook guessed #{stamped['snippet_id']}")
if evidence.get("copies"):
parts.append("other copies of it exist")
if evidence.get("diverges_from"):
parts.append(f"#{evidence['diverges_from']} is canon in this directory")
return f" ({'; '.join(parts)})" if parts else ""
def block_reason(unjudged: list[dict], *, project_id: int, repo: str) -> str:
"""What the agent is told at the end of a turn that wrote unjudged shapes.
Phrased as the practice wanted (note #3565): what to decide and the one
call that records it, not a prohibition. The reusing-code skill owns the
longer version of what each verdict means.
"""
lines = []
for item in unjudged[:_LISTED]:
new = " — new" if item.get("status") == "new" else ""
lines.append(
f"- {item['path']} · {item['symbol']} ({item['kind']}){new}"
f"{_evidence_note(item.get('evidence') or {})}"
)
more = len(unjudged) - _LISTED
if more > 0:
lines.append(f"- … and {more} more (list_shapes(project_id={project_id}, path=…) shows them)")
repo_arg = f', repo="{repo}"' if repo else ""
return (
f"This turn wrote {len(unjudged)} definition{'s' if len(unjudged) != 1 else ''} "
"that nobody has judged yet. You built them, so you know what each one is — "
"record it now, while that is still true, so the next session starts from your "
"answer instead of rebuilding it:\n"
+ "\n".join(lines)
+ "\n\nFor each: something another part of the code should reuse → record it "
"with create_snippet (name, code, when to reach for it, location); built from a "
"recorded snippet → `instance` of it; a deliberate departure from one → "
"`variant` with the why; a genuine one-off → `exempt` with a reason "
"(reason_code one-off-handler, test-helper, scoped-css, pure-helper … indexes it). "
"One call records them all: "
f"classify_shapes(project_id={project_id}{repo_arg}, classifications=[{{path, "
"symbol, kind, status, snippet_id?, reason?, reason_code?}}, …]). "
"`repo` lets a verdict land on a shape the ledger has not synced yet."
)
async def record_shape_check(
user_id: int | None,
outcome: str,
*,
written: int,
unjudged: int,
project_id: int | None = None,
) -> None:
if outcome not in OUTCOMES:
raise ValueError(f"unknown shape-check outcome {outcome!r}")
details: dict = {"outcome": outcome, "written": written, "unjudged": unjudged}
if project_id:
details["project_id"] = project_id
async with async_session() as session:
session.add(AppLog(
category="plugin",
user_id=user_id,
action="shape_check",
details=json.dumps(details),
))
await session.commit()
def outcome_for(phase: str, unjudged: list[dict]) -> str:
"""The recorded outcome: a first stop blocks or passes; the stop after a
block records whether the agent answered, and never blocks."""
if phase == "after":
return "left_after_block" if unjudged else "judged_after_block"
return "blocked" if unjudged else "passed"
def parse_written(raw: str, *, cap: int = 200) -> list[tuple[str, str, str]]:
"""The hook's ledger lines, `path<TAB>kind<TAB>name`, as triples.
Kinds outside the ledger's vocabulary and malformed lines are dropped
rather than echoed into an instruction; duplicates collapse; capped."""
out: list[tuple[str, str, str]] = []
for line in (raw or "").splitlines():
parts = line.split("\t")
if len(parts) != 3:
continue
path, kind, name = (p.strip() for p in parts)
if kind not in ("css", "sym", "file") or not path or not name:
continue
if (path, kind, name) not in out:
out.append((path, kind, name))
if len(out) >= cap:
break
return out
+114 -1
View File
@@ -535,6 +535,7 @@ async def classify_shapes(
classifications: list[dict],
*,
via: str = "agent",
repo: str = "",
) -> dict:
"""Apply a batch of judgments to a project's live ledger rows.
@@ -544,7 +545,19 @@ async def classify_shapes(
item carries one — and a target no live row matches is reported in
``unmatched``, not an error: the tree may simply have moved since the
caller listed. Idempotent by construction.
``repo`` — a repo bound to the project — lets a judgment land on a shape
the ledger has not synced yet (milestone 439): the shape the agent wrote
this turn. Such an item becomes a PROVISIONAL row under that repo, the
same row the write-path hook has always created, with no seen commit;
the next sync confirms it, or marks it vanished until the code reaches
the bound ref, and either way the judgment rides along (a vanished row
that reappears keeps its status). Without ``repo`` an unknown shape is
`unmatched`, exactly as before. An unbound ``repo`` is refused rather
than ignored: a verdict silently dropped reads as one recorded.
"""
from scribe.services import repo_bindings as repo_bindings_svc
from scribe.services import access
from scribe.services import snippets as snippets_svc
@@ -569,9 +582,18 @@ async def classify_shapes(
for sid in sorted(target_ids):
if await snippets_svc.get_snippet(user_id, sid) is None:
raise ValueError(f"snippet {sid} not found (or not readable)")
repo_key = ""
if (repo or "").strip():
repo_key = await repo_bindings_svc.is_bound(project_id, repo)
if not repo_key:
raise ValueError(
f"repo {repo!r} is not bound to project {project_id} — "
"bind_repo it, or omit repo to judge synced shapes only"
)
now = datetime.now(timezone.utc)
classified = 0
provisional = 0
unmatched: list[dict] = []
async with async_session() as session:
rows = (
@@ -592,6 +614,17 @@ async def classify_shapes(
kind = (item.get("kind") or "").strip()
if kind:
matches = [r for r in matches if r.kind == kind]
if not matches and repo_key and item.get("status") != "unclassified":
path = (item.get("path") or "").strip()
symbol = (item.get("symbol") or "").strip()
row = CodeShape(
project_id=project_id, repo_key=repo_key, path=path,
symbol=symbol, kind=kind or snippet_kind(symbol, ""),
)
session.add(row)
by_key.setdefault((path, symbol), []).append(row)
matches = [row]
provisional += 1
if not matches:
unmatched.append({
"path": item.get("path"), "symbol": item.get("symbol"),
@@ -610,7 +643,10 @@ async def classify_shapes(
evidence=item.get("reason"))
classified += 1
await session.commit()
return {"classified": classified, "unmatched": unmatched}
out = {"classified": classified, "unmatched": unmatched}
if provisional:
out["provisional"] = provisional
return out
def rule_matches(row: CodeShape, *, path: str, pattern: str, kind: str) -> bool:
@@ -2550,6 +2586,83 @@ async def flag_divergence(project_id: int, *, since: datetime | None) -> int:
return flagged
# --- the end-of-turn question (milestone 439): what did this turn write that
# nobody has judged? ---------------------------------------------------------
#
# The write hooks keep a session ledger of the definitions each turn wrote; at
# the end of the turn the client asks this, and the agent that wrote them
# answers. "Judged" means a verdict SOMEONE READ: an agent's, an audit's, an
# import's. The sync's `scoped` stamp, `unclassified`, a hook stamp and no row
# at all are all unjudged — each is a machine's guess or nothing, and the
# point of the question is that the writer, who knows, says.
def _is_judged(row) -> bool:
return (
row.status not in _MECHANICAL_TODO
and row.classified_by not in _UNATTENDED_BY
)
async def unjudged_shapes(
project_id: int, written: list[tuple[str, str, str]], *, repo_key: str = "",
) -> list[dict]:
"""The (path, kind, name) shapes in ``written`` that carry no judgment.
Each comes back as {path, kind, symbol, status, evidence} — status is the
row's, or "new" when the ledger has none yet; evidence is what the
machinery holds that the writer may want to weigh: the proposer's
candidate canon, a hook's stamp, a divergence flag. Evidence, never a
verdict — the writer is asked because the machine's guesses were the
thing that poisoned two ledgers. Order follows ``written``; duplicates
collapse. The caller has already resolved the project and checked access.
"""
wanted: list[tuple[str, str, str]] = []
for path, kind, name in written:
key = ((path or "").strip(), (kind or "").strip(), (name or "").strip())
if all(key) and key not in wanted:
wanted.append(key)
if not project_id or not wanted:
return []
paths = sorted({p for p, _k, _n in wanted})
query = select(CodeShape).where(
CodeShape.project_id == project_id,
CodeShape.vanished_at.is_(None),
CodeShape.path.in_(paths),
)
if repo_key:
query = query.where(CodeShape.repo_key == repo_key)
async with async_session() as session:
rows = (await session.execute(query)).scalars().all()
by_key = {(r.path, r.kind, r.symbol): r for r in rows}
out: list[dict] = []
for path, kind, name in wanted:
row = by_key.get((path, kind, name))
if row is not None and _is_judged(row):
continue
evidence: dict = {}
if row is not None:
if row.proposed_snippet_id:
evidence["looks_like"] = {
"snippet_id": row.proposed_snippet_id,
"basis": row.proposal_basis,
"score": row.proposal_score,
}
if row.classified_by in _UNATTENDED_BY and row.snippet_id:
evidence["stamped"] = {
"snippet_id": row.snippet_id, "score": stamp_score(row.reason),
}
if row.proposal_group:
evidence["copies"] = row.proposal_group
if row.diverges_from:
evidence["diverges_from"] = row.diverges_from
out.append({
"path": path, "kind": kind, "symbol": name,
"status": row.status if row is not None else "new",
"evidence": evidence,
})
return out
# How many of a canon's rows a review listing shows before "…" — enough to
# judge from, not the whole table.
_REVIEW_ROWS_SHOWN = 12
+78
View File
@@ -1138,3 +1138,81 @@ async def test_history_records_what_was_used_when_and_drift_asks_for_a_recheck(s
# reads nothing.
assert len((await shape_history(owner, pid, "src"))["events"]) == 6
assert await shape_history(other, pid, "src/app.py") == {}
# --- milestone 439: the writer judges what it wrote, before the sync has seen it
@pytest.mark.integration
async def test_a_shape_written_this_turn_is_judged_before_the_sync_sees_it(seeded):
"""The end-of-turn verdict lands on a shape no sync has read yet: with a
bound `repo` it becomes a provisional row and is judged, the sync that
then reads the shape keeps the judgment, and a sync that does NOT have it
yet (the code has not reached the bound ref) keeps it through the vanish
and the reappearance. Without `repo` the old contract holds."""
from scribe.services import repo_bindings as repo_bindings_svc
owner, pid = seeded["owner"], seeded["pid"]
await repo_bindings_svc.set_binding(owner, REPO, pid)
verdict = [{"path": "web/src/lib/StatusChip.svelte", "symbol": "tone",
"kind": "sym", "status": "exempt",
"reason": "the chip's own colour switch", "reason_code": "pure-helper"}]
out = await classify_shapes(owner, pid, verdict)
assert out["unmatched"] == [{"path": "web/src/lib/StatusChip.svelte", "symbol": "tone"}]
with pytest.raises(ValueError, match="not bound"):
await classify_shapes(owner, pid, verdict, repo="git.example.com/alice/elsewhere")
out = await classify_shapes(owner, pid, verdict, repo=REPO)
assert out == {"classified": 1, "unmatched": [], "provisional": 1}
# A sync that has not got the code yet: the row vanishes, judgment intact…
await sync_repo_shapes(pid, REPO, SHAPES, seen_marker="c1")
# …and the sync that has it brings the row back with the verdict.
await sync_repo_shapes(
pid, REPO, SHAPES + [("web/src/lib/StatusChip.svelte", "sym", "tone")], seen_marker="c2",
)
rows, _ = await list_project_shapes(owner, pid, path="web/src/lib/StatusChip.svelte")
assert len(rows) == 1
row = rows[0]
assert (row.status, row.classified_by, row.reason_code) == ("exempt", "agent", "pure-helper")
assert row.vanished_at is None and row.last_seen_commit == "c2"
@pytest.mark.integration
async def test_unjudged_is_what_nobody_read(seeded):
"""The end-of-turn question: a shape with no row, an unclassified one, a
sync-scoped one and a hook stamp are all asked about; an agent's or an
audit's verdict is not. Evidence travels; a verdict is never inferred."""
from sqlalchemy import update
from scribe.services.shape_ledger import _RESEMBLE_REASON, unjudged_shapes
owner, pid, sid = seeded["owner"], seeded["pid"], seeded["snippet"]
await classify_shapes(owner, pid, [
{"path": "src/app.py", "symbol": "make_app", "status": "instance", "snippet_id": sid},
{"path": "src/util.py", "symbol": "helper", "status": "exempt", "reason": "x"},
])
async with async_session() as session:
await session.execute(
update(CodeShape)
.where(CodeShape.project_id == pid, CodeShape.symbol == "helper")
.values(classified_by="hook", status="instance", snippet_id=sid,
reason=_RESEMBLE_REASON.format(sid=sid, score=0.7))
)
await session.commit()
written = [
("src/app.py", "sym", "make_app"), # judged by an agent → not asked
("src/app.py", "sym", "Config"), # unclassified → asked
("src/util.py", "sym", "helper"), # a hook stamp → asked, with its evidence
("src/new.py", "sym", "fresh"), # no row → asked as new
("src/new.py", "sym", "fresh"), # duplicates collapse
]
got = await unjudged_shapes(pid, written, repo_key=REPO)
assert [(g["symbol"], g["status"]) for g in got] == [
("Config", "unclassified"), ("helper", "instance"), ("fresh", "new"),
]
assert got[1]["evidence"]["stamped"] == {"snippet_id": sid, "score": 0.7}
assert got[2]["evidence"] == {}
assert await unjudged_shapes(pid, []) == []
+81
View File
@@ -0,0 +1,81 @@
"""The server half of the end-of-turn shape check (milestone 439): what the
agent is asked, what the hook's ledger may say, and the outcome record."""
import json
from unittest.mock import patch
import pytest
from tests.helpers import make_mock_session
def test_the_ledger_lines_parse_and_garbage_is_dropped():
from scribe.services.shape_check import parse_written
raw = ("a.py\tsym\thelper\n"
"a.py\tsym\thelper\n" # duplicate
"web/A.svelte\tfile\tA\n"
"x.py\tbogus\tthing\n" # unknown kind
"only-two\tsym\n" # malformed
"\tsym\tnameless-path\n")
assert parse_written(raw) == [("a.py", "sym", "helper"), ("web/A.svelte", "file", "A")]
assert len(parse_written("".join(f"p\tsym\tn{i}\n" for i in range(500)), cap=7)) == 7
def test_a_first_stop_blocks_or_passes_and_the_next_never_blocks():
from scribe.services.shape_check import outcome_for
one = [{"path": "a", "kind": "sym", "symbol": "b"}]
assert outcome_for("check", one) == "blocked"
assert outcome_for("check", []) == "passed"
assert outcome_for("after", one) == "left_after_block"
assert outcome_for("after", []) == "judged_after_block"
def test_the_reason_names_each_shape_its_evidence_and_the_one_call():
from scribe.services.shape_check import block_reason
unjudged = [
{"path": "web/A.svelte", "kind": "file", "symbol": "A", "status": "new", "evidence": {}},
{"path": "src/x.py", "kind": "sym", "symbol": "load", "status": "unclassified",
"evidence": {"looks_like": {"snippet_id": 12, "basis": "semantic", "score": 0.84},
"diverges_from": 9}},
]
reason = block_reason(unjudged, project_id=3, repo="git.example.com/a/w")
assert "2 definitions" in reason
assert "web/A.svelte · A (file) — new" in reason
assert "looks like #12 0.84" in reason and "#9 is canon in this directory" in reason
assert 'classify_shapes(project_id=3, repo="git.example.com/a/w"' in reason
for verdict in ("create_snippet", "`instance`", "`variant`", "`exempt`"):
assert verdict in reason, verdict
def test_a_long_turn_is_listed_with_a_tail_not_dropped():
from scribe.services.shape_check import _LISTED, block_reason
many = [{"path": "p.py", "kind": "sym", "symbol": f"f{i}", "status": "new", "evidence": {}}
for i in range(_LISTED + 3)]
reason = block_reason(many, project_id=1, repo="")
assert "… and 3 more" in reason
assert "repo=" not in reason
async def test_the_outcome_is_recorded_as_a_plugin_event():
from scribe.services.shape_check import record_shape_check
session = make_mock_session()
with patch("scribe.services.shape_check.async_session", return_value=session):
await record_shape_check(7, "blocked", written=5, unjudged=2, project_id=2)
row = session.add.call_args.args[0]
assert (row.category, row.action, row.user_id) == ("plugin", "shape_check", 7)
assert json.loads(row.details) == {"outcome": "blocked", "written": 5,
"unjudged": 2, "project_id": 2}
async def test_an_unknown_outcome_is_refused_before_anything_is_written():
from scribe.services.shape_check import record_shape_check
session = make_mock_session()
with patch("scribe.services.shape_check.async_session", return_value=session), \
pytest.raises(ValueError):
await record_shape_check(7, "skipped", written=1, unjudged=1)
session.add.assert_not_called()
+171
View File
@@ -0,0 +1,171 @@
"""The end-of-turn shape check (milestone 439): the write hooks note what each
write defined, and the Stop hook asks the instance which of those nobody has
judged, blocking once in the server's words.
Runs the real shell against the shared HTTP sink. What it pins: the write
hooks' ledger (a Write of a new file adds a `file` line; a skipped path adds
nothing); silence with nothing written; a block only on a reason the instance
returned; the post-block stop recorded and let through; another hook's block
loop left alone; the ledger kept when the instance could not be reached.
"""
from __future__ import annotations
import json
import os
import shutil
import subprocess
from pathlib import Path
import pytest
from tests.helpers import http_sink
HOOKS = Path(__file__).resolve().parents[1] / "plugin" / "hooks"
CHECK = HOOKS / "scribe_shape_check.sh"
PRIOR_ART = HOOKS / "scribe_prior_art.sh"
REASON = "SERVER REASON: judge what you wrote"
def _env(tmp_path, url="http://127.0.0.1:9"):
for tool in ("curl", "bash", "git"):
if shutil.which(tool) is None:
pytest.skip(f"hook runtime tool {tool!r} not installed")
return {"PATH": os.environ["PATH"], "SCRIBE_URL": url, "SCRIBE_TOKEN": "t",
"TMPDIR": str(tmp_path), "HOME": str(tmp_path)}
def _ledger(tmp_path, session="s1"):
return tmp_path / "scribe-priorart" / f"{session}.written.ids"
def _write_ledger(tmp_path, lines, session="s1"):
path = _ledger(tmp_path, session)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("".join(f"{line}\n" for line in lines))
return path
def _stop(env, cwd, active=False, session="s1"):
out = subprocess.run(
["bash", str(CHECK)],
input=json.dumps({"session_id": session, "transcript_path": "/nonexistent",
"cwd": str(cwd), "hook_event_name": "Stop",
"stop_hook_active": active}),
capture_output=True, text=True, env=env, timeout=30,
)
assert out.returncode == 0, out.stderr
return out.stdout.strip()
def _repo(tmp_path):
repo = tmp_path / "repo"
repo.mkdir()
subprocess.run(["git", "init", "-q", str(repo)], check=True)
subprocess.run(["git", "-C", str(repo), "remote", "add", "origin",
"https://git.example.com/alice/widget.git"], check=True)
return repo
# ── the write hooks keep the ledger ───────────────────────────────────────
def _pre_write(env, repo, rel, content, tool="Write"):
field = "content" if tool == "Write" else "new_string"
out = subprocess.run(
["bash", str(PRIOR_ART)],
input=json.dumps({"session_id": "s1", "cwd": str(repo), "tool_name": tool,
"tool_input": {"file_path": str(repo / rel), field: content}}),
capture_output=True, text=True, env=env, timeout=30,
)
assert out.returncode == 0, out.stderr
def test_a_write_notes_what_it_defined_and_a_new_file_is_a_candidate(tmp_path):
with http_sink(b'{"note_ids":[]}') as (port, _seen):
env = _env(tmp_path, f"http://127.0.0.1:{port}")
repo = _repo(tmp_path)
(repo / "web").mkdir()
_pre_write(env, repo, "web/StatusChip.svelte",
"<script>\nfunction tone(s) {\n return s;\n}\n</script>\n"
"<style>\n.chip {\n color: red;\n}\n</style>\n")
lines = _ledger(tmp_path).read_text().splitlines()
assert "web/StatusChip.svelte\tfile\tStatusChip" in lines
assert "web/StatusChip.svelte\tsym\ttone" in lines
assert "web/StatusChip.svelte\tcss\tchip" in lines
def test_an_edit_to_an_existing_file_adds_no_file_line(tmp_path):
with http_sink(b'{"note_ids":[]}') as (port, _seen):
env = _env(tmp_path, f"http://127.0.0.1:{port}")
repo = _repo(tmp_path)
(repo / "lib.py").write_text("x = 1\n")
_pre_write(env, repo, "lib.py", "def helper():\n return 1\n", tool="Edit")
lines = _ledger(tmp_path).read_text().splitlines()
assert lines == ["lib.py\tsym\thelper"]
def test_a_skipped_path_notes_nothing(tmp_path):
with http_sink(b'{"note_ids":[]}') as (port, _seen):
env = _env(tmp_path, f"http://127.0.0.1:{port}")
repo = _repo(tmp_path)
_pre_write(env, repo, "README.md", "# def helper():\n")
assert not _ledger(tmp_path).exists()
# ── the Stop hook asks ────────────────────────────────────────────────────
def test_nothing_written_is_silent_and_asks_nothing(tmp_path):
with http_sink(b'{"status":"ok","reason":"x"}') as (port, seen):
env = _env(tmp_path, f"http://127.0.0.1:{port}")
assert _stop(env, _repo(tmp_path)) == ""
assert seen == []
def test_all_judged_passes_silently_and_empties_the_ledger(tmp_path):
with http_sink(b'{"status":"ok","outcome":"passed","unjudged":[]}') as (port, seen):
env = _env(tmp_path, f"http://127.0.0.1:{port}")
ledger = _write_ledger(tmp_path, ["a.py\tsym\thelper", "a.py\tsym\thelper"])
assert _stop(env, _repo(tmp_path)) == ""
assert seen[0]["phase"] == ["check"]
assert seen[0]["written"] == ["a.py\tsym\thelper"], "duplicates collapse"
assert seen[0]["repo"] == ["https://git.example.com/alice/widget.git"]
assert not ledger.exists()
def test_unjudged_blocks_once_in_the_servers_words_then_records_the_answer(tmp_path):
reply = json.dumps({"status": "ok", "outcome": "blocked", "reason": REASON}).encode()
with http_sink(reply) as (port, seen):
env = _env(tmp_path, f"http://127.0.0.1:{port}")
repo = _repo(tmp_path)
ledger = _write_ledger(tmp_path, ["web/A.svelte\tfile\tA"])
assert json.loads(_stop(env, repo)) == {"decision": "block", "reason": REASON}
assert ledger.exists(), "kept, so the stop after the block asks about the same set"
# The stop after the block: recorded, never blocked, ledger consumed.
assert _stop(env, repo, active=True) == ""
assert [q["phase"] for q in seen] == [["check"], ["after"]]
assert not ledger.exists()
assert _stop(env, repo, active=True) == ""
assert len(seen) == 2
def test_no_block_without_a_reason_from_the_instance(tmp_path):
with http_sink(b'{"status":"ok"}') as (port, _seen):
env = _env(tmp_path, f"http://127.0.0.1:{port}")
_write_ledger(tmp_path, ["a.py\tsym\thelper"])
assert _stop(env, _repo(tmp_path)) == ""
def test_another_hooks_block_loop_is_left_alone(tmp_path):
with http_sink(b'{"status":"ok","reason":"x"}') as (port, seen):
env = _env(tmp_path, f"http://127.0.0.1:{port}")
ledger = _write_ledger(tmp_path, ["a.py\tsym\thelper"])
assert _stop(env, _repo(tmp_path), active=True) == ""
assert seen == []
assert ledger.exists(), "this hook's own question is still owed"
def test_an_unreachable_instance_keeps_the_question_for_later(tmp_path):
env = _env(tmp_path) # port 9: nothing listens
ledger = _write_ledger(tmp_path, ["a.py\tsym\thelper"])
assert _stop(env, _repo(tmp_path)) == ""
assert ledger.exists()