feat(shapes): the agent judges what it wrote, at the end of the turn (milestone 439 steps 1-3)
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 18s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m38s
CI & Build / Build & push image (push) Successful in 33s

Recording used to be decided by machinery — the only "record it" prompt
fired when a same-named copy already existed (#2664), so a first instance of
a reusable piece was never asked about, and judgment arrived only through
audits. Now the question is asked where the knowledge is: the end of the
turn that wrote the code, of the agent that wrote it.

- Write hooks keep `<sid>.written.ids` (path, kind, name) for every
  definition a write names; a new file adds a `file` line for its stem — a
  candidate in any language without a framework rule (scribe_written_append).
- Stop hook scribe_shape_check.sh sends the ledger to GET
  /api/plugin/shape-check and blocks once, in the server's words, when
  anything is unjudged. Same discipline as the report check: never twice,
  never without a recorded check, another hook's loop left alone; the ledger
  is kept when the instance cannot be reached.
- shape_ledger.unjudged_shapes: no row, unclassified, scoped and hook stamps
  are unjudged; an agent/audit/import verdict is not. A snippet recorded at
  the shape answers for it until the refresh stamps it canonical.
- services/shape_check owns the reason text and records every outcome in
  app_logs (passed / blocked / judged_after_block / left_after_block).
- classify_shapes(repo=…) judges a shape the ledger has not synced yet via a
  provisional row under a bound repo; the sync confirms it, or vanishes and
  revives it with the verdict intact. An unbound repo is refused.

Plugin version minted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-01 08:39:36 -04:00
co-authored by Claude Opus 5.5
parent eb5cc6d3a7
commit f1fbdf746a
17 changed files with 854 additions and 9 deletions
+147
View File
@@ -0,0 +1,147 @@
"""The end-of-turn shape check: what a turn wrote that nobody has judged, and what the agent is asked.
WHY THIS EXISTS (milestone 439)
The shape ledger used to be filled by machinery: extraction found the
definitions, framework rules decided which were one-offs, and the write-path
hook stamped "instance of #N" with nobody reading. The agent that WROTE the
code — the one participant who knows what it is — was asked only in an audit,
weeks later, if anyone ran one. A first instance of a reusable piece was never
asked about at all, so it was never recorded, and the next session rebuilt it.
So the question moves to the moment the knowledge exists. The client's write
hooks keep a ledger of the definitions each turn wrote; its Stop hook sends
them here at the end of the turn. Two jobs live on this side, as with the
report check (services/report_check.py):
- DECIDING what is unjudged, against the ledger and the project's recorded
snippets — something only the server can see.
- OWNING THE WORDS. A hook carries timing and transport (plugin/PACKAGING.md);
the instruction comes from here, so every client asks the same question
and the wording changes in one place.
And it RECORDS every outcome, so "how much did agents leave unjudged after
being asked" is a number rather than an impression. app_logs, for the reason
report_check gives.
"""
from __future__ import annotations
import json
from scribe.models import async_session
from scribe.models.app_log import AppLog
# check — the turn's first stop: block if anything is unjudged.
# after — the stop that follows a block: never block again, only record
# whether the agent answered.
PHASES = ("check", "after")
OUTCOMES = ("passed", "blocked", "judged_after_block", "left_after_block")
# How many shapes the reason names before "+N more". Enough to judge a normal
# turn in one call; a turn that wrote more than this generated something.
_LISTED = 25
def _evidence_note(evidence: dict) -> str:
"""One parenthesis of what the machinery holds — offered, never asserted."""
parts = []
looks = evidence.get("looks_like") or {}
if looks.get("snippet_id"):
score = looks.get("score")
at = f" {score:.2f}" if isinstance(score, (int, float)) else ""
parts.append(f"looks like #{looks['snippet_id']}{at}")
stamped = evidence.get("stamped") or {}
if stamped.get("snippet_id"):
parts.append(f"the hook guessed #{stamped['snippet_id']}")
if evidence.get("copies"):
parts.append("other copies of it exist")
if evidence.get("diverges_from"):
parts.append(f"#{evidence['diverges_from']} is canon in this directory")
return f" ({'; '.join(parts)})" if parts else ""
def block_reason(unjudged: list[dict], *, project_id: int, repo: str) -> str:
"""What the agent is told at the end of a turn that wrote unjudged shapes.
Phrased as the practice wanted (note #3565): what to decide and the one
call that records it, not a prohibition. The reusing-code skill owns the
longer version of what each verdict means.
"""
lines = []
for item in unjudged[:_LISTED]:
new = " — new" if item.get("status") == "new" else ""
lines.append(
f"- {item['path']} · {item['symbol']} ({item['kind']}){new}"
f"{_evidence_note(item.get('evidence') or {})}"
)
more = len(unjudged) - _LISTED
if more > 0:
lines.append(f"- … and {more} more (list_shapes(project_id={project_id}, path=…) shows them)")
repo_arg = f', repo="{repo}"' if repo else ""
return (
f"This turn wrote {len(unjudged)} definition{'s' if len(unjudged) != 1 else ''} "
"that nobody has judged yet. You built them, so you know what each one is — "
"record it now, while that is still true, so the next session starts from your "
"answer instead of rebuilding it:\n"
+ "\n".join(lines)
+ "\n\nFor each: something another part of the code should reuse → record it "
"with create_snippet (name, code, when to reach for it, location); built from a "
"recorded snippet → `instance` of it; a deliberate departure from one → "
"`variant` with the why; a genuine one-off → `exempt` with a reason "
"(reason_code one-off-handler, test-helper, scoped-css, pure-helper … indexes it). "
"One call records them all: "
f"classify_shapes(project_id={project_id}{repo_arg}, classifications=[{{path, "
"symbol, kind, status, snippet_id?, reason?, reason_code?}}, …]). "
"`repo` lets a verdict land on a shape the ledger has not synced yet."
)
async def record_shape_check(
user_id: int | None,
outcome: str,
*,
written: int,
unjudged: int,
project_id: int | None = None,
) -> None:
if outcome not in OUTCOMES:
raise ValueError(f"unknown shape-check outcome {outcome!r}")
details: dict = {"outcome": outcome, "written": written, "unjudged": unjudged}
if project_id:
details["project_id"] = project_id
async with async_session() as session:
session.add(AppLog(
category="plugin",
user_id=user_id,
action="shape_check",
details=json.dumps(details),
))
await session.commit()
def outcome_for(phase: str, unjudged: list[dict]) -> str:
"""The recorded outcome: a first stop blocks or passes; the stop after a
block records whether the agent answered, and never blocks."""
if phase == "after":
return "left_after_block" if unjudged else "judged_after_block"
return "blocked" if unjudged else "passed"
def parse_written(raw: str, *, cap: int = 200) -> list[tuple[str, str, str]]:
"""The hook's ledger lines, `path<TAB>kind<TAB>name`, as triples.
Kinds outside the ledger's vocabulary and malformed lines are dropped
rather than echoed into an instruction; duplicates collapse; capped."""
out: list[tuple[str, str, str]] = []
for line in (raw or "").splitlines():
parts = line.split("\t")
if len(parts) != 3:
continue
path, kind, name = (p.strip() for p in parts)
if kind not in ("css", "sym", "file") or not path or not name:
continue
if (path, kind, name) not in out:
out.append((path, kind, name))
if len(out) >= cap:
break
return out