Files
FabledScribe/tests/test_services_shape_check.py
T
bvandeusenandClaude Opus 5.5 f1fbdf746a
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 18s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m38s
CI & Build / Build & push image (push) Successful in 33s
feat(shapes): the agent judges what it wrote, at the end of the turn (milestone 439 steps 1-3)
Recording used to be decided by machinery — the only "record it" prompt
fired when a same-named copy already existed (#2664), so a first instance of
a reusable piece was never asked about, and judgment arrived only through
audits. Now the question is asked where the knowledge is: the end of the
turn that wrote the code, of the agent that wrote it.

- Write hooks keep `<sid>.written.ids` (path, kind, name) for every
  definition a write names; a new file adds a `file` line for its stem — a
  candidate in any language without a framework rule (scribe_written_append).
- Stop hook scribe_shape_check.sh sends the ledger to GET
  /api/plugin/shape-check and blocks once, in the server's words, when
  anything is unjudged. Same discipline as the report check: never twice,
  never without a recorded check, another hook's loop left alone; the ledger
  is kept when the instance cannot be reached.
- shape_ledger.unjudged_shapes: no row, unclassified, scoped and hook stamps
  are unjudged; an agent/audit/import verdict is not. A snippet recorded at
  the shape answers for it until the refresh stamps it canonical.
- services/shape_check owns the reason text and records every outcome in
  app_logs (passed / blocked / judged_after_block / left_after_block).
- classify_shapes(repo=…) judges a shape the ledger has not synced yet via a
  provisional row under a bound repo; the sync confirms it, or vanishes and
  revives it with the verdict intact. An unbound repo is refused.

Plugin version minted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 08:39:36 -04:00

82 lines
3.5 KiB
Python

"""The server half of the end-of-turn shape check (milestone 439): what the
agent is asked, what the hook's ledger may say, and the outcome record."""
import json
from unittest.mock import patch
import pytest
from tests.helpers import make_mock_session
def test_the_ledger_lines_parse_and_garbage_is_dropped():
from scribe.services.shape_check import parse_written
raw = ("a.py\tsym\thelper\n"
"a.py\tsym\thelper\n" # duplicate
"web/A.svelte\tfile\tA\n"
"x.py\tbogus\tthing\n" # unknown kind
"only-two\tsym\n" # malformed
"\tsym\tnameless-path\n")
assert parse_written(raw) == [("a.py", "sym", "helper"), ("web/A.svelte", "file", "A")]
assert len(parse_written("".join(f"p\tsym\tn{i}\n" for i in range(500)), cap=7)) == 7
def test_a_first_stop_blocks_or_passes_and_the_next_never_blocks():
from scribe.services.shape_check import outcome_for
one = [{"path": "a", "kind": "sym", "symbol": "b"}]
assert outcome_for("check", one) == "blocked"
assert outcome_for("check", []) == "passed"
assert outcome_for("after", one) == "left_after_block"
assert outcome_for("after", []) == "judged_after_block"
def test_the_reason_names_each_shape_its_evidence_and_the_one_call():
from scribe.services.shape_check import block_reason
unjudged = [
{"path": "web/A.svelte", "kind": "file", "symbol": "A", "status": "new", "evidence": {}},
{"path": "src/x.py", "kind": "sym", "symbol": "load", "status": "unclassified",
"evidence": {"looks_like": {"snippet_id": 12, "basis": "semantic", "score": 0.84},
"diverges_from": 9}},
]
reason = block_reason(unjudged, project_id=3, repo="git.example.com/a/w")
assert "2 definitions" in reason
assert "web/A.svelte · A (file) — new" in reason
assert "looks like #12 0.84" in reason and "#9 is canon in this directory" in reason
assert 'classify_shapes(project_id=3, repo="git.example.com/a/w"' in reason
for verdict in ("create_snippet", "`instance`", "`variant`", "`exempt`"):
assert verdict in reason, verdict
def test_a_long_turn_is_listed_with_a_tail_not_dropped():
from scribe.services.shape_check import _LISTED, block_reason
many = [{"path": "p.py", "kind": "sym", "symbol": f"f{i}", "status": "new", "evidence": {}}
for i in range(_LISTED + 3)]
reason = block_reason(many, project_id=1, repo="")
assert "… and 3 more" in reason
assert "repo=" not in reason
async def test_the_outcome_is_recorded_as_a_plugin_event():
from scribe.services.shape_check import record_shape_check
session = make_mock_session()
with patch("scribe.services.shape_check.async_session", return_value=session):
await record_shape_check(7, "blocked", written=5, unjudged=2, project_id=2)
row = session.add.call_args.args[0]
assert (row.category, row.action, row.user_id) == ("plugin", "shape_check", 7)
assert json.loads(row.details) == {"outcome": "blocked", "written": 5,
"unjudged": 2, "project_id": 2}
async def test_an_unknown_outcome_is_refused_before_anything_is_written():
from scribe.services.shape_check import record_shape_check
session = make_mock_session()
with patch("scribe.services.shape_check.async_session", return_value=session), \
pytest.raises(ValueError):
await record_shape_check(7, "skipped", written=1, unjudged=1)
session.add.assert_not_called()