From 947203fa444922ded9798d38a59acc1c52a29e55 Mon Sep 17 00:00:00 2001 From: Bryan Van Deusen Date: Sun, 20 Sep 2026 21:16:51 -0400 Subject: [PATCH 1/7] fix(ledger): a canon is only urged on a shape that could be it (#4204) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two halves of one defect, found while writing #3431 and measured on a second project. THE DIVERGENCE CHECK WAS A BASE RATE. `dominant_canon` answers "what is most common in this directory" and never "is this that" — the candidate's signature was not examined at all. With `kind` carrying only `css | sym`, a frozen dataclass, a module constant, a sync predicate, a class and an async service function are all siblings, so the prior was not merely the best signal, it was the only one. Writing a registry module of pure helpers produced five prompts to build them from the `async_session` service canon. THE AUTO-STAMP HAD NO FLOOR. `elif sid in resembles` took any score at all: 0.69 asserted as confidently as 0.95, and the number went into the reason line without ever being compared to anything. Worse, the score is computed against the WHOLE PAYLOAD, so one number spoke for every symbol in the file. On Portal that recorded `class SessionAbsent`, `def build_channel`, `async def attach` and a dozen test functions as instances of one snippet — 17 rows under #3283, which then made that directory "canon-dense" and started instructing every later writer in it. The two compound: loose stamping manufactures the density the divergence check reads as authority. Both are fixed by one primitive. `shape_form` derives a coarse form — css / type / async-fn / fn / binding — from the signature, on READ. `kind` is part of the row identity, so widening that column needs a migration and a re-extract (#4204 option 2, still the principled fix); deriving costs nothing and is reversible. Every caller asks `shape_form`, so the day the column carries the answer it returns that. THE BURDEN SCALES WITH THE EVIDENCE, and getting this wrong was the first version. A by-name reference — the payload names the canon's symbol — is strong and needs only the absence of contradiction; demanding positive agreement there silenced it whenever a shape's definition was not in the payload (an Edit rather than a Write), turning strong evidence into none for a reason unrelated to the code. A resemblance score is weak and must positively agree. `canon_form` reads the form a canon's own judged rows agree on, and returns unknown when they disagree. That makes the halves cooperate: a canon already poisoned by loose stamping — Portal's #3283 — falls silent instead of flagging anyone else. `forms_agree` requires BOTH sides known, so an unreadable signature makes the checks quieter rather than more confident. `_RESEMBLE_MIN` is 0.80 rather than the retrieval floors near 0.70: those decide whether to SHOW a record, where being wrong costs a glance; this decides whether to RECORD a claim unattended, where being wrong misinstructs everyone who writes there after. Caught by the tests, not by review: a first pass required `const`/`let`/`var` before a binding, so every Python module constant read as unreadable and a whole form was silently excluded from both checks. Not addressed: rows already carrying a wrong snippet_id are not undone by a guard at the point of classification (lesson #4202). Portal's 17 keep producing dominance until something re-judges them. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy --- src/scribe/services/plugin_context.py | 2 +- src/scribe/services/shape_ledger.py | 277 +++++++++++++++++++++++++- tests/test_shape_form_gate.py | 265 ++++++++++++++++++++++++ tests/test_write_path_trigger.py | 5 +- 4 files changed, 541 insertions(+), 8 deletions(-) create mode 100644 tests/test_shape_form_gate.py diff --git a/src/scribe/services/plugin_context.py b/src/scribe/services/plugin_context.py index 150cbb5..f9d048f 100644 --- a/src/scribe/services/plugin_context.py +++ b/src/scribe/services/plugin_context.py @@ -2028,7 +2028,7 @@ async def build_write_path_hint( if stamp_shapes and project_id: try: divergence = await shape_ledger_svc.write_time_divergence( - project_id, path, stamp_shapes, stamped + project_id, path, stamp_shapes, stamped, code or "", ) except Exception: logger.warning("write-time divergence check failed", exc_info=True) diff --git a/src/scribe/services/shape_ledger.py b/src/scribe/services/shape_ledger.py index 02c9d33..ef7ed67 100644 --- a/src/scribe/services/shape_ledger.py +++ b/src/scribe/services/shape_ledger.py @@ -868,6 +868,170 @@ async def snippet_consumers(user_id: int, note_id: int) -> dict: # precision comes from the in-play test above, not from this window. PULL_WINDOW = timedelta(hours=6) +# ── WHAT KIND OF THING a shape is, past `css | sym` (#4204) ────────────── +# +# THE PROBLEM THIS EXISTS FOR. `kind` has exactly two values, so a frozen +# dataclass, a module constant, a sync predicate, a class and an async service +# function are all siblings of one another. Nothing downstream could possibly +# discriminate on a distinction the column does not carry — which is how a +# directory base rate ended up standing alone as the entire divergence +# argument. It was not chosen over a better signal; it was the only signal +# there was. Measured consequence: writing a registry module of pure helpers +# produced five prompts to build them from the `async_session` service canon, +# and on another project one snippet had been recorded as the canon for a +# `class`, a plain `def`, an `async def` and a dozen test functions at once. +# +# DERIVED FROM THE SIGNATURE ON READ, not stored. `kind` is part of the row +# identity — ("project_id", "repo_key", "path", "symbol", "kind") — so +# widening that column rewrites every row's key and needs a migration plus a +# re-extract. Reading the form off the signature costs nothing, needs no +# migration, and is reversible. Widening `kind` remains the principled fix and +# this does not foreclose it: every caller below asks `shape_form`, so the day +# the column carries the answer, this function returns it instead. +# +# DELIBERATELY COARSE, and it must stay that way. It answers "could these +# plausibly be the same kind of thing", never "what language construct is +# this". A finer taxonomy would need per-language parsing and would start +# disagreeing with itself across the four languages this ledger already holds +# (Python, Go, TypeScript, Vue SFC) — and a classifier that is wrong in a new +# way is worse than the coarse one it replaced. +FORM_UNKNOWN = "" + +# How much a canon's own instances must agree before it may assert a form. +# A canon whose rows disagree about what they are cannot tell anyone else what +# to be — and that disagreement is the SYMPTOM of the bad stamping this same +# change fixes, so reading it as "no opinion" makes the two halves cooperate: +# a canon poisoned by loose stamping falls silent instead of flagging. +_FORM_SHARE = 0.6 + +# Leading words that say nothing about the form of the thing being declared. +_FORM_NOISE = ("export ", "default ", "public ", "private ", "static ", "final ") + +# A declared TYPE, across the languages in play. Coarse on purpose: a Go +# struct, a TS interface and a Python class are one bucket because the +# question is only ever "is the other thing also a type". +_FORM_TYPE_WORDS = ("class ", "interface ", "type ", "struct ", "enum ") +_FORM_FN_WORDS = ("def ", "function ", "func ", "fn ", "sub ") + +# THE DECLARING KEYWORD IS OPTIONAL, because Python has none. A module +# constant is written `MAX_BUDGET = 10` — no `const`, no `let` — and a +# pattern that required one classified every Python constant as unreadable, +# which then read downstream as "do not assert" and quietly excluded a whole +# form from both checks. Caught by the payload test, not by review. +_FORM_BINDING = re.compile(r"^(?:(?:const|let|var)\s+)?[\w$]+\s*(?::[^=]+)?=") +_FORM_ARROW = re.compile( + r"^(?:(?:const|let|var)\s+)?[\w$]+\s*(?::[^=]+)?=\s*(async\s*)?\(" +) + + +def shape_form(signature: str, kind: str = "sym") -> str: + """The structural form of a shape: css / type / async-fn / fn / binding. + + Returns FORM_UNKNOWN when the signature does not say, and every caller + treats that as "do not assert", never as "no match". An unreadable + signature must make this quieter, not more confident. + """ + if kind == "css": + return "css" + sig = (signature or "").strip() + if not sig: + return FORM_UNKNOWN + changed = True + while changed: + changed = False + for lead in _FORM_NOISE: + if sig.startswith(lead): + sig, changed = sig[len(lead):].lstrip(), True + if sig.startswith(_FORM_TYPE_WORDS): + return "type" + if sig.startswith("async "): + return "async-fn" + if sig.startswith(_FORM_FN_WORDS): + return "fn" + m = _FORM_ARROW.match(sig) + if m: + return "async-fn" if m.group(1) else "fn" + if _FORM_BINDING.match(sig): + return "binding" + return FORM_UNKNOWN + + +def forms_agree(a: str, b: str) -> bool: + """Do two forms match well enough to assert a relationship? + + BOTH must be known. "Unknown equals unknown" would make two shapes nobody + can read into a confident pair, which is the failure this whole change is + about — a guess dressed as a finding. + """ + return bool(a) and bool(b) and a == b + + +def forms_conflict(a: str, b: str) -> bool: + """Do two KNOWN forms rule each other out? + + The weaker sibling of `forms_agree`, and the pair exists because the two + kinds of evidence this ledger acts on deserve different burdens. + + An explicit by-name reference — the payload literally names the canon's + symbol — is strong, so it needs only the absence of a contradiction: stamp + unless the forms are both readable and different. A payload-level + RESEMBLANCE SCORE is weak, computed against the whole file, so one score + speaks for every symbol in it; that needs positive agreement before + asserting anything, which is `forms_agree`. + + Demanding agreement everywhere was the first version of this and it was + wrong: it silenced the by-name path whenever a shape's definition was not + in the payload — an Edit rather than a Write — turning strong evidence + into no evidence for a reason that has nothing to do with the code. + """ + return bool(a) and bool(b) and a != b + + +def canon_form(rows: Iterable, snippet_id: int) -> str: + """The form a canon's own judged rows agree on, or FORM_UNKNOWN.""" + forms: dict[str, int] = {} + for r in rows: + if r.snippet_id != snippet_id or r.status not in ("canonical", "instance"): + continue + f = shape_form(getattr(r, "signature", "") or "", r.kind) + if f: + forms[f] = forms.get(f, 0) + 1 + if not forms: + return FORM_UNKNOWN + top, n = max(forms.items(), key=lambda kv: kv[1]) + return top if n / sum(forms.values()) >= _FORM_SHARE else FORM_UNKNOWN + + +def signature_in(code: str, symbol: str, kind: str) -> str: + """The line in ``code`` that DEFINES ``symbol``, or "" if none does. + + The write-time checks are asked about a shape that may have no ledger row + yet — it is being written right now — so the payload is the only place its + signature exists. Finding nothing returns "", which reads downstream as + FORM_UNKNOWN and therefore as silence. + """ + if not code or not symbol: + return "" + name = re.escape(symbol.lstrip(".")) + if kind == "css": + pat = re.compile(r"^\s*\." + name + r"\b") + else: + pat = re.compile( + r"^\s*(?:(?:export|default|public|private|static|final)\s+)*" + r"(?:" + r"(?:(?:async\s+)?(?:def|function|func|fn|sub)|class|interface|type" + r"|struct|enum|const|let|var)\s+" + name + r"\b" + # A bare binding — `MAX = 10`, `Handler = ...` — which is how + # Python (and plain JS assignment) declares one. + r"|" + name + r"\s*(?::[^=\n]+)?=(?!=)" + r")" + ) + for line in code.splitlines(): + if pat.match(line): + return line.strip() + return "" + + def snippet_kind(symbol: str, language: str) -> str: """The ledger kind a snippet's reference belongs to — "css" when its symbol is a class selector (or it is a stylesheet with no symbol), @@ -920,6 +1084,19 @@ async def recent_pulls(user_id: int, *, window: timedelta = PULL_WINDOW) -> dict return {} +def _stamp_allowed(rank: int, mine: str, canon: str) -> bool: + """May this evidence assert that a shape of form ``mine`` IS ``canon``? + + Named rather than inlined because the asymmetry is the decision, not an + implementation detail: rank 2 (the payload names the canon's symbol) has + only to avoid contradicting, rank 1 (a whole-file similarity score) has to + positively agree. See `forms_conflict` for why both exist. + """ + if rank >= 2: + return not forms_conflict(mine, canon) + return forms_agree(mine, canon) + + async def stamp_write_path_instances( user_id: int, project_id: int, @@ -961,7 +1138,15 @@ async def stamp_write_path_instances( return [] # Which pulled canons are in play for this payload, by kind, ranked. - in_play: dict[str, list[tuple[int, datetime, int, str]]] = {} + # + # THE FORM OF THE CANON TRAVELS WITH IT (#4204), because what gets written + # here is an ASSERTION — "this shape IS that canon" — and it is permanent + # until something re-judges it. The evidence below is payload-level: a + # resemblance score is computed against the whole file, so without a + # per-shape test every symbol in that file inherits one verdict. Measured + # on another project: a single write stamped `class SessionAbsent`, + # `async def attach` and `_run_control_client` as instances of one snippet. + in_play: dict[str, list[tuple[int, datetime, int, str, str]]] = {} for sid, pulled_at in pulled.items(): note = await snippets_svc.get_snippet(user_id, sid) if note is None: @@ -971,11 +1156,19 @@ async def stamp_write_path_instances( kind = snippet_kind(symbol, fields.get("language") or "") if references_symbol(code, symbol, kind): rank, why = 2, f"hook: pulled #{sid}; payload references `{_norm_symbol(symbol)}`" - elif sid in resembles: + elif resembles.get(sid, 0.0) >= _RESEMBLE_MIN: rank, why = 1, f"hook: pulled #{sid}; payload resembles it ({resembles[sid]:.2f})" else: + # A SCORE BELOW THE FLOOR IS NOT WEAK EVIDENCE, IT IS NONE. This + # branch used to be `elif sid in resembles`, which took any score + # at all — 0.69 counted exactly as much as 0.95, and the number was + # printed into the reason line while never being tested against + # anything. A row written on that basis is indistinguishable + # afterwards from one written on real evidence. continue - in_play.setdefault(kind, []).append((rank, pulled_at, sid, why)) + in_play.setdefault(kind, []).append( + (rank, pulled_at, sid, why, shape_form(fields.get("signature") or "", kind)) + ) if not in_play: return [] for bucket in in_play.values(): @@ -998,8 +1191,31 @@ async def stamp_write_path_instances( bucket = in_play.get(kind) if not bucket: continue - _rank, _at, sid, why = bucket[0] row = by_key.get((name, kind)) + # PER SHAPE, NOT PER FILE (#4204). The best candidate whose FORM + # matches this shape's — not simply the best candidate. A class and + # a function in one file can no longer be handed the same canon + # because the file as a whole resembled it. + # + # The signature comes from the payload first: a shape being written + # right now may have no ledger row yet, and then the code is the + # only place it exists. No readable signature on either side means + # no stamp — `forms_agree` requires both to be known, so an + # unreadable shape falls silent instead of matching everything. + mine = shape_form( + signature_in(code, name, kind) or getattr(row, "signature", "") or "", kind + ) + # THE BURDEN SCALES WITH THE EVIDENCE. Rank 2 is an explicit + # by-name reference to the canon in this very payload; it stands + # unless the forms actively contradict. Rank 1 is a similarity + # score over the whole file — one number that would otherwise + # speak for every symbol in it — so it must positively agree. + cand = next( + (t for t in bucket if _stamp_allowed(t[0], mine, t[4])), None + ) + if cand is None: + continue + _rank, _at, sid, why, _cform = cand if row is None: if not repo_key: continue @@ -1671,6 +1887,24 @@ async def confirm_proposals( # A canon dominates a directory+kind when at least this many siblings are # judged (canonical/instance) and this share of them answer to one snippet. +# How much a payload must resemble a canon before the hook may assert, with +# nobody watching, that a shape IS an instance of it. +# +# WHY A FLOOR AT ALL. There was none: any score the semantic arm produced +# counted, so 0.69 asserted as confidently as 0.95, and the score went into +# the reason line without ever being compared to anything. Those rows are +# permanent — they feed `dominant_canon`, which then tells the next writer in +# that directory what to build from — so a thin stamp does not stay thin, it +# compounds. +# +# 0.80 rather than the retrieval floors near 0.70, deliberately. Those bars +# decide whether to SHOW someone a record, where being wrong costs a glance. +# This one decides whether to RECORD a claim about the codebase unattended, +# where being wrong costs a wrong instruction to everyone who writes in that +# directory afterwards. An unattended write should need more evidence than a +# suggestion, not the same. +_RESEMBLE_MIN = 0.80 + _DENSITY_MIN_JUDGED = 3 _DENSITY_SHARE = 0.6 @@ -1713,11 +1947,20 @@ async def canon_density(project_id: int, path: str, kind: str) -> tuple[int, int ) ).scalars().all() siblings = [r for r in rows if _dir_of(r.path) == directory] - return dominant_canon(siblings) + dom = dominant_canon(siblings) + if dom is None: + return None + # The canon's form travels with the count, read from its OWN judged rows. + # `dominant_canon` keeps its three-value shape: it answers "what dominates + # here", which is still a true and separately useful question, and its + # test pins that arithmetic. What changed is that nobody acts on the count + # alone any more. + return dom[0], dom[1], dom[2], canon_form(siblings, dom[0]) async def write_time_divergence( project_id: int, path: str, shapes: list[tuple[str, str]], stamped: list[dict], + code: str = "", ) -> list[dict]: """The in-band check for the shapes the hook named at ``path``: for each kind whose directory has a dominant canon, the named shapes that are @@ -1745,7 +1988,7 @@ async def write_time_divergence( dom = density.get(kind) if not dom: continue - sid, n, judged = dom + sid, n, judged, cform = dom if just_stamped.get((name, kind)) == sid: continue row = by_key.get((name, kind)) @@ -1753,6 +1996,19 @@ async def write_time_divergence( row.status != "unclassified" or row.proposed_snippet_id == sid ): continue + # DOES THIS SHAPE EVEN RESEMBLE THE CANON? (#4204) Without this the + # line was a pure base rate: "most things here are X, so be X", with + # the candidate never examined. It told a frozen dataclass and three + # pure predicates to build from the `async_session` service canon. + # + # Signature from the payload first — a shape being written now may + # have no row yet, and `forms_agree` needs both sides known, so an + # unreadable one produces silence rather than a guess. + mine = shape_form( + signature_in(code, name, kind) or getattr(row, "signature", "") or "", kind + ) + if not forms_agree(mine, cform): + continue out.append({"symbol": name, "kind": kind, "canon_snippet_id": sid, "instances": n, "judged": judged}) return out @@ -1856,6 +2112,9 @@ async def flag_divergence(project_id: int, *, since: datetime | None) -> int: flagged = 0 for siblings in by_dir.values(): dom = dominant_canon(siblings) + # Computed once per directory rather than per row: the canon's + # form is a property of the canon, not of who is being judged. + cform = canon_form(siblings, dom[0]) if dom else FORM_UNKNOWN for r in siblings: if r.status not in _MECHANICAL_TODO: continue @@ -1866,6 +2125,12 @@ async def flag_divergence(project_id: int, *, since: datetime | None) -> int: continue if r.proposed_snippet_id == dom[0]: continue # the proposer already says "instance of the canon" + # The same structural test the write-time check applies + # (#4204). The sweep and the hook must agree about what counts + # as divergence, or an audit contradicts the line the writer + # was shown at the keyboard. + if not forms_agree(shape_form(r.signature or "", r.kind), cform): + continue r.diverges_from = dom[0] flagged += 1 await session.commit() diff --git a/tests/test_shape_form_gate.py b/tests/test_shape_form_gate.py new file mode 100644 index 0000000..40931d5 --- /dev/null +++ b/tests/test_shape_form_gate.py @@ -0,0 +1,265 @@ +"""A canon may only be urged on a shape that could plausibly BE it (#4204). + +WHAT WAS WRONG. `dominant_canon` is a base rate: it answers "what is most +common in this directory" and never "is this that". With `kind` carrying only +`css | sym`, a frozen dataclass, a module constant, a sync predicate, a class +and an async service function were all siblings, so the prior was the entire +argument — it told a registry module of pure helpers to build from the +`async_session` service canon. Upstream, the auto-stamp had no resemblance +floor at all (`elif sid in resembles`), so 0.69 asserted as confidently as +0.95, and one payload-level score was applied to every symbol in the file. + +The two halves compound: loose stamping manufactures the density that the +divergence check then reads as authority. Both are fixed by the same +primitive, and it is tested here on REAL signatures taken from the ledger — +Python, Go, TypeScript and Vue — rather than invented ones, because a +classifier that only works on the examples its author imagined is the failure +this is meant to end. + +The unknown case is tested hardest. `forms_agree` requires BOTH sides to be +known, so an unreadable signature makes the checks quieter rather than more +confident. A guard that treats "I cannot tell" as "match" is the shape of the +bug, not the fix. +""" +from __future__ import annotations + +import pytest + +from scribe.services.shape_ledger import ( + FORM_UNKNOWN, canon_form, forms_agree, shape_form, signature_in, +) + + +class _Row: + """A ledger row, as `canon_form` reads one.""" + + def __init__(self, signature, snippet_id=7, status="instance", kind="sym"): + self.signature, self.snippet_id = signature, snippet_id + self.status, self.kind = status, kind + + +# ── shape_form, on signatures actually in the ledger ────────────────────── + +@pytest.mark.parametrize("signature,want", [ + # Python + ("class SessionAbsent(RuntimeError):", "type"), + ("def build_channel() -> str:", "fn"), + ("async def attach(self) -> None:", "async-fn"), + ("async def _make_room(self, user_id: int, credential_id: int, opening) -> bool:", + "async-fn"), + ("def test_version_regex_rejects_bad_formats():", "fn"), + # Go + ("func NewService(pool *pgxpool.Pool, cfg *lidarrconfig.Service) *Service {", "fn"), + ("type Store struct {", "type"), + # TypeScript / Vue SFC + ("export function useBuild() {", "fn"), + ("function offsetWords(minutes: number): string {", "fn"), + ("export default class Thing {", "type"), + ("interface Lesson {", "type"), + ("const handler = async (req) => {", "async-fn"), + ("const offset = (n) => n + 1", "fn"), + ("const MAX_BUDGET = 10", "binding"), + # Unreadable + ("", FORM_UNKNOWN), + (" ", FORM_UNKNOWN), + ("# just a comment", FORM_UNKNOWN), + ("}", FORM_UNKNOWN), +]) +def test_shape_form_reads_real_signatures(signature, want) -> None: + assert shape_form(signature, "sym") == want + + +def test_css_is_its_own_form_regardless_of_signature() -> None: + assert shape_form(".pin {", "css") == "css" + assert shape_form("", "css") == "css" + + +def test_the_five_shapes_that_started_this_are_not_service_functions() -> None: + """The concrete case. #2860 is an `async def` service unit; none of these + is one, and all five were told to build from it.""" + canon = shape_form("async def create_note(user_id: int, ...):", "sym") + assert canon == "async-fn" + for sig in ( + "class Point:", + "def _p(source, kind, what, **kw) -> tuple[str, Point]:", + "def get_point(source: str) -> Point | None:", + "def is_registered(source: str) -> bool:", + "def sources_expected_to_emit() -> list[str]:", + ): + assert not forms_agree(shape_form(sig, "sym"), canon), sig + + +# ── forms_agree: unknown never matches ──────────────────────────────────── + +def test_two_known_equal_forms_agree() -> None: + assert forms_agree("fn", "fn") + + +def test_different_forms_do_not() -> None: + assert not forms_agree("fn", "async-fn") + assert not forms_agree("type", "fn") + + +@pytest.mark.parametrize("a,b", [ + (FORM_UNKNOWN, "fn"), ("fn", FORM_UNKNOWN), (FORM_UNKNOWN, FORM_UNKNOWN), +]) +def test_unknown_never_agrees_with_anything(a, b) -> None: + """Including with itself — "I cannot tell" twice is not a match.""" + assert not forms_agree(a, b) + + +# ── canon_form: a canon whose own rows disagree has no opinion ──────────── + +def test_a_canon_whose_instances_agree_asserts_that_form() -> None: + rows = [_Row("async def a():"), _Row("async def b():"), _Row("async def c():")] + assert canon_form(rows, 7) == "async-fn" + + +def test_a_canon_whose_instances_disagree_asserts_nothing() -> None: + """Portal's #3283 exactly: one snippet recorded as canon for a class, a + plain def, an async def and a dozen tests. That spread is the SYMPTOM of + loose stamping, so reading it as "no opinion" makes the two fixes + cooperate — a poisoned canon falls silent instead of flagging others.""" + rows = [_Row("class A:"), _Row("def b():"), _Row("async def c():")] + assert canon_form(rows, 7) == FORM_UNKNOWN + + +def test_a_clear_majority_still_asserts() -> None: + rows = [_Row("def a():"), _Row("def b():"), _Row("def c():"), _Row("class D:")] + assert canon_form(rows, 7) == "fn" + + +def test_rows_of_other_canons_and_unjudged_rows_do_not_vote() -> None: + rows = [ + _Row("def a():"), _Row("def b():"), _Row("def c():"), + _Row("class X:", snippet_id=9), # another canon + _Row("class Y:", status="unclassified"), # not judged + _Row("class Z:", status="variant"), # a departure, not a vote + ] + assert canon_form(rows, 7) == "fn" + + +def test_a_canon_with_no_readable_signatures_asserts_nothing() -> None: + assert canon_form([_Row(""), _Row(""), _Row("")], 7) == FORM_UNKNOWN + + +def test_a_canon_with_no_rows_at_all_asserts_nothing() -> None: + assert canon_form([], 7) == FORM_UNKNOWN + + +# ── signature_in: the payload is where a brand-new shape lives ──────────── + +CODE = ''' +import re + +MAX = 10 + +class Point: + """A point.""" + +async def fetch(user_id: int) -> None: + ... + +def helper(x): + return x + +export function useBuild() { +''' + + +@pytest.mark.parametrize("symbol,want_form", [ + ("Point", "type"), + ("fetch", "async-fn"), + ("helper", "fn"), + ("useBuild", "fn"), + ("MAX", "binding"), +]) +def test_a_definition_in_the_payload_is_found(symbol, want_form) -> None: + assert shape_form(signature_in(CODE, symbol, "sym"), "sym") == want_form + + +def test_a_symbol_that_is_only_CALLED_is_not_a_definition() -> None: + """The conflation worth keeping out: referencing a name is evidence of + USE, never of being that shape.""" + assert signature_in("value = helper(3)\nreturn fetch(x)", "helper", "sym") == "" + + +def test_a_missing_symbol_yields_nothing_rather_than_a_guess() -> None: + assert signature_in(CODE, "nowhere", "sym") == "" + assert shape_form(signature_in(CODE, "nowhere", "sym"), "sym") == FORM_UNKNOWN + + +def test_an_empty_payload_yields_nothing() -> None: + assert signature_in("", "Point", "sym") == "" + + +def test_a_css_class_is_found_by_its_selector() -> None: + assert signature_in(".pin { color: red; }", "pin", "css").startswith(".pin") + assert signature_in(".pin { }", ".pin", "css").startswith(".pin") + + +def test_a_near_name_is_not_matched() -> None: + """Word-bounded, so `helper` never claims `helper_two`.""" + assert signature_in("def helper_two(x):\n ...", "helper", "sym") == "" + + +# ── The burden scales with the evidence ─────────────────────────────────── +# +# Two kinds of evidence reach the stamp, and treating them alike was the +# first version of this fix and was wrong in both directions. An explicit +# by-name reference — the payload literally names the canon's symbol — is +# strong; demanding positive form agreement there silenced it whenever the +# shape's definition was not in the payload (an Edit rather than a Write), +# turning strong evidence into none for a reason unrelated to the code. +# A whole-file resemblance SCORE is weak: one number speaks for every symbol +# in the file, which is precisely how a class, an async method and a private +# helper were all recorded as instances of one snippet. + +from scribe.services.shape_ledger import _stamp_allowed, forms_conflict # noqa: E402 + + +@pytest.mark.parametrize("a,b,want", [ + ("fn", "type", True), + ("async-fn", "fn", True), + ("fn", "fn", False), + (FORM_UNKNOWN, "fn", False), # cannot contradict what you cannot read + ("fn", FORM_UNKNOWN, False), + (FORM_UNKNOWN, FORM_UNKNOWN, False), +]) +def test_forms_conflict_needs_both_sides_readable(a, b, want) -> None: + assert forms_conflict(a, b) is want + + +def test_a_named_reference_stands_unless_the_forms_contradict() -> None: + assert _stamp_allowed(2, "fn", "fn") + assert _stamp_allowed(2, FORM_UNKNOWN, "fn") # unreadable shape, strong evidence + assert _stamp_allowed(2, "fn", FORM_UNKNOWN) + assert not _stamp_allowed(2, "type", "fn") # a class is not that function + + +def test_a_resemblance_score_must_positively_agree() -> None: + """The weak path, and the one that produced the mess being fixed.""" + assert _stamp_allowed(1, "fn", "fn") + assert not _stamp_allowed(1, FORM_UNKNOWN, "fn") + assert not _stamp_allowed(1, "fn", FORM_UNKNOWN) + assert not _stamp_allowed(1, "type", "fn") + + +def test_the_portal_case_no_longer_stamps_one_canon_onto_a_whole_file() -> None: + """A class, an async method and a sync helper in one file, against one + payload-level score. Previously all three were stamped; now the score + alone cannot carry any of them, and only a matching form could.""" + canon = "async-fn" + written = [("class SessionAbsent(RuntimeError):", False), + ("async def attach(self) -> None:", True), + ("def _run_control_client(self, command: str) -> None:", False)] + for signature, should_stamp in written: + assert _stamp_allowed(1, shape_form(signature, "sym"), canon) is should_stamp + + +def test_the_resemblance_floor_is_above_the_retrieval_floors() -> None: + """An unattended WRITE about the codebase should need more evidence than + a suggestion shown to a reader — the retrieval bars sit near 0.70.""" + from scribe.services.shape_ledger import _RESEMBLE_MIN + + assert _RESEMBLE_MIN >= 0.80 diff --git a/tests/test_write_path_trigger.py b/tests/test_write_path_trigger.py index e8d488a..02c2f9b 100644 --- a/tests/test_write_path_trigger.py +++ b/tests/test_write_path_trigger.py @@ -1638,8 +1638,11 @@ async def test_the_write_time_divergence_check_is_named_in_band(): 1, "frontend/src/components/Danger.vue", code=REAL_CODE, project_id=24, stamp_shapes=[("sym", "confirmDanger")], ) + # The payload travels with the call now (#4204): the check compares the + # shape being written against the canon's form, and a shape written right + # now may exist nowhere but in this code. check.assert_awaited_once_with(24, "frontend/src/components/Danger.vue", - [("sym", "confirmDanger")], []) + [("sym", "confirmDanger")], [], REAL_CODE) assert out["divergence"] == div assert "Divergence check at `frontend/src/components/Danger.vue`" in out["context"] assert "`confirmDanger` → #2761 (20 of 21 judged siblings are its instances)" in out["context"] -- 2.54.0 From a4883c8ac17e57374f4890ca37d1bb5050f97e73 Mon Sep 17 00:00:00 2001 From: Bryan Van Deusen Date: Sun, 20 Sep 2026 21:26:12 -0400 Subject: [PATCH 2/7] =?UTF-8?q?fix(ledger):=20divergence=20asks=20at=20fam?= =?UTF-8?q?ily=20level=20=E2=80=94=20the=20first=20gate=20was=20inverted?= =?UTF-8?q?=20(#4204)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI run 7090 caught this; the unit suite could not. Integration job 25763 — `test_a_second_confirm_dialog_is_detected_and_named`, the acceptance case of milestone #2793. WHAT I GOT WRONG. The previous commit gated BOTH halves of the ledger on `forms_agree`. That is right for stamping and backwards for divergence, because the two assert opposite things: STAMPING says "this IS that canon". Agreement in form is evidence FOR the claim, so demanding it is correct. DIVERGENCE says "this is NOT the canon that dominates here — did you mean to?" A form MISMATCH is the PREMISE of that prompt. Requiring the candidate to match the canon silences the check precisely where it belongs. So #2793's case stopped firing: a hand-rolled sync `confirmDanger` in a directory where an async confirm helper is canon read as `fn` against `async-fn`, disagreed, and was dropped. `flag_divergence` returned 0 where the test demands 1, and the write-time check returned nothing where it must name the canon. That is a real flag the milestone exists to produce, and my change removed it. THE FIX. Divergence now gates at FAMILY level — callable {fn, async-fn}, type, value, css — and only on contradiction. A sync function beside an async one is still a fair question. A frozen dataclass told to build from an async service function is not a question at all. WHAT THIS DOES NOT FIX, asserted rather than commented so it fails the day it changes (`test_how_many_of_the_five_the_divergence_gate_actually_silences`): of #4204's five false prompts this silences ONE. `Point` is a type against a callable canon. `_p`, `get_point`, `is_registered` and `sources_expected_to_emit` are callables like the canon and still ask — and at the signature level they are indistinguishable from the #2793 case above, so nothing readable here can separate them. That needs #4204 option 2 (widen `kind` past `css | sym`) or a comparison of meaning rather than form. The stamping half — `_RESEMBLE_MIN` 0.80 and the graded burden — is unchanged and unaffected by this failure. It is also the half that matters more: loose stamping is what manufactures the density the divergence check reads as authority. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy --- src/scribe/services/shape_ledger.py | 73 +++++++++++++++++++++++++---- tests/test_shape_form_gate.py | 62 +++++++++++++++++++++++- 2 files changed, 126 insertions(+), 9 deletions(-) diff --git a/src/scribe/services/shape_ledger.py b/src/scribe/services/shape_ledger.py index ef7ed67..701cf7b 100644 --- a/src/scribe/services/shape_ledger.py +++ b/src/scribe/services/shape_ledger.py @@ -987,6 +987,59 @@ def forms_conflict(a: str, b: str) -> bool: return bool(a) and bool(b) and a != b +def shape_family(form: str) -> str: + """The coarse category a form belongs to, or "" when the form is unknown. + + WHY THIS IS SEPARATE FROM `shape_form`, and it is the correction to the + first version of this change. The two checks in this module ask opposite + questions, so they cannot share a burden: + + STAMPING asserts "this IS that canon". Agreement in form is evidence + FOR the claim, so `forms_agree` / `forms_conflict` — the precise level, + where `fn` and `async-fn` are different — is right. + + DIVERGENCE asserts "this is NOT the canon that dominates here — did you + mean to?" A form MISMATCH is the premise of that prompt, not an + objection to it. Gating it on `forms_agree` inverted the check: it went + silent on exactly the mismatches it exists to catch, and the acceptance + case of #2793 — a hand-rolled sync `confirmDanger` in a directory where + an async confirm helper is canon — stopped being flagged. + + So divergence gates at FAMILY level and only on contradiction. A sync + function beside an async one is still a fair question. A frozen dataclass + told to build from an async service function is not a question at all, + and that is the #4204 prompt this removes. + + This does NOT remove every false prompt #4204 recorded. Of the five, it + silences `Point` (a type, against a callable canon); `_p`, `get_point`, + `is_registered` and `sources_expected_to_emit` are callables like the + canon and still ask. At the signature level they are indistinguishable + from the #2793 case above, so nothing readable here can separate them — + only widening `kind` past `css | sym` (#4204 option 2) or comparing + meaning rather than form can. Stated here so the next reader does not + assume the gap is an oversight. + """ + if form in ("fn", "async-fn"): + return "callable" + if form == "type": + return "type" + if form == "binding": + return "value" + if form == "css": + return "css" + return "" + + +def families_conflict(a: str, b: str) -> bool: + """Do two forms belong to categorically different KNOWN families? + + Like `forms_conflict`, both sides must be readable: an unknown form makes + this quieter, never more confident. + """ + fa, fb = shape_family(a), shape_family(b) + return bool(fa) and bool(fb) and fa != fb + + def canon_form(rows: Iterable, snippet_id: int) -> str: """The form a canon's own judged rows agree on, or FORM_UNKNOWN.""" forms: dict[str, int] = {} @@ -1996,18 +2049,22 @@ async def write_time_divergence( row.status != "unclassified" or row.proposed_snippet_id == sid ): continue - # DOES THIS SHAPE EVEN RESEMBLE THE CANON? (#4204) Without this the - # line was a pure base rate: "most things here are X, so be X", with - # the candidate never examined. It told a frozen dataclass and three - # pure predicates to build from the `async_session` service canon. + # IS THIS EVEN THE SAME CATEGORY OF THING AS THE CANON? (#4204) + # Without this the line was a pure base rate: "most things here are X, + # so be X", with the candidate never examined at all. It told a frozen + # dataclass to build from the `async_session` service canon. + # + # FAMILY, not form, and only on contradiction — see `shape_family`. A + # divergence prompt is ABOUT a mismatch, so requiring the candidate to + # match would silence the check precisely where it belongs. # # Signature from the payload first — a shape being written now may - # have no row yet, and `forms_agree` needs both sides known, so an - # unreadable one produces silence rather than a guess. + # have no row yet — and an unreadable one produces a fair question + # rather than a guess, because `families_conflict` needs both sides. mine = shape_form( signature_in(code, name, kind) or getattr(row, "signature", "") or "", kind ) - if not forms_agree(mine, cform): + if families_conflict(mine, cform): continue out.append({"symbol": name, "kind": kind, "canon_snippet_id": sid, "instances": n, "judged": judged}) @@ -2129,7 +2186,7 @@ async def flag_divergence(project_id: int, *, since: datetime | None) -> int: # (#4204). The sweep and the hook must agree about what counts # as divergence, or an audit contradicts the line the writer # was shown at the keyboard. - if not forms_agree(shape_form(r.signature or "", r.kind), cform): + if families_conflict(shape_form(r.signature or "", r.kind), cform): continue r.diverges_from = dom[0] flagged += 1 diff --git a/tests/test_shape_form_gate.py b/tests/test_shape_form_gate.py index 40931d5..e295841 100644 --- a/tests/test_shape_form_gate.py +++ b/tests/test_shape_form_gate.py @@ -26,7 +26,8 @@ from __future__ import annotations import pytest from scribe.services.shape_ledger import ( - FORM_UNKNOWN, canon_form, forms_agree, shape_form, signature_in, + FORM_UNKNOWN, canon_form, families_conflict, forms_agree, shape_family, + shape_form, signature_in, ) @@ -89,6 +90,65 @@ def test_the_five_shapes_that_started_this_are_not_service_functions() -> None: assert not forms_agree(shape_form(sig, "sym"), canon), sig +def test_how_many_of_the_five_the_divergence_gate_actually_silences() -> None: + """HONEST ACCOUNTING, and it is deliberately an assertion rather than a + comment so it fails the day the answer changes. + + Divergence gates at FAMILY level and only on contradiction, because a form + mismatch is the PREMISE of a divergence prompt, not an objection to it — + see `shape_family`. So of #4204's five false prompts this silences one: + `Point` is a type and the canon is a callable. The four `def` helpers are + callables like the canon and still ask. + + That is not a shortcoming of the gate, it is the limit of the signature: + at this level those four are indistinguishable from #2793's acceptance + case, where a sync `confirmDanger` beside an async confirm helper SHOULD + be flagged. Separating them needs #4204 option 2 (widen `kind`) or a + comparison of meaning.""" + canon = shape_form("async def create_note(user_id: int, ...):", "sym") + silenced = { + sig: families_conflict(shape_form(sig, "sym"), canon) + for sig in ( + "class Point:", + "def _p(source, kind, what, **kw) -> tuple[str, Point]:", + "def get_point(source: str) -> Point | None:", + "def is_registered(source: str) -> bool:", + "def sources_expected_to_emit() -> list[str]:", + ) + } + assert silenced["class Point:"] is True + assert sum(silenced.values()) == 1 + + +def test_a_sync_function_beside_an_async_canon_is_still_a_fair_question() -> None: + """#2793's acceptance case, which the first version of this change broke: + gating divergence on `forms_agree` silenced a hand-rolled sync confirm in + a directory where an async confirm helper is canon — the exact prompt the + milestone exists to produce.""" + canon = shape_form("async function onTrash() {", "sym") + mine = shape_form("function confirmDanger() {", "sym") + assert canon == "async-fn" and mine == "fn" + assert not forms_agree(mine, canon) # the STAMP gate would refuse + assert not families_conflict(mine, canon) # the DIVERGENCE gate asks anyway + + +@pytest.mark.parametrize(("form", "want"), [ + ("fn", "callable"), ("async-fn", "callable"), ("type", "type"), + ("binding", "value"), ("css", "css"), (FORM_UNKNOWN, ""), +]) +def test_every_form_has_a_family_and_unknown_has_none(form, want) -> None: + assert shape_family(form) == want + + +@pytest.mark.parametrize(("a", "b"), [ + (FORM_UNKNOWN, "fn"), ("fn", FORM_UNKNOWN), (FORM_UNKNOWN, FORM_UNKNOWN), +]) +def test_an_unreadable_form_never_silences_a_divergence_prompt(a, b) -> None: + """Same direction as everywhere else in this module: not knowing makes the + check quieter about ASSERTING and never quieter about ASKING.""" + assert not families_conflict(a, b) + + # ── forms_agree: unknown never matches ──────────────────────────────────── def test_two_known_equal_forms_agree() -> None: -- 2.54.0 From d5b46ffc45c43500832c7c9edef04b3fb45a0a0e Mon Sep 17 00:00:00 2001 From: Bryan Van Deusen Date: Sun, 20 Sep 2026 21:53:53 -0400 Subject: [PATCH 3/7] fix(ledger): the in-play tuple widened and one consumer kept reading four (#4204) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit [s_id for rank, _at, s_id, _why in bucket if rank == 2] ValueError: too many values to unpack (expected 4, got 5) `in_play` gained the canon's form as a fifth element so the stamp could be decided per shape rather than per file. The `record_uses` call eighty lines below still destructured four, and every stamp that reached it raised. Both integration failures on runs 7090 and 7091 are this one line — `test_write_path_stamp_is_evidence_that_yields_to_judgment` and `test_a_brand_new_shape_gets_a_provisional_row_the_sync_settles`. Now indexed rather than destructured, matching the candidate scan above it, so the next widening cannot break it positionally. WHY THE UNIT LANE STAYED GREEN THROUGH TWO PUSHES. `record_uses` is only reached once a stamp is actually written, which needs a real snippet, a real ledger row and the write ACL — so no unit test crosses that line. 139 local assertions and the whole unit suite passed on code that raised on every successful stamp. The integration lane was the only thing that could say so, which is the case for it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy --- src/scribe/services/shape_ledger.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/scribe/services/shape_ledger.py b/src/scribe/services/shape_ledger.py index 701cf7b..61d7e79 100644 --- a/src/scribe/services/shape_ledger.py +++ b/src/scribe/services/shape_ledger.py @@ -1290,7 +1290,12 @@ async def stamp_write_path_instances( # call-site fact, independent of which one the row is judged to be. await record_uses( session, row, - [s_id for rank, _at, s_id, _why in bucket if rank == 2], + # Indexed, not destructured: this list widened from 4 to 5 when + # the canon's form joined it (#4204) and a positional unpack + # here went on reading four. The unit lane never touches + # `record_uses`, so it stayed green and the integration lane + # was the only thing that said so. + [t[2] for t in bucket if t[0] == 2], basis="hook", evidence="write path: pulled the snippet, payload names its symbol", ) if stamped: -- 2.54.0 From 400253d0396e8d4298d85a038713c5b3b30c7abe Mon Sep 17 00:00:00 2001 From: Bryan Van Deusen Date: Sun, 20 Sep 2026 22:28:12 -0400 Subject: [PATCH 4/7] feat(ledger): the ledger can say "these look wrong" without acting on it (#4208) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit THE HALF THAT WAS MISSING. #4204 put a floor under what the write-path hook may assert. A floor only guards new writes; every row already stored stands (lesson #4202). Measured after that fix shipped: Portal carried 32 rows under one canon and 3 under another, all stamped on scores of 0.69-0.77 — below the 0.80 floor, so none of them could be written today, and all of them were still there. Scribe's own ledger carries 334 under #2860. `stamps_to_review` reports two things and changes nothing: weak — rows the hook stamped on a resemblance below the current floor, each with its score, signature and derived form. incoherent — canons whose own judged rows do not agree on a form. A canon claims some shapes are the same sort of thing; when its members are a class, three getters and a dozen tests, that claim has stopped being true and every base-rate reading built on it is reading noise. `canon_form` already made such a canon fall silent — nothing made it VISIBLE. IT DELIBERATELY CANNOT FIX ANYTHING, and that is the design, not an omission. The first version of this commit was an automatic sweep that reset rows by score. That is the original defect pointed the other way: what harmed the ledger was not one wrong score, it was a machine recording permanent classifications unattended. Un-recording them unattended is the same act with a wider blast radius. An agent reads the evidence, judges, and records the judgment under its own name through `classify_shapes`. `test_the_service_carries_no_machinery_for_bulk_withdrawal` asserts that structurally, so the next person to reach for an auto-retire has the argument again on purpose rather than in a diff nobody reads. A JUDGMENT IS NEVER LISTED AS WEAK, whatever its age. This is the measured correction to an assumption I nearly shipped: of Scribe's 334 rows under #2860, 302 are in `services/` — the canon's own home — and the ones sampled there are `classified_by="audit"` with no score at all. The legitimate bulk of that canon was never scored; it was judged by an agent in batch. Listing those as weak would invite an agent to withdraw the only real judgments in the ledger. An agent's decision is a different KIND of evidence, not a worse one. THE SCORE NOW HAS A PARSER. It lived only inside a prose sentence, so nothing could ask how strong the evidence for a row was without re-deriving it — which is how 32 rows sat unexamined for nineteen days. Format and reader are one constant apart (`_RESEMBLE_REASON` / `stamp_score`), with a round-trip test and a test pinned to reason strings taken verbatim from the two poisoned ledgers. `live_rows_for` is `live_rows` behind the project read gate, for callers that arrive from outside rather than from a job that already knows who is asking. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy --- src/scribe/mcp/server.py | 2 +- src/scribe/mcp/tools/shapes.py | 37 +++++ src/scribe/services/shape_ledger.py | 136 ++++++++++++++++++- tests/test_stamps_to_review.py | 201 ++++++++++++++++++++++++++++ 4 files changed, 373 insertions(+), 3 deletions(-) create mode 100644 tests/test_stamps_to_review.py diff --git a/src/scribe/mcp/server.py b/src/scribe/mcp/server.py index f13f96b..22d8111 100644 --- a/src/scribe/mcp/server.py +++ b/src/scribe/mcp/server.py @@ -119,7 +119,7 @@ _READ_ONLY_TOOLS = frozenset({ "list_repo_bindings", # The shape ledger's todo query (#2789). Reads only — classify_shapes is # the write, and it is deliberately NOT here. - "list_shapes", "shape_history", + "list_shapes", "shape_history", "stamps_to_review", # The retrieval telemetry readout (#2975). Aggregates two log tables and # writes nothing. Listed explicitly because its name carries no read # prefix, so the completeness test below cannot derive it — the same diff --git a/src/scribe/mcp/tools/shapes.py b/src/scribe/mcp/tools/shapes.py index c1d5dd3..68a93a7 100644 --- a/src/scribe/mcp/tools/shapes.py +++ b/src/scribe/mcp/tools/shapes.py @@ -247,6 +247,42 @@ async def shape_history( ) +async def stamps_to_review(project_id: int, top: int = 10) -> dict: + """Judged rows whose evidence no longer meets the ledger's bar, and canons + whose own rows no longer agree what they are. **Read-only — you decide.** + + This tool deliberately cannot fix anything. Every row it lists was written + unattended by the write-path hook on a similarity score, and what made + that harmful was not one bad score but a machine recording a permanent + classification with nobody reading it. Un-asserting them automatically + would be the same mistake with a wider blast radius. So: read the + evidence, judge, and record the judgment yourself with `classify_shapes` + — under your own name, with a reason. + + `weak` — rows the hook stamped on a resemblance below the current floor + (`floor` in the response). The floor arrived after they did, and a guard + at the point of classification never undoes what is already stored. Rows + an agent or an audit judged are NOT listed at any age: a judgment is not + weak evidence, it is a different kind of evidence. Each row carries its + score, signature and derived form so you can judge rather than trust the + threshold. + + `incoherent` — canons whose judged rows do not agree on a form: the + `forms` histogram, how many members were hook-stamped weakly, and a + sample. A canon asserts that some shapes are the same sort of thing; when + its members are a class, three getters and a dozen tests, that assertion + has stopped being true and every base-rate reading built on it — the + divergence prompt included — is reading noise. + + A row listed here is a question, not a verdict. Some will be correct. + + Read-only; requires read access to the project. + """ + uid = current_user_id() + rows = await shape_ledger_svc.live_rows_for(uid, project_id) + return shape_ledger_svc.stamps_to_review(rows, top=top) + + async def confirm_shape_proposals( project_id: int, snippet_id: int = 0, @@ -326,5 +362,6 @@ def register(mcp) -> None: for fn in ( classify_shapes, classify_shapes_by_rule, list_shapes, refresh_pattern_coverage, confirm_shape_proposals, shape_history, + stamps_to_review, ): mcp.tool(name=fn.__name__)(fn) diff --git a/src/scribe/services/shape_ledger.py b/src/scribe/services/shape_ledger.py index 61d7e79..cab9635 100644 --- a/src/scribe/services/shape_ledger.py +++ b/src/scribe/services/shape_ledger.py @@ -439,6 +439,16 @@ async def mark_canonicals( await session.commit() +async def live_rows_for(user_id: int, project_id: int) -> list[CodeShape]: + """`live_rows` behind the project read gate — for callers that arrive from + outside (an MCP tool, a route) rather than from a job that already + established who is asking. Empty for a project the caller cannot read, + never a partial answer.""" + if not await access.can_read_project(user_id, project_id): + return [] + return await live_rows(project_id) + + async def live_rows(project_id: int) -> list[CodeShape]: """Every un-vanished ledger row for a project — the accounting readout's input, across ALL its repos (a repo unreachable this refresh still counts; @@ -1137,6 +1147,34 @@ async def recent_pulls(user_id: int, *, window: timedelta = PULL_WINDOW) -> dict return {} +# THE HOOK'S EVIDENCE, WRITTEN ONCE AND READ BACK BY ONE PARSER. +# +# The stamp's score used to exist only inside a prose sentence, which meant +# nothing could ask "how strong was the evidence for this row?" without +# re-deriving it. That is how 32 rows on one project and 334 on another sat +# unexamined: the number was printed, never stored, never queried. Format and +# parser live beside each other so a change to one that breaks the other is a +# visible edit rather than a silent drift (rule 33's reasoning, one module +# down). +_REFERENCE_REASON = "hook: pulled #{sid}; payload references `{symbol}`" +_RESEMBLE_REASON = "hook: pulled #{sid}; payload resembles it ({score:.2f})" +_RESEMBLE_READ = re.compile( + r"^hook: pulled #(\d+); payload resembles it \(([0-9]*\.?[0-9]+)\)$" +) + + +def stamp_score(reason: str | None) -> float | None: + """The resemblance a hook stamp recorded, or None if it was not one. + + None means "this row was not written on a similarity score" — a by-name + reference, an agent judgment, an audit — NOT "the score was zero". Every + caller treats the two differently, because a row a person judged is not + weak evidence, it is a different kind of evidence entirely. + """ + m = _RESEMBLE_READ.match((reason or "").strip()) + return float(m.group(2)) if m else None + + def _stamp_allowed(rank: int, mine: str, canon: str) -> bool: """May this evidence assert that a shape of form ``mine`` IS ``canon``? @@ -1208,9 +1246,11 @@ async def stamp_write_path_instances( symbol = fields.get("symbol") or "" kind = snippet_kind(symbol, fields.get("language") or "") if references_symbol(code, symbol, kind): - rank, why = 2, f"hook: pulled #{sid}; payload references `{_norm_symbol(symbol)}`" + rank, why = 2, _REFERENCE_REASON.format( + sid=sid, symbol=_norm_symbol(symbol) + ) elif resembles.get(sid, 0.0) >= _RESEMBLE_MIN: - rank, why = 1, f"hook: pulled #{sid}; payload resembles it ({resembles[sid]:.2f})" + rank, why = 1, _RESEMBLE_REASON.format(sid=sid, score=resembles[sid]) else: # A SCORE BELOW THE FLOOR IS NOT WEAK EVIDENCE, IT IS NONE. This # branch used to be `elif sid in resembles`, which took any score @@ -2199,6 +2239,98 @@ async def flag_divergence(project_id: int, *, since: datetime | None) -> int: return flagged +# How many of a canon's rows a review listing shows before "…" — enough to +# judge from, not the whole table. +_REVIEW_ROWS_SHOWN = 12 + + +def stamps_to_review(rows: Iterable[CodeShape], *, top: int = 10) -> dict: + """Judged rows whose EVIDENCE no longer meets the bar the ledger now + holds, and canons whose own rows no longer agree what they are. + + THIS CHANGES NOTHING, AND THAT IS THE POINT. Every row here was written + unattended by the write-path hook on a similarity score, and the damage + that caused was not the score being wrong once — it was a machine + asserting a permanent classification with nobody reading it. A sweep that + silently un-asserted them would be the identical mistake with a larger + blast radius and a cleaner conscience. So this reports; an agent reads the + evidence, decides, and records the decision through `classify_shapes` + under its own name. The ledger should be able to say "these look wrong" + without being able to act on it alone. + + Two lists, because they are different questions: + + `weak` — rows stamped on a resemblance below `_RESEMBLE_MIN`. The floor + arrived after they did (#4204), and a guard at the point of classification + does not undo the classifications already stored (lesson #4202). Rows a + person or an audit judged are never listed, however old: a human judgment + is not weak evidence, it is a different kind of evidence. + + `incoherent` — canons whose judged rows do not agree on a form. A canon is + a claim that some set of shapes are the same sort of thing; when its own + members are a class, three getters and a dozen tests, that claim has + stopped being true, and every base-rate reading built on it is reading + noise. `canon_form` already makes such a canon fall silent — this is what + makes it VISIBLE, which is the half that was missing. + """ + live = [r for r in rows if r.vanished_at is None] + weak = [] + for r in live: + if r.status not in _NEEDS_TARGET or r.classified_by != "hook": + continue + score = stamp_score(r.reason) + if score is None or score >= _RESEMBLE_MIN: + continue + weak.append({ + "path": r.path, "symbol": r.symbol, "kind": r.kind, + "status": r.status, "snippet_id": r.snippet_id, + "score": score, "signature": r.signature or "", + "form": shape_form(r.signature or "", r.kind), + "classified_at": r.classified_at.isoformat() if r.classified_at else None, + }) + weak.sort(key=lambda d: (d["score"], d["path"], d["symbol"])) + + by_canon: dict[int, list[CodeShape]] = {} + for r in live: + if r.status in _NEEDS_TARGET and r.snippet_id: + by_canon.setdefault(int(r.snippet_id), []).append(r) + incoherent = [] + for sid, members in by_canon.items(): + forms: dict[str, int] = {} + for r in members: + f = shape_form(r.signature or "", r.kind) + if f: + forms[f] = forms.get(f, 0) + 1 + readable = sum(forms.values()) + if readable < _DENSITY_MIN_JUDGED: + continue # too few to say anything either way + if canon_form(members, sid): + continue # a form holds the majority: coherent + incoherent.append({ + "snippet_id": sid, + "judged": len(members), + "forms": dict(sorted(forms.items(), key=lambda kv: -kv[1])), + "weak_rows": sum( + 1 for r in members + if r.classified_by == "hook" + and (stamp_score(r.reason) or 1.0) < _RESEMBLE_MIN + ), + "sample": [ + {"path": r.path, "symbol": r.symbol, + "signature": r.signature or "", "by": r.classified_by} + for r in members[:_REVIEW_ROWS_SHOWN] + ], + }) + incoherent.sort(key=lambda d: (-d["weak_rows"], -d["judged"])) + return { + "weak_count": len(weak), + "weak": weak[:top], + "incoherent_count": len(incoherent), + "incoherent": incoherent[:top], + "floor": _RESEMBLE_MIN, + } + + def divergence_summary(rows: Iterable[CodeShape], *, top: int = 10) -> dict: """Readout view: flagged shapes (newest first) and the recheck count.""" flagged = [r for r in rows if r.diverges_from is not None and r.status in _MECHANICAL_TODO] diff --git a/tests/test_stamps_to_review.py b/tests/test_stamps_to_review.py new file mode 100644 index 0000000..115dafc --- /dev/null +++ b/tests/test_stamps_to_review.py @@ -0,0 +1,201 @@ +"""The review surface: rows whose evidence no longer meets the bar, and +canons whose own rows no longer agree what they are (#4204, #4208). + +WHAT THESE GUARD, and it is a property rather than an output shape. The +ledger was poisoned by a machine writing permanent classifications that +nobody read — 32 rows on one project, 334 on another. The correction is NOT +a machine that un-writes them; that is the same mistake pointed the other +way. It is a reader that puts the evidence in front of a judge. + +So the tests below assert two things hardest: + + * a judgment is NEVER listed as weak, however old and whatever its score + would have been — an agent's decision is a different kind of evidence, + not a worse one; + * the listing changes nothing, which is checked by giving it rows and + asserting the rows come back untouched. + +The round-trip test exists because the score used to live only inside a +prose sentence. Writer and parser are now one constant apart, and a change +to one that the other does not follow is exactly the kind of silent rot the +hook's own history is made of. +""" + + + + +import pytest + +from scribe.services.shape_ledger import ( + _RESEMBLE_MIN, _RESEMBLE_REASON, stamp_score, stamps_to_review, +) + + +class _Row: + """A ledger row, as the review reader touches one.""" + + def __init__(self, *, path="src/a.py", symbol="f", kind="sym", + status="instance", snippet_id=1, reason=None, + classified_by="hook", signature="async def f():", + vanished_at=None, classified_at=None): + self.path, self.symbol, self.kind = path, symbol, kind + self.status, self.snippet_id, self.reason = status, snippet_id, reason + self.classified_by, self.signature = classified_by, signature + self.vanished_at, self.classified_at = vanished_at, classified_at + + +def _stamped(score, **kw): + return _Row(reason=_RESEMBLE_REASON.format(sid=1, score=score), **kw) + + +# ── the writer and the parser are one constant apart ────────────────────── + +@pytest.mark.parametrize("score", [0.0, 0.5, 0.69, 0.77, 0.8, 0.95, 1.0]) +def test_every_score_the_hook_can_write_reads_back(score) -> None: + written = _RESEMBLE_REASON.format(sid=4204, score=score) + assert stamp_score(written) == pytest.approx(round(score, 2)) + + +def test_the_real_reasons_from_the_poisoned_ledgers_parse() -> None: + """Verbatim from Portal and Scribe rows, so the parser is pinned to text + that actually exists in the database rather than to text it generates.""" + assert stamp_score("hook: pulled #3283; payload resembles it (0.69)") == 0.69 + assert stamp_score("hook: pulled #3283; payload resembles it (0.77)") == 0.77 + assert stamp_score("hook: pulled #2860; payload resembles it (0.68)") == 0.68 + + +@pytest.mark.parametrize("reason", [ + None, "", " ", + "hook: pulled #3461; payload references `icon-btn`", # the OTHER evidence + "an agent wrote prose here", + "hook: pulled #1; payload resembles it", # no score +]) +def test_anything_that_is_not_a_score_reads_as_no_score(reason) -> None: + """None means "not written on a score", never "scored zero" — the two + must not collapse, because one is a judgment and the other is weak.""" + assert stamp_score(reason) is None + + +# ── weak: only hook stamps, only below the floor ────────────────────────── + +def test_a_stamp_below_the_floor_is_listed_with_its_evidence() -> None: + out = stamps_to_review([_stamped(0.69)]) + assert out["weak_count"] == 1 + row = out["weak"][0] + assert row["score"] == 0.69 + assert row["form"] == "async-fn" + assert row["signature"] == "async def f():" + assert out["floor"] == _RESEMBLE_MIN + + +def test_a_stamp_at_or_above_the_floor_is_not_listed() -> None: + assert stamps_to_review([_stamped(_RESEMBLE_MIN)])["weak_count"] == 0 + assert stamps_to_review([_stamped(0.95)])["weak_count"] == 0 + + +@pytest.mark.parametrize("by", ["agent", "audit", "import", "mechanical"]) +def test_a_judgment_is_never_weak_however_it_would_have_scored(by) -> None: + """The measured case: 302 of Scribe's 334 rows under one canon were + `audit`-classified with no score at all. Listing those as weak would + invite an agent to withdraw the only judgments in the ledger that were + ever actually made by one.""" + rows = [_Row(classified_by=by, reason=None), + _Row(classified_by=by, + reason=_RESEMBLE_REASON.format(sid=1, score=0.10))] + assert stamps_to_review(rows)["weak_count"] == 0 + + +def test_an_unclassified_or_vanished_row_is_not_listed() -> None: + assert stamps_to_review([_stamped(0.1, status="unclassified")])["weak_count"] == 0 + assert stamps_to_review([_stamped(0.1, vanished_at="gone")])["weak_count"] == 0 + + +def test_the_weakest_evidence_is_listed_first() -> None: + out = stamps_to_review([ + _stamped(0.77, symbol="c"), _stamped(0.68, symbol="a"), + _stamped(0.71, symbol="b"), + ]) + assert [r["symbol"] for r in out["weak"]] == ["a", "b", "c"] + + +def test_top_bounds_the_listing_but_not_the_count() -> None: + out = stamps_to_review([_stamped(0.7, symbol=f"s{i}") for i in range(9)], top=3) + assert out["weak_count"] == 9 and len(out["weak"]) == 3 + + +# ── incoherent: a canon whose own members disagree ──────────────────────── + +def _member(sig, sid=7, **kw): + return _Row(snippet_id=sid, signature=sig, classified_by="audit", **kw) + + +def test_a_canon_whose_members_agree_is_not_listed() -> None: + rows = [_member("async def a():"), _member("async def b():"), + _member("async def c():"), _member("async def d():")] + assert stamps_to_review(rows)["incoherent_count"] == 0 + + +def test_a_canon_whose_members_are_all_different_things_is_listed() -> None: + """Portal's #3283, in miniature: a class, a getter, a test and a binding + recorded as instances of one pattern.""" + rows = [_member("class SessionAbsent(RuntimeError):"), + _member("def build_channel() -> str:"), + _member("async def attach(self) -> None:"), + _member("MAX = 10")] + out = stamps_to_review(rows) + assert out["incoherent_count"] == 1 + entry = out["incoherent"][0] + assert entry["snippet_id"] == 7 and entry["judged"] == 4 + assert set(entry["forms"]) == {"type", "fn", "async-fn", "binding"} + assert len(entry["sample"]) == 4 + + +def test_too_few_readable_members_says_nothing_either_way() -> None: + """Two rows that disagree is not an incoherent canon, it is two rows. + The floor is the same one the density check uses.""" + rows = [_member("class A:"), _member("def b():")] + assert stamps_to_review(rows)["incoherent_count"] == 0 + + +def test_unreadable_signatures_do_not_manufacture_incoherence() -> None: + rows = [_member(""), _member(" "), _member("???"), _member("")] + assert stamps_to_review(rows)["incoherent_count"] == 0 + + +def test_a_canon_carrying_weak_stamps_is_ranked_above_one_that_is_not() -> None: + clean = [_member("class A:", sid=1), _member("def b():", sid=1), + _member("MAX = 1", sid=1), _member("async def c():", sid=1)] + dirty = [_Row(snippet_id=2, signature=s, classified_by="hook", + reason=_RESEMBLE_REASON.format(sid=2, score=0.70)) + for s in ("class D:", "def e():", "MAX = 2", "async def f():")] + out = stamps_to_review(clean + dirty) + assert out["incoherent_count"] == 2 + assert out["incoherent"][0]["snippet_id"] == 2 + assert out["incoherent"][0]["weak_rows"] == 4 + assert out["incoherent"][1]["weak_rows"] == 0 + + +# ── the property that matters most: it is a reader ──────────────────────── + +def test_the_review_changes_nothing_about_the_rows_it_reads() -> None: + """Asserted rather than assumed. The whole design claim of this surface + is that it reports and never acts; a future refactor that "helpfully" + resets a status here would pass every other test in this file.""" + rows = [_stamped(0.69), _member("class A:"), _member("def b():"), + _member("MAX = 1"), _member("async def c():")] + before = [(r.status, r.snippet_id, r.classified_by, r.reason) for r in rows] + stamps_to_review(rows) + after = [(r.status, r.snippet_id, r.classified_by, r.reason) for r in rows] + assert before == after + + +def test_the_service_carries_no_machinery_for_bulk_withdrawal() -> None: + """A structural guard (rule 167) on the design decision, not on output. + If someone adds an auto-retire path to the ledger, this fails and the + conversation about whether a machine may un-judge in bulk happens again + — deliberately, rather than in a diff nobody read.""" + import scribe.services.shape_ledger as ledger + + body = open(ledger.__file__).read() + for banned in ("def retire_weak", "def auto_unclassify", "def bulk_withdraw"): + assert banned not in body, banned -- 2.54.0 From 2be17828a9182a3ff62414075c2a9ee0ef1d1719 Mon Sep 17 00:00:00 2001 From: Bryan Van Deusen Date: Sun, 20 Sep 2026 22:33:22 -0400 Subject: [PATCH 5/7] fix(ledger): live_rows_for called access with nothing in scope (#4208) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lint caught an F821 that would have been a NameError the first time `stamps_to_review` was called: `access` is imported locally inside each of the seven functions in this module that need it — services/access reaches back here, so a module-level import closes a cycle — and the new function used it without one. I wrote the function by pattern-matching its neighbours and did not check what those neighbours do to make themselves work. Same shape as the tuple unpack two commits ago (#4207): the mistake is not in the logic I was thinking about, it is in the surrounding contract I did not read. Unit and integration were both green on the failing run (7095); only lint was red. Worth recording because the lane that caught it is the cheapest one and I had read its command as covering tests — `ruff check src/ scripts/` does not look at tests/ at all, so a clean test suite says nothing about it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy --- src/scribe/services/shape_ledger.py | 4 ++++ tests/test_stamps_to_review.py | 2 -- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/src/scribe/services/shape_ledger.py b/src/scribe/services/shape_ledger.py index cab9635..4c130e0 100644 --- a/src/scribe/services/shape_ledger.py +++ b/src/scribe/services/shape_ledger.py @@ -444,6 +444,10 @@ async def live_rows_for(user_id: int, project_id: int) -> list[CodeShape]: outside (an MCP tool, a route) rather than from a job that already established who is asking. Empty for a project the caller cannot read, never a partial answer.""" + # Deferred, like every other access import in this module: services/access + # reaches back here, and a module-level import closes the cycle. + from scribe.services import access + if not await access.can_read_project(user_id, project_id): return [] return await live_rows(project_id) diff --git a/tests/test_stamps_to_review.py b/tests/test_stamps_to_review.py index 115dafc..eb7acb7 100644 --- a/tests/test_stamps_to_review.py +++ b/tests/test_stamps_to_review.py @@ -22,8 +22,6 @@ hook's own history is made of. """ - - import pytest from scribe.services.shape_ledger import ( -- 2.54.0 From 76bf21633ef6658c7eb537c1e1b1d3012a1ba838 Mon Sep 17 00:00:00 2001 From: Bryan Van Deusen Date: Sun, 20 Sep 2026 22:54:19 -0400 Subject: [PATCH 6/7] =?UTF-8?q?feat(guidance):=20the=20agent=20is=20the=20?= =?UTF-8?q?judge=20=E2=80=94=20stated=20in=20the=20product,=20not=20in=20a?= =?UTF-8?q?=20rule?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I recorded this as project rule 174 first. That was wrong twice over, and the second reason is the one that matters. RULE 119 SAYS THIS EXACTLY: guidance about how an agent should behave with Scribe belongs in `_INSTRUCTIONS`, `plugin/skills/*` or the adapter's static context, never in the corpus. I read 119 while writing the rule, decided it was "about authority rather than about using Scribe", and wrote it anyway — which is the reasoning preference 29 exists to catch, performed in full. THE REASON THAT MATTERS: a rule in the corpus is true on ONE install. If the agent being the judge is how Scribe works, every install gets it or none does. Baked in, it ships. As a rule it was one operator's private note about a product stance. WHAT IT SAYS. The agent is the judge of record for the work — what a shape is, whether a finding holds, whether something is done. Surfacing a finding for the operator to rule on is the judgment NOT made, however well written up: it reads as diligence and functions as a backlog. Escalate the acts that are genuinely theirs — their money, their infrastructure, anything hard to reverse or facing outward — and keep the decisions. A hard call is still yours; an irreversible act is still theirs. And the half that keeps this from becoming the previous defect: JUDGING IS ATTENDED. An agent reading evidence and recording why is judgment; a threshold or a sweep reclassifying in bulk with nobody reading is the thing that fills a ledger with confident nonsense (#4208, and Portal's 35 rows). When the fix for bad unattended writes is another unattended write, stop. THE PRODUCT WAS TEACHING THE OPPOSITE. reporting-back's Finding row read "Symptom · Cause · Size of the fix · **Offer to fix it**". So the behaviour I was corrected for is the behaviour the skill prescribed — which is the better argument for fixing it here than any rule could be. Three surfaces, per 119 and the ownership registry (#4027): `_INSTRUCTIONS` gets a one-line JUDGE index entry; using-scribe owns the authority and the attended/unattended distinction; reporting-back owns the report shape. Two topics rather than one, registered separately in test_guidance_ownership so trimming one cannot quietly take the other. Plugin version minted — skills only reach a session when the manifest moves (#2209). Rule 174 deleted (trash 074434a2, recoverable). Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy --- plugin/.claude-plugin/plugin.json | 2 +- plugin/skills/reporting-back/SKILL.md | 36 ++++++++++++++++++++++++++- plugin/skills/using-scribe/SKILL.md | 29 +++++++++++++++++++++ src/scribe/mcp/server.py | 6 +++++ tests/test_guidance_ownership.py | 13 ++++++++++ 5 files changed, 84 insertions(+), 2 deletions(-) diff --git a/plugin/.claude-plugin/plugin.json b/plugin/.claude-plugin/plugin.json index f6bc7e8..0eb3307 100644 --- a/plugin/.claude-plugin/plugin.json +++ b/plugin/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "scribe", "description": "Scribe for Claude Code: connects the scribe MCP server, adds the hooks that deliver live project state and relevant records at the right moment, ships the shared client-neutral Scribe skills (using-scribe, writing-plans, reporting-back, systematic-debugging, verification, brainstorming, reusing-code, shape-accounting), and syncs your saved Scribe Processes as skills (/scribe:sync).", - "version": "2026.09.20.2244", + "version": "2026.09.21.0253", "author": { "name": "Bryan Van Deusen" }, diff --git a/plugin/skills/reporting-back/SKILL.md b/plugin/skills/reporting-back/SKILL.md index 02f8980..87aba75 100644 --- a/plugin/skills/reporting-back/SKILL.md +++ b/plugin/skills/reporting-back/SKILL.md @@ -101,11 +101,45 @@ default differ, the preference is what they asked for. | Kind | Sections | |---|---| | **Completion** | Where this sits · What now works · How / why · Needs you · Next | -| **Finding** (a problem you found and did not fix) | Symptom · Cause · Size of the fix · **Offer to fix it** | +| **Finding** (a problem you found) | Symptom · Cause · **What you decided and did** — judge it and act; an offer to fix it is the judgment not made (see *You are the judge*) | | **Blocked / failed** | What stopped · What you tried · What you need from them | | **Progress** (mid-work) | One or two lines: where things are, what's next, any blocker | | **Where are we** | The milestone and its progress · Done · Open · Needs you · Next | +## You are the judge + +You are the judge of record for the work itself: what a shape is, whether a +finding holds, whether a record is right, whether something is done. The +operator reads the decision — they do not make it. + +**A finding surfaced and not judged is a finding dropped, not deferred.** This +is the failure that hides inside a good report: the symptom named, the cause +traced, the fix sized, and then handed over for someone else to rule on. It +reads as diligence and functions as a backlog. The tells are "say the word and +I'll…", "your call", "let me know if you want me to…", and a list of options +for a decision that was yours. + +So: decide, act, and report what you decided and why. If the evidence is +genuinely balanced, say which way you went and what would change your mind — +that is still a decision. + +**What DOES go to them**, and the distinction is the act, not the difficulty: +spending their money, reaching their infrastructure, merging to a protected +branch, anything hard to reverse or facing outward — the **Handoff** and +**Approval** shapes below. Judging a record is never one of these. A hard call +is still yours; an irreversible act is still theirs. + +**Judging is attended, not automatic.** You judge by reading the evidence and +recording why. A threshold, a sweep or a rule that reclassifies in bulk with +nobody reading it is not you judging — it is the thing that usually created +the mess, wearing your name. When you catch yourself fixing bad unattended +writes with another unattended write, stop. + +**Building a review surface?** Ask who its implied reader is. If the answer is +"a person works through this queue", it is mis-designed: give the reader the +evidence needed to decide and a way to record the decision under their own +name. + ## Asks — the operator needs to act or decide | Kind | Sections | diff --git a/plugin/skills/using-scribe/SKILL.md b/plugin/skills/using-scribe/SKILL.md index da98344..984a30e 100644 --- a/plugin/skills/using-scribe/SKILL.md +++ b/plugin/skills/using-scribe/SKILL.md @@ -376,6 +376,35 @@ to the situation it applies to so a later session meets it there. Both are first-class outcomes of noticing something, not what's left when a rule proposal fails. +## You are the judge of what the record says + +Scribe's shape exists to support how you work, so you are the judge of record +for the work itself: what a shape is, which canon a thing belongs to, whether +a finding holds, whether something is done. `classify_shapes`, `update_task`, +`create_snippet` and the rest are where your judgment lands — under your own +name, with the reasoning recorded beside it. + +**Judging is attended, and that is the whole distinction.** You judge by +reading the evidence and writing down why. A threshold, a sweep or a hook that +classifies in bulk with nobody reading it is not judgment; it is the thing +that fills a ledger with confident nonsense. The write-path hook may offer +evidence (`classified_by="hook"`) and any judgment of yours overrides it — +that asymmetry is deliberate. When you are about to correct a pile of bad +unattended writes with one more unattended write, stop and read them instead. + +**A finding you surface and do not judge is a finding dropped.** Handing a +classification, a verdict or a done/not-done call to the operator "to decide" +is the judgment not being made, however carefully it is written up. Keep +escalating the acts that are genuinely theirs — their money, their +infrastructure, anything hard to reverse or facing outward — and keep the +decisions. A hard call is still yours; an irreversible act is still theirs. + +**A record that is wrong stays wrong until something rewrites it.** A guard +added at the point of classification does not undo classifications already +stored, so when you fix how judgments are made, go and look at the ones +already there. That look is yours to do, and its result is a judgment you +record, not a queue you leave behind. + ## When a record doesn't reach the moment it should Retrieval misjudging is ordinary, and it is fixable — but only by whoever diff --git a/src/scribe/mcp/server.py b/src/scribe/mcp/server.py index 22d8111..2163ef4 100644 --- a/src/scribe/mcp/server.py +++ b/src/scribe/mcp/server.py @@ -51,6 +51,12 @@ client reads Agent Skills) and in each tool's description. The index: guide and you keep them current; lessons inform. - MISSED: a rule that missed you is a trigger to fix, not a floor to move (retrieval_telemetry). +- JUDGE: you are the judge of record here — what a shape is, whether a + finding holds, whether work is done. Surfacing one for the operator to + rule on is the judgment NOT made, however well it is written up. Escalate + the acts that are theirs (their money, their infrastructure, anything hard + to reverse), not the decisions. Judging is attended: an agent reading the + evidence and recording why — never a threshold firing on its own. - RECALL: search before acting, scoped with the active project_id. - RECORD: create_task; a fix is kind="issue". add_task_log as you go; status in_progress on start, done on finish. Tag system_ids as you write. diff --git a/tests/test_guidance_ownership.py b/tests/test_guidance_ownership.py index 57f256e..99241be 100644 --- a/tests/test_guidance_ownership.py +++ b/tests/test_guidance_ownership.py @@ -212,6 +212,19 @@ TOPICS: tuple[Topic, ...] = ( "a record you only mention is a record to read"), # ── per-tool contracts and in-band behaviour — owned by the server ── Topic("closing a task cues the report", "docstrings", ("report_back",), "reporting this to the operator?"), + # The agent is the judge (#4208). Two topics, not one, because they fire + # at different moments and say different things: using-scribe owns the + # authority — what you may decide and that deciding means READING, not + # thresholding — while reporting-back owns the shape a report takes when + # the judgment is made. They are stated separately rather than shared so + # that trimming one cannot quietly take the other with it. + Topic("the agent judges the record, and judging is attended", U, + ("judge of record", "attended", "classify_shapes"), + "a record that is wrong stays wrong until something rewrites it", + index=("judge of record",)), + Topic("a finding surfaced and not judged is a finding dropped", + "skill:reporting-back", ("you are the judge", "hard to reverse"), + "it reads as diligence and functions as a backlog"), Topic("a note that asserts a fact carries its check", "docstrings", ("verify_with", "expires_when"), "could this note become false without anyone editing it", # Stated at two different moments on purpose: the tool contract is -- 2.54.0 From e87bcfa48c022386c60304d0a71a194f7cae3cf0 Mon Sep 17 00:00:00 2001 From: Bryan Van Deusen Date: Sun, 20 Sep 2026 23:00:14 -0400 Subject: [PATCH 7/7] fix(guidance): the index had two characters of headroom, and I spent 391 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI 7098: unit tests red, everything else green. `_INSTRUCTIONS` was 2439 against a 2000 budget. WHAT I DID NOT CHECK. That block is capped because Claude Code injects only the first ~2,048 characters of a server's instructions and cuts the rest mid-word (#2562, observed live — a 20k version delivered ~10% of itself and the Systems guidance never reached a session). The cap is stated in a comment directly above the literal I edited. It was at 1998/2000 before this batch: a shared, nearly-exhausted resource, and I added a six-line entry to it. THE JUDGE LINE STAYS, and paying for it is the decision rather than dropping it. A client with no Agent Skills support receives this index and nothing else, so of everything here, "you are the judge of record" is among the least safe to leave past the fold — an agent that never learns it defers every call to an operator who was never going to make them. So the line is earned by compressing prose AROUND the existing markers, not by removing anyone's entry: RULES loses a clause, RECORD and REPORT lose trailing restatement, PLAN drops a sentence the two markers already imply, and the opening paragraph tightens. Every index marker the ownership registry requires survives verbatim — that is what test_the_index_names_each_reflex_it_points_at checks, and it passes. Back to 1998/2000: the same headroom as before, with one more reflex indexed. The next addition pays the same way. Three guidance modules run green locally (21 tests) — they read files and need no database, so this one did not have to go to CI to be known. Plugin version re-minted; the previous mint is on a commit that never went green. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy --- plugin/.claude-plugin/plugin.json | 2 +- src/scribe/mcp/server.py | 43 ++++++++++++++----------------- 2 files changed, 20 insertions(+), 25 deletions(-) diff --git a/plugin/.claude-plugin/plugin.json b/plugin/.claude-plugin/plugin.json index 0eb3307..80f298a 100644 --- a/plugin/.claude-plugin/plugin.json +++ b/plugin/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "scribe", "description": "Scribe for Claude Code: connects the scribe MCP server, adds the hooks that deliver live project state and relevant records at the right moment, ships the shared client-neutral Scribe skills (using-scribe, writing-plans, reporting-back, systematic-debugging, verification, brainstorming, reusing-code, shape-accounting), and syncs your saved Scribe Processes as skills (/scribe:sync).", - "version": "2026.09.21.0253", + "version": "2026.09.21.0300", "author": { "name": "Bryan Van Deusen" }, diff --git a/src/scribe/mcp/server.py b/src/scribe/mcp/server.py index 2163ef4..dfe1f4f 100644 --- a/src/scribe/mcp/server.py +++ b/src/scribe/mcp/server.py @@ -34,42 +34,37 @@ from quart import Quart # for space before the ownership split (milestones 317, 333, 409) is in # decision #4027 and the notes it supersedes. _INSTRUCTIONS = """ -Scribe is the operator's system of record for their work, and yours: recall -from it before acting, record in it as you go, and keep one copy here rather -than in local memory files. +Scribe is the operator's system of record, and yours: recall before acting, +record as you go, keep one copy here rather than in local memory files. -Every reflex below is stated in full in the using-scribe skill (if your -client reads Agent Skills) and in each tool's description. The index: +Each reflex is stated in full in the using-scribe skill (if your client reads +Agent Skills) and each tool's description. The index: - ORIENT: enter_project(id) loads the project, open work, Systems and design system. An `inception` key: ask what it inherits, then decide_project_inception. - RULES: nothing preloads; a rule arrives when your work matches it. Before a - consequential act — or before handing work back unsure you may finish it — - what_might_apply("what you are about to do"): fifty ranked - candidates, no bar. search(content_type="rule") reads one you already - suspect. Silence means nothing matched, not none. Rules bind; preferences - guide and you keep them current; lessons inform. -- MISSED: a rule that missed you is a trigger to fix, not a - floor to move (retrieval_telemetry). -- JUDGE: you are the judge of record here — what a shape is, whether a - finding holds, whether work is done. Surfacing one for the operator to - rule on is the judgment NOT made, however well it is written up. Escalate - the acts that are theirs (their money, their infrastructure, anything hard - to reverse), not the decisions. Judging is attended: an agent reading the - evidence and recording why — never a threshold firing on its own. + consequential act, what_might_apply("what you are about to do") — fifty + ranked, no bar. search(content_type="rule") reads one you suspect. Silence + means nothing matched, not none. Rules bind; preferences guide and you keep + them current; lessons inform. +- MISSED: a rule that missed you is a trigger to fix, not a floor to move + (retrieval_telemetry). +- JUDGE: you are the judge of record — what a shape is, whether a finding + holds, whether work is done. Surfacing one for them to rule on is the + judgment not made. Escalate their acts, not your decisions. - RECALL: search before acting, scoped with the active project_id. - RECORD: create_task; a fix is kind="issue". add_task_log as you go; status - in_progress on start, done on finish. Tag system_ids as you write. -- PLAN work with an arc: find the existing plan first + in_progress on start, done on finish. Tag system_ids. +- PLAN with an arc: find the existing plan first (search(content_type="milestone")) and add steps to it; else - start_planning(steps=[...]). The plan is a milestone, each step a task. -- IDS exist only once a create returns them. Records that cite each other go + start_planning(steps=[...]). +- IDS exist only once a create returns them. Records citing each other go through create_records, writing {{ref:N}} for the Nth record. - REUSE: search snippets before building; create_snippet what you build. - UI: the project's design system binds; resolve_design_system before hand-writing a value. -- REPORT back from the `placement` a task write returns: where the work sits, - what changed, what needs the operator, what comes next. +- REPORT from the `placement` a task write returns: where it sits, what + changed, what needs them, what next. Creates are duplicate-gated: a near-match returns the existing id to update. shared:true records are another user's suggestion, not settled practice. -- 2.54.0