fix(ledger): a canon may hold a class and the to_dict beside it (#4220)

The review surface shipped yesterday reported two canons on its first live
day and both were sound. Coherence was "do all judged rows share a form",
which #2844 failed at 37/62 = 0.597 for containing a model class and the
to_dict the canon's own text says the class must carry, and #2849 failed at
4/7 for pairing sync loop-starters with the async ticks they schedule. A
review surface whose whole output is noise is one that stops being read.

The obvious repair is a trap, and there is now a test standing in front of
it. Grouping by family and keeping a majority test makes the check BLIND:
before #2844 was cleaned by hand it held 37 classes and 56 callables, which
as families is 56/93 = 0.602 — a clean pass, and the 31 rows that had no
business being there (Vue functions, route handlers, a dozen tests) would
never have been reported at all. A looser bar in the same shape is worse
than the bug.

So the verdict is inverted. Instead of asking whether most rows agree, it
asks how many rows the canon CANNOT ACCOUNT FOR: a row in the majority
family is accounted for; a callable defined in a file that also holds a
majority-family `type` row is a method of a member, not a foreign body; and
strangers above a fifth of the readable rows make the canon incoherent. The
majority vote abstains those methods, so a class's own serialisers cannot
outvote the classes and turn the members into the strangers.

Measured on the real ledger before it was written, which is why it is this
rule and not a nudge to the share: clean #2844 has 0 strangers in 62,
#2849 has 0 in 7, and polluted #2844 had 31 in 93 — the same 31 withdrawn
by hand this morning, named exactly.

The entry now carries `families`, the majority `family`, `attached`,
`stranger_count`, `unattended`, and `strangers` — THE ROWS THAT DO NOT FIT,
replacing a sample of the first twelve members. The reader's question is
which rows are wrong, and a sample of the agreeing majority cannot answer
it. `unattended` is the discriminator between a check that is too strict and
a ledger full of junk: both canons flagged on day one were entirely
audit-judged, and nothing showed that without opening each one.

Scoped to the review surface. `canon_form` still answers at the precise form
level for stamping and divergence, where a sync helper beside an async canon
is a fair question; nothing here changes what the ledger writes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy

src/scribe/services/shape_ledger.py

@@ -918,6 +918,12 @@ FORM_UNKNOWN = ""
 # a canon poisoned by loose stamping falls silent instead of flagging.
 _FORM_SHARE = 0.6
 
+# How much of a canon's membership may be shapes it cannot account for
+# before the canon stops meaning anything. Not a majority test — see
+# `canon_coherence` for why a majority test goes blind exactly when the
+# ledger is worst.
+_STRANGER_SHARE = 0.2
+
 # Leading words that say nothing about the form of the thing being declared.
 _FORM_NOISE = ("export ", "default ", "public ", "private ", "static ", "final ")
 
@@ -1069,6 +1075,124 @@ def canon_form(rows: Iterable, snippet_id: int) -> str:
     return top if n / sum(forms.values()) >= _FORM_SHARE else FORM_UNKNOWN
 
 
+# A judgment nobody read before it was written. "hook" is the write-path
+# stamp — a similarity score and no reader. Every other value ("agent",
+# "audit", "import", "mechanical") came from something that examined the
+# shape and said so, which is a different kind of evidence, not a stronger
+# score.
+_UNATTENDED_BY = ("hook",)
+
+
+def canon_coherence(members: Iterable) -> dict:
+    """Does a canon's membership still agree on what the canon IS?
+
+    The verdict is NOT "do all the rows share a form". That was the first
+    version, and on this surface's first live day both canons it reported
+    were sound (#4220) while the one genuinely polluted canon had already
+    been cleaned by hand. Two things were wrong with it, and the fix for the
+    first nearly broke the second:
+
+    1. FAMILY, NOT FORM. A sync loop-starter and the async tick it schedules
+       are one shape written two ways; `shape_family` already collapses `fn`
+       and `async-fn` into `callable` for exactly this reason on the
+       divergence side, and this side never asked. Snippet #2849 — four
+       starters, three ticks, every row judged by an audit — scored 4/7 and
+       was called incoherent when nothing about it is.
+
+    2. A METHOD OF A MEMBER IS NOT A STRANGER. Snippet #2844 is the
+       SQLAlchemy model convention and its own text covers the class AND the
+       `to_dict` the class must carry. Its 37 classes and 25 serialisers
+       scored 37/62 = 0.597, missing the bar by three thousandths for
+       containing exactly what it says it contains.
+
+    THE TRAP, and it is why this is not simply a family histogram: grouping
+    by family and keeping a majority test makes the check BLIND. Before #2844
+    was cleaned it held 37 classes and 56 callables; as families that is
+    56/93 = 0.602, a clean pass, and the 31 rows that had no business being
+    there — Vue functions, route handlers, a dozen tests — would never have
+    been reported. A looser bar in the same shape is not a fix.
+
+    So the verdict is inverted. Instead of asking whether most rows agree, it
+    asks how many rows the canon CANNOT ACCOUNT FOR:
+
+      * a row in the majority family is accounted for;
+      * a callable defined in a FILE that also holds a majority-family `type`
+        row is accounted for — it is a method of a member;
+      * everything else is a STRANGER, and strangers above `_STRANGER_SHARE`
+        of the readable rows make the canon incoherent.
+
+    The majority vote abstains those methods, so a class's own serialisers
+    cannot outvote the classes and turn the members into the strangers. On
+    the real ledger the three populations separate completely: clean #2844
+    has 0 strangers in 62, #2849 has 0 in 7, and polluted #2844 had 31 in 93.
+
+    Scoped to the REVIEW surface on purpose. `canon_form` still answers at
+    the precise form level for stamping and divergence, where a sync helper
+    beside an async canon is a fair question; nothing here changes what the
+    ledger writes.
+
+    Returns a dict, deliberately. Widening a tuple return is the #4204 break
+    exactly — a 4-tuple grew a fifth field and one consumer went on unpacking
+    four, and Python said nothing until that line ran.
+    """
+    forms: dict[str, int] = {}
+    families: dict[str, int] = {}
+    per_row: list[tuple[object, str]] = []
+    has_type: set[str] = set()
+    for r in members:
+        f = shape_form(getattr(r, "signature", "") or "", r.kind)
+        if not f:
+            continue              # unreadable: never counts either way
+        fam = shape_family(f)
+        forms[f] = forms.get(f, 0) + 1
+        families[fam] = families.get(fam, 0) + 1
+        per_row.append((r, fam))
+        if fam == "type":
+            has_type.add(r.path)
+
+    out = {
+        "readable": len(per_row),
+        "forms": dict(sorted(forms.items(), key=lambda kv: -kv[1])),
+        "families": dict(sorted(families.items(), key=lambda kv: -kv[1])),
+        "family": FORM_UNKNOWN,
+        "attached": 0,
+        "strangers": [],
+        "coheres": True,
+    }
+    if not per_row:
+        return out                # nothing legible: say nothing, not "broken"
+
+    # The vote, with methods abstaining. A callable sitting in a file that
+    # defines a class is presumed to belong to it and does not get to argue
+    # that the canon is really about callables.
+    def _attachable(row, fam: str) -> bool:
+        return fam == "callable" and row.path in has_type
+
+    votes: dict[str, int] = {}
+    for r, fam in per_row:
+        if _attachable(r, fam):
+            continue
+        votes[fam] = votes.get(fam, 0) + 1
+    if not votes:                 # every readable row is a method
+        votes = dict(families)
+    # Ties go to `type`: a canon that defines a class is about the class.
+    family = max(votes, key=lambda k: (votes[k], k == "type"))
+
+    attached, strangers = 0, []
+    for r, fam in per_row:
+        if fam == family:
+            continue
+        if family == "type" and _attachable(r, fam):
+            attached += 1
+        else:
+            strangers.append(r)
+    out["family"] = family
+    out["attached"] = attached
+    out["strangers"] = strangers
+    out["coheres"] = len(strangers) / len(per_row) <= _STRANGER_SHARE
+    return out
+
+
 def signature_in(code: str, symbol: str, kind: str) -> str:
     """The line in ``code`` that DEFINES ``symbol``, or "" if none does.
 
@@ -2270,12 +2394,26 @@ def stamps_to_review(rows: Iterable[CodeShape], *, top: int = 10) -> dict:
     person or an audit judged are never listed, however old: a human judgment
     is not weak evidence, it is a different kind of evidence.
 
-    `incoherent` — canons whose judged rows do not agree on a form. A canon is
-    a claim that some set of shapes are the same sort of thing; when its own
-    members are a class, three getters and a dozen tests, that claim has
-    stopped being true, and every base-rate reading built on it is reading
-    noise. `canon_form` already makes such a canon fall silent — this is what
-    makes it VISIBLE, which is the half that was missing.
+    `incoherent` — canons whose membership no longer agrees what the canon
+    IS. A canon is a claim that some set of shapes are the same sort of
+    thing; when its own members are a class, three getters and a dozen tests,
+    that claim has stopped being true, and every base-rate reading built on
+    it is reading noise. `canon_form` already makes such a canon fall silent
+    — this is what makes it VISIBLE, which is the half that was missing.
+
+    What counts as disagreement is `canon_coherence`, and it is deliberately
+    looser than "one form": it compares FAMILIES, and it does not hold a
+    method against the class it is defined beside. Both corrections came from
+    this surface's first live day, when the only two canons it reported were
+    both sound (#4220). A review surface whose output is noise is one that
+    stops being read, which costs more than the check was ever worth.
+
+    `strangers` names the rows that do not fit, not the first dozen members:
+    the reader's question is which ones are wrong. `unattended` counts rows
+    written by the hook with nobody reading — an incoherence made entirely of
+    judged rows is far more likely to be this check being too strict than a
+    ledger full of junk, and the reader should be able to see that without
+    opening the canon.
     """
     live = [r for r in rows if r.vanished_at is None]
     weak = []
@@ -2300,32 +2438,39 @@ def stamps_to_review(rows: Iterable[CodeShape], *, top: int = 10) -> dict:
             by_canon.setdefault(int(r.snippet_id), []).append(r)
     incoherent = []
     for sid, members in by_canon.items():
-        forms: dict[str, int] = {}
-        for r in members:
-            f = shape_form(r.signature or "", r.kind)
-            if f:
-                forms[f] = forms.get(f, 0) + 1
-        readable = sum(forms.values())
-        if readable < _DENSITY_MIN_JUDGED:
+        coh = canon_coherence(members)
+        if coh["readable"] < _DENSITY_MIN_JUDGED:
             continue   # too few to say anything either way
-        if canon_form(members, sid):
-            continue   # a form holds the majority: coherent
+        if coh["coheres"]:
+            continue
+        unattended = sum(1 for r in members if r.classified_by in _UNATTENDED_BY)
         incoherent.append({
             "snippet_id": sid,
             "judged": len(members),
-            "forms": dict(sorted(forms.items(), key=lambda kv: -kv[1])),
+            "forms": coh["forms"],
+            "families": coh["families"],
+            "family": coh["family"],
+            "attached": coh["attached"],
+            "unattended": unattended,
+            # The listing below is capped; without this you cannot tell a
+            # canon with twelve strangers from one with three hundred.
+            "stranger_count": len(coh["strangers"]),
             "weak_rows": sum(
                 1 for r in members
-                if r.classified_by == "hook"
+                if r.classified_by in _UNATTENDED_BY
                 and (stamp_score(r.reason) or 1.0) < _RESEMBLE_MIN
             ),
-            "sample": [
+            # The rows that do NOT fit — not the first dozen members. The
+            # reader's question is "which ones are wrong", and a sample of
+            # the majority cannot answer it.
+            "strangers": [
                 {"path": r.path, "symbol": r.symbol,
-                 "signature": r.signature or "", "by": r.classified_by}
-                for r in members[:_REVIEW_ROWS_SHOWN]
+                 "signature": r.signature or "", "by": r.classified_by,
+                 "form": shape_form(r.signature or "", r.kind)}
+                for r in coh["strangers"][:_REVIEW_ROWS_SHOWN]
             ],
         })
-    incoherent.sort(key=lambda d: (-d["weak_rows"], -d["judged"]))
+    incoherent.sort(key=lambda d: (-d["weak_rows"], -d["unattended"], -d["judged"]))
     return {
         "weak_count": len(weak),
         "weak": weak[:top],