docs(6205): shape_ledger comments state the reason, not the instance that taught it
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / integration (push) Successful in 1m9s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 24s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / integration (push) Successful in 1m9s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 24s
Record ids, snippet numbers, dated measurements and the proposer's version history are removed; each block now states the design reason in the present tense. Calibration figures that justify a floor stay, without the instance they were measured on. One error string drops a note id. The density threshold comment moves above the constants it describes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+220
-259
@@ -1,4 +1,4 @@
|
||||
"""The shape ledger's write side — sync and mechanical marking (#2788).
|
||||
"""The shape ledger's write side — sync and mechanical marking.
|
||||
|
||||
The coverage walk (services/coverage.py) is the only feed that sees every
|
||||
shape, so it is the ledger's sync point: each refresh upserts one repo's
|
||||
@@ -68,7 +68,7 @@ def location_covers(loc_path: str, loc_symbol: str, path: str, name: str) -> boo
|
||||
|
||||
|
||||
# The rows the machine may still speak about: nobody's judgment stands on
|
||||
# them. `scoped` (#2869) is the sync's own by-construction stamp — the
|
||||
# them. `scoped` is the sync's own by-construction stamp — the
|
||||
# proposer, derive grouping, divergence, hook evidence and sweeps treat it
|
||||
# like the todo; only the human todo (`unclassified`) excludes it.
|
||||
_MECHANICAL_TODO = ("unclassified", "scoped")
|
||||
@@ -88,8 +88,8 @@ async def sync_repo_shapes(
|
||||
"""Upsert one repo's extracted shapes into the ledger.
|
||||
|
||||
``shapes`` are (path, kind, name) triples, or the richer ArchiveShape
|
||||
records (#2792) whose 4th/5th fields — signature, body_sha — refresh the
|
||||
row's content fingerprint, and whose 7th (#2869) says the shape is a
|
||||
records whose 4th/5th fields — signature, body_sha — refresh the
|
||||
row's content fingerprint, and whose 7th says the shape is a
|
||||
one-off by construction: such rows are stamped `scoped` (mechanical)
|
||||
while unjudged, and un-stamped if a later tree makes them reachable. ``seen_marker`` is the commit the archive was
|
||||
read at when the forge can say, else the ref name — provenance sugar;
|
||||
@@ -147,7 +147,7 @@ async def sync_repo_shapes(
|
||||
row.classified_sha = body_sha
|
||||
elif judged and row.classified_sha != body_sha and row.recheck_at is None:
|
||||
# The body moved under a standing judgment: the judgment
|
||||
# stands, but asks to be confirmed again (#2793).
|
||||
# stands, but asks to be confirmed again.
|
||||
row.recheck_at = now
|
||||
session.add(_event(row, "drifted", now, commit=seen_marker))
|
||||
row.body_sha = body_sha
|
||||
@@ -214,7 +214,7 @@ async def _judge(
|
||||
async def record_uses(
|
||||
session, row: CodeShape, snippet_ids, *, basis: str, evidence: str | None = None,
|
||||
) -> int:
|
||||
"""Upsert consumption edges shape → snippet (#2870). A judgment-grade
|
||||
"""Upsert consumption edges shape → snippet. A judgment-grade
|
||||
basis (agent/audit/import) overwrites a mechanical one (reference/hook)
|
||||
on the same edge; mechanical never overwrites a judgment. Returns the
|
||||
number of edges written or refreshed. The row must be persisted (flushed)
|
||||
@@ -264,7 +264,7 @@ async def uses_of(shape_ids) -> dict[int, list[CodeShapeUse]]:
|
||||
return out
|
||||
|
||||
|
||||
# --- the CSS consumer map (milestone 302) ------------------------------------
|
||||
# --- the CSS consumer map ---------------------------------------------------
|
||||
|
||||
|
||||
def resolve_consumers(
|
||||
@@ -275,13 +275,13 @@ def resolve_consumers(
|
||||
consumes. ``css_rows`` are (id, path, symbol) of the repo's live css rows;
|
||||
``references`` is scan_archive's path → class token → count.
|
||||
|
||||
Resolution (note 2917): a class named in file F resolves to F's OWN row
|
||||
Resolution: a class named in file F resolves to F's OWN row
|
||||
of that name when F defines it (a scoped rule is consumed by its own
|
||||
template); otherwise to every other file's row of that name — a shared
|
||||
sheet, or, when several files define it, all of them: the map says
|
||||
"ambiguous" by fanning out rather than guessing one.
|
||||
|
||||
A token ending in ``PREFIX_MARK`` is a PREFIX reference (#2970) — the
|
||||
A token ending in ``PREFIX_MARK`` is a PREFIX reference — the
|
||||
static head of a name the template concatenates, `status-*` from
|
||||
`` `status-${s}` ``. It stands for every row whose symbol starts with
|
||||
that head, each resolved by the same own-file-else-fan-out rule. The
|
||||
@@ -410,7 +410,7 @@ async def mark_canonicals(
|
||||
usable = [(nid, p, s) for nid, p, s in recorded if (s or "").strip()]
|
||||
# A location naming a FILE and no symbol is a whole-file record: it makes
|
||||
# no claim about a definition inside (`location_covers`), but it is
|
||||
# exactly the claim a `file` row asks about (milestone 439).
|
||||
# exactly the claim a `file` row asks about.
|
||||
whole_files = {(p or "").strip(): nid for nid, p, s in recorded
|
||||
if (p or "").strip() and not (s or "").strip()}
|
||||
now = datetime.now(timezone.utc)
|
||||
@@ -477,7 +477,7 @@ async def live_rows(project_id: int) -> list[CodeShape]:
|
||||
)
|
||||
|
||||
|
||||
# --- classification (#2789): the judgment write path --------------------------
|
||||
# --- classification: the judgment write path --------------------------------
|
||||
|
||||
# Statuses an explicit classification may set. All five: setting a row back to
|
||||
# `unclassified` is how a judgment is deliberately withdrawn.
|
||||
@@ -493,8 +493,8 @@ def validate_classifications(items: list[dict]) -> str | None:
|
||||
"""The structural error a classification batch would earn, or None.
|
||||
|
||||
Pure and checked BEFORE anything is touched: a batch either applies or
|
||||
errors whole — the StrictArgs lesson (#2709), a caller must never learn
|
||||
later that half a batch silently happened.
|
||||
errors whole, so a caller never learns later that half a batch silently
|
||||
happened.
|
||||
"""
|
||||
if not items:
|
||||
return "classifications is empty — nothing to apply"
|
||||
@@ -520,7 +520,7 @@ def validate_classifications(items: list[dict]) -> str | None:
|
||||
if status in ("variant", "exempt") and not (item.get("reason") or "").strip():
|
||||
return (
|
||||
f"classifications[{i}]: status {status!r} needs a reason — "
|
||||
"the WHY is the record (note 2786)"
|
||||
"the WHY is the record"
|
||||
)
|
||||
code = (item.get("reason_code") or "").strip()
|
||||
if code and code not in REASON_CODES:
|
||||
@@ -554,7 +554,7 @@ async def classify_shapes(
|
||||
caller listed. Idempotent by construction.
|
||||
|
||||
``repo`` — a repo bound to the project — lets a judgment land on a shape
|
||||
the ledger has not synced yet (milestone 439): the shape the agent wrote
|
||||
the ledger has not synced yet: the shape the agent wrote
|
||||
this turn. Such an item becomes a PROVISIONAL row under that repo, the
|
||||
same row the write-path hook has always created, with no seen commit;
|
||||
the next sync confirms it, or marks it vanished until the code reaches
|
||||
@@ -578,7 +578,7 @@ async def classify_shapes(
|
||||
raise ValueError(f"project {project_id} not found or no write access")
|
||||
|
||||
# Snippet targets resolve through the caller's own read access — a
|
||||
# family-canon snippet in another project counts (note 2786), a snippet
|
||||
# family-canon snippet in another project counts, a snippet
|
||||
# the caller cannot read does not exist for them.
|
||||
target_ids = {
|
||||
int(item["snippet_id"])
|
||||
@@ -663,10 +663,10 @@ async def classify_shapes(
|
||||
|
||||
|
||||
async def _resolve_ideas(user_id: int, items: list[dict]) -> list[dict]:
|
||||
"""An item may name a family idea (`idea_id`) in place of a snippet
|
||||
(milestone 463 step 5): it is judged against the idea's reference in the
|
||||
shape's language, or its first when there is none in that language — an
|
||||
idea is shared across languages. An explicit snippet_id wins."""
|
||||
"""An item may name a family idea (`idea_id`) in place of a snippet: it
|
||||
is judged against the idea's reference in the shape's language, or its
|
||||
first when there is none in that language — an idea is shared across
|
||||
languages. An explicit snippet_id wins."""
|
||||
from scribe.services import family_adoption
|
||||
|
||||
out = []
|
||||
@@ -684,7 +684,7 @@ async def _resolve_ideas(user_id: int, items: list[dict]) -> list[dict]:
|
||||
|
||||
|
||||
async def _sync_family(user_id: int, project_id: int, snippet_ids: set[int]) -> list[dict]:
|
||||
"""The adoption ledger follows the shapes (milestone 463 step 5): after a
|
||||
"""The adoption ledger follows the shapes: after a
|
||||
judgment commits, the canon ideas whose references it touched — as the
|
||||
new target or the one it replaced — are answered from the shapes.
|
||||
|
||||
@@ -703,7 +703,7 @@ async def _sync_family(user_id: int, project_id: int, snippet_ids: set[int]) ->
|
||||
|
||||
|
||||
def rule_matches(row: CodeShape, *, path: str, pattern: str, kind: str) -> bool:
|
||||
"""Does a ledger row fall under a rule-form classification (#2868)?
|
||||
"""Does a ledger row fall under a rule-form classification?
|
||||
``path`` is a file or a directory (everything beneath it), ``pattern``
|
||||
a shell glob on the symbol (``""`` = every symbol), ``kind`` narrows to
|
||||
sym/css. Pure, so the sweep's reach can be tested without a database."""
|
||||
@@ -734,7 +734,7 @@ async def classify_shapes_where(
|
||||
reason_code: str | None = None,
|
||||
uses: list[int] | None = None,
|
||||
) -> dict:
|
||||
"""The sweep form of classify_shapes (#2868): one judgment applied to
|
||||
"""The sweep form of classify_shapes: one judgment applied to
|
||||
every live row under ``path`` whose symbol matches ``pattern`` (and
|
||||
``kind``). By default only unjudged rows are touched — `unclassified`
|
||||
and the sync's mechanical `scoped` stamp — a sweep must never silently
|
||||
@@ -816,9 +816,9 @@ async def list_project_shapes(
|
||||
to rows the proposer has spoken about: "any", "canon" (an instance-of-#N
|
||||
suggestion), "derive" (a repeats-with-no-canon group), or one basis
|
||||
name (symbol/reference/text/signature/semantic). ``flag`` narrows to
|
||||
the readout's asks (#2793): "divergence" (new where a canon dominates,
|
||||
the readout's asks: "divergence" (new where a canon dominates,
|
||||
`diverges_from` names it), "recheck" (a judged shape whose body moved),
|
||||
or "unused-css" (milestone 302: a css rule no template names).
|
||||
or "unused-css" (a css rule no template names).
|
||||
"""
|
||||
from sqlalchemy import func, or_
|
||||
|
||||
@@ -854,14 +854,14 @@ async def list_project_shapes(
|
||||
elif flag == "recheck":
|
||||
conds.append(CodeShape.recheck_at.isnot(None))
|
||||
elif flag == "unused-css":
|
||||
# The consumer map's negative space (milestone 302): a live css rule
|
||||
# The consumer map's negative space: a live css rule
|
||||
# no file's markup names. A candidate for deletion, surfaced — never
|
||||
# deleted — because the map reads templates only (a class built at
|
||||
# runtime, or used from a script, is invisible to it).
|
||||
conds.append(CodeShape.kind == "css")
|
||||
conds.append(~CodeShape.id.in_(select(CodeShapeConsumer.shape_id)))
|
||||
if uses:
|
||||
# Consumers of a canon (#2870): rows with a uses edge to it, whatever
|
||||
# Consumers of a canon: rows with a uses edge to it, whatever
|
||||
# shape they themselves are.
|
||||
conds.append(CodeShape.id.in_(
|
||||
select(CodeShapeUse.shape_id).where(CodeShapeUse.snippet_id == uses)
|
||||
@@ -899,11 +899,11 @@ def _consumer_dict(row: CodeShape) -> dict:
|
||||
|
||||
|
||||
async def snippet_consumers(user_id: int, note_id: int) -> dict:
|
||||
"""The structured consumer map for one snippet (#2789): its `instances`
|
||||
"""The structured consumer map for one snippet: its `instances`
|
||||
(rows judged to conform) and `variants` (named departures, each carrying
|
||||
its why). Rows are filtered to projects the CALLER can read — a shared
|
||||
snippet must not become a side channel into someone else's project
|
||||
layout. Empty lists mean "attach nothing" (#2483)."""
|
||||
layout. Empty lists mean "attach nothing"."""
|
||||
from scribe.services import access
|
||||
|
||||
async with async_session() as session:
|
||||
@@ -916,7 +916,7 @@ async def snippet_consumers(user_id: int, note_id: int) -> dict:
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
# The consumption edges (#2870): rows that USE this canon, whatever
|
||||
# The consumption edges: rows that USE this canon, whatever
|
||||
# shape they are themselves — the call-site map.
|
||||
using = (
|
||||
await session.execute(
|
||||
@@ -951,7 +951,7 @@ async def snippet_consumers(user_id: int, note_id: int) -> dict:
|
||||
return out
|
||||
|
||||
|
||||
# --- write-path stamping (#2791): hook evidence lands as rows ----------------
|
||||
# --- write-path stamping: hook evidence lands as rows -----------------------
|
||||
#
|
||||
# The write-path hook (plugin/hooks/scribe_prior_art.sh) fires on every
|
||||
# Write/Edit and already carries the two halves of a consumer-map row: the
|
||||
@@ -959,7 +959,7 @@ async def snippet_consumers(user_id: int, note_id: int) -> dict:
|
||||
# it could not say on its own is WHICH canon the session is instantiating.
|
||||
# The pull stream answers that: a snippet the session opened in full
|
||||
# (get_snippet) and is now writing code that references or resembles is being
|
||||
# reused — and a reused canon's call site is an `instance` (note 2786).
|
||||
# reused — and a reused canon's call site is an `instance`.
|
||||
#
|
||||
# The rule, deliberately two-sided so it cannot fire on noise:
|
||||
# PULLED — a PULLED usage event by this user inside PULL_WINDOW. Offered-
|
||||
@@ -968,9 +968,9 @@ async def snippet_consumers(user_id: int, note_id: int) -> dict:
|
||||
# semantic arm scored it above the write-path threshold for this
|
||||
# very payload. Either is evidence; the pull alone is not.
|
||||
# Both hold → every shape the hook named at that path, of the snippet's kind,
|
||||
# carries N as a PROPOSAL (milestone 439). Until then it became instance-of-N
|
||||
# with classified_by="hook" — a verdict nobody read, and the rows written that
|
||||
# way are what `stamps_to_review` still lists.
|
||||
# carries N as a PROPOSAL. Older rows stamped instance-of-N with
|
||||
# classified_by="hook" — a verdict nobody read — are what `stamps_to_review`
|
||||
# lists.
|
||||
#
|
||||
# A proposal is EVIDENCE, not judgment: it lands only on rows nobody has
|
||||
# judged, never touches a status, and is what the end-of-turn question shows
|
||||
@@ -980,7 +980,7 @@ async def snippet_consumers(user_id: int, note_id: int) -> dict:
|
||||
# precision comes from the in-play test above, not from this window.
|
||||
PULL_WINDOW = timedelta(hours=6)
|
||||
|
||||
# ── WHAT KIND OF THING a shape is, past `css | sym` (#4204) ──────────────
|
||||
# ── WHAT KIND OF THING a shape is, past `css | sym` ──────────────────────
|
||||
#
|
||||
# THE PROBLEM THIS EXISTS FOR. `kind` has exactly two values, so a frozen
|
||||
# dataclass, a module constant, a sync predicate, a class and an async service
|
||||
@@ -1052,7 +1052,7 @@ def shape_form(signature: str, kind: str = "sym") -> str:
|
||||
if kind == "css":
|
||||
return "css"
|
||||
if kind == "file":
|
||||
# A whole file (milestone 439): its own family, comparable only with
|
||||
# A whole file: its own family, comparable only with
|
||||
# other files — a component is never told to build from a helper.
|
||||
return "file"
|
||||
sig = (signature or "").strip()
|
||||
@@ -1101,10 +1101,10 @@ def forms_conflict(a: str, b: str) -> bool:
|
||||
speaks for every symbol in it; that needs positive agreement before
|
||||
asserting anything, which is `forms_agree`.
|
||||
|
||||
Demanding agreement everywhere was the first version of this and it was
|
||||
wrong: it silenced the by-name path whenever a shape's definition was not
|
||||
in the payload — an Edit rather than a Write — turning strong evidence
|
||||
into no evidence for a reason that has nothing to do with the code.
|
||||
Demanding agreement everywhere would silence the by-name path whenever a
|
||||
shape's definition is not in the payload — an Edit rather than a Write —
|
||||
turning strong evidence into no evidence for a reason that has nothing to
|
||||
do with the code.
|
||||
"""
|
||||
return bool(a) and bool(b) and a != b
|
||||
|
||||
@@ -1112,9 +1112,8 @@ def forms_conflict(a: str, b: str) -> bool:
|
||||
def shape_family(form: str) -> str:
|
||||
"""The coarse category a form belongs to, or "" when the form is unknown.
|
||||
|
||||
WHY THIS IS SEPARATE FROM `shape_form`, and it is the correction to the
|
||||
first version of this change. The two checks in this module ask opposite
|
||||
questions, so they cannot share a burden:
|
||||
WHY THIS IS SEPARATE FROM `shape_form`. The two checks in this module ask
|
||||
opposite questions, so they cannot share a burden:
|
||||
|
||||
STAMPING asserts "this IS that canon". Agreement in form is evidence
|
||||
FOR the claim, so `forms_agree` / `forms_conflict` — the precise level,
|
||||
@@ -1122,24 +1121,21 @@ def shape_family(form: str) -> str:
|
||||
|
||||
DIVERGENCE asserts "this is NOT the canon that dominates here — did you
|
||||
mean to?" A form MISMATCH is the premise of that prompt, not an
|
||||
objection to it. Gating it on `forms_agree` inverted the check: it went
|
||||
silent on exactly the mismatches it exists to catch, and the acceptance
|
||||
case of #2793 — a hand-rolled sync `confirmDanger` in a directory where
|
||||
an async confirm helper is canon — stopped being flagged.
|
||||
objection to it. Gating it on `forms_agree` would invert the check: it
|
||||
would go silent on exactly the mismatches it exists to catch, such as a
|
||||
hand-rolled sync `confirmDanger` in a directory where an async confirm
|
||||
helper is canon.
|
||||
|
||||
So divergence gates at FAMILY level and only on contradiction. A sync
|
||||
function beside an async one is still a fair question. A frozen dataclass
|
||||
told to build from an async service function is not a question at all,
|
||||
and that is the #4204 prompt this removes.
|
||||
told to build from an async service function is not a question at all.
|
||||
|
||||
This does NOT remove every false prompt #4204 recorded. Of the five, it
|
||||
silences `Point` (a type, against a callable canon); `_p`, `get_point`,
|
||||
`is_registered` and `sources_expected_to_emit` are callables like the
|
||||
canon and still ask. At the signature level they are indistinguishable
|
||||
from the #2793 case above, so nothing readable here can separate them —
|
||||
only widening `kind` past `css | sym` (#4204 option 2) or comparing
|
||||
meaning rather than form can. Stated here so the next reader does not
|
||||
assume the gap is an oversight.
|
||||
This does NOT remove every false prompt. A type against a callable canon
|
||||
is silenced; an unrelated callable beside a callable canon still asks,
|
||||
because at the signature level it is indistinguishable from the
|
||||
`confirmDanger` case above. Only widening `kind` past `css | sym` or
|
||||
comparing meaning rather than form can separate them. The gap is known,
|
||||
not an oversight.
|
||||
"""
|
||||
if form in ("fn", "async-fn"):
|
||||
return "callable"
|
||||
@@ -1192,31 +1188,27 @@ _UNATTENDED_BY = ("hook",)
|
||||
def canon_coherence(members: Iterable) -> dict:
|
||||
"""Does a canon's membership still agree on what the canon IS?
|
||||
|
||||
The verdict is NOT "do all the rows share a form". That was the first
|
||||
version, and on this surface's first live day both canons it reported
|
||||
were sound (#4220) while the one genuinely polluted canon had already
|
||||
been cleaned by hand. Two things were wrong with it, and the fix for the
|
||||
first nearly broke the second:
|
||||
The verdict is NOT "do all the rows share a form". A form-agreement
|
||||
check reports sound canons and misses polluted ones, for two reasons, and
|
||||
the fix for the first can break the second:
|
||||
|
||||
1. FAMILY, NOT FORM. A sync loop-starter and the async tick it schedules
|
||||
are one shape written two ways; `shape_family` already collapses `fn`
|
||||
and `async-fn` into `callable` for exactly this reason on the
|
||||
divergence side, and this side never asked. Snippet #2849 — four
|
||||
starters, three ticks, every row judged by an audit — scored 4/7 and
|
||||
was called incoherent when nothing about it is.
|
||||
are one shape written two ways; `shape_family` collapses `fn` and
|
||||
`async-fn` into `callable` for exactly this reason on the divergence
|
||||
side. A canon of four starters and three ticks is coherent, and a
|
||||
form count would score it 4/7.
|
||||
|
||||
2. A METHOD OF A MEMBER IS NOT A STRANGER. Snippet #2844 is the
|
||||
SQLAlchemy model convention and its own text covers the class AND the
|
||||
`to_dict` the class must carry. Its 37 classes and 25 serialisers
|
||||
scored 37/62 = 0.597, missing the bar by three thousandths for
|
||||
containing exactly what it says it contains.
|
||||
2. A METHOD OF A MEMBER IS NOT A STRANGER. A model convention can cover
|
||||
the class AND the `to_dict` the class must carry. Thirty-seven classes
|
||||
and twenty-five serialisers score 37/62 on form, under the bar, for
|
||||
containing exactly what the canon says it contains.
|
||||
|
||||
THE TRAP, and it is why this is not simply a family histogram: grouping
|
||||
by family and keeping a majority test makes the check BLIND. Before #2844
|
||||
was cleaned it held 37 classes and 56 callables; as families that is
|
||||
56/93 = 0.602, a clean pass, and the 31 rows that had no business being
|
||||
there — Vue functions, route handlers, a dozen tests — would never have
|
||||
been reported. A looser bar in the same shape is not a fix.
|
||||
by family and keeping a majority test makes the check BLIND. The same
|
||||
model canon polluted with 31 unrelated callables — Vue functions, route
|
||||
handlers, tests — holds 37 classes and 56 callables; as families that is
|
||||
56/93, a clean pass, and the 31 rows that have no business being there
|
||||
are never reported. A looser bar in the same shape is not a fix.
|
||||
|
||||
So the verdict is inverted. Instead of asking whether most rows agree, it
|
||||
asks how many rows the canon CANNOT ACCOUNT FOR:
|
||||
@@ -1228,18 +1220,18 @@ def canon_coherence(members: Iterable) -> dict:
|
||||
of the readable rows make the canon incoherent.
|
||||
|
||||
The majority vote abstains those methods, so a class's own serialisers
|
||||
cannot outvote the classes and turn the members into the strangers. On
|
||||
the real ledger the three populations separate completely: clean #2844
|
||||
has 0 strangers in 62, #2849 has 0 in 7, and polluted #2844 had 31 in 93.
|
||||
cannot outvote the classes and turn the members into the strangers. The
|
||||
three cases above separate completely: the clean model canon has 0
|
||||
strangers in 62, the loop canon 0 in 7, and the polluted one 31 in 93.
|
||||
|
||||
Scoped to the REVIEW surface on purpose. `canon_form` still answers at
|
||||
the precise form level for stamping and divergence, where a sync helper
|
||||
beside an async canon is a fair question; nothing here changes what the
|
||||
ledger writes.
|
||||
|
||||
Returns a dict, deliberately. Widening a tuple return is the #4204 break
|
||||
exactly — a 4-tuple grew a fifth field and one consumer went on unpacking
|
||||
four, and Python said nothing until that line ran.
|
||||
Returns a dict, deliberately. Widening a tuple return breaks every
|
||||
consumer that still unpacks the old width, and Python says nothing until
|
||||
that line runs.
|
||||
"""
|
||||
forms: dict[str, int] = {}
|
||||
families: dict[str, int] = {}
|
||||
@@ -1383,13 +1375,11 @@ async def recent_pulls(user_id: int, *, window: timedelta = PULL_WINDOW) -> dict
|
||||
|
||||
# THE HOOK'S EVIDENCE, WRITTEN ONCE AND READ BACK BY ONE PARSER.
|
||||
#
|
||||
# The stamp's score used to exist only inside a prose sentence, which meant
|
||||
# nothing could ask "how strong was the evidence for this row?" without
|
||||
# re-deriving it. That is how 32 rows on one project and 334 on another sat
|
||||
# unexamined: the number was printed, never stored, never queried. Format and
|
||||
# The stamp's score is stored in a fixed format so something can ask "how
|
||||
# strong was the evidence for this row?" without re-deriving it. A score that
|
||||
# is only printed is never queried, and weak rows sit unexamined. Format and
|
||||
# parser live beside each other so a change to one that breaks the other is a
|
||||
# visible edit rather than a silent drift (rule 33's reasoning, one module
|
||||
# down).
|
||||
# visible edit rather than a silent drift.
|
||||
_REFERENCE_REASON = "hook: pulled #{sid}; payload references `{symbol}`"
|
||||
_RESEMBLE_REASON = "hook: pulled #{sid}; payload resembles it ({score:.2f})"
|
||||
_RESEMBLE_READ = re.compile(
|
||||
@@ -1435,12 +1425,11 @@ async def suggest_write_path_instances(
|
||||
) -> list[dict]:
|
||||
"""Offer hook evidence as a PROPOSAL on the shapes being written.
|
||||
|
||||
This used to land the evidence as `instance` rows, classified_by=hook —
|
||||
a permanent classification written with nobody reading it. That is what
|
||||
poisoned two ledgers (132 and 488 rows below today's floor, #4608), and
|
||||
what milestone 439 retires: the agent that wrote the code says what it is,
|
||||
at the end of the turn, and this evidence is what it is shown when it
|
||||
does. So the row keeps its status; the candidate canon goes into the
|
||||
Evidence never lands as an `instance` row: that would be a permanent
|
||||
classification written with nobody reading it. The agent that wrote the
|
||||
code says what it is, at the end of the turn, and this evidence is what
|
||||
it is shown when it does. So the row keeps its status; the candidate
|
||||
canon goes into the
|
||||
proposal fields (`proposed_snippet_id`, `proposal_basis` "reference" for a
|
||||
by-name hit or "semantic" for a resemblance, `proposal_score`), where the
|
||||
end-of-turn question reads it as "looks like #N". A judged row is left
|
||||
@@ -1477,7 +1466,7 @@ async def suggest_write_path_instances(
|
||||
|
||||
# Which pulled canons are in play for this payload, by kind, ranked.
|
||||
#
|
||||
# THE FORM OF THE CANON TRAVELS WITH IT (#4204), because what gets written
|
||||
# THE FORM OF THE CANON TRAVELS WITH IT, because what gets written
|
||||
# here is an ASSERTION — "this shape IS that canon" — and it is permanent
|
||||
# until something re-judges it. The evidence below is payload-level: a
|
||||
# resemblance score is computed against the whole file, so without a
|
||||
@@ -1499,12 +1488,10 @@ async def suggest_write_path_instances(
|
||||
elif resembles.get(sid, 0.0) >= _RESEMBLE_MIN:
|
||||
rank, why = 1, _RESEMBLE_REASON.format(sid=sid, score=resembles[sid])
|
||||
else:
|
||||
# A SCORE BELOW THE FLOOR IS NOT WEAK EVIDENCE, IT IS NONE. This
|
||||
# branch used to be `elif sid in resembles`, which took any score
|
||||
# at all — 0.69 counted exactly as much as 0.95, and the number was
|
||||
# printed into the reason line while never being tested against
|
||||
# anything. A row written on that basis is indistinguishable
|
||||
# afterwards from one written on real evidence.
|
||||
# A SCORE BELOW THE FLOOR IS NOT WEAK EVIDENCE, IT IS NONE.
|
||||
# Taking any score at all (`elif sid in resembles`) would count
|
||||
# 0.69 exactly as much as 0.95, and a row written on that basis is
|
||||
# indistinguishable afterwards from one written on real evidence.
|
||||
continue
|
||||
in_play.setdefault(kind, []).append(
|
||||
(rank, pulled_at, sid, why, shape_form(fields.get("signature") or "", kind))
|
||||
@@ -1531,10 +1518,10 @@ async def suggest_write_path_instances(
|
||||
if not bucket:
|
||||
continue
|
||||
row = by_key.get((name, kind))
|
||||
# PER SHAPE, NOT PER FILE (#4204). The best candidate whose FORM
|
||||
# PER SHAPE, NOT PER FILE. The best candidate whose FORM
|
||||
# matches this shape's — not simply the best candidate. A class and
|
||||
# a function in one file can no longer be handed the same canon
|
||||
# because the file as a whole resembled it.
|
||||
# a function in one file are never handed the same canon because
|
||||
# the file as a whole resembled it.
|
||||
#
|
||||
# The signature comes from the payload first: a shape being written
|
||||
# right now may have no ledger row yet, and then the code is the
|
||||
@@ -1566,7 +1553,7 @@ async def suggest_write_path_instances(
|
||||
by_key[(name, kind)] = row
|
||||
elif not (row.status in _MECHANICAL_TODO or row.classified_by == "hook"):
|
||||
continue # a judgment — or the canon itself — stands
|
||||
# A proposal, never a status: the writer decides (milestone 439).
|
||||
# A proposal, never a status: the writer decides.
|
||||
row.proposed_snippet_id = sid
|
||||
row.proposal_basis = "reference" if _rank >= 2 else "semantic"
|
||||
row.proposal_score = resembles.get(sid) if _rank < 2 else 1.0
|
||||
@@ -1575,15 +1562,14 @@ async def suggest_write_path_instances(
|
||||
"path": path, "symbol": name, "kind": kind,
|
||||
"snippet_id": sid, "reason": why,
|
||||
})
|
||||
# Every pulled canon the payload NAMES is a uses edge (#2870) — the
|
||||
# Every pulled canon the payload NAMES is a uses edge — the
|
||||
# call-site fact, independent of which one the row is judged to be.
|
||||
await record_uses(
|
||||
session, row,
|
||||
# Indexed, not destructured: this list widened from 4 to 5 when
|
||||
# the canon's form joined it (#4204) and a positional unpack
|
||||
# here went on reading four. The unit lane never touches
|
||||
# `record_uses`, so it stayed green and the integration lane
|
||||
# was the only thing that said so.
|
||||
# Indexed, not destructured: the bucket tuples carry the
|
||||
# canon's form as a fifth field, and a positional unpack breaks
|
||||
# silently whenever they widen. The unit lane never reaches
|
||||
# `record_uses`; only the integration lane would catch it.
|
||||
[t[2] for t in bucket if t[0] == 2],
|
||||
basis="hook", evidence="write path: pulled the snippet, payload names its symbol",
|
||||
)
|
||||
@@ -1592,7 +1578,7 @@ async def suggest_write_path_instances(
|
||||
return suggested
|
||||
|
||||
|
||||
# --- the mechanical proposer (#2792): the machine proposes, judgment classifies
|
||||
# --- the mechanical proposer: the machine proposes, judgment classifies
|
||||
#
|
||||
# Runs inside the coverage refresh — the one moment the shape BODIES exist
|
||||
# (the archive is in memory; the ledger stores fingerprints, never code).
|
||||
@@ -1605,8 +1591,7 @@ async def suggest_write_path_instances(
|
||||
# text whitespace-insensitive containment either way between the
|
||||
# body and the canon's recorded code (git-grep, in effect —
|
||||
# a verbatim copy, or the canon's own call-site example);
|
||||
# reference the body calls/uses a canon's symbol — the call-site shape
|
||||
# the P7 backfill classified by hand (#2790);
|
||||
# reference the body calls/uses a canon's symbol — the call-site shape;
|
||||
# signature a sym whose definition line, name blanked, resembles the
|
||||
# canon's (the move_*/resequence family shape);
|
||||
# semantic the widest net: the body (concept-queried, as the write-path
|
||||
@@ -1617,7 +1602,7 @@ async def suggest_write_path_instances(
|
||||
# A hit is a PROPOSAL on the row (proposed_snippet_id/basis/score), never a
|
||||
# classification: an agent confirms in batches (confirm_proposals) or judges
|
||||
# otherwise (classify_shapes clears it). Rows with no canon hit are grouped
|
||||
# by the derive-first rule (note 2786): the same body fingerprint in ≥2
|
||||
# by the derive-first rule: the same body fingerprint in ≥2
|
||||
# places, or the same name defined in ≥3 files, is "a repeating shape with
|
||||
# no canon — derive one first", carried as proposal_basis="derive" + a group
|
||||
# key so sessions see the consolidation candidates as one thing.
|
||||
@@ -1627,7 +1612,7 @@ async def suggest_write_path_instances(
|
||||
# recur by convention, not by duplication).
|
||||
_DERIVE_MIN_DUP = 2
|
||||
_DERIVE_MIN_NAME = 3
|
||||
# CSS is never grouped by body (note 2917): classes for different purposes
|
||||
# CSS is never grouped by body: classes for different purposes
|
||||
# share declarations because the style system makes them alike — `.text-muted`
|
||||
# and `.pin-badge-auto` carrying the same `color: var(--fs-text-tertiary)` are
|
||||
# two meanings, not two copies. A CSS family is a NAME defined in more than
|
||||
@@ -1639,40 +1624,36 @@ _DERIVE_MIN_NAME_CSS = 2
|
||||
_SEMANTIC_CAP = 150
|
||||
# The semantic arm has NO floor of its own: it uses the write-path surface's,
|
||||
# because it asks the write-path hint's question — "which canon does this body
|
||||
# resemble?" — of the same snippet documents (#4208).
|
||||
# resemble?" — of the same snippet documents.
|
||||
#
|
||||
# It had one, 0.8, set after the first live run paired alembic
|
||||
# upgrade()/downgrade() with unrelated canons at 0.68–0.75. That floor was
|
||||
# sized for a proposal nobody reads, and nobody-reads is not how proposals are
|
||||
# consumed: a judge reviews each semantic one (confirm_proposals refuses an
|
||||
# unnamed batch, and the tool says "semantic deserves a look"). Measured on
|
||||
# 2026-09-22 it proposed 0 of 150, while judged instances of the service-unit
|
||||
# canon score 0.68–0.71 against it. A wrong proposal costs the judge a look; a
|
||||
# floor that yields nothing costs the signal. Noise in this band is expected —
|
||||
# the score rides on the proposal so the judge can weigh it.
|
||||
# A higher floor (0.8) would be sized for a proposal nobody reads, and that is
|
||||
# not how proposals are consumed: a judge reviews each semantic one
|
||||
# (confirm_proposals refuses an unnamed batch, and the tool says "semantic
|
||||
# deserves a look"). Judged instances of a canon typically score 0.68–0.71
|
||||
# against it, so 0.8 proposes nothing. A wrong proposal costs the judge a
|
||||
# look; a floor that yields nothing costs the signal. Noise in this band is
|
||||
# expected — the score rides on the proposal so the judge can weigh it.
|
||||
#
|
||||
# How many hits the arm scans for an ALLOWED canon. Wider than the hint's
|
||||
# budget because disallowed snippets (other language families) share the
|
||||
# ranking: one measured true instance sat 4th, behind three it could not use.
|
||||
# ranking: a true instance can sit 4th, behind three it could not use.
|
||||
_SEMANTIC_LIMIT = 8
|
||||
# A MISS FROM THIS ARM IS NOT EVIDENCE, and nothing may read it as one (#4208).
|
||||
# #4208 briefly stored "compared, nothing cleared the floor" and let it
|
||||
# withdraw divergence prompts. It silenced real divergences exactly as it
|
||||
# silenced helpers: most judged instances of a canon score below any floor
|
||||
# that keeps noise out (0.66 and under, in the measurement above), and helpers
|
||||
# score 0.66–0.75 against snippets they have nothing to do with. A code body
|
||||
# against a prose-forward snippet document measures the wrong field (#2518);
|
||||
# no floor separates the two bands. The arm PROPOSES on a hit; its silence
|
||||
# says nothing.
|
||||
# A MISS FROM THIS ARM IS NOT EVIDENCE, and nothing may read it as one.
|
||||
# Letting "compared, nothing cleared the floor" withdraw a divergence prompt
|
||||
# silences real divergences exactly as it silences helpers: many judged
|
||||
# instances of a canon score below any floor that keeps noise out (0.66 and
|
||||
# under), and helpers score 0.66–0.75 against snippets they have nothing to do
|
||||
# with. A code body against a prose-forward snippet document measures the
|
||||
# wrong field; no floor separates the two bands. The arm PROPOSES on a hit;
|
||||
# its silence says nothing.
|
||||
#
|
||||
# THE FAMILY ARM (milestone 463 step 5). The arm above is held to the shape's
|
||||
# own project and language family, because across projects it was pure noise
|
||||
# (#2871). A family idea is the exception the hold was waiting for: an idea
|
||||
# shared across projects ON PURPOSE, whose reference may be in another
|
||||
# language (a Python throttle is an instance of a Go one). So the arm also
|
||||
# proposes the reference of a canon idea that reaches the shape's project
|
||||
# through a shared platform — any language, any project — under a stricter
|
||||
# rule of its own, measured 2026-10-06 (#4991) on cross-project pairs:
|
||||
# THE FAMILY ARM. The arm above is held to the shape's own project and
|
||||
# language family, because across projects it is noise. A family idea is the
|
||||
# exception: an idea shared across projects ON PURPOSE, whose reference may be
|
||||
# in another language (a Python throttle is an instance of a Go one). So the
|
||||
# arm also proposes the reference of a canon idea that reaches the shape's
|
||||
# project through a shared platform — any language, any project — under a
|
||||
# stricter rule of its own, calibrated on cross-project pairs:
|
||||
#
|
||||
# true pairs A py→go 0.728 (next 0.703) B py→go 0.715 (next 0.675)
|
||||
# D kt→kt 0.816 (next 0.734) E go→go 0.610 (next 0.604)
|
||||
@@ -1690,16 +1671,14 @@ _FAMILY_FLOOR = 0.70
|
||||
|
||||
|
||||
def _semantic_priority(row) -> tuple:
|
||||
"""Order for the capped semantic pass: new shapes first (#4208, measured
|
||||
on the first live refresh after the proposer version was bumped).
|
||||
"""Order for the capped semantic pass: new shapes first.
|
||||
|
||||
The pass is capped (`_SEMANTIC_CAP`), and `flag_divergence` acts only on
|
||||
shapes NEW since the previous refresh — and skips one the proposer matched
|
||||
to the dominant canon. In plain row order the new shapes are the highest
|
||||
ids, the back of the queue, so whenever the backlog exceeds the cap (every
|
||||
`_PROPOSER_VERSION` bump queues the whole todo at once) they were flagged
|
||||
before the arm could propose for them: 1,533 rows queued, 150 checked, and
|
||||
every new shape left unread.
|
||||
`_PROPOSER_VERSION` bump queues the whole todo at once) they would be
|
||||
flagged before the arm could propose for them.
|
||||
|
||||
So: the human todo (`unclassified`) before `scoped`, newest first within
|
||||
each. A row with no creation time sorts last in its group.
|
||||
@@ -1707,27 +1686,19 @@ def _semantic_priority(row) -> tuple:
|
||||
created = row.created_at.timestamp() if row.created_at is not None else float("-inf")
|
||||
return (row.status != "unclassified", -created)
|
||||
# Bump when a basis's rule changes: rows remember the (body, ruleset) they
|
||||
# were examined under, so a tightened rule re-examines everything once.
|
||||
# v3: language-family gate on the sym bases, reference stoplist, semantic
|
||||
# restricted to the shape's own project (#2871).
|
||||
# v4: the semantic arm recorded its misses too (#4208). Retired — a miss is
|
||||
# not evidence (see _SEMANTIC_LIMIT) — but the proposals themselves did not
|
||||
# change, so no re-examination is owed; a stale miss basis is inert.
|
||||
# v5: the semantic arm's floor dropped from 0.8 to the write-path floor (#4208),
|
||||
# and the signature floor from 0.8 to 0.75 (#4306), so rows examined and
|
||||
# found nothing for must be read once more.
|
||||
# v6: the family arm (milestone 463 step 5) — a canon idea's reference on a
|
||||
# shared platform, any language, as the top hit at _FAMILY_FLOOR.
|
||||
# were examined under, so a tightened rule re-examines everything once. A
|
||||
# change that does not alter what is proposed needs no bump. A stale "miss"
|
||||
# basis on an old row is inert: a miss is not evidence (see _SEMANTIC_LIMIT).
|
||||
_PROPOSER_VERSION = 6
|
||||
# Signature resemblance floor, name blanked (difflib ratio) — and a length
|
||||
# floor, because `def NAME():` resembles `def NAME(x):` at 0.95 while saying
|
||||
# nothing; a family shape has parameters to resemble.
|
||||
#
|
||||
# A proposal is read before it is confirmed, so this admits the band where
|
||||
# true instances live rather than the band where nothing is wrong (#4306).
|
||||
# Measured 2026-09-22 over 459 judged instance rows against every same-family
|
||||
# canon: true instances score 1.0 or ~0.795 (a model class whose mixin list
|
||||
# differs from the canon's), and 0.8 cut the whole second band. At 0.75 all
|
||||
# true instances live rather than the band where nothing is wrong. Over
|
||||
# judged instance rows against every same-family canon, true instances score
|
||||
# 1.0 or ~0.795 (a model class whose mixin list differs from the canon's), and
|
||||
# 0.8 cuts the whole second band. At 0.75 all
|
||||
# of them clear; the price is `async def NAME(user_id: int, …) -> dict`
|
||||
# look-alikes (35 wrong-canon pairings vs 14 at 0.8), which read as wrong at
|
||||
# a glance. Below 0.75 the next true row is at 0.6 behind 150+ wrong ones.
|
||||
@@ -1749,12 +1720,11 @@ class Canon(NamedTuple):
|
||||
language: str = "" # the snippet's recorded language; "" = unknown, no gate
|
||||
|
||||
|
||||
# Language families: the sym bases only propose within one. The 2026-08
|
||||
# audit (#2871) found every cross-language hit wrong — a Python tool-module
|
||||
# canon named `register` proposed for Vue `handleSubmit`s that call
|
||||
# `authStore.register()`, and a TS store's `register` matched it by symbol;
|
||||
# Minstrel/Forge TS canon proposed for Python bodies by resemblance. CSS is
|
||||
# its own kind and is not gated here.
|
||||
# Language families: the sym bases only propose within one. Cross-language
|
||||
# hits are wrong — a Python tool-module canon named `register` would be
|
||||
# proposed for Vue `handleSubmit`s that call `authStore.register()`, a TS
|
||||
# store's `register` would match it by symbol, and TS canon would be proposed
|
||||
# for Python bodies by resemblance. CSS is its own kind and is not gated here.
|
||||
_FAMILY_BY_LANG = {
|
||||
"python": "py", "py": "py",
|
||||
"typescript": "js", "ts": "js", "tsx": "js", "javascript": "js", "js": "js",
|
||||
@@ -1796,8 +1766,7 @@ def same_family(path: str, canon_language: str) -> bool:
|
||||
|
||||
# Reference basis: generic verbs name too many unrelated things to count a
|
||||
# bare mention as a call site of THIS canon (`register`, `load`, `save` …).
|
||||
# The symbol basis still catches a second definition of such a name; the
|
||||
# call-site relation for these becomes a `uses` edge once #2870 lands.
|
||||
# The symbol basis still catches a second definition of such a name.
|
||||
_REFERENCE_STOPLIST = frozenset({
|
||||
"get", "set", "put", "post", "load", "save", "run", "main", "init", "setup",
|
||||
"register", "restore", "reset", "toggle", "close", "open", "submit", "handler",
|
||||
@@ -1836,7 +1805,7 @@ def text_contains(body: str, code: str) -> bool:
|
||||
|
||||
|
||||
def reference_canons(kind: str, path: str, symbol: str, body: str, canons: Iterable[Canon]) -> list[int]:
|
||||
"""Every canon this body NAMES (#2870) — the uses edges the proposer can
|
||||
"""Every canon this body NAMES — the uses edges the proposer can
|
||||
write mechanically: same kind, same language family, symbol not in the
|
||||
generic-verb stoplist, and not the shape's own name."""
|
||||
norm_sym = _norm_symbol(symbol)
|
||||
@@ -1903,7 +1872,7 @@ def match_canon(
|
||||
|
||||
async def canon_catalog(user_id: int) -> list[Canon]:
|
||||
"""Every snippet this user can browse, as matchable canon — own projects
|
||||
and shared ones alike, because family canon counts (note 2786)."""
|
||||
and shared ones alike, because family canon counts."""
|
||||
from scribe.models.note import Note
|
||||
from scribe.services.access import browsable_notes_clause
|
||||
from scribe.services.coverage import extract_definitions
|
||||
@@ -2022,10 +1991,9 @@ async def propose_for_repo(
|
||||
if canons is None:
|
||||
canons = await canon_catalog(user_id)
|
||||
by_key = {(d[0], d[1], d[2]): d for d in definitions}
|
||||
# The semantic arm is the widest net and, across projects, was pure noise
|
||||
# in the 2026-08 audit (#2871): it is held to the shape's own project and
|
||||
# language family. The precise bases (symbol/text) still reach family
|
||||
# canon in other projects (note 2786).
|
||||
# The semantic arm is the widest net and, across projects, is noise: it
|
||||
# is held to the shape's own project and language family. The precise
|
||||
# bases (symbol/text) still reach family canon in other projects.
|
||||
sym_canons = [c for c in canons if c.kind == "sym" and c.project_id == project_id]
|
||||
|
||||
def semantic_allowed(path: str) -> set[int]:
|
||||
@@ -2062,7 +2030,7 @@ async def propose_for_repo(
|
||||
if row.proposed_at is not None and row.proposed_sha == examined_as:
|
||||
continue
|
||||
if row.status not in _MECHANICAL_TODO:
|
||||
# A judged row gets no proposal — but its uses edges (#2870)
|
||||
# A judged row gets no proposal — but its uses edges
|
||||
# are a fact about the body, judged or not: the consumer map
|
||||
# of a canon must include the call sites someone already
|
||||
# classified. Mark it examined so the scan runs once per body.
|
||||
@@ -2083,7 +2051,7 @@ async def propose_for_repo(
|
||||
row.proposal_group = group
|
||||
row.proposed_at = now
|
||||
row.proposed_sha = examined_as
|
||||
# Consumption is recorded for every canon the body names (#2870),
|
||||
# Consumption is recorded for every canon the body names,
|
||||
# whatever the row is then judged to be.
|
||||
used = reference_canons(row.kind, row.path, row.symbol, body, canons)
|
||||
if used:
|
||||
@@ -2126,7 +2094,7 @@ def derive_groups(
|
||||
(kind `sym`) identical bodies in ≥2 places group as `dup:<sha>` and the
|
||||
same name defined in ≥3 files groups as `name:sym:<symbol>`, the copy
|
||||
before the name. CSS groups by name only — the same class defined in
|
||||
≥2 files is `name:css:<symbol>`; its body never groups it (note 2917)."""
|
||||
≥2 files is `name:css:<symbol>`; its body never groups it."""
|
||||
by_sha: dict[str, list[tuple[str, str, str]]] = {}
|
||||
by_name: dict[tuple[str, str], list[tuple[str, str, str]]] = {}
|
||||
for path, kind, symbol, sha in rows:
|
||||
@@ -2233,7 +2201,7 @@ def proposal_summary(
|
||||
g["files"] = len(files[key])
|
||||
if key in consumers:
|
||||
g["consumers"] = consumer_summary(consumers[key])
|
||||
# Body-identical groups first (#2872): the things an audit actually
|
||||
# Body-identical groups first: the things an audit actually
|
||||
# consolidated were identical bodies under different names/files; a
|
||||
# name repeated across modules is usually convention. Within a tier,
|
||||
# the group spread over more files is the bigger copy.
|
||||
@@ -2251,7 +2219,7 @@ def proposal_summary(
|
||||
def derive_new_summary(
|
||||
rows: Iterable[CodeShape], *, since: datetime | None, top: int = 3
|
||||
) -> dict:
|
||||
"""The arrival-moment drift signal (#2899): derive-grouped rows FIRST
|
||||
"""The arrival-moment drift signal: derive-grouped rows FIRST
|
||||
SEEN after ``since`` — the previous refresh's stamp, the same one
|
||||
flag_divergence uses. "Since the last refresh, N more copies joined a
|
||||
duplicate family" is the sentence that makes the derive queue a thing
|
||||
@@ -2341,7 +2309,7 @@ async def confirm_proposals(
|
||||
return out
|
||||
|
||||
|
||||
# --- the divergence readout (#2793): button B where button A is canon -------
|
||||
# --- the divergence readout: button B where button A is canon --------------
|
||||
#
|
||||
# Three answers the ledger can now give mechanically:
|
||||
# DIVERGENCE a shape NEW since the previous refresh, in a directory+kind
|
||||
@@ -2355,27 +2323,21 @@ async def confirm_proposals(
|
||||
# RECHECK an instance/variant whose body moved since it was judged is
|
||||
# flagged recheck_at (sync) — the judgment stands, re-confirm it.
|
||||
|
||||
# A canon dominates a directory+kind when at least this many siblings are
|
||||
# judged (canonical/instance) and this share of them answer to one snippet.
|
||||
# How much a payload must resemble a canon before the hook SUGGESTS it as the
|
||||
# shape's canon — and, for the rows stamped before milestone 439, the bar below
|
||||
# which a stored stamp is weak (`is_weak_stamp`). Until milestone 439 this was
|
||||
# the bar for the hook to ASSERT, unattended, that a shape IS an instance; the
|
||||
# hook no longer asserts anything, and the history below is why.
|
||||
# shape's canon — and the bar below which a stored hook stamp (an `instance`
|
||||
# row written unattended, classified_by="hook") is weak (`is_weak_stamp`).
|
||||
# The hook itself never asserts a classification.
|
||||
#
|
||||
# WHY A FLOOR AT ALL. There was none: any score the semantic arm produced
|
||||
# counted, so 0.69 asserted as confidently as 0.95, and the score went into
|
||||
# the reason line without ever being compared to anything. Those rows are
|
||||
# permanent — they feed `dominant_canon`, which then tells the next writer in
|
||||
# that directory what to build from — so a thin stamp does not stay thin, it
|
||||
# compounds.
|
||||
# WHY A FLOOR AT ALL. Without one, any score the semantic arm produced would
|
||||
# count, 0.69 as confidently as 0.95. A stamp feeds `dominant_canon`, which
|
||||
# tells the next writer in that directory what to build from, so a thin stamp
|
||||
# does not stay thin, it compounds.
|
||||
#
|
||||
# 0.80 rather than the retrieval floors near 0.70, deliberately. Those bars
|
||||
# decide whether to SHOW someone a record, where being wrong costs a glance.
|
||||
# This one decides whether to RECORD a claim about the codebase unattended,
|
||||
# where being wrong costs a wrong instruction to everyone who writes in that
|
||||
# directory afterwards. An unattended write should need more evidence than a
|
||||
# suggestion, not the same.
|
||||
# This one weighs a claim about the codebase, where being wrong costs a wrong
|
||||
# instruction to everyone who writes in that directory afterwards. A claim
|
||||
# should need more evidence than a suggestion to read, not the same.
|
||||
_RESEMBLE_MIN = 0.80
|
||||
|
||||
|
||||
@@ -2384,12 +2346,11 @@ def is_weak_stamp(row) -> bool:
|
||||
the floor it would have to clear today?
|
||||
|
||||
ONE PREDICATE FOR "WEAK", read by the review surface that lists these rows
|
||||
and by every reader that would otherwise take them as evidence. The floor
|
||||
guards new writes only; the rows stamped before it (#4204) are still
|
||||
stored, and until an agent re-judges them they must not speak. They are
|
||||
not withdrawn here — `stamps_to_review` still lists them for judgment —
|
||||
they are simply not counted, which is the module's rule for evidence it
|
||||
cannot trust: quieter, never more confident (#4608).
|
||||
and by every reader that would otherwise take them as evidence. Rows
|
||||
stamped below the floor stay stored, and until an agent re-judges them
|
||||
they must not speak. They are not withdrawn here — `stamps_to_review`
|
||||
lists them for judgment — they are simply not counted, which is the
|
||||
module's rule for evidence it cannot trust: quieter, never more confident.
|
||||
"""
|
||||
if getattr(row, "classified_by", None) not in _UNATTENDED_BY:
|
||||
return False
|
||||
@@ -2397,6 +2358,8 @@ def is_weak_stamp(row) -> bool:
|
||||
return score is not None and score < _RESEMBLE_MIN
|
||||
|
||||
|
||||
# A canon dominates a directory+kind when at least this many siblings are
|
||||
# judged (canonical/instance) and this share of them answer to one snippet.
|
||||
_DENSITY_MIN_JUDGED = 3
|
||||
_DENSITY_SHARE = 0.6
|
||||
|
||||
@@ -2436,9 +2399,9 @@ def dominant_canon(rows: Iterable[CodeShape]) -> tuple[int, int, int] | None:
|
||||
counts: dict[int, int] = {}
|
||||
judged = 0
|
||||
for r in rows:
|
||||
# A weak stamp is not a judgment: counted, it let a YAML CI snippet
|
||||
# "dominate" a Svelte directory and every write there was told it
|
||||
# diverged from it (#4608).
|
||||
# A weak stamp is not a judgment: counted, it could let a YAML CI
|
||||
# snippet "dominate" a Svelte directory, and every write there would
|
||||
# be told it diverged from it.
|
||||
if is_weak_stamp(r):
|
||||
continue
|
||||
if r.status in ("canonical", "instance") and r.snippet_id is not None:
|
||||
@@ -2466,12 +2429,11 @@ async def canon_density(
|
||||
`shape_form` returns it, NOT a family. `families_conflict` coarsens both
|
||||
sides itself, and handing it a family makes `shape_family("callable")`
|
||||
return "" so nothing is ever excluded: the narrowing silently becomes a
|
||||
no-op that still reads as applied. It narrows the DENOMINATOR
|
||||
(#4208). Without it the base rate was computed over every code symbol in
|
||||
the directory as one bucket: "372 judged siblings" counted a dataclass, a
|
||||
CSS-less constant and an async service unit as three comparable things,
|
||||
and the share that came out of that was a statement about a population
|
||||
nobody had asked a question about.
|
||||
no-op that still reads as applied. It narrows the DENOMINATOR. Without
|
||||
it the base rate is computed over every code symbol in the directory as
|
||||
one bucket, counting a dataclass, a constant and an async service unit as
|
||||
three comparable things, and the share that comes out is a statement about
|
||||
a population nobody asked a question about.
|
||||
|
||||
EXCLUDES ONLY A KNOWN CONFLICT, using `families_conflict` — the same
|
||||
predicate the divergence gate uses, so the two cannot drift apart. A
|
||||
@@ -2482,10 +2444,10 @@ async def canon_density(
|
||||
this module goes the same way — quieter, never more confident.
|
||||
|
||||
NOT narrowed to the candidate's exact form, for the reason `shape_family`
|
||||
gives at length: `fn` beside `async-fn` is the acceptance case of
|
||||
milestone #2793, not noise. Bucketing the denominator by form would take
|
||||
the async canon out of a sync candidate's count and silence that flag —
|
||||
the same inversion the first form gate made, one layer down.
|
||||
gives at length: `fn` beside `async-fn` is a fair question, not noise.
|
||||
Bucketing the denominator by form would take the async canon out of a
|
||||
sync candidate's count and silence that flag — the inversion a form gate
|
||||
makes, one layer down.
|
||||
"""
|
||||
directory = _dir_of(path)
|
||||
async with async_session() as session:
|
||||
@@ -2508,8 +2470,8 @@ async def canon_density(
|
||||
# The canon's form travels with the count, read from its OWN judged rows.
|
||||
# `dominant_canon` keeps its three-value shape: it answers "what dominates
|
||||
# here", which is still a true and separately useful question, and its
|
||||
# test pins that arithmetic. What changed is that nobody acts on the count
|
||||
# alone any more.
|
||||
# test pins that arithmetic. Callers act on the count together with the
|
||||
# canon's form, never on the count alone.
|
||||
return dom[0], dom[1], dom[2], canon_form(siblings, dom[0])
|
||||
|
||||
|
||||
@@ -2526,13 +2488,13 @@ async def write_time_divergence(
|
||||
out: list[dict] = []
|
||||
if not shapes:
|
||||
return out
|
||||
# DENSITY IS NOW PER CANDIDATE, not per kind (#4208): the denominator
|
||||
# DENSITY IS PER CANDIDATE, not per kind: the denominator
|
||||
# excludes siblings whose family contradicts what is being written, so it
|
||||
# cannot be computed until the candidate's own form is known. Cached on
|
||||
# (kind, family) — a write names a handful of shapes and they collapse to
|
||||
# one or two buckets, so this is the same one-or-two queries as before.
|
||||
# one or two buckets, so this stays one or two queries.
|
||||
#
|
||||
# The cost is that the row load below no longer sits behind an early exit
|
||||
# The cost is that the row load below does not sit behind an early exit
|
||||
# on "nothing is dense here". That is one indexed lookup on
|
||||
# (project_id, path), and it has to happen first regardless: the
|
||||
# candidate's signature comes from its stored row when the payload does
|
||||
@@ -2575,7 +2537,7 @@ async def write_time_divergence(
|
||||
row.status != "unclassified" or row.proposed_snippet_id == sid
|
||||
):
|
||||
continue
|
||||
# IS THIS EVEN THE SAME CATEGORY OF THING AS THE CANON? (#4204)
|
||||
# IS THIS EVEN THE SAME CATEGORY OF THING AS THE CANON?
|
||||
# Without this the line was a pure base rate: "most things here are X,
|
||||
# so be X", with the candidate never examined at all. It told a frozen
|
||||
# dataclass to build from the `async_session` service canon.
|
||||
@@ -2603,13 +2565,13 @@ _DERIVE_FILES_SHOWN = 4
|
||||
async def write_time_derive(
|
||||
project_id: int, path: str, shapes: list[tuple[str, str]]
|
||||
) -> list[dict]:
|
||||
"""The in-band DERIVE check (#2900): for each (kind, name) the hook
|
||||
"""The in-band DERIVE check: for each (kind, name) the hook
|
||||
named at ``path``, what the ledger already knows about that name
|
||||
elsewhere in the project —
|
||||
|
||||
family the name sits in a derive-first group (code: identical body
|
||||
in N files or the same name in ≥3; CSS: the same class in
|
||||
≥2 files, note 2917): "this is a known family with no canon
|
||||
≥2 files): "this is a known family with no canon
|
||||
— derive it now, don't add a copy";
|
||||
canon a `canonical` row of that name at another path: "this is
|
||||
canon #N at <path> — reuse, don't redefine".
|
||||
@@ -2660,7 +2622,7 @@ async def write_time_derive(
|
||||
"size": len(members) + (1 if here is not None else 0),
|
||||
}
|
||||
if kind == "css":
|
||||
# What renders the family (milestone 302): the members' consumer
|
||||
# What renders the family: the members' consumer
|
||||
# files, the row at `path` included when it already exists.
|
||||
ids = [r.id for r in members] + ([here.id] if here is not None else [])
|
||||
edges = await consumers_of(ids)
|
||||
@@ -2702,10 +2664,10 @@ async def flag_divergence(project_id: int, *, since: datetime | None) -> int:
|
||||
if r.diverges_from is not None:
|
||||
# A flag is a mechanical prompt resting on one premise —
|
||||
# "this canon dominates here". When the premise no longer
|
||||
# holds (the rows that elected it were weak stamps, #4608,
|
||||
# or were re-judged away) the prompt is false, and a false
|
||||
# prompt on every write is how the flag stopped being
|
||||
# read. Withdrawing it judges nothing: the row stays
|
||||
# holds (the rows that elected it were weak stamps, or
|
||||
# were re-judged away) the prompt is false, and a false
|
||||
# prompt on every write is a flag nobody reads.
|
||||
# Withdrawing it judges nothing: the row stays
|
||||
# unclassified, and a judgment is never touched here.
|
||||
if dom is None or dom[0] != r.diverges_from:
|
||||
r.diverges_from = None
|
||||
@@ -2716,16 +2678,16 @@ async def flag_divergence(project_id: int, *, since: datetime | None) -> int:
|
||||
continue
|
||||
if r.proposed_snippet_id == dom[0]:
|
||||
continue # the proposer already says "instance of the canon"
|
||||
# NOT the converse (#4208). A semantic MISS does not mean "not
|
||||
# the canon": measured, true instances of a canon rarely clear
|
||||
# the arm's floor either (see the note above _SEMANTIC_LIMIT).
|
||||
# A helper beside a canon and a genuine divergence — #2793's
|
||||
# sync `confirmDanger` beside an async confirm canon — are the
|
||||
# same case to every signal this ledger holds, so the prompt
|
||||
# asks, and the judge answers it (`exempt` with a reason
|
||||
# retires a false one for good).
|
||||
# The same structural test the write-time check applies
|
||||
# (#4204). The sweep and the hook must agree about what counts
|
||||
# NOT the converse. A semantic MISS does not mean "not the
|
||||
# canon": true instances of a canon often miss the arm's floor
|
||||
# too (see the note above _SEMANTIC_LIMIT). A helper beside a
|
||||
# canon and a genuine divergence — a sync `confirmDanger`
|
||||
# beside an async confirm canon — are the same case to every
|
||||
# signal this ledger holds, so the prompt asks, and the judge
|
||||
# answers it (`exempt` with a reason retires a false one for
|
||||
# good).
|
||||
# The same structural test the write-time check applies. The
|
||||
# sweep and the hook must agree about what counts
|
||||
# as divergence, or an audit contradicts the line the writer
|
||||
# was shown at the keyboard.
|
||||
if families_conflict(shape_form(r.signature or "", r.kind), cform):
|
||||
@@ -2736,7 +2698,7 @@ async def flag_divergence(project_id: int, *, since: datetime | None) -> int:
|
||||
return flagged
|
||||
|
||||
|
||||
# --- the end-of-turn question (milestone 439): what did this turn write that
|
||||
# --- the end-of-turn question: what did this turn write that
|
||||
# nobody has judged? ---------------------------------------------------------
|
||||
#
|
||||
# The write hooks keep a session ledger of the definitions each turn wrote; at
|
||||
@@ -2822,21 +2784,21 @@ def stamps_to_review(rows: Iterable[CodeShape], *, top: int = 10) -> dict:
|
||||
"""Judged rows whose EVIDENCE no longer meets the bar the ledger now
|
||||
holds, and canons whose own rows no longer agree what they are.
|
||||
|
||||
THIS CHANGES NOTHING, AND THAT IS THE POINT. Every row here was written
|
||||
unattended by the write-path hook on a similarity score, and the damage
|
||||
that caused was not the score being wrong once — it was a machine
|
||||
asserting a permanent classification with nobody reading it. A sweep that
|
||||
silently un-asserted them would be the identical mistake with a larger
|
||||
blast radius and a cleaner conscience. So this reports; an agent reads the
|
||||
THIS CHANGES NOTHING, AND THAT IS THE POINT. Every `weak` row was written
|
||||
unattended by the write-path hook on a similarity score, and the harm in
|
||||
that is not a score being wrong once — it is a machine asserting a
|
||||
permanent classification with nobody reading it. A sweep that silently
|
||||
un-asserted them would be the identical mistake with a larger blast
|
||||
radius and a cleaner conscience. So this reports; an agent reads the
|
||||
evidence, decides, and records the decision through `classify_shapes`
|
||||
under its own name. The ledger should be able to say "these look wrong"
|
||||
without being able to act on it alone.
|
||||
|
||||
Two lists, because they are different questions:
|
||||
|
||||
`weak` — rows stamped on a resemblance below `_RESEMBLE_MIN`. The floor
|
||||
arrived after they did (#4204), and a guard at the point of classification
|
||||
does not undo the classifications already stored (lesson #4202). Rows a
|
||||
`weak` — rows stamped on a resemblance below `_RESEMBLE_MIN`. A guard at
|
||||
the point of classification does not undo classifications already stored,
|
||||
so rows written before the floor need reading. Rows a
|
||||
person or an audit judged are never listed, however old: a human judgment
|
||||
is not weak evidence, it is a different kind of evidence.
|
||||
|
||||
@@ -2845,14 +2807,13 @@ def stamps_to_review(rows: Iterable[CodeShape], *, top: int = 10) -> dict:
|
||||
thing; when its own members are a class, three getters and a dozen tests,
|
||||
that claim has stopped being true, and every base-rate reading built on
|
||||
it is reading noise. `canon_form` already makes such a canon fall silent
|
||||
— this is what makes it VISIBLE, which is the half that was missing.
|
||||
— this is what makes it VISIBLE.
|
||||
|
||||
What counts as disagreement is `canon_coherence`, and it is deliberately
|
||||
looser than "one form": it compares FAMILIES, and it does not hold a
|
||||
method against the class it is defined beside. Both corrections came from
|
||||
this surface's first live day, when the only two canons it reported were
|
||||
both sound (#4220). A review surface whose output is noise is one that
|
||||
stops being read, which costs more than the check was ever worth.
|
||||
method against the class it is defined beside. A review surface whose
|
||||
output is noise stops being read, which costs more than the check is
|
||||
worth.
|
||||
|
||||
`strangers` names the rows that do not fit, not the first dozen members:
|
||||
the reader's question is which ones are wrong. `unattended` counts rows
|
||||
|
||||
Reference in New Issue
Block a user