Compare commits

...
2 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 57773c8b55 docs(6205): snippets comments state the reason, not the instance that taught it
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m8s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 22s
Issue, decision, milestone and rule citations are removed; the snippet-mirror
docstring drops the paragraph about its own earlier wording, and the
pointer-model and freshness comments state the design rather than the plan
step that introduced it. Access notes point at services/access.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:14:09 -04:00
bvandeusenandClaude Opus 5.5 5598b307fa docs(6205): coverage comments state the reason, not the instance that taught it
Issue, note and milestone citations are removed along with first-deploy
counts and the measured phantom-definition story; "button B" jargon becomes
the divergence it names. Each block keeps its present-tense reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:12:53 -04:00
2 changed files with 148 additions and 158 deletions
+82 -84
View File
@@ -1,5 +1,5 @@
"""Pattern-library coverage — what fraction of a bound repo's shapes have a
recorded snippet (#2692, forge job 3 of decision #2686).
recorded snippet.
The all-shapes doctrine says every shape gets recorded at first build. This
module is the hoping→knowing move: it enumerates the definitions that exist in
@@ -43,9 +43,9 @@ logger = logging.getLogger(__name__)
# Cache key in the settings KV, on the project OWNER's user_id — the same
# channel the scheduler's last-run summary uses for machine-written state.
# v2 suffix with #2788: the payload shape inverted (accounted/unclassified);
# pre-ledger blobs under the old key simply stop being found, so the card
# honestly reads "not measured yet" until the first ledger-era refresh.
# The v2 suffix marks the accounted/unclassified payload shape; a blob under
# an older key is simply not found, so the card honestly reads "not measured
# yet" until the first refresh writes the current shape.
_CACHE_KEY_PREFIX = "pattern_coverage_v2_"
# Files whose content can't hold definitions — the hook's skip list, verbatim,
@@ -93,18 +93,15 @@ _ARROW_RE = re.compile(
)
# --- comment and string spans (#4222) ----------------------------------------
# --- comment and string spans -----------------------------------------------
#
# The matchers above are line-oriented and know nothing about what a line is
# INSIDE. A wrapped docstring whose line happens to begin "class AND the …"
# reads as a definition of a shape called `AND`, and that phantom reaches the
# agent mid-edit as a divergence prompt about a symbol that does not exist.
# This repo's own source minted twenty-two of them, measured by running the
# extractor over every scannable file with and without this scan. Two —
# `with` and `nobody`, out of one module docstring in
# scripts/check_dangling_styles.py — are persisted, JUDGED `code_shapes`
# rows, so the cost was never only noise in the moment. Those clear
# themselves: sync_shapes marks a row it no longer extracts as vanished.
# agent mid-edit as a divergence prompt about a symbol that does not exist —
# and, once judged, persists as a `code_shapes` row. Masking comment and
# string spans stops the phantoms; any already stored clear themselves, since
# sync_shapes marks a row it no longer extracts as vanished.
#
# The honest tool for .py would be `ast`, which cannot be fooled by prose at
# all. It is not what this uses, because this extractor is mirrored rule for
@@ -221,7 +218,7 @@ def _masked_lines(text: str, count: int) -> list[str]:
class Definition(NamedTuple):
"""One extracted definition with its content fingerprint (#2792).
"""One extracted definition with its content fingerprint.
`signature` is the definition line itself; `body_sha` hashes the block
whitespace- and comment-insensitively; `body` is the block's text, held
@@ -234,7 +231,7 @@ class Definition(NamedTuple):
signature: str
body_sha: str
body: str
line: int = -1 # 0-based line the definition starts on (#2869)
line: int = -1 # 0-based line the definition starts on
def _definition_on(raw: str) -> tuple[str, str] | None:
@@ -253,7 +250,7 @@ def _definition_on(raw: str) -> tuple[str, str] | None:
# `type` announces a definition only when something is declared after
# the name (`type Foo = …`, `type Foo struct {`); an import specifier
# (`import { type Foo, bar }`) is the same two words and defines
# nothing — it showed up as a two-file "identical body" family (#2904).
# nothing, and would read as a two-file "identical body" family.
if line.startswith("type") and not re.search(r"[={]", line[m.end():]):
return None
return ("sym", name)
@@ -325,10 +322,10 @@ def extract_definitions(text: str) -> list[Definition]:
end = j
break
block = lines[i:end]
# A CSS rule's fingerprint is its DECLARATIONS, not its selector
# (#2872): the row's identity already carries the selector. Since
# note 2917 the derive grouping no longer reads CSS bodies at all (a
# class is grouped by name only), so for CSS the fingerprint is the
# A CSS rule's fingerprint is its DECLARATIONS, not its selector:
# the row's identity already carries the selector. The derive
# grouping does not read CSS bodies at all (a class is grouped by
# name only), so for CSS the fingerprint is the
# recheck identity — "did this rule's body change since it was
# judged?" — and nothing more. The shape of the hash is kept as-is on
# purpose: changing it would flip every judged CSS row to recheck on
@@ -336,20 +333,20 @@ def extract_definitions(text: str) -> list[Definition]:
if kind == "css":
# One-line rules (`.x { color: red; }`) carry their declarations on
# the selector line itself; a block that is only the selector plus
# trailing blanks must not hash to the empty string (which grouped
# 68 unrelated one-liners as one "copy" on first deploy, #2872).
# trailing blanks must not hash to the empty string, which would
# group every unrelated one-liner as one "copy".
first = lines[i]
brace = first.find("{")
head = [first[brace + 1:]] if brace >= 0 and first[brace + 1:].strip() else []
hashed = head + block[1:]
if not any(x.strip() for x in hashed):
hashed = block
# A SINGLE declaration is not a shape (#2903): `color: var(--fs-
# A SINGLE declaration is not a shape: `color: var(--fs-
# text-tertiary)` under .text-muted, .task-mark and .pin-badge-auto
# is three meanings sharing one line, not three copies of one
# rule. Keep the selector in the hash for one-liners; two
# declarations and up stay selector-agnostic. (Moot for grouping
# since note 2917, kept for fingerprint stability — see above.)
# declarations and up stay selector-agnostic. (Moot for grouping,
# kept for fingerprint stability — see above.)
elif _declaration_count(hashed) < 2:
hashed = block
else:
@@ -361,19 +358,20 @@ def extract_definitions(text: str) -> list[Definition]:
return out
# --- by-construction scope (#2869) -------------------------------------------
# --- by-construction scope ---------------------------------------------------
#
# A Vue single-file component's `<style scoped>` rules and its `<script setup>`
# functions cannot be reached from any other file: they are one-offs by
# construction, not by judgment. The sync stamps them `scoped` (mechanical) so
# the human todo holds only shapes a person should look at, while the bodies
# stay in play for the proposer, derive grouping and divergence — the five
# auth views' identical rules were found exactly there. Unscoped `<style>` in
# stay in play for the proposer, derive grouping and divergence — identical
# rules copied across sibling components are found exactly there. Unscoped
# `<style>` in
# a .vue and every file that is neither .vue nor .svelte stay ordinary.
_STYLE_OPEN_RE = re.compile(r"^\s*<style\b[^>]*\bscoped\b", re.IGNORECASE)
_STYLE_CLOSE_RE = re.compile(r"^\s*</style\s*>", re.IGNORECASE)
# Svelte scopes the other way round (#4608): every `<style>` is component-
# Svelte scopes the other way round: every `<style>` is component-
# scoped unless it opts out — `<style global>` (svelte-preprocess) for a whole
# block, `:global(...)` for one selector. Instance `<script>` code is the
# component's own; `<script context="module">` / `<script module>` is the one
@@ -443,12 +441,12 @@ def scoped_definitions(path: str, text: str, defs: list[Definition]) -> set[tupl
return out
# --- template class references: the CSS consumer map (milestone 302) ---------
# --- template class references: the CSS consumer map -------------------------
# Files whose MARKUP can consume a class. Styling consumers are templates —
# `querySelector('.x')` / classList in scripts are deliberately not read in
# v1 (note 2917: watch CSS by name, by recipe, by token and by what uses it;
# "what uses it" is the template).
# `querySelector('.x')` / classList in scripts are deliberately not read: CSS
# is watched by name, by recipe, by token and by what uses it, and "what uses
# it" is the template.
_TEMPLATE_SUFFIXES = (
".vue", ".html", ".htm", ".jsx", ".tsx", ".js", ".ts", ".svelte", ".astro",
)
@@ -473,7 +471,7 @@ _SVELTE_CLASS_RE = re.compile(r"(?<![:\w.-])class:([A-Za-z_][A-Za-z0-9_-]*)\s*="
# runtime, and React's <CSSTransition classNames="fade"> does the same. A
# reader of `class=` attributes alone therefore calls every one of those
# rules unused, which is a false positive no amount of care in the
# stylesheet can avoid (#2970). A dynamic `:name="…"` stays unknowable.
# stylesheet can avoid. A dynamic `:name="…"` stays unknowable.
_TRANSITION_NAME_RE = re.compile(
r"""<\s*[Tt]ransition(?:-[Gg]roup|Group)?\b[^>]*?(?<![:\w.-])name\s*=\s*"""
r"""(?:"([^"]*)"|'([^']*)')"""
@@ -552,8 +550,8 @@ def class_references(path: str, text: str) -> dict[str, int]:
`class:x` directive; never a CSS selector (`.x {` is a definition, read
by extract_definitions) and never a script's `querySelector('.x')`.
Two forms name classes without spelling them out, and both are read
(#2970): a transition `name=` stands for every class the framework
Two forms name classes without spelling them out, and both are read:
a transition `name=` stands for every class the framework
generates from it, and a concatenated name contributes the prefix
reference `head-*` — which resolve_consumers matches against every row
whose symbol starts with `head-`."""
@@ -597,7 +595,7 @@ def scannable(path: str) -> bool:
return not path.lower().endswith(_SKIP_SUFFIXES)
# --- the file as a unit (milestone 439) ---------------------------------------
# --- the file as a unit ------------------------------------------------------
#
# A component is the thing people reuse in a UI, and in a single-file
# component nothing inside is named after it: CatalogueBookCard.svelte defines
@@ -644,7 +642,7 @@ class ArchiveShape(NamedTuple):
signature: str
body_sha: str
body: str
scoped: bool = False # one-off by construction (#2869)
scoped: bool = False # one-off by construction
def shapes_from_archive(blob: bytes) -> list[tuple[str, str, str]]:
@@ -656,11 +654,11 @@ def shapes_from_archive(blob: bytes) -> list[tuple[str, str, str]]:
class ArchiveScan(NamedTuple):
"""One walk of a repo tarball: what each file DEFINES (the ledger rows)
and which class names each file's markup REFERENCES (the CSS consumer
map, milestone 302) — read together because the bodies are in hand once."""
map) — read together because the bodies are in hand once."""
definitions: list[ArchiveShape]
references: dict[str, dict[str, int]] # path → class token → count
# EVERY file's repo-relative path, scannable or not (milestone 463). The
# EVERY file's repo-relative path, scannable or not. The
# platform markers are files this scan otherwise skips — go.mod,
# AndroidManifest.xml, a Dockerfile — so detection reads the names here
# rather than re-walking the tarball.
@@ -722,9 +720,9 @@ def scan_archive(blob: bytes) -> ArchiveScan:
# --- matching shapes against recorded locations ------------------------------
# The covering predicate (location_covers) lives in services/shape_ledger.py
# since #2788 — the ledger's canonical marking and this module's readout are
# two consumers of ONE doctrine, and the ledger is its home.
# The covering predicate (location_covers) lives in services/shape_ledger.py:
# the ledger's canonical marking and this module's readout are two consumers
# of ONE doctrine, and the ledger is its home.
def largest_gaps(
@@ -756,7 +754,7 @@ async def _recorded_locations(
user_id: int, project_id: int
) -> list[tuple[int, str, str]]:
"""(snippet_note_id, path, symbol) for every location of every live
snippet in a project — the canonical-marking input (#2788): the id is what
snippet in a project — the canonical-marking input: the id is what
lets a ledger row point back at the snippet it references."""
from sqlalchemy import select
@@ -788,15 +786,14 @@ async def compute_coverage(
) -> dict | None:
"""Sync the shape ledger from the bound repos and read the accounting.
Since #2788 this is the ledger's sync point, not just a measurement:
every walk upserts the extracted shapes (new → unclassified, vanished →
stamped), re-stamps snippet reference locations as canonical, and then
reports the ACCOUNTING — how many shapes carry a classification at all —
rather than the old "has a snippet" fraction. Unclassified is the todo
(note 2786).
This is the ledger's sync point, not just a measurement: every walk
upserts the extracted shapes (new → unclassified, vanished → stamped),
re-stamps snippet reference locations as canonical, and then reports the
ACCOUNTING — how many shapes carry a classification at all — rather than
a "has a snippet" fraction. Unclassified is the todo.
None means "nothing to measure" — the owner's keyring serves none of the
project's bound repos (#2778). That is the ordinary state for a
project's bound repos. That is the ordinary state for a
forge-less user and every caller treats it as silence, not failure; the
ledger is untouched in that case. Forge errors (unreachable, bad token)
RAISE — the two callers are a refresh button and a background task, and
@@ -818,7 +815,7 @@ async def compute_coverage(
tree_paths: list[str] = []
recorded = await _recorded_locations(user_id, project_id)
# The proposer's canon catalog, read once per refresh and shared across
# the project's repos (#2792).
# the project's repos.
canons = None
proposer_stats = {"examined": 0, "proposed": 0, "semantic_checked": 0}
for binding in await bindings_for_project(user_id, project_id):
@@ -827,8 +824,8 @@ async def compute_coverage(
if hit is None:
continue # bound to a host no connection serves
forge, api_repo = hit
# The binding's own ref when it names one (#2873: a dev-first project
# has its ledger follow dev), else the forge's default branch.
# The binding's own ref when it names one (a dev-first project has
# its ledger follow dev), else the forge's default branch.
ref = binding.ref or await forge.default_branch(api_repo)
scan = scan_archive(await forge.archive(api_repo, ref))
definitions = scan.definitions
@@ -844,7 +841,7 @@ async def compute_coverage(
project_id, key, definitions, seen_marker=marker
)
served.append((key, ref))
# The CSS consumer map (milestone 302) rides the same archive: which
# The CSS consumer map rides the same archive: which
# files' markup names each class. Mechanical and recomputable, so it
# must not be able to fail the refresh either.
try:
@@ -868,7 +865,7 @@ async def compute_coverage(
if not served:
return None
# Platform detection (milestone 463) rides the same walk: the paths are in
# Platform detection rides the same walk: the paths are in
# hand once. It only ever ADDS membership the project has no answer for,
# and it must not be able to fail the refresh it rides on.
try:
@@ -879,7 +876,7 @@ async def compute_coverage(
logger.warning("platform detection failed for project %s", project_id, exc_info=True)
await shape_ledger.mark_canonicals(project_id, recorded)
# The adoption ledger follows the shapes (milestone 463 step 5). The
# The adoption ledger follows the shapes. The
# judge paths sync the ideas they touch; the refresh is the one place
# that sees canonical stamps land and shapes vanish, so it answers every
# canon idea reaching the project. It must not fail the refresh.
@@ -895,7 +892,7 @@ async def compute_coverage(
logger.warning("derive-first grouping failed", exc_info=True)
# "Since the previous computation" — the cache's stamp. A first seed has
# none, so nothing is new then. Read once; two passes use it: the
# button-B flag (#2793) and the derive-new drift count (#2899).
# divergence flag and the derive-new drift count.
since = None
try:
previous = await get_setting(user_id, f"{_CACHE_KEY_PREFIX}{project_id}")
@@ -904,7 +901,7 @@ async def compute_coverage(
since = datetime.fromisoformat(stamp) if stamp else None
except Exception:
logger.warning("previous coverage stamp unreadable", exc_info=True)
# The button-B pass (#2793): shapes new since the PREVIOUS computation,
# The divergence pass: shapes new since the PREVIOUS computation,
# where a canon dominates.
try:
await shape_ledger.flag_divergence(project_id, since=since)
@@ -925,7 +922,7 @@ async def compute_coverage(
agg["accounted"] += row.status != "unclassified"
unclassified = counts.pop("unclassified")
# The CSS consumer map's readout (milestone 302): which files render each
# The CSS consumer map's readout: which files render each
# css row — on the derive groups (a shared recipe vs a scoped one is a
# count), and the negative space: css rules no template names. "Unused"
# is measured only where the map has evidence of templates at all (one
@@ -944,16 +941,16 @@ async def compute_coverage(
proposals = shape_ledger.proposal_summary(rows, consumer_paths=consumer_paths)
divergence = shape_ledger.divergence_summary(rows)
derive_new = shape_ledger.derive_new_summary(rows, since=since)
# What `stamps_to_review` would list (#4608). The review tool existed and
# nobody on two projects knew to call it: 132 and 488 weak stamps sat
# electing canons for weeks. The arrival line is the one place a
# project's own session is sure to look, so the count goes there.
# What `stamps_to_review` would list. A review tool nobody knows to call
# leaves weak stamps electing canons indefinitely; the arrival line is the
# one place a project's own session is sure to look, so the count goes
# there.
try:
review = shape_ledger.stamps_to_review(rows, top=0)
except Exception:
logger.warning("stamp review read failed", exc_info=True)
review = {}
# The write-time figure (milestone 439): how often the end-of-turn
# The write-time figure: how often the end-of-turn
# question was put on this project, and how often it was walked past.
# An audit reads the ledger; this reads whether the reflex held.
try:
@@ -968,27 +965,28 @@ async def compute_coverage(
"accounted": len(rows) - unclassified,
"unclassified": unclassified,
"counts": counts,
# The proposer's standing (#2792): canon proposals awaiting a
# The proposer's standing: canon proposals awaiting a
# confirm, the largest derive-first groups, and what this refresh did.
"proposed": proposals["proposed"],
"derive_groups": proposals["derive_groups"],
"top_canon": proposals.get("top_canon"),
# Drift since the previous refresh (#2899): copies that joined a
# Drift since the previous refresh: copies that joined a
# duplicate family — what the arrival line names so drift is noticed
# on entering, not found by an audit.
"derive_new": derive_new,
# The consumer map's negative space (milestone 302): live css rules
# The consumer map's negative space: live css rules
# no template names — None when the map has no evidence of templates.
"unused_css": unused_css,
"proposer": proposer_stats,
# The divergence readout (#2793): button B where button A is canon,
# and judged shapes whose bodies moved since they were judged.
# The divergence readout: new shapes where a different canon
# dominates, and judged shapes whose bodies moved since they were
# judged.
"divergent": divergence["divergent"],
"divergence": divergence["divergence"],
"recheck": divergence["recheck"],
# Unattended stamps below today's floor, and canons whose members no
# longer agree what they are — the judgment queue `stamps_to_review`
# holds (#4608).
# holds.
"weak_stamps": review.get("weak_count", 0),
"incoherent_canons": review.get("incoherent_count", 0),
# {checked, asked, left, days} from the Stop hook's recorded outcomes.
@@ -1021,9 +1019,9 @@ async def refresh_coverage(
return coverage
# The arrival-moment self-seed (#2802). A ledger that has never been computed
# used to wait for someone to find the UI Refresh button; entering the project
# is the moment the number is wanted, so entering is the moment it seeds.
# The arrival-moment self-seed. A ledger that has never been computed should
# not wait for someone to find the UI Refresh button; entering the project is
# the moment the number is wanted, so entering is the moment it seeds.
_SEED_MAX_AGE = timedelta(hours=24)
# Projects with a refresh already running — concurrent enters must not fetch
@@ -1056,7 +1054,7 @@ async def refresh_if_stale(
return
selector = await get_forges(user_id, project_id)
if not selector.configured:
return # rule #115: forge-less stays exactly as it was
return # forge-less stays exactly as it was
_inflight_seed.add(project_id)
try:
await refresh_coverage(user_id, project_id, selector=selector)
@@ -1070,7 +1068,7 @@ async def refresh_if_stale(
async def refresh_for_caller(caller_id: int, project_id: int) -> dict:
"""The explicit agent-facing refresh (#2802) — synchronous, named errors.
"""The explicit agent-facing refresh — synchronous, named errors.
Write-gated: recomputing spends the owner's forge API budget and rewrites
ledger rows' seen-markers, so a read share doesn't grant it. Resolution
@@ -1134,10 +1132,10 @@ def coverage_line(coverage: dict) -> str:
line += f" — {breakdown}"
line += f" (estimate{', computed ' + day if day else ''})"
unclassified = coverage.get("unclassified", 0)
# The standing work, built whatever the todo count (#2899). Since the
# scoped bucket (#2869) a ledger can read 100% accounted and still carry
# derive groups, proposals and divergence; gating this block on
# `unclassified > 0` is how 439 derive rows went unmentioned.
# The standing work, built whatever the todo count. With the scoped
# bucket a ledger can read 100% accounted and still carry derive groups,
# proposals and divergence; gating this block on `unclassified > 0` would
# leave them unmentioned.
standing = []
if coverage.get("proposed"):
standing.append(f"{coverage['proposed']} proposed")
@@ -1157,7 +1155,7 @@ def coverage_line(coverage: dict) -> str:
standing.append(f"+{n} new cop{'y' if n == 1 else 'ies'} since last refresh{where}")
if coverage.get("divergent"):
standing.append(f"{coverage['divergent']} DIVERGENT")
# The next action, on the line (#2874): the canon with the biggest
# The next action, on the line: the canon with the biggest
# queue to confirm, and the widest body-identical copy to consolidate.
top = coverage.get("top_canon") or {}
if top.get("snippet_id"):
@@ -1165,7 +1163,7 @@ def coverage_line(coverage: dict) -> str:
first = (coverage.get("derive_groups") or [{}])[0]
if first.get("label") and first.get("files"):
top_copy = f"top copy {first['label']} ×{first['files']} files"
# A css family says what renders it (milestone 302): the count that
# A css family says what renders it: the count that
# tells a shared recipe from a scoped convention.
if "consumers" in first:
n_t = (first.get("consumers") or {}).get("count", 0)
@@ -1185,8 +1183,8 @@ def coverage_line(coverage: dict) -> str:
line += f"; standing: {', '.join(standing)}"
if coverage.get("recheck"):
line += f"; {coverage['recheck']} judged shape{'s' if coverage['recheck'] != 1 else ''} changed since judged — recheck"
# Rows the hook asserted on evidence the ledger no longer accepts (#4608).
# They no longer elect a canon, but they still claim to be instances
# Rows the hook asserted on evidence below the ledger's floor. They do
# not elect a canon, but they still claim to be instances
# until someone reads them — the tool to call is named, not implied.
review = []
if coverage.get("weak_stamps"):
@@ -1197,7 +1195,7 @@ def coverage_line(coverage: dict) -> str:
review.append(f"{n_i} incoherent canon{'s' if n_i != 1 else ''}")
if review:
line += f"; {' · '.join(review)} to judge — stamps_to_review"
# Whether the writers judged what they wrote (milestone 439). Silent until
# Whether the writers judged what they wrote. Silent until
# the question has been put at least once: a project nobody has written to
# through the plugin has no figure, not a perfect one.
wt = coverage.get("write_time") or {}
+66 -74
View File
@@ -11,7 +11,7 @@ has (embeddings, ACL, project/System association, dedup) and, crucially, becomes
eligible for semantic recall the moment it's embedded:
- ``title`` = ``name``. The trigger joins it only in the embedded document
(``embeddings.document_title``, milestone 427).
(``embeddings.document_title``).
- ``tags`` = ``[language, "snippet", *caller_tags]``.
- ``body`` = templated markdown (When to use / Signature / Location, then a
fenced code block, then an optional ``## Notes`` section).
@@ -44,7 +44,7 @@ logger = logging.getLogger(__name__)
SNIPPET_NOTE_TYPE = "snippet"
SNIPPET_TAG = "snippet"
# The body section free-text notes live under (#4378). A heading rather than a
# The body section free-text notes live under. A heading rather than a
# `**Notes:**` line because notes are paragraphs, and a heading is the boundary
# `embeddings.chunk_document` splits at.
NOTES_HEADING = "## Notes"
@@ -103,11 +103,11 @@ def resolve_locations(
are the single-location shorthand and apply only when `locations` was not
given — passing both is not a merge, it is the caller having decided.
Extracted because compose_body, create_snippet and the dedup gate must all
read the shorthand the SAME way. They each had their own copy of the
`if locations is None` fallback, which is fine until one of them gains a
rule the others don't — and the gate (#2518) is the one where a disagreement
would mean comparing a location the record won't actually be stored with.
One function because compose_body, create_snippet and the dedup gate must
all read the shorthand the SAME way. Three copies of the
`if locations is None` fallback are fine until one of them gains a rule
the others don't — and the gate is the one where a disagreement would mean
comparing a location the record won't actually be stored with.
"""
if locations is None:
locations = [{"repo": repo, "path": path, "symbol": symbol}]
@@ -138,8 +138,8 @@ def _normalize_merged_from(entries: list | None) -> list[dict]:
Order is history, not sorting — earlier merges stay first, so the list reads
as the sequence of things folded in.
Each entry records WHAT THAT SOURCE CONTRIBUTED, which is what makes un-merge
(#2165) exact. Two problems it solves at once:
Each entry records WHAT THAT SOURCE CONTRIBUTED, which is what makes
un-merge exact. Two problems it solves at once:
- A location can arrive from a source AND genuinely be the survivor's own.
Recording only what the source ADDED means reversing it can never strip a
@@ -200,7 +200,7 @@ def compose_body(
"""Render structured fields into the snippet body markdown. Empty fields are
omitted so the body stays clean.
``notes`` is the free text a snippet had no home for (#4378): why it is
``notes`` is the free text a snippet has no other home for: why it is
shaped this way, what superseded what, the caveats. Without it that prose
went into ``when_to_use`` — the trigger, which is joined onto the title of
EVERY chunk the snippet is embedded as, so a 3 KB write-up there blurred
@@ -371,7 +371,7 @@ _DATA_FIELDS = (
"verification", "provenance",
)
# --- drift check (#2086) -----------------------------------------------------
# --- drift check -------------------------------------------------------------
# A recorded snippet points at a repo · path · symbol that WILL rot: files move,
# symbols get renamed, implementations diverge from the copy stored here. Left
# undetected, the record degrades from "canonical reference" to "confidently
@@ -379,9 +379,9 @@ _DATA_FIELDS = (
# same authority either way.
#
# WHERE THE CHECK RUNS. Not here. Scribe has no checkout of the operator's repos
# and must not acquire one (rule #115 — the instance stays agnostic about where
# code lives; giving the server repo access would make every install a
# credential problem). The agent already has the working tree, so IT does the
# and must not acquire one (the instance stays agnostic about where code
# lives; giving the server repo access would make every install a credential
# problem). The agent already has the working tree, so IT does the
# comparing and reports a verdict; the server's job is to remember the verdict,
# make it queryable, and know when it has expired.
#
@@ -445,10 +445,10 @@ def compose_verification(
out["detail"] = detail.strip()
if (path or "").strip():
out["path"] = path.strip()
# The repo commit the working tree was at when the check ran (#2688). The
# The repo commit the working tree was at when the check ran. The
# code_sha above expires a verdict when the RECORD is edited; this makes
# "the REPO moved on since the check" computable too, once the forge
# integration can compare it against the current head.
# "the REPO moved on since the check" computable too, against the forge's
# current head (`_verdict_still_vouches`).
if (commit_sha or "").strip():
out["commit_sha"] = commit_sha.strip()
return out
@@ -458,12 +458,11 @@ def compose_provenance(*, commit_sha: str, fetched_at: str = "") -> dict | None:
"""Build the `data.provenance` record: which commit the cached body was
read at, and when.
This is the pointer-model half of decision #2686 — the recorded location
is the source of truth for the code and the stored body is a CACHE of it.
Provenance says what that cache is a cache OF, so a reader (and later the
forge fetch, step 5 of milestone 288) can judge staleness instead of
guessing. Absent provenance is valid and means exactly what every snippet
meant before this existed: a body captured by hand at an unknown point.
The pointer model: the recorded location is the source of truth for the
code and the stored body is a CACHE of it. Provenance says what that cache
is a cache OF, so a reader (and the forge fetch at pull time) can judge
staleness instead of guessing. Absent provenance is valid and means a body
captured by hand at an unknown point.
"""
sha = (commit_sha or "").strip()
if not sha:
@@ -493,7 +492,7 @@ def verification_view(note, fields: dict) -> dict:
"detail": stored.get("detail"),
"path": stored.get("path"),
"commit_sha": stored.get("commit_sha"),
# A push touched the recorded location since this verdict (#2691) —
# A push touched the recorded location since this verdict —
# the repo moved under it. Cleared by the next verdict, which builds
# a fresh dict.
"invalidated_by": stored.get("invalidated_by"),
@@ -548,7 +547,7 @@ def compose_data(
# either: the verdict's code_sha expires it on read if the code moved on.
if verification:
out["verification"] = verification
# Also carried: what commit the cached body was read at (#2688). The caller
# Also carried: what commit the cached body was read at. The caller
# owns the live-or-die rule — update_snippet drops it when the code changes
# without a fresh SHA, because keeping it would claim the new body came
# from the old commit.
@@ -565,7 +564,7 @@ def compose_data(
return out
# Past this, a trigger is advised to move its explanation into `notes` (#4378).
# Past this, a trigger is advised to move its explanation into `notes`.
# The trigger is joined onto the title of EVERY chunk (`chunk_document`), so at
# 600 characters it is already over 40% of a 1,400-character chunk — each
# vector is then more about the trigger's prose than about the code or the
@@ -605,14 +604,13 @@ def recompose_data(note) -> dict:
For the GENERIC note door. `update_snippet` composes the mirror itself from
the field set it just merged and never needs this; a plain
`update_note(body=...)` — which the Knowledge feed's editor issues, because
a snippet card there routes to /notes/:id — has no idea the mirror exists,
and left it stale. `snippet_fields` then PREFERS the stale mirror, so the
row reported its old repo/path/symbol to prior-art recall while displaying
its new body: confidently wrong, which is worse than no record (#3128).
a snippet card there routes to /notes/:id — has no idea the mirror exists.
Left stale, the mirror wins: `snippet_fields` PREFERS it, so the row would
report its old repo/path/symbol to prior-art recall while displaying its
new body — confidently wrong, which is worse than no record.
The body is the authority; the mirror is derived. That is already the rule
this file states — it just had no enforcement on the path that bypasses
`update_snippet`.
The body is the authority; the mirror is derived. This is that rule's
enforcement on the path that bypasses `update_snippet`.
`verification` and `provenance` are CARRIED, not recomposed, because
neither is in the body to parse — the same carry `compose_data` does for
@@ -642,15 +640,11 @@ def snippet_fields(note) -> dict:
THE BODY IS THE AUTHORITY; `data` is a mirror derived from it. Every writer
keeps them in step — the snippet service composes the mirror from the field
set it just merged, `update_note` recomposes it when a body reaches the
generic door (#3128), and `backfill_snippet_data` filled the pre-0070 rows
generic door, and `backfill_snippet_data` fills any row without a mirror
at startup. The fallback below is therefore a belt to that braces, not a
second source of truth: it is what a row looks like before the backfill has
run, and it must keep agreeing with the mirror.
(This docstring used to say those rows were "never backfilled". That was
true when 0070 landed and stopped being true when the backfill shipped; it
is corrected here because the sentence read as licence for a stale mirror,
which is exactly the bug #3128 found.)
run, and it must keep agreeing with the mirror. A stale mirror is a bug,
never a state to tolerate.
`code` only ever comes from the body, since `data` doesn't carry it.
"""
@@ -678,7 +672,7 @@ async def backfill_snippet_data(*, batch: int = 500) -> int:
Migration 0070 deliberately left `data` NULL on existing rows, because
*reading* a snippet never needed it — `snippet_fields` falls back to parsing
the body. Querying does: the location reverse lookup (#2083) is a jsonpath
the body. Querying does: the location reverse lookup is a jsonpath
over this column, so a NULL-`data` snippet would be reported as "nothing
recorded here" and the caller would write the helper again — silently worse
than having no reverse lookup at all. Every consumer of the column
@@ -779,8 +773,8 @@ async def create_snippet(
``repo``/``path``/``symbol`` are the one-location shorthand.
``commit_sha`` stamps the body's provenance — the commit the recording
session read the code at (#2688). Optional: absent means what it always
meant, a body captured at an unknown point."""
session read the code at. Optional: absent means a body captured at an
unknown point."""
locations = resolve_locations(repo, path, symbol, locations)
note = await notes_svc.create_note(
user_id,
@@ -807,9 +801,10 @@ async def get_snippet(user_id: int, snippet_id: int):
"""Fetch a snippet by id, or None if it doesn't exist / isn't a snippet /
isn't readable by this user.
Share-aware (rule #78): a fetch by id is an explicit act, so it resolves the
caller's full read scope rather than ownership alone. Without this, a snippet
that a search legitimately surfaced could not then be opened — see #2093."""
Share-aware (services/access.py): a fetch by id is an explicit act, so it
resolves the caller's full read scope rather than ownership alone. Without
this, a snippet that a search legitimately surfaced could not then be
opened."""
result = await notes_svc.get_note_for_user(user_id, snippet_id)
if result is None:
return None
@@ -883,12 +878,12 @@ async def update_snippet(
merged field set back into title/body/tags. Returns the Note, or None if the
id isn't a snippet the caller can see.
``commit_sha`` restamps the body's provenance (#2688). It lives or dies
``commit_sha`` restamps the body's provenance. It lives or dies
with the code: passed → restamped at that commit; code changed without it →
dropped, because keeping it would claim the new body came from the old
commit; code untouched → carried.
Share-aware (rule #47/#78): resolves the read scope, then requires WRITE —
Share-aware (services/access.py): resolves the read scope, then requires WRITE —
so an editor/admin grant lets the holder edit, and a viewer grant does not.
Raises PermissionError when the caller can read but not write, because "not
found" would be a lie about a record they can plainly open. The write itself
@@ -930,7 +925,7 @@ async def update_snippet(
else:
merged_locations = cur["locations"]
# Provenance follows the code (#2688): a fresh SHA restamps it; a code
# Provenance follows the code: a fresh SHA restamps it; a code
# change without one drops it; an edit that leaves the code alone carries
# it. Order matters — the explicit SHA wins even when the code changed,
# because that is precisely the caller saying where the new body came from.
@@ -1037,7 +1032,7 @@ async def record_verification(
# An "ok" verdict at a known commit IS a provenance claim — the checker
# just established that the cached body matches the source there — so
# it restamps. Any other verdict carries what was known: a verdict is
# about the code, not a change to it, and must not erase it (#2688).
# about the code, not a change to it, and must not erase it.
provenance=(
compose_provenance(commit_sha=commit_sha)
if status == VERIFY_OK and (commit_sha or "").strip()
@@ -1047,7 +1042,7 @@ async def record_verification(
return await notes_svc.update_note(note.user_id, snippet_id, data=data)
# --- pull-time freshness (#2690) ---------------------------------------------
# --- pull-time freshness -----------------------------------------------------
# A pull is the moment freshness matters: the reader is about to trust the
# cached body. When the record owner's keyring serves a forge, the pull
# fetches the recorded file and answers the one mechanically-answerable
@@ -1057,12 +1052,11 @@ async def record_verification(
# fetching can honestly deliver, and divergence is reported, not overwritten.
#
# With no forge configured this function attaches NOTHING — the response is
# byte-identical to pre-forge behavior (rule #115's baseline).
# exactly what an install without a forge gets everywhere else.
# Total budget for the in-pull fetch. Tighter than the adapter's own timeout:
# the pull is the moment a session decides whether pulling is worth it
# (#2663's pull-through finding), so a slow forge must cost bounded time and
# then the cache serves.
# the pull is the moment a session decides whether pulling is worth it, so a
# slow forge must cost bounded time and then the cache serves.
PULL_FETCH_BUDGET_S = 2.5
@@ -1099,8 +1093,8 @@ def _verdict_still_vouches(note, fields: dict, fetched_commit_sha: str) -> bool:
"""Does a standing `ok` verdict still speak for this body, at this commit?
Containment (cached code ∈ fetched file) is the fast path, and it is right
for a record kept verbatim. It is WRONG for a deliberately annotated one
(#2782): a record whose job is to say why the shape is what it is carries
for a record kept verbatim. It is WRONG for a deliberately annotated one:
a record whose job is to say why the shape is what it is carries
commentary the source does not, so containment fails forever and the record
reads `diverged` on every pull. That turns the one honest drift signal into
a permanent false positive — and annotation is a sanctioned record style,
@@ -1109,15 +1103,14 @@ def _verdict_still_vouches(note, fields: dict, fetched_commit_sha: str) -> bool:
The escape hatch is the verdict itself. `verify_snippet` is precisely where
a human or agent already judged this body a faithful rendering of that
source, and `verification.commit_sha` records the repo commit they judged
it at — a field whose own docstring (#2688) anticipated this use: "makes
'the REPO moved on since the check' computable, once the forge integration
can compare it against the current head." This is that comparison.
it at, which makes "the REPO moved on since the check" computable. This
is that comparison.
All four conditions, and none is optional:
- the verdict says `ok`;
- it has not EXPIRED — `verification_view` recomputes `code_sha` against
the record's current body, so editing the record retires the verdict;
- it was not INVALIDATED by a push touching the location (#2691);
- it was not INVALIDATED by a push touching the location;
- the file we just fetched is at the very commit the verdict was stamped
at. Any later commit means nobody has judged what is there now.
@@ -1136,10 +1129,9 @@ def _verdict_still_vouches(note, fields: dict, fetched_commit_sha: str) -> bool:
async def attach_live_body(note, data: dict) -> None:
"""Decorate a PULL response with forge-checked freshness (#2690).
"""Decorate a PULL response with forge-checked freshness.
Adds, when (and only when) the record owner's keyring serves a forge
(#2778):
Adds, when (and only when) the record owner's keyring serves a forge:
- ``body_source``: "forge" (confirmed against the source just now) or
"cache" (the stored body, for whatever reason follows)
- ``body_freshness``: "current" | "diverged" | "missing" |
@@ -1156,7 +1148,7 @@ async def attach_live_body(note, data: dict) -> None:
from scribe.services.forge import ForgeError, ForgeNotFound, get_forges
try:
# The OWNER's keyring, honoring the project pin (#2778) — freshness
# The OWNER's keyring, honoring the project pin — freshness
# for a record is checked with its owner's credential, never the
# reader's.
selector = await get_forges(note.user_id, getattr(note, "project_id", None))
@@ -1183,7 +1175,7 @@ async def attach_live_body(note, data: dict) -> None:
return
# Recorded location repos are free-form names ("Scribe"), which can't
# address a forge API — the project's repo BINDING is the identity that
# can (#2691). Try the location string first (it may be a real remote),
# can. Try the location string first (it may be a real remote),
# then fall back to the bindings of the snippet's project.
resolved = selector.resolve(loc["repo"])
if resolved is None and getattr(note, "project_id", None):
@@ -1202,7 +1194,7 @@ async def attach_live_body(note, data: dict) -> None:
stored_prov_sha = (fields.get("provenance") or {}).get("commit_sha") or ""
async def _probe():
# Cached-SHA short-circuit (#2693): provenance names the commit the
# Cached-SHA short-circuit: provenance names the commit the
# cached code was last confirmed at, so one cheap "newest commit
# touching this path" call can prove the file hasn't moved since —
# no content transfer. That economy is what fits pull-time freshness
@@ -1266,7 +1258,7 @@ async def attach_live_body(note, data: dict) -> None:
elif _verdict_still_vouches(note, fields, fetched.commit_sha or ""):
# Containment failed, but an unexpired `ok` verdict stamped at exactly
# this commit already judged this body a faithful rendering of it —
# the annotated-record case (#2782). Trust the judgment over the
# the annotated-record case. Trust the judgment over the
# substring test; `data["verification"]` travels in the same payload,
# so a reader can see the basis rather than take "current" on faith.
data["body_source"] = "forge"
@@ -1276,7 +1268,7 @@ async def attach_live_body(note, data: dict) -> None:
data["body_freshness"] = "diverged"
# --- push-time drift flagging (#2691) ----------------------------------------
# --- push-time drift flagging ------------------------------------------------
def _path_touches(recorded: str, changed: str) -> bool:
"""The location-path semantics, applied to a pushed file: the recorded path
@@ -1294,7 +1286,7 @@ async def invalidate_for_push(
removed: list[str],
commit_sha: str,
) -> int:
"""Flag snippets whose recorded location a push just touched (#2691).
"""Flag snippets whose recorded location a push just touched.
Writes ``verification.invalidated_by = {commit_sha, at, path, removed}``
onto matched snippets that CARRY a verdict — the flag means "the repo
@@ -1449,7 +1441,7 @@ async def merge_snippets(user_id: int, target_id: int, source_ids: list[int]):
Share-aware and write-gated, like update: an editor/admin grant is enough, a
viewer grant is not. Raises PermissionError when the caller can read the
target but not write it. Every source must share the TARGET'S OWNER —
cross-owner merge stays out of scope (#231) — and must itself be writable;
cross-owner merge is out of scope — and must itself be writable;
sources failing either test are skipped rather than silently half-merged.
The survivor records what it absorbed as `merged_from` — in the `data` mirror
@@ -1499,7 +1491,7 @@ async def merge_snippets(user_id: int, target_id: int, source_ids: list[int]):
# a target merged twice keeps both histories.
#
# Each entry also carries what THAT source contributed, which is what makes
# un-merge exact (#2165) rather than a blind subtraction that would strip
# un-merge exact rather than a blind subtraction that would strip
# call sites the survivor legitimately owns.
merged_from = _normalize_merged_from(
list(tgt_fields.get("merged_from") or [])
@@ -1561,8 +1553,8 @@ async def unmerge_snippet(user_id: int, survivor_id: int, source_id: int):
one record type. Instead un-merge performs the restore itself, so the inverse
is one operation with one authorization check and the trash path stays
ignorant. Restoring from the trash directly is still allowed and still leaves
both records claiming the same call sites — which is why this exists — but it
is no longer the only way back.
both records claiming the same call sites — which is why this exists — so it
is not the only way back.
EXACTNESS. Subtraction uses the contribution recorded at merge time, not the
source's current locations. A source that was itself edited after being