feat(snippets): drift check — verify a snippet still matches its source
CI & Build / Python lint (push) Successful in 3s
CI & Build / integration (push) Successful in 19s
CI & Build / TypeScript typecheck (push) Failing after 20s
CI & Build / Python tests (push) Failing after 22s
CI & Build / Build & push image (push) Has been skipped

A recorded snippet points at a repo · path · symbol that WILL rot: files
move, symbols get renamed, implementations diverge from the copy stored
here. Nothing detected any of it, so a record degraded silently from
"canonical reference" to "confidently wrong" — worse than no record, since
it is surfaced with the same authority either way.

WHERE THE CHECK RUNS. Agent-side, which the task flagged as the design
question to settle first. Scribe has no checkout of the operator's repos
and must not acquire one: giving the server repo access would make every
install a credential problem and break instance-agnosticism (rule #115).
The agent already has the working tree, so it does the comparing; the
server remembers the verdict, makes it queryable, and knows when it has
expired. New MCP tool verify_snippet teaches the four-step procedure and
records the result; a REST endpoint mirrors it so the UI can clear a
marker after a manual fix.

WHY THE VERDICT CARRIES A CODE HASH. A verdict describes the code it was
checked against. Invalidating it on edit means deciding which edits count
— a when_to_use tweak shouldn't void a code check, a rewrite must — which
is fiddly and easy to get subtly wrong, and easy for a new write path to
forget entirely. Stamping the verdict with a hash sidesteps all of it: one
whose code_sha no longer matches is self-evidently expired, computed at
read time, no invalidation branch to maintain.

That makes "expired" the interesting filter case. It is not `drifted`
(nothing was found wrong) and not `unverified` (a check did happen), yet
it plainly needs looking at — so `verification=attention` covers both. To
keep that one index-served predicate rather than a post-filter that would
make the pagination total a lie, data now also mirrors the CURRENT code's
fingerprint as data.code_sha, and a jsonpath compares the two fields
within the row. The filter is implemented in both dialects, SQL and
Python, for the same reason the location filter is: the semantic arm's
candidates arrive already fetched.

A merge deliberately carries no verdict forward — the survivor's code is a
union of several sources, so no prior check describes it, and unverified
is the honest answer.

UI: a danger-toned drift badge on each card (an actively misleading record
outranks a merely unused one), and a "Needs attention" filter. Its empty
state says plainly that never-verified snippets don't appear there —
otherwise `attention` would mean "everything" on day one and be useless as
a worklist.

Refs #2086

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
This commit is contained in:
2026-07-28 18:20:33 -04:00
co-authored by Claude Opus 5
parent 2b85443dd1
commit 35f3f09d12
7 changed files with 767 additions and 21 deletions
+191 -5
View File
@@ -29,8 +29,10 @@ came from.
from __future__ import annotations
import asyncio
import hashlib
import logging
import re
from datetime import datetime, timezone
from sqlalchemy import func, or_, select
@@ -303,8 +305,104 @@ def parse_snippet_fields(
# and copying a blob into the column we index *around* would be pure weight.
_DATA_FIELDS = (
"name", "when_to_use", "signature", "language", "locations", "merged_from",
"verification",
)
# --- drift check (#2086) -----------------------------------------------------
# A recorded snippet points at a repo · path · symbol that WILL rot: files move,
# symbols get renamed, implementations diverge from the copy stored here. Left
# undetected, the record degrades from "canonical reference" to "confidently
# wrong" — which is worse than having no record, because it is surfaced with the
# same authority either way.
#
# WHERE THE CHECK RUNS. Not here. Scribe has no checkout of the operator's repos
# and must not acquire one (rule #115 — the instance stays agnostic about where
# code lives; giving the server repo access would make every install a
# credential problem). The agent already has the working tree, so IT does the
# comparing and reports a verdict; the server's job is to remember the verdict,
# make it queryable, and know when it has expired.
#
# WHY THE VERDICT CARRIES A CODE HASH. A stored verdict describes the code it
# was checked against. Edit the snippet afterwards and that verdict is no longer
# about anything — but invalidating it on write means deciding which edits count
# (a `when_to_use` tweak shouldn't void a code check; a code rewrite must). That
# rule is fiddly and easy to get subtly wrong. Recording the hash sidesteps it
# entirely: a verdict whose `code_sha` no longer matches the body is self-
# evidently expired, computed at read time, with no invalidation logic to
# maintain and no way for an edit path to forget to call it.
VERIFY_OK = "ok"
VERIFY_MISSING = "missing" # the recorded path is gone
VERIFY_MOVED = "moved" # path is there, the symbol isn't in it
VERIFY_CHANGED = "changed" # both present, but the source no longer matches
VERIFY_STATUSES = (VERIFY_OK, VERIFY_MISSING, VERIFY_MOVED, VERIFY_CHANGED)
# Everything that isn't a clean bill of health. "Stale" in the UI and the filter
# means this set — the operator wants one list of things to look at, not four.
VERIFY_DRIFTED = (VERIFY_MISSING, VERIFY_MOVED, VERIFY_CHANGED)
def code_sha(code: str) -> str:
"""Stable fingerprint of a snippet's code, for expiring stale verdicts.
Trailing whitespace per line and leading/trailing blank lines are stripped
before hashing: those change when a file is reformatted without the code
meaning anything different, and a verdict shouldn't expire over an editor's
trailing-newline habit.
"""
normalized = "\n".join(line.rstrip() for line in (code or "").splitlines()).strip()
return hashlib.sha256(normalized.encode("utf-8")).hexdigest()[:32]
def compose_verification(
*,
status: str,
checked_code_sha: str,
detail: str = "",
path: str = "",
checked_at: str = "",
) -> dict:
"""Build the `data.verification` record. Unknown statuses are rejected here
rather than stored, so the filter never has to cope with a typo'd status."""
if status not in VERIFY_STATUSES:
raise ValueError(
f"unknown verification status {status!r} — expected one of {VERIFY_STATUSES}"
)
out = {
"status": status,
"code_sha": checked_code_sha,
"checked_at": checked_at or datetime.now(timezone.utc).isoformat(),
}
if (detail or "").strip():
out["detail"] = detail.strip()
if (path or "").strip():
out["path"] = path.strip()
return out
def verification_view(note, fields: dict) -> dict:
"""The verification readout for one snippet, including whether it's expired.
`status` is what was last reported; `current` says whether that verdict still
describes the code in the record. A verdict that no longer matches reads as
unverified, because that is what it is — nobody has checked THIS code.
"""
stored = (fields.get("verification") or {}) if isinstance(fields, dict) else {}
if not stored or not stored.get("status"):
return {"status": "unverified", "current": False, "checked_at": None}
current = stored.get("code_sha") == code_sha(fields.get("code") or "")
return {
"status": stored["status"],
"current": current,
"checked_at": stored.get("checked_at"),
"detail": stored.get("detail"),
"path": stored.get("path"),
# What the operator actually wants to know: is there something to fix?
# An expired verdict counts as "needs looking at" even if it said ok,
# since the code it blessed is not the code that's there now.
"needs_attention": (not current) or stored["status"] in VERIFY_DRIFTED,
}
def compose_data(
*,
@@ -312,8 +410,10 @@ def compose_data(
when_to_use: str = "",
signature: str = "",
language: str = "",
code: str = "",
locations: list[dict] | None = None,
merged_from: list[int] | None = None,
verification: dict | None = None,
) -> dict:
"""Build the `notes.data` mirror of a snippet's structured fields.
@@ -337,6 +437,19 @@ def compose_data(
merged = _normalize_merged_from(merged_from)
if merged:
out["merged_from"] = merged
# Carried, never composed here — like merged_from. An ordinary edit must not
# silently drop the last drift check, and it doesn't need to invalidate it
# either: the verdict's code_sha expires it on read if the code moved on.
if verification:
out["verification"] = verification
# The current code's fingerprint — NOT the code, which stays in the body
# (see _DATA_FIELDS). Its only job is to make "this verdict has expired"
# expressible in SQL: a jsonpath can compare `@.verification.code_sha` to
# `@.code_sha` within the same row, so "show me everything that needs
# looking at" stays one index-served query instead of a post-filter that
# would break pagination counts.
if (code or "").strip():
out["code_sha"] = code_sha(code)
return out
@@ -419,6 +532,7 @@ async def backfill_snippet_data(*, batch: int = 500) -> int:
when_to_use=fields["when_to_use"],
signature=fields["signature"],
language=fields["language"],
code=fields["code"],
locations=fields["locations"],
merged_from=fields["merged_from"],
)
@@ -439,7 +553,13 @@ def snippet_to_dict(note) -> dict:
``snippet`` already reports, and shipping both would give API consumers two
sources of truth for the same facts."""
data = note.to_dict()
data["snippet"] = snippet_fields(note)
fields = snippet_fields(note)
data["snippet"] = fields
# Promoted out of `snippet` because it is a computed READOUT, not a recorded
# field: `current` and `needs_attention` are derived at read time by hashing
# the code, and burying them among the stored fields would invite a caller
# to try writing them back.
data["verification"] = verification_view(note, fields)
return data
@@ -479,7 +599,7 @@ async def create_snippet(
# body so the two can never describe different things.
data=compose_data(
name=name, when_to_use=when_to_use, signature=signature,
language=language, locations=locations,
language=language, code=code, locations=locations,
),
)
_embed_snippet(note)
@@ -513,6 +633,7 @@ async def list_snippets(
repo: str = "",
path: str = "",
symbol: str = "",
verification: str = "",
) -> tuple[list[dict], int]:
"""List snippets (id/title/tags/preview dicts), most-recently-updated first.
@@ -523,7 +644,11 @@ async def list_snippets(
``repo`` / ``path`` / ``symbol`` are the reverse lookup: "what canonical
helpers already live here?" They narrow to snippets recorded at a matching
location, ANDed within one location entry, with ``path`` also matching as a
directory prefix. Combinable with ``q`` — search *and* place."""
directory prefix. Combinable with ``q`` — search *and* place.
``verification`` narrows on the drift check: ``attention`` is the useful one
— everything whose recorded location or code no longer checks out, plus
everything whose verdict expired because the snippet was edited since."""
return await knowledge_svc.query_knowledge(
user_id=user_id,
note_type=SNIPPET_NOTE_TYPE,
@@ -535,6 +660,7 @@ async def list_snippets(
project_id=project_id,
locations=knowledge_svc.location_parts(repo=repo, path=path, symbol=symbol)
or None,
verification=verification,
)
@@ -615,8 +741,12 @@ async def update_snippet(
"data": compose_data(
name=merged["name"], when_to_use=merged["when_to_use"],
signature=merged["signature"], language=merged["language"],
locations=merged_locations,
code=merged["code"], locations=merged_locations,
merged_from=merged.get("merged_from"),
# Carried through the edit rather than cleared. If this edit changed
# the code, the verdict's code_sha stops matching and it reads as
# unverified from here on — no invalidation branch to get wrong.
verification=merged.get("verification"),
),
}
# Recompute tags: keep any non-language, non-marker tags the note already had
@@ -639,6 +769,59 @@ async def update_snippet(
return updated
async def record_verification(
user_id: int,
snippet_id: int,
*,
status: str,
detail: str = "",
path: str = "",
):
"""Record the result of a drift check against the snippet's source.
The CHECK happens agent-side — Scribe has no checkout and shouldn't want one
(see the drift-check note above). This just remembers the verdict, stamped
with a hash of the code it was checked against so it expires by itself when
the snippet is edited.
Requires WRITE access: a verdict changes how the record is presented and
whether it shows up in the operator's "needs attention" list, so being able
to read a shared snippet must not let you mark it broken.
Returns the updated note, or None if the id isn't a snippet this user may
write. Raises ValueError on an unknown status.
"""
note = await get_snippet(user_id, snippet_id)
if note is None:
return None
from scribe.services.access import can_write_note
if not await can_write_note(user_id, snippet_id):
return None
fields = snippet_fields(note)
verification = compose_verification(
status=status,
checked_code_sha=code_sha(fields.get("code") or ""),
detail=detail,
path=path or fields.get("path") or "",
)
# Rebuilt from the CURRENT stored fields plus the new verdict, so recording a
# check can't quietly rewrite anything else about the record. Note the body
# is untouched — a verdict is metadata about the snippet, not part of it, and
# writing it into the body would put it into the embedding.
data = compose_data(
name=fields.get("name", ""),
when_to_use=fields.get("when_to_use", ""),
signature=fields.get("signature", ""),
language=fields.get("language", ""),
code=fields.get("code", ""),
locations=fields.get("locations") or [],
merged_from=fields.get("merged_from") or [],
verification=verification,
)
return await notes_svc.update_note(note.user_id, snippet_id, data=data)
async def delete_snippet(user_id: int, snippet_id: int) -> bool:
"""Retire a snippet to the trash (recoverable). Returns False if the id isn't
a snippet this user may WRITE.
@@ -760,7 +943,10 @@ async def merge_snippets(user_id: int, target_id: int, source_ids: list[int]):
data=compose_data(
name=tgt_fields["name"], when_to_use=tgt_fields["when_to_use"],
signature=tgt_fields["signature"], language=tgt_fields["language"],
locations=locations, merged_from=merged_from,
code=tgt_fields["code"], locations=locations, merged_from=merged_from,
# No verification carried: the survivor's code is a union of several
# sources, so no prior verdict describes it. It reads as unverified,
# which is the honest answer — nobody has checked THIS code.
),
)
if updated is None: