feat(snippets): drift check — verify a snippet still matches its source
CI & Build / Python lint (push) Successful in 3s
CI & Build / integration (push) Successful in 19s
CI & Build / TypeScript typecheck (push) Failing after 20s
CI & Build / Python tests (push) Failing after 22s
CI & Build / Build & push image (push) Has been skipped

A recorded snippet points at a repo · path · symbol that WILL rot: files
move, symbols get renamed, implementations diverge from the copy stored
here. Nothing detected any of it, so a record degraded silently from
"canonical reference" to "confidently wrong" — worse than no record, since
it is surfaced with the same authority either way.

WHERE THE CHECK RUNS. Agent-side, which the task flagged as the design
question to settle first. Scribe has no checkout of the operator's repos
and must not acquire one: giving the server repo access would make every
install a credential problem and break instance-agnosticism (rule #115).
The agent already has the working tree, so it does the comparing; the
server remembers the verdict, makes it queryable, and knows when it has
expired. New MCP tool verify_snippet teaches the four-step procedure and
records the result; a REST endpoint mirrors it so the UI can clear a
marker after a manual fix.

WHY THE VERDICT CARRIES A CODE HASH. A verdict describes the code it was
checked against. Invalidating it on edit means deciding which edits count
— a when_to_use tweak shouldn't void a code check, a rewrite must — which
is fiddly and easy to get subtly wrong, and easy for a new write path to
forget entirely. Stamping the verdict with a hash sidesteps all of it: one
whose code_sha no longer matches is self-evidently expired, computed at
read time, no invalidation branch to maintain.

That makes "expired" the interesting filter case. It is not `drifted`
(nothing was found wrong) and not `unverified` (a check did happen), yet
it plainly needs looking at — so `verification=attention` covers both. To
keep that one index-served predicate rather than a post-filter that would
make the pagination total a lie, data now also mirrors the CURRENT code's
fingerprint as data.code_sha, and a jsonpath compares the two fields
within the row. The filter is implemented in both dialects, SQL and
Python, for the same reason the location filter is: the semantic arm's
candidates arrive already fetched.

A merge deliberately carries no verdict forward — the survivor's code is a
union of several sources, so no prior check describes it, and unverified
is the honest answer.

UI: a danger-toned drift badge on each card (an actively misleading record
outranks a merely unused one), and a "Needs attention" filter. Its empty
state says plainly that never-verified snippets don't appear there —
otherwise `attention` would mean "everything" on day one and be useless as
a worklist.

Refs #2086

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
This commit is contained in:
2026-07-28 18:20:33 -04:00
parent 2b85443dd1
commit 35f3f09d12
7 changed files with 767 additions and 21 deletions
+29
View File
@@ -56,6 +56,19 @@ export interface SnippetUsage {
last_pulled_at: string | null; last_pulled_at: string | null;
} }
/** Result of the last drift check — does the recorded location and code still
* match source? The check runs agent-side (Scribe has no checkout); this is the
* remembered verdict. `current` is false once the snippet has been edited since
* the check, at which point the verdict describes code that's no longer there. */
export interface SnippetVerification {
status: "ok" | "missing" | "moved" | "changed" | "unverified";
current: boolean;
checked_at: string | null;
detail?: string | null;
path?: string | null;
needs_attention?: boolean;
}
/** Lightweight list item from the knowledge preview feed. Note: the `snippet` /** Lightweight list item from the knowledge preview feed. Note: the `snippet`
* field here is a truncated *body preview* (the knowledge feed's naming), not * field here is a truncated *body preview* (the knowledge feed's naming), not
* the parsed fields above. */ * the parsed fields above. */
@@ -73,6 +86,9 @@ export interface SnippetListItem {
owner?: string | null; owner?: string | null;
/** Always present from the backend, zero-filled for records with no events. */ /** Always present from the backend, zero-filled for records with no events. */
usage?: SnippetUsage; usage?: SnippetUsage;
/** Present on the detail record; the list feed carries it when a check has
* been recorded. */
verification?: SnippetVerification;
} }
/** Create/update payload — discrete fields the backend serializes into the /** Create/update payload — discrete fields the backend serializes into the
@@ -103,6 +119,8 @@ export async function listSnippets(
repo?: string; repo?: string;
path?: string; path?: string;
symbol?: string; symbol?: string;
/** Drift check: "attention" | "ok" | "unverified" | "drifted" | a status. */
verification?: string;
} = {}, } = {},
): Promise<{ snippets: SnippetListItem[]; total: number }> { ): Promise<{ snippets: SnippetListItem[]; total: number }> {
const qs = new URLSearchParams(); const qs = new URLSearchParams();
@@ -112,6 +130,7 @@ export async function listSnippets(
if (params.repo) qs.set("repo", params.repo); if (params.repo) qs.set("repo", params.repo);
if (params.path) qs.set("path", params.path); if (params.path) qs.set("path", params.path);
if (params.symbol) qs.set("symbol", params.symbol); if (params.symbol) qs.set("symbol", params.symbol);
if (params.verification) qs.set("verification", params.verification);
const query = qs.toString(); const query = qs.toString();
return apiGet(`/api/snippets${query ? `?${query}` : ""}`); return apiGet(`/api/snippets${query ? `?${query}` : ""}`);
} }
@@ -135,6 +154,16 @@ export async function deleteSnippet(id: number): Promise<void> {
return apiDelete(`/api/snippets/${id}`); return apiDelete(`/api/snippets/${id}`);
} }
/** Record a drift-check verdict. The check itself runs where the code is — an
* agent with the working tree — since Scribe has no checkout. This stores what
* was found, and is how the UI clears a stale marker after a manual fix. */
export async function verifySnippet(
id: number,
verdict: { status: string; detail?: string; path?: string },
): Promise<Snippet> {
return apiPost(`/api/snippets/${id}/verify`, verdict);
}
/** Unify `sourceIds` into the canonical snippet `targetId`. Returns the merged /** Unify `sourceIds` into the canonical snippet `targetId`. Returns the merged
* survivor plus `merged_ids` — the sources actually folded in and trashed. */ * survivor plus `merged_ids` — the sources actually folded in and trashed. */
export async function mergeSnippets( export async function mergeSnippets(
+84 -11
View File
@@ -34,6 +34,15 @@ function toggleLocationFilter() {
if (!showLocationFilter.value && locationActive.value) clearLocation(); if (!showLocationFilter.value && locationActive.value) clearLocation();
} }
// Drift check (#2086) — "attention" is everything whose recorded location or
// code no longer checks out, plus everything whose verdict expired because the
// snippet was edited since it was checked.
const needsAttentionOnly = ref(false);
function toggleAttention() {
needsAttentionOnly.value = !needsAttentionOnly.value;
loadSnippets();
}
// Multi-select → merge // Multi-select → merge
const selectMode = ref(false); const selectMode = ref(false);
const selectedIds = ref<Set<number>>(new Set()); const selectedIds = ref<Set<number>>(new Set());
@@ -96,6 +105,7 @@ async function loadSnippets() {
repo: locRepo.value.trim() || undefined, repo: locRepo.value.trim() || undefined,
path: locPath.value.trim() || undefined, path: locPath.value.trim() || undefined,
symbol: locSymbol.value.trim() || undefined, symbol: locSymbol.value.trim() || undefined,
verification: needsAttentionOnly.value ? "attention" : undefined,
}); });
snippets.value = data.snippets; snippets.value = data.snippets;
} catch { } catch {
@@ -144,6 +154,39 @@ function usageBadge(s: SnippetListItem): string {
return `${u.pull_count}/${u.surfaced_count} used`; return `${u.pull_count}/${u.surfaced_count} used`;
} }
/** Short label for the drift verdict, or "" when there's nothing to say.
* An expired verdict is reported as "unchecked" whatever it used to say —
* it was about code that is no longer in the record. */
function driftBadge(s: SnippetListItem): string {
const v = s.verification;
if (!v) return "";
if (!v.current) return "unchecked since edit";
return { ok: "", missing: "path gone", moved: "symbol moved", changed: "code drifted" }[
v.status
] ?? "";
}
function driftTitle(s: SnippetListItem): string {
const v = s.verification;
if (!v) return "";
const when = v.checked_at
? `Checked ${new Date(v.checked_at).toLocaleDateString()}`
: "Checked";
if (!v.current) {
return (
`${when}, but the snippet has been edited since — that verdict was about ` +
`code this record no longer holds. Re-verify it.`
);
}
const what =
{
missing: "the recorded path no longer exists",
moved: "the file is there but the symbol isn't in it",
changed: "the source no longer matches the recorded code",
}[v.status] ?? "";
return v.detail ? `${when}: ${what}. ${v.detail}` : `${when}: ${what}.`;
}
function usageTitle(s: SnippetListItem): string { function usageTitle(s: SnippetListItem): string {
const u = s.usage; const u = s.usage;
if (!u) return ""; if (!u) return "";
@@ -201,6 +244,15 @@ function usageTitle(s: SnippetListItem): string {
a screen reader too. --> a screen reader too. -->
{{ locationActive ? "Location · filtering" : "Location" }} {{ locationActive ? "Location · filtering" : "Location" }}
</button> </button>
<button
class="btn-ghost"
:class="{ 'filter-on': needsAttentionOnly }"
:aria-pressed="needsAttentionOnly"
title="Show only snippets whose recorded location or code no longer checks out — including ones edited since they were last verified"
@click="toggleAttention"
>
{{ needsAttentionOnly ? "Needs attention · filtering" : "Needs attention" }}
</button>
</div> </div>
<!-- Reverse lookup: what's already kept in this repo / file / symbol. --> <!-- Reverse lookup: what's already kept in this repo / file / symbol. -->
@@ -243,20 +295,27 @@ function usageTitle(s: SnippetListItem): string {
<div v-else-if="snippets.length === 0" class="empty-state-rich"> <div v-else-if="snippets.length === 0" class="empty-state-rich">
<div class="empty-icon">_</div> <div class="empty-icon">_</div>
<p class="empty-title"> <p class="empty-title">
{{ locationActive {{ needsAttentionOnly
? "Nothing kept at that location yet" ? "Everything checks out"
: search.trim() : locationActive
? "No snippets match your search" ? "Nothing kept at that location yet"
: "No snippets kept yet" }} : search.trim()
? "No snippets match your search"
: "No snippets kept yet" }}
</p> </p>
<p class="empty-sub"> <p class="empty-sub">
{{ locationActive {{ needsAttentionOnly
? "No recorded snippet lives there — so whatever you're about to write is new. Widen the path, or clear the filter." ? "No snippet has drifted from its recorded location or code — as far as anything has been checked. Snippets nobody has verified yet don't appear here."
: search.trim() : locationActive
? "Try a different term, or clear the search." ? "No recorded snippet lives there — so whatever you're about to write is new. Widen the path, or clear the filter."
: "Record a reusable function or component and it will be offered back to you later." }} : search.trim()
? "Try a different term, or clear the search."
: "Record a reusable function or component and it will be offered back to you later." }}
</p> </p>
<button v-if="locationActive" class="empty-action" @click="clearLocation"> <button v-if="needsAttentionOnly" class="empty-action" @click="toggleAttention">
Show all snippets
</button>
<button v-else-if="locationActive" class="empty-action" @click="clearLocation">
Clear location filter Clear location filter
</button> </button>
<button <button
@@ -295,6 +354,9 @@ function usageTitle(s: SnippetListItem): string {
</p> </p>
<div class="card-footer"> <div class="card-footer">
<span class="meta-date">Updated {{ new Date(s.updated_at).toLocaleDateString() }}</span> <span class="meta-date">Updated {{ new Date(s.updated_at).toLocaleDateString() }}</span>
<span v-if="driftBadge(s)" class="drift-tag" :title="driftTitle(s)">
{{ driftBadge(s) }}
</span>
<span <span
v-if="usageBadge(s)" v-if="usageBadge(s)"
class="usage-tag" class="usage-tag"
@@ -618,6 +680,17 @@ function usageTitle(s: SnippetListItem): string {
color: var(--color-text-muted); color: var(--color-text-muted);
} }
/* Drift is a stronger signal than dead weight: the record may be actively
misleading, not merely unused. Danger tone, and it sits first in the footer. */
.drift-tag {
font-size: 0.7rem;
padding: 0.1rem 0.4rem;
border-radius: 4px;
white-space: nowrap;
background: color-mix(in srgb, var(--color-danger, #b91c1c) 15%, transparent);
color: var(--color-danger, #b91c1c);
}
.usage-tag { .usage-tag {
font-size: 0.7rem; font-size: 0.7rem;
padding: 0.1rem 0.4rem; padding: 0.1rem 0.4rem;
+61 -3
View File
@@ -20,7 +20,7 @@ from scribe.services import systems as systems_svc
async def list_snippets( async def list_snippets(
q: str = "", tag: str = "", limit: int = 50, project_id: int = 0, q: str = "", tag: str = "", limit: int = 50, project_id: int = 0,
repo: str = "", path: str = "", symbol: str = "", repo: str = "", path: str = "", symbol: str = "", verification: str = "",
) -> dict: ) -> dict:
"""List recorded snippets (reusable functions/components). """List recorded snippets (reusable functions/components).
@@ -44,6 +44,12 @@ async def list_snippets(
OR anything beneath it, so "frontend/src" finds OR anything beneath it, so "frontend/src" finds
"frontend/src/lib/x.ts" as well as itself. "frontend/src/lib/x.ts" as well as itself.
symbol: Narrow to snippets recorded under this symbol name, exactly. symbol: Narrow to snippets recorded under this symbol name, exactly.
verification: Narrow on the drift check (see verify_snippet).
"attention" is the one to reach for — everything whose recorded
location or code no longer checks out, plus everything whose verdict
expired because the snippet was edited after it was checked. Also
accepts "ok", "unverified", "drifted", or a specific failure:
"missing", "moved", "changed".
`repo`/`path`/`symbol` must all match the SAME recorded location, so a `repo`/`path`/`symbol` must all match the SAME recorded location, so a
snippet that lives in repo A and, separately, at path B in another repo is snippet that lives in repo A and, separately, at path B in another repo is
@@ -70,7 +76,7 @@ async def list_snippets(
items, total = await snippets_svc.list_snippets( items, total = await snippets_svc.list_snippets(
uid, q=q or None, tag=tag, limit=max(1, min(limit, 100)), uid, q=q or None, tag=tag, limit=max(1, min(limit, 100)),
project_id=project_id or None, project_id=project_id or None,
repo=repo, path=path, symbol=symbol, repo=repo, path=path, symbol=symbol, verification=verification,
) )
labeled = await access_svc.label_shared_items(uid, items) labeled = await access_svc.label_shared_items(uid, items)
usage = await usage_for_notes([int(it["id"]) for it in labeled]) usage = await usage_for_notes([int(it["id"]) for it in labeled])
@@ -184,6 +190,58 @@ async def get_snippet(snippet_id: int) -> dict:
return data return data
async def verify_snippet(
snippet_id: int, status: str, detail: str = "", path: str = "",
) -> dict:
"""Record whether a snippet's recorded location and code still match source.
YOU do the checking — Scribe has no copy of the repo and deliberately never
gets one. This tool only remembers your verdict so it becomes queryable and
so the operator can see what has rotted.
The procedure, once per snippet you're checking:
1. `get_snippet(id)` — read its `snippet.locations` and `snippet.code`.
2. Does the recorded path still exist in the working tree? If not →
status="missing".
3. Does the recorded symbol still appear in that file? If not →
status="moved" (the file is there, the thing isn't).
4. Does the source still match the recorded code, allowing for formatting?
Judge whether it still does the same thing — an added parameter or a
changed branch is "changed"; a reindent is not. If it diverged →
status="changed".
5. All three hold → status="ok".
Put what you actually found in `detail` ("renamed to parse_location_str",
"moved to services/knowledge.py"). It's what makes the record fixable later
by someone who wasn't here, so write it for them, not as a status echo.
A verdict expires automatically if the snippet is edited afterwards: it is
stamped with a hash of the code it was checked against, so it can never go
on vouching for code nobody checked. Re-verify after fixing a record.
Args:
snippet_id: The snippet you checked.
status: "ok" | "missing" | "moved" | "changed".
detail: What you found — free text, shown to the operator.
path: The path you actually checked, if it differs from the recorded
one (e.g. you found the symbol at its new home). Defaults to the
recorded path.
Requires write access: a verdict changes how the record is presented, so
being able to read a snippet someone shared with you doesn't let you mark
it broken.
"""
uid = current_user_id()
note = await snippets_svc.record_verification(
uid, snippet_id, status=status, detail=detail, path=path,
)
if note is None:
raise ValueError(
f"snippet {snippet_id} not found, or you don't have write access to it"
)
return snippets_svc.snippet_to_dict(note)
async def update_snippet( async def update_snippet(
snippet_id: int, snippet_id: int,
name: str | None = None, name: str | None = None,
@@ -307,6 +365,6 @@ async def merge_snippets(target_id: int, source_ids: list[int]) -> dict:
def register(mcp) -> None: def register(mcp) -> None:
for fn in ( for fn in (
list_snippets, create_snippet, get_snippet, update_snippet, list_snippets, create_snippet, get_snippet, update_snippet,
delete_snippet, merge_snippets, delete_snippet, merge_snippets, verify_snippet,
): ):
mcp.tool(name=fn.__name__)(fn) mcp.tool(name=fn.__name__)(fn)
+41 -1
View File
@@ -62,10 +62,12 @@ async def list_snippets_route():
repo = request.args.get("repo", "") repo = request.args.get("repo", "")
path = request.args.get("path", "") path = request.args.get("path", "")
symbol = request.args.get("symbol", "") symbol = request.args.get("symbol", "")
# Drift check (#2086). "attention" is what the UI's filter chip sends.
verification = request.args.get("verification", "")
limit, offset = parse_pagination() limit, offset = parse_pagination()
items, total = await snippets_svc.list_snippets( items, total = await snippets_svc.list_snippets(
uid, q=q, tag=tag, limit=limit, offset=offset, project_id=project_id, uid, q=q, tag=tag, limit=limit, offset=offset, project_id=project_id,
repo=repo, path=path, symbol=symbol, repo=repo, path=path, symbol=symbol, verification=verification,
) )
# Mark rows owned by someone else so the UI can show whose they are — an # Mark rows owned by someone else so the UI can show whose they are — an
# unmarked row in your own list reads as one you recorded and vetted. # unmarked row in your own list reads as one you recorded and vetted.
@@ -203,6 +205,44 @@ async def update_snippet_route(snippet_id: int):
return jsonify(out) return jsonify(out)
@snippets_bp.route("/<int:snippet_id>/verify", methods=["POST"])
@login_required
async def verify_snippet_route(snippet_id: int):
"""Record a drift-check verdict. Body: {"status": ..., "detail", "path"}.
The CHECK itself runs wherever the code is — an agent with the working tree
— because Scribe has no checkout and shouldn't have one. This endpoint just
stores what was found. It's here for parity with the MCP tool and so the UI
can clear a stale marker after the operator fixes a record by hand."""
uid = get_current_user_id()
if await _load_snippet(uid, snippet_id) is None:
return not_found("Snippet")
# Distinguished from not-found deliberately: the service returns None for
# both, and telling a shared reader "no such snippet" about one they can
# plainly see is a confusing lie.
if not await can_write_note(uid, snippet_id):
return jsonify({"error": "Permission denied"}), 403
data = await request.get_json() or {}
status = (data.get("status") or "").strip()
if not status:
return jsonify({"error": "status is required"}), 400
try:
updated = await snippets_svc.record_verification(
uid, snippet_id,
status=status,
detail=data.get("detail") or "",
path=data.get("path") or "",
)
except ValueError as exc:
# An unknown status — reject it rather than storing a value the filter
# would then never match.
return jsonify({"error": str(exc)}), 400
if updated is None:
return not_found("Snippet")
return jsonify(snippets_svc.snippet_to_dict(updated))
@snippets_bp.route("/<int:snippet_id>/merge", methods=["POST"]) @snippets_bp.route("/<int:snippet_id>/merge", methods=["POST"])
@login_required @login_required
async def merge_snippet_route(snippet_id: int): async def merge_snippet_route(snippet_id: int):
+110 -1
View File
@@ -18,7 +18,7 @@ in your ambient lists.
import json import json
import logging import logging
from sqlalchemy import func, select from sqlalchemy import and_, func, or_, select
from scribe.models import async_session from scribe.models import async_session
from scribe.models.note import Note from scribe.models.note import Note
@@ -108,6 +108,83 @@ def _location_clause(parts: dict[str, str]):
return Note.data.path_exists(location_jsonpath(parts)) return Note.data.path_exists(location_jsonpath(parts))
# --- drift-check filter (#2086) ----------------------------------------------
# `verification` selects on the drift-check verdict stored in `data.verification`
# (see services/snippets.py). Statuses are the service's own constants; the two
# composite values are what the operator actually asks for.
#
# The interesting one is `attention`. A verdict describes the code it was checked
# against, so an OK verdict on code that has since been edited is not an OK
# record — nobody has checked what's actually there. That is expressible in SQL
# only because `data.code_sha` mirrors the current code's fingerprint alongside
# the verdict's: jsonpath compares the two fields within the row, so this stays
# one index-served predicate rather than a post-filter that would make the
# pagination total a lie.
def verification_matches(data: dict | None, value: str) -> bool:
"""Python dialect of the verification predicate.
Keep in step with _verification_clause — the semantic arm's candidates are
already fetched, so there is no query left to narrow and the same rule has
to be expressible twice. Same arrangement as location_matches.
"""
want = (value or "").strip().lower()
if not want:
return True
verdict = (data or {}).get("verification") or {}
status = verdict.get("status") or ""
if not status:
return want == "unverified"
expired = verdict.get("code_sha") != (data or {}).get("code_sha")
if want == "unverified":
return False
if want == "drifted":
return status != "ok"
if want == "attention":
return status != "ok" or expired
if want == "ok":
return status == "ok" and not expired
return status == want
_VERIFY_DRIFTED_JSONPATH = '$.verification ? (@.status != "ok")'
_VERIFY_EXPIRED_JSONPATH = "$ ? (@.verification.code_sha != @.code_sha)"
_VERIFY_ANY_JSONPATH = "$.verification"
def _verification_clause(value: str):
"""SQL predicate for one `verification` filter value, or None for no filter."""
want = (value or "").strip().lower()
if not want:
return None
has_verdict = Note.data.path_exists(_VERIFY_ANY_JSONPATH)
if want == "unverified":
# Never checked at all. Rows predating migration 0070 have no `data`
# whatsoever and land here correctly — which is right, they haven't been.
return ~has_verdict
if want == "drifted":
return Note.data.path_exists(_VERIFY_DRIFTED_JSONPATH)
if want == "attention":
# Everything worth looking at: a failing verdict, OR an expired one.
return or_(
Note.data.path_exists(_VERIFY_DRIFTED_JSONPATH),
and_(has_verdict, Note.data.path_exists(_VERIFY_EXPIRED_JSONPATH)),
)
if want == "ok":
# A clean bill of health that still describes the current code. The
# `~expired` half matters: without it this would quietly include records
# whose blessing has lapsed, which is the exact failure the feature is
# meant to catch.
return and_(
Note.data.path_exists('$.verification ? (@.status == "ok")'),
~Note.data.path_exists(_VERIFY_EXPIRED_JSONPATH),
)
# A specific status: 'missing' | 'moved' | 'changed'.
return Note.data.path_exists(
f"$.verification ? (@.status == {json.dumps(want)})"
)
def _note_to_item(note: Note) -> dict: def _note_to_item(note: Note) -> dict:
item: dict = { item: dict = {
"id": note.id, "id": note.id,
@@ -122,6 +199,22 @@ def _note_to_item(note: Note) -> dict:
"created_at": note.created_at.isoformat(), "created_at": note.created_at.isoformat(),
"updated_at": note.updated_at.isoformat(), "updated_at": note.updated_at.isoformat(),
} }
# Drift verdict (#2086), when one has been recorded. Included here rather
# than decorated on by the snippet layer because `current` is derivable from
# `data` alone — the verdict's code_sha against the row's — so this needs no
# body parsing and stays a plain projection of the column. Omitted entirely
# when unchecked, so "no key" and "never verified" don't become two states
# the client has to tell apart.
verdict = (note.data or {}).get("verification") if note.data else None
if verdict and verdict.get("status"):
item["verification"] = {
"status": verdict["status"],
"current": verdict.get("code_sha") == (note.data or {}).get("code_sha"),
"checked_at": verdict.get("checked_at"),
"detail": verdict.get("detail"),
"path": verdict.get("path"),
}
# Task fields — override note_type and add status/priority/due_date # Task fields — override note_type and add status/priority/due_date
if note.is_task: if note.is_task:
item["note_type"] = "task" item["note_type"] = "task"
@@ -161,6 +254,7 @@ async def query_knowledge(
offset: int, offset: int,
project_id: int | None = None, project_id: int | None = None,
locations: dict[str, str] | None = None, locations: dict[str, str] | None = None,
verification: str = "",
) -> tuple[list[dict], int]: ) -> tuple[list[dict], int]:
"""Query knowledge objects (non-task notes) with filters. """Query knowledge objects (non-task notes) with filters.
@@ -171,6 +265,10 @@ async def query_knowledge(
Today only snippets carry locations, but the column is general, so the filter Today only snippets carry locations, but the column is general, so the filter
lives here with the query rather than in one type's service. lives here with the query rather than in one type's service.
`verification` narrows on the drift-check verdict: 'ok', 'drifted',
'unverified', 'attention', or one specific failure ('missing' | 'moved' |
'changed'). Empty means no filter.
Returns (items, total_count). Returns (items, total_count).
""" """
# Semantic search path — scores take priority over sort # Semantic search path — scores take priority over sort
@@ -178,6 +276,7 @@ async def query_knowledge(
return await _semantic_knowledge_search( return await _semantic_knowledge_search(
user_id, q, note_type=note_type, tags=tags, limit=limit, user_id, q, note_type=note_type, tags=tags, limit=limit,
offset=offset, project_id=project_id, locations=locations, offset=offset, project_id=project_id, locations=locations,
verification=verification,
) )
# No query = browsing. Narrower scope: a record shared directly with the # No query = browsing. Narrower scope: a record shared directly with the
@@ -196,6 +295,10 @@ async def query_knowledge(
if locations: if locations:
base = base.where(_location_clause(locations)) base = base.where(_location_clause(locations))
verify_clause = _verification_clause(verification)
if verify_clause is not None:
base = base.where(verify_clause)
# Count before pagination # Count before pagination
count_stmt = select(func.count()).select_from(base.subquery()) count_stmt = select(func.count()).select_from(base.subquery())
total: int = (await session.execute(count_stmt)).scalar_one() total: int = (await session.execute(count_stmt)).scalar_one()
@@ -224,6 +327,7 @@ async def _semantic_knowledge_search(
offset: int, offset: int,
project_id: int | None = None, project_id: int | None = None,
locations: dict[str, str] | None = None, locations: dict[str, str] | None = None,
verification: str = "",
) -> tuple[list[dict], int]: ) -> tuple[list[dict], int]:
"""Hybrid search: keyword matches first (title/body ILIKE), then semantic results. """Hybrid search: keyword matches first (title/body ILIKE), then semantic results.
@@ -259,6 +363,9 @@ async def _semantic_knowledge_search(
base = base.where(Note.tags.contains([tag])) base = base.where(Note.tags.contains([tag]))
if locations: if locations:
base = base.where(_location_clause(locations)) base = base.where(_location_clause(locations))
verify_clause = _verification_clause(verification)
if verify_clause is not None:
base = base.where(verify_clause)
# Title matches first, then body-only matches, newest first within each # Title matches first, then body-only matches, newest first within each
base = base.order_by( base = base.order_by(
Note.title.ilike(pattern).desc(), Note.title.ilike(pattern).desc(),
@@ -301,6 +408,8 @@ async def _semantic_knowledge_search(
# narrow. See the comment on location_matches. # narrow. See the comment on location_matches.
if locations and not location_matches(note.data, locations): if locations and not location_matches(note.data, locations):
continue continue
if verification and not verification_matches(note.data, verification):
continue
semantic_notes.append(note) semantic_notes.append(note)
except Exception: except Exception:
logger.warning("Semantic search unavailable, using keyword results only", exc_info=True) logger.warning("Semantic search unavailable, using keyword results only", exc_info=True)
+191 -5
View File
@@ -29,8 +29,10 @@ came from.
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import hashlib
import logging import logging
import re import re
from datetime import datetime, timezone
from sqlalchemy import func, or_, select from sqlalchemy import func, or_, select
@@ -303,8 +305,104 @@ def parse_snippet_fields(
# and copying a blob into the column we index *around* would be pure weight. # and copying a blob into the column we index *around* would be pure weight.
_DATA_FIELDS = ( _DATA_FIELDS = (
"name", "when_to_use", "signature", "language", "locations", "merged_from", "name", "when_to_use", "signature", "language", "locations", "merged_from",
"verification",
) )
# --- drift check (#2086) -----------------------------------------------------
# A recorded snippet points at a repo · path · symbol that WILL rot: files move,
# symbols get renamed, implementations diverge from the copy stored here. Left
# undetected, the record degrades from "canonical reference" to "confidently
# wrong" — which is worse than having no record, because it is surfaced with the
# same authority either way.
#
# WHERE THE CHECK RUNS. Not here. Scribe has no checkout of the operator's repos
# and must not acquire one (rule #115 — the instance stays agnostic about where
# code lives; giving the server repo access would make every install a
# credential problem). The agent already has the working tree, so IT does the
# comparing and reports a verdict; the server's job is to remember the verdict,
# make it queryable, and know when it has expired.
#
# WHY THE VERDICT CARRIES A CODE HASH. A stored verdict describes the code it
# was checked against. Edit the snippet afterwards and that verdict is no longer
# about anything — but invalidating it on write means deciding which edits count
# (a `when_to_use` tweak shouldn't void a code check; a code rewrite must). That
# rule is fiddly and easy to get subtly wrong. Recording the hash sidesteps it
# entirely: a verdict whose `code_sha` no longer matches the body is self-
# evidently expired, computed at read time, with no invalidation logic to
# maintain and no way for an edit path to forget to call it.
VERIFY_OK = "ok"
VERIFY_MISSING = "missing" # the recorded path is gone
VERIFY_MOVED = "moved" # path is there, the symbol isn't in it
VERIFY_CHANGED = "changed" # both present, but the source no longer matches
VERIFY_STATUSES = (VERIFY_OK, VERIFY_MISSING, VERIFY_MOVED, VERIFY_CHANGED)
# Everything that isn't a clean bill of health. "Stale" in the UI and the filter
# means this set — the operator wants one list of things to look at, not four.
VERIFY_DRIFTED = (VERIFY_MISSING, VERIFY_MOVED, VERIFY_CHANGED)
def code_sha(code: str) -> str:
"""Stable fingerprint of a snippet's code, for expiring stale verdicts.
Trailing whitespace per line and leading/trailing blank lines are stripped
before hashing: those change when a file is reformatted without the code
meaning anything different, and a verdict shouldn't expire over an editor's
trailing-newline habit.
"""
normalized = "\n".join(line.rstrip() for line in (code or "").splitlines()).strip()
return hashlib.sha256(normalized.encode("utf-8")).hexdigest()[:32]
def compose_verification(
*,
status: str,
checked_code_sha: str,
detail: str = "",
path: str = "",
checked_at: str = "",
) -> dict:
"""Build the `data.verification` record. Unknown statuses are rejected here
rather than stored, so the filter never has to cope with a typo'd status."""
if status not in VERIFY_STATUSES:
raise ValueError(
f"unknown verification status {status!r} — expected one of {VERIFY_STATUSES}"
)
out = {
"status": status,
"code_sha": checked_code_sha,
"checked_at": checked_at or datetime.now(timezone.utc).isoformat(),
}
if (detail or "").strip():
out["detail"] = detail.strip()
if (path or "").strip():
out["path"] = path.strip()
return out
def verification_view(note, fields: dict) -> dict:
"""The verification readout for one snippet, including whether it's expired.
`status` is what was last reported; `current` says whether that verdict still
describes the code in the record. A verdict that no longer matches reads as
unverified, because that is what it is — nobody has checked THIS code.
"""
stored = (fields.get("verification") or {}) if isinstance(fields, dict) else {}
if not stored or not stored.get("status"):
return {"status": "unverified", "current": False, "checked_at": None}
current = stored.get("code_sha") == code_sha(fields.get("code") or "")
return {
"status": stored["status"],
"current": current,
"checked_at": stored.get("checked_at"),
"detail": stored.get("detail"),
"path": stored.get("path"),
# What the operator actually wants to know: is there something to fix?
# An expired verdict counts as "needs looking at" even if it said ok,
# since the code it blessed is not the code that's there now.
"needs_attention": (not current) or stored["status"] in VERIFY_DRIFTED,
}
def compose_data( def compose_data(
*, *,
@@ -312,8 +410,10 @@ def compose_data(
when_to_use: str = "", when_to_use: str = "",
signature: str = "", signature: str = "",
language: str = "", language: str = "",
code: str = "",
locations: list[dict] | None = None, locations: list[dict] | None = None,
merged_from: list[int] | None = None, merged_from: list[int] | None = None,
verification: dict | None = None,
) -> dict: ) -> dict:
"""Build the `notes.data` mirror of a snippet's structured fields. """Build the `notes.data` mirror of a snippet's structured fields.
@@ -337,6 +437,19 @@ def compose_data(
merged = _normalize_merged_from(merged_from) merged = _normalize_merged_from(merged_from)
if merged: if merged:
out["merged_from"] = merged out["merged_from"] = merged
# Carried, never composed here — like merged_from. An ordinary edit must not
# silently drop the last drift check, and it doesn't need to invalidate it
# either: the verdict's code_sha expires it on read if the code moved on.
if verification:
out["verification"] = verification
# The current code's fingerprint — NOT the code, which stays in the body
# (see _DATA_FIELDS). Its only job is to make "this verdict has expired"
# expressible in SQL: a jsonpath can compare `@.verification.code_sha` to
# `@.code_sha` within the same row, so "show me everything that needs
# looking at" stays one index-served query instead of a post-filter that
# would break pagination counts.
if (code or "").strip():
out["code_sha"] = code_sha(code)
return out return out
@@ -419,6 +532,7 @@ async def backfill_snippet_data(*, batch: int = 500) -> int:
when_to_use=fields["when_to_use"], when_to_use=fields["when_to_use"],
signature=fields["signature"], signature=fields["signature"],
language=fields["language"], language=fields["language"],
code=fields["code"],
locations=fields["locations"], locations=fields["locations"],
merged_from=fields["merged_from"], merged_from=fields["merged_from"],
) )
@@ -439,7 +553,13 @@ def snippet_to_dict(note) -> dict:
``snippet`` already reports, and shipping both would give API consumers two ``snippet`` already reports, and shipping both would give API consumers two
sources of truth for the same facts.""" sources of truth for the same facts."""
data = note.to_dict() data = note.to_dict()
data["snippet"] = snippet_fields(note) fields = snippet_fields(note)
data["snippet"] = fields
# Promoted out of `snippet` because it is a computed READOUT, not a recorded
# field: `current` and `needs_attention` are derived at read time by hashing
# the code, and burying them among the stored fields would invite a caller
# to try writing them back.
data["verification"] = verification_view(note, fields)
return data return data
@@ -479,7 +599,7 @@ async def create_snippet(
# body so the two can never describe different things. # body so the two can never describe different things.
data=compose_data( data=compose_data(
name=name, when_to_use=when_to_use, signature=signature, name=name, when_to_use=when_to_use, signature=signature,
language=language, locations=locations, language=language, code=code, locations=locations,
), ),
) )
_embed_snippet(note) _embed_snippet(note)
@@ -513,6 +633,7 @@ async def list_snippets(
repo: str = "", repo: str = "",
path: str = "", path: str = "",
symbol: str = "", symbol: str = "",
verification: str = "",
) -> tuple[list[dict], int]: ) -> tuple[list[dict], int]:
"""List snippets (id/title/tags/preview dicts), most-recently-updated first. """List snippets (id/title/tags/preview dicts), most-recently-updated first.
@@ -523,7 +644,11 @@ async def list_snippets(
``repo`` / ``path`` / ``symbol`` are the reverse lookup: "what canonical ``repo`` / ``path`` / ``symbol`` are the reverse lookup: "what canonical
helpers already live here?" They narrow to snippets recorded at a matching helpers already live here?" They narrow to snippets recorded at a matching
location, ANDed within one location entry, with ``path`` also matching as a location, ANDed within one location entry, with ``path`` also matching as a
directory prefix. Combinable with ``q`` — search *and* place.""" directory prefix. Combinable with ``q`` — search *and* place.
``verification`` narrows on the drift check: ``attention`` is the useful one
— everything whose recorded location or code no longer checks out, plus
everything whose verdict expired because the snippet was edited since."""
return await knowledge_svc.query_knowledge( return await knowledge_svc.query_knowledge(
user_id=user_id, user_id=user_id,
note_type=SNIPPET_NOTE_TYPE, note_type=SNIPPET_NOTE_TYPE,
@@ -535,6 +660,7 @@ async def list_snippets(
project_id=project_id, project_id=project_id,
locations=knowledge_svc.location_parts(repo=repo, path=path, symbol=symbol) locations=knowledge_svc.location_parts(repo=repo, path=path, symbol=symbol)
or None, or None,
verification=verification,
) )
@@ -615,8 +741,12 @@ async def update_snippet(
"data": compose_data( "data": compose_data(
name=merged["name"], when_to_use=merged["when_to_use"], name=merged["name"], when_to_use=merged["when_to_use"],
signature=merged["signature"], language=merged["language"], signature=merged["signature"], language=merged["language"],
locations=merged_locations, code=merged["code"], locations=merged_locations,
merged_from=merged.get("merged_from"), merged_from=merged.get("merged_from"),
# Carried through the edit rather than cleared. If this edit changed
# the code, the verdict's code_sha stops matching and it reads as
# unverified from here on — no invalidation branch to get wrong.
verification=merged.get("verification"),
), ),
} }
# Recompute tags: keep any non-language, non-marker tags the note already had # Recompute tags: keep any non-language, non-marker tags the note already had
@@ -639,6 +769,59 @@ async def update_snippet(
return updated return updated
async def record_verification(
user_id: int,
snippet_id: int,
*,
status: str,
detail: str = "",
path: str = "",
):
"""Record the result of a drift check against the snippet's source.
The CHECK happens agent-side — Scribe has no checkout and shouldn't want one
(see the drift-check note above). This just remembers the verdict, stamped
with a hash of the code it was checked against so it expires by itself when
the snippet is edited.
Requires WRITE access: a verdict changes how the record is presented and
whether it shows up in the operator's "needs attention" list, so being able
to read a shared snippet must not let you mark it broken.
Returns the updated note, or None if the id isn't a snippet this user may
write. Raises ValueError on an unknown status.
"""
note = await get_snippet(user_id, snippet_id)
if note is None:
return None
from scribe.services.access import can_write_note
if not await can_write_note(user_id, snippet_id):
return None
fields = snippet_fields(note)
verification = compose_verification(
status=status,
checked_code_sha=code_sha(fields.get("code") or ""),
detail=detail,
path=path or fields.get("path") or "",
)
# Rebuilt from the CURRENT stored fields plus the new verdict, so recording a
# check can't quietly rewrite anything else about the record. Note the body
# is untouched — a verdict is metadata about the snippet, not part of it, and
# writing it into the body would put it into the embedding.
data = compose_data(
name=fields.get("name", ""),
when_to_use=fields.get("when_to_use", ""),
signature=fields.get("signature", ""),
language=fields.get("language", ""),
code=fields.get("code", ""),
locations=fields.get("locations") or [],
merged_from=fields.get("merged_from") or [],
verification=verification,
)
return await notes_svc.update_note(note.user_id, snippet_id, data=data)
async def delete_snippet(user_id: int, snippet_id: int) -> bool: async def delete_snippet(user_id: int, snippet_id: int) -> bool:
"""Retire a snippet to the trash (recoverable). Returns False if the id isn't """Retire a snippet to the trash (recoverable). Returns False if the id isn't
a snippet this user may WRITE. a snippet this user may WRITE.
@@ -760,7 +943,10 @@ async def merge_snippets(user_id: int, target_id: int, source_ids: list[int]):
data=compose_data( data=compose_data(
name=tgt_fields["name"], when_to_use=tgt_fields["when_to_use"], name=tgt_fields["name"], when_to_use=tgt_fields["when_to_use"],
signature=tgt_fields["signature"], language=tgt_fields["language"], signature=tgt_fields["signature"], language=tgt_fields["language"],
locations=locations, merged_from=merged_from, code=tgt_fields["code"], locations=locations, merged_from=merged_from,
# No verification carried: the survivor's code is a union of several
# sources, so no prior verdict describes it. It reads as unverified,
# which is the honest answer — nobody has checked THIS code.
), ),
) )
if updated is None: if updated is None:
+251
View File
@@ -0,0 +1,251 @@
"""Tests for the drift check (#2086) — does a snippet still match its source?
The check itself runs agent-side (Scribe has no checkout), so what's testable
here is everything around it: the verdict's shape, the rule that a verdict
expires when the code it blessed is edited away, and the two dialects of the
filter predicate — SQL for the browse/keyword arms, Python for the semantic
arm's already-fetched candidates.
"""
from types import SimpleNamespace
import pytest
from scribe.services import knowledge as knowledge_svc
from scribe.services import snippets as snippets_svc
from scribe.services.snippets import (
code_sha,
compose_data,
compose_verification,
verification_view,
)
def _note(data=None, body=""):
return SimpleNamespace(data=data, body=body, title="n — w", tags=["snippet"])
# --- the verdict record ---------------------------------------------------
def test_unknown_status_is_rejected_not_stored():
"""A typo'd status would be stored happily and then never match any filter —
invisible rot in the thing built to make rot visible."""
with pytest.raises(ValueError):
compose_verification(status="broekn", checked_code_sha="abc")
@pytest.mark.parametrize("status", ["ok", "missing", "moved", "changed"])
def test_every_documented_status_is_accepted(status):
out = compose_verification(status=status, checked_code_sha="abc")
assert out["status"] == status
assert out["code_sha"] == "abc"
assert out["checked_at"]
def test_blank_detail_and_path_are_omitted_not_stored_empty():
"""Keeps `data` sparse, and keeps a containment match from tripping on ""."""
out = compose_verification(status="ok", checked_code_sha="abc")
assert "detail" not in out and "path" not in out
# --- expiry by edit -------------------------------------------------------
def test_code_sha_ignores_reformatting_but_not_meaning():
"""A verdict must not expire because an editor stripped trailing whitespace,
and must expire when the code actually changes."""
assert code_sha("def f():\n return 1") == code_sha("def f(): \n return 1\n")
assert code_sha("def f():\n return 1") != code_sha("def f():\n return 2")
def test_verdict_stays_current_while_code_is_unchanged():
code = "def f():\n return 1"
fields = {"code": code, "verification": compose_verification(
status="ok", checked_code_sha=code_sha(code)
)}
view = verification_view(_note(), fields)
assert view["status"] == "ok"
assert view["current"] is True
assert view["needs_attention"] is False
def test_an_ok_verdict_expires_once_the_code_is_edited():
"""The whole reason the verdict carries a hash: an OK verdict must never go
on vouching for code nobody checked."""
fields = {"code": "def f():\n return 2", "verification": compose_verification(
status="ok", checked_code_sha=code_sha("def f():\n return 1")
)}
view = verification_view(_note(), fields)
assert view["current"] is False
assert view["needs_attention"] is True
def test_never_checked_reads_as_unverified():
view = verification_view(_note(), {"code": "x = 1"})
assert view["status"] == "unverified"
assert view["current"] is False
def test_a_failing_verdict_needs_attention_even_while_current():
code = "x = 1"
fields = {"code": code, "verification": compose_verification(
status="missing", checked_code_sha=code_sha(code)
)}
assert verification_view(_note(), fields)["needs_attention"] is True
# --- the data mirror ------------------------------------------------------
def test_compose_data_mirrors_the_current_code_hash():
"""`data.code_sha` is what makes "this verdict expired" an SQL-expressible
comparison rather than a post-filter that would break pagination totals."""
out = compose_data(name="n", code="def f():\n return 1")
assert out["code_sha"] == code_sha("def f():\n return 1")
def test_compose_data_omits_the_hash_when_there_is_no_code():
assert "code_sha" not in compose_data(name="n")
def test_the_code_itself_is_never_copied_into_data():
"""data indexes AROUND the code; duplicating the blob would be pure weight."""
out = compose_data(name="n", code="secret_looking_blob")
assert "code" not in out
assert "secret_looking_blob" not in str(out)
# --- the filter, Python dialect -------------------------------------------
def _data(status, checked_sha, current_sha):
return {
"verification": {"status": status, "code_sha": checked_sha},
"code_sha": current_sha,
}
@pytest.mark.parametrize(
"value, expected",
[
("", True), # no filter
("ok", True),
("attention", False),
("drifted", False),
("unverified", False),
],
)
def test_python_dialect_on_a_current_ok_verdict(value, expected):
data = _data("ok", "aaa", "aaa")
assert knowledge_svc.verification_matches(data, value) is expected
@pytest.mark.parametrize(
"value, expected",
[("ok", False), ("attention", True), ("drifted", False), ("unverified", False)],
)
def test_python_dialect_on_an_expired_ok_verdict(value, expected):
"""Expired-but-ok is the case that motivated `attention` existing at all: it
is not `drifted` (nothing was found wrong) and not `unverified` (a check did
happen), yet it plainly needs looking at."""
data = _data("ok", "aaa", "bbb")
assert knowledge_svc.verification_matches(data, value) is expected
@pytest.mark.parametrize(
"value, expected",
[("ok", False), ("attention", True), ("drifted", True), ("changed", True),
("missing", False)],
)
def test_python_dialect_on_a_drifted_verdict(value, expected):
data = _data("changed", "aaa", "aaa")
assert knowledge_svc.verification_matches(data, value) is expected
def test_python_dialect_on_a_row_with_no_data_at_all():
"""Rows predating migration 0070 have no `data`. They are unverified, which
is true — nobody has checked them."""
assert knowledge_svc.verification_matches(None, "unverified") is True
assert knowledge_svc.verification_matches(None, "attention") is False
assert knowledge_svc.verification_matches(None, "ok") is False
# --- the filter, SQL dialect ----------------------------------------------
def test_no_filter_produces_no_clause():
assert knowledge_svc._verification_clause("") is None
@pytest.mark.parametrize(
"value", ["ok", "drifted", "unverified", "attention", "missing", "moved", "changed"]
)
def test_every_filter_value_compiles_to_sql(value):
"""The two dialects have to agree, and the SQL one has to be expressible at
all — the expired comparison relies on jsonpath reading two fields of the
same row, which is easy to get wrong silently."""
clause = knowledge_svc._verification_clause(value)
assert clause is not None
assert str(clause)
def test_a_status_value_is_json_quoted_into_the_jsonpath():
"""Statuses are validated on WRITE, but the filter takes caller input
directly — a quote must not be able to break out of the jsonpath and turn
a narrowing filter into a widening one."""
hostile = 'x" || @.status == "ok'
clause = knowledge_svc._verification_clause(hostile)
params = list(clause.compile().params.values())
jsonpath = next(p for p in params if isinstance(p, str) and "status" in p)
# The injected quote is escaped, so the whole hostile value stays ONE string
# literal rather than closing it and appending a second condition. Asserting
# on the exact json.dumps form rather than counting "@.status" — the hostile
# value legitimately contains that text inside the literal.
import json as _json
assert jsonpath == f"$.verification ? (@.status == {_json.dumps(hostile)})"
assert '\\"' in jsonpath
# --- write path -----------------------------------------------------------
async def test_recording_a_verdict_does_not_touch_the_body():
"""A verdict is metadata about the snippet, not part of it. Writing it into
the body would put it into the embedding and change what the record recalls
against."""
from unittest.mock import AsyncMock, patch
note = SimpleNamespace(
id=5, user_id=1, note_type="snippet", deleted_at=None,
title="n — w", body="```python\nx = 1\n```", tags=["snippet"], data=None,
)
with (
patch.object(snippets_svc, "get_snippet", AsyncMock(return_value=note)),
patch("scribe.services.access.can_write_note", AsyncMock(return_value=True)),
patch.object(
snippets_svc.notes_svc, "update_note", AsyncMock(return_value=note)
) as upd,
):
await snippets_svc.record_verification(1, 5, status="moved", detail="renamed")
assert set(upd.call_args.kwargs) == {"data"}
stored = upd.call_args.kwargs["data"]["verification"]
assert stored["status"] == "moved"
assert stored["detail"] == "renamed"
async def test_recording_a_verdict_requires_write_access():
"""Being able to read a snippet someone shared must not let you mark it
broken — a verdict changes how the record is presented to them."""
from unittest.mock import AsyncMock, patch
note = SimpleNamespace(id=5, user_id=2, note_type="snippet", deleted_at=None,
title="n — w", body="", tags=[], data=None)
with (
patch.object(snippets_svc, "get_snippet", AsyncMock(return_value=note)),
patch("scribe.services.access.can_write_note", AsyncMock(return_value=False)),
patch.object(snippets_svc.notes_svc, "update_note", AsyncMock()) as upd,
):
assert await snippets_svc.record_verification(1, 5, status="ok") is None
upd.assert_not_called()