feat(snippets): reverse lookup — find snippets by repo/path/symbol
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 12s
CI & Build / integration (push) Failing after 27s
CI & Build / Python tests (push) Successful in 50s
CI & Build / Build & push image (push) Successful in 1m42s

"What canonical helpers already live in this file?" was unanswerable:
location lived only in the body markdown. It is now a jsonpath containment
query over the `notes.data` mirror added by migration 0070.

- One predicate in two dialects in services/knowledge.py: SQL (`data @?`,
  applied in the browse arm and the keyword arm before count/pagination, so
  totals stay honest) and Python (`location_matches`, for the semantic arm
  which post-filters candidates it already holds). Both must change together.
- Parts are ANDed within a SINGLE locations entry — repo A in one entry and
  path B in another is not "recorded at A/B". `path` also matches as a
  directory prefix, via jsonpath `starts with` rather than `@>`, which the
  same GIN index serves.
- `repo`/`path`/`symbol` reach the service, the REST list and the MCP tool
  under one name with one default (rule #33); the MCP docstring teaches the
  place form, and so does the reusing-code skill (plugin.json bumped).
- UI: a Location disclosure beside the snippet search, with its own empty
  state — "nothing kept there, so what you're about to write is new."

Settles #2083's open question (pre-0070 NULL `data`) by backfilling after
all: `backfill_snippet_data` runs at startup, deriving the mirror from the
body with the same parser the read path trusts. 0070's caution was about
mangling a hand-edited body; this never touches the body. The alternative
was a permanent second body-regex arm, or a query that silently answers
"nothing here" for an old snippet and gets the helper written twice.

Refs #2083, milestone #232.
This commit is contained in:
2026-07-27 23:09:37 -04:00
parent 0d396de215
commit dd1b5e5ddb
12 changed files with 899 additions and 16 deletions
+86 -7
View File
@@ -5,10 +5,10 @@ component recorded once so a later session can recall it before writing a
one-off. It carries a name, language, signature, canonical location
(repo · path · symbol), a one-line "when to reach for it", and the code itself.
Structured fields are stored as a **body-convention** — no dedicated column, so
a snippet inherits everything a note has (embeddings, ACL, project/System
association, dedup) and, crucially, becomes eligible for semantic recall the
moment it's embedded:
Structured fields are stored as a **body-convention** — the body is the readable
form and the thing that gets embedded, so a snippet inherits everything a note
has (embeddings, ACL, project/System association, dedup) and, crucially, becomes
eligible for semantic recall the moment it's embedded:
- ``title`` = ``"{name}{when_to_use}"``. The title is exactly what the
title-first auto-inject surfaces, so this one line self-describes the snippet
@@ -17,18 +17,30 @@ moment it's embedded:
- ``body`` = templated markdown (When to use / Signature / Location, then a
fenced code block).
Since migration 0070 the same fields are ALSO written to ``notes.data`` (see
``compose_data``) — a queryable mirror, not a second source of truth: it carries
no code, and it exists so location/language can be indexed rather than regexed
out of every body. Reads prefer it and fall back to the body parse.
The public field API (name/language/signature/location/when_to_use/code) lives
here, so the underlying storage could later move to a structured column without
changing any caller (MCP tool, REST route, UI).
here, so callers (MCP tool, REST route, UI) never see which of the two a field
came from.
"""
from __future__ import annotations
import asyncio
import logging
import re
from sqlalchemy import select
from scribe.models import async_session
from scribe.models.note import Note
from scribe.services import knowledge as knowledge_svc
from scribe.services import notes as notes_svc
logger = logging.getLogger(__name__)
SNIPPET_NOTE_TYPE = "snippet"
SNIPPET_TAG = "snippet"
@@ -354,6 +366,63 @@ def snippet_fields(note) -> dict:
return merged
async def backfill_snippet_data(*, batch: int = 500) -> int:
"""Populate `notes.data` for snippet rows that predate migration 0070.
Runs once at startup (see app.py). Returns how many rows were filled.
Migration 0070 deliberately left `data` NULL on existing rows, because
*reading* a snippet never needed it — `snippet_fields` falls back to parsing
the body. Querying does: the location reverse lookup (#2083) is a jsonpath
over this column, so a NULL-`data` snippet would be reported as "nothing
recorded here" and the caller would write the helper again — silently worse
than having no reverse lookup at all. Every consumer of the column
(reverse lookup, and the write-path trigger built on it) can then assume the
mirror is present instead of carrying a second body-regex arm.
This does not reverse 0070's actual caution, which was about mangling a
hand-edited *body*: the body is never touched here, only the mirror derived
from it, by the same parser the read path already trusts. Idempotent — a
filled row is skipped forever after, and a snippet with no structured fields
at all settles at `{}` rather than staying NULL and being re-scanned. Trashed
rows are included so a later restore comes back queryable.
"""
filled = 0
async with async_session() as session:
while True:
rows = list(
(
await session.execute(
select(Note)
.where(Note.note_type == SNIPPET_NOTE_TYPE)
.where(Note.data.is_(None))
.limit(batch)
)
)
.scalars()
.all()
)
if not rows:
break
for note in rows:
fields = parse_snippet_fields(note.title, note.body, note.tags)
note.data = compose_data(
name=fields["name"],
when_to_use=fields["when_to_use"],
signature=fields["signature"],
language=fields["language"],
locations=fields["locations"],
merged_from=fields["merged_from"],
)
await session.commit()
filled += len(rows)
if len(rows) < batch:
break
if filled:
logger.info("Snippet data backfill: populated `data` for %d snippet(s)", filled)
return filled
def snippet_to_dict(note) -> dict:
"""Note serialization plus a parsed ``snippet`` sub-object of structured
fields, so callers get both the raw record and the typed view.
@@ -433,12 +502,20 @@ async def list_snippets(
limit: int = 50,
offset: int = 0,
project_id: int | None = None,
repo: str = "",
path: str = "",
symbol: str = "",
) -> tuple[list[dict], int]:
"""List snippets (id/title/tags/preview dicts), most-recently-updated first.
``project_id`` narrows to one project; omit it to reach across every project
— which is the point when the thing you're about to write was already solved
somewhere else."""
somewhere else.
``repo`` / ``path`` / ``symbol`` are the reverse lookup: "what canonical
helpers already live here?" They narrow to snippets recorded at a matching
location, ANDed within one location entry, with ``path`` also matching as a
directory prefix. Combinable with ``q`` — search *and* place."""
return await knowledge_svc.query_knowledge(
user_id=user_id,
note_type=SNIPPET_NOTE_TYPE,
@@ -448,6 +525,8 @@ async def list_snippets(
limit=max(1, min(limit, 100)),
offset=max(0, offset),
project_id=project_id,
locations=knowledge_svc.location_parts(repo=repo, path=path, symbol=symbol)
or None,
)