Commit Graph

8 Commits

Author SHA1 Message Date
bvandeusen 35f3f09d12 feat(snippets): drift check — verify a snippet still matches its source
CI & Build / Python lint (push) Successful in 3s
CI & Build / integration (push) Successful in 19s
CI & Build / TypeScript typecheck (push) Failing after 20s
CI & Build / Python tests (push) Failing after 22s
CI & Build / Build & push image (push) Has been skipped
A recorded snippet points at a repo · path · symbol that WILL rot: files
move, symbols get renamed, implementations diverge from the copy stored
here. Nothing detected any of it, so a record degraded silently from
"canonical reference" to "confidently wrong" — worse than no record, since
it is surfaced with the same authority either way.

WHERE THE CHECK RUNS. Agent-side, which the task flagged as the design
question to settle first. Scribe has no checkout of the operator's repos
and must not acquire one: giving the server repo access would make every
install a credential problem and break instance-agnosticism (rule #115).
The agent already has the working tree, so it does the comparing; the
server remembers the verdict, makes it queryable, and knows when it has
expired. New MCP tool verify_snippet teaches the four-step procedure and
records the result; a REST endpoint mirrors it so the UI can clear a
marker after a manual fix.

WHY THE VERDICT CARRIES A CODE HASH. A verdict describes the code it was
checked against. Invalidating it on edit means deciding which edits count
— a when_to_use tweak shouldn't void a code check, a rewrite must — which
is fiddly and easy to get subtly wrong, and easy for a new write path to
forget entirely. Stamping the verdict with a hash sidesteps all of it: one
whose code_sha no longer matches is self-evidently expired, computed at
read time, no invalidation branch to maintain.

That makes "expired" the interesting filter case. It is not `drifted`
(nothing was found wrong) and not `unverified` (a check did happen), yet
it plainly needs looking at — so `verification=attention` covers both. To
keep that one index-served predicate rather than a post-filter that would
make the pagination total a lie, data now also mirrors the CURRENT code's
fingerprint as data.code_sha, and a jsonpath compares the two fields
within the row. The filter is implemented in both dialects, SQL and
Python, for the same reason the location filter is: the semantic arm's
candidates arrive already fetched.

A merge deliberately carries no verdict forward — the survivor's code is a
union of several sources, so no prior check describes it, and unverified
is the honest answer.

UI: a danger-toned drift badge on each card (an actively misleading record
outranks a merely unused one), and a "Needs attention" filter. Its empty
state says plainly that never-verified snippets don't appear there —
otherwise `attention` would mean "everything" on day one and be useless as
a worklist.

Refs #2086

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
2026-07-28 18:20:33 -04:00
bvandeusen 2b85443dd1 feat(snippets): usage signal — was a surfaced record ever actually pulled?
CI & Build / Python lint (push) Successful in 2s
CI & Build / integration (push) Successful in 31s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Python tests (push) Successful in 56s
CI & Build / Build & push image (push) Successful in 44s
retrieval_logs answers "what did the ranker return, at what scores" — the
right substrate for tuning a threshold. It cannot answer the question the
snippet corpus actually needs: did anyone open this? A snippet nobody
opens is not neutral. It takes a slot in every future auto-inject menu and
crowds out something useful.

Adds note_usage_events (migration 0071): one row per note per event,
either 'surfaced' (we put its title in front of an agent) or 'pulled'
(someone opened it in full), tagged with which surface produced it.

Closes the gap #2082 recorded against this work. The write-path PLACE arm
carries no score, so it has no home in retrieval_logs — folding it in
would corrupt the score distribution that table exists to capture. The
result was that the arm firing on the STRONGEST claim ("there is already a
canonical helper in this exact file") was the one arm nobody could
measure. Both arms now emit usage events under distinct sources, so their
pull-through rates are finally comparable.

Deliberate departures from the task as written:

- Not in-session correlation. The original framing was "correlate
  result_ids against a later get_note in the same session." There is no
  session identity server-side — the MCP endpoint is stateless and the
  hooks send no session id — and adding one would mean threading an
  opaque client-supplied token through every read path. Two independent
  counters answer the question without it: surfaced 40×, pulled 0 is dead
  weight regardless of how those events distribute across sessions.

- Pulls record at the ENTRY POINTS (MCP tools, REST detail route), not in
  snippets_svc.get_snippet, which update and merge also reach. Counting
  those would inflate precisely the number meant to say "someone chose to
  look at this."

- get_note records for every note kind, not just snippets. The auto-inject
  menu surfaces tasks and processes too; scoping this to snippets would
  pin those at zero pulls forever and make them read as dead weight next
  to snippets that merely had a counter.

Surfaced in the Snippets list as an "N/M used" badge, warning-toned once a
record has been offered 3+ times and never opened, with the tooltip saying
what to do about it (usually: its "when to reach for it" doesn't say
when). No badge at all below one surfacing — "0/0" reads as a verdict when
it's an absence of evidence. Also returned from MCP list_snippets so the
agent can see dead weight without opening the UI.

Telemetry keeps the retrieval_telemetry contract throughout: writes are
fire-and-forget, reads degrade to zeroes, and no path can raise into the
surface it observes.

Refs #2085

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
2026-07-28 18:09:17 -04:00
bvandeusen dd1b5e5ddb feat(snippets): reverse lookup — find snippets by repo/path/symbol
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 12s
CI & Build / integration (push) Failing after 27s
CI & Build / Python tests (push) Successful in 50s
CI & Build / Build & push image (push) Successful in 1m42s
"What canonical helpers already live in this file?" was unanswerable:
location lived only in the body markdown. It is now a jsonpath containment
query over the `notes.data` mirror added by migration 0070.

- One predicate in two dialects in services/knowledge.py: SQL (`data @?`,
  applied in the browse arm and the keyword arm before count/pagination, so
  totals stay honest) and Python (`location_matches`, for the semantic arm
  which post-filters candidates it already holds). Both must change together.
- Parts are ANDed within a SINGLE locations entry — repo A in one entry and
  path B in another is not "recorded at A/B". `path` also matches as a
  directory prefix, via jsonpath `starts with` rather than `@>`, which the
  same GIN index serves.
- `repo`/`path`/`symbol` reach the service, the REST list and the MCP tool
  under one name with one default (rule #33); the MCP docstring teaches the
  place form, and so does the reusing-code skill (plugin.json bumped).
- UI: a Location disclosure beside the snippet search, with its own empty
  state — "nothing kept there, so what you're about to write is new."

Settles #2083's open question (pre-0070 NULL `data`) by backfilling after
all: `backfill_snippet_data` runs at startup, deriving the mirror from the
body with the same parser the read path trusts. 0070's caution was about
mangling a hand-edited body; this never touches the body. The alternative
was a permanent second body-regex arm, or a query that silently answers
"nothing here" for an old snippet and gets the helper written twice.

Refs #2083, milestone #232.
2026-07-27 23:09:37 -04:00
bvandeusen 04b58ce01e feat(acl): shared records are search-only and always labelled as someone else's
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 28s
CI & Build / integration (push) Successful in 35s
CI & Build / Python tests (push) Failing after 41s
CI & Build / Build & push image (push) Has been skipped
Narrows the b7d6fc7 widening per the operator's call, and fixes a regression it
introduced. Decision recorded as note 2094.

Two scopes now, deliberately different:

  readable_notes_clause  — everything the ACL permits, including records reached
                           only via a direct/group note share. For EXPLICIT acts:
                           a search the caller typed, a fetch by id.
  browsable_notes_clause — the caller's own records plus anything in a project
                           they can reach. For PASSIVE surfaces: browse lists,
                           facet counts, the process->skill manifest.

The split is a trust boundary. Anything appearing unasked — in your own list,
your own counts, or as a skill installed on your machine — reads as material you
endorsed. A one-off someone shared with you hasn't earned that standing, so it
waits until you go looking. This also dissolves the shared-Process problem by
construction rather than by special case: the manifest is a passive surface, so a
directly-shared Process is never installed as an auto-surfacing skill.

Regression fix (#2093): b7d6fc7 widened the list queries but left the fetch path
owner-only, so on the MCP path a record could be listed and then not opened —
get_snippet raised not-found, get_process couldn't resolve, and the manifest
emitted stubs whose get_process call would fail. snippets.get_snippet and
notes.resolve_process now resolve the read scope. delete_snippet gained an
explicit can_write_note guard, since being able to SEE a shared snippet must not
imply being able to bin it.

Provenance, so nothing arrives looking like the operator's own work:
- access.describe_provenance / label_shared_items add shared/owner/permission;
  labelling costs no query when everything is the caller's own.
- MCP: get_snippet, list_snippets, get_process and list_processes carry it, and
  get_process now says outright NOT to follow a shared process verbatim — its
  follow-as-written contract was the sharpest instance of the problem.
- The skill stub for a shared Process names its author and asks for a go-ahead,
  instead of describing it as "the operator's saved Scribe process".
- Policy stated once in the MCP _INSTRUCTIONS and the reusing-code skill: a
  shared record is that person's suggestion, weigh it, attribute it, ask before
  adopting it.
- UI: shared snippets show "by <owner>" in the list and a notice above the code
  in the detail view, reusing SharedWithMeView's vocabulary.

Plugin 0.1.15 -> 0.1.16 (skill text changed).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt
2026-07-25 22:43:00 -04:00
bvandeusen b33e2a79c6 fix(snippets): close the recall-surface gaps found reviewing the Drafter
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 11s
CI & Build / integration (push) Successful in 19s
CI & Build / Python tests (push) Successful in 43s
CI & Build / Build & push image (push) Successful in 1m7s
Four defects from the 2026-07-25 review of the recall (#227) and merge (#231)
milestones. The theme: a snippet could be recorded but not fully corrected, and
the agent and web surfaces had drifted apart.

- #2076 language was mis-derived from the first caller tag, so a snippet created
  with tags and no language read that tag back as its language — corrupting the
  tag set and the code fence on the next update. Only the FIRST tag can carry
  the language, since compose_tags emits [language, "snippet", *caller].
- #2077 MCP update_snippet mapped "" to "unchanged", so no field could ever be
  cleared and no snippet detached from its project. Now an omitted field is left
  alone, an empty string clears, and project_id follows the -1 = detach
  convention. A service-level UNSET sentinel keeps None available as the clear.
- #2078 surface parity: adds delete_snippet (MCP had none, so a wrong snippet
  could not be retired by the agent that recorded it), locations on MCP create
  and update, system_ids through the REST routes and the editor, and the
  near-duplicate gate on REST create with a "record it anyway" escape.
- #2079 project scoping: list_snippets takes project_id through the service, the
  MCP tool and the REST route, defaulting to every project — reaching across
  projects is the point when the helper you need was written elsewhere.

Sharing the list across owners is deliberately NOT in here: query_knowledge is
shared with the Knowledge browse surface, so widening it changes behaviour well
beyond snippets. Left open on #2079 for a scope decision.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt
2026-07-25 19:00:05 -04:00
bvandeusen 7a81b7333e feat(scribe): snippet merge UI — multi-select merge, repeatable locations
CI & Build / Python lint (push) Successful in 4s
CI & Build / integration (push) Successful in 29s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Python tests (push) Successful in 53s
CI & Build / Build & push image (push) Successful in 1m2s
Step 3 of the snippet-merge milestone (#231): the human surfaces for
merge + multi-location, at v1 quality.

Frontend:
- SnippetListView: a Select mode (checkbox on each card) → a sticky action
  bar → a merge modal that lets you pick which selected snippet is the
  canonical (the others fold in and go to trash). Accent border on selected
  cards, Moss action buttons (Hybrid rule).
- SnippetEditorView: the single Location fieldset becomes a repeatable
  locations list (add/remove rows), so editing a merged snippet no longer
  collapses its call sites — no data loss. Sends `locations`.
- SnippetDetailView: renders every location (Location vs Locations label).
- api/snippets.ts: SnippetLocation type, `locations` on fields/input,
  mergeSnippets().

Backend (editor enablement):
- create_snippet service + POST route accept an optional `locations` list;
  PATCH route forwards `locations` — so the editor's location list works
  uniformly on create and edit. Single repo/path/symbol remain the
  one-location shorthand (MCP create contract unchanged).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pa2EsuB54BuWQ8GfJq9c7t
2026-07-25 17:22:01 -04:00
bvandeusen 85625de394 feat(scribe): merge_snippets — unify found one-off snippets into one canonical
CI & Build / Python lint (push) Successful in 4s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / integration (push) Successful in 38s
CI & Build / Python tests (push) Successful in 59s
CI & Build / Build & push image (push) Successful in 1m10s
Step 2 of the snippet-merge milestone (#231). The dedup gate only PREVENTS
new near-duplicates; merge is the CURE for the ones already scattered.

- services/snippets.py: merge_snippets(user_id, target_id, source_ids) —
  keep the target's scalar fields (name/when_to_use/signature/language/
  code), union the sources' locations + extra tags onto it (so the survivor
  carries every call site as a location), trash the sources (recoverable),
  re-embed the survivor. Pure merge_snippet_fields() factored out for unit
  testing. Returns (survivor_note, merged_ids).
- mcp/tools/snippets.py: merge_snippets(target_id, source_ids) tool (5th),
  and a create_snippet dedup-path nudge toward merge over a forced copy.
- routes/snippets.py: POST /api/snippets/<id>/merge {source_ids} — share-
  aware (can_write target + every source, rule #78) with a same-owner guard
  (cross-owner merge is out of scope).
- plugin reusing-code skill + MCP _INSTRUCTIONS: point found-duplicates at
  merge as the cure (rule #119 surfaces, not a Scribe rule). plugin.json
  0.1.13 -> 0.1.14 in the same change (the #1040 marketplace-ship lesson).
- Tests: pure merge-helper union/dedup; MCP tool (requires a source,
  survivor+merged_ids, not-found); route handler + 5-tool registration.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pa2EsuB54BuWQ8GfJq9c7t
2026-07-25 17:11:47 -04:00
bvandeusen d257c0fd67 feat(scribe): snippet management UI + REST routes; embed snippets on create
CI & Build / Python lint (push) Successful in 3s
CI & Build / integration (push) Successful in 23s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 48s
CI & Build / Build & push image (push) Successful in 1m4s
Step 6 of the Drafter recall milestone (#227): a human-facing surface for
the reusable-code snippets that agents record via MCP, plus the REST API
behind it. Backend embeds snippets inline on create/update so they're
recallable immediately, not only after a restart.

Backend:
- routes/snippets.py: GET/POST /api/snippets, GET/PATCH/DELETE
  /api/snippets/<id>. Share-aware per rule #78 (get_note_for_user +
  can_write_note), writes performed as the owner; list owner-scoped —
  mirrors routes/notes.py. Registered in app.py.
- services/snippets.py: embed on create/update via a _embed_snippet
  fire-and-forget helper, covering BOTH the MCP tool and the REST route
  by construction. A snippet's value is immediate recall, so it can't wait
  for the startup-only backfill (see issue: MCP create path doesn't embed
  inline for notes/tasks generally).
- tests/test_routes_snippets.py: structural registration + handler/service
  contract + PATCH-field ↔ update_snippet-kwarg parity (rule #33).

Frontend (Vue 3 + TS):
- api/snippets.ts: typed client, modeled on api/systems.ts.
- views: SnippetListView (search, skeleton/empty/error states),
  SnippetDetailView (read + copy-to-clipboard, ConfirmDialog delete),
  SnippetEditorView (create/edit all fields, Ctrl/Cmd+S, Esc, autofocus,
  validation). v1 quality per rules #24/#27.
- router: /snippets, /snippets/new, /snippets/:id, /snippets/:id/edit.
- NoteType union widened to include 'snippet'; Snippets nav link added to
  AppHeader (desktop pill bar + mobile menu).
- Design system: Moss --color-action-primary for action buttons, accent
  --color-primary reserved for tags/brand (Hybrid rule); focus rings;
  JetBrains Mono for code/name/signature/location.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pa2EsuB54BuWQ8GfJq9c7t
2026-07-25 14:16:02 -04:00