feat(snippets): near-duplicate finder — surface the sets worth merging
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 14s
CI & Build / integration (push) Successful in 36s
CI & Build / Python tests (push) Successful in 55s
CI & Build / Build & push image (push) Successful in 44s

#231's premise was unifying reusable things already scattered as one-offs.
The create gate PREVENTS a new duplicate and merge_snippets CURES one you
point it at, but nothing FOUND the duplicates already in the record —
someone had to notice them by hand, which is the exact failure the Drafter
exists to remove.

One indexed self-join over note_embeddings, not an N² Python scan:
pgvector's cosine distance is the same operator semantic search uses, so a
similarity floor is a distance ceiling and the work stays in Postgres.
`left.note_id < right.note_id` yields each unordered pair once and drops
the self-pair that would otherwise dominate the ranking.

Pairs are collapsed into merge SETS by connected components. Transitive on
purpose: A~B plus B~C puts all three together even when A and C don't
directly clear the bar, which is what merge actually does (it folds every
source into one survivor). The cost is that a chain of mild resemblances
can rope in a member that isn't really alike — so the UI presents a set as
a proposal, shows the members, and never merges without a confirm.

Two scope decisions worth naming:

- OWN snippets only. merge_snippets requires one owner across the set, so
  surfacing someone else's would propose a merge that cannot be performed.
  The report is bounded by what the operator can act on, not what they can
  see.

- Threshold defaults to 0.82, LOOSER than the write gate's 0.90, and is a
  setting rather than a constant (rule #25). The gate blocks a create and
  has to be unforgiving of noise; this only suggests a merge under review,
  so it must reach further or it would never surface the pairs the gate
  already let through — which are precisely the ones that accumulated.

Fixes a real bug in the merge flow while wiring the UI: selectedList
filtered the selection against the CURRENT PAGE, and doMerge derives its
source ids from that list. A corpus-wide suggested group with off-page
members would have rendered incomplete and silently merged only the
visible subset. A group under review is now the authority for that list.

Refs #2088

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
This commit is contained in:
2026-07-28 18:32:40 -04:00
parent 84c5c0dc81
commit 6db791965f
10 changed files with 547 additions and 8 deletions
+6 -1
View File
@@ -20,7 +20,7 @@ def test_snippet_handlers_callable():
for name in (
"list_snippets_route", "create_snippet_route", "get_snippet_route",
"update_snippet_route", "delete_snippet_route", "merge_snippet_route",
"verify_snippet_route",
"verify_snippet_route", "duplicate_snippets_route",
):
assert callable(getattr(routes, name))
@@ -68,6 +68,11 @@ def test_agent_and_web_surfaces_stay_at_parity():
assert "verification" in inspect.signature(tools.list_snippets).parameters
assert "verification" in inspect.getsource(routes.list_snippets_route)
# The near-duplicate report (#2088) reaches both. The web side can only hand
# a group to the merge flow if it can get the groups in the first place.
assert callable(getattr(tools, "find_duplicate_snippets", None))
assert callable(getattr(routes, "duplicate_snippets_route", None))
def test_project_scoping_reaches_every_caller():
"""A snippet search has to be narrowable to one project from both surfaces."""