feat(snippets): near-duplicate finder — surface the sets worth merging
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 14s
CI & Build / integration (push) Successful in 36s
CI & Build / Python tests (push) Successful in 55s
CI & Build / Build & push image (push) Successful in 44s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 14s
CI & Build / integration (push) Successful in 36s
CI & Build / Python tests (push) Successful in 55s
CI & Build / Build & push image (push) Successful in 44s
#231's premise was unifying reusable things already scattered as one-offs. The create gate PREVENTS a new duplicate and merge_snippets CURES one you point it at, but nothing FOUND the duplicates already in the record — someone had to notice them by hand, which is the exact failure the Drafter exists to remove. One indexed self-join over note_embeddings, not an N² Python scan: pgvector's cosine distance is the same operator semantic search uses, so a similarity floor is a distance ceiling and the work stays in Postgres. `left.note_id < right.note_id` yields each unordered pair once and drops the self-pair that would otherwise dominate the ranking. Pairs are collapsed into merge SETS by connected components. Transitive on purpose: A~B plus B~C puts all three together even when A and C don't directly clear the bar, which is what merge actually does (it folds every source into one survivor). The cost is that a chain of mild resemblances can rope in a member that isn't really alike — so the UI presents a set as a proposal, shows the members, and never merges without a confirm. Two scope decisions worth naming: - OWN snippets only. merge_snippets requires one owner across the set, so surfacing someone else's would propose a merge that cannot be performed. The report is bounded by what the operator can act on, not what they can see. - Threshold defaults to 0.82, LOOSER than the write gate's 0.90, and is a setting rather than a constant (rule #25). The gate blocks a create and has to be unforgiving of noise; this only suggests a merge under review, so it must reach further or it would never surface the pairs the gate already let through — which are precisely the ones that accumulated. Fixes a real bug in the merge flow while wiring the UI: selectedList filtered the selection against the CURRENT PAGE, and doMerge derives its source ids from that list. A corpus-wide suggested group with off-page members would have rendered incomplete and silently merged only the visible subset. A group under review is now the authority for that list. Refs #2088 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
This commit is contained in:
@@ -154,6 +154,25 @@ export async function deleteSnippet(id: number): Promise<void> {
|
||||
return apiDelete(`/api/snippets/${id}`);
|
||||
}
|
||||
|
||||
/** A set of snippets that resemble each other closely enough to be worth
|
||||
* merging. Grouping is transitive, so a set can hold members that don't
|
||||
* directly resemble each other — read it as a proposal, not a verdict. */
|
||||
export interface DuplicateGroup {
|
||||
note_ids: number[];
|
||||
snippets: { id: number; title: string }[];
|
||||
/** The strongest resemblance within the set — how confident the suggestion is. */
|
||||
top_score: number;
|
||||
}
|
||||
|
||||
/** Near-duplicates already in the record. The create gate prevents new ones and
|
||||
* merge cures the ones you point it at; this is what finds them. */
|
||||
export async function findDuplicateSnippets(
|
||||
threshold?: number,
|
||||
): Promise<{ groups: DuplicateGroup[]; threshold: number }> {
|
||||
const qs = threshold ? `?threshold=${threshold}` : "";
|
||||
return apiGet(`/api/snippets/duplicates${qs}`);
|
||||
}
|
||||
|
||||
/** Record a drift-check verdict. The check itself runs where the code is — an
|
||||
* agent with the working tree — since Scribe has no checkout. This stores what
|
||||
* was found, and is how the UI clears a stale marker after a manual fix. */
|
||||
|
||||
Reference in New Issue
Block a user