feat(snippets): near-duplicate finder — surface the sets worth merging
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 14s
CI & Build / integration (push) Successful in 36s
CI & Build / Python tests (push) Successful in 55s
CI & Build / Build & push image (push) Successful in 44s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 14s
CI & Build / integration (push) Successful in 36s
CI & Build / Python tests (push) Successful in 55s
CI & Build / Build & push image (push) Successful in 44s
#231's premise was unifying reusable things already scattered as one-offs. The create gate PREVENTS a new duplicate and merge_snippets CURES one you point it at, but nothing FOUND the duplicates already in the record — someone had to notice them by hand, which is the exact failure the Drafter exists to remove. One indexed self-join over note_embeddings, not an N² Python scan: pgvector's cosine distance is the same operator semantic search uses, so a similarity floor is a distance ceiling and the work stays in Postgres. `left.note_id < right.note_id` yields each unordered pair once and drops the self-pair that would otherwise dominate the ranking. Pairs are collapsed into merge SETS by connected components. Transitive on purpose: A~B plus B~C puts all three together even when A and C don't directly clear the bar, which is what merge actually does (it folds every source into one survivor). The cost is that a chain of mild resemblances can rope in a member that isn't really alike — so the UI presents a set as a proposal, shows the members, and never merges without a confirm. Two scope decisions worth naming: - OWN snippets only. merge_snippets requires one owner across the set, so surfacing someone else's would propose a merge that cannot be performed. The report is bounded by what the operator can act on, not what they can see. - Threshold defaults to 0.82, LOOSER than the write gate's 0.90, and is a setting rather than a constant (rule #25). The gate blocks a create and has to be unforgiving of noise; this only suggests a merge under review, so it must reach further or it would never surface the pairs the gate already let through — which are precisely the ones that accumulated. Fixes a real bug in the merge flow while wiring the UI: selectedList filtered the selection against the CURRENT PAGE, and doMerge derives its source ids from that list. A corpus-wide suggested group with off-page members would have rendered incomplete and silently merged only the visible subset. A group under review is now the authority for that list. Refs #2088 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
This commit is contained in:
@@ -23,6 +23,10 @@ const kbInjectEnabled = ref(true);
|
||||
const kbInjectThreshold = ref("0.55");
|
||||
const kbInjectTopK = ref("3");
|
||||
const kbWritePathEnabled = ref(true);
|
||||
// Near-duplicate report floor. Deliberately looser than the 0.90 write-time
|
||||
// gate: that one BLOCKS a create and must be unforgiving of noise, this one only
|
||||
// suggests a merge the operator reviews (services/dedup.py).
|
||||
const kbDuplicateThreshold = ref("0.82");
|
||||
const savingKbInject = ref(false);
|
||||
const kbInjectSaved = ref(false);
|
||||
|
||||
@@ -68,8 +72,13 @@ async function saveRetention() {
|
||||
async function saveKbInject() {
|
||||
const t = Math.min(1, Math.max(0, Number(kbInjectThreshold.value) || 0));
|
||||
const k = Math.min(10, Math.max(1, Math.floor(Number(kbInjectTopK.value) || 1)));
|
||||
// `|| 0.82` not `|| 0`: an unparseable value here should fall back to the
|
||||
// default, not to 0 — a 0 floor would report every snippet as a duplicate of
|
||||
// every other one.
|
||||
const dupT = Math.min(1, Math.max(0, Number(kbDuplicateThreshold.value) || 0.82));
|
||||
kbInjectThreshold.value = String(t);
|
||||
kbInjectTopK.value = String(k);
|
||||
kbDuplicateThreshold.value = String(dupT);
|
||||
savingKbInject.value = true;
|
||||
kbInjectSaved.value = false;
|
||||
try {
|
||||
@@ -80,6 +89,7 @@ async function saveKbInject() {
|
||||
// Its own switch, but deliberately the same threshold/ceiling — see
|
||||
// WRITEPATH_ENABLED_KEY in services/plugin_context.py.
|
||||
kb_writepath_enabled: kbWritePathEnabled.value ? 'true' : 'false',
|
||||
kb_duplicate_threshold: String(dupT),
|
||||
});
|
||||
kbInjectSaved.value = true;
|
||||
setTimeout(() => (kbInjectSaved.value = false), 2000);
|
||||
@@ -464,6 +474,9 @@ onMounted(async () => {
|
||||
kbInjectTopK.value = allSettings.kb_autoinject_top_k;
|
||||
}
|
||||
kbWritePathEnabled.value = allSettings.kb_writepath_enabled !== "false";
|
||||
if (allSettings.kb_duplicate_threshold !== undefined) {
|
||||
kbDuplicateThreshold.value = allSettings.kb_duplicate_threshold;
|
||||
}
|
||||
if (allSettings.notify_task_reminders !== undefined) {
|
||||
notifyTaskReminders.value = allSettings.notify_task_reminders !== "false";
|
||||
}
|
||||
@@ -1211,6 +1224,25 @@ function formatUserDate(iso: string): string {
|
||||
edit. Off = prior art surfaces only on your own prompts.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-duplicate-threshold">Near-duplicate report threshold</label>
|
||||
<input
|
||||
id="kb-duplicate-threshold"
|
||||
v-model="kbDuplicateThreshold"
|
||||
type="number"
|
||||
min="0"
|
||||
max="1"
|
||||
step="0.01"
|
||||
class="input"
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">
|
||||
How alike two snippets must be before the Snippets page suggests merging
|
||||
them. Lower = more suggestions, more false pairs. Looser than the 0.90
|
||||
used to block a duplicate at creation, because this only proposes a merge
|
||||
you review — it never acts on its own.
|
||||
</p>
|
||||
</div>
|
||||
<div class="actions">
|
||||
<button class="btn-save" @click="saveKbInject" :disabled="savingKbInject">
|
||||
{{ savingKbInject ? 'Saving…' : 'Save' }}
|
||||
|
||||
Reference in New Issue
Block a user