fix(write-path): give the semantic arm its own threshold + a payload floor
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 6s
CI & Build / integration (push) Successful in 20s
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / Python tests (push) Successful in 41s
CI & Build / Build & push image (push) Successful in 49s
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 6s
CI & Build / integration (push) Successful in 20s
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / Python tests (push) Successful in 41s
CI & Build / Build & push image (push) Successful in 49s
Closes #2223. The write-path prior-art trigger's semantic arm shared auto-inject's 0.55 threshold, which was tuned on prose. Code embeddings sit on a much higher similarity floor — any two Python-shaped payloads share keywords, indentation and structure — so 0.55 landed INSIDE the noise band. Measured against the live instance: near-duplicate of a recorded helper 0.73-0.74 true positive unrelated colour math / Vue SFC / CSS 0.55-0.63 false positive `x = 1` 0.58 false positive 6 of 8 probe payloads produced a nudge; 4 were noise. The margin gate couldn't help — _AUTOINJECT_BAND is relative to the top hit, so with a single hit it never engages. Two gates are now the write-path arm's own: - kb_writepath_threshold, default 0.68 — above every measured false positive, still 0.05 below both true positives. Auto-inject keeps 0.55; it was tuned on prose and is not implicated. The comment this replaces explicitly reserved the split for when telemetry showed the surfaces wanted different values, so this is the change it described, not a reversal of it. - WRITEPATH_MIN_CODE_CHARS = 48 non-whitespace chars, below which the semantic arm doesn't run at all. Whitespace is excluded so a deeply indented one-liner can't pass on padding. 48 sits under the smallest plausible reusable helper (~60) and well over a degenerate edit, so it errs toward keeping recall — precision is the threshold's job. This is the cheap half of the operator's #89 idea; the full length<->threshold curve stays open there, since they asked to brainstorm it rather than have a scale invented for them. top_k stays shared — "how many titles at once" means the same thing on both surfaces. The existing tests were passing `code="x"` / `code="def f(): ..."` into the semantic arm, i.e. exactly the payloads the floor now drops, so the gate tests were never exercising a realistic payload. They now use a REAL_CODE fixture, plus new coverage for the floor (trivial payload, padding, place-arm unaffected, real helper passes) and a guard on the constant itself. Settings UI carries the new knob with the reasoning in its hint, and the write-path checkbox no longer claims it shares the threshold. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
This commit is contained in:
@@ -23,6 +23,10 @@ const kbInjectEnabled = ref(true);
|
||||
const kbInjectThreshold = ref("0.55");
|
||||
const kbInjectTopK = ref("3");
|
||||
const kbWritePathEnabled = ref(true);
|
||||
// The write-path arm's OWN threshold, stricter than auto-inject's 0.55 above:
|
||||
// code embeddings sit on a much higher similarity floor than prose, so 0.55 let
|
||||
// unrelated code through (#2223). Shares top-k, not the threshold.
|
||||
const kbWritePathThreshold = ref("0.68");
|
||||
// Near-duplicate report floor. Deliberately looser than the 0.90 write-time
|
||||
// gate: that one BLOCKS a create and must be unforgiving of noise, this one only
|
||||
// suggests a merge the operator reviews (services/dedup.py).
|
||||
@@ -76,9 +80,13 @@ async function saveKbInject() {
|
||||
// default, not to 0 — a 0 floor would report every snippet as a duplicate of
|
||||
// every other one.
|
||||
const dupT = Math.min(1, Math.max(0, Number(kbDuplicateThreshold.value) || 0.82));
|
||||
// Same `|| default` reasoning as dupT: falling back to 0 would surface every
|
||||
// snippet in the corpus on every edit, which is the failure this knob fixes.
|
||||
const wpT = Math.min(1, Math.max(0, Number(kbWritePathThreshold.value) || 0.68));
|
||||
kbInjectThreshold.value = String(t);
|
||||
kbInjectTopK.value = String(k);
|
||||
kbDuplicateThreshold.value = String(dupT);
|
||||
kbWritePathThreshold.value = String(wpT);
|
||||
savingKbInject.value = true;
|
||||
kbInjectSaved.value = false;
|
||||
try {
|
||||
@@ -86,9 +94,11 @@ async function saveKbInject() {
|
||||
kb_autoinject_enabled: kbInjectEnabled.value ? 'true' : 'false',
|
||||
kb_autoinject_threshold: String(t),
|
||||
kb_autoinject_top_k: String(k),
|
||||
// Its own switch, but deliberately the same threshold/ceiling — see
|
||||
// WRITEPATH_ENABLED_KEY in services/plugin_context.py.
|
||||
// Its own switch AND its own threshold (shares only the ceiling) — see
|
||||
// WRITEPATH_DEFAULT_THRESHOLD in services/plugin_context.py for the
|
||||
// measurements that split them.
|
||||
kb_writepath_enabled: kbWritePathEnabled.value ? 'true' : 'false',
|
||||
kb_writepath_threshold: String(wpT),
|
||||
kb_duplicate_threshold: String(dupT),
|
||||
});
|
||||
kbInjectSaved.value = true;
|
||||
@@ -474,6 +484,9 @@ onMounted(async () => {
|
||||
kbInjectTopK.value = allSettings.kb_autoinject_top_k;
|
||||
}
|
||||
kbWritePathEnabled.value = allSettings.kb_writepath_enabled !== "false";
|
||||
if (allSettings.kb_writepath_threshold !== undefined) {
|
||||
kbWritePathThreshold.value = allSettings.kb_writepath_threshold;
|
||||
}
|
||||
if (allSettings.kb_duplicate_threshold !== undefined) {
|
||||
kbDuplicateThreshold.value = allSettings.kb_duplicate_threshold;
|
||||
}
|
||||
@@ -1220,8 +1233,31 @@ function formatUserDate(iso: string): string {
|
||||
Checks the file Claude is about to write or edit against your recorded
|
||||
snippets — what's already kept at that path, and what resembles the code
|
||||
being written — so a helper you already have is offered before it's
|
||||
rewritten. Uses the same threshold and ceiling above, and never blocks the
|
||||
edit. Off = prior art surfaces only on your own prompts.
|
||||
rewritten. Uses the ceiling above with its own threshold below, and never
|
||||
blocks the edit. Off = prior art surfaces only on your own prompts.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-writepath-threshold">Prior-art confidence threshold (0–1)</label>
|
||||
<input
|
||||
id="kb-writepath-threshold"
|
||||
v-model="kbWritePathThreshold"
|
||||
type="number"
|
||||
min="0"
|
||||
max="1"
|
||||
step="0.01"
|
||||
class="input"
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">
|
||||
Stricter than the prompt threshold above on purpose. Any two pieces of
|
||||
code look somewhat alike — shared keywords, indentation, structure — so
|
||||
resemblance scores start higher for code than for prose, and a bar tuned
|
||||
for prompts flags unrelated code as prior art. Lower this if genuine
|
||||
duplicates go unnoticed; raise it if you're being offered snippets that
|
||||
have nothing to do with what's being written. Snippets recorded at the
|
||||
exact file are always shown regardless — those are prior art by
|
||||
location, not by resemblance.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
|
||||
Reference in New Issue
Block a user