fix(planning): the plan gate's default drops to 0.80, where reworded plans actually score (#4079)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 55s
CI & Build / Python tests (push) Successful in 1m38s
CI & Build / Build & push image (push) Successful in 36s

Measured live after deploy: three plans reworded from existing active
milestones scored 0.83-0.87 against the milestone they restated, and the
nearest distinct plans 0.72-0.77. At the inherited 0.90 the semantic arm
matched nothing, so only an identical title was caught. 0.80 sits in the gap.
The Settings form's default moves with it (test_settings_defaults_agree).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
This commit is contained in:
2026-09-15 14:53:14 -04:00
co-authored by Claude Opus 5
parent fb36599f2d
commit 21a701c572
2 changed files with 17 additions and 11 deletions
+2 -2
View File
@@ -105,7 +105,7 @@ const kbDupThresholdSnippet = ref("0.82");
const kbDupThresholdNote = ref("0.93"); const kbDupThresholdNote = ref("0.93");
const kbDupThresholdTask = ref("0.93"); const kbDupThresholdTask = ref("0.93");
// PLAN_MATCH_DEFAULT_THRESHOLD in services/dedup.py. // PLAN_MATCH_DEFAULT_THRESHOLD in services/dedup.py.
const kbPlanMatchThreshold = ref("0.90"); const kbPlanMatchThreshold = ref("0.80");
const savingKbInject = ref(false); const savingKbInject = ref(false);
const kbInjectSaved = ref(false); const kbInjectSaved = ref(false);
@@ -159,7 +159,7 @@ async function saveKbInject() {
const dupTask = Math.min(1, Math.max(0, Number(kbDupThresholdTask.value) || 0.93)); const dupTask = Math.min(1, Math.max(0, Number(kbDupThresholdTask.value) || 0.93));
// Same `|| default` guard: a floor of 0 would hand back an existing plan // Same `|| default` guard: a floor of 0 would hand back an existing plan
// for every new one, and no plan could be started without force. // for every new one, and no plan could be started without force.
const planT = Math.min(1, Math.max(0, Number(kbPlanMatchThreshold.value) || 0.9)); const planT = Math.min(1, Math.max(0, Number(kbPlanMatchThreshold.value) || 0.8));
// Same `|| default` reasoning: falling back to 0 would surface every // Same `|| default` reasoning: falling back to 0 would surface every
// snippet in the corpus on every edit, which is the failure this knob fixes. // snippet in the corpus on every edit, which is the failure this knob fixes.
const wpT = Math.min(1, Math.max(0, Number(kbWritePathThreshold.value) || 0.68)); const wpT = Math.min(1, Math.max(0, Number(kbWritePathThreshold.value) || 0.68));
+15 -9
View File
@@ -739,16 +739,22 @@ async def find_duplicate_rule(
# project is exactly what adding steps to its existing plan needs, so a caller # project is exactly what adding steps to its existing plan needs, so a caller
# who reaches this gate can act on whatever it returns. # who reaches this gate can act on whatever it returns.
# #
# Its own threshold, as a setting (rule 25). The note gate's 0.90 is where it # Its own threshold, as a setting (rule 25), and lower than the note gate's
# starts, because it asks the same question ("the same thing, reworded") with # 0.90 because plan documents are shaped differently: a milestone is embedded
# the same embedder. Plan documents are shaped differently, though: a milestone # as title, description and plan (embeddings.milestone_document), while the
# is embedded as title, description and plan (embeddings.milestone_document), # candidate usually has no description and carries its steps instead, so even
# while the candidate usually has no description and carries its steps # a faithful rewording never scores like a copy.
# instead. That difference has not been measured, and a gate that blocks on #
# noise teaches sessions to pass force=true every time, which is worse than no # Measured after the first deploy (2026-09-15, bge-small-en-v1.5, #4079): three
# gate. So the default is conservative and the operator can lower it. # plans reworded from existing active milestones scored 0.83, 0.85 and 0.87
# against the milestone they restated; the nearest DIFFERENT plan for each, and
# a distinct plan on a neighbouring topic, scored 0.72-0.77. At 0.90 the
# semantic arm matched nothing, leaving only the title arm. 0.80 sits in the
# gap. A gate that blocks on noise teaches sessions to pass force=true every
# time, which is why it is not lower; four samples is thin, which is why the
# operator can move it. Retune alongside the embedder, not the corpus.
PLAN_MATCH_THRESHOLD_KEY = "kb_plan_match_threshold" PLAN_MATCH_THRESHOLD_KEY = "kb_plan_match_threshold"
PLAN_MATCH_DEFAULT_THRESHOLD = 0.90 PLAN_MATCH_DEFAULT_THRESHOLD = 0.80
async def get_plan_match_threshold(user_id: int) -> float: async def get_plan_match_threshold(user_id: int) -> float: