feat(settings): the two new retrieval bars get their controls (#3927)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 48s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m27s
CI & Build / Build & push image (push) Successful in 36s

#3852 and #3853 each added a threshold and neither added its input, so two
of the eleven retrieval settings were reachable only through the database.
The other nine have had UI all along.

That matters because the issue said otherwise. #3927 claimed none of the ten
appeared in the frontend, on the strength of a grep across `web/src` — a
directory this repo does not have. An empty result from a path that cannot
match was read as "no UI anywhere", and a rule-25 argument was written on top
of it. The issue is corrected rather than quietly rewritten: it is #3720's
defect, absence read as non-existence, committed while filing issues about
the product doing the same thing. A grep that returns nothing and a grep that
cannot match produce the same output, and only a positive control tells them
apart.

So this is small, which is the honest size:

- `kb_toolrule_threshold` — the command arm's bar. The hint says why it sits
  BELOW the write-path one rather than leaving that looking like a mistake: a
  shell command is short, so it scores lower for the same relevance, and at a
  shared bar this arm spoke on 2% of calls against the write path's 37%.
- `kb_promptrule_threshold` — the prompt boundary, a third query shape again,
  and the only moment that reaches a rule about how to ANSWER.

Both follow the five-site pattern the existing controls use: ref, clamp on
save, write-back, payload key, load. Separate keys, because the finding of
#3853 is that one number cannot serve arms whose queries differ in shape.

Also corrects copy that went stale this morning. The standing-rule hint still
said the arm surfaces "only rules marked conditional, since always-on ones
are already loaded" — describing a tier milestone 394 removed, on the surface
whose whole job is telling the operator what the bar does.

The guard is the part worth keeping. A form's initial value is a CLAIM about
the server's default, and nothing connected the two: retune the Python
constant and the input keeps rendering the old number, which the operator
reads as the bar in force. It pins the relationship across all five
thresholds, never the values, so retuning stays free as long as both move —
and it is falsified against a drifted form value.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
This commit is contained in:
2026-09-11 20:30:06 -04:00
co-authored by Claude Opus 5
parent fe2f88cdb6
commit 2e8d8461cc
2 changed files with 162 additions and 8 deletions
+74 -8
View File
@@ -87,6 +87,15 @@ const kbWritePathEnabled = ref(true);
// unrelated code through (#2223). Shares top-k, not the threshold.
const kbWritePathThreshold = ref("0.68");
const kbRuleHintThreshold = ref("0.72");
// The two ACT arms no longer share a bar (#3853). A write-path query is a code
// payload; a pre-tool query is a shell command, often under a dozen words —
// less text, less signal, lower scores for the same relevance. Measured at one
// shared 0.72 the two behaved like different subsystems: the write-path arm
// spoke on 37% of its calls, the command arm on 2% of 11,768.
const kbToolRuleThreshold = ref("0.68");
// And the prompt boundary is a third query shape again — the operator's own
// prose rather than anything a tool produced (#3852).
const kbPromptRuleThreshold = ref("0.72");
// Near-duplicate report floors, one per record kind (services/dedup.py).
// Snippets are single-chunk, so their floor sits below the 0.90 write-time
// gate and catches what it lets through. Notes/tasks are scored at chunk
@@ -153,6 +162,10 @@ async function saveKbInject() {
// fires on every write, so a fallback of 0 would attach a standing rule to
// every edit in the session.
const rhT = Math.min(1, Math.max(0, Number(kbRuleHintThreshold.value) || 0.72));
// Same `|| default` guard, and the same reason: this arm fires before every
// Bash call, so a fallback of 0 would put a rule in front of every command.
const trT = Math.min(1, Math.max(0, Number(kbToolRuleThreshold.value) || 0.68));
const prT = Math.min(1, Math.max(0, Number(kbPromptRuleThreshold.value) || 0.72));
kbInjectThreshold.value = String(t);
kbInjectTopK.value = String(k);
kbDupThresholdSnippet.value = String(dupSnip);
@@ -160,6 +173,8 @@ async function saveKbInject() {
kbDupThresholdTask.value = String(dupTask);
kbWritePathThreshold.value = String(wpT);
kbRuleHintThreshold.value = String(rhT);
kbToolRuleThreshold.value = String(trT);
kbPromptRuleThreshold.value = String(prT);
savingKbInject.value = true;
kbInjectSaved.value = false;
try {
@@ -176,6 +191,11 @@ async function saveKbInject() {
// in services/plugin_context.py for why rules cannot share the
// code threshold any more than code could share the prose one.
kb_rulehint_threshold: String(rhT),
// A FOURTH and FIFTH bar, and they are separate keys on purpose: the
// whole finding of #3853 is that one number cannot serve arms whose
// queries are different shapes. Moving one must not move the others.
kb_toolrule_threshold: String(trT),
kb_promptrule_threshold: String(prT),
kb_duplicate_threshold_snippet: String(dupSnip),
kb_duplicate_threshold_note: String(dupNote),
kb_duplicate_threshold_task: String(dupTask),
@@ -624,6 +644,12 @@ onMounted(async () => {
if (allSettings.kb_rulehint_threshold !== undefined) {
kbRuleHintThreshold.value = allSettings.kb_rulehint_threshold;
}
if (allSettings.kb_toolrule_threshold !== undefined) {
kbToolRuleThreshold.value = allSettings.kb_toolrule_threshold;
}
if (allSettings.kb_promptrule_threshold !== undefined) {
kbPromptRuleThreshold.value = allSettings.kb_promptrule_threshold;
}
if (allSettings.kb_writepath_threshold !== undefined) {
kbWritePathThreshold.value = allSettings.kb_writepath_threshold;
}
@@ -1482,14 +1508,54 @@ async function deleteUser(userId: number) {
style="max-width: 8rem"
/>
<p class="field-hint">
The same hint can mention a standing rule whose trigger resembles what's
being written — only rules marked <em>conditional</em>, since always-on
ones are already loaded. Stricter again than the threshold above, because
there are far fewer rules than snippets: with a small set, something
always ranks first, so the bar has to carry more of the judgement.
Raise it if rules keep arriving unread; lower it if a rule you needed
never showed up. Settings → check the pull-through in
<code>retrieval_telemetry</code> to see which is happening.
The same hint can mention a standing rule whose trigger resembles the
code being written. <em>Every</em> rule is eligible nothing is
preloaded any more, so this is the only way a rule reaches a write.
Stricter than the threshold above, because there are far fewer rules
than snippets: with a small set something always ranks first, so the
bar has to carry more of the judgement. Raise it if rules keep
arriving unread; lower it if a rule you needed never showed up.
</p>
</div>
<div class="field">
<label for="kb-toolrule-threshold">Command confidence threshold (01)</label>
<input
id="kb-toolrule-threshold"
v-model="kbToolRuleThreshold"
type="number"
min="0"
max="1"
step="0.01"
class="fs-input input"
style="max-width: 8rem"
/>
<p class="field-hint">
The bar for a rule surfacing before a <em>command</em> runs, where the
query is the command text rather than code. Lower than the one above
on purpose: a shell command is short, so it scores lower for the same
relevance at a shared bar this arm spoke on 2% of calls against the
write path's 37%. Raise it if commands attract rules that do not
apply; lower it if a <code>git push</code> arrives with nothing.
</p>
</div>
<div class="field">
<label for="kb-promptrule-threshold">Prompt confidence threshold (01)</label>
<input
id="kb-promptrule-threshold"
v-model="kbPromptRuleThreshold"
type="number"
min="0"
max="1"
step="0.01"
class="fs-input input"
style="max-width: 8rem"
/>
<p class="field-hint">
The bar for a rule or preference surfacing against <em>what you just
said</em>, before anything is done. It is the only moment that reaches
a rule about how to answer rather than how to act, so it runs once a
turn rather than once a tool call. A third query shape — your prose,
not a command or a file — which is why it carries its own number.
</p>
</div>
<!-- A design system belongs to a PROJECT, and the picker for it lives on