align(4909d): every rule arm that annotates reads one bar, kb_rule_threshold at 0.70
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m7s
CI & Build / Python tests (push) Failing after 1m30s
CI & Build / Build & push image (push) Skipped
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m7s
CI & Build / Python tests (push) Failing after 1m30s
CI & Build / Build & push image (push) Skipped
prompt_rule, pre_tool_rule and write_path_rule now share one settings key, RULE_FLOOR_KEY = "kb_rule_threshold", default 0.70. The reply backstop keeps its own 0.80, because there the floor is a stop bar. Moving the shared floor through tune_retrieval or the Settings form records a tuning event on every surface that reads it (set_dial, via retrieval_surfaces.sharing_floor), and the result names the surfaces it carried as `moved_with`. Settings shows one "Standing-rule confidence threshold" control where it showed three. Operator ruling (2026-10-05): one shared bar for the rule arms, "especially since we're only taking the first x number of them" -- the budget limits what is shown; the bar only decides what is worth ranking. Kept a dial. Why 0.70, from records read rather than percentiles: - write_path_rule near-misses at 0.7199-0.72 were all false positives (rule 149 against a Lidarr comment, 157 against a retention Svelte file, 121 against an Archives restore, 164 against a config test, 131 against a Go package), so the old 0.72 was not hiding real hits -- but the band does noise control on this arm, so dropping 0.02 costs little. - prompt_rule: the operator set 0.70 by hand on 2026-09-18, after a 0.60 experiment admitted junk and was reverted. 0.70 is the number with a human decision behind it. - pre_tool_rule lines near 0.68 this session were mostly noise; rising to 0.70 trims them, and the band still narrows what clears. Retired keys: kb_promptrule_threshold, kb_toolrule_threshold and kb_rulehint_threshold are no longer read. Rows an install stored under them stay in the settings table, inert. On this instance the effective bars move pre_tool_rule 0.68 -> 0.70, write_path_rule 0.72 -> 0.70, prompt_rule stays 0.70; record_release_defaults writes those per-surface release rows on boot, so the telemetry shift has an event to explain it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -159,16 +159,10 @@ const kbWritePathEnabled = ref(true);
|
||||
// code embeddings sit on a much higher similarity floor than prose, so 0.55
|
||||
// lets unrelated code through. Shares top-k, not the threshold.
|
||||
const kbWritePathThreshold = ref("0.68");
|
||||
const kbRuleHintThreshold = ref("0.72");
|
||||
// One bar for every rule hint — prompt, command and edit alike. They search
|
||||
// the same rules, and what limits each is its budget, not the bar.
|
||||
const kbRuleThreshold = ref("0.70");
|
||||
const kbCheckpointThreshold = ref("0.80");
|
||||
// The two ACT arms do not share a bar. A write-path query is a code payload; a
|
||||
// pre-tool query is a shell command, often under a dozen words — less text,
|
||||
// less signal, lower scores for the same relevance — so one shared bar makes
|
||||
// them behave like different subsystems.
|
||||
const kbToolRuleThreshold = ref("0.68");
|
||||
// And the prompt boundary is a third query shape again — the operator's own
|
||||
// prose rather than anything a tool produced.
|
||||
const kbPromptRuleThreshold = ref("0.72");
|
||||
// The reply arm: its bar is the bar at which a finished reply
|
||||
// is HELD for one read, so it sits with the checkpoint, not with the hints.
|
||||
const kbReplyRuleThreshold = ref("0.8");
|
||||
@@ -367,14 +361,10 @@ async function saveKbInject() {
|
||||
// Same `|| default` reasoning: falling back to 0 would surface every
|
||||
// snippet in the corpus on every edit, which is the failure this knob fixes.
|
||||
const wpT = asBar(kbWritePathThreshold.value, 0.68);
|
||||
// Same `|| default` reasoning again, and it bites harder here: a rule hint
|
||||
// fires on every write, so a fallback of 0 would attach a standing rule to
|
||||
// every edit in the session.
|
||||
const rhT = asBar(kbRuleHintThreshold.value, 0.72);
|
||||
// Same `|| default` guard, and the same reason: this arm fires before every
|
||||
// Bash call, so a fallback of 0 would put a rule in front of every command.
|
||||
const trT = asBar(kbToolRuleThreshold.value, 0.68);
|
||||
const prT = asBar(kbPromptRuleThreshold.value, 0.72);
|
||||
// Same `|| default` reasoning again, and it bites harder here: rule hints
|
||||
// fire on every edit and before every command, so a fallback of 0 would put
|
||||
// a standing rule beside each of them.
|
||||
const ruleT = asBar(kbRuleThreshold.value, 0.7);
|
||||
// The checkpoint bar, and the `|| default` guard matters most here of all:
|
||||
// this is the only number that can STOP a call, so a fallback of 0 would
|
||||
// hold the first command of every session behind whatever ranked first.
|
||||
@@ -409,10 +399,8 @@ async function saveKbInject() {
|
||||
kbGateNoteOverlapFloor.value = String(gateOverlap);
|
||||
kbPlanMatchThreshold.value = String(planT);
|
||||
kbWritePathThreshold.value = String(wpT);
|
||||
kbRuleHintThreshold.value = String(rhT);
|
||||
kbRuleThreshold.value = String(ruleT);
|
||||
kbCheckpointThreshold.value = String(cpT);
|
||||
kbToolRuleThreshold.value = String(trT);
|
||||
kbPromptRuleThreshold.value = String(prT);
|
||||
kbReplyRuleThreshold.value = String(ryT);
|
||||
savingKbInject.value = true;
|
||||
kbInjectSaved.value = false;
|
||||
@@ -426,20 +414,15 @@ async function saveKbInject() {
|
||||
// measurements that split them.
|
||||
kb_writepath_enabled: kbWritePathEnabled.value ? 'true' : 'false',
|
||||
kb_writepath_threshold: String(wpT),
|
||||
// A THIRD corpus with a third bar — see RULEHINT_DEFAULT_THRESHOLD
|
||||
// in services/plugin_context.py for why rules cannot share the
|
||||
// code threshold any more than code could share the prose one.
|
||||
kb_rulehint_threshold: String(rhT),
|
||||
// A third corpus with a third bar — see RULE_FLOOR_DEFAULT in
|
||||
// services/retrieval_pipeline.py for why every rule hint shares it.
|
||||
kb_rule_threshold: String(ruleT),
|
||||
// The one bar that stops a call rather than annotating it. Its own key,
|
||||
// never derived from the two rule bars above: it answers a different
|
||||
// never derived from the rule bar above: it answers a different
|
||||
// question of the same scores — not "is this worth showing" but "is the
|
||||
// corpus confident enough to be read first" — and a number that moves
|
||||
// when another moves is a number nobody can reason about.
|
||||
kb_checkpoint_threshold: String(cpT),
|
||||
// A FOURTH and FIFTH bar, and they are separate keys on purpose: one
|
||||
// number cannot serve arms whose queries are different shapes. Moving one must not move the others.
|
||||
kb_toolrule_threshold: String(trT),
|
||||
kb_promptrule_threshold: String(prT),
|
||||
// The reply backstop's own bar: it HOLDS a reply, so like the
|
||||
// checkpoint it is never derived from a hint bar.
|
||||
kb_replyrule_threshold: String(ryT),
|
||||
@@ -864,14 +847,8 @@ onMounted(async () => {
|
||||
if (allSettings.kb_checkpoint_threshold !== undefined) {
|
||||
kbCheckpointThreshold.value = allSettings.kb_checkpoint_threshold;
|
||||
}
|
||||
if (allSettings.kb_rulehint_threshold !== undefined) {
|
||||
kbRuleHintThreshold.value = allSettings.kb_rulehint_threshold;
|
||||
}
|
||||
if (allSettings.kb_toolrule_threshold !== undefined) {
|
||||
kbToolRuleThreshold.value = allSettings.kb_toolrule_threshold;
|
||||
}
|
||||
if (allSettings.kb_promptrule_threshold !== undefined) {
|
||||
kbPromptRuleThreshold.value = allSettings.kb_promptrule_threshold;
|
||||
if (allSettings.kb_rule_threshold !== undefined) {
|
||||
kbRuleThreshold.value = allSettings.kb_rule_threshold;
|
||||
}
|
||||
if (allSettings.kb_replyrule_threshold !== undefined) {
|
||||
kbReplyRuleThreshold.value = allSettings.kb_replyrule_threshold;
|
||||
@@ -1772,10 +1749,10 @@ async function deleteUser(userId: number) {
|
||||
<p class="field-hint">How many snippets and issues one edit may be offered (1–10). Reach for this rather than the threshold when the hint feels long: lowering it keeps the best matches and drops the tail, while raising the bar above throws away good matches along with weak ones.</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-rulehint-threshold">Standing-rule confidence threshold (0–1)</label>
|
||||
<label for="kb-rule-threshold">Standing-rule confidence threshold (0–1)</label>
|
||||
<input
|
||||
id="kb-rulehint-threshold"
|
||||
v-model="kbRuleHintThreshold"
|
||||
id="kb-rule-threshold"
|
||||
v-model="kbRuleThreshold"
|
||||
type="number"
|
||||
min="0"
|
||||
max="1"
|
||||
@@ -1784,14 +1761,14 @@ async function deleteUser(userId: number) {
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">
|
||||
The same hint can mention a standing rule whose trigger resembles the
|
||||
code being written. <em>Every</em> rule is eligible — nothing is
|
||||
preloaded any more, so this is the only way a rule reaches a write.
|
||||
Stricter than the threshold above, because there are far fewer rules
|
||||
than snippets: with a small set something always ranks first, so the
|
||||
bar has to carry more of the judgement. If rules arrive unread, or one
|
||||
you needed never showed up, that is Claude's to notice and correct
|
||||
from the telemetry — and the budget below is usually the better lever.
|
||||
The bar for a standing rule or preference to be offered at all,
|
||||
whether against the code being written, a command about to run, or
|
||||
what you just said. <em>Every</em> rule is eligible — nothing is
|
||||
preloaded, so this is how a rule reaches the work. One bar serves all
|
||||
three: they search the same rules, and how many each may show is set
|
||||
by its own budget below, which is usually the better lever. If rules
|
||||
arrive unread, or one you needed never showed up, that is Claude's to
|
||||
notice and correct from the telemetry.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
@@ -1834,26 +1811,6 @@ async function deleteUser(userId: number) {
|
||||
only — file edits are never held.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-toolrule-threshold">Command confidence threshold (0–1)</label>
|
||||
<input
|
||||
id="kb-toolrule-threshold"
|
||||
v-model="kbToolRuleThreshold"
|
||||
type="number"
|
||||
min="0"
|
||||
max="1"
|
||||
step="0.01"
|
||||
class="fs-input input"
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">
|
||||
The bar for a rule surfacing before a <em>command</em> runs, where the
|
||||
query is the command text rather than code. Lower than the one above
|
||||
on purpose: a shell command is short, so it scores lower for the same
|
||||
relevance — at a shared bar this arm spoke on 2% of calls against the
|
||||
write path's 37%.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-toolrule-topk">Max rules per command</label>
|
||||
<input
|
||||
@@ -1868,26 +1825,6 @@ async function deleteUser(userId: number) {
|
||||
/>
|
||||
<p class="field-hint">How many standing rules one command may be shown (1–10). This is the busiest arm there is — it fires before every command — so its budget is the one most worth keeping small.</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-promptrule-threshold">Prompt confidence threshold (0–1)</label>
|
||||
<input
|
||||
id="kb-promptrule-threshold"
|
||||
v-model="kbPromptRuleThreshold"
|
||||
type="number"
|
||||
min="0"
|
||||
max="1"
|
||||
step="0.01"
|
||||
class="fs-input input"
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">
|
||||
The bar for a rule or preference surfacing against <em>what you just
|
||||
said</em>, before anything is done. It is the only moment that reaches
|
||||
a rule about how to answer rather than how to act, so it runs once a
|
||||
turn rather than once a tool call. A third query shape — your prose,
|
||||
not a command or a file — which is why it carries its own number.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-promptrule-topk">Max rules per prompt</label>
|
||||
<input
|
||||
|
||||
Reference in New Issue
Block a user