align(4909d): every rule arm that annotates reads one bar, kb_rule_threshold at 0.70
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m7s
CI & Build / Python tests (push) Failing after 1m30s
CI & Build / Build & push image (push) Skipped

prompt_rule, pre_tool_rule and write_path_rule now share one settings key,
RULE_FLOOR_KEY = "kb_rule_threshold", default 0.70. The reply backstop keeps
its own 0.80, because there the floor is a stop bar. Moving the shared floor
through tune_retrieval or the Settings form records a tuning event on every
surface that reads it (set_dial, via retrieval_surfaces.sharing_floor), and
the result names the surfaces it carried as `moved_with`. Settings shows one
"Standing-rule confidence threshold" control where it showed three.

Operator ruling (2026-10-05): one shared bar for the rule arms, "especially
since we're only taking the first x number of them" -- the budget limits
what is shown; the bar only decides what is worth ranking. Kept a dial.

Why 0.70, from records read rather than percentiles:
- write_path_rule near-misses at 0.7199-0.72 were all false positives
  (rule 149 against a Lidarr comment, 157 against a retention Svelte file,
  121 against an Archives restore, 164 against a config test, 131 against a
  Go package), so the old 0.72 was not hiding real hits -- but the band does
  noise control on this arm, so dropping 0.02 costs little.
- prompt_rule: the operator set 0.70 by hand on 2026-09-18, after a 0.60
  experiment admitted junk and was reverted. 0.70 is the number with a human
  decision behind it.
- pre_tool_rule lines near 0.68 this session were mostly noise; rising to
  0.70 trims them, and the band still narrows what clears.

Retired keys: kb_promptrule_threshold, kb_toolrule_threshold and
kb_rulehint_threshold are no longer read. Rows an install stored under them
stay in the settings table, inert. On this instance the effective bars move
pre_tool_rule 0.68 -> 0.70, write_path_rule 0.72 -> 0.70, prompt_rule stays
0.70; record_release_defaults writes those per-surface release rows on boot,
so the telemetry shift has an event to explain it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-22 18:32:09 -04:00
co-authored by Claude Opus 5.5
parent f77d18cb63
commit c8ede427ca
10 changed files with 175 additions and 211 deletions
+25 -88
View File
@@ -159,16 +159,10 @@ const kbWritePathEnabled = ref(true);
// code embeddings sit on a much higher similarity floor than prose, so 0.55
// lets unrelated code through. Shares top-k, not the threshold.
const kbWritePathThreshold = ref("0.68");
const kbRuleHintThreshold = ref("0.72");
// One bar for every rule hint — prompt, command and edit alike. They search
// the same rules, and what limits each is its budget, not the bar.
const kbRuleThreshold = ref("0.70");
const kbCheckpointThreshold = ref("0.80");
// The two ACT arms do not share a bar. A write-path query is a code payload; a
// pre-tool query is a shell command, often under a dozen words — less text,
// less signal, lower scores for the same relevance — so one shared bar makes
// them behave like different subsystems.
const kbToolRuleThreshold = ref("0.68");
// And the prompt boundary is a third query shape again — the operator's own
// prose rather than anything a tool produced.
const kbPromptRuleThreshold = ref("0.72");
// The reply arm: its bar is the bar at which a finished reply
// is HELD for one read, so it sits with the checkpoint, not with the hints.
const kbReplyRuleThreshold = ref("0.8");
@@ -367,14 +361,10 @@ async function saveKbInject() {
// Same `|| default` reasoning: falling back to 0 would surface every
// snippet in the corpus on every edit, which is the failure this knob fixes.
const wpT = asBar(kbWritePathThreshold.value, 0.68);
// Same `|| default` reasoning again, and it bites harder here: a rule hint
// fires on every write, so a fallback of 0 would attach a standing rule to
// every edit in the session.
const rhT = asBar(kbRuleHintThreshold.value, 0.72);
// Same `|| default` guard, and the same reason: this arm fires before every
// Bash call, so a fallback of 0 would put a rule in front of every command.
const trT = asBar(kbToolRuleThreshold.value, 0.68);
const prT = asBar(kbPromptRuleThreshold.value, 0.72);
// Same `|| default` reasoning again, and it bites harder here: rule hints
// fire on every edit and before every command, so a fallback of 0 would put
// a standing rule beside each of them.
const ruleT = asBar(kbRuleThreshold.value, 0.7);
// The checkpoint bar, and the `|| default` guard matters most here of all:
// this is the only number that can STOP a call, so a fallback of 0 would
// hold the first command of every session behind whatever ranked first.
@@ -409,10 +399,8 @@ async function saveKbInject() {
kbGateNoteOverlapFloor.value = String(gateOverlap);
kbPlanMatchThreshold.value = String(planT);
kbWritePathThreshold.value = String(wpT);
kbRuleHintThreshold.value = String(rhT);
kbRuleThreshold.value = String(ruleT);
kbCheckpointThreshold.value = String(cpT);
kbToolRuleThreshold.value = String(trT);
kbPromptRuleThreshold.value = String(prT);
kbReplyRuleThreshold.value = String(ryT);
savingKbInject.value = true;
kbInjectSaved.value = false;
@@ -426,20 +414,15 @@ async function saveKbInject() {
// measurements that split them.
kb_writepath_enabled: kbWritePathEnabled.value ? 'true' : 'false',
kb_writepath_threshold: String(wpT),
// A THIRD corpus with a third bar — see RULEHINT_DEFAULT_THRESHOLD
// in services/plugin_context.py for why rules cannot share the
// code threshold any more than code could share the prose one.
kb_rulehint_threshold: String(rhT),
// A third corpus with a third bar — see RULE_FLOOR_DEFAULT in
// services/retrieval_pipeline.py for why every rule hint shares it.
kb_rule_threshold: String(ruleT),
// The one bar that stops a call rather than annotating it. Its own key,
// never derived from the two rule bars above: it answers a different
// never derived from the rule bar above: it answers a different
// question of the same scores — not "is this worth showing" but "is the
// corpus confident enough to be read first" — and a number that moves
// when another moves is a number nobody can reason about.
kb_checkpoint_threshold: String(cpT),
// A FOURTH and FIFTH bar, and they are separate keys on purpose: one
// number cannot serve arms whose queries are different shapes. Moving one must not move the others.
kb_toolrule_threshold: String(trT),
kb_promptrule_threshold: String(prT),
// The reply backstop's own bar: it HOLDS a reply, so like the
// checkpoint it is never derived from a hint bar.
kb_replyrule_threshold: String(ryT),
@@ -864,14 +847,8 @@ onMounted(async () => {
if (allSettings.kb_checkpoint_threshold !== undefined) {
kbCheckpointThreshold.value = allSettings.kb_checkpoint_threshold;
}
if (allSettings.kb_rulehint_threshold !== undefined) {
kbRuleHintThreshold.value = allSettings.kb_rulehint_threshold;
}
if (allSettings.kb_toolrule_threshold !== undefined) {
kbToolRuleThreshold.value = allSettings.kb_toolrule_threshold;
}
if (allSettings.kb_promptrule_threshold !== undefined) {
kbPromptRuleThreshold.value = allSettings.kb_promptrule_threshold;
if (allSettings.kb_rule_threshold !== undefined) {
kbRuleThreshold.value = allSettings.kb_rule_threshold;
}
if (allSettings.kb_replyrule_threshold !== undefined) {
kbReplyRuleThreshold.value = allSettings.kb_replyrule_threshold;
@@ -1772,10 +1749,10 @@ async function deleteUser(userId: number) {
<p class="field-hint">How many snippets and issues one edit may be offered (1–10). Reach for this rather than the threshold when the hint feels long: lowering it keeps the best matches and drops the tail, while raising the bar above throws away good matches along with weak ones.</p>
</div>
<div class="field">
<label for="kb-rulehint-threshold">Standing-rule confidence threshold (0–1)</label>
<label for="kb-rule-threshold">Standing-rule confidence threshold (0–1)</label>
<input
id="kb-rulehint-threshold"
v-model="kbRuleHintThreshold"
id="kb-rule-threshold"
v-model="kbRuleThreshold"
type="number"
min="0"
max="1"
@@ -1784,14 +1761,14 @@ async function deleteUser(userId: number) {
style="max-width: 8rem"
/>
<p class="field-hint">
The same hint can mention a standing rule whose trigger resembles the
code being written. <em>Every</em> rule is eligible — nothing is
preloaded any more, so this is the only way a rule reaches a write.
Stricter than the threshold above, because there are far fewer rules
than snippets: with a small set something always ranks first, so the
bar has to carry more of the judgement. If rules arrive unread, or one
you needed never showed up, that is Claude's to notice and correct
from the telemetry — and the budget below is usually the better lever.
The bar for a standing rule or preference to be offered at all,
whether against the code being written, a command about to run, or
what you just said. <em>Every</em> rule is eligible — nothing is
preloaded, so this is how a rule reaches the work. One bar serves all
three: they search the same rules, and how many each may show is set
by its own budget below, which is usually the better lever. If rules
arrive unread, or one you needed never showed up, that is Claude's to
notice and correct from the telemetry.
</p>
</div>
<div class="field">
@@ -1834,26 +1811,6 @@ async function deleteUser(userId: number) {
only — file edits are never held.
</p>
</div>
<div class="field">
<label for="kb-toolrule-threshold">Command confidence threshold (0–1)</label>
<input
id="kb-toolrule-threshold"
v-model="kbToolRuleThreshold"
type="number"
min="0"
max="1"
step="0.01"
class="fs-input input"
style="max-width: 8rem"
/>
<p class="field-hint">
The bar for a rule surfacing before a <em>command</em> runs, where the
query is the command text rather than code. Lower than the one above
on purpose: a shell command is short, so it scores lower for the same
relevance — at a shared bar this arm spoke on 2% of calls against the
write path's 37%.
</p>
</div>
<div class="field">
<label for="kb-toolrule-topk">Max rules per command</label>
<input
@@ -1868,26 +1825,6 @@ async function deleteUser(userId: number) {
/>
<p class="field-hint">How many standing rules one command may be shown (1–10). This is the busiest arm there is — it fires before every command — so its budget is the one most worth keeping small.</p>
</div>
<div class="field">
<label for="kb-promptrule-threshold">Prompt confidence threshold (0–1)</label>
<input
id="kb-promptrule-threshold"
v-model="kbPromptRuleThreshold"
type="number"
min="0"
max="1"
step="0.01"
class="fs-input input"
style="max-width: 8rem"
/>
<p class="field-hint">
The bar for a rule or preference surfacing against <em>what you just
said</em>, before anything is done. It is the only moment that reaches
a rule about how to answer rather than how to act, so it runs once a
turn rather than once a tool call. A third query shape — your prose,
not a command or a file — which is why it carries its own number.
</p>
</div>
<div class="field">
<label for="kb-promptrule-topk">Max rules per prompt</label>
<input
+20 -64
View File
@@ -144,7 +144,7 @@ WRITEPATH_THRESHOLD_KEY = SURFACES["write_path"].floor_key
WRITEPATH_DEFAULT_ENABLED = True
WRITEPATH_DEFAULT_THRESHOLD = SURFACES["write_path"].floor_default
# The standing-rule arm gets its own bar, for a third corpus: 0.68 above is
# The standing-rule arms get their own bar, for a third corpus: 0.68 above is
# calibrated on code against note PROSE, not code against RULE TEXT.
#
# THE STRUCTURAL ARGUMENT, which holds on any install, including one with six
@@ -155,64 +155,27 @@ WRITEPATH_DEFAULT_THRESHOLD = SURFACES["write_path"].floor_default
# pool always returns something, so "the best match cleared the bar"
# drifts from "a good match exists" toward "N things were ranked". A bar
# calibrated for best-of-thousands is cleared by best-of-forty as
# arithmetic rather than relevance. (The arms search every tier, which
# makes the pool larger; the bar still errs toward silence on purpose.)
# arithmetic rather than relevance.
# 2. Rules are short imperative technical English — a far more HOMOGENEOUS
# corpus than note prose, and a more homogeneous corpus has a HIGHER
# noise floor than prose's 0.55-0.63.
#
# WHY 0.72 AND NOT A NUMBER OFF A HISTOGRAM. The exact offset between prose's
# floor and rule-text's is not derivable in general — it depends on how an
# install writes its rules — so the default errs deliberately toward SILENCE
# rather than toward recall, on an asymmetry that is itself structural: this
# hint fires on EVERY write. A missed rule is recoverable, because the rule is
# still in Grimoire and the agent can search it. A hint that cries wolf is not:
# it teaches the reader to skip the whole block, and the surface is lost along
# with the true positives it would have carried.
# ONE BAR FOR EVERY RULE ARM THAT ANNOTATES. The write path, the command arm
# and the prompt arm read the same key (`RULE_FLOOR_KEY` in retrieval_pipeline):
# they search the same rules, and what limits each one is its budget and band.
#
# TUNE IT FROM YOUR OWN INSTANCE: `retrieval_telemetry` reports
# `rule_usage.pull_through`. Raise this if rules arrive unread; lower it if
# rules you needed never arrived. What would RETIRE it: a cross-encoder rerank,
# which would make a similarity bar the wrong control entirely.
#
# This bar is the WRITE-PATH arm's. A code payload is long and rich, which is
# the case 0.72 suits; the command arm has its own bar below.
# `rule_usage.pull_through` and `near_miss_samples`. Raise it if rules arrive
# unread; lower it if rules you needed never arrived. What would RETIRE it: a
# cross-encoder rerank, which would make a similarity bar the wrong control
# entirely.
RULEHINT_THRESHOLD_KEY = SURFACES["write_path_rule"].floor_key
RULEHINT_DEFAULT_THRESHOLD = SURFACES["write_path_rule"].floor_default
# THE COMMAND ARM'S OWN BAR, AND WHY IT IS NOT THE WRITE PATH'S.
#
# A write-path query is a code payload, long and rich, while a pre-tool query
# is a shell command — often under a dozen words. Less text, less signal, lower
# scores for the same relevance. At the write path's 0.72 the command arm
# turns nearly everything away, with its refused mass piled just under the
# line, which is the shape a bar set too high leaves.
#
# WHAT 0.68 IS CALIBRATED AGAINST. Replayed queries, consequential acts
# against innocuous ones, scored against a rule corpus with symptom-bearing
# triggers:
#
# 0.7571 git push origin dev consequential
# 0.7245 cd ...; git fetch; git add -A consequential
# 0.7193 git pull --rebase origin dev consequential
# 0.6850 docker compose up -d consequential
# ---------------------------------------- 0.68
# 0.6735 wc -l src/*.py && date innocuous
# 0.6544 grep -rn useState src/ innocuous
# 0.6099 sed -n '120,160p' package.json innocuous
# 0.6056 ls -la && cat README.md innocuous
#
# At 0.72 three of the four consequential acts would retrieve nothing, though
# the right rules ranked first for each.
#
# THE SEPARATION IS NARROW (about 0.01), and that is a caveat, not a result.
# Eight probes set a direction; they do not settle a number.
# `near_miss_samples` on real traffic is what settles it, and this is the bar
# to re-read first.
#
# WHY A LOWER BAR IS SAFE HERE. The band does noise control downstream: a marginal hit that
# clears the bar still has to score within `_RULEHINT_BAND` of the top to be
# rendered. The bar's job shrank, so the bar can.
# THE COMMAND ARM READS THE SAME BAR. A shell command is short, so it scores
# lower than a code payload for the same relevance and the shared bar admits
# less here. The band does noise control downstream: a hit that clears the bar
# still has to score within `_RULEHINT_BAND` of the top to be rendered.
#
# The innocuous scores near the line are largely CROSS-PROJECT BLEED rather
# than bad ranking: another project's shell-command rule matching a shell
@@ -478,17 +441,11 @@ def _autoinject_query(prompt: str, context: str) -> str:
# The operator's message is the only query that exists before one is composed,
# and this arm is what runs against it.
PROMPTRULE_THRESHOLD_KEY = SURFACES["prompt_rule"].floor_key
# INHERITED FROM THE ACT ARMS. 0.72 is tuned against
# code and shell commands. An operator's prose is a different query shape
# against the same documents, and nothing yet says the two distributions line
# up — triggers are written in the vocabulary of the MOMENT, which for most
# rules is act vocabulary, so prose may well score lower across the board.
#
# Starting at the act arms' number anyway is deliberate: it is the only value
# with evidence behind it, and guessing lower would put an unmeasured bar in
# front of a corpus that binds. Every call is logged under `prompt_rule`, so
# real traffic settles it — read `near_miss_samples` before moving this, not
# the percentile alone.
# THE SAME BAR AS THE ACT ARMS. An operator's prose is a different query shape
# against the same documents, and triggers are written in the vocabulary of
# the moment, which for most rules is act vocabulary. Every call is logged
# under `prompt_rule`, so read `near_miss_samples` before moving the shared
# bar, not the percentile alone.
PROMPTRULE_DEFAULT_THRESHOLD = SURFACES["prompt_rule"].floor_default
# ANCHORED ON THIS HOOK'S BUDGET rather than the act arms'. This arm fires
@@ -1171,9 +1128,8 @@ async def get_writepath_config(user_id: int) -> dict:
independent pairs, resolved from the registry.
Each floor's reason is recorded where its default is: WRITEPATH (code
embeddings sit on a much higher similarity floor than prose), RULEHINT (a
third corpus again), TOOLRULE (a shell command is a different query shape
from a code payload and scores lower for the same relevance).
embeddings sit on a much higher similarity floor than prose) and RULEHINT
(a third corpus again, whose one bar both rule arms read).
THE BUDGET IS SPLIT TOO. "How many titles at once" does not mean the same
thing on both surfaces: this arm fires before every Write and Edit while
+20 -6
View File
@@ -460,6 +460,20 @@ class RuleArm:
declared: Declared | None = None
# ONE BAR FOR EVERY RULE ARM THAT ANNOTATES. The prompt, command and write
# arms search the same corpus, and what limits what they show is the budget
# and the band, not the floor. The floor only answers "is this worth ranking
# at all", and that question has one answer per corpus, not one per query
# shape — so the three share one settings key, and moving it moves all three.
# `set_dial` records the move on every surface that shares the key.
#
# Set low on purpose: a hit under it is never shown, so a bar set too high
# costs a rule that binds, while one set a little low costs a line the band
# usually drops. The reply backstop keeps its own number, because there the
# floor is a stop bar and not a hint bar.
RULE_FLOOR_KEY = "kb_rule_threshold"
RULE_FLOOR_DEFAULT = 0.70
# The arms differ on purpose where these flags differ. Whether the prompt arm
# should band, and whether the act arms should reserve a preference, are open
# questions to settle on telemetry, not by symmetry.
@@ -468,8 +482,8 @@ PROMPT_RULE = RuleArm(
preference_slot=True,
tuning=Surface(
name="prompt_rule",
floor_key="kb_promptrule_threshold",
floor_default=0.72,
floor_key=RULE_FLOOR_KEY,
floor_default=RULE_FLOOR_DEFAULT,
budget_key="kb_promptrule_top_k",
budget_default=3,
asks="the operator's message, against rule triggers",
@@ -483,8 +497,8 @@ PRE_TOOL_RULE = RuleArm(
preference_slot=False,
tuning=Surface(
name="pre_tool_rule",
floor_key="kb_toolrule_threshold",
floor_default=0.68,
floor_key=RULE_FLOOR_KEY,
floor_default=RULE_FLOOR_DEFAULT,
budget_key="kb_toolrule_top_k",
budget_default=5,
asks="the command about to run, against rule triggers",
@@ -498,8 +512,8 @@ WRITE_PATH_RULE = RuleArm(
preference_slot=False,
tuning=Surface(
name="write_path_rule",
floor_key="kb_rulehint_threshold",
floor_default=0.72,
floor_key=RULE_FLOOR_KEY,
floor_default=RULE_FLOOR_DEFAULT,
budget_key="kb_rulehint_top_k",
budget_default=5,
asks="the code being written, against rule triggers",
@@ -110,6 +110,9 @@ def dial_for_key(key: str) -> tuple[str, str] | None:
Derived rather than listed so a seventh surface is covered the moment it is
added here, which is the only way this stays true.
A floor key several surfaces share resolves to the first of them;
`set_dial` records the move on every surface in `sharing_floor`.
"""
for surface in SURFACES.values():
if key == surface.floor_key:
@@ -119,6 +122,16 @@ def dial_for_key(key: str) -> tuple[str, str] | None:
return None
def sharing_floor(name: str) -> list[str]:
"""Every surface that reads the same floor key as `name`, itself included.
The rule arms share one bar, so moving one surface's floor moves its
siblings, and each of them needs the event that says so.
"""
key = get_surface(name).floor_key
return [n for n, s in SURFACES.items() if s.floor_key == key]
async def floor_for(user_id: int, name: str) -> float:
"""This install's current floor for a surface, clamped to [0, 1]."""
s = get_surface(name)
+15 -6
View File
@@ -44,6 +44,7 @@ from grimoire.services.retrieval_surfaces import (
budget_for,
floor_for,
get_surface,
sharing_floor,
surface_names,
)
from grimoire.services.settings import set_setting
@@ -257,6 +258,10 @@ async def set_dial(
"reason": text, "actor": actor, "unchanged": True,
}
# A shared floor moves every surface that reads it, and each one's history
# has to show the move, or its telemetry changes with no event to explain it.
moved = sharing_floor(surface) if dial == "floor" else [surface]
await set_setting(user_id, key, stored)
async with async_session() as session:
# Stamped with the space this number was chosen in. Read at
@@ -264,12 +269,13 @@ async def set_dial(
# the caller recording it are the same call, so there is no window in
# which they could disagree.
stamp = calibration_stamp()
session.add(RetrievalTuningEvent(
user_id=user_id, surface=surface, dial=dial,
old_value=old, new_value=applied, actor=actor, reason=text,
embedding_model=stamp["embedding_model"],
shape_version=stamp["shape_version"],
))
for name in moved:
session.add(RetrievalTuningEvent(
user_id=user_id, surface=name, dial=dial,
old_value=old, new_value=applied, actor=actor, reason=text,
embedding_model=stamp["embedding_model"],
shape_version=stamp["shape_version"],
))
await session.commit()
return {
@@ -286,6 +292,9 @@ async def set_dial(
# Always present, both ways round: a caller that has to test for the
# key's absence to learn the answer will eventually forget to.
"unchanged": False,
# The other surfaces this move carried with it: a shared floor is one
# number, and a caller tuning one rule arm has tuned them all.
"moved_with": [n for n in moved if n != surface],
}
+21 -5
View File
@@ -71,13 +71,29 @@ def test_every_surface_name_is_a_real_telemetry_source():
)
def test_no_two_surfaces_share_a_settings_key():
"""One dial, one label."""
floors = [s.floor_key for s in rs.SURFACES.values()]
def test_a_shared_settings_key_is_one_bar_with_one_default():
"""One dial, one label. Surfaces may share a floor key, as the rule arms
do, but then they must ship the same default: otherwise the one number
means something different on each surface until somebody sets it."""
by_key: dict[str, set[float]] = {}
for s in rs.SURFACES.values():
by_key.setdefault(s.floor_key, set()).add(s.floor_default)
split = {k: v for k, v in by_key.items() if len(v) > 1}
assert not split, f"a shared floor key ships two defaults: {split}"
budgets = [s.budget_key for s in rs.SURFACES.values()]
assert len(set(floors)) == len(floors), f"duplicate floor key in {floors}"
assert len(set(budgets)) == len(budgets), f"duplicate budget key in {budgets}"
assert not (set(floors) & set(budgets)), "a floor key doubles as a budget key"
assert not (set(by_key) & set(budgets)), "a floor key doubles as a budget key"
def test_the_rule_arms_that_annotate_share_one_bar_and_the_reply_stop_does_not():
from grimoire.services.retrieval_pipeline import RULE_FLOOR_KEY
assert sorted(rs.sharing_floor("prompt_rule")) == [
"pre_tool_rule", "prompt_rule", "write_path_rule",
]
assert rs.SURFACES["prompt_rule"].floor_key == RULE_FLOOR_KEY
assert rs.sharing_floor("reply_rule") == ["reply_rule"]
def test_every_surface_says_what_it_asks_over_what_and_how_often():
+33 -2
View File
@@ -86,9 +86,9 @@ async def test_a_good_change_writes_the_setting_and_the_event():
setter.assert_awaited_once()
_uid, key, stored = setter.await_args.args
assert key == "kb_promptrule_threshold" and stored == "0.66"
assert key == "kb_rule_threshold" and stored == "0.66"
event = session.add.call_args.args[0]
event = session.add.call_args_list[0].args[0]
assert event.surface == "prompt_rule" and event.dial == "floor"
# The before-value is what makes the history answerable.
assert event.old_value == 0.72 and event.new_value == 0.66
@@ -97,6 +97,37 @@ async def test_a_good_change_writes_the_setting_and_the_event():
assert out["clamped"] is False
@pytest.mark.asyncio
async def test_a_shared_floor_records_the_move_on_every_surface_that_reads_it():
"""One key, one write, an event per surface. A rule arm whose bar moved
with no event of its own would show a telemetry shift nothing explains."""
session, ctx = _patches(floor=0.70)
with ctx[0], patch.object(rt, "set_setting", AsyncMock()) as setter, \
ctx[2], ctx[3]:
out = await rt.set_dial(
1, "pre_tool_rule", "floor", 0.74,
reason="read the 10 highest declines: every one was a real miss",
)
setter.assert_awaited_once()
recorded = {c.args[0].surface for c in session.add.call_args_list}
assert recorded == {"pre_tool_rule", "prompt_rule", "write_path_rule"}
assert sorted(out["moved_with"]) == ["prompt_rule", "write_path_rule"]
@pytest.mark.asyncio
async def test_a_budget_moves_only_its_own_surface():
session, ctx = _patches(budget=3)
with ctx[0], patch.object(rt, "set_setting", AsyncMock()), ctx[2], ctx[3]:
out = await rt.set_dial(
1, "prompt_rule", "budget", 4,
reason="on-point hits sit at ranks 4 and 5 as often as rank 1",
)
assert [c.args[0].surface for c in session.add.call_args_list] == ["prompt_rule"]
assert out["moved_with"] == []
@pytest.mark.asyncio
@pytest.mark.parametrize("dial, sent, applied", [
("floor", 1.4, 1.0),
+7 -2
View File
@@ -76,10 +76,15 @@ def test_the_route_does_not_take_the_actor_from_the_caller():
def test_every_registry_key_is_recognised_as_a_dial():
"""Derived from the registry, both directions. A seventh surface added
without a row here is a floor that can be written silently again."""
from grimoire.services.retrieval_surfaces import SURFACES, dial_for_key
from grimoire.services.retrieval_surfaces import (
SURFACES, dial_for_key, sharing_floor,
)
for name, surface in SURFACES.items():
assert dial_for_key(surface.floor_key) == (name, "floor")
# A shared floor resolves to one of the surfaces that read it, and
# `set_dial` records the move on all of them.
owner, dial = dial_for_key(surface.floor_key)
assert dial == "floor" and name in sharing_floor(owner)
assert dial_for_key(surface.budget_key) == (name, "budget")
+5 -4
View File
@@ -41,7 +41,7 @@ ROOT = pathlib.Path(__file__).resolve().parents[1]
_SERVICES = ROOT / "src" / "grimoire" / "services"
_VUE = ROOT / "frontend" / "src" / "views" / "SettingsView.vue"
# THE SIX RETRIEVAL FLOORS live in one registry, so their side of
# THE RETRIEVAL FLOORS live in one registry, so their side of
# the pairing is a SURFACE NAME rather than a module constant. That is strictly
# better for this guard: the surface name is also the telemetry source, so a row
# here names the same arm the readout does, and a floor that moved cannot be
@@ -49,9 +49,10 @@ _VUE = ROOT / "frontend" / "src" / "views" / "SettingsView.vue"
_SURFACE_PAIRS = (
("auto_inject", "kbInjectThreshold"),
("write_path", "kbWritePathThreshold"),
("write_path_rule", "kbRuleHintThreshold"),
("pre_tool_rule", "kbToolRuleThreshold"),
("prompt_rule", "kbPromptRuleThreshold"),
# The three rule hints read one bar, so one control serves them.
("write_path_rule", "kbRuleThreshold"),
("pre_tool_rule", "kbRuleThreshold"),
("prompt_rule", "kbRuleThreshold"),
("reply_rule", "kbReplyRuleThreshold"),
)
+16 -34
View File
@@ -485,13 +485,13 @@ async def test_writepath_threshold_is_operator_tunable_and_clamped():
@pytest.mark.asyncio
async def test_the_rule_arm_has_its_own_tunable_bar():
async def test_the_rule_arms_have_their_own_tunable_bar():
"""Settable and clamped again, for the THIRD corpus.
Separate from the code threshold above and separately settable, because the
two are measured against different things: 0.68 was derived from code
against note PROSE, and rules are short imperative technical
English — a more homogeneous corpus whose noise floor sits higher.
two are measured against different things: the code bar against note
PROSE, the rule bar against short imperative technical English — a more
homogeneous corpus whose noise floor sits higher.
"""
from grimoire.services import plugin_context as pc
@@ -524,33 +524,28 @@ async def test_the_two_write_path_bars_are_independent():
@pytest.mark.asyncio
async def test_the_two_act_arms_read_independent_rule_bars():
"""The command arm's bar moves without the write path's.
async def test_both_act_arms_read_the_one_rule_bar():
"""One stored value moves the command arm and the write arm together.
The two act arms behave like different subsystems at the same number,
because a code payload is long and rich where a shell command is short. A config assembler that
reads one key into both fields would silently undo that, and the symptom
would be invisible: both arms would simply agree again.
They search the same rules, and what limits each is its budget and band.
A config assembler that read a second key into either field would split
the bar again with nothing in the settings to show it.
"""
from grimoire.services import plugin_context as pc
stored = {pc.RULEHINT_THRESHOLD_KEY: "0.75", pc.TOOLRULE_THRESHOLD_KEY: "0.61"}
assert pc.TOOLRULE_THRESHOLD_KEY == pc.RULEHINT_THRESHOLD_KEY
stored = {pc.RULEHINT_THRESHOLD_KEY: "0.61"}
with _stored_settings(stored):
cfg = await pc.get_writepath_config(1)
assert cfg["rule_threshold"] == 0.75
assert cfg["rule_threshold"] == 0.61
assert cfg["tool_rule_threshold"] == 0.61
@pytest.mark.asyncio
async def test_a_garbage_command_bar_falls_back_to_its_own_default():
"""Not to 0.0, and not to the write path's default.
Falling back to 0.0 would attach a rule to every Bash call in the session;
falling back to the sibling's default would quietly re-merge the two bars
that are kept separate, which is the harder failure to see because the arm
keeps working.
"""
async def test_a_garbage_rule_bar_falls_back_to_its_default():
"""Not to 0.0, which would attach a rule to every Bash call and every
write in the session."""
from grimoire.services import plugin_context as pc
stored = {pc.TOOLRULE_THRESHOLD_KEY: "banana"}
@@ -558,20 +553,7 @@ async def test_a_garbage_command_bar_falls_back_to_its_own_default():
cfg = await pc.get_writepath_config(1)
assert cfg["tool_rule_threshold"] == pc.TOOLRULE_DEFAULT_THRESHOLD
def test_the_command_bar_defaults_below_the_write_path_bar():
"""A DIRECTION check, like its sibling above, and for the same reason: the
value is measured against one corpus, the relationship is not.
A shell command carries less text than a code payload and therefore scores
lower for the same relevance; at the write path's bar, consequential
commands retrieve nothing. Tuning either value stays free; inverting the
relationship would mute the command arm.
"""
from grimoire.services import plugin_context as pc
assert pc.TOOLRULE_DEFAULT_THRESHOLD < pc.RULEHINT_DEFAULT_THRESHOLD
assert cfg["rule_threshold"] == pc.RULEHINT_DEFAULT_THRESHOLD
def test_the_rule_bar_defaults_above_the_code_bar():