feat(500): retire the fixed-question preference arm - completion-report preferences ride the moments
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Python tests (push) Successful in 1m54s
CI & Build / Build & push image (push) Successful in 31s

report_preference searched one constant string when a task closed and handed
matches back as update_task's reply_preferences. Milestone 500 step 3 delivers
the reply shapes and the preferences mounted beside them on the moments, so
the arm goes, whole (rule 22):

- services/reply_preferences.py, the REPORT_PREFERENCE RuleArm, its re-scorer
  and corpus entries, and the REPORTPREF constants
- update_task's reply_preferences block and its cue; the docstring now points
  at moment_rules
- the fixed_query field and the fixed_query_never_clears warning: this arm was
  the only one that set it, so the concept and the cannot_decline exemption
  go with it
- the two Settings fields for its floor and budget
- migration 0121 deletes its rows (logs, judgments, rule-usage events, tuning
  history, settings keys), operator-approved 2026-10-09. Without it the
  readout would call the source unregistered and its surfacings would count
  as ambient. The unindexed rule_usage delete is bounded by created_at.

#5496 (step 4 of milestone 500).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 15:51:04 -04:00
co-authored by Claude Opus 5.5
parent cee1eb1328
commit f68b73f922
19 changed files with 94 additions and 619 deletions
-55
View File
@@ -168,7 +168,6 @@ const kbToolRuleThreshold = ref("0.68");
// And the prompt boundary is a third query shape again — the operator's own
// prose rather than anything a tool produced (#3852).
const kbPromptRuleThreshold = ref("0.72");
const kbReportPrefThreshold = ref("0.72");
// The reply arm (milestone 458): its bar is the bar at which a finished reply
// is HELD for one read, so it sits with the checkpoint, not with the hints.
const kbReplyRuleThreshold = ref("0.8");
@@ -181,7 +180,6 @@ const kbWritePathTopK = ref("3");
const kbRuleHintTopK = ref("5");
const kbToolRuleTopK = ref("5");
const kbPromptRuleTopK = ref("3");
const kbReportPrefTopK = ref("3");
const kbReplyRuleTopK = ref("1");
// What has been changed about retrieval, newest first — the review surface for
// changes the model made on the operator's behalf (#4102).
@@ -387,7 +385,6 @@ async function saveKbInject() {
// this is the only number that can STOP a call, so a fallback of 0 would
// hold the first command of every session behind whatever ranked first.
const cpT = asBar(kbCheckpointThreshold.value, 0.8);
const rpT = asBar(kbReportPrefThreshold.value, 0.72);
// A stop bar like the checkpoint's: a fallback of 0 would hold every reply.
const ryT = asBar(kbReplyRuleThreshold.value, 0.8);
// The budgets, clamped the way the server clamps them: a whole number in
@@ -400,13 +397,11 @@ async function saveKbInject() {
const rhK = asK(kbRuleHintTopK.value, 5);
const trK = asK(kbToolRuleTopK.value, 5);
const prK = asK(kbPromptRuleTopK.value, 3);
const rpK = asK(kbReportPrefTopK.value, 3);
const ryK = asK(kbReplyRuleTopK.value, 1);
kbWritePathTopK.value = String(wpK);
kbRuleHintTopK.value = String(rhK);
kbToolRuleTopK.value = String(trK);
kbPromptRuleTopK.value = String(prK);
kbReportPrefTopK.value = String(rpK);
kbReplyRuleTopK.value = String(ryK);
kbInjectThreshold.value = String(t);
kbInjectTopK.value = String(k);
@@ -424,7 +419,6 @@ async function saveKbInject() {
kbCheckpointThreshold.value = String(cpT);
kbToolRuleThreshold.value = String(trT);
kbPromptRuleThreshold.value = String(prT);
kbReportPrefThreshold.value = String(rpT);
kbReplyRuleThreshold.value = String(ryT);
savingKbInject.value = true;
kbInjectSaved.value = false;
@@ -453,12 +447,6 @@ async function saveKbInject() {
// queries are different shapes. Moving one must not move the others.
kb_toolrule_threshold: String(trT),
kb_promptrule_threshold: String(prT),
// A SIXTH, and the one that most needed its own key: this arm's query
// is a fixed string, so its score is a constant for a given corpus.
// While it borrowed the prompt bar, tuning prose silently retuned it —
// and a constant that lands under the bar is a dead arm, not a quiet
// one (#3860).
kb_reportpref_threshold: String(rpT),
// The reply backstop's own bar: it HOLDS a reply, so like the
// checkpoint it is never derived from a hint bar.
kb_replyrule_threshold: String(ryT),
@@ -470,7 +458,6 @@ async function saveKbInject() {
kb_rulehint_top_k: String(rhK),
kb_toolrule_top_k: String(trK),
kb_promptrule_top_k: String(prK),
kb_reportpref_top_k: String(rpK),
kb_replyrule_top_k: String(ryK),
kb_duplicate_threshold_snippet: String(dupSnip),
kb_duplicate_threshold_note: String(dupNote),
@@ -941,9 +928,6 @@ onMounted(async () => {
if (allSettings.kb_promptrule_threshold !== undefined) {
kbPromptRuleThreshold.value = allSettings.kb_promptrule_threshold;
}
if (allSettings.kb_reportpref_threshold !== undefined) {
kbReportPrefThreshold.value = allSettings.kb_reportpref_threshold;
}
if (allSettings.kb_replyrule_threshold !== undefined) {
kbReplyRuleThreshold.value = allSettings.kb_replyrule_threshold;
}
@@ -967,9 +951,6 @@ onMounted(async () => {
if (allSettings.kb_promptrule_top_k !== undefined) {
kbPromptRuleTopK.value = allSettings.kb_promptrule_top_k;
}
if (allSettings.kb_reportpref_top_k !== undefined) {
kbReportPrefTopK.value = allSettings.kb_reportpref_top_k;
}
if (allSettings.kb_replyrule_top_k !== undefined) {
kbReplyRuleTopK.value = allSettings.kb_replyrule_top_k;
}
@@ -1974,42 +1955,6 @@ async function deleteUser(userId: number) {
/>
<p class="field-hint">How many rules or preferences one message may be shown (1–10).</p>
</div>
<div class="field">
<label for="kb-reportpref-threshold">Completion-report confidence threshold (0–1)</label>
<input
id="kb-reportpref-threshold"
v-model="kbReportPrefThreshold"
type="number"
min="0"
max="1"
step="0.01"
class="fs-input input"
style="max-width: 8rem"
/>
<p class="field-hint">
The bar for a <em>preference about how a completion report should be
written</em>, looked up when a task closes. Unlike every other bar
here, the question this arm asks never changes — so its score is
fixed by your preferences alone, and it will either always find one
or never find one, and no run of calls will reveal a dead one on its
own. That is why this arm is worth looking up in the panel below when
a report preference never seems to arrive.
</p>
</div>
<div class="field">
<label for="kb-reportpref-topk">Max report preferences</label>
<input
id="kb-reportpref-topk"
v-model="kbReportPrefTopK"
type="number"
min="1"
max="10"
step="1"
class="fs-input input"
style="max-width: 8rem"
/>
<p class="field-hint">How many preferences a finished task may be shown (1–10).</p>
</div>
<div class="field">
<label for="kb-replyrule-threshold">Reply check threshold (0–1)</label>
<input