feat(500): retire the fixed-question preference arm - completion-report preferences ride the moments
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Python tests (push) Successful in 1m54s
CI & Build / Build & push image (push) Successful in 31s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Python tests (push) Successful in 1m54s
CI & Build / Build & push image (push) Successful in 31s
report_preference searched one constant string when a task closed and handed matches back as update_task's reply_preferences. Milestone 500 step 3 delivers the reply shapes and the preferences mounted beside them on the moments, so the arm goes, whole (rule 22): - services/reply_preferences.py, the REPORT_PREFERENCE RuleArm, its re-scorer and corpus entries, and the REPORTPREF constants - update_task's reply_preferences block and its cue; the docstring now points at moment_rules - the fixed_query field and the fixed_query_never_clears warning: this arm was the only one that set it, so the concept and the cannot_decline exemption go with it - the two Settings fields for its floor and budget - migration 0121 deletes its rows (logs, judgments, rule-usage events, tuning history, settings keys), operator-approved 2026-10-09. Without it the readout would call the source unregistered and its surfacings would count as ambient. The unindexed rule_usage delete is bounded by created_at. #5496 (step 4 of milestone 500). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -168,7 +168,6 @@ const kbToolRuleThreshold = ref("0.68");
|
||||
// And the prompt boundary is a third query shape again — the operator's own
|
||||
// prose rather than anything a tool produced (#3852).
|
||||
const kbPromptRuleThreshold = ref("0.72");
|
||||
const kbReportPrefThreshold = ref("0.72");
|
||||
// The reply arm (milestone 458): its bar is the bar at which a finished reply
|
||||
// is HELD for one read, so it sits with the checkpoint, not with the hints.
|
||||
const kbReplyRuleThreshold = ref("0.8");
|
||||
@@ -181,7 +180,6 @@ const kbWritePathTopK = ref("3");
|
||||
const kbRuleHintTopK = ref("5");
|
||||
const kbToolRuleTopK = ref("5");
|
||||
const kbPromptRuleTopK = ref("3");
|
||||
const kbReportPrefTopK = ref("3");
|
||||
const kbReplyRuleTopK = ref("1");
|
||||
// What has been changed about retrieval, newest first — the review surface for
|
||||
// changes the model made on the operator's behalf (#4102).
|
||||
@@ -387,7 +385,6 @@ async function saveKbInject() {
|
||||
// this is the only number that can STOP a call, so a fallback of 0 would
|
||||
// hold the first command of every session behind whatever ranked first.
|
||||
const cpT = asBar(kbCheckpointThreshold.value, 0.8);
|
||||
const rpT = asBar(kbReportPrefThreshold.value, 0.72);
|
||||
// A stop bar like the checkpoint's: a fallback of 0 would hold every reply.
|
||||
const ryT = asBar(kbReplyRuleThreshold.value, 0.8);
|
||||
// The budgets, clamped the way the server clamps them: a whole number in
|
||||
@@ -400,13 +397,11 @@ async function saveKbInject() {
|
||||
const rhK = asK(kbRuleHintTopK.value, 5);
|
||||
const trK = asK(kbToolRuleTopK.value, 5);
|
||||
const prK = asK(kbPromptRuleTopK.value, 3);
|
||||
const rpK = asK(kbReportPrefTopK.value, 3);
|
||||
const ryK = asK(kbReplyRuleTopK.value, 1);
|
||||
kbWritePathTopK.value = String(wpK);
|
||||
kbRuleHintTopK.value = String(rhK);
|
||||
kbToolRuleTopK.value = String(trK);
|
||||
kbPromptRuleTopK.value = String(prK);
|
||||
kbReportPrefTopK.value = String(rpK);
|
||||
kbReplyRuleTopK.value = String(ryK);
|
||||
kbInjectThreshold.value = String(t);
|
||||
kbInjectTopK.value = String(k);
|
||||
@@ -424,7 +419,6 @@ async function saveKbInject() {
|
||||
kbCheckpointThreshold.value = String(cpT);
|
||||
kbToolRuleThreshold.value = String(trT);
|
||||
kbPromptRuleThreshold.value = String(prT);
|
||||
kbReportPrefThreshold.value = String(rpT);
|
||||
kbReplyRuleThreshold.value = String(ryT);
|
||||
savingKbInject.value = true;
|
||||
kbInjectSaved.value = false;
|
||||
@@ -453,12 +447,6 @@ async function saveKbInject() {
|
||||
// queries are different shapes. Moving one must not move the others.
|
||||
kb_toolrule_threshold: String(trT),
|
||||
kb_promptrule_threshold: String(prT),
|
||||
// A SIXTH, and the one that most needed its own key: this arm's query
|
||||
// is a fixed string, so its score is a constant for a given corpus.
|
||||
// While it borrowed the prompt bar, tuning prose silently retuned it —
|
||||
// and a constant that lands under the bar is a dead arm, not a quiet
|
||||
// one (#3860).
|
||||
kb_reportpref_threshold: String(rpT),
|
||||
// The reply backstop's own bar: it HOLDS a reply, so like the
|
||||
// checkpoint it is never derived from a hint bar.
|
||||
kb_replyrule_threshold: String(ryT),
|
||||
@@ -470,7 +458,6 @@ async function saveKbInject() {
|
||||
kb_rulehint_top_k: String(rhK),
|
||||
kb_toolrule_top_k: String(trK),
|
||||
kb_promptrule_top_k: String(prK),
|
||||
kb_reportpref_top_k: String(rpK),
|
||||
kb_replyrule_top_k: String(ryK),
|
||||
kb_duplicate_threshold_snippet: String(dupSnip),
|
||||
kb_duplicate_threshold_note: String(dupNote),
|
||||
@@ -941,9 +928,6 @@ onMounted(async () => {
|
||||
if (allSettings.kb_promptrule_threshold !== undefined) {
|
||||
kbPromptRuleThreshold.value = allSettings.kb_promptrule_threshold;
|
||||
}
|
||||
if (allSettings.kb_reportpref_threshold !== undefined) {
|
||||
kbReportPrefThreshold.value = allSettings.kb_reportpref_threshold;
|
||||
}
|
||||
if (allSettings.kb_replyrule_threshold !== undefined) {
|
||||
kbReplyRuleThreshold.value = allSettings.kb_replyrule_threshold;
|
||||
}
|
||||
@@ -967,9 +951,6 @@ onMounted(async () => {
|
||||
if (allSettings.kb_promptrule_top_k !== undefined) {
|
||||
kbPromptRuleTopK.value = allSettings.kb_promptrule_top_k;
|
||||
}
|
||||
if (allSettings.kb_reportpref_top_k !== undefined) {
|
||||
kbReportPrefTopK.value = allSettings.kb_reportpref_top_k;
|
||||
}
|
||||
if (allSettings.kb_replyrule_top_k !== undefined) {
|
||||
kbReplyRuleTopK.value = allSettings.kb_replyrule_top_k;
|
||||
}
|
||||
@@ -1974,42 +1955,6 @@ async function deleteUser(userId: number) {
|
||||
/>
|
||||
<p class="field-hint">How many rules or preferences one message may be shown (1–10).</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-reportpref-threshold">Completion-report confidence threshold (0–1)</label>
|
||||
<input
|
||||
id="kb-reportpref-threshold"
|
||||
v-model="kbReportPrefThreshold"
|
||||
type="number"
|
||||
min="0"
|
||||
max="1"
|
||||
step="0.01"
|
||||
class="fs-input input"
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">
|
||||
The bar for a <em>preference about how a completion report should be
|
||||
written</em>, looked up when a task closes. Unlike every other bar
|
||||
here, the question this arm asks never changes — so its score is
|
||||
fixed by your preferences alone, and it will either always find one
|
||||
or never find one, and no run of calls will reveal a dead one on its
|
||||
own. That is why this arm is worth looking up in the panel below when
|
||||
a report preference never seems to arrive.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-reportpref-topk">Max report preferences</label>
|
||||
<input
|
||||
id="kb-reportpref-topk"
|
||||
v-model="kbReportPrefTopK"
|
||||
type="number"
|
||||
min="1"
|
||||
max="10"
|
||||
step="1"
|
||||
class="fs-input input"
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">How many preferences a finished task may be shown (1–10).</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-replyrule-threshold">Reply check threshold (0–1)</label>
|
||||
<input
|
||||
|
||||
Reference in New Issue
Block a user