feat(500): retire the fixed-question preference arm - completion-report preferences ride the moments
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Python tests (push) Successful in 1m54s
CI & Build / Build & push image (push) Successful in 31s

report_preference searched one constant string when a task closed and handed
matches back as update_task's reply_preferences. Milestone 500 step 3 delivers
the reply shapes and the preferences mounted beside them on the moments, so
the arm goes, whole (rule 22):

- services/reply_preferences.py, the REPORT_PREFERENCE RuleArm, its re-scorer
  and corpus entries, and the REPORTPREF constants
- update_task's reply_preferences block and its cue; the docstring now points
  at moment_rules
- the fixed_query field and the fixed_query_never_clears warning: this arm was
  the only one that set it, so the concept and the cannot_decline exemption
  go with it
- the two Settings fields for its floor and budget
- migration 0121 deletes its rows (logs, judgments, rule-usage events, tuning
  history, settings keys), operator-approved 2026-10-09. Without it the
  readout would call the source unregistered and its surfacings would count
  as ambient. The unindexed rule_usage delete is bounded by created_at.

#5496 (step 4 of milestone 500).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 15:51:04 -04:00
co-authored by Claude Opus 5.5
parent cee1eb1328
commit f68b73f922
19 changed files with 94 additions and 619 deletions
-77
View File
@@ -471,83 +471,6 @@ def test_a_quiet_arm_is_not_suspended_either_way() -> None:
assert "floor_moved_mid_window" not in codes(ws)
# ── a fixed-query arm: decline rate is arithmetic, not evidence (#4232) ─────
#
# `report_preference` searches one constant string (COMPLETION_QUERY), so it
# scores against one number on every call. Its decline rate is therefore 0% or
# 100% and never in between, and which one depends only on where the bar sits
# relative to that constant.
#
# So the two warnings swap roles for these arms. "Never declined" stops being
# evidence about the floor — `cannot_decline`'s own remedy, "check that it
# applies its floor", is unanswerable from it. "Always declined" starts being
# evidence, because for a constant score it means the bar is above it and no
# further traffic will ever say otherwise.
def test_cannot_decline_is_silent_on_a_fixed_query_arm():
"""The live readout fired this on `report_preference` at 45 calls, 0
empty, with p10 = p50 = p90 = min = max = 0.791 — five identical
percentiles, which is one record at one score rather than a ranking."""
ws = warn({"report_preference": src(calls=45, zero_result_calls=0, p10=0.791)})
assert "cannot_decline" not in codes(ws, "report_preference")
def test_cannot_decline_still_fires_where_the_rate_means_something():
"""The falsifier for the case above (rule 167). If this passes only
because the check was disabled rather than narrowed, this fails."""
ws = warn({"auto_inject": src(calls=N, zero_result_calls=0)})
assert "cannot_decline" in codes(ws, "auto_inject")
def test_a_fixed_query_arm_that_never_clears_its_bar_is_named():
"""69 consecutive declines at 0.0006 under the bar is a state this arm has
actually been in. Nothing else in the readout would have said so: it looks
exactly like an arm with nothing to report."""
ws = warn({"report_preference": src(calls=45, zero_result_calls=45)})
assert "fixed_query_never_clears" in codes(ws, "report_preference")
detail = next(w["detail"] for w in ws if w["code"] == "fixed_query_never_clears")
assert "near_miss_samples" in detail, (
"the last time this fired, every percentile said lower the floor and "
"the refused record showed the refusal was right — so the warning has "
"to send the reader to the record, not to the dial"
)
def test_an_ordinary_arm_returning_nothing_all_window_is_not_dead():
"""For an arm whose score can vary, an empty window means nothing matched,
which is an answer rather than a fault."""
ws = warn({"auto_inject": src(calls=N, zero_result_calls=N)})
assert "fixed_query_never_clears" not in codes(ws, "auto_inject")
def test_a_fixed_query_arm_that_sometimes_clears_is_not_dead():
"""Only ALL-empty says the bar is above the constant. Anything in between
means the score is not actually constant, and the premise is wrong."""
ws = warn({"report_preference": src(calls=45, zero_result_calls=44)})
assert "fixed_query_never_clears" not in codes(ws, "report_preference")
def test_the_dead_arm_warning_still_needs_volume():
ws = warn({"report_preference": src(calls=N - 1, zero_result_calls=N - 1)})
assert "fixed_query_never_clears" not in codes(ws)
def test_the_registry_declares_which_arms_ask_a_fixed_question():
"""Asserted on structure (rule 167), and able to fail: if `fixed_query`
is dropped or defaults to True, one of these two halves breaks."""
from scribe.services.retrieval_registry import POINTS
assert POINTS["report_preference"].fixed_query is True, (
"services/reply_preferences.py::COMPLETION_QUERY is a module constant"
)
# An arm whose query is built from the prompt, the file or the command is
# not fixed, and marking one would silence a warning that works there.
for varying in ("auto_inject", "write_path", "pre_tool_rule", "prompt_rule"):
assert POINTS[varying].fixed_query is False, varying
# ── floor_history_unknown: the window reaches back past the ledger ─────────
#
# THE SAME SUSPENSION, FOR THE CASE THE ONE ABOVE CANNOT SEE.