fix(telemetry): a warning fired on an arm whose decline rate is arithmetic, not evidence (#4232)

Found by reading a live `retrieval_telemetry` readout after milestone 419
deployed, not by inspection. The readout said:

    cannot_decline / report_preference — "45 calls, 0 of them returned
    nothing. An arm that fires unasked has to be able to say nothing; this one
    never has. Check that it applies its floor at all."

And printed, beside it, that arm's band: p10 = p50 = p90 = min = max = 0.791.

FIVE IDENTICAL PERCENTILES IS THE TELL. That is not a ranking, it is one
record at one score on every call — because `report_preference` searches a
fixed string (`reply_preferences.COMPLETION_QUERY`, a module constant, and
deliberately so).

For a fixed query against a stable corpus the top score is a CONSTANT, so the
arm's decline rate is 0% or 100% and never in between; which of the two it is
depends only on where the bar sits relative to that one number. "Never
returned nothing" is therefore arithmetic, not evidence, and the warning's own
remedy — check whether it applies a floor — cannot be answered from it.

The arm already knew this about itself; the warning did not:

    "a fixed query makes this arm's score a constant and a floor a hair above
     it produces a dead arm no amount of traffic will ever reveal"
        — services/reply_preferences.py

THIS CLASS OF BUG ALREADY HAS A GUARD, which is the argument for the shape of
the fix. `Point.logs_unconditionally` exists because of #3497: both rule arms
once logged only their hits, so their zero count was structurally 0 and this
same warning would have fired on a LOGGING property while sending the reader
to move a threshold that was never involved. This is that one step over — a
QUERY-SHAPE property — and gets the same treatment: a declared field on
`Point`, and exclusion rather than trust.

AND THE WARNING THAT WOULD BE INFORMATIVE HERE DID NOT EXIST. For a fixed-query
arm the dangerous state is the mirror image: every call empty, meaning the bar
is above the constant and no further traffic will ever move it. The arm is off
rather than quiet, and nothing in the readout said so — `expects_traffic`
covers an arm with NO calls, not one with calls and a 100% decline rate. That
state is real and reached: `report_preference` once logged 69 consecutive
declines at 0.0006 under its bar.

So `fixed_query_never_clears` sends the reader to `near_miss_samples` and not
to the dial — because that incident is also the one where the statistic and
the correct action pointed opposite ways. Every percentile said lower the
floor; opening the refused record showed it was rule 77 arriving as a false
positive, and lowering it would have delivered that rule on every completion
report ever written.

Guards in tests/test_retrieval_warnings.py, including the falsifier that
matters most here: `cannot_decline` must still fire on an arm whose query
varies, or this change is a disabled check wearing a narrowed one's clothes.
Both boundaries tested from both sides, per that module's own standard.

The new code is documented in the `retrieval_telemetry` tool docstring beside
the others (rule 33) — an undocumented code in a readout is a reader meeting a
verdict with no way to disagree with it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy

src/scribe/mcp/tools/search.py

@@ -421,7 +421,19 @@ It is an UPPER BOUND per surface: a pull records the door it came
       Only ever raised for unbidden arms known to log unconditionally: a
       search returning a list every time is doing its job, and an arm whose
       zeros were never written would flag a LOGGING bug while pointing you at
-      a threshold, which is #3497 exactly.
+      a threshold, which is #3497 exactly. Nor for an arm whose query never
+      changes — see the next entry.
+    - `fixed_query_never_clears` — an arm that always searches the SAME query
+      returned nothing on every call. Its score is one constant, so this is
+      not a quiet window: the bar sits above that constant and no amount of
+      further traffic will produce a different result. The arm is off rather
+      than silent, and nothing else here would say so. The same property is
+      why `cannot_decline` is not raised for these arms: with a constant
+      score the decline rate is 0% or 100% by construction, so "never
+      declined" is arithmetic and not evidence about the floor. Read the
+      refused record (`near_miss_samples`) BEFORE moving the dial — the last
+      time an arm sat here, every percentile said lower it and the refused
+      record showed the refusal was right.
     - `band_hugs_floor` — the weakest tenth of what an arm returns sits on
       its floor. The bar is doing the selecting and the score is not, so
       moving that floor changes how MUCH you get, not how good it is.

src/scribe/services/retrieval_registry.py

@@ -86,6 +86,29 @@ class Point:
 
     quiet_because: str = ""
 
+    fixed_query: bool = False
+    """Whether this arm always searches the SAME query string.
+
+    THE #3497 GUARD, ONE STEP OVER. `logs_unconditionally` below exists
+    because a warning computed over a LOGGING property read as a ranking
+    problem. This field exists because a warning computed over a QUERY-SHAPE
+    property does the same thing.
+
+    An arm with a fixed query scores against one constant. Its decline rate is
+    therefore 0% or 100% and nothing in between — which of the two depends
+    only on whether the bar sits below or above that single number. So "never
+    returned nothing" says nothing at all about whether a floor is applied,
+    and `cannot_decline` — whose whole remedy is "check that it applies its
+    floor" — is uninformative here and skips these arms.
+
+    What IS informative for them is the mirror image, and `reply_preferences`
+    names it in its own docstring: every call returning nothing means the bar
+    sits above the constant, no traffic will ever move it, and the arm is
+    dead. That has happened — 69 consecutive declines at 0.0006 under the bar
+    (see `retrieval_surfaces`) — so it gets its own warning rather than
+    inheriting one written for arms whose score can vary.
+    """
+
     logs_unconditionally: bool = True
     """Whether this arm writes a row even when it returns NOTHING.
 
@@ -114,8 +137,13 @@ POINTS: dict[str, Point] = dict([
        "the one line reserved for a preference at the prompt boundary"),
     _p("reuse_slot", UNBIDDEN, "the one line reserved for a reusable snippet"),
     _p("lesson_slot", UNBIDDEN, "the one line reserved for a lesson"),
+    # `fixed_query`: COMPLETION_QUERY is a module constant in
+    # services/reply_preferences.py, so this arm's top score is the same number
+    # on every call — measured at 0.791 across 45 consecutive calls, with p10,
+    # p50, p90, min and max all identical. Five equal percentiles is the tell.
     _p("report_preference", UNBIDDEN,
-       "the fixed question asked when a task finishes: how should this report read"),
+       "the fixed question asked when a task finishes: how should this report read",
+       fixed_query=True),
 
     # ── Asked: a caller wanted a ranked list ─────────────────────────────
     _p("mcp_search", ASKED, "an agent called search"),

src/scribe/services/retrieval_telemetry.py

@@ -556,12 +556,22 @@ def _compute_warnings(sources: dict, usage: dict, rule_usage: dict,
         # arms once recorded only their hits, so their decline count was
         # structurally zero and this warning would have fired on a LOGGING
         # defect while pointing the reader at the threshold.
+        #
+        # FOUR NOW. A FIXED-QUERY arm is exempt for the same reason one step
+        # over (#4232): it scores against one constant, so its decline rate is
+        # 0% or 100% and never in between, and which one it is depends only on
+        # where the bar sits relative to that single number. "Never returned
+        # nothing" is then not evidence about the floor — it is arithmetic —
+        # and this warning's own remedy, "check that it applies its floor",
+        # cannot be answered from it. Those arms get `fixed_query_never_clears`
+        # below, which asks the question that IS answerable for them.
         if (
             calls >= min_calls
             and (b.get("zero_result_calls") or 0) == 0
             and point is not None
             and point.kind == UNBIDDEN
             and point.logs_unconditionally
+            and not point.fixed_query
         ):
             out.append(_warn(
                 "cannot_decline",
@@ -572,6 +582,46 @@ def _compute_warnings(sources: dict, usage: dict, rule_usage: dict,
                 source=name, calls=calls, zero_result_calls=0,
             ))
 
+        # ── A fixed-query arm that never clears its bar ──────────────────
+        #
+        # The mirror image of `cannot_decline`, and the state that actually
+        # threatens these arms. `reply_preferences` names it in its own
+        # docstring: "a fixed query makes this arm's score a constant and a
+        # floor a hair above it produces a dead arm no amount of traffic will
+        # ever reveal."
+        #
+        # For an arm whose score can vary, a window of all-empty calls is
+        # ordinary — it means nothing matched, which is an answer. For one
+        # whose score is a constant it means the bar is above that constant,
+        # and no volume of further calls will ever produce a different result.
+        # The arm is not quiet; it is switched off, and nothing else in this
+        # readout would say so.
+        #
+        # It has happened: `report_preference` logged 69 consecutive declines
+        # at 0.0006 under the bar. Note what that incident also proves — the
+        # fix is NOT automatically to lower the floor. Reading the refused
+        # record showed the refusal was correct, so this warning sends the
+        # reader to `near_miss_samples` rather than to the dial.
+        if (
+            calls >= min_calls
+            and point is not None
+            and point.fixed_query
+            and point.logs_unconditionally
+            and (b.get("zero_result_calls") or 0) == calls
+        ):
+            out.append(_warn(
+                "fixed_query_never_clears",
+                f"{calls} calls, every one of them empty — and this arm always "
+                f"searches the same query, so its score is a constant. That "
+                f"means the bar sits above it and no amount of further traffic "
+                f"will change the result: the arm is off, not quiet. Read the "
+                f"record it refused (`near_miss_samples`) before touching the "
+                f"floor — the last time this arm sat here, every percentile "
+                f"said lower it and the refused record showed the refusal was "
+                f"right.",
+                source=name, calls=calls, zero_result_calls=calls,
+            ))
+
         # ── Band hugs its floor ──────────────────────────────────────────
         #
         # Read on p10, the WEAKEST tenth of what the arm returned. If even

tests/test_retrieval_warnings.py

@@ -446,3 +446,80 @@ def test_a_quiet_arm_is_not_suspended_either_way() -> None:
     ws = warn({"auto_inject": src(calls=1, zero_result_calls=0, p10=0.705)},
               floors={"auto_inject": 0.70}, floor_moves={"auto_inject": MOVED})
     assert "floor_moved_mid_window" not in codes(ws)
+
+
+# ── a fixed-query arm: decline rate is arithmetic, not evidence (#4232) ─────
+#
+# `report_preference` searches one constant string (COMPLETION_QUERY), so it
+# scores against one number on every call. Its decline rate is therefore 0% or
+# 100% and never in between, and which one depends only on where the bar sits
+# relative to that constant.
+#
+# So the two warnings swap roles for these arms. "Never declined" stops being
+# evidence about the floor — `cannot_decline`'s own remedy, "check that it
+# applies its floor", is unanswerable from it. "Always declined" starts being
+# evidence, because for a constant score it means the bar is above it and no
+# further traffic will ever say otherwise.
+
+
+def test_cannot_decline_is_silent_on_a_fixed_query_arm():
+    """The live readout fired this on `report_preference` at 45 calls, 0
+    empty, with p10 = p50 = p90 = min = max = 0.791 — five identical
+    percentiles, which is one record at one score rather than a ranking."""
+    ws = warn({"report_preference": src(calls=45, zero_result_calls=0, p10=0.791)})
+    assert "cannot_decline" not in codes(ws, "report_preference")
+
+
+def test_cannot_decline_still_fires_where_the_rate_means_something():
+    """The falsifier for the case above (rule 167). If this passes only
+    because the check was disabled rather than narrowed, this fails."""
+    ws = warn({"auto_inject": src(calls=N, zero_result_calls=0)})
+    assert "cannot_decline" in codes(ws, "auto_inject")
+
+
+def test_a_fixed_query_arm_that_never_clears_its_bar_is_named():
+    """69 consecutive declines at 0.0006 under the bar is a state this arm has
+    actually been in. Nothing else in the readout would have said so: it looks
+    exactly like an arm with nothing to report."""
+    ws = warn({"report_preference": src(calls=45, zero_result_calls=45)})
+    assert "fixed_query_never_clears" in codes(ws, "report_preference")
+
+    detail = next(w["detail"] for w in ws if w["code"] == "fixed_query_never_clears")
+    assert "near_miss_samples" in detail, (
+        "the last time this fired, every percentile said lower the floor and "
+        "the refused record showed the refusal was right — so the warning has "
+        "to send the reader to the record, not to the dial"
+    )
+
+
+def test_an_ordinary_arm_returning_nothing_all_window_is_not_dead():
+    """For an arm whose score can vary, an empty window means nothing matched,
+    which is an answer rather than a fault."""
+    ws = warn({"auto_inject": src(calls=N, zero_result_calls=N)})
+    assert "fixed_query_never_clears" not in codes(ws, "auto_inject")
+
+
+def test_a_fixed_query_arm_that_sometimes_clears_is_not_dead():
+    """Only ALL-empty says the bar is above the constant. Anything in between
+    means the score is not actually constant, and the premise is wrong."""
+    ws = warn({"report_preference": src(calls=45, zero_result_calls=44)})
+    assert "fixed_query_never_clears" not in codes(ws, "report_preference")
+
+
+def test_the_dead_arm_warning_still_needs_volume():
+    ws = warn({"report_preference": src(calls=N - 1, zero_result_calls=N - 1)})
+    assert "fixed_query_never_clears" not in codes(ws)
+
+
+def test_the_registry_declares_which_arms_ask_a_fixed_question():
+    """Asserted on structure (rule 167), and able to fail: if `fixed_query`
+    is dropped or defaults to True, one of these two halves breaks."""
+    from scribe.services.retrieval_registry import POINTS
+
+    assert POINTS["report_preference"].fixed_query is True, (
+        "services/reply_preferences.py::COMPLETION_QUERY is a module constant"
+    )
+    # An arm whose query is built from the prompt, the file or the command is
+    # not fixed, and marking one would silence a warning that works there.
+    for varying in ("auto_inject", "write_path", "pre_tool_rule", "prompt_rule"):
+        assert POINTS[varying].fixed_query is False, varying