fix(retrieval): the rules never-opened warning stops sending the reader to a review tool that refuses rules (#4798)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m7s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 26s

#4798 "The rules corpus's surfaced_never_pulled warning sends the reader to
menus_to_review, which can only review auto_inject". #4772 gave the warning
one remedy for both corpora: "judge a sample with menus_to_review". That is
right for notes, where a line carries its passage. For rules it is a dead
end: retrieval_review.REVIEWABLE holds only auto_inject, so the tool
refuses every rule arm.

- The reading is now per corpus (_NEVER_PULLED_READING). For rules, the
  text says:
  - the count includes rules that only arrived in a listing;
  - a rule can rightly be set aside on its trigger alone;
  - no judged sample exists for the rule arms;
  - a rule set aside again and again is a trigger to fix (update_rule
    when_to_apply, then what_might_apply), not a floor.
- The tool docstring says the same.
- The guard is tied to REVIEWABLE, so it can fail in both directions: if
  the rules text names menus_to_review while no rule arm is reviewable, or
  if a rule arm becomes reviewable and the text still says there is no
  sample.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-03 23:02:32 -04:00
co-authored by Claude Opus 5.5
parent f95ffae972
commit 9909cd2450
3 changed files with 54 additions and 5 deletions
+23
View File
@@ -182,6 +182,29 @@ def test_records_shown_and_never_opened_are_reported_per_corpus() -> None:
assert found["rules"]["never_pulled"] == 58
def test_each_corpus_is_sent_only_to_a_tool_that_takes_it() -> None:
"""#4798: one remedy for both corpora sent the rules reader to
`menus_to_review`, which refuses every source but the notes menu. Tied to
`REVIEWABLE` itself, so it goes red in either direction: the rules text
naming the tool while no rule arm is reviewable, or a rule arm becoming
reviewable while the text still says there is no sample."""
from scribe.services.retrieval_review import REVIEWABLE
ws = warn(
{},
usage={"distinct_notes_surfaced": 9, "distinct_notes_pulled": 1},
rule_usage={"distinct_rules_surfaced": 9, "distinct_rules_pulled": 1},
)
detail = {w["numbers"]["corpus"]: w["detail"]
for w in ws if w["code"] == "surfaced_never_pulled"}
assert set(detail) == {"notes", "rules"}
assert "auto_inject" in REVIEWABLE
assert "menus_to_review" in detail["notes"]
rule_arms = {"prompt_rule", "pre_tool_rule", "write_path_rule"}
assert ("menus_to_review" in detail["rules"]) == bool(rule_arms & set(REVIEWABLE))
assert "when_to_apply" in detail["rules"], "the rules text names no next step"
def test_everything_opened_reports_nothing() -> None:
ws = warn({}, usage={"distinct_notes_surfaced": 5, "distinct_notes_pulled": 5})
assert "surfaced_never_pulled" not in codes(ws)