fix(retrieval): the rules never-opened warning stops sending the reader to a review tool that refuses rules (#4798)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m7s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 26s
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m7s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 26s
#4798 "The rules corpus's surfaced_never_pulled warning sends the reader to menus_to_review, which can only review auto_inject". #4772 gave the warning one remedy for both corpora: "judge a sample with menus_to_review". That is right for notes, where a line carries its passage. For rules it is a dead end: retrieval_review.REVIEWABLE holds only auto_inject, so the tool refuses every rule arm. - The reading is now per corpus (_NEVER_PULLED_READING). For rules, the text says: - the count includes rules that only arrived in a listing; - a rule can rightly be set aside on its trigger alone; - no judged sample exists for the rule arms; - a rule set aside again and again is a trigger to fix (update_rule when_to_apply, then what_might_apply), not a floor. - The tool docstring says the same. - The guard is tied to REVIEWABLE, so it can fail in both directions: if the rules text names menus_to_review while no rule arm is reviewable, or if a rule arm becomes reviewable and the text still says there is no sample. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -182,6 +182,29 @@ def test_records_shown_and_never_opened_are_reported_per_corpus() -> None:
|
||||
assert found["rules"]["never_pulled"] == 58
|
||||
|
||||
|
||||
def test_each_corpus_is_sent_only_to_a_tool_that_takes_it() -> None:
|
||||
"""#4798: one remedy for both corpora sent the rules reader to
|
||||
`menus_to_review`, which refuses every source but the notes menu. Tied to
|
||||
`REVIEWABLE` itself, so it goes red in either direction: the rules text
|
||||
naming the tool while no rule arm is reviewable, or a rule arm becoming
|
||||
reviewable while the text still says there is no sample."""
|
||||
from scribe.services.retrieval_review import REVIEWABLE
|
||||
|
||||
ws = warn(
|
||||
{},
|
||||
usage={"distinct_notes_surfaced": 9, "distinct_notes_pulled": 1},
|
||||
rule_usage={"distinct_rules_surfaced": 9, "distinct_rules_pulled": 1},
|
||||
)
|
||||
detail = {w["numbers"]["corpus"]: w["detail"]
|
||||
for w in ws if w["code"] == "surfaced_never_pulled"}
|
||||
assert set(detail) == {"notes", "rules"}
|
||||
assert "auto_inject" in REVIEWABLE
|
||||
assert "menus_to_review" in detail["notes"]
|
||||
rule_arms = {"prompt_rule", "pre_tool_rule", "write_path_rule"}
|
||||
assert ("menus_to_review" in detail["rules"]) == bool(rule_arms & set(REVIEWABLE))
|
||||
assert "when_to_apply" in detail["rules"], "the rules text names no next step"
|
||||
|
||||
|
||||
def test_everything_opened_reports_nothing() -> None:
|
||||
ws = warn({}, usage={"distinct_notes_surfaced": 5, "distinct_notes_pulled": 5})
|
||||
assert "surfaced_never_pulled" not in codes(ws)
|
||||
|
||||
Reference in New Issue
Block a user