feat(retrieval): a review pass judges whether injected lines related — menus_to_review, judge_menu and a judged readout (#4772)

An open rate cannot say whether a menu line related: every line carries its matched passage (#4364), so "not opened" covers unrelated, enough as shown, and already in context. #4772 "Injected notes are never judged". - retrieval_judgments (0115): a reviewer verdict per line of a logged call, on_point / adjacent / unrelated, with its reason, rank, budget side and whether the agent opened it within the hour. - menus_to_review re-runs a random sample of unjudged auto_inject calls with the arm's own parameters, past its budget, passage on every line. judge_menu records verdicts, re-deriving rank from a fresh re-run. - retrieval_telemetry gains a judged block (by rank, within/beyond budget, on_point_unopened). surfaced_never_pulled stops blaming titles. - missed-retrieval guidance names the review before a budget move. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 15:34:14 -04:00
parent 0a1bb68808
commit 6598c7fa85
13 changed files with 879 additions and 11 deletions
@@ -61,6 +61,7 @@ from scribe.models.retrieval_tuning import RetrievalTuningEvent  # noqa: E402, F
 from scribe.models.note_usage import NoteUsageEvent  # noqa: E402, F401
 from scribe.models.rule_usage import RuleUsageEvent  # noqa: E402, F401
 from scribe.models.system_usage import SystemUsageEvent  # noqa: E402, F401
+from scribe.models.retrieval_judgment import RetrievalJudgment  # noqa: E402, F401
 from scribe.models.project import Project  # noqa: E402, F401
 from scribe.models.milestone import Milestone  # noqa: E402, F401
 from scribe.models.task_log import TaskLog  # noqa: E402, F401