"""Judging what a retrieval arm offered. `retrieval_telemetry` says what the reader did with each line — opened it or not. These two say whether the line RELATED, which is a reading only someone looking at the query beside the line can do. """ from grimoire.mcp._context import current_user_id from grimoire.services import retrieval_review as review_svc async def menus_to_review( source: str = "auto_inject", n: int = 5, days: int = 14, depth: int = 6, ) -> dict: """A sample of logged injection menus, re-run so you can judge each line. Reach for this before moving a budget or a floor, and whenever an open rate is about to be read as a verdict on relevance. It is not one: every menu line carries the passage that matched, so a record left unopened may have been unrelated, enough as shown, or already in context, and the usage counters cannot tell those apart. A judged sample can. Each menu is one logged call: the query the arm searched (enriched, as it was asked), its `budget`, and `lines` — the query re-run with the arm's own parameters to `depth`, each line with `rank`, `score`, `kind`, `name`, the `passage` that matched, `within_budget` (was it inside the cut) and `opened_after` (did your agent open it within an hour). Lines past the budget are there on purpose: they are what a larger budget would add. `missing` names ids the call logged that the re-run no longer finds — the corpus has moved since. Judge what is in front of you. The re-run is against today's corpus, so it guards what has been written since. Records created after the call are left out and named in `postdated`: the call could not have offered them, and the session that made it often wrote them, quoting it. A line marked `changed_since_call` was written to afterwards; if its passage is plainly the later text, leave it unjudged. `logged` says the line was shown fresh at the time. JUDGE FROM THE LINE; DO NOT OPEN THE RECORD. Opening counts as a pull and would credit the arm with an open it never earned. Then record verdicts with `judge_menu`. `how_to_judge` in the response repeats the vocabulary. Args: source: the arm to review. Only arms whose search can be re-run exactly are offered: `auto_inject`, and the rule arms `prompt_rule`, `pre_tool_rule` and `write_path_rule`. A rule line's passage is the trigger the arm matched, and for the command and write arms `within_budget` also needs the line to sit inside the band of the top hit, as it did when shown. n: how many calls to sample, 1–20. Random among the calls you have not judged, so a review does not describe one afternoon. days: how far back to sample from. depth: how many candidates to re-run per call, 1–10. Past the budget, so the cut itself can be judged. """ return await review_svc.menus_to_review( current_user_id(), source=source, n=n, days=days, depth=depth, ) async def judge_menu(log_id: int, verdicts: list[dict]) -> dict: """Record your verdict on lines of one menu from `menus_to_review`. `verdicts` is a list of `{record_id, verdict, reason}`: "on_point" — someone in the query's situation should read this. "adjacent" — the same area, but it would not change what they do. "unrelated" — close in words only. `reason` is REQUIRED for every verdict: what in the query and the passage decided it. A verdict nobody can re-read is a threshold, and this exists to replace one. Rank, score and which side of the budget the line sat are taken from a fresh re-run, not from you. Judging a line again replaces your earlier verdict on it. The verdicts are read back as `retrieval_telemetry`'s `judged` block. """ return await review_svc.judge_menu( current_user_id(), log_id=log_id, verdicts=verdicts, ) def register(mcp) -> None: mcp.tool(name="menus_to_review")(menus_to_review) mcp.tool(name="judge_menu")(judge_menu)