feat(4636): rule pull-through per surface, and per-rule usage with confirmed lessons on list_rules
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m3s
CI & Build / Python tests (push) Successful in 1m56s
CI & Build / Build & push image (push) Successful in 22s

retrieval_telemetry gains rule_usage.by_source, the twin of usage.by_source:
distinct rules each surface showed, how many an agent opened, and a ratio
that is null on a bulk delivery. list_rules(with_usage=True, days=N) adds
each rule's usage, windowed, and how many lessons are confirmed instances
of it, so linked and unlinked rules can be compared through MCP.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-22 19:20:32 -04:00
co-authored by Claude Opus 5.5
parent 71260a28e2
commit d05d4f3632
9 changed files with 287 additions and 9 deletions
+30 -6
View File
@@ -10,13 +10,16 @@ preview-style warning.
"""
from __future__ import annotations
from datetime import datetime, timedelta, timezone
from grimoire.mcp._context import current_user_id
from grimoire.services import dedup as dedup_svc
from grimoire.services import moments as moments_svc
from grimoire.services import rulebooks as rulebooks_svc
from grimoire.services import trash as trash_svc
from grimoire.services.lesson_rules import confirmed_lesson_counts
from grimoire.services.rule_usage import (
record_rule_outcome, record_rule_pulled,
empty_rule_usage, record_rule_outcome, record_rule_pulled, usage_for_rules,
)
from grimoire.services import moment_delivery
@@ -204,6 +207,7 @@ def _rule_summary(r, moments: list[str] | None = None) -> dict:
async def list_rules(
rulebook_id: int = 0, topic_id: int = 0, project_id: int = 0,
with_usage: bool = False, days: int = 0,
) -> dict:
"""List rules — filter by rulebook, topic, and/or project.
@@ -213,9 +217,21 @@ async def list_rules(
project_id: 0 = no filter; positive = that project's OWN rules.
Global rules apply to every project, so they are listed by
rulebook or topic (or unfiltered), not under each project.
with_usage: add each rule's `usage` and `confirmed_lessons`. Off by
default: a listing is for applying rules, and this is for judging
them.
days: with `with_usage`, count only the last N days; 0 = everything
recorded.
rulebook_id and topic_id AND-combine; project_id lists a project's rules
on its own. Ownership-scoped.
`usage` is the rule's own telemetry: `surfaced_count` (a ranked surface
chose it), `ambient_count` (a bulk delivery carried it), `pull_count`,
`applied_count`, `departed_count` and the latest of each.
`confirmed_lessons` counts the lessons judged instances of the rule. Read
together they answer whether rules that situations keep proving are the
ones that surface and get opened.
"""
uid = current_user_id()
rows = await rulebooks_svc.list_rules(
@@ -224,12 +240,20 @@ async def list_rules(
topic_id=topic_id or None,
project_id=project_id or None,
)
ids = [r.id for r in rows]
# One batched read for the page, not one per row.
mounted = await rulebooks_svc.list_rule_moments([r.id for r in rows])
return {
"rules": [_rule_summary(r, mounted.get(r.id)) for r in rows],
"total": len(rows),
}
mounted = await rulebooks_svc.list_rule_moments(ids)
rules = [_rule_summary(r, mounted.get(r.id)) for r in rows]
if with_usage:
since = (
datetime.now(timezone.utc) - timedelta(days=days) if days > 0 else None
)
usage = await usage_for_rules(ids, since=since)
lessons = await confirmed_lesson_counts(uid, ids)
for row in rules:
row["usage"] = usage.get(row["id"], empty_rule_usage())
row["confirmed_lessons"] = lessons.get(row["id"], 0)
return {"rules": rules, "total": len(rows)}
async def get_rule(rule_id: int) -> dict:
+9
View File
@@ -509,6 +509,15 @@ async def retrieval_telemetry(
Read it against `sources["write_path_rule"]`, which declines the large
majority of its calls like any other surface.
`rule_usage.by_source` is the per-surface split, the twin of
`usage.by_source`: for each surface that showed rules, `rules_surfaced`
(distinct rules), `rules_pulled` (how many of those an agent opened in the
window) and `pull_through`, null on an ambient surface. It is the readout
that says which arm — the prompt arm, the tool arm, a lesson pointing at
its rule — earns its lines. The same upper bound as the note split: a rule
two surfaces showed and an agent opened once counts for both.
`by_source_failed: true` means that one query failed.
`system_usage` — the RULINGS arm, from `system_usage_events`: how
often an area's rulings were shown because a command (`rulings_pre_tool`)
or an edit (`rulings_write_path`) touched its files, how often a System