feat(rules): the staleness sweep — which standing rules assert a fact nobody has confirmed (#3097, milestone 312 step 3)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m8s
CI & Build / Build & push image (push) Successful in 35s

The query the last two steps were storage for. `rules_due_for_verification`
returns every rule carrying a `verify_with`, ordered by `verified_at` ASC
NULLS FIRST, each row carrying the check IN FULL — the opposite call from
rule_brief, because the reader is about to go and run it.

NULLS FIRST is the ordering this turns on. Postgres sorts NULLs last on an
ASC ordering, which would put the rules nobody has ever confirmed BEHIND
every rule someone once looked at. Exactly backwards: a claim with no
evidence at all outranks an old one.

Rules with no check never appear, and that is the property that keeps the
list worth reading. Most rules are decisions — no truth value, nothing to go
and check. If they appeared here the sweep would be the rulebook.

`mark_rule_verified(rule_id, still_true)` closes the loop, asymmetrically:
passing writes a stamp, FAILING WRITES NOTHING. There is no "verified false"
state because a rule whose check failed is not in a special condition, it is
wrong — and recording the failure as a flag would let it sit there being
false with the sweep satisfied that someone had looked. So it stays at the
top until someone corrects or retires it, and the response says so.

An unrecognised `tier` filter raises rather than falling back. _valid_tier's
silent always_on default is right for a WRITE — a typo should leave a rule
binding — and wrong for a FILTER, where the same fallback quietly answers a
different question and returns a short list that reads as good news.

Deliberately NOT filterable by project: a project reaches rules through
project scope, subscriptions, always-on rulebooks and exclusions, and a
filter missing one of those paths would UNDER-report — the exact failure
this surface exists to prevent. Said so in the docstring rather than
shipping a half-correct filter.

Ownership-scoped like every other rule read (owned rulebook, or owned
project), in ONE statement with an OR across the XOR rather than two queries
merged in Python, so the ordering is the database's and cannot disagree with
itself. Note that rules have no sharing ACL in this schema — no rule_shares,
no rulebook_shares — so there is no wider set for access.py to consult here.

Also fixes a test title that had been lying for ten tools: "all sixteen
tools" asserted 26. The number now lives only in the assertion.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-27 10:49:47 -04:00
co-authored by Claude Opus 5
parent 469b43f222
commit 410d616c22
7 changed files with 560 additions and 4 deletions
+92
View File
@@ -691,6 +691,97 @@ async def unrelate_rules(relation_id: int) -> dict:
raise ValueError(f"relation {relation_id} not found")
return {"deleted": relation_id}
# ── The staleness sweep (milestone 312) ────────────────────────────────
async def rules_due_for_verification(
older_than_days: int = 0, tier: str = "", never_only: bool = False,
) -> dict:
"""Which standing rules assert a FACT that nobody has confirmed lately.
A rulebook holds two kinds of thing. Most rules are DECISIONS — how the
operator wants to work. They have no truth value and cannot rot. A few
assert a fact about someone else's software: what a CI runner does, which
tools exist, what a setting is currently set to. Those go false silently,
with nobody present, and they keep being handed to every session as
binding instructions long after they stopped being true.
This lists the second kind, oldest verification first, never-checked at
the top. Each row carries the rule's `verify_with` in full — you are
about to go and run it — plus `expires_when`, and `days_since_verified`.
Reach for it when you are curating the rulebook, when a rule's advice
just contradicted what you observed, or periodically. Then, for each row:
run the check, and call mark_rule_verified with what you found.
Rules with no `verify_with` never appear here. That is correct: they are
decisions, and there is nothing to go and check. Do not "fix" their
absence by giving them checks — the list is only worth reading while
everything on it genuinely can go false.
Args:
older_than_days: only rules last verified longer ago than this.
Never-checked rules always qualify. 0 = no age filter.
tier: "always_on" or "conditional" to narrow. An always-on constraint
that has gone false is the expensive kind — it is preloaded into
every session, so a wrong one is wrong everywhere at once.
never_only: only rules nobody has ever verified.
NOT filterable by project, deliberately: a project reaches rules through
project scope, subscriptions, always-on rulebooks and exclusions, and a
filter that missed one of those paths would UNDER-report — which is the
exact failure this whole surface exists to prevent. Read the whole list.
"""
uid = current_user_id()
rules = await rulebooks_svc.rules_due_for_verification(
uid, older_than_days=older_than_days, tier=tier, never_only=never_only,
)
return {
"rules": [rulebooks_svc.verification_row(r) for r in rules],
"total": len(rules),
}
async def mark_rule_verified(rule_id: int, still_true: bool = True) -> dict:
"""Record that you ran a rule's check — and what it said.
Call this AFTER actually running the rule's `verify_with`, never on the
strength of the rule sounding plausible. A stamp nobody earned is worse
than no stamp: it moves the rule to the bottom of the sweep and buys it
another long silence.
`still_true=False` writes NOTHING. A rule whose check failed is not in a
special state to be recorded — it is WRONG, and the only honest next
moves are to correct it, retire it, or find out why. So it stays at the
top of the sweep until someone deals with it, and the response tells you
what the rule said would end it.
Args:
rule_id: the rule whose check you ran.
still_true: True if the check passed. False if the fact it asserts is
no longer true — say so, that is the outcome worth having.
"""
uid = current_user_id()
rule = await rulebooks_svc.mark_rule_verified(rule_id, uid, still_true)
if rule is None:
raise ValueError(
f"rule {rule_id} not found, or carries no verify_with "
f"(nothing to verify is not the same as verified)"
)
data = await rulebooks_svc.rule_detail(uid, rule)
if still_true:
data["verified"] = True
return data
data["verified"] = False
data["next"] = (
"This rule is no longer true and is still binding on every session "
"that loads it. Correct it with update_rule, retire it with "
"delete_rule, or open a task to work out what replaced it. Its "
"verified_at is deliberately untouched, so it stays at the top of "
"rules_due_for_verification until one of those happens."
)
return data
def register(mcp) -> None:
for fn in (
list_rulebooks, get_rulebook, create_rulebook, update_rulebook, delete_rulebook,
@@ -702,5 +793,6 @@ def register(mcp) -> None:
suppress_rule_for_project, unsuppress_rule_for_project,
suppress_topic_for_project, unsuppress_topic_for_project,
exclude_always_on_rulebook, include_always_on_rulebook,
rules_due_for_verification, mark_rule_verified,
):
mcp.tool(name=fn.__name__)(fn)