feat(rules): the staleness sweep — which standing rules assert a fact nobody has confirmed (#3097, milestone 312 step 3)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m8s
CI & Build / Build & push image (push) Successful in 35s
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m8s
CI & Build / Build & push image (push) Successful in 35s
The query the last two steps were storage for. `rules_due_for_verification` returns every rule carrying a `verify_with`, ordered by `verified_at` ASC NULLS FIRST, each row carrying the check IN FULL — the opposite call from rule_brief, because the reader is about to go and run it. NULLS FIRST is the ordering this turns on. Postgres sorts NULLs last on an ASC ordering, which would put the rules nobody has ever confirmed BEHIND every rule someone once looked at. Exactly backwards: a claim with no evidence at all outranks an old one. Rules with no check never appear, and that is the property that keeps the list worth reading. Most rules are decisions — no truth value, nothing to go and check. If they appeared here the sweep would be the rulebook. `mark_rule_verified(rule_id, still_true)` closes the loop, asymmetrically: passing writes a stamp, FAILING WRITES NOTHING. There is no "verified false" state because a rule whose check failed is not in a special condition, it is wrong — and recording the failure as a flag would let it sit there being false with the sweep satisfied that someone had looked. So it stays at the top until someone corrects or retires it, and the response says so. An unrecognised `tier` filter raises rather than falling back. _valid_tier's silent always_on default is right for a WRITE — a typo should leave a rule binding — and wrong for a FILTER, where the same fallback quietly answers a different question and returns a short list that reads as good news. Deliberately NOT filterable by project: a project reaches rules through project scope, subscriptions, always-on rulebooks and exclusions, and a filter missing one of those paths would UNDER-report — the exact failure this surface exists to prevent. Said so in the docstring rather than shipping a half-correct filter. Ownership-scoped like every other rule read (owned rulebook, or owned project), in ONE statement with an OR across the XOR rather than two queries merged in Python, so the ordering is the database's and cannot disagree with itself. Note that rules have no sharing ACL in this schema — no rule_shares, no rulebook_shares — so there is no wider set for access.py to consult here. Also fixes a test title that had been lying for ten tools: "all sixteen tools" asserted 26. The number now lives only in the assertion. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -691,6 +691,97 @@ async def unrelate_rules(relation_id: int) -> dict:
|
||||
raise ValueError(f"relation {relation_id} not found")
|
||||
return {"deleted": relation_id}
|
||||
|
||||
# ── The staleness sweep (milestone 312) ────────────────────────────────
|
||||
|
||||
async def rules_due_for_verification(
|
||||
older_than_days: int = 0, tier: str = "", never_only: bool = False,
|
||||
) -> dict:
|
||||
"""Which standing rules assert a FACT that nobody has confirmed lately.
|
||||
|
||||
A rulebook holds two kinds of thing. Most rules are DECISIONS — how the
|
||||
operator wants to work. They have no truth value and cannot rot. A few
|
||||
assert a fact about someone else's software: what a CI runner does, which
|
||||
tools exist, what a setting is currently set to. Those go false silently,
|
||||
with nobody present, and they keep being handed to every session as
|
||||
binding instructions long after they stopped being true.
|
||||
|
||||
This lists the second kind, oldest verification first, never-checked at
|
||||
the top. Each row carries the rule's `verify_with` in full — you are
|
||||
about to go and run it — plus `expires_when`, and `days_since_verified`.
|
||||
|
||||
Reach for it when you are curating the rulebook, when a rule's advice
|
||||
just contradicted what you observed, or periodically. Then, for each row:
|
||||
run the check, and call mark_rule_verified with what you found.
|
||||
|
||||
Rules with no `verify_with` never appear here. That is correct: they are
|
||||
decisions, and there is nothing to go and check. Do not "fix" their
|
||||
absence by giving them checks — the list is only worth reading while
|
||||
everything on it genuinely can go false.
|
||||
|
||||
Args:
|
||||
older_than_days: only rules last verified longer ago than this.
|
||||
Never-checked rules always qualify. 0 = no age filter.
|
||||
tier: "always_on" or "conditional" to narrow. An always-on constraint
|
||||
that has gone false is the expensive kind — it is preloaded into
|
||||
every session, so a wrong one is wrong everywhere at once.
|
||||
never_only: only rules nobody has ever verified.
|
||||
|
||||
NOT filterable by project, deliberately: a project reaches rules through
|
||||
project scope, subscriptions, always-on rulebooks and exclusions, and a
|
||||
filter that missed one of those paths would UNDER-report — which is the
|
||||
exact failure this whole surface exists to prevent. Read the whole list.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rules = await rulebooks_svc.rules_due_for_verification(
|
||||
uid, older_than_days=older_than_days, tier=tier, never_only=never_only,
|
||||
)
|
||||
return {
|
||||
"rules": [rulebooks_svc.verification_row(r) for r in rules],
|
||||
"total": len(rules),
|
||||
}
|
||||
|
||||
|
||||
async def mark_rule_verified(rule_id: int, still_true: bool = True) -> dict:
|
||||
"""Record that you ran a rule's check — and what it said.
|
||||
|
||||
Call this AFTER actually running the rule's `verify_with`, never on the
|
||||
strength of the rule sounding plausible. A stamp nobody earned is worse
|
||||
than no stamp: it moves the rule to the bottom of the sweep and buys it
|
||||
another long silence.
|
||||
|
||||
`still_true=False` writes NOTHING. A rule whose check failed is not in a
|
||||
special state to be recorded — it is WRONG, and the only honest next
|
||||
moves are to correct it, retire it, or find out why. So it stays at the
|
||||
top of the sweep until someone deals with it, and the response tells you
|
||||
what the rule said would end it.
|
||||
|
||||
Args:
|
||||
rule_id: the rule whose check you ran.
|
||||
still_true: True if the check passed. False if the fact it asserts is
|
||||
no longer true — say so, that is the outcome worth having.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rule = await rulebooks_svc.mark_rule_verified(rule_id, uid, still_true)
|
||||
if rule is None:
|
||||
raise ValueError(
|
||||
f"rule {rule_id} not found, or carries no verify_with "
|
||||
f"(nothing to verify is not the same as verified)"
|
||||
)
|
||||
data = await rulebooks_svc.rule_detail(uid, rule)
|
||||
if still_true:
|
||||
data["verified"] = True
|
||||
return data
|
||||
data["verified"] = False
|
||||
data["next"] = (
|
||||
"This rule is no longer true and is still binding on every session "
|
||||
"that loads it. Correct it with update_rule, retire it with "
|
||||
"delete_rule, or open a task to work out what replaced it. Its "
|
||||
"verified_at is deliberately untouched, so it stays at the top of "
|
||||
"rules_due_for_verification until one of those happens."
|
||||
)
|
||||
return data
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (
|
||||
list_rulebooks, get_rulebook, create_rulebook, update_rulebook, delete_rulebook,
|
||||
@@ -702,5 +793,6 @@ def register(mcp) -> None:
|
||||
suppress_rule_for_project, unsuppress_rule_for_project,
|
||||
suppress_topic_for_project, unsuppress_topic_for_project,
|
||||
exclude_always_on_rulebook, include_always_on_rulebook,
|
||||
rules_due_for_verification, mark_rule_verified,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
|
||||
Reference in New Issue
Block a user