Rules that can go stale say so — verify_with, expires_when, the sweep, and task_kind='spike' (milestone 312, steps 1–5) #132

Merged

bvandeusen merged 8 commits from dev into main 2026-08-27 13:43:58 -04:00

Conversation 0 Commits 8 Files Changed 29

+1856 -56

bvandeusen commented 2026-08-27 13:43:01 -04:00

Owner

Copy Link

Copy Source

A rulebook holds two kinds of row. A norm is a decision: no truth value, it changes only when its author changes it, and they know they did. A constraint asserts a fact about someone else's software — a runner's shell, a bot's config, whether a tool exists — and it goes false silently, with nobody present.

Milestone 307's rulebook audit found nine stale sites. Every one was a constraint; not one norm had rotted. One of them had been telling every session to skip database-backed tests for weeks while the integration lane sat green in the workflow.

This lets a rule carry its own check, and turns staleness from something you discover into something you query.

What lands

Schema (0090) — three nullable columns on rules: verify_with (how to check it is still true), expires_when (the state that ends it, deliberately not a date), verified_at. Nothing backfilled: a migration cannot invent a check any more than 0088 could invent a trigger. No index — the sweep reads a whole rulebook on operator demand, never on a request path.

Write path — both doors, each keeping its own idiom. MCP keeps "" = "leave unchanged" (an agent filling three fields must not wipe the other five), so clearing is explicit via clear_fields=[...]. REST takes "" from a cleared form input, and the service normalises "" → NULL for every nullable rule column. That normalisation is what makes the sweep query mean anything: without it verify_with IS NOT NULL would be true for every rule ever touched through the UI.

Editing verify_with drops verified_at. A stamp certifies a check, not a rule; reword the check and the old stamp vouches for something that no longer exists.

The sweep — rules_due_for_verification, ordered verified_at ASC NULLS FIRST. Postgres sorts NULLs last by default, which would put the rules nobody has ever confirmed behind every rule someone once looked at — exactly backwards. mark_rule_verified(rule_id, still_true) is asymmetric: passing writes a stamp, failing writes nothing. There is no "verified false" state, because a rule whose check failed is not in a special condition — it is wrong, and recording the failure would let it sit there being false with the sweep satisfied that someone had looked.

UI (rule 27) — the fields in the rule editor under a legend that asks the real question and says empty is the normal answer; a verify affordance on the saved rule; a chip on rules that carry a check; a dedicated sweep pane. No age-graded colour anywhere: the list is already ordered by urgency, so a red/amber ramp would restate the ordering and require an invented "stale after N days" threshold. Only "never" is marked, by weight rather than hue.

task_kind = 'spike' (0091) — a spike is time-boxed and its output is knowledge rather than a change; it succeeds with an empty diff. It is the record a failed check asks for. Rule 36 honoured: the value and the widened CHECK land in the same migration, DROP then ADD, as 0065 did for issue.

Two fixes found along the way

• arose_from_id was exported by the backup service and silently dropped on restore, since 0088 — every restore lost every rule's provenance link. note_id_map was already in scope; one-line remap.
• _dt() substitutes now() for an absent timestamp. Right for created_at; exactly wrong for verified_at, where it would restore a never-checked rule as freshly checked. Added _dt_or_none.

Verification

CI green on 15659e2 — typecheck, plugin hooks, lint, unit tests, integration, image build. The integration lane ran alembic upgrade head against real Postgres for both migrations; integration selection went 66 → 83, all passing. Dangling-style and design-token reports compared against the previous run: unchanged, no new entries.

Known gap: the UI is unexercised beyond vue-tsc — no component test, no browser. The sweep is reached by a new control that nothing else routes through; the editor changes are two textareas and a conditional block.

After merging

The sweep will be empty — no rule carries a check yet, so it will correctly read "Nothing to check." Milestone 312 step 6 (#3100) is the instance track that fills it, and it has to run against the deployed instance.

Confirmed while filing #3126: create_task(kind="spike") currently fails against the running instance with CheckViolationError — 0091 is not deployed there yet. That is what this merge fixes.

Steps 1–5 of milestone 312; step 6 is rulebook curation, not product code, and is deliberately not in this PR.

🤖 Generated with Claude Code

A rulebook holds two kinds of row. A **norm** is a decision: no truth value, it changes only when its author changes it, and they know they did. A **constraint** asserts a fact about someone else's software — a runner's shell, a bot's config, whether a tool exists — and it goes false silently, with nobody present. Milestone 307's rulebook audit found nine stale sites. **Every one was a constraint; not one norm had rotted.** One of them had been telling every session to skip database-backed tests for weeks while the integration lane sat green in the workflow. This lets a rule carry its own check, and turns staleness from something you discover into something you query. ## What lands **Schema (0090)** — three nullable columns on `rules`: `verify_with` (how to check it is still true), `expires_when` (the *state* that ends it, deliberately not a date), `verified_at`. Nothing backfilled: a migration cannot invent a check any more than 0088 could invent a trigger. No index — the sweep reads a whole rulebook on operator demand, never on a request path. **Write path** — both doors, each keeping its own idiom. MCP keeps `""` = "leave unchanged" (an agent filling three fields must not wipe the other five), so clearing is explicit via `clear_fields=[...]`. REST takes `""` from a cleared form input, and the service normalises `""` → NULL for every nullable rule column. That normalisation is what makes the sweep query mean anything: without it `verify_with IS NOT NULL` would be true for every rule ever touched through the UI. Editing `verify_with` drops `verified_at`. A stamp certifies *a check*, not a rule; reword the check and the old stamp vouches for something that no longer exists. **The sweep** — `rules_due_for_verification`, ordered `verified_at` ASC **NULLS FIRST**. Postgres sorts NULLs last by default, which would put the rules nobody has ever confirmed *behind* every rule someone once looked at — exactly backwards. `mark_rule_verified(rule_id, still_true)` is asymmetric: passing writes a stamp, **failing writes nothing**. There is no "verified false" state, because a rule whose check failed is not in a special condition — it is wrong, and recording the failure would let it sit there being false with the sweep satisfied that someone had looked. **UI (rule 27)** — the fields in the rule editor under a legend that asks the real question and says empty is the normal answer; a verify affordance on the saved rule; a chip on rules that carry a check; a dedicated sweep pane. No age-graded colour anywhere: the list is already ordered by urgency, so a red/amber ramp would restate the ordering *and* require an invented "stale after N days" threshold. Only "never" is marked, by weight rather than hue. **`task_kind = 'spike'` (0091)** — a spike is time-boxed and its output is knowledge rather than a change; it succeeds with an empty diff. It is the record a failed check asks for. Rule 36 honoured: the value and the widened CHECK land in the same migration, DROP then ADD, as 0065 did for `issue`. ## Two fixes found along the way - **`arose_from_id` was exported by the backup service and silently dropped on restore**, since 0088 — every restore lost every rule's provenance link. `note_id_map` was already in scope; one-line remap. - **`_dt()` substitutes `now()` for an absent timestamp.** Right for `created_at`; exactly wrong for `verified_at`, where it would restore a never-checked rule as freshly checked. Added `_dt_or_none`. ## Verification CI green on `15659e2` — typecheck, plugin hooks, lint, unit tests, integration, image build. The integration lane ran `alembic upgrade head` against real Postgres for both migrations; integration selection went 66 → 83, all passing. Dangling-style and design-token reports compared against the previous run: unchanged, no new entries. **Known gap:** the UI is unexercised beyond `vue-tsc` — no component test, no browser. The sweep is reached by a new control that nothing else routes through; the editor changes are two textareas and a conditional block. ## After merging **The sweep will be empty** — no rule carries a check yet, so it will correctly read "Nothing to check." Milestone 312 step 6 (#3100) is the instance track that fills it, and it has to run against the deployed instance. Confirmed while filing #3126: `create_task(kind="spike")` currently fails against the running instance with `CheckViolationError` — 0091 is not deployed there yet. That is what this merge fixes. Steps 1–5 of milestone 312; step 6 is rulebook curation, not product code, and is deliberately not in this PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 8 commits 2026-08-27 13:43:01 -04:00

feat(rules): a rule can carry its own check — verify_with, expires_when, verified_at (#3095, milestone 312 step 1) ... e08e999406

A rulebook holds two kinds of row in one table. A NORM is a decision: no
truth value, changes only when its author changes it, and they know they
did. A CONSTRAINT asserts a fact about someone else's software, and goes
false with nobody present. Milestone 307's audit found nine stale sites;
every one was a constraint, and not one norm had rotted.

Three nullable columns so a rule can say how to check itself. expires_when
is a STATE, not a date — constraints expire when the ground moves, not on a
schedule. verified_at NULL means never checked and sorts FIRST in the sweep
to come: unexamined outranks examined-long-ago. Most rules set none of the
three; a null verify_with is the marker for "this is a decision, there is
nothing to go and check," and it only reads that way while it stays honest.

Nothing is backfilled and nothing is indexed. A migration cannot invent a
check any more than 0088 could invent a trigger, and the sweep reads a whole
rulebook — hundreds of rows, on operator demand, never on a request path.

Also, in the backup service the fields had to pass through:

- Restore now remaps arose_from_id through note_id_map. It has been exported
  since 0088 and silently dropped on the way back in ever since, so every
  restore lost every rule's provenance link.
- _dt_or_none, because _dt substitutes now() for an absent value. That is
  right for created_at/updated_at and wrong here: a rule nobody ever checked
  would restore looking freshly checked and fall to the bottom of the sweep
  it should top.

Column additions do not move BACKUP_VERSION; only new sections do, as when
0088 added when_to_apply/tier/arose_from_id to the same helper.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

feat(rules): the write path carries a rule's check, and empty finally means empty (#3096, milestone 312 step 2) ... c61925be76

verify_with / expires_when now reach a rule through both doors and come back
on every read. The open question this step existed to settle was how to
UNSET a nullable field, and the answer is one convention per door:

- MCP: "" still means "leave unchanged" — an agent filling three fields must
  not wipe the other five — so clearing is explicit, clear_fields=["..."].
  Naming the field is the one form that cannot happen by accident.
- REST: a cleared form input arrives as "", and the service normalises "" to
  NULL for every nullable rule column, so an emptied input does what it looks
  like it does.

Two idioms, one outcome, and the normalisation is what makes the step-3 sweep
correct: `verify_with IS NOT NULL` would otherwise be true for every rule ever
touched through the UI, and the sweep would list the whole rulebook and mean
nothing. to_dict renders "" and NULL identically, so this is only visible
against a real column — hence the integration module rather than a mock.

Editing verify_with drops verified_at. A stamp certifies A CHECK, not a rule;
reword the check and the old stamp vouches for something that no longer
exists. Safe direction, same asymmetry as _valid_tier: a rule wrongly listed
as due costs one look, a rule wrongly vouched for costs the thing the sweep
exists to catch. Editing anything else leaves the stamp alone, or a rulebook
tidy-up would reset every constraint and the ordering would carry nothing.

Reads: rule_brief attaches `last_verified` ONLY to a rule that carries a
check — its presence is the signal, and it says both "this asserts a fact
that can go false" and "here is how long ago anyone confirmed it". "never"
rather than null, per #2483. The check text itself stays in get_rule; a
listing needs to know which rules can rot, not how to test them. Search hits
carry the full trio, since a hit is exactly the moment someone is about to
act on a rule.

Also folds in the #3078 finding, which had been sitting as a note: create_rule
now teaches that when_to_apply is the retrieval surface and must carry the
SYMPTOM — the words you would type while stuck — not just the situation.

fake_rule gains the three fields as None for the reason the helper already
documents one line up: unnamed, verify_with is a truthy MagicMock and every
stand-in rule would claim a check it does not have.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

test(rules): the kwargs assertion learns about clear (#3096, milestone 312 step 2) ... 469b43f222

test_update_rule_only_sends_non_default_fields pins that the MCP door
forwards only what the caller actually gave. `clear` is now always
forwarded — an empty tuple is "clear nothing", a value rather than an
absent argument — so the expected kwargs gained it. The property under
test is unchanged: everything left at its default still stays out.

Two tests added beside it while the shape is in view: naming a field for
clearing reaches the service as `clear`, and the check fields are
forwarded when given.

CI 4630 otherwise green — the integration lane ran all six of the new
real-Postgres cases (72 selected, was 66) and applied 0089 -> 0090.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

feat(rules): the staleness sweep — which standing rules assert a fact nobody has confirmed (#3097, milestone 312 step 3) ... 410d616c22

The query the last two steps were storage for. `rules_due_for_verification`
returns every rule carrying a `verify_with`, ordered by `verified_at` ASC
NULLS FIRST, each row carrying the check IN FULL — the opposite call from
rule_brief, because the reader is about to go and run it.

NULLS FIRST is the ordering this turns on. Postgres sorts NULLs last on an
ASC ordering, which would put the rules nobody has ever confirmed BEHIND
every rule someone once looked at. Exactly backwards: a claim with no
evidence at all outranks an old one.

Rules with no check never appear, and that is the property that keeps the
list worth reading. Most rules are decisions — no truth value, nothing to go
and check. If they appeared here the sweep would be the rulebook.

`mark_rule_verified(rule_id, still_true)` closes the loop, asymmetrically:
passing writes a stamp, FAILING WRITES NOTHING. There is no "verified false"
state because a rule whose check failed is not in a special condition, it is
wrong — and recording the failure as a flag would let it sit there being
false with the sweep satisfied that someone had looked. So it stays at the
top until someone corrects or retires it, and the response says so.

An unrecognised `tier` filter raises rather than falling back. _valid_tier's
silent always_on default is right for a WRITE — a typo should leave a rule
binding — and wrong for a FILTER, where the same fallback quietly answers a
different question and returns a short list that reads as good news.

Deliberately NOT filterable by project: a project reaches rules through
project scope, subscriptions, always-on rulebooks and exclusions, and a
filter missing one of those paths would UNDER-report — the exact failure
this surface exists to prevent. Said so in the docstring rather than
shipping a half-correct filter.

Ownership-scoped like every other rule read (owned rulebook, or owned
project), in ONE statement with an OR across the XOR rather than two queries
merged in Python, so the ordering is the database's and cannot disagree with
itself. Note that rules have no sharing ACL in this schema — no rule_shares,
no rulebook_shares — so there is no wider set for access.py to consult here.

Also fixes a test title that had been lying for ten tools: "all sixteen
tools" asserted 26. The number now lives only in the assertion.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

docs(rules): a project rule is shaped differently, not just scoped differently (milestone 312) ... 9d7485df2d

The three surfaces already agree on WHERE a rule goes — the using-scribe
skill's "Where a new rule goes" section and both tool docstrings frame it
as one question, who should this bind. What they did not say is that the
two homes want differently SHAPED rules, and one deferral was actively
misleading.

`create_project_rule` said `tier: "always_on" or "conditional" — see
create_rule`. That imports a bar calibrated for a different blast radius.
On a rulebook rule always_on means every session in every project, so the
test is severe: the trigger must be nameless. A project rule is already
scoped by construction, so always_on costs only that project's sessions —
and being specific, which the family test treats as the signal for
conditional, is what project rules are FOR. The instance's own data says
so: rules 78, 115 and 119 are all project rules and all always_on.

Not zero bar, a different one: conditional is right when the rule is about
one AREA of a large project, because forty always-on rules on one project
reproduces locally the preload bloat milestone 307 fixed globally.

Also:
- create_rule now says to write the general form WITHOUT hedging for
  exceptions — a project needing to narrow it writes its own and links
  with overrides/elaborates. A rulebook rule padded with "unless…" for two
  projects is two project rules that were never written. Only the project
  side mentioned that relationship; the side that benefits from it did not.
- arose_from_id: reach for it harder on a project rule, which usually comes
  from one traceable incident in the repo, where a family rule is more
  often a standing preference with no single origin.
- system_ids is worth setting on a project rule too — it is what lets a
  conditional one arrive with its area.
- when_to_apply no longer claims to "decide" the tier here, which stopped
  being true one entry down.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

feat(rules): the check is editable, visible, and sweepable in the UI (#3098, milestone 312 step 4) ... c83bedf3be

Rule 27 — the milestone was backend-only until this. Four surfaces:

RULE EDITOR — verify_with and expires_when under a legend that asks the
actual question ("Can this rule go stale?") and says empty is the normal
answer, because most rules are decisions and a form that implies a missing
field would get them filled in out of tidiness. When the SAVED rule carries
a check, the stamp shows with Still true / No longer true beside it. The
stamp reads the stored value, not the draft: an unsaved edit to the textarea
has not been run against anything.

SWEEP PANE — its own surface, not a filter on the rule list. That list can
only ever show one topic of one rulebook, and a rule that has gone false
belongs to no one rulebook; filtering it would under-report, which is the
failure this whole surface exists to catch. Reached from the rulebook list,
below the rulebooks, because that is where you go to look at rules.

RULE ROWS — a chip only on rules carrying a check, so its presence is the
signal. PROJECT RULES TAB — the check shows beside `why` when a rule has
one, read-only: that tab is the project's view of what binds it.

NO AGE-GRADED COLOUR anywhere, deliberately. The sweep is already ordered by
urgency, so a red/amber ramp would restate the ordering AND require an
invented "stale after N days" threshold — a magic number nobody could defend
and the first thing to go out of date. --fs-overdue is error red and reserved
for a broken promise like a missed due date; a verification age is not one,
and colouring it that way makes a rule someone just wrote look broken. Only
"never" is marked, because it is categorically different from a date rather
than a worse one — and it is marked by weight, not hue.

An empty sweep says "Nothing to check", not nothing: good news must not read
as a broken page.

Two chips (tier, then verification) turned out byte-identical, so .rule-chip
moves to rules-shared.css and snippet #2906 is updated to match rather than
left describing a file that has moved on. Its header comment counted the
panes it served; that count went stale the moment a fourth arrived, so it no
longer counts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

feat(tasks): task_kind gains 'spike' — the investigation, not the change (#3099, milestone 312 step 5) ... 88e9c0b0bd

A spike is a shape the other kinds cannot hold. `work` ships a change;
`issue` fixes something broken. A spike is time-boxed and its output is
KNOWLEDGE — it succeeds by producing an answer, and nothing ships at the end
of it. Filing one as `work` makes a finished investigation look like an
abandoned change, which is why the distinction earns a value rather than a
convention.

It is also the record a failed check asks for. This milestone gave rules a
verify_with; when one fails the rule is wrong, and the next move is often to
go and find out what replaced it. notes.arose_from_id already exists (0065),
so constraint -> spike provenance needed no schema at all — only a docstring
saying it is there.

Rule 36: the value and the widened CHECK land in the same migration, DROP
then ADD, exactly as 0065 did for 'issue'. The two whitelists live in one
tuple each so upgrade and downgrade cannot disagree about what the list was
on either side. The downgrade demotes existing spikes to 'work' first —
lossy, deliberately, because the alternative is a downgrade that fails on
real data, and one that says what it did beats one that cannot run.

'plan' stays whitelisted though retired: historical plan-tasks carry it, and
a row that cannot be rewritten cannot be edited, restored or migrated.

The integration test asserts both halves. A test that only proved 'spike' is
accepted would pass just as happily against a table whose CHECK had been
dropped and never re-added — which is the other way rule 36's failure
happens — so an unknown kind is asserted to still raise.

Not in scope, deliberately: any special lifecycle, time-box enforcement, or
gating relationship. It is a kind, not a workflow.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(tests): a Note's is_task cannot be set — status is what makes one (#3099) ... 15659e2c57

The spike CHECK tests constructed Note(is_task=True). `is_task` is a derived
read-only property — `status is not None` — so SQLAlchemy raised
"property 'is_task' of 'Note' object has no setter" before any row reached
the database. All three failed for that, not for anything about migration
0091; the other 80 integration tests passed, including 0090's.

status="todo" is what makes a note a task. Noted inline, since the field
appears in to_dict output and reads like an ordinary column from there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

bvandeusen merged commit 5aabc31ee7 into main 2026-08-27 13:43:58 -04:00

bvandeusen referenced this pull request 2026-08-27 17:59:14 -04:00

A task's kind is correctable — the Kind select stops lying (#3129) #133

bvandeusen referenced this issue from a commit 2026-08-27 18:00:34 -04:00

merge: bring main's rebased history into dev after PR #132

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

renovate-bot bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledScribe#132