14 Commits

Include renames

Author	SHA1	Message	Date
bvandeusen	b134fe9aa1	fix(tests): the column guard names join tables as _BACKED_UP holds them (#3182) ... _BACKED_UP carries REAL table names — project_rulebook_subscriptions, project_rule_suppressions, project_topic_suppressions, project_rulebook_exclusions — not the shorter keys the payload uses for the same sections. The registry-coverage assertion used the payload spelling and reported four tables unguarded. The integration round trip passed on this run, which is the half that matters: the real restore_full_backup remaps arose_from_id onto the restored origin.	2026-08-28 15:24:38 -04:00
bvandeusen	a6ef3a6a5a	fix(backup): a restore stops flattening the record vocabulary, and a column guard stops the next one (#3182) ... `_note_rows` carried 16 of the `notes` table's 27 columns. A backup -> restore cycle reported success and handed back a corpus with every snippet and process flattened into a plain note, every issue and spike into `work`, every provenance edge gone, and recurring tasks no longer recurring. The record-type and kind vocabulary is what #3128 and milestone 312 were about, and a restore erased it. Two more found by auditing every row helper rather than only the one being edited: `_milestone_rows` dropped `body` — a milestone IS the plan (0066), so every plan restored as a title with no reasoning behind it — and `_repo_binding_rows` dropped `ref`, the branch a ledger follows (#2873), so a restored binding silently accounts for a different tree. `arose_from_id` is deferred to a second pass beside `parent_id`, never written in the constructor: it is an id in the SOURCE database, so copying it through lands the edge on whatever record happens to hold that number here. An edge whose target did not survive stays NULL rather than being guessed at. This is the trap that kept the fix out of milestone 317 step 1. THE STRUCTURAL HALF. The coverage guard from #2293 checks TABLES against Base.metadata; nothing checked COLUMNS, which is how nine went missing from a table that had been "covered" for years — added to the model and the migration, both of which fail loudly, and never to the serialiser, which fails silently. `_COLUMN_EXCLUSIONS` now declares, per table, every column deliberately not exported and why, and a parameterised guard walks all 23 helpers and asserts the two agree. Forgetting is no longer expressible. Reconciling all 23 turned up one more deliberate exclusion worth naming: the `code_shapes` proposal columns are the machine's standing suggestion, cleared by judgment and recomputed by every refresh, so carrying them would restore stale guesses over a tree the proposer has not seen. Tests: the round trip drives the REAL restore_full_backup against Postgres, not a reimplementation of its loop — a test that re-derives the remap it is checking would agree with whatever the product does, including nothing. Backup v12.	2026-08-28 15:22:50 -04:00
bvandeusen	2263fd04a4	fix(tests): the notes table has 27 columns — deleted_batch_id is the second deliberate exclusion (#3165) ... The pin test caught its own inaccuracy on the first run, which is what it is for. `deleted_batch_id` comes from SoftDeleteMixin alongside `deleted_at` and is excluded for the same reason: trashed rows are not exported, so neither is the batch id that groups them for restore(). The nine-field gap #3182 tracks is unchanged.	2026-08-28 14:55:44 -04:00
bvandeusen	2065781302	feat(notes): a note can carry its own check — verify_with, expires_when, verified_at (#3165, milestone 317 step 1) ... The sibling of migration 0090, one table over. Same distinction: a NORM is a decision with no truth value; a CONSTRAINT asserts a fact about someone else's software and goes false with nobody watching. Notes hold far more constraints than rules do and hold them longer — a cross-project reference asserting what a signing service does on a duplicate upload is believed by every project that reads it, and nothing in the record says when anyone last looked. note_supersessions only fires once a human has already believed it. Three nullable columns, no backfill, no index. The index margin is thinner than 0090's — thousands of note rows against hundreds of rules — so the comment says to decide it in step 3 against a real query plan rather than guessing here. The columns land on every row in `notes`, but only non-task, non-snippet records will be OFFERED them (gated at the service in step 2): a task's decay is its status, and a snippet already carries a richer location-aware verdict in data.verification. A schema-level gate would have meant a CHECK across three columns to say what the write path says in two lines. Backup carries the trio (v11), with `verified_at` restored through _dt_or_none — _dt substitutes now(), which would restore every never-checked note as checked at the moment of the restore, inverting the one signal the sweep reads. Found while doing that, NOT fixed here, and now pinned by a test: `_note_rows` carries 16 of the `notes` table's 26 columns. note_type, task_kind, arose_from_id, the recurrence pair, the lifecycle stamps, description and data have all been missing for a long time, so a restore flattens every snippet and process into a plain note and every issue and spike into `work`. The coverage guard cannot see it — it checks TABLES, not columns, which is #2293's failure mode one level down. #3182 tracks it; arose_from_id needs the second id-remapping pass parent_id gets, which is why it is not a drive-by fix.	2026-08-28 14:54:07 -04:00
bvandeusenandClaude Opus 5	e08e999406	feat(rules): a rule can carry its own check — verify_with, expires_when, verified_at (#3095, milestone 312 step 1) ... A rulebook holds two kinds of row in one table. A NORM is a decision: no truth value, changes only when its author changes it, and they know they did. A CONSTRAINT asserts a fact about someone else's software, and goes false with nobody present. Milestone 307's audit found nine stale sites; every one was a constraint, and not one norm had rotted. Three nullable columns so a rule can say how to check itself. expires_when is a STATE, not a date — constraints expire when the ground moves, not on a schedule. verified_at NULL means never checked and sorts FIRST in the sweep to come: unexamined outranks examined-long-ago. Most rules set none of the three; a null verify_with is the marker for "this is a decision, there is nothing to go and check," and it only reads that way while it stays honest. Nothing is backfilled and nothing is indexed. A migration cannot invent a check any more than 0088 could invent a trigger, and the sweep reads a whole rulebook — hundreds of rows, on operator demand, never on a request path. Also, in the backup service the fields had to pass through: - Restore now remaps arose_from_id through note_id_map. It has been exported since 0088 and silently dropped on the way back in ever since, so every restore lost every rule's provenance link. - _dt_or_none, because _dt substitutes now() for an absent value. That is right for created_at/updated_at and wrong here: a rule nobody ever checked would restore looking freshly checked and fall to the bottom of the sweep it should top. Column additions do not move BACKUP_VERSION; only new sections do, as when 0088 added when_to_apply/tier/arose_from_id to the same helper. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 07:44:13 -04:00
bvandeusenandClaude Fable 5	e9b8f525c8	feat(inception): projects.inception record + project_rulebook_exclusions — migration 0085 with legacy backfill; backup v10 (#2879, milestone 297 step 1) ... A project's inheritance becomes a decision, not a default (milestone 297). - projects.inception (JSONB, NULL = undecided): {decided_at, decided_by, via mcp|ui|legacy, choices {exclude_always_on_rulebooks, subscribe_rulebooks, design_system_id, seed_systems}}; on to_dict. - project_rulebook_exclusions: a project's opt-out of a whole always-on rulebook — the sibling of the rule/topic suppressions, CASCADE both ways. - services/inception.py (first cut): the vocabulary, validate_inception (pure, all-or-nothing), normalize_choices, is_decided. Effects come in step 3. - Migration 0085 backfills every existing project via="legacy" with its current standing (no exclusions, its subscriptions, its design_system_id, no seed) so the ask fires only for projects created after this ships. - Backup v10: rulebook_exclusions section; project rows carry inception and design_system_id, restored in a post-pass once rulebooks/design systems are mapped (design_system_id was not restored before — fixed in passing). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-21 21:57:02 -04:00
bvandeusenandClaude Fable 5	6a0f8ad328	feat(backup): code_shape_uses travels (v9); consumer-map test expects uses (#2870) ... Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-21 15:21:13 -04:00
bvandeusenandClaude Fable 5	d74a244b3a	feat(ledger): divergence readout — button B where button A is canon, shape history, and judged-shape recheck (#2793, milestone 294 step 7) ... Every judgment now goes through one helper that remembers the fingerprint judged (classified_sha) and writes a code_shape_events row; the sync writes vanished / reappeared / drifted events and flags recheck_at when a body moves under an instance/variant. The refresh flags diverges_from on shapes new since the previous computation that sit where one canon dominates the judged siblings of their directory+kind and were not proposed as that canon (a first seed flags nothing); the write-path hint asks the same question in-band for the shapes the hook names. list_shapes(flag=divergence|recheck), shape_history(project_id, path, symbol) (read-only), coverage line/payload/ card carry divergent + recheck. Backup v8 carries the history. Plugin 0.1.36. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-20 23:13:39 -04:00
bvandeusenandClaude Fable 5	19fdc9aa89	feat(ledger): code_shapes — the shape ledger schema (#2787, milestone 294 step 1) ... The accounting half of the pattern system (governing note 2786): the snippet library records canon (small), this table accounts for EVERY extracted shape (total). Identity is (project, repo_key, path, symbol, kind) — kind included because one file can define '.foo' (css) and 'foo' (sym) as distinct shapes. Status vocabulary: canonical / instance / variant / exempt / unclassified, with unclassified as the default and THE todo state; classifications carry who judged (agent|audit|hook|mechanical|import), when, and the why for variants/exemptions. first/last-seen commits + vanished_at keep history instead of deleting it; a rename reads as vanish+new (accepted for v1). snippet_id is SET NULL on snippet deletion so accounting rows outlive their target and rejoin the todo via the step-2 sync, never dangle silently. Backups: v7 carries code_shapes (judgment data, worth moving) — full and per-user export sections, and a restore that keeps a judgment only when its snippet survives the id re-mapping, downgrading to unclassified otherwise. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-19 19:25:25 -04:00
bvandeusen	5dcb738ce8	fix(backup): carry note_supersessions — the coverage guard caught the omission ... CI failed on 45c6b1c. `tests/test_services_backup.py` asserts every table is either in `_BACKED_UP` or explicitly in `_NOT_INCLUDED`, and the new table was in neither. That guard exists for exactly this: a new table gets a model and a migration — both of which fail loudly if wrong — and then silently never gets a backup section. No error, no warning, and a restore that reports success. Its comment says it was written to "stop the seventh" table slipping through. This was the seventh. Supersession claims are backed up rather than excluded because they are a JUDGEMENT. Someone decided this note replaced that one, and nothing in either note's text records the decision. Lose them and a restored corpus silently returns to ranking stale material alongside current material — with no symptom that says why. Scoping differs by backup kind, deliberately. A whole-instance export takes every row. A single-user export takes only claims where BOTH ends are that user's notes: a claim spanning out to someone else's record cannot be restored into a single-user import, since the far id is not in the map, so exporting it would write a row that silently vanishes on the way back in. Restore maps both ids and skips the row unless both resolve. A claim is about a PAIR — half of one is not a weaker claim, it is a dangling row pointing at whatever note holds that id next. Guarded by `data.get` like every post-v2 section, so v5 and older payloads restore cleanly without it. BACKUP_VERSION 5 -> 6, and the version test moved with it. A payload section added without moving the version produces backups that are structurally different and indistinguishable by inspection. Refs #278	2026-08-07 21:49:08 -04:00
bvandeusenandClaude Opus 5	2cc9e1380e	test(backup): assert the version constant, not a copy of it ... The export test pinned `out["version"] == 4`, so bumping BACKUP_VERSION broke a test that was only ever checking the payload carries the version — which it still did. Asserts against backup.BACKUP_VERSION now, and covers the six v5 sections alongside the v3 ones. The guard itself passed on the first run: every table in Base.metadata was accounted for, in both directions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 23:13:16 -04:00
bvandeusenandClaude Opus 5	84541f392b	fix(backup): six tables were silently absent, and nothing would catch a seventh ... services/backup.py enumerated its tables as hand-maintained literals with nothing tying them to the schema. Tables added since that list was last extended were absent from every backup — no error, no warning, and a restore that reports success. Missing: systems, record_systems (0065), note_usage_events (0071), design_systems, design_tokens (0072), and repo_bindings — which the issue itself had not spotted, found only by diffing the model tablenames against the two lists instead of trusting either. _NOT_INCLUDED was worse than incomplete: it named "embeddings", "invitations" and "password_resets", none of which are tables. It read as coverage while naming nothing the schema could confirm. Now real names, plus retrieval_logs — observational telemetry that grows per query and that nothing reads for correctness. THE DELIVERABLE IS THE GUARD, not the six sections. Extending a list fixes today and changes nothing about the next table; a new one now fails a test until someone either backs it up or states that it shouldn't be. It checks both directions — an unaccounted table, and a listed name that no longer exists, which is what the three phantom entries above would have tripped. Design systems need ordering care: parent_id is a self-FK. The export orders parent-first (parent_id NULLS FIRST, then id — a parent always has the smaller id), so restore resolves each parent from the map as it goes, with no second pass. A child whose parent is missing lands as a root rather than failing the whole restore. Usage events are kept because pull-through is the evidence base for whether recall works, and it only ever accumulates — a restore that dropped it would reset that measurement to zero while everything still looked fine. BACKUP_VERSION 4 -> 5. Every new restore section is data.get()-guarded, so v2/v3/v4 payloads restore unchanged. Closes #2293. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 23:09:25 -04:00
bvandeusenandClaude Opus 4.8	b49efdcb11	refactor(scribe): retire calendar/events + person/place/list entities (backend) ... Narrow Scribe to a Claude-Code work system-of-record (milestone #194, decision note #1759). Wholesale removal per rule #22 — backend + schema half. Calendar/events + CalDAV: delete models/event, services/{events,caldav, caldav_sync}, routes/events, mcp/tools/events; strip event branches from backup (bump v3->v4), dashboard (upcoming_events), trash, recent, and the mcp server read-only allowlist + instructions. Typed entities (person/place/list): delete mcp/tools/entities; drop the notes.metadata (entity_meta) column from model/service/routes and the knowledge browse service. note_type STAYS — it also marks 'process' notes. Scheduler: event_scheduler -> recurrence_scheduler, keeping only the recurring-task spawn job (drops event reminders + CalDAV sync). Schema: migration 0069 drops the events table + notes.metadata column + orphan caldav settings rows (faithful downgrade recreates them). KEEP: recurrence.py (task recurrence), notifications task reminders, graph view, and every work surface. Frontend + plugin/docs true-up follow next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPtbSzA4JLMAKgFZ8VTg7Q	2026-07-19 13:29:14 -04:00
bvandeusenandClaude Opus 4.8	d6c8470ab2	feat(backup): v3 backup covers rulebooks, rules, events + join tables ... The v2 backup silently dropped the entire rulebook system (rulebooks, topics, rules), the project subscription/suppression join tables, and events — so a 'full' backup wasn't. v3 adds all of them with FK re-mapping on restore, and a _not_included field that names the still-deferred tables (ACL groups/shares, api_keys, embeddings, transient/operational) so the gap is explicit, not silent. restore_full_backup routes v2 and v3 through one path; v3-only sections are guarded by data.get so a v2 payload still restores cleanly. Tests: version/coverage constants, pure join-table row helpers, and the export contract via a mocked session (CI has no DB; full round-trip is a manual check). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-09 23:52:14 -04:00