services/backup.py enumerated its tables as hand-maintained literals with
nothing tying them to the schema. Tables added since that list was last
extended were absent from every backup — no error, no warning, and a restore
that reports success.
Missing: systems, record_systems (0065), note_usage_events (0071),
design_systems, design_tokens (0072), and repo_bindings — which the issue
itself had not spotted, found only by diffing the model tablenames against the
two lists instead of trusting either.
_NOT_INCLUDED was worse than incomplete: it named "embeddings", "invitations"
and "password_resets", none of which are tables. It read as coverage while
naming nothing the schema could confirm. Now real names, plus retrieval_logs —
observational telemetry that grows per query and that nothing reads for
correctness.
THE DELIVERABLE IS THE GUARD, not the six sections. Extending a list fixes
today and changes nothing about the next table; a new one now fails a test
until someone either backs it up or states that it shouldn't be. It checks
both directions — an unaccounted table, and a listed name that no longer
exists, which is what the three phantom entries above would have tripped.
Design systems need ordering care: parent_id is a self-FK. The export orders
parent-first (parent_id NULLS FIRST, then id — a parent always has the smaller
id), so restore resolves each parent from the map as it goes, with no second
pass. A child whose parent is missing lands as a root rather than failing the
whole restore.
Usage events are kept because pull-through is the evidence base for whether
recall works, and it only ever accumulates — a restore that dropped it would
reset that measurement to zero while everything still looked fine.
BACKUP_VERSION 4 -> 5. Every new restore section is data.get()-guarded, so
v2/v3/v4 payloads restore unchanged.
Closes#2293.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
The v2 backup silently dropped the entire rulebook system (rulebooks, topics,
rules), the project subscription/suppression join tables, and events — so a
'full' backup wasn't. v3 adds all of them with FK re-mapping on restore, and a
_not_included field that names the still-deferred tables (ACL groups/shares,
api_keys, embeddings, transient/operational) so the gap is explicit, not silent.
restore_full_backup routes v2 and v3 through one path; v3-only sections are
guarded by data.get so a v2 payload still restores cleanly.
Tests: version/coverage constants, pure join-table row helpers, and the export
contract via a mocked session (CI has no DB; full round-trip is a manual check).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bvandeusenandClaude Opus 5