Files
FabledScribe/tests/test_services_backup.py
T
bvandeusenandClaude Opus 5 84541f392b
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / integration (push) Successful in 24s
CI & Build / TypeScript typecheck (push) Successful in 25s
CI & Build / Python tests (push) Failing after 38s
CI & Build / Build & push image (push) Skipped
fix(backup): six tables were silently absent, and nothing would catch a seventh
services/backup.py enumerated its tables as hand-maintained literals with
nothing tying them to the schema. Tables added since that list was last
extended were absent from every backup — no error, no warning, and a restore
that reports success.

Missing: systems, record_systems (0065), note_usage_events (0071),
design_systems, design_tokens (0072), and repo_bindings — which the issue
itself had not spotted, found only by diffing the model tablenames against the
two lists instead of trusting either.

_NOT_INCLUDED was worse than incomplete: it named "embeddings", "invitations"
and "password_resets", none of which are tables. It read as coverage while
naming nothing the schema could confirm. Now real names, plus retrieval_logs —
observational telemetry that grows per query and that nothing reads for
correctness.

THE DELIVERABLE IS THE GUARD, not the six sections. Extending a list fixes
today and changes nothing about the next table; a new one now fails a test
until someone either backs it up or states that it shouldn't be. It checks
both directions — an unaccounted table, and a listed name that no longer
exists, which is what the three phantom entries above would have tripped.

Design systems need ordering care: parent_id is a self-FK. The export orders
parent-first (parent_id NULLS FIRST, then id — a parent always has the smaller
id), so restore resolves each parent from the map as it goes, with no second
pass. A child whose parent is missing lands as a root rather than failing the
whole restore.

Usage events are kept because pull-through is the evidence base for whether
recall works, and it only ever accumulates — a restore that dropped it would
reset that measurement to zero while everything still looked fine.

BACKUP_VERSION 4 -> 5. Every new restore section is data.get()-guarded, so
v2/v3/v4 payloads restore unchanged.

Closes #2293.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
2026-07-31 23:09:25 -04:00

111 lines
4.0 KiB
Python

"""Unit tests for the v4 backup export contract.
CI runs pytest with no database, so these cover the parts that don't need one:
the version/coverage constants, the pure join-table row helpers, and the export
dict shape (via a mocked session). Full FK-remapping round-trip is exercised
manually against a real DB (export a backup, confirm rulebooks appear).
"""
from types import SimpleNamespace
from unittest.mock import patch
import pytest
from scribe.services import backup
def test_backup_version_is_v5():
assert backup.BACKUP_VERSION == 5
def test_not_included_lists_the_known_gaps():
# The deferred tables must be surfaced explicitly, not silently dropped.
for table in ("groups", "project_shares", "note_shares", "api_keys",
"note_embeddings", "retrieval_logs"):
assert table in backup._NOT_INCLUDED
def test_every_table_is_either_backed_up_or_explicitly_excluded():
"""THE GUARD (#2293), and the only shape of test that catches an ABSENCE.
A new table gets a model and a migration — both fail loudly if wrong — and
then silently never gets a backup section. No error, no warning, and a
restore that reports success. That is how `systems`, `record_systems`,
`note_usage_events`, `design_systems`, `design_tokens` and `repo_bindings`
all went missing, over five migrations, with nothing to notice.
Extending the export fixes today. THIS fixes the next one: adding a table
now fails here until someone either backs it up or states in
`_NOT_INCLUDED` that it shouldn't be. Either is fine; silence is not.
"""
from scribe.models import Base
schema = set(Base.metadata.tables)
accounted = set(backup._BACKED_UP) | set(backup._NOT_INCLUDED)
unaccounted = schema - accounted
assert not unaccounted, (
f"{len(unaccounted)} table(s) are neither backed up nor explicitly "
f"excluded: {sorted(unaccounted)}. Add each to backup._BACKED_UP (and "
f"give it an export + restore section) or to backup._NOT_INCLUDED with "
f"a reason in the comment above it."
)
# And the reverse: a name in either list that no longer exists is a lie the
# guard would otherwise keep telling. This half is what caught "embeddings",
# "invitations" and "password_resets" — three entries that named nothing.
phantom = accounted - schema
assert not phantom, (
f"backup lists table(s) that are not in the schema: {sorted(phantom)}. "
f"Renamed or dropped — fix the list rather than leaving it to read as "
f"coverage."
)
def test_join_table_row_helpers_are_pure():
subs = [SimpleNamespace(project_id=1, rulebook_id=2)]
rsup = [SimpleNamespace(project_id=1, rule_id=9)]
tsup = [SimpleNamespace(project_id=1, topic_id=7)]
assert backup._subscription_rows(subs) == [{"project_id": 1, "rulebook_id": 2}]
assert backup._rule_suppression_rows(rsup) == [{"project_id": 1, "rule_id": 9}]
assert backup._topic_suppression_rows(tsup) == [{"project_id": 1, "topic_id": 7}]
class _Result:
def scalars(self):
return self
def all(self):
return []
class _Session:
async def execute(self, *a, **k):
return _Result()
async def get(self, *a, **k):
return None
class _CM:
async def __aenter__(self):
return _Session()
async def __aexit__(self, *a):
return False
@pytest.mark.asyncio
async def test_export_full_backup_contains_v3_sections():
with patch("scribe.services.backup.async_session", lambda: _CM()):
out = await backup.export_full_backup()
assert out["version"] == 4
assert out["scope"] == "full"
assert "api_keys" in out["_not_included"]
# The sections v2 silently dropped must now be present (empty here).
for key in ("rulebooks", "rulebook_topics", "rules",
"rulebook_subscriptions", "rule_suppressions",
"topic_suppressions"):
assert key in out, f"missing v3 section: {key}"
assert out[key] == []