Settings → Maintenance → Backups gains "Restore from the bucket": pick one of
the bucket's dated dumps, type its confirmation, and FabledCurator saves the
current database to the bucket, replaces it with the dump, migrates it to this
version, then fetches every image and attachment back to the path its row
records. Each file is checked against its sha256 on arrival and moved into
place only then; a file already on disk is left alone, so "Fetch missing
files" repairs a library that lost files without touching the database.
- services/backup_restore.py: list/find/fetch dumps, migrate, keep this
install's destination (the dump's sealed secret can't be opened here),
library_files work list, restore_file (one fetch per sha, copies for
duplicate paths, refuses paths outside /images).
- restore_offsite_task: the database phase, then 45-minute file chunks with a
heartbeat; a failed load leaves an empty migrated database and says why.
The dump's own run row is marked ok, other runs then going "Interrupted".
- backup_s3.get_file: sha/size check on arrival, ObjectMissing for a 404,
download pacing.
- backup_service: every dump leaves the credential table's rows out;
restore_db reports pg_restore's last line, stops at the first error, ends
other connections and waits at most 60 s for locks.
- Backups and orphan repair wait while a restore runs; the stall sweep judges
restores by heartbeat; restore runs are pruned to 20.
- CLI: python -m backend.app.scripts.restore list|database|files.
- README: backup and restore section; logins are in no backup.
- Tests: a round trip through an in-process S3 bucket (tests/fake_s3.py) with
real pg_dump/pg_restore, plus file, run, API and get_file cases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The nightly database backup (and "Run DB backup now") now sends pg_dump's
output straight to the off-site bucket as a multipart upload, with no copy
staged on this server. The object is named database/fabledcurator-<UTC
time>.dump and appears only once pg_dump has exited cleanly and every part
has landed; a failed dump aborts the upload and records pg_dump's last line.
Without a destination, dumps are written to /images/_backups as before.
- backup_s3: Upload (start / add part / complete / abort), with a busy or
silent bucket rested and asked again (15 s, 60 s, Retry-After obeyed), and
a Pacer that holds uploads to the new bandwidth limit.
- Settings: "Upload limit (Mbit/s)" on the destination card, 200 by default,
0 for none (migration 0120).
- pg_dump runs under nice (and ionice where installed), locally too.
- Backups card says where dumps go; the runs table gains "Saved to" and
shows a failed run's error. Off-site rows have no restore/tag/delete here:
restoring from the bucket is a later step (#6880).
Task #6878, milestone 586.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings → Maintenance → Storage gains "Off-site backup destination": the
address (with its port), bucket, region, access key and secret key of an S3
bucket such as Fabled Archives. The secret key is sealed with the credential
key, never returned by the API, and kept when the field is left blank. Test
destination lists the bucket with the fields as typed and writes nothing, and
names the cause when it fails ("nothing is listening there (check the port)",
"the secret key is wrong", "answered, but not as S3").
services/backup_s3 is a standard-library SigV4 client (put, get, paged list)
ported from Grimoire's; it is held to AWS's published signing examples and
exercised against a local HTTP server. Migration 0118 adds the columns; no
destination is set up until someone saves one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Removed the app/client fixtures duplicated across 36 test files (two
variants: separate app + client(app), and a self-contained client() that
called create_app inline) and the now-unused create_app imports. Both
fixtures now live once in conftest.py. test_suggestions_bulk keeps its
import (builds the app inline in two tests); test_health drops its local
client + unused pytest_asyncio.
Net -415 lines.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bvandeusenandClaude Opus 5.5