feat: restore a new install from the off-site bucket, database first, then every file
CI and images / lint (push) Successful in 2s
CI and images / extension-version (push) Successful in 1s
CI and images / extension-test (push) Successful in 17s
CI and images / frontend-build (push) Successful in 19s
CI and images / backend-lint-and-test (push) Successful in 45s
CI and images / integration (push) Failing after 2m26s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / refresh-dev (push) Skipped
CI and images / unattended-failure (push) Skipped
CI and images / build-agent (push) Skipped
CI and images / lint (push) Successful in 2s
CI and images / extension-version (push) Successful in 1s
CI and images / extension-test (push) Successful in 17s
CI and images / frontend-build (push) Successful in 19s
CI and images / backend-lint-and-test (push) Successful in 45s
CI and images / integration (push) Failing after 2m26s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / refresh-dev (push) Skipped
CI and images / unattended-failure (push) Skipped
CI and images / build-agent (push) Skipped
Settings → Maintenance → Backups gains "Restore from the bucket": pick one of the bucket's dated dumps, type its confirmation, and FabledCurator saves the current database to the bucket, replaces it with the dump, migrates it to this version, then fetches every image and attachment back to the path its row records. Each file is checked against its sha256 on arrival and moved into place only then; a file already on disk is left alone, so "Fetch missing files" repairs a library that lost files without touching the database. - services/backup_restore.py: list/find/fetch dumps, migrate, keep this install's destination (the dump's sealed secret can't be opened here), library_files work list, restore_file (one fetch per sha, copies for duplicate paths, refuses paths outside /images). - restore_offsite_task: the database phase, then 45-minute file chunks with a heartbeat; a failed load leaves an empty migrated database and says why. The dump's own run row is marked ok, other runs then going "Interrupted". - backup_s3.get_file: sha/size check on arrival, ObjectMissing for a 404, download pacing. - backup_service: every dump leaves the credential table's rows out; restore_db reports pg_restore's last line, stops at the first error, ends other connections and waits at most 60 s for locks. - Backups and orphan repair wait while a restore runs; the stall sweep judges restores by heartbeat; restore runs are pruned to 20. - CLI: python -m backend.app.scripts.restore list|database|files. - README: backup and restore section; logins are in no backup. - Tests: a round trip through an in-process S3 bucket (tests/fake_s3.py) with real pg_dump/pg_restore, plus file, run, API and get_file cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -114,9 +114,10 @@ that cannot decrypt anything, leaving an instance that looks healthy while every
|
||||
paywalled download fails. Making you say so once, on an empty install, is the
|
||||
price of that not happening silently later.
|
||||
|
||||
**Which means: back up `./images/secrets/` alongside your database.** It is the
|
||||
only thing that can read your stored credentials. A database restored without it
|
||||
needs every credential entered again by hand.
|
||||
**Your account logins are sealed with that key, and no backup carries either.**
|
||||
FabledCurator's backups (below) leave the key and the logins out on purpose, so
|
||||
a stolen backup holds no way into your accounts. After a restore, sign in to
|
||||
each account again through the browser extension.
|
||||
|
||||
A few other things are worth knowing about the first few minutes:
|
||||
|
||||
@@ -177,6 +178,37 @@ Run it for a burst of tagging, stop it to get your card back. It deploys from
|
||||
`agent/docker-compose.yml`, not the main stack — see
|
||||
[agent/README.md](agent/README.md).
|
||||
|
||||
## Backing up and restoring
|
||||
|
||||
Backups live under **Settings → Maintenance → Backups**. With nothing else set
|
||||
up, the database is dumped nightly into `./images/_backups`, which is on the
|
||||
same disk as what it protects.
|
||||
|
||||
To keep a copy off the machine, set up an off-site destination on the same
|
||||
screen: any S3 bucket (Fabled Archives, MinIO, AWS) and a key that may add and
|
||||
read objects. Each night FabledCurator then sends the database as a dated dump
|
||||
under `database/`, and every image and attachment once, named by its
|
||||
fingerprint, under `media/`. It never deletes from the bucket; the bucket's own
|
||||
retention thins old dumps.
|
||||
|
||||
To rebuild after losing the server, start a fresh install (first boot as
|
||||
above), enter the same destination, and under **Restore from the bucket** choose
|
||||
a dump. The database is replaced, then every file is fetched back to its place
|
||||
and checked against its fingerprint as it arrives; progress shows on the same
|
||||
screen. **Fetch missing files** does only the second half, for a library that
|
||||
lost files but kept its database.
|
||||
|
||||
If the web page isn't reachable, the same restore runs from a shell:
|
||||
|
||||
```bash
|
||||
docker compose exec web python -m backend.app.scripts.restore list
|
||||
docker compose exec web python -m backend.app.scripts.restore database --dump latest --yes
|
||||
```
|
||||
|
||||
On an install with no destination saved yet, add `--endpoint`, `--bucket` and
|
||||
`--access-key`, and put the secret key in `FC_BACKUP_SECRET_KEY` (or type it at
|
||||
the prompt).
|
||||
|
||||
## Upgrading
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user