feat(backup): restore at first-run setup, restore --in-place, docs lead with the UI (#5818, #5819)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m28s
CI / unit (push) Successful in 1m37s
CI / publish (push) Successful in 1m17s

#5818 "First-run setup: Restore from a backup, by upload": beside creating
the first admin, setup offers "Or restore from a backup". Only while no
user exists; the upload is checked (a newer backup or one whose key can't
open its secrets is refused and not kept), then restored with no safety
backup (nothing to keep), and Steward restarts into the backup's users.

#5819 "Docs and CI": backups.md leads with restoring from Settings →
Backups and from setup; `steward restore --in-place` does the same-schema
restore from the shell, and CI's image check now uses it with a real
pg_restore: a changed setting comes back and the audit log stays.

Also exempts errors/restoring.html from the breadcrumb guard, as with
the other whole-app error pages: it replaces every page while a restore
runs, so there is nowhere to trail back to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-15 11:05:39 -04:00
co-authored by Claude Opus 5.5
parent fb99f8b551
commit e445c83e67
9 changed files with 225 additions and 19 deletions
+57 -5
View File
@@ -1,8 +1,9 @@
# Backups: Steward backing itself up, and coming back
Steward sends a backup of itself to an S3 bucket once a day. Fabled Archives
is the intended target. A backup is one file, and `steward restore` brings
Steward back from it into an empty database.
is the intended target. A backup is one file. Settings → Backups restores it
on a running Steward, first-run setup restores it on a new one, and
`steward restore` brings Steward back from it when Steward can't start.
## What a backup holds
@@ -126,8 +127,57 @@ as Steward itself.
## Restoring
### From the web UI
**On a running Steward: Settings → Backups.** Press **Restore** on one of
the recent backups, or choose a file on the "Check or restore a file" card.
Steward first shows what the backup would change: the same comparison as
Check. Then it says what a restore does and asks you to type `restore`.
- **A backup of this Steward's schema goes back in place.** Settings,
users, hosts, monitors, alert rules and dashboards become the backup's.
The audit log and the history of everything still here stay. History of
anything the backup doesn't have goes with it. It is one transaction: if
anything fails part-way, nothing changes, and Backups says why.
- **A backup from an older Steward replaces everything**, history included,
and you tick "Replace everything" as well. Steward brings it up to date
as it starts.
- **A backup from a newer Steward isn't offered.** Update Steward first.
Before anything changes, Steward makes a **safety backup** of itself as it
stands. It goes to `/data/backups` (the newest five are kept) and to the
bucket, and it is listed under Recent backups as "before restore". A
restore can be undone by restoring that.
While the restore runs:
- every page says Steward is restoring;
- agents get a 503 and keep their readings until Steward is back;
- scheduled work holds.
Then Steward restarts (it exits with code 3 so Swarm, compose or systemd
starts it again), and you sign in as a user from the backup. If an older
backup's full replace fails after the database was emptied, Steward starts
at first-run setup, where the safety backup can be uploaded.
**On a new Steward: first-run setup.** Next to "Create the initial admin
account" is **Or restore from a backup**: upload a backup file instead,
and Steward comes up with its users, settings and key. There is nothing to
keep yet, so no safety backup is made.
The backup's secret key is put in place as `/data/secret.key`. If
`STEWARD_SECRET_KEY` is set instead, it must equal the backup's key, or the
restore stops before anything changes.
### From the command line
`steward restore` writes into an **empty** database. Pass `--wipe` to drop
everything in the database's public schema first.
everything in the database's public schema first. Use it when Steward
can't start at all.
`steward restore --in-place` does what Settings → Backups does for a backup
of the same schema, from the shell: it replaces what the backup carries and
keeps the audit log and history, in one transaction. Restart Steward after
it so the restored settings load.
1. Stop Steward (scale its service to 0) so nothing writes during the restore.
2. Download the backup file from Archives.
@@ -159,5 +209,7 @@ Postgres 16 with one setting line dropped (`transaction_timeout`, which 16
doesn't know).
CI checks the whole path on every build. The built image backs up its own
database, restores into a second one, checks a setting came back and the
metrics table is empty, then starts Steward on it.
database and checks the backup against it. It changes a setting and puts
the backup back in place: the setting returns and the audit log stays.
Then it restores into a second, empty database, checks a setting came back
and the metrics table and audit log are empty, and starts Steward on it.