feat(backup): daily backups to an S3 bucket, Settings → Backups, and the guide (#5673, #5675)

#5673 "S3 client and the backup job: settings, schedule, history, metrics":
- steward/backup/s3.py: SigV4 over httpx, path-style; put and list only
  (an append-only key is enough; retention is the bucket's). Signing is held
  to AWS's published examples.
- steward/backup/job.py: a 15-minute tick. The daily backup goes at the hour
  chosen on the reader's clock; a missed day is made up, and a failure is
  retried an hour later. History of the last 30 runs, audit
  (backup.made / backup.failed), and backup_age_secs / backup_bytes under
  "Steward itself", resource backup, for alert rules.
- Settings → Backups: verdict first, then the bucket (secret sealed; a
  blank box keeps it), the schedule, Test the bucket, Back up now, and the
  recent runs.

#5675 "Docs: backing up to Archives, and restoring": docs/core/backups.md.

Also: the audit page now words the Prometheus target actions from #5668,
which showed as codes, and a guard fails on any logged action without
wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/core/backups.md

@@ -0,0 +1,117 @@
+# Backups: Steward backing itself up, and coming back
+
+Steward sends a backup of itself to an S3 bucket once a day. Fabled Archives
+is the intended target. A backup is one file, and `steward restore` brings
+Steward back from it into an empty database.
+
+## What a backup holds
+
+`steward-<UTC time>.tar`, containing:
+
+| Member | What it is |
+|---|---|
+| `steward.dump` | `pg_dump --format=custom` of Steward's database |
+| `secret.key` | The key Steward's stored passwords and tokens are sealed with |
+| `manifest.json` | Format, Steward version, migration heads, time, and the tables whose rows were left out |
+
+**Kept:**
+- configuration: settings, users, API tokens, hosts, monitors, alert rules, maintenance windows, Ansible inventory and schedules, dashboards, SNMP devices, Prometheus targets, UniFi client names;
+- the audit log;
+- alert history.
+
+**Left out** (the tables come back empty and fill again as Steward runs):
+- metrics and their rollups;
+- monitor check results;
+- Docker, Traefik and UniFi history;
+- Ansible run records;
+- the email queue;
+- password-reset links.
+
+The list is `steward/backup/contents.py`. A test fails when a new table
+appears in neither list, so whoever adds a table decides.
+
+**The secret key is in the file.** That makes a restore complete on its own.
+It also means anyone holding a copy can read every password and key Steward
+keeps. Keep the bucket private.
+
+## Setting it up with Fabled Archives
+
+1. In Archives, make a backup target for Steward with an S3 key. The default
+   append-only key is enough: Steward only uploads and never deletes.
+   How long backups are kept is set on the Archives side.
+2. In Steward, open **Settings → Backups** and fill in:
+   - **Endpoint**: the S3 address Archives shows for the target, without the
+     bucket (for example `http://archives:8081`). Requests are path-style:
+     `endpoint/bucket/file`.
+   - **Bucket**: the target's bucket name.
+   - **Access key** and **Secret key**: from Archives. The secret is sealed at
+     rest; a blank box on save keeps the stored one.
+3. Press **Test the bucket**. A key that may not list ("AccessDenied") still
+   counts as reached: an append-only key can't list, and uploads work anyway.
+4. Tick **Back up every day**, pick the hour (on your clock), and save.
+   **Back up now** makes one straight away.
+
+A day missed while Steward was down is made up when it comes back. A failed
+backup is tried again an hour later, never sooner. The last 30 runs are
+listed on the page, and each one is in the audit log (`backup.made` /
+`backup.failed`).
+
+### Alert when backups stop
+
+While backups are on, Steward records two metrics under the source
+**Steward itself**, resource `backup`:
+
+| Metric | Meaning |
+|---|---|
+| `backup_age_secs` ("Newest backup's age") | Seconds since the last backup was sent, updated every 15 minutes |
+| `backup_bytes` ("Backup size") | The size of the last backup sent |
+
+A rule such as `backup_age_secs > 93600` (26 hours) fires when a daily backup
+has been missed.
+
+## Making a backup by hand
+
+```sh
+docker exec -u app <steward container> steward backup --to /tmp/backups
+```
+
+This writes the file and sends it nowhere. It needs the same database access
+as Steward itself.
+
+## Restoring
+
+`steward restore` writes into an **empty** database. Pass `--wipe` to drop
+everything in the database's public schema first.
+
+1. Stop Steward (scale its service to 0) so nothing writes during the restore.
+2. Download the backup file from Archives.
+3. Run the restore with the image, pointed at the database:
+
+   ```sh
+   docker run --rm \
+     -e STEWARD_DATABASE_URL=postgresql+asyncpg://steward:…@db:5432/steward \
+     -v steward_data:/data -v "$PWD":/restore \
+     git.fabledsword.com/bvandeusen/steward:latest \
+     steward restore /restore/steward-20261012T030000Z.tar
+   ```
+
+   - The backup's secret key is written to `/data/secret.key`.
+   - If `STEWARD_SECRET_KEY` is set instead, it must equal the backup's key,
+     or the restore stops and says so. Read the key with
+     `tar -xOf <file> secret.key`.
+4. Start Steward. On start it runs any migrations the backup is behind on.
+
+**How the restore works.** pg_restore runs in its three sections: schema,
+then data, then constraints and indexes. Between the data and the
+constraints, a kept row that points at a left-out row is put right by its
+foreign key's ON DELETE:
+- an alert event's link to an Ansible run is cleared;
+- a row whose foreign key would cascade or block is removed.
+
+pg_restore 17 writes each section as a script for psql. It runs against
+Postgres 16 with one setting line dropped (`transaction_timeout`, which 16
+doesn't know).
+
+CI checks the whole path on every build. The built image backs up its own
+database, restores into a second one, checks a setting came back and the
+metrics table is empty, then starts Steward on it.