4 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 fb99f8b551 feat(backup): restore from Settings → Backups, with restoring mode (#5816, #5817)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m27s
CI / unit (push) Failing after 1m37s
CI / publish (push) Skipped
#5816 "Restoring mode": while a backup goes back in, the scheduler holds,
ingest and HTMX answer 503 with Retry-After (agents keep their samples),
and every page says what is happening. A safety backup of Steward as it
stands is made first, to /data/backups and the bucket. The backup's key
is put in place, the restore audited, and Steward exits with code 3 so
Swarm, compose or systemd starts it fresh. A failed in-place restore
changes nothing and says why on Backups; a full replace that fails after
the wipe restarts into first-run setup.

#5817 "Settings → Backups: Restore": a Restore button on each sent backup
and on the file card. The chosen backup is checked (the same comparison
as Check, now a shared partial) and held server-side for an hour under a
token; it starts only once the admin types "restore", and an older
backup also needs "Replace everything" ticked. A newer backup, or one
whose key can't open its secrets, gets no button.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:03:01 -04:00
bvandeusenandClaude Opus 5.5 e8b75143f1 feat(backup): check a backup against what is in place, changing nothing (#5823)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m23s
CI / unit (push) Successful in 1m37s
CI / publish (push) Successful in 1m8s
#5823 "Check a backup: compare what it would restore with what is in
place, without restoring".

- steward/backup/check.py: pg_restore writes the backup's rows as COPY
  text with no database involved. The rows in place go out through COPY in
  the same form. Tables are compared on their primary keys, over the columns
  both sides have, and grouped by area.
- Sealed values are opened with each side's own key and compared as
  plaintext digests, so a secret sealed twice still matches. They are never
  shown. A backup whose key can't open its own secrets fails the check.
- What Steward records as it runs (last-seen times, a schedule's last run,
  backup history, scraped UniFi fields) is left out of the comparison:
  contents.ACTIVITY, STATE_SETTINGS and HAND_SET_ONLY.
- Settings → Backups:
  - a Check button on each sent backup, which fetches it from the bucket
    (new s3.get_file);
  - "Check a file" to upload one, with the size limit raised before the
    CSRF check reads the form;
  - a results page.
- Each backup Steward sends checks itself once written, and its history
  row says how that went.
- New CLI command `steward check-backup FILE`. CI's image smoke step checks a
  real backup against its source and expects a match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 10:48:48 -04:00
bvandeusenandClaude Opus 5.5 e6da0ff299 feat(backup): a backup holds settings and keys only, for recovery (#5822)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m25s
CI / unit (push) Successful in 1m36s
CI / publish (push) Successful in 1m10s
#5822 "Backup contents: settings and keys only". The operator's ruling
(System 180, 2026-10-15) overturns 2026-10-12's "Config + audit + alerts":
a backup is for system recovery and carries what would otherwise have to
be set up again, with no logs or history.

- contents.py: audit_events, alert_events, alert_states, metric_anomalies,
  metric_anomaly_spans and user_invites move to EXCLUDED.
- Alert state is no longer restored, and evaluation skips a rule with no
  state row. ensure_rule_states (alerts/service.py) gives every
  single-resource rule lacking one an inactive row; alert_reconcile runs
  it each minute.
- Docs, the Backups page and the CLI help say what a backup holds now.
  CI's image check also finds the audit log empty after a restore.
- tests/integration/test_backup_restore_fixes.py is removed: its case (an
  alert event naming a left-out Ansible run) no longer arises. The SET NULL
  fix-up is covered by a unit test on synthetic tables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 10:39:55 -04:00
bvandeusenandClaude Opus 5.5 49799bd0b3 feat(backup): daily backups to an S3 bucket, Settings → Backups, and the guide (#5673, #5675)
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 1m15s
CI / unit (push) Failing after 1m28s
CI / publish (push) Skipped
#5673 "S3 client and the backup job: settings, schedule, history, metrics":
- steward/backup/s3.py: SigV4 over httpx, path-style; put and list only
  (an append-only key is enough; retention is the bucket's). Signing is held
  to AWS's published examples.
- steward/backup/job.py: a 15-minute tick. The daily backup goes at the hour
  chosen on the reader's clock; a missed day is made up, and a failure is
  retried an hour later. History of the last 30 runs, audit
  (backup.made / backup.failed), and backup_age_secs / backup_bytes under
  "Steward itself", resource backup, for alert rules.
- Settings → Backups: verdict first, then the bucket (secret sealed; a
  blank box keeps it), the schedule, Test the bucket, Back up now, and the
  recent runs.

#5675 "Docs: backing up to Archives, and restoring": docs/core/backups.md.

Also: the audit page now words the Prometheus target actions from #5668,
which showed as codes, and a guard fails on any logged action without
wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:23:58 -04:00