Commit Graph
8 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 e445c83e67 feat(backup): restore at first-run setup, restore --in-place, docs lead with the UI (#5818, #5819)
CI / integration (push) Successful in 1m28s
CI / unit (push) Successful in 1m37s
CI / publish (push) Successful in 1m17s
CI / lint (push) Successful in 2s
#5818 "First-run setup: Restore from a backup, by upload": beside creating
the first admin, setup offers "Or restore from a backup". Only while no
user exists; the upload is checked (a newer backup or one whose key can't
open its secrets is refused and not kept), then restored with no safety
backup (nothing to keep), and Steward restarts into the backup's users.

#5819 "Docs and CI": backups.md leads with restoring from Settings →
Backups and from setup; `steward restore --in-place` does the same-schema
restore from the shell, and CI's image check now uses it with a real
pg_restore: a changed setting comes back and the audit log stays.

Also exempts errors/restoring.html from the breadcrumb guard, as with
the other whole-app error pages: it replaces every page while a restore
runs, so there is nowhere to trail back to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:05:39 -04:00
bvandeusenandClaude Opus 5.5 fb99f8b551 feat(backup): restore from Settings → Backups, with restoring mode (#5816, #5817)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m27s
CI / unit (push) Failing after 1m37s
CI / publish (push) Skipped
#5816 "Restoring mode": while a backup goes back in, the scheduler holds,
ingest and HTMX answer 503 with Retry-After (agents keep their samples),
and every page says what is happening. A safety backup of Steward as it
stands is made first, to /data/backups and the bucket. The backup's key
is put in place, the restore audited, and Steward exits with code 3 so
Swarm, compose or systemd starts it fresh. A failed in-place restore
changes nothing and says why on Backups; a full replace that fails after
the wipe restarts into first-run setup.

#5817 "Settings → Backups: Restore": a Restore button on each sent backup
and on the file card. The chosen backup is checked (the same comparison
as Check, now a shared partial) and held server-side for an hour under a
token; it starts only once the admin types "restore", and an older
backup also needs "Replace everything" ticked. A newer backup, or one
whose key can't open its secrets, gets no button.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:03:01 -04:00
bvandeusenandClaude Opus 5.5 e8b75143f1 feat(backup): check a backup against what is in place, changing nothing (#5823)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m23s
CI / unit (push) Successful in 1m37s
CI / publish (push) Successful in 1m8s
#5823 "Check a backup: compare what it would restore with what is in
place, without restoring".

- steward/backup/check.py: pg_restore writes the backup's rows as COPY
  text with no database involved. The rows in place go out through COPY in
  the same form. Tables are compared on their primary keys, over the columns
  both sides have, and grouped by area.
- Sealed values are opened with each side's own key and compared as
  plaintext digests, so a secret sealed twice still matches. They are never
  shown. A backup whose key can't open its own secrets fails the check.
- What Steward records as it runs (last-seen times, a schedule's last run,
  backup history, scraped UniFi fields) is left out of the comparison:
  contents.ACTIVITY, STATE_SETTINGS and HAND_SET_ONLY.
- Settings → Backups:
  - a Check button on each sent backup, which fetches it from the bucket
    (new s3.get_file);
  - "Check a file" to upload one, with the size limit raised before the
    CSRF check reads the form;
  - a results page.
- Each backup Steward sends checks itself once written, and its history
  row says how that went.
- New CLI command `steward check-backup FILE`. CI's image smoke step checks a
  real backup against its source and expects a match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 10:48:48 -04:00
bvandeusenandClaude Opus 5.5 e6da0ff299 feat(backup): a backup holds settings and keys only, for recovery (#5822)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m25s
CI / unit (push) Successful in 1m36s
CI / publish (push) Successful in 1m10s
#5822 "Backup contents: settings and keys only". The operator's ruling
(System 180, 2026-10-15) overturns 2026-10-12's "Config + audit + alerts":
a backup is for system recovery and carries what would otherwise have to
be set up again, with no logs or history.

- contents.py: audit_events, alert_events, alert_states, metric_anomalies,
  metric_anomaly_spans and user_invites move to EXCLUDED.
- Alert state is no longer restored, and evaluation skips a rule with no
  state row. ensure_rule_states (alerts/service.py) gives every
  single-resource rule lacking one an inactive row; alert_reconcile runs
  it each minute.
- Docs, the Backups page and the CLI help say what a backup holds now.
  CI's image check also finds the audit log empty after a restore.
- tests/integration/test_backup_restore_fixes.py is removed: its case (an
  alert event naming a left-out Ansible run) no longer arises. The SET NULL
  fix-up is covered by a unit test on synthetic tables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 10:39:55 -04:00
bvandeusenandClaude Opus 5.5 357bb6480d fix(backup): restore finds plugin tables by path, as the app loads them (#5674)
Run 9075: the restore got through schema and data, then all_metadata's
`import plugins` failed in the image, where plugins/ is a directory the app
loads by path (PLUGIN_DIRS), not an installed package. It now loads each
plugin under the loader's own module name from the bootstrap's plugin
dirs, which `steward restore` passes. A plugin whose models are already
mapped under either name is skipped rather than mapped twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:23:58 -04:00
bvandeusenandClaude Opus 5.5 b875160c40 feat(backup): steward backup and steward restore; CI restores a dump and boots on it (#5672, #5674)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m14s
CI / unit (push) Failing after 1m28s
CI / publish (push) Skipped
The backup file (#5672 "Backup file: pg_dump of config + audit + alert
history, the secret key and a manifest"): steward-<UTC>.tar holding a
pg_dump --format=custom with metrics', check results' and Docker/Traefik/
UniFi rows left out, the secret key, and a manifest. Every table is named
KEPT or EXCLUDED in steward/backup/contents.py, and a guard test fails on a
new one. The image gains postgresql-client-17 (approved, System 180).

The restore (#5674 "Restore: `steward restore` and a CI test that a dump
brings Steward back"): pg_restore in its three sections. Between data and
constraints, a kept row pointing at a left-out row gets what its foreign
key's ON DELETE says (alert_events.ansible_run_id is cleared). It refuses
a non-empty database unless --wipe, and puts the backup's key in
/data/secret.key. Migrations run on Steward's next start.

`steward` stays the server command: it is now a click group that serves
when given no subcommand.

The publish smoke step backs up the booted image's database, restores it
into a second one, checks a marker setting came back and plugin_metrics
is empty, and starts Steward on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:14:04 -04:00
bvandeusenandClaude Opus 5.5 98854d94f9 feat(platform): tini as PID 1, Hypercorn, a log-level setting, the image on CI's Python, an airgapped boot gate before push (#5159)
CI / lint (push) Successful in 4s
CI / integration (push) Successful in 1m11s
CI / unit (push) Successful in 1m31s
CI / publish (push) Successful in 2m34s
- Dockerfile: python:3.14-slim, the Python CI tests on (was 3.13). tini from
  apt is PID 1 (ENTRYPOINT tini -- entrypoint.sh): it reaps the zombies ssh
  and ansible leave and forwards SIGTERM.
- cli: Hypercorn's serve() with one worker (run, stream and scheduler state is
  in-process), a 10 s graceful timeout, and SIGTERM/SIGINT as the shutdown
  trigger. Access and error lines go through Steward's logging. --debug keeps
  Quart's reloading dev server.
- Logging: steward/core/logging_setup.py installs one root handler before
  anything logs (until now a library's basicConfig on import was the only
  configuration). Its level is the `log.level` setting (DEBUG/INFO/WARNING/
  ERROR, default INFO), applied at boot and on save in Settings → General.
- CI publish: build, then "The image boots with no internet" (rule 164,
  snippet #3552), then push, so a broken image never reaches a tag (rule 177).
  On an --internal network with its own Postgres, the image must migrate and
  answer /health, run tini as PID 1, serve first-run setup with its assets
  from /static and no third-party host, and exit 0 on SIGTERM. Logs and
  clean-up run always.

Tests: tests/core/test_logging_setup.py; the smoke step itself.
Not in this commit: the dependency lockfile (it conflicts with rule 154 and
is waiting on the operator).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 21:29:21 -04:00
bvandeusenandClaude Opus 4.7 88ab5b917e chore: rename project Roundtable → Steward
Renames the Python package directory, CLI command, env var prefix,
docker-compose service/container/image, Postgres role/db, and all
visible branding. Marketing form is "Fabled Steward".

Clean break from the previous rebrand: drops the fabledscryer→roundtable
import shim in __init__.py and the FABLEDSCRYER_* env var fallback in
config.py and migrations/env.py. Env vars are now STEWARD_* only.

Heads-up for existing deployments:
- Postgres user/db renamed fabledscryer → steward in docker-compose.yml.
  Existing volumes need the role/db renamed inside Postgres, or override
  POSTGRES_USER/POSTGRES_DB to keep the old names.
- Host-agent systemd unit is now steward-agent.service. Existing agents
  keep running under the old name; reinstall to switch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 16:20:14 -04:00