feat(backup): restore from Settings → Backups, with restoring mode (#5816, #5817)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m27s
CI / unit (push) Failing after 1m37s
CI / publish (push) Skipped

#5816 "Restoring mode": while a backup goes back in, the scheduler holds,
ingest and HTMX answer 503 with Retry-After (agents keep their samples),
and every page says what is happening. A safety backup of Steward as it
stands is made first, to /data/backups and the bucket. The backup's key
is put in place, the restore audited, and Steward exits with code 3 so
Swarm, compose or systemd starts it fresh. A failed in-place restore
changes nothing and says why on Backups; a full replace that fails after
the wipe restarts into first-run setup.

#5817 "Settings → Backups: Restore": a Restore button on each sent backup
and on the file card. The chosen backup is checked (the same comparison
as Check, now a shared partial) and held server-side for an hour under a
token; it starts only once the admin types "restore", and an older
backup also needs "Replace everything" ticked. A newer backup, or one
whose key can't open its secrets, gets no button.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-15 11:03:01 -04:00
co-authored by Claude Opus 5.5
parent 9e5f6e183a
commit fb99f8b551
14 changed files with 858 additions and 72 deletions
+18
View File
@@ -0,0 +1,18 @@
{# errors/restoring.html — every page while a backup is being restored (#5816):
what is going back in, by whom, the step it is on. It reloads itself, so
the sign-in page appears once Steward is back. #}
{% extends "base.html" %}
{% block title %}Restoring a backup — Steward{% endblock %}
{% block head %}<meta http-equiv="refresh" content="5">{% endblock %}
{% block content %}
<div style="text-align:center; padding: 4rem 2rem;">
<h1 class="tone-warn" style="font-size: 1.6rem; margin-bottom: 0.75rem;">Steward is restoring a backup</h1>
<p style="margin-bottom: 0.5rem;">{{ state.name }}{% if state.by %}, started by {{ state.by }}{% endif %}
{%- if state.since %} {{ state.since|ago }}{% endif %}.</p>
<p class="tone-dim" style="margin-bottom: 1rem;">{{ state.step }}…</p>
<p class="field-hint" style="max-width: 34rem; margin: 0 auto;">
Nothing else reads or writes Steward's records until it is done. Agents keep their readings
and send them when Steward is back. This page reloads by itself.
</p>
</div>
{% endblock %}
@@ -0,0 +1,61 @@
{# settings/_backup_report.html — what a check found (#5823), shown on the check
page and on the restore confirm page (#5817): the verdict, what the backup
is, each area's count, then each difference. Needs `report`. #}
{% set tone = "tone-crit" if not report.ok else ("tone-ok" if not report.changes else "tone-warn") %}
<p class="page-verdict"><span class="{{ tone }}">{{ report.summary() }}</span></p>
<div class="card" style="max-width:640px;margin-bottom:1.25rem;">
<dl class="kv">
<dt>File</dt><dd>{{ report.name }}</dd>
<dt>Made</dt><dd>{% if report.made_at %}{{ report.made_at|when }}{% else %}—{% endif %}
by Steward {{ report.manifest.steward_version or "—" }}</dd>
<dt>Schema</dt><dd>
{%- if report.schema == "same" %}The same as this Steward's.
{%- elif report.schema == "older" %}<span class="tone-warn">Older than this Steward's.</span> A restore would replace everything, and Steward would bring it up to date on start.
{%- else %}<span class="tone-crit">Newer than this Steward's.</span> This Steward can't restore it; update Steward first.{% endif %}</dd>
<dt>Secret key</dt><dd>
{%- if report.key_same %}The one in use here.{% else %}<span class="tone-warn">A different one.</span> A restore would put it in place, and passwords and keys stored since would need entering again.{% endif %}
{%- if report.sealed %} It opens {{ report.sealed - report.unopened }} of the backup's {{ report.sealed }} sealed value{{ "s" if report.sealed != 1 }}.{% endif %}</dd>
</dl>
</div>
<div class="card-flush" style="margin-bottom:1.25rem;">
<table class="table">
<thead><tr><th>Area</th><th>Same</th><th>Differ</th><th>Only in the backup</th><th>Only here</th></tr></thead>
<tbody>
{% for a in report.areas %}
{% set differ = a.tables|sum(attribute="differ", start=[])|length %}
{% set only_b = a.tables|sum(attribute="only_backup", start=[])|length %}
{% set only_h = a.tables|sum(attribute="only_here", start=[])|length %}
<tr>
<td>{{ a.name }}</td>
<td>{{ a.same }}</td>
<td>{% if differ %}<span class="tone-warn">{{ differ }}</span>{% else %}—{% endif %}</td>
<td>{% if only_b %}<span class="tone-warn">{{ only_b }}</span>{% else %}—{% endif %}</td>
<td>{% if only_h %}<span class="tone-warn">{{ only_h }}</span>{% else %}—{% endif %}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% for a in report.areas if a.changes or a.tables|selectattr("note")|list %}
<h2 class="section-title" style="margin:1.5rem 0 0.75rem;">{{ a.name }}</h2>
<ul class="check-list">
{% for t in a.tables %}
{% for d in t.differ %}
<li><strong>{{ d.label }}</strong> differs
{%- if d.secret %} (a sealed value; not shown).
{%- elif d.backup_value is not none or d.here_value is not none %}: the backup has <code>{{ d.backup_value if d.backup_value is not none else "nothing" }}</code>, here it is <code>{{ d.here_value if d.here_value is not none else "nothing" }}</code>.
{%- else %} in {{ d.columns|join(", ") }}.{% endif %}</li>
{% endfor %}
{% for label in t.only_backup %}<li><strong>{{ label }}</strong> is only in the backup: a restore brings it back.</li>{% endfor %}
{% for label in t.only_here %}<li><strong>{{ label }}</strong> is only here: a restore removes it.</li>{% endfor %}
{% if t.note %}<li class="tone-dim">{{ t.table }}: {{ t.note }}</li>{% endif %}
{% endfor %}
</ul>
{% endfor %}
<p class="field-hint" style="margin-top:1rem;">Left out of the comparison: what Steward records as it runs
(when an agent or monitor was last seen, a schedule's last run, the backup history), since it changes by the
minute. A restore still brings those back as they were in the backup.</p>
+1 -58
View File
@@ -11,64 +11,7 @@
{% if error %}
<p class="page-verdict"><span class="tone-crit">{{ name or "The backup" }} couldn't be checked. {{ error }}</span></p>
{% else %}
{% set tone = "tone-crit" if not report.ok else ("tone-ok" if not report.changes else "tone-warn") %}
<p class="page-verdict"><span class="{{ tone }}">{{ report.summary() }}</span></p>
<div class="card" style="max-width:640px;margin-bottom:1.25rem;">
<dl class="kv">
<dt>File</dt><dd>{{ report.name }}</dd>
<dt>Made</dt><dd>{% if report.made_at %}{{ report.made_at|when }}{% else %}—{% endif %}
by Steward {{ report.manifest.steward_version or "—" }}</dd>
<dt>Schema</dt><dd>
{%- if report.schema == "same" %}The same as this Steward's.
{%- elif report.schema == "older" %}<span class="tone-warn">Older than this Steward's.</span> A restore would replace everything, and Steward would bring it up to date on start.
{%- else %}<span class="tone-crit">Newer than this Steward's.</span> This Steward can't restore it; update Steward first.{% endif %}</dd>
<dt>Secret key</dt><dd>
{%- if report.key_same %}The one in use here.{% else %}<span class="tone-warn">A different one.</span> A restore would put it in place, and passwords and keys stored since would need entering again.{% endif %}
{%- if report.sealed %} It opens {{ report.sealed - report.unopened }} of the backup's {{ report.sealed }} sealed value{{ "s" if report.sealed != 1 }}.{% endif %}</dd>
</dl>
</div>
<div class="card-flush" style="margin-bottom:1.25rem;">
<table class="table">
<thead><tr><th>Area</th><th>Same</th><th>Differ</th><th>Only in the backup</th><th>Only here</th></tr></thead>
<tbody>
{% for a in report.areas %}
{% set differ = a.tables|sum(attribute="differ", start=[])|length %}
{% set only_b = a.tables|sum(attribute="only_backup", start=[])|length %}
{% set only_h = a.tables|sum(attribute="only_here", start=[])|length %}
<tr>
<td>{{ a.name }}</td>
<td>{{ a.same }}</td>
<td>{% if differ %}<span class="tone-warn">{{ differ }}</span>{% else %}—{% endif %}</td>
<td>{% if only_b %}<span class="tone-warn">{{ only_b }}</span>{% else %}—{% endif %}</td>
<td>{% if only_h %}<span class="tone-warn">{{ only_h }}</span>{% else %}—{% endif %}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% for a in report.areas if a.changes or a.tables|selectattr("note")|list %}
<h2 class="section-title" style="margin:1.5rem 0 0.75rem;">{{ a.name }}</h2>
<ul class="check-list">
{% for t in a.tables %}
{% for d in t.differ %}
<li><strong>{{ d.label }}</strong> differs
{%- if d.secret %} (a sealed value; not shown).
{%- elif d.backup_value is not none or d.here_value is not none %}: the backup has <code>{{ d.backup_value if d.backup_value is not none else "nothing" }}</code>, here it is <code>{{ d.here_value if d.here_value is not none else "nothing" }}</code>.
{%- else %} in {{ d.columns|join(", ") }}.{% endif %}</li>
{% endfor %}
{% for label in t.only_backup %}<li><strong>{{ label }}</strong> is only in the backup: a restore brings it back.</li>{% endfor %}
{% for label in t.only_here %}<li><strong>{{ label }}</strong> is only here: a restore removes it.</li>{% endfor %}
{% if t.note %}<li class="tone-dim">{{ t.table }}: {{ t.note }}</li>{% endif %}
{% endfor %}
</ul>
{% endfor %}
<p class="field-hint" style="margin-top:1rem;">Left out of the comparison: what Steward records as it runs
(when an agent or monitor was last seen, a schedule's last run, the backup history), since it changes by the
minute. A restore still brings those back as they were in the backup.</p>
{% include "settings/_backup_report.html" %}
{% endif %}
<p style="margin-top:1rem;"><a href="/settings/backups/">Back to Backups</a></p>
{% endblock %}
@@ -0,0 +1,58 @@
{# settings/backup_restore.html — restoring a backup, before anything changes
(#5817): what it would bring back (the same check as Check, #5823), what a
restore does, then a typed confirmation. A backup that can't go in says
why and offers no button. #}
{% extends "base.html" %}
{% from "_macros.html" import crumbs %}
{% from "settings/_tabs.html" import settings_tabs %}
{% block title %}Settings — Backups — Restore — Steward{% endblock %}
{% block section_tabs %}{{ settings_tabs("backups") }}{% endblock %}
{% block breadcrumb %}{{ crumbs([("Settings", "/settings/"), ("Backups", "/settings/backups/"), ("Restore " ~ (name or "a backup"), "")]) }}{% endblock %}
{% block content %}
{% if not report %}
<p class="page-verdict"><span class="tone-crit">{{ name or "The backup" }} can't be restored. {{ error }}</span></p>
{% else %}
{% include "settings/_backup_report.html" %}
<div class="card" style="max-width:640px;margin-top:1.25rem;">
<h2 class="section-title" style="margin-bottom:0.5rem;">Restore {{ name }}</h2>
{% if report.schema == "newer" %}
<p class="tone-crit">This Steward is older than the one that made the backup. Update Steward, then restore it.</p>
{% elif not report.ok %}
<p class="tone-crit">Its key doesn't open all of its secrets, so passwords and keys wouldn't come back. Restore a
different backup.</p>
{% else %}
<ul class="field-hint" style="margin:0 0 1rem 1.1rem;line-height:1.6;">
<li>A safety backup of Steward as it stands is made first, on <code>/data/backups</code> and the bucket, so
this can be undone the same way.</li>
{% if report.schema == "same" %}
<li>Settings, users, hosts, monitors, rules and dashboards become the backup's. The audit log and the history
of what is still here stay; history of anything the backup doesn't have goes with it.</li>
<li>If anything goes wrong part-way, nothing changes.</li>
{% else %}
<li><span class="tone-warn">The backup is from an older Steward, so it replaces everything</span>: the audit
log, alert history, metrics and check results go too, and Steward brings the backup up to date as it
starts. If this fails part-way, Steward starts empty, at first-run setup, where the safety backup can be
uploaded.</li>
{% endif %}
<li>While it runs, pages say Steward is restoring and agents hold their readings. Steward then restarts, and
you sign in again as a user from the backup.</li>
</ul>
{% if error %}<p class="tone-crit" style="margin-bottom:0.75rem;">{{ error }}</p>{% endif %}
<form method="post" action="/settings/backups/restore/start/">{{ csrf_field() }}
<input type="hidden" name="token" value="{{ held.token }}">
{% if report.schema == "older" %}
<label style="display:flex;align-items:center;gap:0.4rem;font-weight:normal;margin-bottom:0.75rem;">
<input type="checkbox" name="replace_all"> Replace everything, history included</label>
{% endif %}
<div class="form-group" style="max-width:260px;">
<label for="r-confirm">Type <code>{{ word }}</code> to confirm</label>
<input type="text" id="r-confirm" name="confirm" autocomplete="off" required>
</div>
<button type="submit" class="btn btn-danger">Restore</button>
</form>
{% endif %}
</div>
{% endif %}
<p style="margin-top:1rem;"><a href="/settings/backups/">Back to Backups</a></p>
{% endblock %}
+18 -7
View File
@@ -1,7 +1,7 @@
{# settings/backups.html — Steward's own backups (#5673): a verdict, the bucket
they go to (Fabled Archives over S3) and the daily hour, then the recent
runs, each with its own check and a Check button (#5823), and a file to
check. Restoring is `steward restore`; see docs/core/backups.md. #}
runs, each with its own check and Check and Restore buttons (#5823, #5817),
and a file to check or restore. #}
{% extends "base.html" %}
{% from "_macros.html" import crumbs %}
{% from "settings/_tabs.html" import settings_tabs %}
@@ -16,6 +16,12 @@
{% if verdict.good %}The newest good backup was sent {{ verdict.good|ago }}.{% else %}No backup has been sent yet.{% endif %}
{%- endif %}</span></p>
{% if restore_failed %}
<p class="tone-crit" style="margin:-0.5rem 0 1.25rem;">The restore of {{ restore_failed.name }}
{%- if restore_failed.since %} {{ restore_failed.since|ago }}{% endif %} failed, and nothing changed:
{{ restore_failed.error }}</p>
{% endif %}
{% if flash %}
<div style="padding:0.75rem 1rem;border-radius:6px;margin-bottom:1.25rem;
background:{% if flash.ok %}var(--green-dim){% else %}var(--red-dim){% endif %};
@@ -130,11 +136,15 @@
<td class="nowrap">{{ r.size|bytes if r.ok else "—" }}</td>
<td class="nowrap">{{ r.secs|duration }}</td>
<td>{{ "the schedule" if r.by == "schedule" else r.by }}</td>
<td>{% if r.ok and r.name %}
<td>{% if r.ok and r.name %}<div style="display:flex;gap:0.35rem;">
<form method="post" action="/settings/backups/check/">{{ csrf_field() }}
<input type="hidden" name="name" value="{{ r.name }}">
<button type="submit" class="btn btn-ghost btn-sm" title="Compare {{ r.name }} with what is in place; nothing changes">Check</button>
</form>{% endif %}</td>
</form>
<form method="post" action="/settings/backups/restore/">{{ csrf_field() }}
<input type="hidden" name="name" value="{{ r.name }}">
<button type="submit" class="btn btn-ghost btn-sm" title="See what restoring {{ r.name }} would change, then confirm">Restore</button>
</form></div>{% endif %}</td>
</tr>
{% endfor %}
</tbody>
@@ -144,15 +154,16 @@
<p class="tone-dim">None yet.</p>
{% endif %}
<p class="field-hint" style="margin-top:0.75rem;">Check compares a backup with what is in place: what
a restore would bring back, change or remove, area by area. It changes nothing. To bring Steward back
from one, see <code>steward restore</code> in the backups guide (docs/core/backups.md).</p>
a restore would bring back, change or remove, area by area. It changes nothing. Restore shows the same
comparison and asks you to confirm before anything changes.</p>
<div class="card" style="max-width:640px;margin-top:1.25rem;">
<h2 class="section-title" style="margin-bottom:0.5rem;">Check a file</h2>
<h2 class="section-title" style="margin-bottom:0.5rem;">Check or restore a file</h2>
<form method="post" action="/settings/backups/check/file/" enctype="multipart/form-data"
style="display:flex;gap:0.75rem;flex-wrap:wrap;align-items:center;">{{ csrf_field() }}
<input type="file" name="file" accept=".tar" required>
<button type="submit" class="btn btn-ghost">Check</button>
<button type="submit" class="btn btn-ghost" formaction="/settings/backups/restore/file/">Restore</button>
</form>
<p class="field-hint" style="margin-top:0.5rem;">A <code>steward-&lt;time&gt;.tar</code> downloaded from
the bucket, or made with <code>steward backup</code>.</p>