#5768 "A maintenance cadence: a group's updates and reboots, one host at a
time, on a schedule". A plan names an inventory group, a weekday and time
in a time zone, a window and a reboot policy (only when required by
default). At its slot a run takes the group's hosts in order and gives each
a #5767 maintenance job. The next host starts only when the last is back in
service: its agent is posting and, for a swarm node, the node is active and
every replicated service is at its replicas. A failure stops the run, and
no host is started once the window has passed. A host's latest failure
shows as host_agent/<host>/maintenance_failed, for an ordinary alert rule.
The page is Hosts → Maintenance.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5767 "Steward hands a host a maintenance job and follows it to the end,
through its own restarts", milestone 514.
A host_maintenance_jobs row per job: queued → updating → [draining →]
rebooting → [undraining →] done, or failed / timed_out. Steward's answer to
an ingest post carries the step that host's agent should carry out now; the
agent (1.18.0) leaves each step id for the root helper once, and the helper
refuses an id it already ran, so repeats are harmless. Results come back as
the #5766 maintenance events, matched by step id; the reboot completes when
the host's `started` event has a boot_at after the reboot was handed out.
The job reboots only when the update says one is needed (or never, if asked).
A swarm node is drained first by itself if it's a manager or by a manager
with the helper, and made active after; with no manager to drain it the job
is refused. A minute tick times out a job past its state's deadline. A
maintenance window over host_agent/<host> lasts the rest of the job.
Everything is on the row, so a Steward restart, or the reboot of the host
Steward runs on, resumes from the next post. Ingest runs it in its own
savepoint so a job going wrong never costs the samples.
The Agent panel leads with a running job and says what a finished one did;
Manage agent has Update now (with "never reboot"), or says why a job can't
start. Integration test: hand off, Steward restarts, the post-boot events
close the job.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5766 "A root helper the agent can start, for a fixed set of maintenance
actions and nothing else", milestone 514.
The agent runs unprivileged under NoNewPrivileges, so it can't update or
reboot its host. It now leaves a request in /var/lib/steward-agent/requests,
the one path its unit lets it write; steward-maint.path starts maint.py as
root, which reads it without following links, accepts only update, reboot,
drain or undrain, deletes it, and writes the outcome to
/var/lib/steward-maint/results. The agent (1.17.0) reports each new result
state once as a `maintenance` event, re-checks pending updates when an update
finishes, and says in its metadata whether the helper is installed.
The units and the helper are served from one copy at
/plugins/host_agent/files/<name> (a fixed list), and install.sh plus the
install, provision and update playbooks all fetch them, replacing three
inline copies of the unit. Update agent installs the helper on existing
hosts; the Agent panel says under Manage agent whether it is there.
Steward handing out jobs is #5767; the cadence is #5768.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 1 of milestone 514 "Updates and reboots on a cadence, carried out by
each host's agent…": #5765 "The agent reports pending updates and whether
a reboot is required".
- Agent 1.16.0: an hourly background check (apt-check, apt list, or dnf -C
with updateinfo and needs-restarting), each command bounded at 120 s, no
root and no network. Debian's reboot-required marker and its packages
are read every sample.
- Ingest records updates_pending, updates_security and reboot_required for
rules, and keeps the newest on the registration (host_agent_005_updates).
- The Agent card leads with "Reboot required since …", then the updates
waiting or "Up to date", with when the check ran. list_agents carries
them too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5756 "The agent says when its host rebooted and when it lost contact, so
downtime is recorded as what it was". Agent 1.14.0 posts events with the
next post that gets through: started (with the host's boot time), stopping
(on SIGTERM, with whether systemd says the host is going down) and
lost_contact (from its first unanswered post to the one carrying it).
Steward keeps them as the host's downtime (host_downtime): reboot,
agent_stop, unclean (a boot after the agent was last heard that nobody
announced) or lost_contact. Retried posts apply once.
An announced stop explains the silence for reboot_grace_seconds (Settings
-> Plugins -> host_agent, 10 minutes): seconds_since_seen reads 0 until
then, and the host's other rules go quiet with it (agents_away beside
#5721's registry). Past the grace, agent offline fires.
The host's Agent card lists the last 7 days of downtime. Rows are pruned
at data.retention_days.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
History (#5316):
- Transitions fold into episodes (steward/alerts/episodes.py): each stretch
out of quiet, the worst state it reached, how long it lasted, its worst
reading against the rule ('3m 21s (fires above 3m)'), who was told across
its changes, and its Ansible runs. The changes themselves fold under it.
- An episode cut by paging says so ('cleared later (above)', 'earlier').
- For one rule: a verdict ('Fired 4 times in the last 7d, 1h 2m in all') and
no Rule column. Notified and Action show only when the range has any.
Tests:
- The confirm guard looks for a confirm( call, not the word in the comment
that says it is gone (lesson #4365).
- The Hosts viewer test looks for the add-host dialog by id, since every page
now carries the confirm dialog.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5294 "Hosts list: search, group and status filters; sortable columns; a Groups column".
Ansible inventory groups are Steward's grouping system (operator's ruling): a host's
groups are its linked target's, and membership overlaps, so the list filters by group
rather than sectioning by it.
- Filter bar: search on name or address, a group (or "No group"), a status (Down,
Agent silent, Up, No monitors). The table swaps in alone and the URL keeps the filters.
- Name, Status, Latency, 30d uptime, CPU and Memory headers sort (sort_th in _macros);
problems-first stays the default, and a host with no value goes last either way.
- A Groups column of chips, each filtering to its group; shown when any group exists.
- A swap is keyed on HX-Target, so Back to a pushed URL still gets the page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5290 "Host page: the health tiles span the card's width; even spacing above Ansible":
the four tiles take equal quarters (halves on a phone) and their sparklines stretch
across them; the Monitors + Agent row drops the Agent card's own margin, so the gap
above Ansible is 1rem like every other.
#5291 "Add host opens as a dialog over the Hosts list": a native <dialog> on /hosts/,
opened by Add host or /hosts/?add=1. A name clash re-renders the list with it open,
the error and what was typed. GET /hosts/new is gone; hosts/form.html is Edit only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hosts list:
- CPU and Memory are their own labelled columns (#5279); a silent
agent's last readings are greyed, and Status says "agent silent for 7m".
- One uptime figure, 30 days, with 24h and 7d on hover. The SLA tab
(/hosts/uptime) repeated it and is removed (#5280; one home per fact).
- Status is a word with how long ("Down for 12m"); host_status now carries
down_since. Edit and Delete live on the host's page only (#5281).
- Figures are right-aligned with tabular digits (.table .num), not a
monospace face.
Agent fleet (#5282): its add-host form is gone, so hosts are added only on
Hosts -> Add host. That form was also the only way to get a token for a
manual curl install, so that moves to the host's Agent card as "Install
manually" (POST /fleet/<host_id>/register), shown on the fleet page as
before. "Delete" there reads "Unregister".
Add host (#5283): laid out like the add-monitor form; a name clash comes
back inline with what was typed instead of a bare text 400; the address
is optional, as the host page already allowed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docker panel (#5276): problems first, then the 8 busiest running
containers by CPU; the other running and the stopped fold behind one
"+N more running · M stopped" line. Containers that are not swarm tasks
read the way the Docker page reads them: CI jobs as "CI: workflow · job",
buildx builders as "BuildKit builder · name".
Monitors card (#5277): the edit/delete icon buttons and the inline add
form are gone. Each row links to its monitor's page; "Add monitor" opens
the add form filed under this host, starting from its address, with the
host in its breadcrumbs and Cancel back to it.
Agent card (#5278): Full metrics stays in view; Update, Rotate token,
Unregister agent (was "Remove") and Re-provision sit under "Manage agent".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DNS monitors take an optional Resolver, a server's IP. Set, the check asks
that server directly for A and AAAA over UDP and records how long it took
to answer; empty keeps today's lookup through the container's resolver.
"Does my local DNS still answer for public names" and "does 1.1.1.1" are
now separate monitors. No new dependency.
On a host's page the Ansible card moves above the Docker list, which can
run to dozens of containers and hid it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bvandeusenandClaude Opus 5.5