feat(host_agent): Steward hands a host a maintenance job and follows it through the reboot (#5767)
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 1m28s
CI / unit (push) Failing after 1m31s
CI / publish (push) Skipped

#5767 "Steward hands a host a maintenance job and follows it to the end,
through its own restarts", milestone 514.

A host_maintenance_jobs row per job: queued → updating → [draining →]
rebooting → [undraining →] done, or failed / timed_out. Steward's answer to
an ingest post carries the step that host's agent should carry out now; the
agent (1.18.0) leaves each step id for the root helper once, and the helper
refuses an id it already ran, so repeats are harmless. Results come back as
the #5766 maintenance events, matched by step id; the reboot completes when
the host's `started` event has a boot_at after the reboot was handed out.

The job reboots only when the update says one is needed (or never, if asked).
A swarm node is drained first by itself if it's a manager or by a manager
with the helper, and made active after; with no manager to drain it the job
is refused. A minute tick times out a job past its state's deadline. A
maintenance window over host_agent/<host> lasts the rest of the job.

Everything is on the row, so a Steward restart, or the reboot of the host
Steward runs on, resumes from the next post. Ingest runs it in its own
savepoint so a job going wrong never costs the samples.

The Agent panel leads with a running job and says what a finished one did;
Manage agent has Update now (with "never reboot"), or says why a job can't
start. Integration test: hand off, Steward restarts, the post-boot events
close the job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-14 18:42:12 -04:00
co-authored by Claude Opus 5.5
parent 6a68bd75d9
commit 22b0be2b6c
12 changed files with 954 additions and 8 deletions
+29
View File
@@ -279,3 +279,32 @@ async def test_manage_agent_says_whether_the_maintenance_helper_is_installed(app
reg=SimpleNamespace(**{**vars(reg), "maint_helper": None}), **kw)
assert "Maintenance helper installed" in installed
assert "No maintenance helper yet: Update agent installs it" in missing
async def test_the_agent_card_follows_a_maintenance_job_and_says_why_it_cant_start_one(app):
"""#5767: a running job leads what the host has waiting; Update now is
offered only when a job can start, else the reason is shown."""
from datetime import datetime, timedelta, timezone
from plugins.host_agent.maintenance import summary
from tests._templates import render_plugin_template
now = datetime.now(timezone.utc)
host = SimpleNamespace(id="h1", name="box", address="10.0.0.5")
reg = SimpleNamespace(agent_version="1.18.0", last_seen_at=now, maint_helper=True,
updates_checked_at=now, updates_pending=4, updates_security=0,
reboot_required=False)
job = SimpleNamespace(state="rebooting", state_at=now - timedelta(minutes=2), finished_at=None,
detail=None, packages=4, back_at=None, reboot_needed=True,
reboot_policy="if_required")
kw = dict(host=host, reg=reg, reporting=True, stale=False, target=None, ansible_available=False,
managed_key_set=False, downtime=[])
async with app.test_request_context("/plugins/host_agent/panel/h1"):
session["user_role"] = "admin"
busy = await render_plugin_template(
app, "host_agent", "panel.html", maint_job=job, maint_summary=summary(job),
maint_blocker="a maintenance job is already running on it", **kw)
free = await render_plugin_template(app, "host_agent", "panel.html", maint_job=None,
maint_blocker=None, **kw)
assert busy.index("Maintenance: Rebooting") < busy.index("4 updates waiting")
assert "Can't update from here: a maintenance job is already running on it." in busy
assert ">Update now</button>" not in busy
assert ">Update now</button>" in free and 'name="reboot_policy" value="never"' in free