fix(maintenance): check first, drain before a Docker upgrade, wait 5 min after a drain, see containers back (#6098)
On 2026-10-18 an update on VDNT-Docker02 restarted the Docker daemon six seconds in, before the drain that came only with the reboot, and every container on the node stopped. Docker's own packages restart the daemon from their install scripts; needrestart's list-only mode doesn't stop that. - The helper gains a `check` action (agent 1.24.0): a dry run of the update for each package manager, whether Docker is among it, and the running containers outside the swarm with their restart policies. - A swarm node is drained before an update that upgrades Docker, else only before a reboot, and at most once a job. - After a drain the job settles 5 minutes before the update or reboot. - Once the work is over, the containers that ran outside the swarm must be seen running again within 10 minutes, or the job fails naming them. - An older agent gets no check and its job runs as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -20,9 +20,11 @@
|
||||
{{ page_header("Maintenance") }}
|
||||
<p style="color:var(--text-muted);font-size:0.84rem;margin:-0.4rem 0 1.25rem;">
|
||||
Each plan updates an inventory group's hosts once a week, one host at a time. A host reboots
|
||||
only when its update needs it, unless the plan says otherwise; a swarm node is drained first
|
||||
and made active again after. The next host starts once the last is back in service. A failure
|
||||
stops the run, and the host's <em>Maintenance failed</em> figure can drive an alert rule.
|
||||
only when its update needs it, unless the plan says otherwise. A swarm node is drained before
|
||||
a reboot, or before the update when it upgrades Docker, given 5 minutes for its tasks to move,
|
||||
and made active again after. Containers running outside the swarm must be running again
|
||||
within 10 minutes. The next host starts once the last is back in service. A failure stops the
|
||||
run, and the host's <em>Maintenance failed</em> figure can drive an alert rule.
|
||||
</p>
|
||||
|
||||
{% if live %}
|
||||
|
||||
@@ -93,7 +93,7 @@
|
||||
{% else %}
|
||||
<form method="post" action="/plugins/host_agent/fleet/{{ host.id }}/maintain"
|
||||
style="display:flex;gap:0.6rem;align-items:center;flex-wrap:wrap;margin:0.6rem 0 0;"
|
||||
data-confirm="Update {{ host.name }} now? It reboots only if the update needs it, after its swarm node is drained.">{{ csrf_field() }}
|
||||
data-confirm="Update {{ host.name }} now? It reboots only if the update needs it. A swarm node is drained first when it reboots or Docker is upgraded.">{{ csrf_field() }}
|
||||
<button type="submit" class="btn btn-sm">Update now</button>
|
||||
<label style="font-size:0.8rem;color:var(--text-muted);display:flex;gap:0.3rem;align-items:center;">
|
||||
<input type="checkbox" name="reboot_policy" value="never"> never reboot
|
||||
|
||||
Reference in New Issue
Block a user