fix(ansible): the swarm prune never runs a docker prune that touches images; it lives in its own folder (#6399)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 1m42s
CI / integration (push) Successful in 1m47s
CI / publish (push) Successful in 1m10s

The first applied run lost VDNT-Docker03 29 images (10.4 GB) before its
script ran. Its `docker system prune` removes dangling images, and Docker
counts an image with no tag as dangling: every image a swarm task pulled by
digest. Docker03 was left holding only its own tasks' images, as under the
old prune. Nothing went down, since a running container's image can't be
removed.

- Step 1 now prunes containers, networks, build cache and (prune_volumes)
  volumes one by one. Every image decision is left to the script, which
  keeps the images services are pinned to.
- A test guards against any image-pruning command in the playbook.
- The playbook moves to maintenance/swarm_prune/, with its script in that
  folder's files/, so a helper's name can't collide with another
  playbook's (the operator's direction for playbooks with helpers).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-25 11:22:41 -04:00
co-authored by Claude Opus 5.5
parent b40e672cda
commit b2f4dde288
3 changed files with 29 additions and 15 deletions
@@ -1,5 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Steward's swarm prune (maintenance/swarm_prune.yml), run on each node. """Steward's swarm prune (maintenance/swarm_prune/swarm_prune.yml), run on each node.
Removes the images on this node that no swarm service is pinned to and no Removes the images on this node that no swarm service is pinned to and no
container uses (by id, or by the reference it was made from), and keeps every image a service is pinned to even when its container uses (by id, or by the reference it was made from), and keeps every image a service is pinned to even when its
@@ -4,10 +4,11 @@
# steward:confirm: true # steward:confirm: true
# Run on the swarm's nodes; each must be a manager (only a manager can list # Run on the swarm's nodes; each must be a manager (only a manager can list
# services). On each node: # services). On each node:
# 1. docker system prune: stopped containers, unused networks, dangling # 1. Stopped containers, unused networks and build cache are pruned
# images, build cache (+ prune_volumes=true: unused anonymous volumes). # (+ prune_volumes=true: unused anonymous volumes). No image is: Docker
# 2. files/swarm_prune.py: removes every other image no container uses and # calls an untagged image dangling, and swarm pulls by digest, untagged.
# no swarm service is pinned to, wherever that service's task runs. A # 2. files/swarm_prune.py: removes every image no container uses and no
# swarm service is pinned to, wherever that service's task runs. A
# node keeps the images for the services running elsewhere, so it can # node keeps the images for the services running elsewhere, so it can
# start them at once when the other node fails or is drained. # start them at once when the other node fails or is drained.
# A node that can't list the swarm's services removes no images and the run # A node that can't list the swarm's services removes no images and the run
@@ -19,13 +20,19 @@
vars: vars:
prune_volumes: false prune_volumes: false
tasks: tasks:
- name: Prune stopped containers, unused networks, dangling images and build cache # Never `docker system prune` or `docker image prune`: Docker counts an
# image with no tag as dangling, and every image a swarm task pulled by
# digest has none. Images are left to the script below.
- name: Prune stopped containers, unused networks and build cache
ansible.builtin.command: ansible.builtin.command:
argv: >- argv: "{{ item }}"
{{ ['docker', 'system', 'prune', '-f'] loop: >-
+ (['--volumes'] if prune_volumes | bool else []) }} {{ [['docker', 'container', 'prune', '-f'],
register: system_prune ['docker', 'network', 'prune', '-f'],
changed_when: "'Total reclaimed space: 0B' not in system_prune.stdout" ['docker', 'builder', 'prune', '-f']]
+ ([['docker', 'volume', 'prune', '-f']] if prune_volumes | bool else []) }}
register: cleanup
changed_when: "'Total reclaimed space: 0B' not in cleanup.stdout and cleanup.stdout | trim | length > 0"
- name: Remove images no container uses and no swarm service is pinned to - name: Remove images no container uses and no swarm service is pinned to
ansible.builtin.script: ansible.builtin.script:
@@ -39,5 +46,6 @@
- name: Report what went - name: Report what went
ansible.builtin.debug: ansible.builtin.debug:
msg: >- msg: >-
{{ (system_prune.stdout_lines | default([]) | select | list)[-1:] {{ (cleanup.results | default([]) | map(attribute='stdout', default='')
| select('search', 'reclaimed') | list)
+ [image_prune.stdout | from_json] }} + [image_prune.stdout | from_json] }}
+9 -3
View File
@@ -16,7 +16,7 @@ BUILTIN = Path(get_sources({"sources": []})[0]["path"])
@pytest.fixture @pytest.fixture
def prune(): def prune():
spec = importlib.util.spec_from_file_location( spec = importlib.util.spec_from_file_location(
"swarm_prune", BUILTIN / "maintenance" / "files" / "swarm_prune.py") "swarm_prune", BUILTIN / "maintenance" / "swarm_prune" / "files" / "swarm_prune.py")
mod = importlib.util.module_from_spec(spec) mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod) spec.loader.exec_module(mod)
return mod return mod
@@ -27,13 +27,19 @@ def _image(id_, tags=(), digests=(), size=100):
def test_the_playbook_is_listed_and_asks_before_it_runs(): def test_the_playbook_is_listed_and_asks_before_it_runs():
assert "maintenance/swarm_prune.yml" in discover_playbooks(str(BUILTIN)) assert "maintenance/swarm_prune/swarm_prune.yml" in discover_playbooks(str(BUILTIN))
content = (BUILTIN / "maintenance" / "swarm_prune.yml").read_text() content = (BUILTIN / "maintenance" / "swarm_prune" / "swarm_prune.yml").read_text()
meta = discover_playbook_meta(content) meta = discover_playbook_meta(content)
assert meta["confirm"] is True and meta["category"] == "maintenance" and meta["description"] assert meta["confirm"] is True and meta["category"] == "maintenance" and meta["description"]
play = yaml.safe_load(content)[0] play = yaml.safe_load(content)[0]
script = next(t for t in play["tasks"] if "ansible.builtin.script" in t) script = next(t for t in play["tasks"] if "ansible.builtin.script" in t)
assert script["check_mode"] is False and "--dry-run" in script["ansible.builtin.script"]["cmd"] assert script["check_mode"] is False and "--dry-run" in script["ansible.builtin.script"]["cmd"]
# No docker prune touches images: an image swarm pulled by digest has no
# tag, and Docker calls that dangling (on 2026-10-25 a `system prune`
# took 29 a node was keeping for the other's services).
commands = yaml.safe_dump([t for t in play["tasks"] if "ansible.builtin.command" in t])
assert "'system'" not in commands and "'image'" not in commands
assert "'container', 'prune'" in commands and "'builder', 'prune'" in commands
def test_services_are_matched_by_digest_or_else_by_tag(prune): def test_services_are_matched_by_digest_or_else_by_tag(prune):