feat(docker): collect + persist /system/df image/disk usage (agent 1.6.0)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 47s
CI / integration (push) Failing after 2m23s
CI / publish (push) Has been skipped

Backend for the image/disk panel (milestone 77 #942). Agent gains
collect_disk_usage() — one /system/df call (gated on containers existing, so
Docker-less hosts pay nothing), surfacing reclaimable bytes (image size held by
unreferenced images), layers/containers/volumes/build-cache sizes, and the top
50 images by size. Emitted as sample["docker_disk"]; host_agent ingest tracks
the newest sample's copy and hands it to the docker capability as a 5th arg.

New current-state tables docker_disk_usage (one row/host) + docker_images
(per-host image rows), docker_007 migration; ingest upserts the summary and
replaces the image set per host (stale images pruned). Unit tests for the df
parsing/reclaimable math + build_sample gating; integration test for
persistence + image-set replacement.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
This commit is contained in:
2026-06-18 21:51:14 -04:00
parent faecac3ec6
commit a840d6f823
7 changed files with 367 additions and 3 deletions
+54
View File
@@ -230,6 +230,60 @@ def test_swarm_topology_persisted(app):
assert node[0] == "manager" and node[2] == "ready" and node[3] is True
@_NEEDS_DB
def test_disk_usage_persisted(app):
"""persist_host_docker stores the /system/df summary + image rows, and
replaces the image set on the next report (stale images pruned)."""
from sqlalchemy import text
from steward.models.hosts import Host
persist = _persist_fn(app)
disk1 = {
"layers_size": 1000, "images_total": 2, "images_active": 1,
"images_size": 1000, "images_reclaimable": 700,
"containers_count": 2, "containers_size": 100,
"volumes_count": 1, "volumes_size": 40, "build_cache_size": 15,
"images": [
{"image_id": "aaaaaaaaaaaa", "repo_tag": "app:1", "size": 300,
"shared_size": 50, "containers": 1},
{"image_id": "bbbbbbbbbbbb", "repo_tag": "<none>", "size": 700,
"shared_size": 0, "containers": 0},
],
}
disk2 = {**disk1, "images_reclaimable": 0, "images": [disk1["images"][0]]}
async def _go():
async with app.db_sessionmaker() as s:
async with s.begin():
await s.execute(text("DELETE FROM docker_images"))
await s.execute(text("DELETE FROM docker_disk_usage"))
h = Host(id=str(uuid.uuid4()), name="dsk", address="10.6.6.6")
s.add(h)
await s.flush()
hid = h.id
await persist(s, h, [], None, disk1)
summary = (await s.execute(text(
"SELECT images_reclaimable, images_total, build_cache_size "
"FROM docker_disk_usage WHERE host_id=:h"), {"h": hid})).first()
img_count = (await s.execute(text(
"SELECT COUNT(*) FROM docker_images WHERE host_id=:h"), {"h": hid})).scalar()
# Re-report with one image dropped — the stale row must be pruned.
async with s.begin():
await persist(s, await s.get(Host, hid), [], None, disk2)
after = (await s.execute(text(
"SELECT image_id FROM docker_images WHERE host_id=:h"), {"h": hid})).all()
reclaimable2 = (await s.execute(text(
"SELECT images_reclaimable FROM docker_disk_usage WHERE host_id=:h"),
{"h": hid})).scalar()
return summary, img_count, [r[0] for r in after], reclaimable2
summary, img_count, after_ids, reclaimable2 = asyncio.run(_go())
assert summary[0] == 700 and summary[1] == 2 and summary[2] == 15
assert img_count == 2
assert after_ids == ["aaaaaaaaaaaa"] # the unreferenced image pruned on re-report
assert reclaimable2 == 0 # summary upserted in place
@_NEEDS_DB
def test_retention_rollup_and_prune(app):
"""Old raw metrics roll up to hourly averages then delete; stale rollup +