feat(docker): Docker's own status reaches Steward — an empty answer clears a host, a lost one greys it, a demoted manager's swarm rows go (#5152)
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 53s
CI / unit (push) Failing after 1m17s
CI / publish (push) Skipped

The agent (1.9.0) said nothing when Docker could not be asked and nothing
when Docker had no containers, so Steward could tell neither apart from the
other, and kept a host's last containers on screen as running either way.

- Agent: collect_docker returns (status, detail, containers). "ok" sends
  docker: [] when there are none; "unavailable" (no socket, nothing
  listening) and "error" (permission, timeout, bad answer) say why.
  collect_swarm says whether /info answered, and a node that is not a
  manager sends swarm_manager: false. A failed /info says nothing.
- Ingest: an empty docker list is a snapshot; the newest sample's status
  goes to the Docker plugin with the rest.
- docker_host_status (docker_011): per host, status, detail, since and
  last report. A host that never had Docker gets no row.
- An "ok" empty snapshot reaps the host's containers; an error keeps them.
  swarm_manager: false drops the host's swarm services and nodes.
- Docker page: a "Docker not answering" card names each such host, why,
  and since when; its containers read as stale. Duplicate header comment
  in rows.html removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-06 18:41:22 -04:00
co-authored by Claude Opus 5.5
parent f802c9f153
commit 8ceb983183
11 changed files with 510 additions and 65 deletions
+38 -2
View File
@@ -251,8 +251,37 @@ async def _persist_disk(session, host, disk: dict) -> None:
await session.execute(stale)
async def _persist_status(session, host, status: dict) -> None:
"""Record whether Docker answered on this host, and why not.
A host reporting "unavailable" with no row yet never had Docker: nothing to
record. Once a host has had Docker, losing it is recorded, so its last
containers read as stale rather than as running."""
from .models import DockerHostStatus
row = await session.get(DockerHostStatus, host.id)
if row is None:
if status["status"] == "unavailable":
return
row = DockerHostStatus(host_id=host.id)
session.add(row)
if row.status != status["status"]:
row.since = status["at"]
row.status = status["status"]
row.detail = (status.get("detail") or "")[:255] or None
row.reported_at = status["at"]
async def _expire_swarm(session, host) -> None:
"""Drop the swarm rows a host reported while it was a manager."""
from .models import DockerSwarmNode, DockerSwarmService
await session.execute(delete(DockerSwarmService).where(DockerSwarmService.host_id == host.id))
await session.execute(delete(DockerSwarmNode).where(DockerSwarmNode.host_id == host.id))
async def persist_host_docker(session, host, snapshots, swarm=None, disk=None,
logs=None) -> None:
logs=None, status=None) -> None:
"""Upsert containers + time-series + lifecycle events + swarm for one host.
`snapshots` is a list of (recorded_at: datetime, containers: list[dict]) —
@@ -264,11 +293,18 @@ async def persist_host_docker(session, host, snapshots, swarm=None, disk=None,
`disk` is the newest sample's /system/df summary (or None on Docker-less
hosts) — persisted when present. `logs` is a list of (recorded_at, records)
log batches (m79) — appended one row per line when present, independent of
whether this sample also carried container metrics.
whether this sample also carried container metrics. `status` is the newest
sample's {"status", "detail", "at", "swarm_manager"} (agent ≥ 1.9.0): an
"ok" with an empty snapshot clears the host's containers, and a node that
says it is no longer a manager loses the swarm rows it reported as one.
"""
from steward.core.alerts import MetricBatch
from .models import DockerContainer, DockerEvent, DockerMetric
if status is not None:
await _persist_status(session, host, status)
if status.get("swarm_manager") is False:
await _expire_swarm(session, host)
if swarm is not None:
await _persist_swarm(session, host, swarm)
if disk is not None: