feat(docker): Docker's own status reaches Steward — an empty answer clears a host, a lost one greys it, a demoted manager's swarm rows go (#5152)
The agent (1.9.0) said nothing when Docker could not be asked and nothing when Docker had no containers, so Steward could tell neither apart from the other, and kept a host's last containers on screen as running either way. - Agent: collect_docker returns (status, detail, containers). "ok" sends docker: [] when there are none; "unavailable" (no socket, nothing listening) and "error" (permission, timeout, bad answer) say why. collect_swarm says whether /info answered, and a node that is not a manager sends swarm_manager: false. A failed /info says nothing. - Ingest: an empty docker list is a snapshot; the newest sample's status goes to the Docker plugin with the rest. - docker_host_status (docker_011): per host, status, detail, since and last report. A host that never had Docker gets no row. - An "ok" empty snapshot reaps the host's containers; an error keeps them. swarm_manager: false drops the host's swarm services and nodes. - Docker page: a "Docker not answering" card names each such host, why, and since when; its containers read as stale. Duplicate header comment in rows.html removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -251,8 +251,37 @@ async def _persist_disk(session, host, disk: dict) -> None:
|
||||
await session.execute(stale)
|
||||
|
||||
|
||||
async def _persist_status(session, host, status: dict) -> None:
|
||||
"""Record whether Docker answered on this host, and why not.
|
||||
|
||||
A host reporting "unavailable" with no row yet never had Docker: nothing to
|
||||
record. Once a host has had Docker, losing it is recorded, so its last
|
||||
containers read as stale rather than as running."""
|
||||
from .models import DockerHostStatus
|
||||
|
||||
row = await session.get(DockerHostStatus, host.id)
|
||||
if row is None:
|
||||
if status["status"] == "unavailable":
|
||||
return
|
||||
row = DockerHostStatus(host_id=host.id)
|
||||
session.add(row)
|
||||
if row.status != status["status"]:
|
||||
row.since = status["at"]
|
||||
row.status = status["status"]
|
||||
row.detail = (status.get("detail") or "")[:255] or None
|
||||
row.reported_at = status["at"]
|
||||
|
||||
|
||||
async def _expire_swarm(session, host) -> None:
|
||||
"""Drop the swarm rows a host reported while it was a manager."""
|
||||
from .models import DockerSwarmNode, DockerSwarmService
|
||||
|
||||
await session.execute(delete(DockerSwarmService).where(DockerSwarmService.host_id == host.id))
|
||||
await session.execute(delete(DockerSwarmNode).where(DockerSwarmNode.host_id == host.id))
|
||||
|
||||
|
||||
async def persist_host_docker(session, host, snapshots, swarm=None, disk=None,
|
||||
logs=None) -> None:
|
||||
logs=None, status=None) -> None:
|
||||
"""Upsert containers + time-series + lifecycle events + swarm for one host.
|
||||
|
||||
`snapshots` is a list of (recorded_at: datetime, containers: list[dict]) —
|
||||
@@ -264,11 +293,18 @@ async def persist_host_docker(session, host, snapshots, swarm=None, disk=None,
|
||||
`disk` is the newest sample's /system/df summary (or None on Docker-less
|
||||
hosts) — persisted when present. `logs` is a list of (recorded_at, records)
|
||||
log batches (m79) — appended one row per line when present, independent of
|
||||
whether this sample also carried container metrics.
|
||||
whether this sample also carried container metrics. `status` is the newest
|
||||
sample's {"status", "detail", "at", "swarm_manager"} (agent ≥ 1.9.0): an
|
||||
"ok" with an empty snapshot clears the host's containers, and a node that
|
||||
says it is no longer a manager loses the swarm rows it reported as one.
|
||||
"""
|
||||
from steward.core.alerts import MetricBatch
|
||||
from .models import DockerContainer, DockerEvent, DockerMetric
|
||||
|
||||
if status is not None:
|
||||
await _persist_status(session, host, status)
|
||||
if status.get("swarm_manager") is False:
|
||||
await _expire_swarm(session, host)
|
||||
if swarm is not None:
|
||||
await _persist_swarm(session, host, swarm)
|
||||
if disk is not None:
|
||||
|
||||
Reference in New Issue
Block a user