feat(docker): ingest swarm topology + lifecycle events + health/restart alerts
Wires the agent's enriched + swarm payloads through the docker.persist_host_
samples capability:
* Swarm topology — persist sample["swarm"] into docker_swarm_services /
docker_swarm_nodes (upsert + prune stale, host-scoped so two managers don't
clobber). Migration docker_005 adds services.placement_json for the
task→node placement the agent now reports.
* Lifecycle events — _derive_events (pure, unit-tested) diffs the newest
snapshot against stored per-container state: start / stop / die (non-zero
exit) / oom / health_change → docker_events rows. Skipped on a host's first
snapshot so the baseline doesn't emit a start per existing container.
* Alerts — record restart_count (always) and is_healthy (1.0/0.0, only when a
HEALTHCHECK exists) alongside cpu/mem, under host-scoped resource names;
METRIC_CATALOG[docker] gains restart_count + is_healthy so they're alertable.
host_agent ingest captures the newest sample's swarm object and threads it to
the capability (now persist_host_docker(session, host, snapshots, swarm=None));
invoked when containers OR swarm are present, under the same SAVEPOINT. Unit
tests cover the event-diff matrix; integration tests cover event derivation
across two snapshots and swarm topology round-trip (incl. placement).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
This commit is contained in:
@@ -197,6 +197,8 @@ async def ingest():
|
||||
accepted = 0
|
||||
latest_ts: datetime | None = None
|
||||
docker_snapshots: list[tuple[datetime, list]] = []
|
||||
latest_swarm: dict | None = None
|
||||
latest_swarm_ts: datetime | None = None
|
||||
for sample in samples:
|
||||
try:
|
||||
recorded_at = _parse_ts(sample["ts"])
|
||||
@@ -208,6 +210,12 @@ async def ingest():
|
||||
docker = sample.get("docker")
|
||||
if isinstance(docker, list) and docker:
|
||||
docker_snapshots.append((recorded_at, docker))
|
||||
# Swarm is current-state, not time-series — keep only the newest
|
||||
# sample's topology (a manager re-reports it every interval).
|
||||
swarm = sample.get("swarm")
|
||||
if isinstance(swarm, dict) and (
|
||||
latest_swarm_ts is None or recorded_at > latest_swarm_ts):
|
||||
latest_swarm, latest_swarm_ts = swarm, recorded_at
|
||||
accepted += 1
|
||||
if latest_ts is None or recorded_at > latest_ts:
|
||||
latest_ts = recorded_at
|
||||
@@ -220,7 +228,7 @@ async def ingest():
|
||||
# (opportunistic synergy via the capability registry — no hard import,
|
||||
# no-op when docker is disabled). A failure here must never sink the
|
||||
# whole ingest, so the metrics above still land.
|
||||
if docker_snapshots:
|
||||
if docker_snapshots or latest_swarm is not None:
|
||||
from steward.core.capabilities import has_capability, invoke_capability
|
||||
if has_capability("docker.persist_host_samples"):
|
||||
try:
|
||||
@@ -229,7 +237,7 @@ async def ingest():
|
||||
async with session.begin_nested():
|
||||
await invoke_capability(
|
||||
"docker.persist_host_samples", UserRole.admin,
|
||||
session, host, docker_snapshots,
|
||||
session, host, docker_snapshots, latest_swarm,
|
||||
)
|
||||
except Exception:
|
||||
current_app.logger.exception(
|
||||
|
||||
Reference in New Issue
Block a user