bvandeusen
7b80552a7d
Docker collection moves off the central single-socket scrape onto the host
agent, giving Docker a real per-host dimension. The Steward host now reports
its own containers like any other host, and same-named containers on different
hosts no longer collide.
- agent: stdlib UDS Docker client (AF_UNIX HTTP/1.1, Connection: close,
chunked-aware), collect_docker() ports the cpu%/mem math; sample["docker"]
added best-effort (silent-skip on absent/unreadable socket). AGENT_VERSION
1.2.0 → 1.3.0; optional docker_socket config key.
- ingest: host_agent ingest hands per-host container snapshots to the docker
plugin via a new "docker.persist_host_samples" capability (no hard import,
no-op when docker disabled), inside a SAVEPOINT so a docker failure never
sinks the host metrics. Resource names are host-scoped ("<host>/<name>").
- schema: docker_containers re-keyed (host_id, name); docker_metrics gains
host_id; docker_002 migration DROP+recreates (dev-only, rule 122).
- ui: Docker page + widgets grouped by host with host links; new per-host
Docker panel embedded on the Hosts hub (gated on docker enabled via a new
enabled_plugins template context). Replaces the SQLite-only strftime
bucketing with DB-agnostic Python bucketing.
- provisioning: install/provision playbooks add steward-agent to the docker
group (best-effort) so the agent can read the socket.
- removed central scrape: docker scheduler.py + scraper.py deleted; plugin.yaml
socket_path/scrape_interval_seconds/include_stopped dropped (plugin 2.0.0).
- tests: agent docker collector units (math, chunked decode, silent-skip,
sample shape, config) + integration (host-scoped schema + persistence).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
90 lines
3.8 KiB
Python
90 lines
3.8 KiB
Python
# plugins/docker/ingest.py
|
|
"""Persist host-scoped Docker samples pushed by the host agent.
|
|
|
|
Published as the "docker.persist_host_samples" capability (see __init__.setup),
|
|
so the host_agent plugin can hand off the `docker` array from a sample WITHOUT
|
|
importing the docker models — the coupling is opportunistic and degrades to a
|
|
no-op when the docker plugin is disabled. Runs inside the caller's open
|
|
transaction (the ingest handler's session); never opens or commits its own.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
from datetime import datetime
|
|
|
|
|
|
def _parse_started_at(value) -> datetime | None:
|
|
"""Parse the agent's ISO-8601 started_at string back to a datetime."""
|
|
if not isinstance(value, str) or not value:
|
|
return None
|
|
try:
|
|
return datetime.fromisoformat(value.replace("Z", "+00:00"))
|
|
except ValueError:
|
|
return None
|
|
|
|
|
|
async def persist_host_docker(session, host, snapshots) -> None:
|
|
"""Upsert containers + append time-series for one host's docker snapshots.
|
|
|
|
`snapshots` is a list of (recorded_at: datetime, containers: list[dict]) —
|
|
one entry per ingested sample carrying docker data (usually just one; more
|
|
when the agent flushes a backlog). Every snapshot contributes time-series
|
|
DockerMetric rows; the newest snapshot drives current container state and
|
|
the alert pipeline (record_metric writes "now", so feeding it stale buffered
|
|
snapshots would be misleading).
|
|
"""
|
|
from steward.core.alerts import record_metric
|
|
from .models import DockerContainer, DockerMetric
|
|
|
|
if not snapshots:
|
|
return
|
|
ordered = sorted(snapshots, key=lambda s: s[0])
|
|
latest_at, latest_containers = ordered[-1]
|
|
|
|
# Time-series points for every snapshot (running containers with a CPU read).
|
|
for recorded_at, containers in ordered:
|
|
for c in containers:
|
|
if c.get("status") == "running" and c.get("cpu_pct") is not None:
|
|
session.add(DockerMetric(
|
|
host_id=host.id,
|
|
container_name=c["name"],
|
|
scraped_at=recorded_at,
|
|
cpu_pct=c["cpu_pct"],
|
|
mem_pct=c.get("mem_pct") or 0.0,
|
|
mem_usage_bytes=c.get("mem_usage_bytes") or 0,
|
|
))
|
|
|
|
# Current state + alerts from the newest snapshot only.
|
|
for c in latest_containers:
|
|
name = c.get("name")
|
|
if not name:
|
|
continue
|
|
existing = await session.get(DockerContainer, (host.id, name))
|
|
if existing is None:
|
|
existing = DockerContainer(host_id=host.id, name=name)
|
|
session.add(existing)
|
|
existing.container_id = c.get("container_id", "") or ""
|
|
existing.image = c.get("image", "") or ""
|
|
existing.status = c.get("status", "unknown") or "unknown"
|
|
existing.cpu_pct = c.get("cpu_pct")
|
|
existing.mem_usage_bytes = c.get("mem_usage_bytes")
|
|
existing.mem_limit_bytes = c.get("mem_limit_bytes")
|
|
existing.mem_pct = c.get("mem_pct")
|
|
existing.ports_json = json.dumps(c.get("ports") or [])
|
|
existing.started_at = _parse_started_at(c.get("started_at"))
|
|
existing.scraped_at = latest_at
|
|
|
|
# Alert pipeline — resource is host-scoped so containers of the same name
|
|
# on different hosts don't collide in the metric/alert namespace.
|
|
if c.get("status") == "running" and c.get("cpu_pct") is not None:
|
|
resource = f"{host.name}/{name}"
|
|
await record_metric(
|
|
session=session, source_module="docker",
|
|
resource_name=resource, metric_name="cpu_pct", value=c["cpu_pct"],
|
|
)
|
|
if c.get("mem_pct") is not None:
|
|
await record_metric(
|
|
session=session, source_module="docker",
|
|
resource_name=resource, metric_name="mem_pct", value=c["mem_pct"],
|
|
)
|