feat(docker): uptime from StartedAt, a restart is an event, host totals count physical devices only (#5153)

- Agent: started_at comes from inspect's State.StartedAt (nanoseconds
  trimmed; Docker's zero time is "never started"), not the list's Created,
  so a restarted container's uptime counts from its restart.
- A rising restart_count between two snapshots emits a "restart" event
  ("restart #3", or "3 restarts, now #5"), said instead of a start; a
  restart then a death gives both. A count that falls (recreated under the
  same name) says nothing. The timeline draws it as ↻.
- Network and disk I/O count what the kernel calls physical: interfaces
  not under /sys/devices/virtual/net (bridges, docker0, veth, vxlan, tun),
  block devices with a device behind them (not dm, md, zram), so a byte is
  not counted twice. Where /sys calls everything virtual (an agent in LXC)
  the name-filtered set is kept rather than reporting no traffic.
- Agent tests updated for #5152's (status, detail, containers) and
  (is_manager, topology) shapes; the no-Docker disk test now states the
  case it was passing for by accident.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-06 18:45:57 -04:00
co-authored by Claude Opus 5.5
parent 8ceb983183
commit bcd0563f39
8 changed files with 170 additions and 32 deletions
+12 -3
View File
@@ -34,12 +34,14 @@ def _derive_events(old_state: dict, new_containers: list) -> list:
"""Diff a fresh snapshot against stored per-container state → lifecycle events.
Pure function (no DB) so it's unit-testable. `old_state` maps name → the
previously stored {status, health, oom_killed, exit_code}; `new_containers`
previously stored {status, health, oom_killed, exit_code, restart_count}; `new_containers`
is the newest snapshot's list of container dicts. Returns a list of
(container_name, event, detail) tuples:
start — a container transitions into running (or a genuinely new
container appears already running)
restart — Docker restarted it (its restart count went up); said
instead of a start, since that is what happened
stop — running → not-running, or a running container is removed
die — same as stop but with a non-zero exit code (abnormal)
oom — OOMKilled flips False→True
@@ -69,7 +71,13 @@ def _derive_events(old_state: dict, new_containers: list) -> list:
continue
old_running = (old.get("status") or "").lower() == "running"
if new_running and not old_running:
new_restarts = c.get("restart_count") or 0
old_restarts = old.get("restart_count") or 0
if new_restarts > old_restarts:
gained = new_restarts - old_restarts
events.append((name, "restart", f"restart #{new_restarts}" if gained == 1
else f"{gained} restarts, now #{new_restarts}"))
if new_running and not old_running and new_restarts <= old_restarts:
events.append((name, "start", c.get("image") or None))
elif old_running and not new_running:
if new_exit not in (None, 0):
@@ -338,7 +346,8 @@ async def persist_host_docker(session, host, snapshots, swarm=None, disk=None,
)).scalars().all()
old_state = {
r.name: {"status": r.status, "health": r.health,
"oom_killed": r.oom_killed, "exit_code": r.exit_code}
"oom_killed": r.oom_killed, "exit_code": r.exit_code,
"restart_count": r.restart_count}
for r in old_rows
}
if old_state:
+1 -1
View File
@@ -308,7 +308,7 @@ async def get_container(container: str, host: str = "", range: str = "24h", even
range: the window for `cpu_pct_history` and `mem_pct_history` (min, avg,
max, p95, last and a ≤60-point series) — "1h", "6h", "24h" (default),
"7d" or "30d". `cpu_pct` and `mem_pct` are the latest reported values.
events: how many lifecycle events (start, stop, die, oom, health_change)
events: how many lifecycle events (start, restart, stop, die, oom, health_change)
to return, newest first (default 20, max 100).
A container Docker has since removed (a dead Swarm task, a `docker rm`)
+1 -1
View File
@@ -146,7 +146,7 @@ class DockerEvent(Base):
)
container_name: Mapped[str] = mapped_column(String(255), nullable=False)
event: Mapped[str] = mapped_column(String(16), nullable=False)
# start | stop | die | oom | health_change
# start | restart | stop | die | oom | health_change
detail: Mapped[str | None] = mapped_column(String(255), nullable=True)
# e.g. "exit 137", "healthy→unhealthy", image on start
at: Mapped[datetime] = mapped_column(
+1
View File
@@ -303,6 +303,7 @@ async def host_panel(host_id: str):
# Glyph + colour per lifecycle event, so the timeline reads at a glance.
_EVENT_STYLE = {
"start": ("▲", "var(--green)"),
"restart": ("↻", "var(--yellow)"),
"stop": ("■", "var(--text-muted)"),
"die": ("✕", "var(--red)"),
"oom": ("☠", "var(--red)"),