fix(host_agent): every Docker read is bounded, and the agent reports its own footprint (#5764)
#5764 "The agent reads a container's whole log backlog into memory, on its main loop, with no overall deadline". Docker03's 1.13.0 agent went silent for a day at 2.1 GB and 46 h of CPU, then ignored SIGTERM. - One shared reader, _docker_get: each answer has an overall deadline and a byte cap, not only a per-recv timeout. A trickling or endless body is cut off (15 s / 32 MiB for JSON, 5 s / 4 MiB for a log read). - Container logs: since the cursor but only the newest 2,000 lines, all within 10 s. A note says when older lines were skipped, and a container whose answer is too big skips ahead to now instead of re-reading its backlog every interval. - The agent reports its own RSS and CPU; Steward records agent_rss_bytes and agent_cpu_pct, so a runaway agent shows before it goes silent. Agent 1.15.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -149,6 +149,14 @@ def _expand_sample_to_metrics(
|
||||
if sample.get("uptime_secs") is not None:
|
||||
row("uptime_secs", host_name, sample["uptime_secs"])
|
||||
|
||||
# The agent's own footprint (#5764), so a runaway agent shows before it
|
||||
# goes silent.
|
||||
me = sample.get("agent") or {}
|
||||
if me.get("rss_bytes") is not None:
|
||||
row("agent_rss_bytes", host_name, me["rss_bytes"])
|
||||
if me.get("cpu_pct") is not None:
|
||||
row("agent_cpu_pct", host_name, me["cpu_pct"])
|
||||
|
||||
# The fullest disk is the fullest by space or by inodes (#5745): out of
|
||||
# inodes, every file create fails with space to spare, so the disk rules
|
||||
# and the overview, which read disk_used_pct_worst, see either.
|
||||
|
||||
Reference in New Issue
Block a user