diff --git a/docs/plugins/host-agent-design.md b/docs/plugins/host-agent-design.md index ef6611a..e81e5bd 100644 --- a/docs/plugins/host-agent-design.md +++ b/docs/plugins/host-agent-design.md @@ -74,8 +74,15 @@ token = a1b2c3d4... interval_seconds = 30 hostname = myhost # optional; defaults to uname -n mounts = /, /mnt/data # optional; defaults to all real mounts (excluding tmpfs/devtmpfs/etc.) +docker_logs_enabled = true # optional; on by default — per-host container-log kill-switch +docker_log_exclude = watchtower, noisy-svc # optional; container names whose logs this host skips collecting ``` +Container-log collection is on by default (no config needed). The two keys above +are the per-host opt-outs; the operator-facing global toggle + exclude live in +Settings and are enforced server-side (the push model has no channel to command +an agent), so they don't require touching a host's conf. + ### Agent internals (function list, not classes) - `read_config(path)` — parses the conf file into a dict. @@ -85,10 +92,17 @@ mounts = /, /mnt/data # optional; defaults to all real mounts (excludin - `collect_load()` — reads `/proc/loadavg`, returns `[1m, 5m, 15m]`. - `collect_uptime()` — reads `/proc/uptime`, returns seconds since boot (int). - `collect_metadata()` — `os.uname()` for kernel + arch, `/etc/os-release` for distro. Called once at startup and cached. +- `collect_docker_logs(socket, containers, state, exclude)` — (m79) per running + container, fetches new log lines over the Docker socket with an incremental + `since` cursor kept per container in `state` (a container's first interval + seeds from a short tail). Demuxes the multiplexed stream, parses each line's + RFC3339 timestamp, caps the whole batch at a byte limit (a marker line records + a truncation; the deferred lines come next interval). Folded into the sample as + `docker_logs`; omitted when empty. - `build_payload()` — assembles a snapshot from all collectors into one dict. - `post_payload(url, token, payloads)` — POSTs a list of samples, returns success/failure. - `RingBuffer(maxlen=20)` — tiny FIFO wrapper, drops oldest when full. -- `main_loop()` — the 30s loop: collect → try POST → on failure push to buffer + backoff → on success flush buffer. +- `main_loop()` — the 30s loop: collect → try POST → on failure push to buffer + backoff → on success flush buffer. Container logs are stripped from a sample before it's buffered (metrics survive an outage; stale logs are dropped). **Target: ~300 lines total including docstrings.** More than that is a smell that the agent is over-scoping. @@ -255,6 +269,7 @@ Content-Type: application/json - **`metadata` is sent on every POST**, not just on change. Server-side diff detects actual changes and only writes on change. Cost per POST is one dict — negligible. Benefit: server can cleanly detect agent restarts. - **Raw bytes, not percentages, for memory and storage.** Percentages are derived server-side. Changing the "what counts as used" math doesn't require re-releasing the agent. - **CPU is the one exception** — reported as a percentage because it's inherently a derivative (delta over time), not a snapshot. The agent must sample twice to compute it. +- **Container logs (m79) ride in the same push** as `docker_logs`: a list of `{container, stream, ts, line}` records — incremental since the previous interval. This pushes the *same direction* as metrics, so batched log history needs no inbound channel (only sub-second live-follow would). The server ingests them into `docker_logs`, enforces the global toggle + exclude list on ingest, and bounds storage with a per-container size+age ring. `docker_logs` is omitted when there's nothing new. ### Server expansion into `PluginMetric` rows