Commit Graph
6 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 613ed4e78b feat(traffic): each container's network rate, and each NFS mount's bytes, so a host's traffic has a sender (#6115)
CI / unit (push) Failing after 1m30s
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m38s
CI / publish (push) Skipped
Docker02's outbound traffic tops out the internet connection and Steward
couldn't say who sends it: a container's counters were kept only as a
running total, and NFS traffic belongs to no container.

- Docker ingest turns each running container's cumulative network counters
  into net_rx_bps / net_tx_bps on "<host>/<container>" (net_rates), counting
  from the stored row so a restart of Steward loses nothing; a container
  new, recreated or restarted has no rate until its next snapshot.
- Agent 1.25.0 reads the bytes sent and received per NFS mount from
  mountstats (already parsed for the delays) as tx_bps / rx_bps; Steward
  stores them as nfs_tx_bps / nfs_rx_bps on "<host>:<mount>".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:35:24 -04:00
bvandeusenandClaude Opus 5.5 3f9aae7f46 feat(host_agent): NFS delay per mount, read from /proc/self/mountstats (#5996)
CI / unit (push) Failing after 1m32s
CI / integration (push) Successful in 1m31s
CI / publish (push) Skipped
CI / lint (push) Successful in 2s
Each sample carries each NFS mount's requests since the last one: how many
a second, their round trip and execute time (READ and WRITE on their own),
and how many were sent again or hit a major timeout. mountstats is the
kernel's own count and never touches the share, so it reads while one is
stuck. Steward stores them per mount and the slowest on the host as
nfs_exe_max_ms, for an "NFS slow" rule and the host page's NFS chart beside
mounts_stalled; each mount's bar says what a request took.

Agent 1.22.0 (which also carries the backlog order from 41fc807).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 09:46:14 -04:00
bvandeusenandClaude Opus 5.5 b6c4ce510d fix(host_agent): a stalled NFS mount no longer freezes the agent (#5760)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m21s
CI / unit (push) Successful in 1m33s
CI / publish (push) Successful in 1m3s
#5760 "One stalled NFS mount freezes the whole agent: statvfs has no
deadline, so the host goes silent while it still answers ping". The agent
read every mount inline in its only loop; a hard-mounted share whose server
stops blocks statvfs in the kernel with no timeout. Each mount is now read
in a daemon thread under one 5 s deadline. One that misses it is sent as
stalled, and isn't probed again while its last probe is still blocked. A
mount that errors (a stale handle) is left out instead of emptying the list.

Steward counts them as mounts_stalled (alertable: "an NFS share stopped"),
and skips the fullest-disk figure when every mount stalled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:12:27 -04:00
bvandeusenandClaude Opus 5.5 5dd9956f35 feat(host_agent): a disk out of inodes reads as full (#5745)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m19s
CI / unit (push) Successful in 1m27s
CI / publish (push) Successful in 1m13s
VDNT-Util01's / ran out of inodes (956 free of 60.8M) with 274G of space
left; every file create failed and Steward's page read healthy at 66%.

- Agent 1.13.0 reports each mount's inodes (statvfs; a filesystem reporting
  none, like btrfs, is left without).
- Ingest records inodes_used_pct per mount, and disk_used_pct_worst is the
  fullest mount by space or inodes, so existing disk-full rules and the
  overview see it.
- The host's health strip, the Hosts widget and the host glance grade / by
  whichever is fuller and name inodes when they are ("Disk / 66% · inodes
  100%", "Inodes on / ... past its critical threshold").

Hosts pick up the inode figures once their agent is updated to 1.13.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:51:46 -04:00
bvandeusenandClaude Opus 4.8 f037b69c58 test(host_agent): update existing collector tests for new agent API
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m15s
CI / publish (push) Successful in 8s
collect_cpu now returns (aggregate, [per_core]) and build_sample takes a
rate-state dict, so the pre-existing tests that asserted the old float
return / no-state signature needed updating. Mocks the new collectors
(temps/psi/net/diskio) for determinism.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 10:15:43 -04:00
bvandeusenandClaude Sonnet 4.6 b6b1c8adca test(host_agent): agent resource collectors unit tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 22:07:39 -04:00