fix(host_agent): a stalled NFS mount no longer freezes the agent (#5760)
#5760 "One stalled NFS mount freezes the whole agent: statvfs has no deadline, so the host goes silent while it still answers ping". The agent read every mount inline in its only loop; a hard-mounted share whose server stops blocks statvfs in the kernel with no timeout. Each mount is now read in a daemon thread under one 5 s deadline. One that misses it is sent as stalled, and isn't probed again while its last probe is still blocked. A mount that errors (a stale handle) is left out instead of emptying the list. Steward counts them as mounts_stalled (alertable: "an NFS share stopped"), and skips the fullest-disk figure when every mount stalled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -63,10 +63,12 @@ def get_alert_metrics() -> dict[str, list[str]]:
|
||||
clock_off_secs is how far the host's clock is from Steward's, either way
|
||||
(agent ≥ 1.12.0): "clock drifting" is > 30. load_pct is load_1m as a %
|
||||
of the host's cores. inodes_used_pct is per mount (agent ≥ 1.13.0), and
|
||||
disk_used_pct_worst is the fullest mount by space or inodes."""
|
||||
disk_used_pct_worst is the fullest mount by space or inodes.
|
||||
mounts_stalled counts the mounts that didn't answer the agent in time
|
||||
(agent ≥ 1.14.0): "an NFS share stopped" is > 0."""
|
||||
return {"host_agent": [
|
||||
"cpu_pct", "mem_used_pct", "mem_available_bytes", "swap_used_bytes",
|
||||
"disk_used_pct_worst", "disk_used_pct", "inodes_used_pct", "load_1m", "load_5m", "load_15m",
|
||||
"load_pct",
|
||||
"uptime_secs", "seconds_since_seen", "clock_off_secs",
|
||||
"uptime_secs", "seconds_since_seen", "clock_off_secs", "mounts_stalled",
|
||||
]}
|
||||
|
||||
Reference in New Issue
Block a user