fix(host_agent): a stalled NFS mount no longer freezes the agent (#5760)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m21s
CI / unit (push) Successful in 1m33s
CI / publish (push) Successful in 1m3s

#5760 "One stalled NFS mount freezes the whole agent: statvfs has no
deadline, so the host goes silent while it still answers ping". The agent
read every mount inline in its only loop; a hard-mounted share whose server
stops blocks statvfs in the kernel with no timeout. Each mount is now read
in a daemon thread under one 5 s deadline. One that misses it is sent as
stalled, and isn't probed again while its last probe is still blocked. A
mount that errors (a stale handle) is left out instead of emptying the list.

Steward counts them as mounts_stalled (alertable: "an NFS share stopped"),
and skips the fullest-disk figure when every mount stalled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-14 17:12:27 -04:00
co-authored by Claude Opus 5.5
parent 7feb2b1ce2
commit b6c4ce510d
7 changed files with 128 additions and 19 deletions
@@ -93,3 +93,14 @@ def test_a_sensor_with_nothing_wired_to_it_does_not_set_the_host_s_temperature()
assert idx[("temp_c_max", "util")] == 60.0
assert idx[("temp_c", "util:temp:AUXTIN0")] == 110.0
assert idx[("temp_c", "util:temp:PCH_CHIP_CPU_MAX_TEMP")] == 0.0
def test_stalled_mounts_are_counted_and_have_no_figures():
"""#5760: a mount the agent couldn't read in time."""
sample = {"ts": "2026-10-14T21:00:00+00:00",
"storage": [{"mount": "/", "total_bytes": 100, "used_bytes": 50},
{"mount": "/nfs/media", "stalled": True}]}
idx = _index(_expand_sample_to_metrics(sample, "d3", datetime.now(timezone.utc)))
assert idx[("mounts_stalled", "d3")] == 1
assert idx[("disk_used_pct_worst", "d3")] == 50.0
assert ("disk_used_pct", "d3:/nfs/media") not in idx