fix(host_agent): a stalled NFS mount no longer freezes the agent (#5760)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m21s
CI / unit (push) Successful in 1m33s
CI / publish (push) Successful in 1m3s

#5760 "One stalled NFS mount freezes the whole agent: statvfs has no
deadline, so the host goes silent while it still answers ping". The agent
read every mount inline in its only loop; a hard-mounted share whose server
stops blocks statvfs in the kernel with no timeout. Each mount is now read
in a daemon thread under one 5 s deadline. One that misses it is sent as
stalled, and isn't probed again while its last probe is still blocked. A
mount that errors (a stale handle) is left out instead of emptying the list.

Steward counts them as mounts_stalled (alertable: "an NFS share stopped"),
and skips the fullest-disk figure when every mount stalled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-14 17:12:27 -04:00
co-authored by Claude Opus 5.5
parent 7feb2b1ce2
commit b6c4ce510d
7 changed files with 128 additions and 19 deletions
+4
View File
@@ -116,6 +116,10 @@ Python 3.8+ stdlib only. `urllib.request` for the POST, plus `json`, `os`, `shut
- **SIGTERM** (from systemd) → finish current poll cycle → flush buffer one last time, with a `stopping` event saying whether the host is going down too (`systemctl is-system-running` says `stopping`) → exit.
- **Systemd restart policy** → `Restart=on-failure`, `RestartSec=10`.
### Mounts that don't answer (1.14.0, #5760)
Each mount is read in a daemon thread under one 5 s deadline. A hard-mounted NFS share whose server stops blocks `statvfs` in the kernel with no timeout; read inline, that froze the whole agent. A mount that misses the deadline is sent as `{"mount", "stalled": true}`, and isn't probed again while its last probe is still blocked. Steward counts them as `mounts_stalled`.
### What happened to the host (1.14.0, #5756)
A post may carry `events`, said once each, with the next post that gets through: