feat(host_agent): NFS delay per mount, read from /proc/self/mountstats (#5996)
CI / lint (push) Successful in 2s
CI / unit (push) Failing after 1m32s
CI / integration (push) Successful in 1m31s
CI / publish (push) Skipped

Each sample carries each NFS mount's requests since the last one: how many
a second, their round trip and execute time (READ and WRITE on their own),
and how many were sent again or hit a major timeout. mountstats is the
kernel's own count and never touches the share, so it reads while one is
stuck. Steward stores them per mount and the slowest on the host as
nfs_exe_max_ms, for an "NFS slow" rule and the host page's NFS chart beside
mounts_stalled; each mount's bar says what a request took.

Agent 1.22.0 (which also carries the backlog order from 41fc807).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-17 09:46:14 -04:00
co-authored by Claude Opus 5.5
parent 2df8431de9
commit 3f9aae7f46
12 changed files with 282 additions and 13 deletions
@@ -114,6 +114,17 @@
<span style="color:var(--text-muted);">{{ m.get('disk_used_bytes')|bytes }} / {{ m.get('disk_total_bytes')|bytes }}{% if m.get('disk_used_pct') is not none %} · {{ "%.0f"|format(m.get('disk_used_pct')) }}%{% endif %}</span>
</div>
{{ bar(m.get('disk_used_pct'), 'disk') }}
{% if m.get('nfs_ops_per_sec') is not none %}
{# NFS delay (#5996): what a request took since the agent's last sample. #}
{% set exe = m.get('nfs_exe_ms') %}
<div style="font-size:0.72rem;color:var(--text-dim);margin-top:0.2rem;">
NFS {% if m.get('nfs_ops_per_sec') > 0 and exe is not none -%}
<span class="{{ 'tone-crit' if exe >= 1000 else ('tone-warn' if exe >= 100 else '') }}">{{ "%.1f"|format(exe) }} ms a request</span> · {{ "%.3g"|format(m.get('nfs_ops_per_sec')) }}/s
{%- else %}idle{% endif %}
{%- if m.get('nfs_retrans') %} · <span class="tone-warn">{{ m.get('nfs_retrans')|int }} sent again</span>{% endif %}
{%- if m.get('nfs_timeouts') %} · <span class="tone-crit">{{ m.get('nfs_timeouts')|int }} timed out</span>{% endif %}
</div>
{% endif %}
</div>
{% endfor %}
</div>
@@ -35,6 +35,12 @@
<div class="section-title" style="margin-bottom:0.5rem;">Load &amp; pressure — last <span class="hm-chart-range">{{ current_range }}</span></div>
<div style="height:220px;"><canvas id="chart-pressure"></canvas></div>
</div>
{% if has_nfs %}
<div class="card" style="margin-bottom:0;">
<div class="section-title" style="margin-bottom:0.5rem;">NFS delay (ms) — last <span class="hm-chart-range">{{ current_range }}</span></div>
<div style="height:220px;"><canvas id="chart-nfs"></canvas></div>
</div>
{% endif %}
</div>
{# Hidden poller: fetches the history series and applies it to the charts above
(no visible swap). Lazy on load → never blocks paint; refreshed slowly + on
@@ -79,6 +85,10 @@
{ key: "psi_mem_some_avg10", label: "Mem PSI %", color: stewardColor("--red") },
{ key: "temp_c_max", label: "Temp °C", color: stewardColor("--yellow") },
] },
"chart-nfs": { ymax: undefined, ds: [
{ key: "nfs_exe_max_ms", label: "Slowest mount, ms a request", color: stewardColor("--blue") },
{ key: "mounts_stalled", label: "Mounts not answering", color: stewardColor("--red") },
] },
};
const charts = {};
for (const id of Object.keys(defs)) {