feat(host_agent): how far each host's clock is from Steward's (milestone 490)
CI / lint (push) Successful in 5s
CI / integration (push) Successful in 1m8s
CI / unit (push) Successful in 1m34s
CI / publish (push) Successful in 2m27s

- Agent 1.12.0 stamps each post with sent_at; ingest compares it with the
  moment the post arrived: clock_offset_s / clock_checked_at on the
  registration (host_agent_003), and a clock_off_secs metric (either way)
  that alert rules can watch. Replaces the old skew log, which measured the
  newest sample and was off by up to an interval.
- Settings -> Thresholds: Clock drift, 30 s warn / 5 min critical.
- Host page: 'clock 0.3s ahead' in the vitals line, toned past the warning.
- Hosts list: names hosts whose clock is off; when every reporting host is
  off by about the same amount, says Steward's own clock is the likely one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-08 08:00:33 -04:00
co-authored by Claude Opus 5.5
parent 766eef04c5
commit 96c1ea9407
15 changed files with 217 additions and 11 deletions
+8
View File
@@ -22,6 +22,14 @@
{% endif %}
</div>
</div>
{% if clock %}
{#- Milestone 490: a host whose clock drifts, or Steward's own when every host disagrees with it. #}
<p class="page-verdict tone-warn">
{%- if clock.steward %}Every host's clock is about {{ clock.typical|duration }} off from Steward's: Steward's own clock is the likely one that is wrong. Check what keeps the time in sync on the machine Steward runs on.
{%- else %}Clock off from Steward's on {% for name, secs in clock.hosts %}<a href="/hosts/{{ host_ids[name] }}">{{ name }}</a> by {{ secs|duration }}{{ ", " if not loop.last }}{% endfor %}. Check what keeps the time in sync there.
{%- endif -%}
</p>
{% endif %}
{% if total %}
<form class="filter-bar" method="get" action="/hosts/" role="search"
hx-target="#hosts-table" hx-push-url="true" hx-include="closest form">
@@ -43,6 +43,7 @@
{{ pair("Ping latency", "latency_warn", "ping.threshold.good_ms", "latency_crit", "ping.threshold.warn_ms", "ms", "Round-trip time. Shared with the Ping widget's good/warn colouring.") }}
{{ pair("Uptime / SLA", "uptime_warn", "thresholds.uptime_warn", "uptime_crit", "thresholds.uptime_crit", "%", "Lower is worse — amber below warn, red below crit.", step="0.1") }}
{{ pair("TLS certificate expiry", "tls_warn", "thresholds.tls_warn_days", "tls_crit", "thresholds.tls_crit_days", "days left", "HTTPS monitors whose certificate expires within these many days are flagged. Lower is worse.") }}
{{ pair("Clock drift", "clock_warn", "thresholds.clock_warn_s", "clock_crit", "thresholds.clock_crit_s", "seconds", "How far a host's clock may be from Steward's, either way, before its page and the Hosts list call it out.") }}
<div class="form-group" style="margin-bottom:0.25rem;">
<label>Container crash loop <span style="color:var(--text-muted);font-size:0.8rem;">(deaths per hour)</span></label>