feat(health): Steward records its own health and Settings → System shows it (#5467)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m15s
CI / unit (push) Failing after 1m29s
CI / publish (push) Skipped

#5467 "Steward's own health: metrics, rules can watch them, a page that shows them".

The self-monitor tick now records, under source "steward", resource "process":
ingest_posts / ingest_failed / ingest_slowest_ms (the host-agent posts since the
last tick, tallied in steward/core/health.py), db_roundtrip_ms (a timed SELECT 1),
and unwatched_secs (the gap before this tick past watch.GAP_S, from Steward's own
last sample). The hourly cleanup records cleanup_secs. The open-file pair is still
recorded, and is now only skipped on its own where /proc is absent. All of these
are in CORE_METRICS, so an alert rule can watch them under "Steward itself".

Settings → System (/settings/system/, admin) shows a verdict first (failed posts,
a gap in watching, open files near the limit), then each measure now and over the
last day, with a sparkline scaled to its own peak.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 14:02:50 -04:00
co-authored by Claude Opus 5.5
parent 656e127fd8
commit c8dbe811f8
12 changed files with 337 additions and 33 deletions
+6
View File
@@ -276,6 +276,12 @@ METRICS: dict[str, tuple[str, str]] = {
"is_up": ("Up or down", "whether the check passed"),
"is_healthy": ("Health check", "the container's own HEALTHCHECK result"),
"response_ms": ("Response time", "how long the check took"),
"ingest_posts": ("Agent posts per minute", "host-agent reports Steward took in the last minute"),
"ingest_failed": ("Failed agent posts", "reports in the last minute that didn't land"),
"ingest_slowest_ms": ("Slowest agent post", "the longest report in the last minute; agents give up at 10 s"),
"db_roundtrip_ms": ("Database round trip", "a trivial query to Steward's database, connection included"),
"unwatched_secs": ("Time not watching", "how long Steward wasn't recording before this minute; 0 when unbroken"),
"cleanup_secs": ("Hourly cleanup took", "how long the last retention and rollup run took"),
"open_fds": ("Open files", "files Steward itself has open"),
"open_fds_pct": ("Open files (% of limit)", "files Steward has open, against its limit"),
"request_rate": ("Requests per second", "requests through this router"),