perf(metrics): a latest-value table for "what is it now"; dashboard cells say loading, failed and as-of (#5208)
The dashboard's host widgets took 18-60 s. Every "latest reading" query (hosts overview, host-agent fleet, host page, vitals, SNMP, MCP get_host) searched plugin_metrics for the newest row per metric. Postgres has no skip scan, so each one read every raw sample in the 7-day window. - plugin_metric_latest: one row per (source, resource, metric). record_metrics upserts it in the same transaction, moving forward only and in sorted key order. Migration 0030 seeds it by stepping the existing index. - Retention drops a latest row once its sample leaves the raw window, so a resource that stopped reporting disappears when its history does, as before. Renames carry it along. - A quiet host keeps its last values, marked stale. A time-bounded query would have shown it as missing. - Vitals sparklines span the 6 h they claim, averaged in SQL to ~120 points, instead of fetching 6 h raw and drawing the last 120 samples. - Dashboard and share-view cells: "Loading…" until the first answer; a 20 s deadline per fetch; "Couldn't load (why)" with Retry now and an automatic retry for cells that don't poll; "as of HH:MM" in the header, and "couldn't refresh · as of" when a refresh fails over shown data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,8 +1,9 @@
|
||||
"""Integration: host-metrics read paths (DISTINCT ON latest + SQL date_bin history).
|
||||
"""Integration: host-metrics read paths (latest values + SQL date_bin history).
|
||||
|
||||
Validates the two host_agent query helpers that back the slow host views, against
|
||||
a live Postgres: the latest-per-(resource,metric) lookup and the bucket-averaged
|
||||
history (which now aggregates in SQL instead of shipping raw rows to Python).
|
||||
a live Postgres: the latest-per-(resource,metric) lookup (plugin_metric_latest,
|
||||
kept by record_metrics) and the bucket-averaged history (which aggregates in SQL
|
||||
instead of shipping raw rows to Python).
|
||||
Requires STEWARD_DATABASE_URL.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
@@ -31,8 +32,9 @@ def app():
|
||||
|
||||
|
||||
@_NEEDS_DB
|
||||
def test_latest_distinct_on_and_sql_bucketed_history(app):
|
||||
def test_latest_values_and_sql_bucketed_history(app):
|
||||
from sqlalchemy import text
|
||||
from steward.core.alerts import record_metrics
|
||||
from steward.models.metrics import PluginMetric
|
||||
from plugins.host_agent.metrics_query import (
|
||||
SOURCE_MODULE, _history_for_host, _latest_metrics_for_host,
|
||||
@@ -62,7 +64,7 @@ def test_latest_distinct_on_and_sql_bucketed_history(app):
|
||||
metric_name="disk_used_pct", value=80.0,
|
||||
recorded_at=now - timedelta(minutes=5),
|
||||
))
|
||||
s.add_all(rows)
|
||||
await record_metrics(s, rows)
|
||||
|
||||
latest = await _latest_metrics_for_host(s, hostname)
|
||||
hist = await _history_for_host(s, hostname, now - timedelta(hours=1))
|
||||
@@ -70,7 +72,7 @@ def test_latest_distinct_on_and_sql_bucketed_history(app):
|
||||
|
||||
latest, hist = asyncio.run(_go())
|
||||
|
||||
# DISTINCT ON returns the newest sample per (resource, metric).
|
||||
# The newest sample per (resource, metric).
|
||||
assert latest[hostname]["cpu_pct"] == 30.0
|
||||
assert latest[hostname + ":/"]["disk_used_pct"] == 80.0
|
||||
|
||||
|
||||
Reference in New Issue
Block a user