perf(metrics): a latest-value table for "what is it now"; dashboard cells say loading, failed and as-of (#5208)
The dashboard's host widgets took 18-60 s. Every "latest reading" query (hosts overview, host-agent fleet, host page, vitals, SNMP, MCP get_host) searched plugin_metrics for the newest row per metric. Postgres has no skip scan, so each one read every raw sample in the 7-day window. - plugin_metric_latest: one row per (source, resource, metric). record_metrics upserts it in the same transaction, moving forward only and in sorted key order. Migration 0030 seeds it by stepping the existing index. - Retention drops a latest row once its sample leaves the raw window, so a resource that stopped reporting disappears when its history does, as before. Renames carry it along. - A quiet host keeps its last values, marked stale. A time-bounded query would have shown it as missing. - Vitals sparklines span the 6 h they claim, averaged in SQL to ~120 points, instead of fetching 6 h raw and drawing the last 120 samples. - Dashboard and share-view cells: "Loading…" until the first answer; a 20 s deadline per fetch; "Couldn't load (why)" with Retry now and an automatic retry for cells that don't poll; "as of HH:MM" in the header, and "couldn't refresh · as of" when a refresh fails over shown data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -30,6 +30,25 @@ class PluginMetric(Base):
|
||||
)
|
||||
|
||||
|
||||
class PluginMetricLatest(Base):
|
||||
"""The newest sample of each (source, resource, metric), kept beside
|
||||
plugin_metrics by record_metrics so "what is it now" never scans history.
|
||||
|
||||
Postgres has no skip scan, so a newest-per-key query over plugin_metrics
|
||||
reads every raw sample in the window: on a seven-day window that took a
|
||||
dashboard widget a minute (#5208). This table is one row per key. Retention
|
||||
drops a row once its sample has aged out of the raw window, so a resource
|
||||
that stopped reporting disappears when its raw history does.
|
||||
"""
|
||||
__tablename__ = "plugin_metric_latest"
|
||||
|
||||
source_module: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||
resource_name: Mapped[str] = mapped_column(String(255), primary_key=True)
|
||||
metric_name: Mapped[str] = mapped_column(String(128), primary_key=True)
|
||||
value: Mapped[float] = mapped_column(Float, nullable=False)
|
||||
recorded_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
|
||||
class PluginMetricHourly(Base):
|
||||
"""Hourly rollup of plugin_metrics — the coarse series that retention rolls
|
||||
raw samples into before pruning them, so multi-day/week history stays cheap.
|
||||
|
||||
Reference in New Issue
Block a user