fix(metrics): seed the latest table in the background, not in the migration (#5208)
Migration 0030 seeded plugin_metric_latest from the whole history: two index probes per distinct metric key. On the live instance that held the first boot for 7+ minutes with Steward down. The migration now only creates the table. Live sources fill it on their next sample, and backfill_latest runs after startup for resources that have history but no latest rows. Per resource it does one backward probe for the last sample, then takes the newest value per metric within the hour before it. Resources that already have rows are skipped, so later boots only walk the resource names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -23,32 +23,10 @@ def upgrade() -> None:
|
||||
sa.Column("value", sa.Float, nullable=False),
|
||||
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
|
||||
)
|
||||
# Seed from history without a full scan: step through the distinct keys of
|
||||
# the (source, resource, metric, recorded_at) index one probe at a time,
|
||||
# then take each key's newest row with a backward probe of the same index.
|
||||
op.execute("""
|
||||
WITH RECURSIVE k AS (
|
||||
(SELECT source_module, resource_name, metric_name FROM plugin_metrics
|
||||
ORDER BY source_module, resource_name, metric_name LIMIT 1)
|
||||
UNION ALL
|
||||
SELECT n.source_module, n.resource_name, n.metric_name
|
||||
FROM k, LATERAL (
|
||||
SELECT p.source_module, p.resource_name, p.metric_name FROM plugin_metrics p
|
||||
WHERE (p.source_module, p.resource_name, p.metric_name)
|
||||
> (k.source_module, k.resource_name, k.metric_name)
|
||||
ORDER BY p.source_module, p.resource_name, p.metric_name LIMIT 1
|
||||
) n
|
||||
)
|
||||
INSERT INTO plugin_metric_latest
|
||||
(source_module, resource_name, metric_name, value, recorded_at)
|
||||
SELECT k.source_module, k.resource_name, k.metric_name, l.value, l.recorded_at
|
||||
FROM k, LATERAL (
|
||||
SELECT p.value, p.recorded_at FROM plugin_metrics p
|
||||
WHERE p.source_module = k.source_module AND p.resource_name = k.resource_name
|
||||
AND p.metric_name = k.metric_name
|
||||
ORDER BY p.recorded_at DESC LIMIT 1
|
||||
) l
|
||||
""")
|
||||
# Not seeded here: a seed over the whole history held the first boot for
|
||||
# minutes. Live sources fill it on their next sample, and
|
||||
# metric_series.backfill_latest fills the quiet ones in the background
|
||||
# after startup.
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
|
||||
Reference in New Issue
Block a user