feat(retention): every growing table is pruned; the audit log has its own window; charts read the raw window from Settings (#5155)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 47s
CI / unit (push) Successful in 1m10s
CI / publish (push) Successful in 55s

- prune_by_age(column) registers a time-series column with the hourly cleanup;
  Traefik (metrics, access summaries, global stats) and UniFi (WAN, client,
  WLAN, DPI, events, alarms, speedtests) register theirs in setup().
- Alert events join monitor results and Ansible runs on data.retention_days,
  read from Settings each run; Ansible run log files older than it are deleted.
- audit.retention_days (365) keeps the audit log longer, set on Settings > General.
- metric_series.points reads metrics.retention.raw_days, so the chart's
  raw/rollup seam is where the rollup put it (RAW_DAYS removed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-06 21:05:22 -04:00
co-authored by Claude Opus 5.5
parent 2f208a84fb
commit 9fdd67a675
9 changed files with 214 additions and 20 deletions
+6 -5
View File
@@ -13,10 +13,6 @@ from sqlalchemy import func, or_, select, text
from steward.core.time_range import bucket_seconds
from steward.models.metrics import PluginMetric, PluginMetricHourly
# Raw rows older than this are rolled into plugin_metrics_hourly and deleted
# (see core/metrics_retention.py), so a range reaching past it reads the rollup.
RAW_DAYS = 7
# Loose index scans: step through the distinct leading values of the
# (source_module, resource_name, …) index one probe at a time instead of
# scanning every sample. Postgres has no native skip scan before 18.
@@ -79,9 +75,14 @@ async def points(db, source_module: str, resource: str, metric: str, since: date
The part of the range older than the raw window comes from the hourly
rollup; the rest is bucket-averaged in SQL (buckets no wider than an hour
when both are present, so the series is evenly grained across the seam).
The raw window is metrics.retention.raw_days, the setting the rollup in
core/metrics_retention.py reads, so the seam is where the rollup put it.
"""
from steward.core.settings import get_setting
raw_days = int(await get_setting(db, "metrics.retention.raw_days") or 7)
now = datetime.now(timezone.utc)
raw_cutoff = (now - timedelta(days=RAW_DAYS)).replace(minute=0, second=0, microsecond=0)
raw_cutoff = (now - timedelta(days=raw_days)).replace(minute=0, second=0, microsecond=0)
out: list[tuple[datetime, float]] = []
if since < raw_cutoff: