feat(docker): retention + hourly rollup for metrics/events with Settings windows
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 47s
CI / integration (push) Successful in 2m19s
CI / publish (push) Successful in 1m6s

Bounds Docker time-series growth (the main scaling concern). New
docker_metrics_hourly table + docker_006 migration; a plugin retention module
(docker.run_retention capability) rolls raw docker_metrics older than the raw
window into hourly averages (idempotent upsert), deletes the rolled raw rows,
then prunes stale rollups + lifecycle events. Core cleanup.py drives it each
hourly run via the capability (no plugin-model import), reading the three
retention windows fresh from settings so changes apply without restart (rule 25).

Settings → "Thresholds & Retention" gains a Docker retention card (raw /
rolled-up / events windows, working defaults 7/90/30 days). Unit tests cover the
hour-aligned cutoff/bucketing helpers; integration test exercises the real
rollup-average + prune across both windows.

Milestone 77 task #941.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
This commit is contained in:
2026-06-18 21:40:57 -04:00
parent 578cc33cc0
commit faecac3ec6
11 changed files with 445 additions and 4 deletions
+35 -2
View File
@@ -16,9 +16,11 @@ logger = logging.getLogger(__name__)
async def run_cleanup(app: "Quart") -> None:
"""Delete rows older than DATA_RETENTION_DAYS from time-series tables."""
"""Delete rows older than DATA_RETENTION_DAYS from time-series tables, then
run Docker-specific rollup + retention (delegated to the docker plugin)."""
retention_days: int = app.config.get("DATA_RETENTION_DAYS", 90)
cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
now = datetime.now(timezone.utc)
cutoff = now - timedelta(days=retention_days)
async with app.db_sessionmaker() as session:
async with session.begin():
@@ -32,3 +34,34 @@ async def run_cleanup(app: "Quart") -> None:
)
if result.rowcount:
logger.info(f"Pruned {result.rowcount} rows from {model.__tablename__}")
await _run_docker_retention(session, now)
async def _run_docker_retention(session, now: datetime) -> None:
"""Drive the docker plugin's rollup + prune via its capability, if loaded.
Windows are read fresh from settings each run (rule 25 — a change in the
Settings UI takes effect on the next hourly cleanup, no restart). Kept in its
own transaction so a docker-side failure can't roll back the generic prune
above. No-op when the docker plugin is disabled (capability absent).
"""
from steward.core.capabilities import has_capability, invoke_capability
if not has_capability("docker.run_retention"):
return
from steward.core.settings import get_setting
from steward.models.users import UserRole
# Reads + rollup/prune share one transaction — get_setting's SELECT would
# otherwise autobegin one, making a later session.begin() raise.
async with session.begin():
raw_days = int(await get_setting(session, "docker.retention.metrics_raw_days") or 7)
rollup_days = int(await get_setting(session, "docker.retention.metrics_rollup_days") or 90)
events_days = int(await get_setting(session, "docker.retention.events_days") or 30)
counts = await invoke_capability(
"docker.run_retention", UserRole.viewer, session,
events_days=events_days, metrics_raw_days=raw_days,
metrics_rollup_days=rollup_days, now=now,
)
if counts and any(counts.values()):
logger.info("Docker retention: %s", counts)
+6
View File
@@ -78,6 +78,12 @@ DEFAULTS: dict[str, Any] = {
"thresholds.load_warn": 80, "thresholds.load_crit": 100,
"thresholds.temp_warn": 70, "thresholds.temp_crit": 85,
"thresholds.uptime_warn": 99.0, "thresholds.uptime_crit": 95.0,
# Docker time-series retention (rule 25 — tunable, no restart). Raw 30s
# samples are heavy, so keep a short raw window then roll up to hourly
# averages kept much longer; lifecycle events are light, keep a month.
"docker.retention.metrics_raw_days": 7,
"docker.retention.metrics_rollup_days": 90,
"docker.retention.events_days": 30,
"plugins.index_url": "https://git.fabledsword.com/bvandeusen/Steward-plugins/raw/branch/main/index.yaml",
# Default-enabled plugins. These are the generic, non-vendor-specific
# bundled plugins (protocols/standards, not a single product) — useful on