feat(norms): Steward learns each metric's normal per hour of day, then week (#5739)
A metric registered with learn_normal() gets, for every resource reporting it, a usual hold (the 90th percentile across past days of each hour's p95) and a usual peak for each slot: - an hour of the day once 5 days of history exist; - an hour of the week once history reaches back 3 weeks, so weekly jobs are learned too. Each slot takes in the hours either side, so a job that runs a little late stays normal. One SQL statement relearns everything hourly into metric_norms, from the rollup plus the raw window not rolled up yet; norms_at() reads the slot a moment falls in. Nothing is registered yet: Traefik (#5741) and hosts (#5742) opt in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -445,6 +445,10 @@ def _register_core_tasks(app: Quart) -> None:
|
||||
from .backup.job import tick
|
||||
await tick(app)
|
||||
|
||||
async def run_metric_norms():
|
||||
from .core.norms import run_norms
|
||||
await run_norms(app)
|
||||
|
||||
async def run_email_outbox():
|
||||
from .core.outbox import send_due
|
||||
await send_due(app)
|
||||
@@ -507,6 +511,15 @@ def _register_core_tasks(app: Quart) -> None:
|
||||
interval_seconds=cleanup_interval,
|
||||
run_on_startup=False,
|
||||
),
|
||||
# Each learned metric's normal per hour (#5739). Hourly, since a slot
|
||||
# gains one hour of history an hour; the table outlives a restart, so
|
||||
# there is nothing to rebuild at startup.
|
||||
ScheduledTask(
|
||||
name="metric_norms",
|
||||
coro_factory=run_metric_norms,
|
||||
interval_seconds=3600,
|
||||
run_on_startup=False,
|
||||
),
|
||||
])
|
||||
|
||||
# Pull Ansible git sources as each falls due. One task reads the sources
|
||||
|
||||
Reference in New Issue
Block a user