feat(anomalies): a metric out of its normal is an anomaly to review (#5740)
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 1m24s
CI / unit (push) Successful in 1m37s
CI / publish (push) Skipped

#5740 "A metric that departs from its normal is recorded as an anomaly to
review, and is findable after it ends", milestone 513.

- Every minute each learned key's last 15 minutes is judged against its
  hour's normal: above the usual hold x1.25 and by the metric's floor opens
  an occurrence, which ends when the level comes back.
- metric_anomalies holds one row per item to review; a repeat while it
  still waits joins it as an occurrence (metric_anomaly_spans).
- An ended row nobody reviews within anomalies.review_days (14, Settings ->
  Thresholds) lapses; an ongoing one never does.
- Learning skips every hour an occurrence covers unless its anomaly was
  marked normal: no normal is learned without review.
- Alerts -> Anomalies lists those to review first, then the reviewed and
  lapsed under day dividers; marking is operator/admin and audited.
- The Overview widget and get_overview warn of open ones and count those
  to review; the host page shows its own; the weekly report lists the
  week's and the lapsed; MCP list_anomalies and review_anomaly.
- Reviewed and lapsed rows are pruned at data.retention_days, and kept
  in backups like alert history.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-14 13:41:57 -04:00
co-authored by Claude Opus 5.5
parent cddf7fc6c4
commit 5569f9934e
34 changed files with 1343 additions and 25 deletions
+17
View File
@@ -43,6 +43,23 @@ This week {{ d.alert_events.fired }} {{ "alert" if d.alert_events.fired == 1 els
<tr><td style="padding:0 24px;font:14px/1.5 {{ c.font }};color:{{ c.muted }};">No monitors are enabled.</td></tr>
{% endif %}
{% if anomalies.seen or anomalies.lapsed %}
{{ section("Metric anomalies this week", base ~ "/alerts/anomalies" if base else "") }}
{% call rows() %}
{% for a in anomalies.seen %}
{{ row([badge(a.tone, a.review), a.name ~ " — " ~ a.when, a.level]) }}
{% endfor %}
{% for a in anomalies.lapsed %}
{{ row([badge(a.tone, a.review), a.name ~ " — lapsed unreviewed; " ~ a.when, a.level]) }}
{% endfor %}
{% endcall %}
{% if anomalies.to_review %}
<tr><td style="padding:8px 24px 0 24px;font:13px/1.5 {{ c.font }};color:{{ c.muted }};">
{{ anomalies.to_review }} {{ "anomaly waits" if anomalies.to_review == 1 else "anomalies wait" }} to be marked normal or real.
</td></tr>
{% endif %}
{% endif %}
{% if d.top_routers %}
{{ section("Busiest Traefik routers, last 24 hours", base ~ "/plugins/traefik/" if base else "") }}
{% call rows() %}