feat(traefik): routers are graded against their own normal (#5741)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m18s
CI / unit (push) Successful in 1m31s
CI / publish (push) Successful in 1m3s

#5741 "Traefik routers are graded against their own normal", milestone 513.

- A router's 4xx share and p95 learn their normal (learn_normal in the
  plugin's setup). Once a router has 5 days of history, a reading over the
  fixed cutoff but inside its normal is healthy, and the healthy table says
  it is usual for that router. One out of its normal is a warning that says
  what is usual, and a metric anomaly.
- The 5xx share is a failure: fixed cutoff always, whatever the history or
  traffic (System 182's ruling).
- A 4xx share over fewer requests than Settings -> Thresholds' new "Fewest
  requests to judge 4xx" (30 per 15 minutes) isn't graded on the page, and
  isn't judged for anomalies.
- Core: learn_normal(gate=(metric, least)) judges a key only while its
  resource's gate metric averages at least `least`; `least` may be a
  function, so it follows a setting. learn_normal(since=...) learns only
  from history after a moment: Traefik latency learns from #5755's
  per-scrape figures only.
- The Traefik page lists the routers' metric anomalies of the last 30 days
  (_anomaly_card.html). The dashboard widget keeps its why to problems.
- Thresholds and docs say when the cutoffs give way to a normal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-14 15:48:34 -04:00
co-authored by Claude Opus 5.5
parent 98a0ced5eb
commit 9f14620271
12 changed files with 321 additions and 46 deletions
+6
View File
@@ -97,6 +97,9 @@ DEFAULTS: dict[str, Any] = {
# than this share of 4xx is worth a look (#5380).
"traefik.threshold.error_5xx_pct": 0.1,
"traefik.threshold.error_4xx_pct": 1.0,
# A router's 4xx share over fewer requests than this in 15 minutes isn't
# graded, on the page or for anomalies: one 404 of three is 33% (#5741).
"traefik.threshold.min_requests": 30,
# Degraded/critical cutoffs for metric coloring (warn=amber, crit=red).
# cpu/mem/disk/load are percentages (load is load-per-core %); temp in °C.
# uptime is "higher is better" so its warn/crit are floors. Ping latency
@@ -534,6 +537,9 @@ def to_thresholds_cfg(settings: dict[str, Any]) -> dict:
"crit": g("traefik.threshold.error_5xx_pct", 0.1), "dir": "high"},
"error_4xx": {"warn": g("traefik.threshold.error_4xx_pct", 1.0),
"crit": g("traefik.threshold.error_4xx_pct", 1.0), "dir": "high"},
# Fewer requests than this in 15 minutes and a router's 4xx share isn't graded.
"router_requests": {"warn": g("traefik.threshold.min_requests", 30),
"crit": g("traefik.threshold.min_requests", 30), "dir": "low"},
}