feat(traefik): a 4xx share is worth a look only once it is a lot of 4xx (#6105)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 1m29s
CI / integration (push) Successful in 1m42s
CI / publish (push) Successful in 1m9s

Three 404s on a quiet router read 7.5% and were raised as an anomaly; most of
the 4xx anomalies on review stood for 3 to 7 errors in 15 minutes. A new
Settings → Thresholds minimum, traefik.threshold.min_4xx (default 20, more
than one a minute held for the window), applies to both the router's grade
on its page and to anomaly detection, as min_5xx does for 5xx.

learn_normal takes `fewest`: for a share gated on a per-second rate, the
fewest events (rate × window × share) it must stand for to be judged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-19 16:02:17 -04:00
co-authored by Claude Opus 5.5
parent 4639ac6e52
commit b75b883326
11 changed files with 81 additions and 15 deletions
+5 -3
View File
@@ -22,15 +22,17 @@ def setup(app: "Quart") -> None:
prune_by_age(TraefikAccessSummary.period_end)
# Each router's 4xx share and p95 learn their normal (#5741), judged only
# while the router answers the Thresholds minimum of requests a window
# (read at each pass, so a saved change applies at once). Latency is
# (read at each pass, so a saved change applies at once), and the 4xx
# share only once it stands for the minimum of 4xx (#6105). Latency is
# learned from the scrape-by-scrape figures only: before #5755 it was a
# percentile of every response since Traefik started.
from datetime import datetime, timezone
from steward.core.norms import floor_setting, learn_normal
from .routes import STATE_WINDOW, min_requests
from .routes import STATE_WINDOW, min_4xx, min_requests
gate = ("request_rate",
lambda: min_requests(app.config.get("THRESHOLDS", {})) / STATE_WINDOW.total_seconds())
learn_normal("traefik", "error_rate_4xx_pct", floor_setting(app, "router_4xx_pct"), gate=gate)
learn_normal("traefik", "error_rate_4xx_pct", floor_setting(app, "router_4xx_pct"), gate=gate,
fewest=lambda: min_4xx(app.config.get("THRESHOLDS", {})))
learn_normal("traefik", "latency_p95_ms", floor_setting(app, "router_p95_ms"), gate=gate,
since=datetime(2026, 10, 14, 18, 47, tzinfo=timezone.utc))