feat(traefik): routers are graded against their own normal (#5741)
#5741 "Traefik routers are graded against their own normal", milestone 513. - A router's 4xx share and p95 learn their normal (learn_normal in the plugin's setup). Once a router has 5 days of history, a reading over the fixed cutoff but inside its normal is healthy, and the healthy table says it is usual for that router. One out of its normal is a warning that says what is usual, and a metric anomaly. - The 5xx share is a failure: fixed cutoff always, whatever the history or traffic (System 182's ruling). - A 4xx share over fewer requests than Settings -> Thresholds' new "Fewest requests to judge 4xx" (30 per 15 minutes) isn't graded on the page, and isn't judged for anomalies. - Core: learn_normal(gate=(metric, least)) judges a key only while its resource's gate metric averages at least `least`; `least` may be a function, so it follows a setting. learn_normal(since=...) learns only from history after a moment: Traefik latency learns from #5755's per-scrape figures only. - The Traefik page lists the routers' metric anomalies of the last 30 days (_anomaly_card.html). The dashboard widget keeps its why to problems. - Thresholds and docs say when the cutoffs give way to a normal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -20,6 +20,19 @@ def setup(app: "Quart") -> None:
|
||||
prune_by_age(TraefikStatusCode.scraped_at)
|
||||
prune_by_age(TraefikGlobalStat.scraped_at)
|
||||
prune_by_age(TraefikAccessSummary.period_end)
|
||||
# Each router's 4xx share and p95 learn their normal (#5741), judged only
|
||||
# while the router answers the Thresholds minimum of requests a window
|
||||
# (read at each pass, so a saved change applies at once). Latency is
|
||||
# learned from the scrape-by-scrape figures only: before #5755 it was a
|
||||
# percentile of every response since Traefik started.
|
||||
from datetime import datetime, timezone
|
||||
from steward.core.norms import learn_normal
|
||||
from .routes import FLOOR_4XX, FLOOR_P95, STATE_WINDOW, min_requests
|
||||
gate = ("request_rate",
|
||||
lambda: min_requests(app.config.get("THRESHOLDS", {})) / STATE_WINDOW.total_seconds())
|
||||
learn_normal("traefik", "error_rate_4xx_pct", FLOOR_4XX, gate=gate)
|
||||
learn_normal("traefik", "latency_p95_ms", FLOOR_P95, gate=gate,
|
||||
since=datetime(2026, 10, 14, 18, 47, tzinfo=timezone.utc))
|
||||
|
||||
|
||||
def get_scheduled_tasks() -> list:
|
||||
|
||||
Reference in New Issue
Block a user