fix(traefik): a router fails on 5xx only once it answered a few server errors (#5865)
One 502 among about 70 requests read steward@swarm critical at 1.4%: past the 30-request minimum and far past the 0.1% cutoff, yet one error. A new Settings → Thresholds field, "Fewest server errors to fail" (traefik.threshold.min_5xx, default 3), must also be reached in the 15-minute window; the 0.1% cutoff still catches a busy router. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -107,7 +107,7 @@ The full Traefik page at `/plugins/traefik/` reads top to bottom, for the time r
|
||||
|
||||
A router is graded on its last 15 minutes against Settings → Thresholds. When a problem started is read from the router's history over the range.
|
||||
|
||||
Once a router has 5 days of history, its 4xx share and p95 latency are judged against its own normal for that hour (see [Normals and anomalies](../architecture.md)) instead of the fixed cutoffs. One over the cutoff but within its normal is healthy, and the healthy table says it is usual for that router (an error-pages service that answers 4xx all day, a login wall's steady 401s). One out of its normal is a warning that says what is usual, and a metric anomaly to review; the Traefik page lists the routers' anomalies of the last 30 days. How far out counts is set on Settings → Thresholds → Normals (2 points of 4xx share, 50 ms of p95 by default). A 4xx or 5xx share over fewer requests in the 15 minutes than Settings → Thresholds allows (30 by default) isn't judged at all: two 500s of sixteen requests would otherwise read 12% and critical. The 5xx share is a failure and keeps its fixed cutoff whatever the router's history. Latency is learned only from #5755's per-scrape figures, so its normal arrives 5 days after that release.
|
||||
Once a router has 5 days of history, its 4xx share and p95 latency are judged against its own normal for that hour (see [Normals and anomalies](../architecture.md)) instead of the fixed cutoffs. One over the cutoff but within its normal is healthy, and the healthy table says it is usual for that router (an error-pages service that answers 4xx all day, a login wall's steady 401s). One out of its normal is a warning that says what is usual, and a metric anomaly to review; the Traefik page lists the routers' anomalies of the last 30 days. How far out counts is set on Settings → Thresholds → Normals (2 points of 4xx share, 50 ms of p95 by default). A 4xx or 5xx share over fewer requests in the 15 minutes than Settings → Thresholds allows (30 by default) isn't judged at all: two 500s of sixteen requests would otherwise read 12% and critical. A 5xx share past its cutoff is failing only once the router answered at least 3 server errors in those 15 minutes (Settings → Thresholds, "Fewest server errors to fail"): one 502 among 70 requests is 1.4%, but it is one error. The 5xx share is a failure and keeps its fixed cutoff whatever the router's history. Latency is learned only from #5755's per-scrape figures, so its normal arrives 5 days after that release.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user