feat(traefik): a 4xx share is worth a look only once it is a lot of 4xx (#6105)

Three 404s on a quiet router read 7.5% and were raised as an anomaly; most of
the 4xx anomalies on review stood for 3 to 7 errors in 15 minutes. A new
Settings → Thresholds minimum, traefik.threshold.min_4xx (default 20, more
than one a minute held for the window), applies to both the router's grade
on its page and to anomaly detection, as min_5xx does for 5xx.

learn_normal takes `fewest`: for a share gated on a per-second rate, the
fewest events (rate × window × share) it must stand for to be judged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

plugins/traefik/routes.py

@@ -65,6 +65,8 @@ def error_cutoffs(thresholds: dict) -> tuple[float, float]:
 # keeps its fixed cutoff whatever the history; it shares the 4xx share's
 # minimum of traffic (the operator, #5859), and is failing only once
 # min_5xx() errors were answered: one 502 of 70 requests is 1.4% (#5865).
+# A 4xx share is worth a look only once min_4xx() were answered: three 404s
+# on a quiet router read 7%, and some 4xx are part of any day (#6105).
 NORMED = ("error_rate_4xx_pct", "latency_p95_ms")
 
 
@@ -80,6 +82,13 @@ def min_5xx(thresholds: dict) -> float:
     return float(thresholds.get("router_5xx", {}).get("warn", 3))
 
 
+def min_4xx(thresholds: dict) -> float:
+    """Fewest 4xx answered over the STATE_WINDOW for a 4xx share to be worth a
+    look, on the page or as an anomaly (Settings → Thresholds,
+    traefik.threshold.min_4xx)."""
+    return float(thresholds.get("router_4xx", {}).get("warn", 20))
+
+
 def router_state(reading, thresholds: dict, over: str = "", norms: dict | None = None) -> tuple[str, str | None]:
     """(grade, why) for a router's reading (recent_window's, or one scrape's):
     crit while it serves 5xx past its cutoff, warn on a p95 past the latency
@@ -87,8 +96,9 @@ def router_state(reading, thresholds: dict, over: str = "", norms: dict | None =
     otherwise ok. `norms` ({metric: Norm}) are the router's normals for this
     hour: a p95 or 4xx share past its cutoff but within its normal is ok, and
     `why` then says it is usual. `over` names the span the reading covers.
-    Neither share is graded over fewer than min_requests() requests, and a
-    5xx share is failing only once about min_5xx() errors were answered."""
+    Neither share is graded over fewer than min_requests() requests, a 5xx
+    share is failing only once about min_5xx() errors were answered, and a
+    4xx share is worth a look only once about min_4xx() were."""
     norms = norms or {}
     error_5xx, error_4xx = error_cutoffs(thresholds)
     requests = getattr(reading, "requests", None)
@@ -106,7 +116,8 @@ def router_state(reading, thresholds: dict, over: str = "", norms: dict | None =
             return "warn", f"p95 latency is {p95:.0f} ms{over}" + (f", {said}" if said else "")
         usual.append(f"p95 {p95:.0f} ms is {said}")
     e4 = reading.error_rate_4xx_pct or 0
-    if enough and e4 > error_4xx:
+    errors4 = None if requests is None else round(requests * e4 / 100.0)
+    if enough and e4 > error_4xx and (errors4 is None or errors4 >= min_4xx(thresholds)):
         out, said = anom.against(e4, norms.get("error_rate_4xx_pct"),
                                   floor_of(thresholds, "router_4xx_pct"), "%")
         if out: