feat(traefik): routers are graded against their own normal (#5741)

#5741 "Traefik routers are graded against their own normal", milestone 513.

- A router's 4xx share and p95 learn their normal (learn_normal in the
  plugin's setup). Once a router has 5 days of history, a reading over the
  fixed cutoff but inside its normal is healthy, and the healthy table says
  it is usual for that router. One out of its normal is a warning that says
  what is usual, and a metric anomaly.
- The 5xx share is a failure: fixed cutoff always, whatever the history or
  traffic (System 182's ruling).
- A 4xx share over fewer requests than Settings -> Thresholds' new "Fewest
  requests to judge 4xx" (30 per 15 minutes) isn't graded on the page, and
  isn't judged for anomalies.
- Core: learn_normal(gate=(metric, least)) judges a key only while its
  resource's gate metric averages at least `least`; `least` may be a
  function, so it follows a setting. learn_normal(since=...) learns only
  from history after a moment: Traefik latency learns from #5755's
  per-scrape figures only.
- The Traefik page lists the routers' metric anomalies of the last 30 days
  (_anomaly_card.html). The dashboard widget keeps its why to problems.
- Thresholds and docs say when the cutoffs give way to a normal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

steward/templates/settings/thresholds.html

@@ -40,7 +40,7 @@
   {{ pair("Disk usage",   "disk_warn", "thresholds.disk_warn", "disk_crit", "thresholds.disk_crit", "%", "Filesystem usage.") }}
   {{ pair("Load",         "load_warn", "thresholds.load_warn", "load_crit", "thresholds.load_crit", "% per core", "Load average normalized by core count (100% = one full core per core).") }}
   {{ pair("Temperature",  "temp_warn", "thresholds.temp_warn", "temp_crit", "thresholds.temp_crit", "°C", "Hottest reported sensor per host.") }}
-  {{ pair("Response time", "latency_warn", "ping.threshold.good_ms", "latency_crit", "ping.threshold.warn_ms", "ms", "How long a monitor's check takes to answer. Also grades Traefik routers' p95 and the UniFi WAN round trip.") }}
+  {{ pair("Response time", "latency_warn", "ping.threshold.good_ms", "latency_crit", "ping.threshold.warn_ms", "ms", "How long a monitor's check takes to answer. Also grades Traefik routers' p95 (until a router has its own normal) and the UniFi WAN round trip.") }}
   {{ pair("Uptime / SLA", "uptime_warn", "thresholds.uptime_warn", "uptime_crit", "thresholds.uptime_crit", "%", "Lower is worse — amber below warn, red below crit.", step="0.1") }}
   {{ pair("TLS certificate expiry", "tls_warn", "thresholds.tls_warn_days", "tls_crit", "thresholds.tls_crit_days", "days left", "HTTPS monitors whose certificate expires within these many days are flagged. Lower is worse.") }}
   {{ pair("Clock drift", "clock_warn", "thresholds.clock_warn_s", "clock_crit", "thresholds.clock_crit_s", "seconds", "How far a host's clock may be from Steward's, either way, before its page and the Hosts list call it out.") }}
@@ -69,13 +69,21 @@
     <label for="error_5xx_pct">Server errors (5xx) <span style="color:var(--text-muted);font-size:0.8rem;">(% of responses)</span></label>
     <input type="number" id="error_5xx_pct" name="error_5xx_pct" min="0" max="100" step="0.01"
            value="{{ settings['traefik.threshold.error_5xx_pct'] }}" style="max-width:110px;margin-top:0.25rem;">
-    <div style="font-size:0.78rem;color:var(--text-muted);margin-top:0.3rem;">Above this share, a router is failing (red).</div>
+    <div style="font-size:0.78rem;color:var(--text-muted);margin-top:0.3rem;">Above this share, a router is failing (red), whatever its history.</div>
   </div>
   <div class="form-group" style="margin-bottom:0.25rem;">
     <label for="error_4xx_pct">Client errors (4xx) <span style="color:var(--text-muted);font-size:0.8rem;">(% of responses)</span></label>
     <input type="number" id="error_4xx_pct" name="error_4xx_pct" min="0" max="100" step="0.1"
            value="{{ settings['traefik.threshold.error_4xx_pct'] }}" style="max-width:110px;margin-top:0.25rem;">
-    <div style="font-size:0.78rem;color:var(--text-muted);margin-top:0.3rem;">Above this share, a router is worth a look (amber): missing routes, failed sign-ins.</div>
+    <div style="font-size:0.78rem;color:var(--text-muted);margin-top:0.3rem;">Above this share, a router is worth a look (amber): missing routes, failed sign-ins.
+      Once a router has 5 days of history it is judged against its own normal for the hour instead, as is its p95,
+      so one that always answers some 4xx stops being listed.</div>
+  </div>
+  <div class="form-group" style="margin:1.1rem 0 0.25rem;">
+    <label for="min_requests">Fewest requests to judge 4xx <span style="color:var(--text-muted);font-size:0.8rem;">(in 15 minutes)</span></label>
+    <input type="number" id="min_requests" name="min_requests" min="0" step="1"
+           value="{{ settings['traefik.threshold.min_requests'] }}" style="max-width:110px;margin-top:0.25rem;">
+    <div style="font-size:0.78rem;color:var(--text-muted);margin-top:0.3rem;">Below this, a router's 4xx share isn't judged, on its page or as an anomaly: one 404 of three requests is 33%. 5xx is always judged.</div>
   </div>
 </div>
 {% endif %}