feat(traefik): keep each router's 4xx and 5xx responses by status code (#5711)
CI / lint (push) Failing after 2s
CI / integration (push) Successful in 1m19s
CI / unit (push) Successful in 1m30s
CI / publish (push) Skipped

The scraper folded traefik_service_requests_total{code} into two shares and
dropped the code, so a problem couldn't say whether it was scanners (404), a
login wall (401/403) or the app down (502/503/504). Each scrape now stores the
responses per router and code that occurred (traefik_status_codes, migration
traefik_006), pruned with the router metrics and left out of backups (System
180 ruling: Traefik history is not backed up). A problem card lists its codes
over the last 15 minutes, most first, or says none occurred, or that codes
aren't recorded yet.

A counter series new since the previous scrape now counts from 0 instead of
being skipped: Traefik creates a router's 502 series at its first 502, so a
first burst of 5xx read as 0% until it happened again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-14 11:07:19 -04:00
co-authored by Claude Opus 5.5
parent 627193b00a
commit 81995051dc
11 changed files with 243 additions and 10 deletions
+9 -1
View File
@@ -51,11 +51,12 @@ async def _do_scrape(app: "Quart") -> None:
global _prev_metrics, _prev_time, _router_domains
from datetime import datetime, timezone
from .models import TraefikCert, TraefikGlobalStat, TraefikMetric
from .models import TraefikCert, TraefikGlobalStat, TraefikMetric, TraefikStatusCode
from .scraper import (
api_base,
compute_global_stats,
compute_router_metrics,
compute_status_codes,
extract_certs,
fetch_metrics,
fetch_router_domains,
@@ -81,6 +82,7 @@ async def _do_scrape(app: "Quart") -> None:
elapsed = now - _prev_time if _prev_time else 60.0
router_metrics = compute_router_metrics(current, _prev_metrics or None, elapsed)
global_stats = compute_global_stats(current, _prev_metrics or None)
status_codes = compute_status_codes(current, _prev_metrics or None)
cert_list = extract_certs(current)
_prev_metrics = current
@@ -125,6 +127,12 @@ async def _do_scrape(app: "Quart") -> None:
recorded_at=scraped_at,
)
# ── 4xx and 5xx responses by code (#5711) ─────────────────────────
for router_name, codes in status_codes.items():
for code, count in codes.items():
session.add(TraefikStatusCode(router_name=router_name, scraped_at=scraped_at,
code=code, count=count))
# ── Global stats ──────────────────────────────────────────────────
session.add(TraefikGlobalStat(scraped_at=scraped_at, **global_stats))