fix(traefik): latency is each scrape's own percentile, every label set summed (#5755)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m23s
CI / unit (push) Successful in 1m27s
CI / publish (push) Successful in 17s

#5755 "Traefik latency was a percentile of every response since Traefik
started, over unsummed label sets". _percentile_ms read the duration
histogram as it stood, which counts from Traefik's start, so p50/p95/p99 were
lifetime figures that barely moved. It also walked every code/method label
set's buckets as one unsummed list, so equal counts from two sets returned
an edge exactly: nextcloud's flat 100.0 ms p95.

service_buckets() now takes each bucket's rise since the previous scrape,
summed per le over every label set. A scrape with no responses, and the
first scrape, record no latency: traefik_007_latency_nullable lets the
columns be NULL, the batch skips the figure, and the page shows "—".

Found while preparing #5741 "Traefik routers are graded against their own
normal", which grades p95 against a learned normal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-14 14:45:07 -04:00
co-authored by Claude Opus 5.5
parent cd17def3ba
commit 98a0ced5eb
8 changed files with 143 additions and 39 deletions
+2 -2
View File
@@ -204,7 +204,7 @@ async def latest_per_router(db) -> list[SimpleNamespace]:
for r in rows:
by_router.setdefault(r.resource_name, {})[r.metric_name] = r
return [SimpleNamespace(router_name=name, scraped_at=max(m.recorded_at for m in figures.values()),
**{k: figures[k].value if k in figures else 0.0 for k in ROUTER_METRICS})
**{k: figures[k].value if k in figures else None for k in ROUTER_METRICS})
for name, figures in by_router.items() if "request_rate" in figures]
@@ -299,7 +299,7 @@ async def rows():
else (None, False))
r["sparkline_req"] = _sparkline([p.request_rate for p in points])
r["sparkline_bytes"] = _sparkline([p.response_bytes_rate for p in points])
r["sparkline_p95"] = _sparkline([p.latency_p95_ms for p in points])
r["sparkline_p95"] = _sparkline([p.latency_p95_ms for p in points if p.latency_p95_ms is not None])
r["sparkline_4xx"] = _sparkline([p.error_rate_4xx_pct for p in points])
r["sparkline_5xx"] = _sparkline([p.error_rate_5xx_pct for p in points])
phases.mark("sparklines")