feat(traefik): keep each router's 4xx and 5xx responses by status code (#5711)
CI / lint (push) Failing after 2s
CI / integration (push) Successful in 1m19s
CI / unit (push) Successful in 1m30s
CI / publish (push) Skipped

The scraper folded traefik_service_requests_total{code} into two shares and
dropped the code, so a problem couldn't say whether it was scanners (404), a
login wall (401/403) or the app down (502/503/504). Each scrape now stores the
responses per router and code that occurred (traefik_status_codes, migration
traefik_006), pruned with the router metrics and left out of backups (System
180 ruling: Traefik history is not backed up). A problem card lists its codes
over the last 15 minutes, most first, or says none occurred, or that codes
aren't recorded yet.

A counter series new since the previous scrape now counts from 0 instead of
being skipped: Traefik creates a router's 502 series at its first 502, so a
first burst of 5xx read as 0% until it happened again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-14 11:07:19 -04:00
co-authored by Claude Opus 5.5
parent 627193b00a
commit 81995051dc
11 changed files with 243 additions and 10 deletions
@@ -42,6 +42,15 @@
</div>
<p class="problem-detail" style="margin-bottom:0.5rem;">{{ r.why }}
{%- if r.since_before %}, all of the last {{ current_range }}{% elif r.since %}, started {{ r.since|ago }}{% endif %}</p>
{% if r.codes %}
<p class="field-hint" style="margin:0 0 0.5rem;">Last 15 min:
{% for code, n in r.codes[:5] %}<span class="tone-{{ 'crit' if code[:1] == '5' else 'warn' }}">{{ code }}</span> ×{{ "{:,}".format(n) }}{{ ", " if not loop.last }}{% endfor %}
{%- if r.codes|length > 5 %}, {{ r.codes|length - 5 }} more code{{ "s" if r.codes|length - 5 != 1 }}{% endif %}</p>
{% elif codes_known %}
<p class="field-hint" style="margin:0 0 0.5rem;">No 4xx or 5xx responses in the last 15 min.</p>
{% else %}
<p class="field-hint" style="margin:0 0 0.5rem;">Status codes not recorded yet: Steward keeps them from this version on.</p>
{% endif %}
<table class="table" style="font-size:0.85rem;">
<tr><td class="field-hint">Req/s</td><td>{{ "%.2f"|format(r.latest.request_rate) }}</td><td>{{ r.sparkline_req|safe }}</td></tr>
<tr><td class="field-hint">Bandwidth</td><td>{{ r.latest.response_bytes_rate|rate }}</td><td>{{ r.sparkline_bytes|safe }}</td></tr>