feat(traefik): scrape every Traefik replica behind the metrics URL and add them up (#5844)
Through one address Steward read whichever replica answered, and with host-mode ports behind a keepalived VIP that can be the replica serving nothing: since the 18:43 restart every scrape landed on Docker03's idle Traefik, which has no traefik_service_* series, so no router was stored. A metrics_url host name that resolves to several addresses (a swarm service's tasks.<service>, a Kubernetes headless Service, round-robin DNS) is now scraped address by address, with the original Host header and TLS name. Answers from one process are kept once; a replica that doesn't answer is skipped with a warning. Requests, bytes and latency buckets are summed over the replicas that have a baseline, so the sum's rise is the sum of each one's own rise; certificates are listed once; Traefik's own figures add connections and memory and keep the newest reload. A name with one address is one scrape, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -64,11 +64,13 @@ async def _do_scrape(app: "Quart") -> None:
|
||||
from .scraper import (
|
||||
Baselines,
|
||||
api_base,
|
||||
combine,
|
||||
compute_global_stats,
|
||||
compute_router_metrics,
|
||||
compute_status_codes,
|
||||
extract_certs,
|
||||
fetch_metrics,
|
||||
fetch_instances,
|
||||
merge_global_stats,
|
||||
unique_certs,
|
||||
fetch_error_page_services,
|
||||
fetch_router_domains,
|
||||
)
|
||||
@@ -77,7 +79,7 @@ async def _do_scrape(app: "Quart") -> None:
|
||||
metrics_url: str = app.config["PLUGINS"]["traefik"]["metrics_url"]
|
||||
|
||||
try:
|
||||
current = await fetch_metrics(metrics_url)
|
||||
scrapes = await fetch_instances(metrics_url)
|
||||
except Exception:
|
||||
logger.exception("Traefik scrape failed (url=%s)", metrics_url)
|
||||
return
|
||||
@@ -95,11 +97,12 @@ async def _do_scrape(app: "Quart") -> None:
|
||||
|
||||
if _baselines is None:
|
||||
_baselines = Baselines()
|
||||
previous, elapsed = _baselines.swap(current, time.monotonic())
|
||||
now = time.monotonic()
|
||||
current, previous, elapsed = combine([(scrape, *_baselines.swap(scrape, now)) for scrape in scrapes])
|
||||
router_metrics = compute_router_metrics(current, previous, elapsed)
|
||||
global_stats = compute_global_stats(current, previous)
|
||||
global_stats = merge_global_stats([compute_global_stats(scrape, None) for scrape in scrapes])
|
||||
status_codes = compute_status_codes(current, previous)
|
||||
cert_list = extract_certs(current)
|
||||
cert_list = unique_certs(scrapes)
|
||||
|
||||
scraped_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user