Files
FabledSteward/steward/core/counter_rates.py
T
bvandeusenandClaude Opus 5.5 3cb2a95054
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 1m12s
CI / unit (push) Failing after 1m27s
CI / publish (push) Skipped
feat(prometheus): scrape any /metrics endpoint into Steward's metrics, with pages (#5668, #5669)
A bundled prometheus plugin, on by default like the other protocol plugins.
Targets are rows edited on its page: address, bearer token (encrypted),
interval, keep/leave-out name patterns and a series cap. Steward does the
filtering; the endpoint serves everything (Archives' ruling).

- A 5 s tick scrapes due targets at once, each within a 30 s deadline, and
  writes the outcome to its row and its series through record_metrics, so
  alert rules can watch any of them.
- Gauges are stored as they are; counters (and a histogram's or summary's
  _sum/_count) as <name>_per_sec rates from the second scrape. Buckets and
  quantiles are skipped. A labelled series is stored under
  "<target>:k=v,…". Each target also records is_up, scrape_ms, series_kept
  and series_capped.
- Pages: targets not answering first, with why; a target's series grouped
  by metric with their newest values; a chart per series; the editor with
  Test, which scrapes once and says what would be stored.
- The counter-to-rate keeper moves to steward/core/counter_rates.py; SNMP's
  RateKeeper now scales its rate on top of it.
- metric_value reads Prometheus's _seconds as a span and _per_sec as a rate.
- The catalog guard treats an empty list as a dynamic source (snmp,
  prometheus), as the rule form does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:04:01 -04:00

35 lines
1.2 KiB
Python

"""A counter's previous sample, to turn the next one into a per-second rate.
Scrapers that read monotonic counters (SNMP octets, Prometheus *_total)
store the rate, not the running total. Keys are tuples whose first element
is the owner (a device, a target), so forgetting an owner drops all of its
counters at once.
"""
from __future__ import annotations
from datetime import datetime
class CounterRates:
"""In memory: after a restart the first sample of each counter gives no
rate and the second does. A sample lower than the one before (the source
restarted, or a 32-bit counter wrapped) gives no rate rather than a huge
negative or wrapped one."""
def __init__(self) -> None:
self._last: dict[tuple, tuple[float, datetime]] = {}
def rate(self, key: tuple, value: float, at: datetime) -> float | None:
prev = self._last.get(key)
self._last[key] = (value, at)
if prev is None:
return None
seconds = (at - prev[1]).total_seconds()
if seconds <= 0 or value < prev[0]:
return None
return (value - prev[0]) / seconds
def forget(self, owner) -> None:
for key in [k for k in self._last if k[0] == owner]:
del self._last[key]