feat(host_agent): a dropped ingest says which step of the metric write it stalled in (#5149)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 48s
CI / unit (push) Successful in 1m11s
CI / publish (push) Successful in 1m8s

The first instrumented drop on dev read "dropped after 10.0s … (completed:
parse=0.1s)": the whole stall is inside record_metrics. It now marks its
three steps (flush of the rows, the rule and state reads, the per-sample
evaluation) on the caller's phases, and the drop line carries the request's
samples and metric row count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-06 18:34:54 -04:00
co-authored by Claude Opus 5.5
parent edea8482b5
commit f802c9f153
4 changed files with 64 additions and 6 deletions
+5 -4
View File
@@ -250,8 +250,8 @@ async def ingest():
# Through the alert pipeline, so rules on host metrics can fire.
from steward.core.alerts import record_metrics
await record_metrics(session, metric_rows)
phases.mark("metrics")
phases.size = f"samples={accepted} metric_rows={len(metric_rows)}"
await record_metrics(session, metric_rows, phases)
# Hand host-scoped container data to the docker plugin if it's enabled
# (opportunistic synergy via the capability registry — no hard import,
@@ -331,6 +331,7 @@ class _Phases:
def __init__(self):
self.start = self.last = time.monotonic()
self.took: dict[str, float] = {}
self.size = ""
def mark(self, name: str) -> None:
now = time.monotonic()
@@ -339,9 +340,9 @@ class _Phases:
def cancelled(self, who: str) -> None:
current_app.logger.warning(
"host_agent ingest: dropped after %.1fs for token=%s… (completed: %s)",
"host_agent ingest: dropped after %.1fs for token=%s… (completed: %s) %s",
time.monotonic() - self.start, who,
" ".join(f"{k}={v:.1f}s" for k, v in self.took.items()) or "nothing")
" ".join(f"{k}={v:.1f}s" for k, v in self.took.items()) or "nothing", self.size)
def report(self, host: str, samples: int, metrics: int, containers: int, log_lines: int) -> None:
total = time.monotonic() - self.start