fix(host_agent): a post that reached Steward counts as the agent seen, even if storing it failed (#5514)
At 20:25-20:39Z the NFS storage under steward_db slowed badly (checkpoint sync 4.6 s, then 70.7 s with one file at 21.9 s, against the usual 0.002 s). Ingest commits stalled past the agent's 10 s timeout and were dropped, and last_seen_at, written in that same transaction, went with them. Agents posting every few seconds read as silent for 3 minutes, and "Agent offline" fired and emailed for all three hosts. Ingest now notes each agent's arrival in memory right after the (read-only) registration lookup, before any write; seconds_since_seen counts from the later of last_seen_at and that arrival. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -24,6 +24,7 @@ from steward.core.time_range import parse_range, RANGE_OPTIONS
|
||||
from steward.models.hosts import Host
|
||||
from steward.models.metrics import PluginMetric
|
||||
from .models import HostAgentRegistration
|
||||
from .scheduler import arrivals
|
||||
# Query helpers live in a model-free module so integration tests can import them
|
||||
# without the plugin-loader double-registration guard tripping (see metrics_query).
|
||||
from .metrics_query import (
|
||||
@@ -242,6 +243,8 @@ async def ingest():
|
||||
)).scalar_one_or_none()
|
||||
if host is None:
|
||||
return _error(401, "invalid_token")
|
||||
# The agent reached us: that holds even if storing its post fails (#5514).
|
||||
arrivals(current_app)[reg.host_id] = received_at
|
||||
|
||||
accepted = 0
|
||||
metric_rows: list[PluginMetric] = []
|
||||
|
||||
Reference in New Issue
Block a user