feat(timeline): maintenance, reboots, deploys, crash-loops and reviewed anomalies on the timeline (#6349)
The host agent writes each maintenance job once it ends, and each stretch a host was away (a reboot, a crash, lost contact, the agent restarting), kept up to date until it is back. Docker judges each snapshot: a unit on a new image, or a container recreated, is a deploy; deaths past the Thresholds crash-loop count are one crash-loop entry that later deaths extend; single restarts are recorded but hidden. An anomaly has one entry whose type follows its review: only those judged real are shown by default. A source whose record changes keeps one entry for it (record_or_update). Plugins register the records their entries point to, so an aged-out job or downtime reads "Details no longer kept". The first fill runs once per source (timeline.filled), the host agent's jobs and downtime included. get_anomaly's "what else was going on" reads the timeline for the host or resource instead of querying alerts, runs, maintenance, downtime and container deaths itself; Docker's own source keeps only starts, stops and health changes. A new kind, Observed, is for what Steward saw happen with no rule or person behind it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -15,9 +15,8 @@ from steward.mcp.tools.overview import finding
|
||||
from steward.models.hosts import Host
|
||||
from steward.models.users import UserRole
|
||||
|
||||
from . import cadence, downtime, maintenance, senders
|
||||
from .models import (HostAgentRegistration, HostDowntime, HostMaintenanceJob, HostMaintenancePlan,
|
||||
HostMaintenanceRun)
|
||||
from . import cadence, maintenance, senders
|
||||
from .models import HostAgentRegistration, HostMaintenanceJob, HostMaintenancePlan, HostMaintenanceRun
|
||||
|
||||
# A registration this old that has never checked in is a failed install, not one in progress.
|
||||
PENDING_GRACE = timedelta(minutes=30)
|
||||
@@ -354,31 +353,6 @@ async def mcp_overview(db) -> list[dict]:
|
||||
return out
|
||||
|
||||
|
||||
async def mcp_events(db, host_id: str, host_name: str, since, until) -> list[dict]:
|
||||
"""What the host agent knows happened on a host in a window (get_anomaly):
|
||||
maintenance jobs, and downtime (reboots, agent stops, lost contact)."""
|
||||
from steward.mcp.tools._events import event
|
||||
out = []
|
||||
jobs = (await db.execute(select(HostMaintenanceJob).where(
|
||||
HostMaintenanceJob.host_id == host_id, HostMaintenanceJob.created_at <= until,
|
||||
(HostMaintenanceJob.finished_at.is_(None)) | (HostMaintenanceJob.finished_at >= since))
|
||||
.order_by(HostMaintenanceJob.created_at).limit(200))).scalars()
|
||||
for job in jobs:
|
||||
out.append(event(job.created_at, "maintenance", f"Maintenance: {maintenance.summary(job)}",
|
||||
until=job.finished_at, ids={"job_id": job.id}))
|
||||
now = now_utc()
|
||||
rows = (await db.execute(select(HostDowntime).where(
|
||||
HostDowntime.host_id == host_id, HostDowntime.started_at <= until,
|
||||
(HostDowntime.ended_at.is_(None)) | (HostDowntime.ended_at >= since))
|
||||
.order_by(HostDowntime.started_at).limit(200))).scalars()
|
||||
for row in rows:
|
||||
seen = downtime.describe(row, now)
|
||||
out.append(event(row.started_at, "downtime",
|
||||
f"{seen['label']} ({int(seen['seconds'])} s{', still going' if seen['ongoing'] else ''})",
|
||||
until=row.ended_at))
|
||||
return out
|
||||
|
||||
|
||||
def register_tools(register) -> None:
|
||||
register(list_agents, role=UserRole.viewer, read_only=True)
|
||||
register(list_maintenance_plans, role=UserRole.viewer, read_only=True)
|
||||
|
||||
Reference in New Issue
Block a user