feat(timeline): record a change by hand from the Timeline, a host's page or MCP log_change (#6350)
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 1m20s
CI / unit (push) Failing after 1m33s
CI / publish (push) Skipped

A change Steward can't see happen (a config edited, a disk swapped) is
recorded by whoever made it: what was done, its area, when (in the reader's
own zone), the hosts, what it was done to and a detail. Its author or an
admin edits or deletes it; recording, editing and deleting are audited and
those audit rows stay off the timeline. log_change records one as
"<user> via MCP", editable by the token's user.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-24 21:26:02 -04:00
co-authored by Claude Opus 5.5
parent 2b97c36fd4
commit 329aa15e3d
15 changed files with 703 additions and 38 deletions
+2 -1
View File
@@ -78,7 +78,7 @@ Each role includes the tools of the roles above it in this list.
| Metric anomalies | `list_anomalies` (metrics that left their normal for the hour, to review first), `get_anomaly` (one anomaly with each occurrence against what is usual at that hour, the metric around it, what else happened on its host — alerts, Ansible runs, container deaths and starts, maintenance, reboots, other anomalies — and how earlier ones on the same metric were judged), `get_metric_normal` (the learned normal slot by slot, and the level that would be flagged), `get_anomaly_settings` (the review window and each floor) |
| Automation | `list_playbooks`, `get_playbook`, `list_inventory`, `list_ansible_runs`, `get_ansible_run`, `wait_for_run`, `list_schedules` |
| Audit | `list_audit_events` |
| Timeline | `get_timeline` (everything that happened, newest first, in one list: alerts firing and clearing, playbook runs and what they changed, changes to Steward's settings; by host, area or kind, and kept after the records it came from age out) |
| Timeline | `get_timeline` (everything that happened, newest first, in one list: alerts firing and clearing, playbook runs and what they changed, maintenance, reboots, deploys and crash-loops, reviewed anomalies, changes recorded by hand, changes to Steward's settings; by host, area or kind, and kept after the records it came from age out) |
| Docker plugin | `list_containers`, `get_container`, `list_swarm_services`, `search_container_logs`, `get_docker_disk_usage` |
| Host agent (built in, always on) | `list_agents`, `list_maintenance_plans` (each plan's group, weekly slot, next run and latest run: how far it got, who it skipped and why), `get_maintenance_job` (a host's update jobs or one job: state, packages updated, whether it rebooted, why it failed), `get_host_senders` (who sent or received a host's network traffic over a window: its containers and NFS mounts ranked at the host's peak, and what none of them accounts for) |
| Traefik plugin | `list_routers` (each router graded as the Traefik page grades it, with the status codes behind a problem, what they mean and the swarm service behind it), `get_router` (one router's history, codes, normal and anomalies), `list_certificates` (served certificates, days left, the routers they cover), `get_traffic` (the access log's clients, countries and routers, and Traefik's own figures) |
@@ -95,6 +95,7 @@ Each role includes the tools of the roles above it in this list.
| Monitors | `create_monitor`, `update_monitor`, `set_monitor_enabled` |
| Alerts | `create_alert_rule`, `update_alert_rule`, `acknowledge_alert`, `send_test_alert`, `create_maintenance_window`, `end_maintenance_window` |
| Metric anomalies | `review_anomaly` (mark one or several normal, so their hours are learned, or real, with a note saying why) |
| Timeline | `log_change` (record a change Steward can't see happen — a config edited by hand, a disk swapped — on the hosts it was done on, under an area; shown as by "<you> via MCP") |
### Admin: destructive and remediation
+10 -3
View File
@@ -1,6 +1,6 @@
# Timeline
The timeline is one list of everything that happened to the infrastructure, newest first: alerts firing and clearing, playbook runs and what they changed, maintenance runs and reboots, deploys and crash-loops, anomalies once reviewed, and changes to Steward's own settings. Open it from **Timeline** in the sidebar, from a host's **History** card, or through the `get_timeline` MCP tool.
The timeline is one list of everything that happened to the infrastructure, newest first: alerts firing and clearing, playbook runs and what they changed, maintenance runs and reboots, deploys and crash-loops, anomalies once reviewed, changes recorded by hand, and changes to Steward's own settings. Open it from **Timeline** in the sidebar, from a host's **History** card, or through the `get_timeline` MCP tool.
Each entry stands on its own. An alert event, an Ansible run, a maintenance job or an audit row is pruned after its own retention, but the entry for it stays. Its link then reads "Details no longer kept".
@@ -22,24 +22,31 @@ Every source writes its entry in the same transaction as its own record, through
| Maintenance jobs, when they end (and again if the host comes back late) | host agent: `maintenance`, `cadence` | `maintenance.done`, `maintenance.failed` |
| A host away: a reboot, a crash, lost contact, the agent restarting; kept up to date until it ends | host agent: `downtime.record_events` | `host.rebooted`, `host.lost_contact`, `host.agent_restarted` |
| Container snapshots: a new image, a recreated container, a restart or death | Docker: `ingest.persist_host_docker` → `timeline.from_snapshot` | `docker.deployed`, `docker.crash_loop`, `docker.restarted` |
| A change recorded by hand, from the Timeline page, a host's History card or MCP `log_change` | `timeline/changes.py` | `change.recorded` |
A source whose record changes after it is written keeps one entry for it (`record.record_or_update`): a downtime gets its length when it ends, an anomaly changes type when it is reviewed.
Docker's units are swarm services where a container has one, so a service's tasks, replaced on every deploy and crash, read as one subject. A swarm task moving to another host is not a deploy. A crash loop uses the same rule as the Docker pages: Settings → Thresholds' deaths within an hour. The death that reaches it opens one `docker.crash_loop` entry, and each later one extends it until the container has been quiet for an hour. The deaths before it stay as hidden `docker.restarted` entries.
Some audit actions are left to another writer, because the event itself is recorded there: starting a run, acknowledging an alert, and starting maintenance. A person's own preferences (time zone, email settings) are not recorded. `catalog.audit_type` holds that mapping.
Some audit actions are left to another writer, because the event itself is recorded there: starting a run, acknowledging an alert, starting maintenance, and recording a change by hand. A person's own preferences (time zone, email settings) are not recorded. `catalog.audit_type` holds that mapping.
On its first start, Steward fills the timeline from the last 90 days of records that predate it (`service.backfill`): core's alert events, finished runs and audit rows, and each loaded plugin's own through its `timeline_backfill(session, since)` (the host agent's maintenance jobs and downtime). It fills from each source once and records which in `timeline.filled`, so a plugin loaded later fills on the next daily turn. An entry for a record that is already covered is skipped. Docker has no fill: its events can't tell a deploy from a restart after the fact.
`get_anomaly`'s "what else was going on" reads the timeline for the host or resource, hidden types included, beside each occurrence.
## Recording a change by hand
Steward can't see every change happen: a config edited on a host, a disk swapped, a service moved. Whoever made it records it, with **Record a change** on the Timeline page or a host's History card, or the `log_change` MCP tool. It says what was done, the area it is about, when, the hosts it was done on (none for a change not on any one host), optionally what it was done to (a container, a file) and a longer detail.
An operator or admin can record one. Its author can edit or delete it from its **Edit** link, and so can an admin. One recorded through MCP is shown as by "<user> via MCP" and is editable by that token's user. Recording, editing and deleting are audited (`timeline.change_recorded`, `…_edited`, `…_deleted`), and those audit rows stay off the timeline, which holds the change itself.
## Areas and kinds
An entry has a **kind**, which says how it came about: Change, Automation, Alert, Anomaly, Ticket, Observed (Steward saw it happen with no rule or person behind it: a reboot, a crash-loop) or Steward. Kinds are fixed in `catalog.KINDS`.
An entry also has an **area**, which says what part of the infrastructure it is about. The areas start as Service, Host OS, Hardware, Storage, Network, Security and Monitoring.
Each entry is written with an area of its own. An alert's area comes from what its rule watches (`catalog.area_of_metric`): drive health is Hardware, a mount is Storage, an HTTP monitor is Service. Other entries use their type's area.
Each entry is written with an area of its own. An alert's area comes from what its rule watches (`catalog.area_of_metric`): drive health is Hardware, a mount is Storage, an HTTP monitor is Service. A change recorded by hand has the area its author picked. Other entries use their type's area.
## Settings → Timeline