feat(mcp): understand and judge a metric anomaly: what happened, what was usual, what else went on (#5837)

get_anomaly gives each occurrence against what is usual at that hour, the
metric around the newest one with its normal, what else happened on the
resource's host around each occurrence (alert transitions, Ansible runs,
other anomalies there and a count elsewhere, plus plugins' own events
through a new mcp_events hook: the host agent's maintenance jobs and
downtime, Docker's container starts, deaths and OOM kills), and how earlier
anomalies on the same metric were judged.

get_metric_normal reads the learned slots and the level that would be
flagged. review_anomaly takes one id or several, all or nothing, with a
note. get_anomaly_settings and (admin) set_anomaly_settings read and change
the review window and each normal's floor, as Settings → Thresholds does.

metric_series.points takes an optional `until`; core.alerts gains
host_of_resource; floor_setting names its floor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/core/mcp.md

@@ -75,7 +75,7 @@ Each role includes the tools of the roles above it in this list.
 | Hosts and metrics | `list_hosts`, `get_host`, `get_host_metrics`, `query_metrics` |
 | Monitors | `list_monitors`, `get_monitor` |
 | Alerts | `list_alerts`, `get_alert_history`, `list_alert_rules`, `get_alert_rule_options`, `list_maintenance_windows` |
-| Metric anomalies | `list_anomalies` (metrics that left their normal for the hour, to review first) |
+| Metric anomalies | `list_anomalies` (metrics that left their normal for the hour, to review first), `get_anomaly` (one anomaly with each occurrence against what is usual at that hour, the metric around it, what else happened on its host — alerts, Ansible runs, container deaths and starts, maintenance, reboots, other anomalies — and how earlier ones on the same metric were judged), `get_metric_normal` (the learned normal slot by slot, and the level that would be flagged), `get_anomaly_settings` (the review window and each floor) |
 | Automation | `list_playbooks`, `get_playbook`, `list_inventory`, `list_ansible_runs`, `get_ansible_run`, `wait_for_run`, `list_schedules` |
 | Audit | `list_audit_events` |
 | Docker plugin | `list_containers`, `get_container`, `list_swarm_services`, `search_container_logs`, `get_docker_disk_usage` |
@@ -88,11 +88,12 @@ Each role includes the tools of the roles above it in this list.
 | Automation | `run_playbook`, `cancel_ansible_run`, `create_schedule`, `update_schedule`, `delete_schedule`, `update_group_members` (put hosts into an inventory group or take them out; a group alert rule follows within a minute) |
 | Monitors | `create_monitor`, `update_monitor`, `set_monitor_enabled` |
 | Alerts | `create_alert_rule`, `update_alert_rule`, `acknowledge_alert`, `send_test_alert`, `create_maintenance_window`, `end_maintenance_window` |
-| Metric anomalies | `review_anomaly` (mark one normal, so its hours are learned, or real) |
+| Metric anomalies | `review_anomaly` (mark one or several normal, so their hours are learned, or real, with a note saying why) |
 
 ### Admin: destructive and remediation
 
-`set_alert_action` (the playbook a rule runs when it fires), `delete_monitor`,
+`set_anomaly_settings` (the anomaly review window and each normal's floor, as
+Settings → Thresholds sets them), `set_alert_action` (the playbook a rule runs when it fires), `delete_monitor`,
 `delete_alert_rule`, `prune_docker`, `run_maintenance` (update a host now, rebooting
 only if the update needs it, as the host page's Update now does).
 
@@ -150,7 +151,10 @@ Every tool follows these conventions (`steward/mcp/tools/_shape.py`):
 
 Core tools: add an `async def` to a module in `steward/mcp/tools/` and list it
 in that module's `register()` with its minimum role and `read_only`. A plugin
-exposes `register_mcp_tools(register)`, and optionally `mcp_overview(db)` to
-contribute findings to `get_overview`. See
+exposes `register_mcp_tools(register)`, optionally `mcp_overview(db)` to
+contribute findings to `get_overview`, and optionally
+`mcp_events(db, host_id, host_name, since, until)`: what it knows happened on a
+host in a window, as a list of `steward.mcp.tools._events.event(...)`, which
+`get_anomaly` shows beside an anomaly. See
 [Writing a plugin](../plugins/writing-a-plugin.md). A tool that changes
 anything calls `audit_mcp(...)` from `steward/mcp/tools/_audit.py`.

docs/plugins/writing-a-plugin.md

@@ -391,6 +391,21 @@ async def mcp_overview(db) -> list[dict]:
 - `severity` is `"critical"` (something is down or about to be) or `"warning"` (degraded, or heading that way). Report problems only; a healthy plugin returns `[]`.
 - It is called only while the plugin is enabled, with its own session. If it raises, the overview still returns, and names the plugin under `unavailable`.
 
+### What happened on a host
+
+`get_anomaly` shows, beside a metric anomaly, what else happened on its host around each occurrence. A plugin that records events on a host adds them by exporting an optional `mcp_events(db, host_id, host_name, since, until)` that returns events built with `steward.mcp.tools._events.event`:
+
+```python
+# plugins/myplugin/__init__.py
+async def mcp_events(db, host_id, host_name, since, until) -> list[dict]:
+    from steward.mcp.tools._events import event
+    return [event(r.at, "backup_job", f"Backup {r.name}: {r.state}", until=r.finished_at)
+            for r in await _jobs_on(db, host_id, since, until)]
+```
+
+- Return what happened between `since` and `until`, oldest first: something that lasted gives `until`. Keep it to tens of rows; the caller keeps the first 40.
+- It is called only while the plugin is enabled, with its own session. If it raises, the anomaly is still returned without the plugin's events. The host agent (maintenance jobs, reboots) and Docker (container starts, deaths, OOM kills) are the examples to read.
+
 ---
 
 ## Shipping a Plugin