A ticket records a known problem: title, area, hosts, subject, description,
links to alert rules and anomalies, and a log of comments, edits, resolving
and reopening. Opening, resolving (with the resolution as the note) and
reopening go on the timeline; every step is audited, and the audit rows stay
off the timeline.
An alert or anomaly an open ticket covers (by link, by subject, or a
host-wide ticket on the host itself) shows a Known issue badge, and offers
"Open a ticket" prefilled from its rule or anomaly when none does. Hosts,
monitors and containers get a Known issues card.
MCP: list_tickets, get_ticket (viewer); open_ticket, update_ticket,
comment_ticket, resolve_ticket, reopen_ticket (operator). list_alerts and
list_anomalies carry known_issue; get_overview, get_host, get_monitor and
get_container carry open_tickets. Also update_change and delete_change for
changes recorded by hand (author or admin).
Tickets are left out of backups, like the timeline (2026-10-15 ruling).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The host agent writes each maintenance job once it ends, and each stretch a
host was away (a reboot, a crash, lost contact, the agent restarting), kept up
to date until it is back. Docker judges each snapshot: a unit on a new image,
or a container recreated, is a deploy; deaths past the Thresholds crash-loop
count are one crash-loop entry that later deaths extend; single restarts are
recorded but hidden. An anomaly has one entry whose type follows its review:
only those judged real are shown by default.
A source whose record changes keeps one entry for it (record_or_update).
Plugins register the records their entries point to, so an aged-out job or
downtime reads "Details no longer kept". The first fill runs once per source
(timeline.filled), the host agent's jobs and downtime included.
get_anomaly's "what else was going on" reads the timeline for the host or
resource instead of querying alerts, runs, maintenance, downtime and container
deaths itself; Docker's own source keeps only starts, stops and health
changes. A new kind, Observed, is for what Steward saw happen with no rule or
person behind it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three 404s on a quiet router read 7.5% and were raised as an anomaly; most of
the 4xx anomalies on review stood for 3 to 7 errors in 15 minutes. A new
Settings → Thresholds minimum, traefik.threshold.min_4xx (default 20, more
than one a minute held for the window), applies to both the router's grade
on its page and to anomaly detection, as min_5xx does for 5xx.
learn_normal takes `fewest`: for a share gated on a per-second rate, the
fewest events (rate × window × share) it must stand for to be judged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
get_anomaly's circumstances gain the host's or SNMP device's syslog
warnings and worse (mcp_events / mcp_resource_events). An anomaly row opens
"Syslog around it", read only when opened. A shared Chart.js plugin
(chart-marks.js) draws a dashed line at each syslog error on the host
metrics charts and an SNMP device's charts, with a line saying how many.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each source in an anomaly's `around` was cut to its oldest 40 with
nothing in the reply to say so; on a busy host the events nearest the
occurrence were the ones dropped. Each source now keeps the 40 nearest
the occurrence's start, and `around_left_out` counts the rest. Sources
return everything their bounded query read (Docker and the host agent
no longer cut at 40 and 20 themselves), so the count is honest.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All 16 Traefik 4xx anomalies on 2026-10-15 came back with nothing
around them, though 15 were Steward's own restarts. A router has no
host, and every events source was read by host.
- Core: each break of more than 90 s in Steward's once-a-minute health
sample is a "steward" event beside every anomaly, placed or not.
- Plugins: an optional mcp_resource_events(db, source, resource, since,
until) for a resource with no host. Traefik's gives the container
events of the swarm service behind a router, on any host.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
get_anomaly gives each occurrence against what is usual at that hour, the
metric around the newest one with its normal, what else happened on the
resource's host around each occurrence (alert transitions, Ansible runs,
other anomalies there and a count elsewhere, plus plugins' own events
through a new mcp_events hook: the host agent's maintenance jobs and
downtime, Docker's container starts, deaths and OOM kills), and how earlier
anomalies on the same metric were judged.
get_metric_normal reads the learned slots and the level that would be
flagged. review_anomaly takes one id or several, all or nothing, with a
note. get_anomaly_settings and (admin) set_anomaly_settings read and change
the review window and each normal's floor, as Settings → Thresholds does.
metric_series.points takes an optional `until`; core.alerts gains
host_of_resource; floor_setting names its floor.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5740 "A metric that departs from its normal is recorded as an anomaly to
review, and is findable after it ends", milestone 513.
- Every minute each learned key's last 15 minutes is judged against its
hour's normal: above the usual hold x1.25 and by the metric's floor opens
an occurrence, which ends when the level comes back.
- metric_anomalies holds one row per item to review; a repeat while it
still waits joins it as an occurrence (metric_anomaly_spans).
- An ended row nobody reviews within anomalies.review_days (14, Settings ->
Thresholds) lapses; an ongoing one never does.
- Learning skips every hour an occurrence covers unless its anomaly was
marked normal: no normal is learned without review.
- Alerts -> Anomalies lists those to review first, then the reviewed and
lapsed under day dividers; marking is operator/admin and audited.
- The Overview widget and get_overview warn of open ones and count those
to review; the host page shows its own; the weekly report lists the
week's and the lapsed; MCP list_anomalies and review_anomaly.
- Reviewed and lapsed rows are pruned at data.retention_days, and kept
in backups like alert history.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bvandeusenandClaude Opus 5.5