Commit Graph
123 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 d19df51ffb feat(alerts): pick the host, then its part, and see what the metric reads (#5475)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m8s
CI / unit (push) Failing after 1m29s
CI / publish (push) Skipped
#5475 "Rule form: pick the host, then its part; metrics that part reports;
a link to groups; a preview of the metric", from the operator's review of
the New rule page: hosts and their parts shared one long picker.

- Host agent rules pick a Host, then a Part: "The whole host", then its
  mounts, CPU cores, disks, interfaces and sensors under a heading each,
  cores in number order. A part that reports nothing a rule can watch
  (an interface's traffic) isn't offered.
- The Metric list holds what the picked resource reports, in catalog order.
- Under it, a preview: the reading now, and the last 24 h low to high with
  an hourly sparkline drawn from zero (UI ruling 8), read from one series.
- The group picker links to Ansible > Inventory to make a group, in a new
  tab, and re-reads the groups when the tab comes back into view.
- Picking a host or part swaps the pickers alone (/alerts/api/rule_target);
  picking a metric swaps the preview (/alerts/api/metric_preview).
- A rule's own resource and metric stay on offer after they go quiet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:54:08 -04:00
bvandeusenandClaude Opus 5.5 d5601575d8 test(alerts): seed metrics the way ingest does, through record_metrics (#5473)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m10s
CI / unit (push) Successful in 1m37s
CI / publish (push) Successful in 1m11s
#5473 "Opening an alert rule takes up to 15 s". With the rule pickers now
reading plugin_metric_latest, nine integration tests failed on rule_fields:
"host_agent has no resource 'box-…'". They seeded a resource by adding a raw
PluginMetric row, which no production path does: host-agent ingest and
record_metric both go through record_metrics, which keeps the latest-value
table. The seeds now go through record_metrics too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:28:53 -04:00
bvandeusenandClaude Opus 5.5 5ef18c992c feat(alerts): Send test on a rule checks that its alerts reach someone (#5466)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m9s
CI / unit (push) Successful in 1m36s
CI / publish (push) Successful in 1m6s
#5466 "Send test: check a rule's delivery from the rule". A rule's page gets
a Send test button (operator role), and MCP gets send_test_alert. Each sends
a TEST alert for that rule now, by email to everyone who gets alerts and by
webhook. The alert carries the rule's name and condition, at the latest
reading of what it watches; a rule covering hosts uses its first host. The
result for each channel shows under the button, or "Nothing is set up to
send to". Nothing fires, no event is written, and the test is audited as
alert_rule.tested. An alert with no reading now says "has no reading yet"
instead of "is None".

Docs: alerting.md also gains the burst email from #5465 "One email per burst
of alerts".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 08:33:32 -04:00
bvandeusenandClaude Opus 5.5 48a0de5600 feat(alerts): one rule can cover every host, or every host in a group (#5464)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m7s
CI / unit (push) Successful in 1m25s
CI / publish (push) Successful in 1m12s
#5464 "Group rules: one rule covers every host, or every host in a group",
step 2 of milestone 499 "Alerts you can trust". Each rule named one host, so
the live instance carried six rules for three hosts, and a new host had no
agent-offline or disk alert until someone copied them.

- alert_rules.covers / cover_group / parent_id (migration 0041). A rule on
  host_agent may cover "all_hosts" or a "group" (an inventory group: the
  Hosts ruling, #5294). A covering rule has no state; Steward keeps one
  ordinary child rule per covered host, named "<rule>: <host>", so each
  host's alert keeps its own state, history and acknowledgement.
- alerts.service.reconcile_children: a covered host that has reported in
  the last 7 days gets a child; a host no longer covered loses its child;
  children take the parent's condition, checks, notify setting, action and
  on/off. Run on save and each minute (alert_reconcile).
- A child is changed through its parent: edit, toggle and delete of a child
  are refused (409 / PartOfCoveringRule); its edit page opens the parent.
- Rule form: a Covers choice for host sources, with the group picker; fixed
  once the rule exists. Rules list: a covering rule reads "every host in
  swarm · 3 hosts" with its hosts' rules nested under it, each opening its
  history. The history picker and the rule count leave covering rules out.
- MCP: create_alert_rule takes covers/group; update_alert_rule takes group
  and passes changes down; list_alert_rules shows covers, hosts and part_of.
- docs/core/alerting.md: "Rules that cover hosts".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 08:25:51 -04:00
bvandeusenandClaude Opus 5.5 734d313a54 feat(alerts): an acknowledgement is on the record, with who made it (#5463)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m7s
CI / unit (push) Successful in 1m27s
CI / publish (push) Successful in 1m8s
#5463 "Acknowledgements on record: an event and who did it", step 1 of
milestone 499 "Alerts you can trust". The live walk in #5461 found that
acknowledging changed only the alert's current state: once it resolved,
nothing said who had acknowledged it or when.

- alert_events.actor_id (migration 0040): the person behind a transition.
- Acknowledging writes a firing → acknowledged event by the acknowledger.
- A rule edit, disable or enable that resets an alert carries its actor
  (web and MCP alike).
- The history page notes "acknowledged by X" and "rule disabled by X"; MCP
  get_alert_history returns `by`.
- One usernames() helper in alerts.service, used by the active list, the
  history page and the MCP tools.
- docs/core/alerting.md: the state diagram no longer shows an acknowledged
  alert re-firing, and says what an event records.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 08:19:48 -04:00
bvandeusenandClaude Opus 5.5 b3a5bbfeff perf(docker): the hourly log prune ranks only the containers over their cap (#5449)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m30s
CI / unit (push) Successful in 1m42s
CI / publish (push) Successful in 1m52s
#5449 "The hourly Docker log prune ranks every log line in the table to find
the few over their cap": the size ring put a running-total window over every
row of docker_logs each hour, sorting the whole table on NFS-backed storage
to find the handful of containers past docker.logs.max_bytes_per_container.

- prune_docker_logs (capability docker.prune_logs) does the age cutoff, then
  a per-container byte total picks the containers at or over the cap, and
  only their rows are ranked and trimmed. Same keep rule as before: the
  newest line always survives.
- run_docker_retention keeps the metrics rollup and the event prune.
- Cleanup runs the log ring in its own transaction after them.

Arose from #5431 "Ingest dropped for ~3 minutes after each deploy while
startup retention runs".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 22:31:00 -04:00
bvandeusenandClaude Opus 5.5 4a33046aef fix(ansible): Run now runs the playbook and leaves the schedule's time alone (#5445)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 1m51s
CI / integration (push) Successful in 1m45s
CI / publish (push) Successful in 2m21s
Run now set last_run_at to the click, and a schedule's next turn counts
from last_run_at, so a manual run of the nightly prune at 14:00 moved
every later run to 14:00. Operator ruling: "run now shouldn't change the
schedule it should just run the playbook and leave the schedule as it
was."

Run now no longer touches last_run_at. It still records the run as
last_run_id, so the list shows it and a firing during it is skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 22:08:18 -04:00
bvandeusenandClaude Opus 5.5 8edc0c30a3 fix(mcp): get_monitor says how often a monitor on the global interval runs (#5433)
check_interval_s was `m.check_interval_seconds or None`, so a monitor
following monitors.poll_interval_seconds read as null and a reader
couldn't tell how often it ran. Monitor results now carry the effective
interval and `interval_from: "monitor" | "global"`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 21:50:20 -04:00
bvandeusenandClaude Opus 5.5 480b465b8e fix(alerts): a series that comes back gets one history event, not two at one instant (#5442)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m29s
CI / unit (push) Successful in 1m41s
CI / publish (push) Successful in 1m25s
When a silent series' sample also moved the rule (fired, resolved), the
resume and the transition were two events: the resume from the
pre-evaluation state at ingest time, the transition at the sample's
time. Ties sort by a random id, and the resume's "→ inactive" landed
after "→ firing", so the history read "silent, firing, back to normal"
at once and the episode closed while the alert was firing.

_track_samples now only clears the mark and returns its note; the first
sample's evaluation records one event from no_data to wherever that
sample took the rule ("firing" or "resolved" notification), or back to
the state it was in ("data_resumed", still only when notify_no_data).

Also fixes the #5441 integration test: its rule needs a recorded
resource to target (CI 8878).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 21:48:43 -04:00
bvandeusenandClaude Opus 5.5 a9bf4b7f16 fix(alerts): Steward's own outage no longer reads as every agent going silent (#5441)
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 1m26s
CI / unit (push) Successful in 1m31s
CI / publish (push) Skipped
After Steward or its database was down, every agent's age and every
series' last sample were old, so all agent-offline rules fired and all
rules went no-data at once, clearing a minute later. With per-account
alert email that would have mailed everyone a dozen notices per restart.

A periodic job now ticks a Watch (steward/core/watch.py) once it has
reached the database; the Watch says since when runs have been unbroken
(process start, or the end of a gap over 120 s). seconds_since_seen and
the no-data sweep count silence from no earlier than that, so a source
must stay quiet while Steward is watching before it counts as silent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 21:45:47 -04:00
bvandeusenandClaude Opus 5.5 593b583d23 feat(auth): reset a forgotten password from an emailed link (milestone 497)
CI / lint (push) Failing after 3s
CI / integration (push) Successful in 1m37s
CI / unit (push) Failing after 1m43s
CI / publish (push) Skipped
Step #5429 "Reset by email: request, link and new-password page" of
milestone 497 "Passwords: change your own, and reset it by email".

- Sign-in page: "Forgot your password?" while a mail server is set.
- /forgot takes a username or email and answers the same either way; only
  an active local account with a real address gets mail. Requests count
  per address on their own throttle, so they never lock out sign-in.
- password_resets (migration 0039): a one-time key, stored as its sha256,
  working for an hour; asking again replaces it.
- /reset/<key>: the new password, then signed straight in; every other
  browser is signed out (sessions_since). Bad keys count as invite keys do.
- Email "[Steward] Reset your password", in the account's own zone.
- Audited: user.password_reset_requested, user.password_reset_by_email.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 20:17:24 -04:00
bvandeusenandClaude Opus 5.5 ea523b8c37 feat(account): change your own password on the Account page (milestone 497)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m10s
CI / unit (push) Successful in 1m31s
CI / publish (push) Successful in 1m26s
Step #5428 "Account page: change your own password" of milestone 497
"Passwords: change your own, and reset it by email". A Password section
in the page's one form, for local accounts: current, new, again. Blank
keeps the password; a new one signs out every other browser
(sessions_since) and keeps this one. A wrong current password counts
toward the login throttle. Audited as account.password_changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 20:14:45 -04:00
bvandeusenandClaude Opus 5.5 ea31deabf9 fix(unifi): the Overview's history buckets on Postgres; devices name their state (#5207)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m24s
CI / unit (push) Successful in 1m35s
CI / publish (push) Successful in 1m29s
#5207 "UniFi Overview tab: func.strftime is SQLite-only": the WAN and
client sparklines bucketed with strftime('%s'), which Postgres lacks, so
/plugins/unifi/rows failed. Traefik's date_bin bucket moves to
core/time_range.time_bucket and both plugins use it. Devices list offline
and failing first, and a state other than online is named (Upgrading,
Adopting, Heartbeat missed...) instead of drawn as Down.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 17:28:19 -04:00
bvandeusenandClaude Opus 5.5 2995cfdd68 fix(users): a password reset signs out every session begun with the old password (#5427)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m7s
CI / unit (push) Failing after 1m24s
CI / publish (push) Skipped
#5427 "A password reset doesn't end the sessions already signed in with
the old password": users.sessions_since (migration 0038) is set on reset;
refresh_session_user ends a session whose login_at is earlier. An admin
resetting their own password carries on signed in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 17:25:35 -04:00
bvandeusenandClaude Opus 5.5 fb4cf098c3 test(invites): match the escaped apostrophe; give test admins a real address
CI / lint (push) Successful in 6s
CI / unit (push) Successful in 1m43s
CI / integration (push) Successful in 1m17s
CI / publish (push) Successful in 1m41s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 17:00:13 -04:00
bvandeusenandClaude Opus 5.5 2b6e12859a feat(users): invite by email — a one-time link where the invitee makes their own account (milestone 496)
CI / lint (push) Successful in 3s
CI / integration (push) Failing after 1m17s
CI / unit (push) Successful in 1m43s
CI / publish (push) Skipped
The operator's ask: an invite link sent by mail, the key in the URL, and
the new user choosing their own username, password and email.

- user_invites (migration 0037): the key's sha256 only, the address, the
  role and days the admin chose, who sent it, used/revoked.
- core/invites: create (revokes an older open invite to the address),
  renew (new key, new clock), find_pending, the four states, signup checks.
- /invite/<key>, outside sign-in and throttled like login: the invitee's
  username, email (prefilled, theirs to change) and password; the account
  takes the admin's role, the key is used up in the same transaction and
  they are signed in. A bad key gets one plain message.
- Settings › Users: Invite by email (address, role, 1/7/30 days), sent from
  the admin in their zone; without a mail server, or if sending fails, the
  link is shown once to copy. Invites not yet used are listed with Resend
  and Withdraw.
- Audit: user.invited, user.invite_resent, user.invite_revoked, user.joined.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:54:44 -04:00
bvandeusenandClaude Opus 5.5 ac38a04642 test(account): count the page's own forms, not base.html's
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m3s
CI / unit (push) Successful in 1m29s
CI / publish (push) Successful in 1m34s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:11:18 -04:00
bvandeusenandClaude Opus 5.5 beae8a33f3 fix(account): one form and one Save for your account; the same for an account's page in Users
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 1m8s
CI / unit (push) Successful in 1m36s
CI / publish (push) Skipped
Operator review of the Account page: two sections with two different
save buttons, one stuck to the next card and one beside a squeezed hint.
- Account: time zone and Email are one form; POST /account/ saves both,
  nothing unless all of it is valid. /account/email is gone.
- Users › account: Name and Email are one form; the rename route also
  saves the email switches. /settings/users/<id>/email is gone.
- The Email section is a plain section now (email_section); its hint sits
  under the switches, names the zone as the picker does, and links
  Notifications only for an admin.
- apply_email_form leaves a switch the form doesn't carry as it is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:05:41 -04:00
bvandeusenandClaude Opus 5.5 31e32c173c test(settings): the adopt-zone test gives the account its zone — the session copy is refreshed from it
CI / lint (push) Successful in 4s
CI / integration (push) Successful in 1m38s
CI / unit (push) Successful in 1m52s
CI / publish (push) Successful in 1m12s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:39:47 -04:00
bvandeusenandClaude Opus 5.5 7ae6f85032 fix(settings): Notifications opens on first visit — no second begin on an autobegun session
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 59s
CI / unit (push) Successful in 1m24s
CI / publish (push) Skipped
Reading the settings autobegins the session's transaction, so the
nested db.begin() that saves the first admin's zone as smtp.tz raised
InvalidRequestError and every visit 500'd until a zone was saved. The
integration test signed in without a zone, so that path never ran;
it now has a test of its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:34:19 -04:00
bvandeusenandClaude Opus 5.5 22090b02d8 feat(email): each account gets its own email, in its own zone (milestone 495)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m0s
CI / unit (push) Successful in 1m24s
CI / publish (push) Successful in 1m32s
Milestone 495 "Email goes to each account, in its own time zone", on the
operator's ruling (System 156): notifications go to each user, times in
their zone, defaults on, toggled per user or per role.

- users.notify_alerts / notify_report (migration 0036): None follows the
  role's default, notify.<kind>.<role> in settings, all on to start.
- mail.pick_recipients: active accounts that want the email and have an
  address mail can reach (an SSO placeholder like name@oidc is skipped and
  flagged), each in its own zone with no UTC beside it, then the Also send
  to addresses in their chosen zone with UTC beside. One copy per address,
  all over one SMTP connection.
- Alerts, the weekly report and the test email (to you, in your zone, plus
  the typed addresses) send that way. The save-time smtp.tz capture is gone.
- Notifications: a role × email table of defaults, who each email reaches
  and who can't be reached, and Also send to with its own zone, which starts
  as the first admin's to open the page.
- Account page: your address and your two switches. Users: an Emailed
  column, and an admin sets an account's switches on its page.
- The Alerts page counts the accounts it emails. zone_select macro and
  formatting.zones_by_region shared by Account and Notifications.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:16:17 -04:00
bvandeusenandClaude Opus 5.5 c533eec33a feat(settings): Users page — list, add, rename, role, password, disable, delete (#5336)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 1m31s
CI / integration (push) Successful in 1m3s
CI / publish (push) Successful in 1m21s
#5336 "Users page: list accounts, change role, disable, reset a local
password, create a local user".

- Settings › Users (admin only, a Settings tab): every account with how it
  signs in, its role, last sign-in (from the audit log) and working AI
  tokens; disabled accounts last. A row opens the account.
- An account's page: rename (any account), role and password (local
  accounts; an LDAP or single sign-on role comes from the provider at
  every sign-in, and the page says so), disable or enable, and Delete
  behind the confirm dialog.
- Refusals are said beside the control: nobody disables, demotes or
  deletes themselves, the last active admin stays, and an account that
  made alert rules, maintenance windows or playbook runs is disabled
  rather than deleted (those records keep who made them).
- Add user: a local account with a role and a password of 8+ characters.
- Every change is audited as user.*, and the audit log names and links the
  account.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:12:56 -04:00
bvandeusenandClaude Opus 5.5 ec55faf737 feat(email): alert and weekly-report emails read as designed messages (#5335)
CI / lint (push) Successful in 4s
CI / integration (push) Successful in 1m22s
CI / unit (push) Successful in 1m49s
CI / publish (push) Successful in 1m13s
#5335 "Emails Steward sends read as designed messages: an HTML alert email
and weekly report, with a plain-text twin".

- core/mail.py renders each email as multipart/alternative from
  templates/email/: a table-and-inline-style HTML part and a plain-text
  twin carrying the same facts. Colours are the design system's light
  theme, resolved from theme.css; nothing is fetched when a mail opens.
- Alert email: state badge, a headline sentence ("Disk used on web01 is
  94%."), rule, resource, reading, the condition in words, and the time in
  the zone Notifications was saved in with UTC beside it; "View in
  Steward" when Steward's address is set. Subject "[Steward] Firing: <rule>
  — <resource> 94%". Alerts now carry their operator.
- Weekly report: monitors down, active alerts, the week's alert events,
  uptime and busiest Traefik routers as small tables with links. Sent from
  the configured From address; the "Generated … UTC" line and the stale
  repo footer are gone.
- condition_text() states a condition from a rule's parts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:08:05 -04:00
bvandeusenandClaude Opus 5.5 501a345aed test(snmp): read and remove the poll test's rows in one transaction (milestone 491)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 58s
CI / unit (push) Successful in 1m24s
CI / publish (push) Successful in 1m34s
Run 8790: a select opened the session's transaction before begin().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:01:08 -04:00
bvandeusenandClaude Opus 5.5 189b492397 test(snmp): the poll test reaches the plugin as the app loaded it (milestone 491)
CI / lint (push) Successful in 3s
CI / integration (push) Failing after 1m6s
CI / unit (push) Successful in 1m41s
CI / publish (push) Skipped
SNMP is on by default, so the loader has already mapped snmp_devices under
_steward_plugin_snmp; importing plugins.snmp.models defined the table twice
(integration lane, run 8788).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:57:16 -04:00
bvandeusenandClaude Opus 5.5 9ca2e38fa1 feat(snmp): devices you add, test and read on the SNMP page (milestone 491)
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 55s
CI / unit (push) Successful in 1m24s
CI / publish (push) Skipped
The operator, reviewing SNMP on #5214: "this system looks incomplete like
there's something missing from it". The one device shown was plugin.yaml's
example (core-switch 192.168.1.1, community public), polled every minute;
devices could only be defined by editing YAML in the image; a failed poll was
logged at DEBUG and stored nowhere, so the page said "waiting for the next
poll" forever; v3 ran as v2c; counters showed as lifetime totals.

- Devices are rows (snmp_devices, migration snmp_001): address, port,
  v1/v2c community or a v3 user with auth/priv protocols and keys (secrets
  encrypted), presets plus custom readings, an optional host binding, and
  the newest poll's outcome. The YAML device list and its example are gone.
- The SNMP page adds, edits and deletes them (admin, audited, with
  get_audit_entities so the audit log names and links a device). Readings
  come from presets (system uptime, interface traffic by number, UPS-MIB,
  Linux net-snmp) and custom "OID name [unit]" lines.
- One GET per device with every OID, devices polled at once. The outcome is
  written to the row in words ("No reply from … within 5 s. Check the address
  and port, that SNMP is on, and the community"), with readings it lacks.
  v1's whole-request error drops that OID and asks again. SNMPv3 signs in
  with USM (MD5/SHA-1/SHA-2, DES/3DES/AES).
- Test asks the unsaved form's device for its name, description, uptime and
  interfaces by number. It is an admin's explicit action; every page reads
  only stored rows (System 178's ruling).
- Each reading has a unit; a counter is stored as a per-second rate (bytes as
  bit/s), a reset or wrap skipped. The page, widget and host panel say values
  in units and why a device is silent; the device page draws a chart per
  reading on a dated axis.

Tests: readings and presets, the form and its secrets, v3 checks, rates,
the poller against a stand-in pysnmp (one GET, v1 retry, timeout wording,
USM), the pages, and an integration test of the poll writing outcomes and
rates on Postgres.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:54:11 -04:00
bvandeusenandClaude Opus 5.5 5f04f6af66 fix(traefik): the newest scrape per router comes from plugin_metric_latest (#5355)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m6s
CI / unit (push) Successful in 1m28s
CI / publish (push) Successful in 1m28s
The phase log on the live instance: 'traefik rows: slow 7.4s (latest=7.04s
history=0.11s window=0.04s ...)'. latest_per_router walked traefik_metrics'
index one router at a time; each step lands on the router's oldest
entries, which retention deletes every minute (likely dead ones). The
scraper already records every router figure through MetricBatch, which
keeps one row per router and figure in plugin_metric_latest (#5208's
table, read as latest_by_resource does); latest_per_router reads that
now, and the router figures carry the scrape's own time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 09:59:48 -04:00
bvandeusenandClaude Opus 5.5 7ce61a2099 fix(traefik): grade routers on their last 15 minutes; the Config tile reads Traefik's reload time (#5355)
CI / lint (push) Failing after 2s
CI / integration (push) Successful in 54s
CI / unit (push) Successful in 1m24s
CI / publish (push) Skipped
The operator: problem cards "can disappear so quickly". Each router was
graded on its latest scrape, so a 4xx share near the 1% cutoff crossed it
on one scrape and fell back on the next. recent_window reads every
router's last 15 minutes in one query, the error shares weighted by
traffic; grade_routers uses it on the page and the dashboard widget, and
the why says "over the last 15 min".

traefik_config_last_reload_success is the Unix time of the last good
reload, not a 0/1 flag, so the tile said "Reload failed" on every scrape.
traefik_005 stores it as config_last_reload_at, and the tile reads
"Config reloaded 3m ago". The router index also carries the 4xx share.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 08:26:10 -04:00
bvandeusenandClaude Opus 5.5 31e93e96cf fix(traefik): the rows page reads every router's history in one index-only query (#5355)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m18s
CI / unit (push) Successful in 1m28s
CI / publish (push) Successful in 2m3s
#5355 "Traefik page doesn't load": /plugins/traefik/rows took 12-28 s.
It ran a query per router ever seen, and each raw row in the range was a
heap fetch. traefik_004 rebuilds the router index to cover the charted
columns and indexes scraped_at for the request chart; router_history reads
the current routers' buckets in one query.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 08:20:09 -04:00
bvandeusenandClaude Opus 5.5 6a81cf4aac feat: times in the reader's own zone; the audit log shows exact moments (milestone 490)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m0s
CI / unit (push) Failing after 1m23s
CI / publish (push) Skipped
- users.time_zone (0034). An account without one takes the browser's zone
  on its first page (POST /account/time-zone/adopt, never over a choice made
  by hand); the session follows the account on every request.
- /account: who you are and the zone every page writes its times in,
  reached from your name in the sidebar.
- formatting.when: 'Today 06:58', 'Yesterday 23:41', 'Tue 6 Oct 14:12',
  '6 Oct 2025 14:12', 24-hour, in the reader's zone; |ago's hover is the
  exact local moment instead of UTC; tz_label() for column headers.
- Audit log: When (EDT) with exact times, one detail fact a line, on classes.
- tzdata in the image, and the airgapped smoke checks a zone resolves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 07:56:27 -04:00
bvandeusenandClaude Opus 5.5 c678379646 fix(ansible): group names Ansible accepts, and interpreter discovery without the warning (#5334)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 57s
CI / unit (push) Successful in 1m24s
CI / publish (push) Successful in 1m26s
Every run warned twice because of Steward's own inventory:
- 'Invalid characters were found in group names': the group 'Docker
  Swarm' went into the inventory as typed. ansible_group_name() cleans it
  to docker_swarm (lower case, [a-z0-9_], never a leading digit); two
  names that clean alike get _2, _3 so neither folds into the other. The
  group page says what plays call it. The display name is unchanged.
- Python interpreter discovery: ansible_env sets
  ANSIBLE_PYTHON_INTERPRETER=auto_silent unless the environment already
  sets one; a target or group's ansible_python_interpreter still wins.

Runs are scoped by target set (steward:group:<id>), not by group name,
and --limit is free text, so neither path depends on the name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 00:01:54 -04:00
bvandeusenandClaude Opus 5.5 7c346354f2 feat(ansible): Inventory on one page, add behind dialogs, delete behind the confirm (#5333)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 57s
CI / unit (push) Successful in 1m23s
CI / publish (push) Successful in 1m25s
/ansible/inventory/ shows every target (address, its linked Steward host,
its groups) and every group (members, the schedules aimed at it) under a
verdict line. The whole row opens the target or group. Add target and
Add group open small dialogs; a failed add comes back with its dialog
open and what was typed. The red x leaves the rows: Delete is on the
target's or group's page, behind the confirm dialog, saying which
schedules would be left with nothing to run on.

The two list pages (/inventory/targets, /inventory/groups) are gone
with no redirect (rule 122); their URLs keep only the Add POST.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 23:57:54 -04:00
bvandeusenandClaude Opus 5.5 72a3261582 feat(ansible): Playbooks as one list, and a page per playbook that runs it (#5331, #5330)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 55s
CI / unit (push) Failing after 1m20s
CI / publish (push) Skipped
- /ansible/playbooks lists every playbook across sources: what it says it
  does, its category and destructive flag, how it last ran; the whole row
  opens the playbook. A source with none is one quiet line.
- /ansible/playbooks/<source>/<path>: the verdict (what it does, flagged
  destructive, last ran), the run form in place (dry run beside the button,
  options folded), the schedules that use it, its recent runs, the playbook
  itself folded. ?from=<run> fills the form from that run: Run again on a
  run's page.
- /ansible/browse, its view page and the /run-form fragment are gone, not
  redirected (rule 122); every link points at the new pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 23:49:05 -04:00
bvandeusenandClaude Opus 5.5 a02f224f22 feat(ansible): one strip of tabs, a run list that opens on its verdict, a run page in one line (#5328, #5329, #5330)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m25s
CI / unit (push) Successful in 1m46s
CI / publish (push) Successful in 1m9s
- section_tabs macro; Runs · Playbooks · Schedules · Inventory on every
  Ansible page, replacing each page's own set of header buttons.
- Runs: a verdict line (last 30 days, failures first with the latest named,
  running now, the schedule due next); under each status what the run did
  across its hosts (ansible/recap.py); a filter bar (playbook, status, who
  started it) that swaps the table and carries into the scroll paging.
- Run page: status, duration, start and who started it in one line; each
  host's result in words, worst first; the playbook it ran links to it;
  the log and failures on classes rather than inline colours.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 23:45:04 -04:00
bvandeusenandClaude Opus 5.5 e37cd0a873 test(alerts): check the history's Action column on one rule's rows, which the test owns (#5316)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 53s
CI / unit (push) Successful in 1m19s
CI / publish (push) Successful in 1m19s
The all-rules page reads a Postgres shared with other tests, whose
Ansible-run events turn the column on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 23:06:33 -04:00
bvandeusenandClaude Opus 5.5 531a193bd1 feat(alerts): history reads as episodes; tests follow the confirm dialog (#5316, #5314)
CI / lint (push) Successful in 3s
CI / integration (push) Failing after 54s
CI / unit (push) Successful in 1m34s
CI / publish (push) Skipped
History (#5316):
- Transitions fold into episodes (steward/alerts/episodes.py): each stretch
  out of quiet, the worst state it reached, how long it lasted, its worst
  reading against the rule ('3m 21s (fires above 3m)'), who was told across
  its changes, and its Ansible runs. The changes themselves fold under it.
- An episode cut by paging says so ('cleared later (above)', 'earlier').
- For one rule: a verdict ('Fired 4 times in the last 7d, 1h 2m in all') and
  no Rule column. Notified and Action show only when the range has any.

Tests:
- The confirm guard looks for a confirm( call, not the word in the comment
  that says it is gone (lesson #4365).
- The Hosts viewer test looks for the add-host dialog by id, since every page
  now carries the confirm dialog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 23:02:01 -04:00
bvandeusenandClaude Opus 5.5 29be9e3590 feat(docker): the service and container pages say each fact once (#5310)
CI / lint (push) Successful in 4s
CI / integration (push) Successful in 1m33s
CI / unit (push) Failing after 1m59s
CI / publish (push) Skipped
Service page:
- The header carries the verdict, mode, stack, image and when it was reported.
  'In trouble since' shows only while the service is in trouble; once recovered,
  'Last failed X ago · its logs'.
- The failure card shows only while the service is down, degraded or crash-looping.
- The tile strip and the kv card are gone.
- One Tasks table: running tasks first with CPU, memory and since when, then
  Swarm's record. The node is named as its Steward host throughout, and
  'N failed in 24h' sits in the caption.

Container page:
- Same shape: header verdict, a 'Last exit' card only while it is stopped
  after a failing exit, set-up and I/O in one kv card, and lifecycle as a table.
- A swarm task's breadcrumb goes through its service.

Both pages:
- Resource history is the one fragment (_usage_history.html): now and peak in
  cores and bytes, drawn with a floor of 1 core / 1 GiB (sparkline_svg floor=),
  so idle noise draws flat.
- The container page's old % sparklines, its private _sparkline and the unused
  bucket()/container_history() helpers are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:33:25 -04:00
bvandeusenandClaude Opus 5.5 2ecf4d57cb test(docker): the disk page's images carry the 12-character ids the agent sends (#5305)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 52s
CI / unit (push) Successful in 1m18s
CI / publish (push) Successful in 1m19s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:20:41 -04:00
bvandeusenandClaude Opus 5.5 8566c8f78e feat(docker): Docker disk is recorded over time per host, the Disk page shows each host's growth and folds images under their repository, and the overview has one disk line per host (#5304–#5306)
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 54s
CI / unit (push) Successful in 1m21s
CI / publish (push) Skipped
- #5304 "Docker disk totals recorded over time per host, with a growth chart and alertable metrics"
- #5305 "Disk page: images grouped by repository, versions folded under it"
- #5306 "Docker overview: one disk line per host linking to the Disk page"

Each `docker system df` report is now also recorded as metrics, with the host
as the resource: disk_total_bytes, its parts, and disk_reclaimable_bytes. An
alert rule can watch them. The Disk page starts with the hosts side by side,
showing how much each grew over the selected range with a line of it, then
gives each host its own section. An image's versions fold under its
repository, which is charged only for the layers its versions don't share.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:16:13 -04:00
bvandeusenandClaude Opus 5.5 0b5cd23d38 test(docker): a filtered overview leaves the service out of its table, not out of the failures list (#5299)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m12s
CI / unit (push) Successful in 1m36s
CI / publish (push) Successful in 1m35s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:14:33 -04:00
bvandeusenandClaude Opus 5.5 a775882bd4 test(docker): a plugin template can take its own name; the overview's assertions follow its new tables (#5299, #5302)
render_plugin_template's first three arguments are positional-only, so the
service page's own `name` reaches the template. The overview names a service
rather than its replica containers, and a stack chip keeps the search it was
clicked under.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:11:46 -04:00
bvandeusenandClaude Opus 5.5 2093f399e6 feat(docker): the Docker page leads with a verdict, lists services and containers as filterable tables, and each service has its own page with logs across its tasks (#5298–#5302)
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 1m4s
CI / unit (push) Failing after 1m28s
CI / publish (push) Skipped
- #5298 "Docker overview: verdict line; only the problem tiles that aren't zero"
- #5299 "Docker overview: swarm services as a filterable, sortable table with a Stack column"
- #5300 "Docker overview: failures listed once, each with a link to the failed task's logs"
- #5301 "Docker overview: standalone containers in one table with a Host column"
- #5302 "Service page: replicas, image, task history, resource history and logs across all its tasks"

overview.py holds the rows, filters and sort as plain functions. The filter
bar sits outside the polled fragment and the URL keeps its state. A service's
page reads every task container Steward still holds anything for, the live
ones and removed ones known by their events, so a failed task's logs are
reachable after Docker has removed it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:08:38 -04:00
bvandeusenandClaude Opus 5.5 7dccb2f303 fix(hosts): add HX-Request to Vary rather than replacing it; the session adds Cookie (#5294)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 52s
CI / unit (push) Successful in 1m17s
CI / publish (push) Successful in 1m1s
CI 8616: the integration test asserted Vary == 'HX-Request', but the session
layer appends 'Cookie'. resp.vary.add keeps both; the test asserts membership.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:57:22 -04:00
bvandeusenandClaude Opus 5.5 2990113e10 feat(hosts): the Hosts list searches, filters by group and status, sorts by column, and shows each host's groups (#5294)
CI / integration (push) Failing after 52s
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 1m17s
CI / publish (push) Skipped
#5294 "Hosts list: search, group and status filters; sortable columns; a Groups column".
Ansible inventory groups are Steward's grouping system (operator's ruling): a host's
groups are its linked target's, and membership overlaps, so the list filters by group
rather than sectioning by it.

- Filter bar: search on name or address, a group (or "No group"), a status (Down,
  Agent silent, Up, No monitors). The table swaps in alone and the URL keeps the filters.
- Name, Status, Latency, 30d uptime, CPU and Memory headers sort (sort_th in _macros);
  problems-first stays the default, and a host with no value goes last either way.
- A Groups column of chips, each filtering to its group; shown when any group exists.
- A swap is keyed on HX-Target, so Back to a pushed URL still gets the page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:53:03 -04:00
bvandeusenandClaude Opus 5.5 bfe3618e88 feat(hosts): the Hosts list labels its figures and says how long; SLA tab gone; one place to add a host (#5279–#5283)
CI / integration (push) Successful in 53s
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 1m16s
CI / publish (push) Successful in 1m23s
Hosts list:
- CPU and Memory are their own labelled columns (#5279); a silent
  agent's last readings are greyed, and Status says "agent silent for 7m".
- One uptime figure, 30 days, with 24h and 7d on hover. The SLA tab
  (/hosts/uptime) repeated it and is removed (#5280; one home per fact).
- Status is a word with how long ("Down for 12m"); host_status now carries
  down_since. Edit and Delete live on the host's page only (#5281).
- Figures are right-aligned with tabular digits (.table .num), not a
  monospace face.

Agent fleet (#5282): its add-host form is gone, so hosts are added only on
Hosts -> Add host. That form was also the only way to get a token for a
manual curl install, so that moves to the host's Agent card as "Install
manually" (POST /fleet/<host_id>/register), shown on the fleet page as
before. "Delete" there reads "Unregister".

Add host (#5283): laid out like the add-monitor form; a name clash comes
back inline with what was typed instead of a bare text 400; the address
is optional, as the host page already allowed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 17:12:17 -04:00
bvandeusenandClaude Opus 5.5 c6fc02def5 feat(monitors): a monitor's page reads as a page — response time over 24h, labelled actions, how long, not "ago"; the add form says what each type checks (#5214)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m3s
CI / unit (push) Successful in 1m22s
CI / publish (push) Successful in 1m22s
Monitor page: a response-time chart for the last day with failed checks
marked; Edit / Pause / Delete as labelled buttons; the state tile says
"for 29m" instead of "for at least 29m 25s ago"; the check strip fills
its card; no failures is one line, not an empty box; Settings names the
type, target and the actual global interval, with readable field names.

Add monitor: a line under the fields says what the selected type checks
and when it counts as up; Name says it defaults to the target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 15:57:09 -04:00
bvandeusenandClaude Opus 5.5 30f4de3612 fix(docker): drop the unused .w-meter style; the bars test reads the loaded plugin's models (#5214)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m13s
CI / unit (push) Successful in 1m35s
CI / publish (push) Successful in 1m24s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 15:08:15 -04:00
bvandeusenandClaude Opus 5.5 05eaca3f2f feat(docker): Busiest Containers shows bars of the last hours beside each figure, scaled to the row's peak with a floor (#5214)
CI / lint (push) Successful in 4s
CI / integration (push) Failing after 59s
CI / unit (push) Failing after 1m27s
CI / publish (push) Skipped
The fill bars were a share of the host's capacity, a sliver on hosts with
80 cores. Each figure now sits beside 24 bars of the last 6 hours (1h/24h
per widget), each row scaled to its own peak but never below 1 core or
1 GiB (configurable per widget), so a 0.05-core container reads flat and
a bar's height means real load. One grouped date_bin query feeds them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 15:05:06 -04:00
bvandeusenandClaude Opus 5.5 c3ba850a9c fix(traefik): bucket history with date_bin; integer-epoch division never grouped scrapes (#5256)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 51s
CI / unit (push) Successful in 1m18s
CI / publish (push) Successful in 14s
CI 8551 integration: "integer out of range" for a 2099 timestamp.

The SQL in the log showed a second problem: `CAST(epoch AS INTEGER) / 60`
compiled to `/ CAST(60 AS NUMERIC)`, i.e. decimal division. Every scrape got
its own "bucket", so the request chart and the Traefik page's sparklines
never averaged anything. The int4 cast would also overflow in 2038.

Both now use date_bin from the epoch, as host_agent/metrics_query.py does.
The integration test adds a second scrape 30 s later in the same minute and
checks that the bucket averages the totals and weights the 5xx share over
both scrapes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 13:33:38 -04:00
bvandeusenandClaude Opus 5.5 39b6066f3b feat(traefik): request chart shows the total, a time axis and a weighted 5xx share; says why if it can't draw (#5256)
CI / integration (push) Failing after 53s
CI / unit (push) Successful in 1m23s
CI / publish (push) Skipped
CI / lint (push) Successful in 2s
#5256 "Traefik — Request chart: blank on the live dashboard; time axis and honest labels".

- "Req/s" was the average of the routers, not the total. Each scrape is now
  summed over routers, then averaged per bucket (request_history).
- The 5xx share is now weighted by each router's traffic. The axis is no
  longer pinned at 0–100 (suggestedMax 5).
- The x axis shows times, in the viewer's local time. The tooltip shows
  both series at that time.
- A verdict line comes first: now / peak req/s and the peak 5xx share. The
  figures read even if the chart doesn't.
- The blank live chart's cause is still unknown. Chart.js and
  chart-colors.js both load on the dashboard. A missing library or a thrown
  error is now written into the widget instead of leaving it empty, so the
  cause will show on the page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 13:28:55 -04:00