5c7a4cf9a0c20eb01fcad3c119c313efda5bef7b
45
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  bvandeusenandClaude Opus 5.5
|
f3433fe3ce |
feat(norms): hosts, containers and monitor response times are graded against their own normal (#5742)
Host CPU, memory and load per core, container CPU and memory, and every monitor's response time learn a normal per hour (milestone 513, System 182). A reading past its fixed cutoff but within its normal reads healthy and says what is usual at this hour; one well out of it keeps its grade and is a metric anomaly. Disk, a monitor down and an agent offline keep their cutoffs. - Each normal's floor, the smallest departure that counts, is a setting: Settings -> Thresholds -> Normals (norms.floor.*). Defaults: host CPU 10 points, host memory 5, load per core 10, container CPU 10 points of one core, container memory 5, monitor response 100 ms; Traefik's 2-point 4xx and 50 ms p95 floors move there from constants. learn_normal takes a floor function (floor_setting) read at each pass. - learn_normal(parts=False): a host learns, its cores don't. - Ingest records load_pct (load_1m / cores x 100), the figure pages grade. - Shared grading: anomalies.against / graded, worst_reading(normals=), hosts' host_grades, docker usage(thresholds=, norms=), monitors' latency_grade. Applied to host vitals, the Hosts widget, MCP overview, Docker usage, and the monitor list, widget and page. - The monitor and container pages list their anomalies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
9f14620271 |
feat(traefik): routers are graded against their own normal (#5741)
#5741 "Traefik routers are graded against their own normal", milestone 513. - A router's 4xx share and p95 learn their normal (learn_normal in the plugin's setup). Once a router has 5 days of history, a reading over the fixed cutoff but inside its normal is healthy, and the healthy table says it is usual for that router. One out of its normal is a warning that says what is usual, and a metric anomaly. - The 5xx share is a failure: fixed cutoff always, whatever the history or traffic (System 182's ruling). - A 4xx share over fewer requests than Settings -> Thresholds' new "Fewest requests to judge 4xx" (30 per 15 minutes) isn't graded on the page, and isn't judged for anomalies. - Core: learn_normal(gate=(metric, least)) judges a key only while its resource's gate metric averages at least `least`; `least` may be a function, so it follows a setting. learn_normal(since=...) learns only from history after a moment: Traefik latency learns from #5755's per-scrape figures only. - The Traefik page lists the routers' metric anomalies of the last 30 days (_anomaly_card.html). The dashboard widget keeps its why to problems. - Thresholds and docs say when the cutoffs give way to a normal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
5a0e150789 |
refactor(docker): container CPU and memory live in the one metric store (#5752)
#5752 "Container CPU and memory history lives in the one metric store; docker_metrics and docker_metrics_hourly are retired". Container cpu_pct and mem_pct already went to plugin_metrics for alerting; now mem_usage_bytes does too, and every snapshot in an ingest (backlog included) is recorded at its own time. The container history chart, Busiest Containers, a service's history and the MCP container tools read plugin_metrics under resource "<host>/<container>", rolled up and pruned by the shared metric retention. docker_012_drop_metrics drops both tables. Their history is not copied (lesson #5221 "A data migration's cost is set by the live table"); the store already holds 7 days of container CPU and memory %. Docker retention now prunes only lifecycle events, and Settings -> Thresholds has one "Metric retention" card for every source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
5569f9934e |
feat(anomalies): a metric out of its normal is an anomaly to review (#5740)
#5740 "A metric that departs from its normal is recorded as an anomaly to review, and is findable after it ends", milestone 513. - Every minute each learned key's last 15 minutes is judged against its hour's normal: above the usual hold x1.25 and by the metric's floor opens an occurrence, which ends when the level comes back. - metric_anomalies holds one row per item to review; a repeat while it still waits joins it as an occurrence (metric_anomaly_spans). - An ended row nobody reviews within anomalies.review_days (14, Settings -> Thresholds) lapses; an ongoing one never does. - Learning skips every hour an occurrence covers unless its anomaly was marked normal: no normal is learned without review. - Alerts -> Anomalies lists those to review first, then the reviewed and lapsed under day dividers; marking is operator/admin and audited. - The Overview widget and get_overview warn of open ones and count those to review; the host page shows its own; the weekly report lists the week's and the lapsed; MCP list_anomalies and review_anomaly. - Reviewed and lapsed rows are pruned at data.retention_days, and kept in backups like alert history. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
49799bd0b3 |
feat(backup): daily backups to an S3 bucket, Settings → Backups, and the guide (#5673, #5675)
#5673 "S3 client and the backup job: settings, schedule, history, metrics": - steward/backup/s3.py: SigV4 over httpx, path-style; put and list only (an append-only key is enough; retention is the bucket's). Signing is held to AWS's published examples. - steward/backup/job.py: a 15-minute tick. The daily backup goes at the hour chosen on the reader's clock; a missed day is made up, and a failure is retried an hour later. History of the last 30 runs, audit (backup.made / backup.failed), and backup_age_secs / backup_bytes under "Steward itself", resource backup, for alert rules. - Settings → Backups: verdict first, then the bucket (secret sealed; a blank box keeps it), the schedule, Test the bucket, Back up now, and the recent runs. #5675 "Docs: backing up to Archives, and restoring": docs/core/backups.md. Also: the audit page now words the Prometheus target actions from #5668, which showed as codes, and a guard fails on any logged action without wording. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
eea29123f9 |
fix(settings): a plugin's password is sealed at rest, and a blank box keeps it (#5671)
The UniFi password sat in plugin.unifi in plain text: SECRET_KEYS covers whole settings, and a plugin's settings are one dict. Saving the plugin's settings page also wiped it, since the password box is never filled back in and its blank value was stored. - A plugin field whose name holds password, secret or token is a credential (is_plugin_secret). set_setting seals it inside the dict, every read opens it, startup seals any plaintext left from before, and the undecryptable-secrets banner names it as plugin.<name>.<field>. - The settings page uses the same test for its password boxes, and a blank box keeps the saved value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
defad2e98b |
feat(email): delivery you can see: each alert's email in history, outgoing email on Settings -> System (#5509)
#5509 "Delivery you can see: each alert's email state in history, and outgoing email on Settings -> System" (milestone 501). - Alerts -> History: an alert's email reads from the outbox (#5506): "gathering" with when it sends, "sending", "retrying" with the attempt, next try and the server's last answer, "sent", or "failed" naming each reader it didn't reach and why (outbox.delivery / email_states). Events from before the outbox keep what they recorded. The Notified column shows once an alert has a queued email. - Settings -> System gets "Outgoing email": every email of the last 7 days not yet sent, failures first, then the longest retrying, with recipient, kind, state and last error, and a count of what was sent. Failures and anything still retrying after 5 minutes lead the page's verdict (outbox.outgoing). - get_alert_history (MCP) adds `email`: the same delivery state. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
9a3e7ddba2 |
feat(email): the report, invites, resets and test sends go through the outbox; only it talks SMTP (#5508)
#5508 "Every other email through the outbox: weekly report, invites, password resets, and the test buttons" (milestone 501). - outbox.send_each queues one email per reader (claimed, so the sender doesn't take it too) and tries them at once, for a page that answers with what happened; outbox.said words the result. attempt takes a cfg for a test of the form's unsaved settings and returns the settled rows. - Weekly report (scheduled and Send now) and invites: queued, tried at once; one the server couldn't take yet is retried for an hour. An invite that didn't go shows its link with the reason, as before. - Password reset: queued in the reset link's own transaction; the sender delivers it on its next tick. - Settings -> Notifications test email and a rule's Send test: queued and recorded with retry=False, tried once (System 156 ruling). - Removed mail.send, notifications._send_email and _smtp_send_sync; dispatch_notifications is webhook only. A guard asserts only the outbox (and smtp_open itself) opens an SMTP connection, with a self-test. Also fixes run 8941 (#5507's integration test): its Also-send-to addresses were "@test", which is_real_address rightly drops; they are example.com now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
0b2fbf13fc |
feat(settings): Settings → System shows table sizes and what the last cleanup removed (#5469)
#5469 "Settings → System: table sizes and what the last cleanup removed". - Storage: each table's size on disk (pg_total_relation_size, indexes and TOAST included) and Postgres's row estimate (reltuples), largest first, the 15 largest by name and the rest summed, under the database's total size. Read from the catalog only, never by counting rows. - Last cleanup: when it ran, how long it took, and what each pass removed (each aged table, run log files, the metric and Docker rollups, the log ring), or "Nothing was past its window". run_cleanup now returns that summary and keeps it as the cleanup.last_run setting, as reports.last_sent_at does; cleanup_secs (#5467) is recorded from the same figure. - A hint links to where each retention window is set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
c8dbe811f8 |
feat(health): Steward records its own health and Settings → System shows it (#5467)
#5467 "Steward's own health: metrics, rules can watch them, a page that shows them". The self-monitor tick now records, under source "steward", resource "process": ingest_posts / ingest_failed / ingest_slowest_ms (the host-agent posts since the last tick, tallied in steward/core/health.py), db_roundtrip_ms (a timed SELECT 1), and unwatched_secs (the gap before this tick past watch.GAP_S, from Steward's own last sample). The hourly cleanup records cleanup_secs. The open-file pair is still recorded, and is now only skipped on its own where /proc is absent. All of these are in CORE_METRICS, so an alert rule can watch them under "Steward itself". Settings → System (/settings/system/, admin) shows a verdict first (failed posts, a gap in watching, open files near the limit), then each measure now and over the last day, with a sparkline scaled to its own peak. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
2995cfdd68 |
fix(users): a password reset signs out every session begun with the old password (#5427)
#5427 "A password reset doesn't end the sessions already signed in with the old password": users.sessions_since (migration 0038) is set on reset; refresh_session_user ends a session whose login_at is earlier. An admin resetting their own password carries on signed in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
2b6e12859a |
feat(users): invite by email — a one-time link where the invitee makes their own account (milestone 496)
The operator's ask: an invite link sent by mail, the key in the URL, and the new user choosing their own username, password and email. - user_invites (migration 0037): the key's sha256 only, the address, the role and days the admin chose, who sent it, used/revoked. - core/invites: create (revokes an older open invite to the address), renew (new key, new clock), find_pending, the four states, signup checks. - /invite/<key>, outside sign-in and throttled like login: the invitee's username, email (prefilled, theirs to change) and password; the account takes the admin's role, the key is used up in the same transaction and they are signed in. A bad key gets one plain message. - Settings › Users: Invite by email (address, role, 1/7/30 days), sent from the admin in their zone; without a mail server, or if sending fails, the link is shown once to copy. Invites not yet used are listed with Resend and Withdraw. - Audit: user.invited, user.invite_resent, user.invite_revoked, user.joined. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
beae8a33f3 |
fix(account): one form and one Save for your account; the same for an account's page in Users
Operator review of the Account page: two sections with two different save buttons, one stuck to the next card and one beside a squeezed hint. - Account: time zone and Email are one form; POST /account/ saves both, nothing unless all of it is valid. /account/email is gone. - Users › account: Name and Email are one form; the rename route also saves the email switches. /settings/users/<id>/email is gone. - The Email section is a plain section now (email_section); its hint sits under the switches, names the zone as the picker does, and links Notifications only for an admin. - apply_email_form leaves a switch the form doesn't carry as it is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
7ae6f85032 |
fix(settings): Notifications opens on first visit — no second begin on an autobegun session
Reading the settings autobegins the session's transaction, so the nested db.begin() that saves the first admin's zone as smtp.tz raised InvalidRequestError and every visit 500'd until a zone was saved. The integration test signed in without a zone, so that path never ran; it now has a test of its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
22090b02d8 |
feat(email): each account gets its own email, in its own zone (milestone 495)
Milestone 495 "Email goes to each account, in its own time zone", on the operator's ruling (System 156): notifications go to each user, times in their zone, defaults on, toggled per user or per role. - users.notify_alerts / notify_report (migration 0036): None follows the role's default, notify.<kind>.<role> in settings, all on to start. - mail.pick_recipients: active accounts that want the email and have an address mail can reach (an SSO placeholder like name@oidc is skipped and flagged), each in its own zone with no UTC beside it, then the Also send to addresses in their chosen zone with UTC beside. One copy per address, all over one SMTP connection. - Alerts, the weekly report and the test email (to you, in your zone, plus the typed addresses) send that way. The save-time smtp.tz capture is gone. - Notifications: a role × email table of defaults, who each email reaches and who can't be reached, and Also send to with its own zone, which starts as the first admin's to open the page. - Account page: your address and your two switches. Users: an Emailed column, and an admin sets an account's switches on its page. - The Alerts page counts the accounts it emails. zone_select macro and formatting.zones_by_region shared by Account and Notifications. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
c533eec33a |
feat(settings): Users page — list, add, rename, role, password, disable, delete (#5336)
#5336 "Users page: list accounts, change role, disable, reset a local password, create a local user". - Settings › Users (admin only, a Settings tab): every account with how it signs in, its role, last sign-in (from the audit log) and working AI tokens; disabled accounts last. A row opens the account. - An account's page: rename (any account), role and password (local accounts; an LDAP or single sign-on role comes from the provider at every sign-in, and the page says so), disable or enable, and Delete behind the confirm dialog. - Refusals are said beside the control: nobody disables, demotes or deletes themselves, the last active admin stays, and an account that made alert rules, maintenance windows or playbook runs is disabled rather than deleted (those records keep who made them). - Add user: a local account with a role and a password of 8+ characters. - Every change is audited as user.*, and the audit log names and links the account. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
ec55faf737 |
feat(email): alert and weekly-report emails read as designed messages (#5335)
#5335 "Emails Steward sends read as designed messages: an HTML alert email and weekly report, with a plain-text twin". - core/mail.py renders each email as multipart/alternative from templates/email/: a table-and-inline-style HTML part and a plain-text twin carrying the same facts. Colours are the design system's light theme, resolved from theme.css; nothing is fetched when a mail opens. - Alert email: state badge, a headline sentence ("Disk used on web01 is 94%."), rule, resource, reading, the condition in words, and the time in the zone Notifications was saved in with UTC beside it; "View in Steward" when Steward's address is set. Subject "[Steward] Firing: <rule> — <resource> 94%". Alerts now carry their operator. - Weekly report: monitors down, active alerts, the week's alert events, uptime and busiest Traefik routers as small tables with links. Sent from the configured From address; the "Generated … UTC" line and the stale repo footer are gone. - condition_text() states a condition from a rule's parts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
24700ee608 |
feat(settings): Settings brought into line with the rest of Steward (milestone 493)
The last page of #5214's walk. Settings still described Steward as it was before monitors were unified and SNMP was rebuilt, drew its tabs below the breadcrumb, and showed the report schedule in UTC. - Tabs (#5377): settings/_tabs.html is a settings_tabs() macro called in each page's section_tabs block, above the trail, as Ansible's are. The breadcrumb guard also fails any page that {% include %}s a tab strip. - Report time (#5378): reports.schedule_tz keeps the zone of whoever saved the schedule; check_and_send compares in it (report_due), and the form shows the day and hour on the reader's clock (schedule_in) under their zone's name. "Last sent" uses |when. - Wording (#5379): the report contents list what the report holds (monitors, not ping-enabled hosts); "Ping latency" is "Response time" and says what it grades; plugin detail says enabling loads now and disabling needs a restart; the monitor interval says it applies on the next check (the scheduler reads it every pass); the dead list-config branch is gone. - Traefik (#5380): the 5xx and 4xx cutoffs are settings (traefik.threshold.error_5xx_pct 0.1, error_4xx_pct 1.0) in a "Traefik routers" group on Thresholds while the plugin runs, read by router_state through error_cutoffs(). The Traefik page has a Thresholds button for admins, to /settings/thresholds/#traefik. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
f67678998c |
refactor(plugins): no plugin catalog; plugins ship in the image or are mounted (#5374)
First-party plugins have shipped inside the image since
|
||
|
bvandeusenandClaude Opus 5.5
|
de3958a362 |
fix(settings): the plugin catalog is fetched by a scheduled task, never by a page load
The operator's ruling (Background Jobs, 2026-10-08): a Steward page reads stored records; the sources are contacted by scheduled collection. An audit of every GET handler, HTMX fragment, widget poll and MCP read tool found one page load that reached out: Settings -> Plugins' catalog fragment fetched each plugin repository (5 min cache, 10 s timeout each, retried on every load when unreachable). plugin_catalog_refresh ticks every minute and fetches a repository whose copy is missing or older than an hour, waiting 5 min after a failure; saving repositories clears the cache, so the next tick fetches. The fragment only reads the cache and says which repositories are not fetched yet. Refresh is now a POST that fetches at once, as does install. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
96c1ea9407 |
feat(host_agent): how far each host's clock is from Steward's (milestone 490)
- Agent 1.12.0 stamps each post with sent_at; ingest compares it with the moment the post arrived: clock_offset_s / clock_checked_at on the registration (host_agent_003), and a clock_off_secs metric (either way) that alert rules can watch. Replaces the old skew log, which measured the newest sample and was off by up to an interval. - Settings -> Thresholds: Clock drift, 30 s warn / 5 min critical. - Host page: 'clock 0.3s ahead' in the vitals line, toned past the warning. - Hosts list: names hosts whose clock is off; when every reporting host is off by about the same amount, says Steward's own clock is the likely one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
98854d94f9 |
feat(platform): tini as PID 1, Hypercorn, a log-level setting, the image on CI's Python, an airgapped boot gate before push (#5159)
- Dockerfile: python:3.14-slim, the Python CI tests on (was 3.13). tini from apt is PID 1 (ENTRYPOINT tini -- entrypoint.sh): it reaps the zombies ssh and ansible leave and forwards SIGTERM. - cli: Hypercorn's serve() with one worker (run, stream and scheduler state is in-process), a 10 s graceful timeout, and SIGTERM/SIGINT as the shutdown trigger. Access and error lines go through Steward's logging. --debug keeps Quart's reloading dev server. - Logging: steward/core/logging_setup.py installs one root handler before anything logs (until now a library's basicConfig on import was the only configuration). Its level is the `log.level` setting (DEBUG/INFO/WARNING/ ERROR, default INFO), applied at boot and on save in Settings → General. - CI publish: build, then "The image boots with no internet" (rule 164, snippet #3552), then push, so a broken image never reaches a tag (rule 177). On an --internal network with its own Postgres, the image must migrate and answer /health, run tini as PID 1, serve first-run setup with its assets from /static and no third-party host, and exit 0 on SIGTERM. Logs and clean-up run always. Tests: tests/core/test_logging_setup.py; the smoke step itself. Not in this commit: the dependency lockfile (it conflicts with rule 154 and is waiting on the operator). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
edcfa4d3db |
fix(ansible): git sources work in the image and survive a redeploy; pulls time out; alert vars go in a JSON file (#5157)
- git ships in the image; checkouts live under /data/ansible/sources (STEWARD_ANSIBLE_SOURCE_DIR), on the persistent volume. - One task, ansible_git_pulls, reads the sources every 30 s and pulls each as its interval passes: a source added in Settings is pulled without a restart, a removed one is forgotten. - git_pull: a 120 s timeout ending the process group, GIT_TERMINAL_PROMPT=0, pull --ff-only, a failed clone removed; it raises GitPullError with git's message, so the Sync button stops saying Synced after a failure (and the message is now escaped). - An alert action's vars and the firing's steward_alert_* vars reach ansible-playbook as one JSON extra-vars file, not -e key=value, which Ansible split on whitespace (a rule named 'High CPU on web'). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
9fdd67a675 |
feat(retention): every growing table is pruned; the audit log has its own window; charts read the raw window from Settings (#5155)
- prune_by_age(column) registers a time-series column with the hourly cleanup; Traefik (metrics, access summaries, global stats) and UniFi (WAN, client, WLAN, DPI, events, alarms, speedtests) register theirs in setup(). - Alert events join monitor results and Ansible runs on data.retention_days, read from Settings each run; Ansible run log files older than it are deleted. - audit.retention_days (365) keeps the audit log longer, set on Settings > General. - metric_series.points reads metrics.retention.raw_days, so the chart's raw/rollup seam is where the rollup put it (RAW_DAYS removed). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
35fca57397 |
feat(notifications): JSON-safe webhook template, SMTP timeout and implicit TLS, test-send buttons (#5064)
#5064 "Notifications that arrive": - render_webhook: alert.* is escaped to sit inside a JSON string (a quote or newline in a rule name no longer breaks the body); raw.* is the plain value for |tojson; a missing value reads "no data"; the template runs sandboxed and the body is checked to be JSON before it is sent. - SMTP: 15s timeout; port 465 connects with implicit TLS, otherwise the box means STARTTLS; a From address setting (falls back to the username). - A webhook error no longer carries the URL, which is a secret. - Settings → Notifications: "Send a test email" / "Send a test webhook" send a sample alert with the form's values (unsaved) and show the result and the body that went out; template help names alert.* and raw.*. - Notification audits go through audit_request; the test send is audited as settings.notification.tested, with the channel and outcome only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
faa605cead |
feat(secrets): webhook URL and git tokens sealed; run output masks its secrets (#5037)
#5037 "Secrets at rest and in run output: webhook URL and git token encrypted, askpass via env, secret values scrubbed from logs". - webhook.url joins SECRET_KEYS; existing values are sealed at startup by migrate_plaintext_secrets. Settings -> Notifications no longer echoes the URL: it names the saved host, keeps it when left blank, and has a Remove box. - Git source http_token is sealed inside ansible.sources on save and at startup, opened in get_sources, and counted by the undecryptable-secrets scan as ansible.sources[<name>].http_token. - The askpass script holds no token: git runs it with STEWARD_GIT_TOKEN in its environment and printf prints it as data. - The executor masks every secret the run is handed (become/vault passwords, bootstrap passwords, sensitive run vars, and the per-host agent tokens the host_agent plugin now passes as redact) in each line before it is broadcast, logged or stored. - Bundled "Write agent config" tasks are no_log; a test fails if any bundled task templating steward_token lacks it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
f944cdb9b0 |
feat(auth): LDAP and SSO users keep their source; directory TLS is real (#5036)
#5036 "SSO and LDAP: users keep their auth source, no merging into local accounts, is_active checked, LDAP TLS real and filter escaped". - users gain auth_source (local/ldap/oidc) and external_id (migration 0027). A provider account is found by (source, subject) only: the LDAP entry DN or the OIDC sub. A first sign-in whose username or email another account holds is refused with a message naming why; a disabled user stays out. - Local passwords sign in local accounts only. - LDAP: port 636 is LDAPS, otherwise StartTLS before any bind; the server certificate is checked against the system CAs or a CA pasted in Settings -> Auth. The login name is escaped into the filter (RFC 4515), and an empty password is refused before the directory sees it. - OIDC: the ID token must carry this sign-in's nonce and the userinfo sub. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
7a3f7869fe |
feat(auth): sessions end when they should; logins are throttled; Secure behind HTTPS (#5035)
#5035 "Sessions: lifetime applied, role and is_active re-read per request, logout is a POST, failed logins throttled". - steward/core/proxy.py: trusted_value / is_https / client_ip read X-Forwarded-Proto and -For only as far as the trusted proxy hops (Settings → General, default 1, 0 = trust none). - The session cookie is Secure, and HSTS is sent, only on requests that arrived over HTTPS by that rule; plain-HTTP and LAN use are unchanged and nothing redirects. - A login lasts Settings → General's session lifetime from when it began (session.permanent + a login_at stamp checked every request — the cookie's own age slides on every response). A deleted or deactivated user is logged out on their next request; a role change applies at once. - Logout is a POST (a form in the sidebar); GET /logout is gone. - Failed logins pause an address after login.max_failures within login.lockout_minutes (Settings → General; defaults 5 in 15 min), keyed by address so nobody can lock a user out by name. Failures are audited as auth.login_failed with the address. - to_access_cfg projects the new settings into app.config at startup and on save, beside the existing projections. - Integration tests seed real users and a login_at, since every request now re-reads the session's user. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
d188a673a6 |
feat(plugins): installs fetch only what the catalog names, verified (#5032)
#5032 "Plugin install takes its URL and checksum from the catalog, requires the checksum, and extracts only inside the plugin folder". The install form carried the download URL and checksum as hidden fields, so any admin POST could make Steward fetch and import code from anywhere, unchecked, and a zip member like ../x.py was written wherever it pointed. - The route takes only the plugin name and reads download_url and checksum_sha256 from the configured catalogs; a name no repository lists is refused. Installs are audited (plugin.install / plugin.install_failed). - The checksum is required; the name must be a plain identifier. - extract_plugin refuses any member that resolves outside the plugin folder, caps the download at 50 MB and the unpacked size at 200 MB. - The zip unpacks into a staging folder and is swapped in whole: a refused or failed install leaves the installed copy untouched, and an update no longer leaves the old version's deleted files behind. - The catalog shows "Can't install: the catalog lists no checksum" in place of the button for an entry without one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
21338dbcd9 |
feat(mcp): docker and host-agent tools; TLS and crash-loop cutoffs in Settings
MCP-5 of milestone 461 "MCP server: Steward as an LLM-inspectable, LLM-operable hub". Contributed through each plugin's register_mcp_tools hook, so they exist only while the plugin is enabled. docker (viewer, read-only): - list_containers: filter by host, status, compose project, or problem (restarting, crash_looping, unhealthy, oom_killed, exited_nonzero); problems sort first and each row names what is wrong. - get_container: state, CPU/memory history across the raw/hourly seam, lifecycle events. A unique part of a swarm task name is enough. - list_swarm_services: replicas per node (agent-less placements as ghosts) and the swarm's nodes. - search_container_logs: text (% and _ literal), stream, range, context lines, paging by a full-precision `before` cursor; over-long lines keep both ends and say how much of the middle was cut. - get_docker_disk_usage: bytes per host, reclaimable first. - mcp_overview: failing containers and under-replicated swarm services join get_overview. host_agent: list_agents (reporting / stale / pending, version, OS), and an overview finding for agents registered over 30 minutes ago that never checked in. Settings > Thresholds gains the TLS-expiry warn/crit days (were 14/3 in code) and the crash-loop deaths-per-hour count (3), which get_overview and the docker tools now read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5
|
0da88c7076 |
fix(plugins): show load-failure reasons on the page the banner links to
The admin banner counts every plugin load failure, but settings/plugins.html only rendered failure reasons inside the plugin card macro -- which iterates discovered plugins. A plugin that failed BECAUSE it has no directory was therefore counted in the banner and shown nowhere on the page the banner links to. That dead end is what the operator hit: "1 plugin failed to load" on the dashboard, nothing wrong on the details page, and no way to clean it up. Folds the reason into the "Configured but not installed" rows rather than adding a competing section, and drives that list from the UNION of stored plugin.* settings and recorded load failures instead of stored settings alone. A failure with no stored row -- a plugin enabled by a DEFAULTS entry, which is exactly what plugin.http was -- would otherwise still be counted and still be invisible. Rows carry `removable`: only a plugin with a real stored row can be cleaned up, so a default-declared one shows the reason and an explanation instead of a Remove button that would delete nothing while the default reasserts it. Rows with a reason read as an error (red, "Failed to load"); rows that merely have leftover config stay a warning. A discovered plugin that failed is excluded -- its own card already shows the reason. Replaces find_orphaned_plugin_names outright rather than adding a second overlapping helper; it had one caller. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5
|
59fece855d |
fix(plugins): remove the phantom http plugin; surface orphaned plugin settings
Operator saw the `http` plugin -- deleted when ping/dns/http were unified into the Monitor entity -- still reported as enabled, with no way to clear it. It was never an orphaned row. `plugin.http` was hardcoded in core DEFAULTS, so deleting the app_settings row did nothing: the default reasserted it on the next settings load. That is precisely why no cleanup UI could have fixed it. Rule 22 says the removed subsystem should have taken its setting with it. Purged the surviving references -- the DEFAULTS key, "http" in CAPABILITY_PLUGINS, the stale plugin_manager docstring example, and the capabilities blurb still advertising HTTP/uptime as a bundled capability -- plus a migration dropping any stored plugin.http row. Untouched: `http` as a MONITOR TYPE (icmp/tcp/dns/http) everywhere it appears, and http_001_initial, which is kept deliberately so existing DBs resolve the revision graph. For the general case, cleanup splits by provenance rather than being uniformly automatic or uniformly manual: Bundled plugins ship in the image and version atomically with core, so they cannot be transiently missing -- a plugin.* default with no bundled directory is unambiguously a bug. Guarded by a unit test that fails CI, which is the only part safe to automate. External plugins live in operator-mounted /data/plugins, where absence is ambiguous: unmounted volume, failed install, mid-upgrade. Auto-deleting their config would silently destroy unrecoverable credentials on a transient condition, so Settings now lists them under "Configured but not installed" with an explicit, confirmed, audit-logged Remove. The section hides entirely when empty, and the remove route refuses if the plugin is actually installed. Orphan detection reads STORED rows, never the DEFAULTS-merged view -- offering to remove a default-only key would be a lie -- and returns names only, since plugin config can hold credentials and listing orphans never needs their values. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
8f1c8c5cf7 |
feat(docker): container log viewer + Settings controls [M79 step 5]
The user-facing half (rule 27). Per-container log viewer + admin controls, so container logs are something the operator can actually touch. Viewer (plugins/docker): - /container/<host>/<name>/logs full page + /logs/lines HTMX fragment polled every 5s (near-live follow); stream filter (all/stdout/stderr) + case-insensitive text search; newest-first so the latest lines survive the poll's scroll reset; empty/loading states; a "View logs" link from the container detail page. Jinja auto-escapes log content (no markup injection). Settings (Thresholds & Retention tab): - global on/off toggle, comma-separated exclude list, retention days + max MB per container — DB-backed, no restart. The toggle + exclude are enforced authoritatively at ingest (_persist_logs drops disabled/excluded lines), since the push model has no channel to tell an agent to stop. Tests: route-defined smoke; template-parse covers the new templates; integration test for the ingest-time toggle + exclude enforcement. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAGR73DUowdVFVvYzLXC5C |
||
|
bvandeusenandClaude Opus 4.8
|
8af297670e |
feat(metrics): roll plugin_metrics up to hourly to bound storage
plugin_metrics grows by (sources × resources × ~30s cadence); keeping 90d of raw
is a large table. Add a raw→hourly rollup (mirroring the Docker plugin) so only a
short raw window is kept at full resolution, with hourly averages archived longer.
- PluginMetricHourly model + core migration 0024 (plugin_metrics_hourly: avg/max/
count per source/resource/metric/hour, unique bucket constraint + lookup index).
- steward/core/metrics_retention.rollup_plugin_metrics: date_trunc('hour') agg of
raw older than the hour-aligned raw window, idempotent pg upsert into hourly,
delete the rolled raw, prune hourly beyond the rollup window.
- cleanup.py: plugin_metrics is no longer blanket-deleted at data.retention_days;
_run_metrics_retention drives the rollup with windows read live from settings.
- Settings: metrics.retention.raw_days (7) + rollup_days (90), tunable on the
Thresholds & Retention page (new "Host metrics retention" card).
- Chart read: _history_for_host merges the hourly rollup (older part of the range)
with raw date_bin (recent part, capped ≤1h), so 30d charts keep working —
recent at full resolution, older at hourly. Route passes raw_days from settings.
- Tests: unit (cutoff helpers) + integration (rollup aggregates/prunes; history
merges hourly + raw) against Postgres.
Speed was already handled by the indexes + SQL aggregation; this is the storage
lever (raw window ~10x smaller).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
|
||
|
bvandeusenandClaude Opus 4.8
|
faecac3ec6 |
feat(docker): retention + hourly rollup for metrics/events with Settings windows
Bounds Docker time-series growth (the main scaling concern). New docker_metrics_hourly table + docker_006 migration; a plugin retention module (docker.run_retention capability) rolls raw docker_metrics older than the raw window into hourly averages (idempotent upsert), deletes the rolled raw rows, then prunes stale rollups + lifecycle events. Core cleanup.py drives it each hourly run via the capability (no plugin-model import), reading the three retention windows fresh from settings so changes apply without restart (rule 25). Settings → "Thresholds & Retention" gains a Docker retention card (raw / rolled-up / events windows, working defaults 7/90/30 days). Unit tests cover the hour-aligned cutoff/bucketing helpers; integration test exercises the real rollup-average + prune across both windows. Milestone 77 task #941. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC |
||
|
bvandeusenandClaude Opus 4.8
|
591706bd39 |
feat(settings): configurable monitoring thresholds
Move the hardcoded warn/crit cutoffs into Settings -> Thresholds (DB-backed, live, no restart). New thresholds.* keys + to_thresholds_cfg() + a threshold_style(value, kind) jinja global that reads them; latency reuses the existing ping good/warn keys, uptime is direction-aware (floors). Replace the _macros metric_style/uptime_style macros (now removed) with the global across Hosts-Overview, host_agent fleet + panel, Uptime/SLA widget, and the ping page uptime column — all now honor the configured cutoffs. Uptime keeps its green 'good' look when not degraded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
7ef1af2184 |
feat(settings): Phase 4 — capabilities vs integrations + nav cleanup
Final phase of the host-IA unification (milestone 70). - Settings → Plugins split into two tiers: "Monitoring capabilities" (host_agent, http, snmp, docker — built-in host facets, surfaced via Hosts/Status, on by default) and "Integrations" (traefik, unifi — external systems, off until configured). Presentation only: a CAPABILITY_PLUGINS set (overridable by plugin.yaml `kind:`) tags each plugin; module loading, optional deps, and migrations are untouched. - Drop the "default-enable a plugin" framing in the UI copy — capabilities are described as built-in, not optional add-ons. - Nav: remove the standalone "Uptime" item (folded into Hosts; still reachable via the SLA button on the Hosts list). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
6a8146b544 |
fix(ansible): replace steward key on reprovision + distinct agent update path
Provisioning review corrections + the matching frontend, plus breadcrumb header integration. - provision.yml: authorize the managed pubkey with a regexp match on the ' steward-managed' comment so rotating the key REPLACES the host's steward key in place instead of stacking a second authorized entry. Hand-added keys (other comments) are untouched. - update.yml (new): refresh agent.py + restart only. Does NOT rotate the token or rewrite /etc/steward-agent.conf — the host keeps its identity. Asserts the agent is already installed and fails clearly otherwise. - host_agent /update route: runs update.yml as the managed steward user (no token minting). Token rotation stays a deliberate action. - settings/ansible/generate-key honors a safe relative `next` redirect, so an inline trigger elsewhere returns to its page. - Host Agents settings: reworked into a clear lifecycle — an intro card that explains it runs Ansible to deploy the agent (+ inline "generate managed key" warning/trigger when none exists), then three labelled cards: 1 Provision, 2 Install/enroll, 3 Update. Each explains what it does. - base.html: breadcrumb now renders as a kicker line directly above the page title (moved below alerts, tightened margin) so nested and top-level views share one consistent header. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
0318f6423f |
feat(ansible): host provisioning via steward managed SSH identity
Turn the agent-install playbook into a full provisioning + maintenance path. Solves the bootstrap chicken-and-egg: first contact uses an operator-supplied password (one run, never stored), which creates a dedicated `steward` login account with NOPASSWD sudo + Steward's managed public key. Every run thereafter connects as `steward` with the managed key — fully unattended (scheduled prune, agent updates). - core/crypto: generate_ssh_keypair() — ed25519, OpenSSH formats. - settings: ansible.ssh_public_key (non-secret, displayed) + ansible.ssh_user (default steward); to_ansible_cfg extended. - settings UI + route: "Generate managed key" (private encrypted, public shown to copy) + SSH-user field. - executor: build_bootstrap() writes a 0600 vars file (-e @file) for the per-run user/password — never argv, never DB, never logged; drops the managed key when a bootstrap password is given; --user floor from the global ssh_user when no override. - runner.trigger_run: pass-through `connection` kwarg, deliberately NOT persisted on AnsibleRun.params (password stays out of the DB). - bundled/host_agent/provision.yml: create steward user + authorized_keys + /etc/sudoers.d/steward (visudo-validated) + agent install. - host_agent: /provision route + "Provision a fresh host" card (bootstrap user/password; injects pubkey + user + token as space-safe JSON hostvars). - Dockerfile: add sshpass (Ansible shells out to it for password SSH). - tests: keypair generation + build_bootstrap (secret stays off argv). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
88857be24e |
feat(ansible): runner robustness — cancel, concurrency, structured results, retention
Closes #550 (all four): - Cancellation: track live subprocesses; POST /ansible/runs/<id>/cancel (operator) SIGTERMs then SIGKILLs after a grace; new 'cancelled' status (+ migration 0019, ALTER TYPE in autocommit). Queued runs cancel cleanly before launch. Cancel button on run detail. - Concurrency: global semaphore (ansible.max_concurrent_runs, default 3, Settings→Ansible) caps simultaneous runs; excess show 'queued' (new status) until a slot frees. Semaphore bound lazily per running loop. - Structured results: parse PLAY RECAP into per-host ok/changed/unreachable/ failed/skipped + capture failed-task lines, stored in new results JSON column (migration 0020); rendered as a host-summary table on run detail. Keeps live streaming (no json-callback swap). - Retention: full output written to a persistent log artifact (/data/ansible/runs/<id>.log, env-overridable) beyond the 1 MB DB cap and across restarts; in-memory replay buffer bounded + GC'd after completion; Download-log route. Boot reconciliation now also sweeps stale 'queued'. Unit tests for recap parsing + cancel flagging. Status colors updated across run list / detail / schedules. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
d906232eb2 | feat(ansible): GIT_ASKPASS per-source HTTP token auth + settings UI | ||
|
bvandeusenandClaude Opus 4.8
|
0bf007173b |
feat(ansible): credentials — SSH key, become, vault, host-key checking (task 548)
Global Ansible credentials, applied to every run (manual + alert-triggered): - core/settings.py: ansible.ssh_private_key / become_password / vault_password (plaintext at rest, masked in UI — encryption tracked in #580) + host_key_checking (default off); surfaced via to_ansible_cfg into app.config[ANSIBLE] - executor.py: pure build_credentials() materializes creds into a 0600 temp dir (--private-key, --vault-password-file, become via -e @vars-file so the password never hits argv) cleaned up in finally; pure ansible_env() sets ANSIBLE_HOST_KEY_CHECKING. build_ansible_command stays param-only - settings/routes.py + ansible.html: admin-only Credentials section, masked-update (blank keeps current, explicit Clear checkbox), reload app config on save - tests: unit (build_credentials per cred + none; ansible_env toggle); integration vault round-trip (ansible-vault encrypt a vars file, run via executor with the vault password, assert it decrypted) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
af60ca446d |
fix(plugins): complete steward rename across bundled plugins; clear lint debt
The fabledscryer->steward rename had only ever reached host_agent. The other five bundled plugins (http, snmp, traefik, unifi, docker) still imported `from fabledscryer.*` (package no longer exists) and read FABLEDSCRYER_* env vars — so every one of them was broken at import since the original rebrand. CI stayed green only because none are enabled by default and migrations don't import plugin modules. Now that they version in-tree, complete the rename: - fabledscryer.* -> steward.* imports across all five plugins - FABLEDSCRYER_* -> STEWARD_* in plugin migration env.py files - author/repository/homepage + user-facing 'Fabled Scryer' strings -> Steward - snmp/scheduler.py: also drop dead `now`/datetime; record_metric from steward Adds tests/test_no_legacy_names.py — fails if 'scryer'/'roundtable' ever reappear in shipped code (the drift bit twice; this stops a third time). Also clears pre-existing ruff lint debt (unused imports, semicolon statements, mid-file import) surfaced by the new lint lane. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
a7a281cb11 |
feat(plugins): fold first-party plugins in-tree; bundled + external roots
First-party plugins (host_agent, http, snmp, traefik, unifi, docker) are now tracked under plugins/ and baked into the image, so they version atomically with core — ending the cross-repo import drift the roundtable->steward rename exposed. History for these files is preserved in the archived Roundtable-plugins repo. Plugin discovery becomes multi-root: PLUGIN_DIR (single) -> PLUGIN_DIRS (bundled first, then external) + PLUGIN_INSTALL_DIR. Bundled ships in the image; third-party plugins still mount at runtime into the external root (STEWARD_PLUGIN_DIR, default /data/plugins) and downloads/installs land there. Bundled shadows external on a name collision. - config.py: load_bootstrap returns plugin_dirs + plugin_install_dir - app.py: iterate PLUGIN_DIRS at the migration + load sites - migration_runner.py: discover_all_in() unions every plugin root - plugin_manager.py: resolve_plugin_path() (pure, first-root-wins); load / install / hot-reload span all roots; installs target the external root - settings/routes.py: _discover_plugins scans all roots, dedup bundled-first - Dockerfile: COPY plugins/ ; docker-compose: drop host bind, document external - tests/test_plugin_dirs.py: resolution, multi-root discovery, bootstrap split Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.7
|
88ab5b917e |
chore: rename project Roundtable → Steward
Renames the Python package directory, CLI command, env var prefix, docker-compose service/container/image, Postgres role/db, and all visible branding. Marketing form is "Fabled Steward". Clean break from the previous rebrand: drops the fabledscryer→roundtable import shim in __init__.py and the FABLEDSCRYER_* env var fallback in config.py and migrations/env.py. Env vars are now STEWARD_* only. Heads-up for existing deployments: - Postgres user/db renamed fabledscryer → steward in docker-compose.yml. Existing volumes need the role/db renamed inside Postgres, or override POSTGRES_USER/POSTGRES_DB to keep the old names. - Host-agent systemd unit is now steward-agent.service. Existing agents keep running under the old name; reinstall to switch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |