b3a5bbfeffaae2ea727909308bf22be2c4afc51a
38
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  bvandeusenandClaude Opus 5.5
|
593b583d23 |
feat(auth): reset a forgotten password from an emailed link (milestone 497)
Step #5429 "Reset by email: request, link and new-password page" of milestone 497 "Passwords: change your own, and reset it by email". - Sign-in page: "Forgot your password?" while a mail server is set. - /forgot takes a username or email and answers the same either way; only an active local account with a real address gets mail. Requests count per address on their own throttle, so they never lock out sign-in. - password_resets (migration 0039): a one-time key, stored as its sha256, working for an hour; asking again replaces it. - /reset/<key>: the new password, then signed straight in; every other browser is signed out (sessions_since). Bad keys count as invite keys do. - Email "[Steward] Reset your password", in the account's own zone. - Audited: user.password_reset_requested, user.password_reset_by_email. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
c533eec33a |
feat(settings): Users page — list, add, rename, role, password, disable, delete (#5336)
#5336 "Users page: list accounts, change role, disable, reset a local password, create a local user". - Settings › Users (admin only, a Settings tab): every account with how it signs in, its role, last sign-in (from the audit log) and working AI tokens; disabled accounts last. A row opens the account. - An account's page: rename (any account), role and password (local accounts; an LDAP or single sign-on role comes from the provider at every sign-in, and the page says so), disable or enable, and Delete behind the confirm dialog. - Refusals are said beside the control: nobody disables, demotes or deletes themselves, the last active admin stays, and an account that made alert rules, maintenance windows or playbook runs is disabled rather than deleted (those records keep who made them). - Add user: a local account with a role and a password of 8+ characters. - Every change is audited as user.*, and the audit log names and links the account. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
f67678998c |
refactor(plugins): no plugin catalog; plugins ship in the image or are mounted (#5374)
First-party plugins have shipped inside the image since
|
||
|
bvandeusenandClaude Opus 5.5
|
de3958a362 |
fix(settings): the plugin catalog is fetched by a scheduled task, never by a page load
The operator's ruling (Background Jobs, 2026-10-08): a Steward page reads stored records; the sources are contacted by scheduled collection. An audit of every GET handler, HTMX fragment, widget poll and MCP read tool found one page load that reached out: Settings -> Plugins' catalog fragment fetched each plugin repository (5 min cache, 10 s timeout each, retried on every load when unreachable). plugin_catalog_refresh ticks every minute and fetches a repository whose copy is missing or older than an hour, waiting 5 min after a failure; saving repositories clears the cache, so the next tick fetches. The fragment only reads the cache and says which repositories are not fetched yet. Refresh is now a POST that fetches at once, as does install. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
6a81cf4aac |
feat: times in the reader's own zone; the audit log shows exact moments (milestone 490)
- users.time_zone (0034). An account without one takes the browser's zone on its first page (POST /account/time-zone/adopt, never over a choice made by hand); the session follows the account on every request. - /account: who you are and the zone every page writes its times in, reached from your name in the sidebar. - formatting.when: 'Today 06:58', 'Yesterday 23:41', 'Tue 6 Oct 14:12', '6 Oct 2025 14:12', 24-hour, in the reader's zone; |ago's hover is the exact local moment instead of UTC; tz_label() for column headers. - Audit log: When (EDT) with exact times, one detail fact a line, on classes. - tzdata in the image, and the airgapped smoke checks a zone resolves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
5ec930bf76 |
feat(ui): Disk, Traefik, UniFi and Settings say things plainly (#5212)
#5212 "Plain labels: no raw ids, scopes, metric keys or unexplained jargon", batch 3: - Disk page: each tile says what it holds. "Writable layers" becomes "Container files" and Build cache says it is safe to clear. Shared is "Shared with other images" with a tooltip, and "dangling" is gone. Untagged images show by id instead of "<none>", a capped list says "The 50 largest of N images", and each host shows "as of" its report. - Traefik: an expired certificate reads "expired 12 days ago", not "-12 days" (cert_left). The Traffic empty state knows whether the access log is on: off points to Settings → Plugins → Traefik; on says to check the mounted log path. - Moments use |ago with an exact UTC tooltip: UniFi last seen, alarms, events, speed test and DPI; AI token last used. - Settings: the Auth tab is "Authentication" like its breadcrumb, plugin fields read "Metrics URL" not metrics_url (field_label), a source's pull interval is "every 1h", and the last report time drops the ISO "T". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
3a772716ae |
feat(ui): audit, alert rules and the rule form say things in words (#5212)
#5212 "Plain labels: no raw ids, scopes, metric keys or unexplained jargon", batch 1: - Audit log: each action reads as a sentence ("paused monitor"), with the code as its tooltip. Detail shows flags as yes/no, lists joined, a run's scope by name, spans as spans, and no ids the entity column already gives. The filter is now a picker of areas. - Alert rules, history and Status say the condition in words: "is down", "Fullest disk >= 85%". Readings use metric_value, and history times use |ago. - Rule form: sources and metrics by name, the threshold's unit, "fires when it is down/up" for an up/down metric instead of 0.5, and a metric reference built from the catalog. - Schedules and Maintenance: viewers no longer see Run now, Pause, Edit, Delete or New. Intervals and next runs read as "every 2h" and "in 3h"; a scope reads "Every alert on Host agent". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
8941ed0372 |
fix(metrics): seed the latest table in the background, not in the migration (#5208)
Migration 0030 seeded plugin_metric_latest from the whole history: two index probes per distinct metric key. On the live instance that held the first boot for 7+ minutes with Steward down. The migration now only creates the table. Live sources fill it on their next sample, and backfill_latest runs after startup for resources that have history but no latest rows. Per resource it does one backward probe for the last sample, then takes the newest value per metric within the hour before it. Resources that already have rows are skipped, so later boots only walk the resource names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
98854d94f9 |
feat(platform): tini as PID 1, Hypercorn, a log-level setting, the image on CI's Python, an airgapped boot gate before push (#5159)
- Dockerfile: python:3.14-slim, the Python CI tests on (was 3.13). tini from apt is PID 1 (ENTRYPOINT tini -- entrypoint.sh): it reaps the zombies ssh and ansible leave and forwards SIGTERM. - cli: Hypercorn's serve() with one worker (run, stream and scheduler state is in-process), a 10 s graceful timeout, and SIGTERM/SIGINT as the shutdown trigger. Access and error lines go through Steward's logging. --debug keeps Quart's reloading dev server. - Logging: steward/core/logging_setup.py installs one root handler before anything logs (until now a library's basicConfig on import was the only configuration). Its level is the `log.level` setting (DEBUG/INFO/WARNING/ ERROR, default INFO), applied at boot and on save in Settings → General. - CI publish: build, then "The image boots with no internet" (rule 164, snippet #3552), then push, so a broken image never reaches a tag (rule 177). On an --internal network with its own Postgres, the image must migrate and answer /health, run tini as PID 1, serve first-run setup with its assets from /static and no third-party host, and exit 0 on SIGTERM. Logs and clean-up run always. Tests: tests/core/test_logging_setup.py; the smoke step itself. Not in this commit: the dependency lockfile (it conflicts with rule 154 and is waiting on the operator). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
edcfa4d3db |
fix(ansible): git sources work in the image and survive a redeploy; pulls time out; alert vars go in a JSON file (#5157)
- git ships in the image; checkouts live under /data/ansible/sources (STEWARD_ANSIBLE_SOURCE_DIR), on the persistent volume. - One task, ansible_git_pulls, reads the sources every 30 s and pulls each as its interval passes: a source added in Settings is pulled without a restart, a removed one is forgotten. - git_pull: a 120 s timeout ending the process group, GIT_TERMINAL_PROMPT=0, pull --ff-only, a failed clone removed; it raises GitPullError with git's message, so the Sync button stops saying Synced after a failure (and the message is now escaped). - An alert action's vars and the firing's steward_alert_* vars reach ansible-playbook as one JSON extra-vars file, not -e key=value, which Ansible split on whitespace (a rule named 'High CPU on web'). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
edea8482b5 |
fix(monitors): run each monitor on its own interval, from its scheduled start (#5151)
The dispatcher ticked once per MONITORS_POLL_INTERVAL and stamped last_checked_at when the whole batch finished. With a 60 s tick and a 60 s monitor the next tick saw 59.x s elapsed and skipped, so the monitor ran every 120 s; a 30 s monitor could never run faster than the tick. - The dispatcher ticks every 5 s (DISPATCH_TICK_SECONDS) and reads the monitors and intervals fresh each tick, so a changed interval applies at the next one. - Due is measured from the scheduled start, stamped before the check runs, with half a tick of slack so tick jitter never costs a whole tick. A long-overdue monitor runs once. - Each check runs in its own task and records its own result, so a slow probe holds up nobody; a monitor still being checked is not started again, and it is freed in a finally. - A check past CHECK_DEADLINE_SECONDS (120) is recorded down with "Check did not finish within 120 s" rather than holding its monitor forever. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
4fbe93bdf7 |
feat(alerts): a sentence per alert, a link to what it watches, a count in the sidebar (#5124)
The Alerts page's active list now says each alert in one line around the resource it watches, e.g. "cpu_pct on web-1 is 94% (> 90%), firing for 12m", with web-1 linking to its host, monitor or container page (or the plugin's page for Traefik, UniFi and SNMP). Values take their unit from the metric name through formatting.metric_value. A silent series reads "No cpu_pct from web-1 for 20m"; an up/down metric says the word. Actions follow the routes' roles: acknowledge, edit, toggle and New rule for operators and admins, Delete for admins only, nothing for viewers. The sidebar shows how many alerts need attention beside Alerts, from an async context processor that skips htmx fragments, logged-out pages and the database-down page, and renders without the count if the query fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
e595bd922a |
feat(monitors): problems first, a page per monitor, adding on its own page (#5121)
The Monitors list now opens with a count strip (down, up with how many degraded, pending, paused) and sorts down, degraded, pending, up, paused, by name within each. Degraded is up but worth a look: uptime at or below its warning floor, or the latest response past its critical cutoff, both from Settings → Thresholds; a slow-but-tolerable reply is coloured, not sorted. A monitor had no page of its own: every link went to the list. Now /monitors/<id> shows its state and since when, latency, uptime for 24h/7d/ 30d, TLS days, the last checks, recent failures with their errors, and its settings (HTTP header names, never their values). Status links to it, and adding or editing returns to it. Adding is its own page (/monitors/new, form.html shared with edit) behind an "Add monitor" button; a refused add re-renders the form with the error and what was typed instead of a bare 400 text. Row actions are labelled icon buttons, type names read Ping/TCP/DNS/HTTP (|monitor_type), and the heartbeat uses pill-up/pill-down, so HEX_CEILING drops to 56. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
477b05b9d0 |
feat(ui): status_badge, status_dot, stat_tile, empty_state, page_header, icon_button and form classes (#5094)
#5094 "Status and layout macros, and form classes". - formatting.STATES: one state -> (tone, label) map for up, down, stale, firing, pending, acknowledged, no_data, ok, warn, crit, resolved and inactive; status_info() reads it (an unmapped state is drawn dim under its own name). required() lets a macro refuse a missing argument. - _macros.html: status_badge and status_dot (a dot always carries its label, sr-only unless shown), stat_tile, empty_state, page_header (actions via a call block) and icon_button, which raises without a label. - steward.css: tone classes (badge-/dot-/stat- ok, warn, crit, info, dim), .sr-only, .page-header, .stat-tile, .empty-state, .btn-icon and the form classes .field, .field-label, .field-hint and .input. Info text is a 40% mix (--blue): the house --fs-info-fg falls to 4.18:1 on its tint over a hovered row, 40% holds 5.09:1. - First users: the Alerts list, history and maintenance pages (headers, state badges, empty states) and the alert forms' hints; the inline inp/lbl style sets on the monitor and host forms become .input / .field-label. - tests/core/test_macros.py renders each macro through the app environment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
bf08169176 |
feat(ui): shared |bytes |rate |duration |ago filters and status_class from Settings → Thresholds (#5093)
#5093 "Shared format filters and status_class driven by Settings → Thresholds". - steward/core/formatting.py: human_bytes (binary units), human_rate, human_duration (two largest units), ago (a <time> titled with the absolute UTC time) and status_class(value, kind, thresholds) -> ok/warn/crit/unknown, which raises on a kind the thresholds don't name. - app.py registers them as the bytes, rate, duration and ago filters and the status_class global; threshold_style is now graded by status_class (and caps its weight at 500). - Replaces the copies: docker's _human_bytes/_human_uptime, core alerts' _duration, the host-agent fmt_bps/fmt_bytes macros, the Traefik and UniFi byte-rate arithmetic, the swarm-failures ago macro and two uptime spans. - Tests: a table per helper, and every template compiled against the app's filters, so an unknown filter fails CI rather than a page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
49607a2441 |
feat(alerts): dispatch after commit; maintenance mutes, not skips; actions via trigger_run (#5063)
Notifications and alert actions now start only once the evaluating
transaction commits, and never for one that rolls back. record_metrics
queues them on the session (_after_commit), and after_commit/after_rollback
listeners on the sync session spawn or drop them. Spawned tasks are held in a
module set so they cannot be garbage-collected before they run.
A maintenance window no longer skips evaluation. Alerts in its scope still
change state, but their notifications and actions are held, and each event
says so ("notification muted: maintenance window <name>"; also on no-data
and resume events). notify_after_maintenance, run every minute with the
silence check as the new core task alert_reconcile, sends one notification
for an alert still firing after its window ends and runs its action. The
event's note records the release.
Alert actions go through runner.trigger_run like every other run path, so
the playbook and the inventory must be ones the source lists (#5034 guards).
The firing event links to the run, or its note says why the action did not
start. The maintenance page and form wording now say what a window does.
Tests:
- integration: a rolled-back evaluation notifies nobody, and dispatch waits
for the commit; maintenance mutes while the state moves and notifies once
when the window ends; an action whose playbook is outside its source is
refused, with the note set;
- the alert-action test uses a listed inventory name and waits, with a
deadline, for the run the action starts;
- unit: _muted_by.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
bvandeusenandClaude Opus 5.5
|
c368e93e2c |
feat(alerts): silence is alertable — agent offline and no-data (#5062)
Agent offline is an ordinary rule now. The host_agent scheduler records host_agent/<host>/seconds_since_seen every minute (from last_seen_at, or registration time for an agent that never reported), replacing the task that only logged stale agents. The catalog offers the metric, and picking it on a new rule fills "> 180". No data: each rule's state tracks its last sample and its usual gap, smoothed so that an outage counts as at most three gaps. A core task, alert_silence_check, runs every 60s and marks a rule no_data_since once its series has had no sample for three usual gaps (at least 5 minutes; an hour while the cadence is unknown). A never-sampled rule counts from when it started watching. The mark sits beside the state, so a firing alert that goes silent stays firing. Each silence records one event and sends one notification (rule option notify_no_data, default on; muted in maintenance windows). The next sample clears it with a "data resumed" event and notification. Creating, changing, disabling or re-enabling a rule restarts its watch. Migration 0029 adds the columns and seeds each existing rule's last sample and gap from its last day of data, so an upgrade does not flag every rule. Shown on the Alerts page (a NO DATA badge with its start time), in MCP list_alerts, list_alert_rules and get_host (state "no_data", no_data_since), in get_overview findings (alert_no_data, or a firing alert marked silent) and in the weekly report. The rule form has the notify option, and its metric reference now matches what traefik records and lists host_agent. Emails say "no data" and carry the event's note. The metric-catalog guard also reads positional MetricBatch.add arguments. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
7a3f7869fe |
feat(auth): sessions end when they should; logins are throttled; Secure behind HTTPS (#5035)
#5035 "Sessions: lifetime applied, role and is_active re-read per request, logout is a POST, failed logins throttled". - steward/core/proxy.py: trusted_value / is_https / client_ip read X-Forwarded-Proto and -For only as far as the trusted proxy hops (Settings → General, default 1, 0 = trust none). - The session cookie is Secure, and HSTS is sent, only on requests that arrived over HTTPS by that rule; plain-HTTP and LAN use are unchanged and nothing redirects. - A login lasts Settings → General's session lifetime from when it began (session.permanent + a login_at stamp checked every request — the cookie's own age slides on every response). A deleted or deactivated user is logged out on their next request; a role change applies at once. - Logout is a POST (a form in the sidebar); GET /logout is gone. - Failed logins pause an address after login.max_failures within login.lockout_minutes (Settings → General; defaults 5 in 15 min), keyed by address so nobody can lock a user out by name. Failures are audited as auth.login_failed with the address. - to_access_cfg projects the new settings into app.config at startup and on save, beside the existing projections. - Integration tests seed real users and a login_at, since every request now re-reads the session's user. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
6a1bc437c9 |
feat(auth): share links open only their own widgets; unmarked routes are closed (#5033)
#5033 "Share links open only their own dashboard; every route without a role is on an explicit allow-list". Any live share token in ?s= made the caller a viewer on every viewer route — host pages, container logs, any dashboard — and the host-agent widget routes had no role check at all. - steward/dashboard/share.py: a share token opens, read-only, exactly the widget fragment URLs its own dashboard loads — same path, same configured parameters; only the time window (hours/range) may change. - require_role marks its view; public_route marks the deliberately open ones (login, setup, health, the share page, agent ingest, install.sh, agent.py) with the reason. A before_request refuses any view with neither, so a route that forgets its decorator is closed, not open. - /plugins/host_agent/widget and /widget/history now require viewer. - The history widget's range buttons keep the rest of the query (and so a share link's token) instead of rebuilding it; the uptime widget's "Full SLA" link shows to logged-in users, not to share viewers. - tests/test_route_access.py: a source scan that fails on any unmarked route in core or bundled plugins, the default-deny refusal, and the share matcher's cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
96126b1c83 |
feat(auth): forged browser requests are refused (#5031)
#5031 "Forged-request protection: SameSite cookie, per-session CSRF token on every non-GET, HTMX header". A page on another site could make a logged-in browser POST to Steward with the session cookie attached. - steward/auth/csrf.py: a per-session token, checked in one before_request for every POST/PUT/PATCH/DELETE (header X-CSRF-Token or form field csrf_token, constant-time compare). The bearer-authenticated host-agent ingest is exempt; /mcp never reaches Quart. - Every POST form renders {{ csrf_field() }}; base.html carries the token in a meta tag and adds it to every HTMX request; the dashboard's fetch() calls send it. - The session cookie is SameSite=Lax; login issues a fresh token. - PUBLIC_BASE_URL is now loaded from Settings at startup (it was only set when Settings → General was saved). - tests/test_csrf.py: refusal, acceptance, rotation, the ingest exemption, and a guard that every POST form in the templates carries the field. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
3984f639e8 |
feat(mcp): /mcp endpoint with bearer auth and role-gated tools
Second step of milestone 461. An MCP client (Claude Code, Claude Desktop via mcp-remote, Open WebUI) connects to /mcp with a personal API token. - mcp>=2.2 (SDK v2 MCPServer). Mounted by wrapping app.asgi_app, so /mcp never enters Quart's pipeline; stateless streamable HTTP so a redeploy never strands a client on a session id the new process didn't issue. - Auth: Authorization: Bearer stw_… resolves to a Caller (user + effective role) in a ContextVar; 401 with WWW-Authenticate otherwise. The web pipeline's database gate runs first, so a DB restart answers 503 instead of failing inside a tool. - Registry, default-deny: register_tool(fn, role=, read_only=) is the only door; role and read_only are required. Tools above the caller's role, or from a disabled plugin, are neither listed nor callable. Undeclared arguments are refused (not silently dropped); a tool's ValueError reaches the agent as its reason; a DB failure mid-tool says so. - Plugins contribute tools via an optional register_mcp_tools(register) export, all-or-nothing, replaced on hot-reload. Documented in docs/plugins/writing-a-plugin.md. - First tool: whoami. - Tests drive app.asgi_app directly (Quart's test client never reaches /mcp); the integration test mints a real token against Postgres and runs initialize → tools/list → whoami → revoke → 401. Task #4966 "MCP-2 — /mcp transport". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5.5
|
1c60fb6b4d |
feat(api): personal API tokens and the AI access page
First step of milestone 461 (MCP server). A token is the credential an MCP client presents as `Authorization: Bearer stw_…` to act as its user. - api_tokens table (0026): sha256 hash only, display prefix, role ceiling, expiry, revocation, throttled last_used_at. - steward/core/api_tokens.py: mint, list, revoke, resolve. Resolution re-reads the user's current role and acts with the lower of that and the token's ceiling, so demoting or disabling a user weakens or kills their tokens with nothing to revoke by hand. - /ai-access/ (sidebar → Automation → AI access): every user manages their own tokens, can't grant above their own role, sees the token once. Session-only: a share-link viewer is refused. Create and revoke are audited. - Integration tests drive the routes against real Postgres so their transactions run for real (#884 escaped CI behind a mocked session). Task #4965 "MCP-1 — Personal API tokens". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5
|
8c50ff242c |
fix(db): ride out a brief database outage mid-request; explain it if it persists
The operator's 500 was NOT the startup gap I attributed it to. They found it by logging in and getting Steward's own error page -- which means create_app had completed and the app was serving, so the gaierror came from a request handler acquiring a connection, not from boot. Neither prior fix covers that: the startup retry never runs, and pool_pre_ping only helps once the database is back, since its replacement connect fails too while the container is gone. Adds a before_request gate that acquires a pooled connection with a short bounded retry (~1.75s over 4 tries) so a database restart is ridden out invisibly, and renders a distinct 503 "database unavailable" page when the budget is exhausted. The request budget is deliberately far shorter than the startup one: nobody watches a container boot, but somebody is watching this page load, and a page that hangs is worse than one that says what is wrong. /health and static are exempt -- a liveness probe must stay answerable while the database is down, or a dependency outage triggers a restart loop. Credential and missing-database errors still propagate rather than being reported as "unavailable", which would send the operator chasing the wrong problem entirely. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5
|
6c9b89390a |
fix(db): survive a database restart and a not-yet-ready database
Two connection-recovery gaps, both surfaced by a Postgres restart that left the app throwing tracebacks while the DB itself was healthy. pool_pre_ping + pool_recycle on the app engine [#2626]: when the database restarts, every connection already in the pool is dead at the socket level. SQLAlchemy only discovered that by failing a real query, so the first operation after a restart errored out on whatever triggered it. pre_ping checks liveness on checkout and swaps the dead connection transparently; pool_recycle caps connection age so a socket stranded by a NAT/conntrack timeout or a Docker network rebuild is retired on a timer instead. wait_for_database() gate at startup [#2627]: create_app touches the DB synchronously (migrations, secret re-encryption, settings load) and assumed it was both resolvable and accepting connections on the first try. Neither holds after a host reboot (Docker DNS not yet serving `db` -> gaierror -2) or an unclean shutdown (Postgres still replaying WAL -> "not yet accepting connections"). Both are transient, so retry with capped backoff behind one gate ahead of the first DB touch. Credential and missing-database errors are classified by SQLSTATE and still fail immediately -- waiting cannot fix those. Budget is bootstrap-configurable (STEWARD_DB_CONNECT_TIMEOUT / database.connect_timeout, default 60s) since it governs reaching the DB and so cannot live in the DB-backed settings. Tests drive the retry loop off a fake clock, so backoff and timeout behaviour are deterministic rather than wall-clock dependent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
f40063a74d |
feat(reliability): fd-leak rails — self-watchdog, poll-overlap guard, fd tests
Guardrails so the fd-leak class of bug (Errno 24 lockup; recent SNMP + UniFi fixes) surfaces early or can't compound, instead of silently killing the app. - Self-fd watchdog (steward/core/self_monitor.py): records open_fds and open_fds_pct (% of soft RLIMIT_NOFILE) as "steward"/"process" metrics each minute through the normal alert pipeline, so the operator can alert on them via the existing alert-rules UI. Built-in WARNING floor at 80% gives a zero-config early signal. Stdlib-only (/proc + resource); degrades to a no-op off Linux. Registered as a core ScheduledTask in app.py. - Poll-overlap guard (steward/core/scheduler.py): extract a pure _DueTracker that skips a tick while a task's prior run is still in flight, so a hung poll can't stack overlapping runs (which amplify per-poll resource/fd use). A skipped task isn't penalised — it retries the next tick after it completes. - fd-stability tests (tests/core/): _DueTracker overlap policy, the watchdog metric/warning/degradation paths, and a real-fd canary that hammers tcp_check and asserts /proc/self/fd doesn't grow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
c8b6719b37 |
feat(ui): plugin get_nav() hook → sidebar Infrastructure links (slice 2)
Plugin UIs had no nav home (only dashboard widgets / typed URLs). Add an optional get_nav() plugin export and surface it in the sidebar. - plugin_manager: _PLUGIN_NAV registry + get_plugin_nav() getter + a tolerant _collect_plugin_nav() (missing hook = fine; raising/malformed = logged & skipped; idempotent per plugin so hot-reload re-runs cleanly). Collected in both the startup load path and the hot-reload path. - app.py: inject plugin_nav into the template context, filtered to enabled plugins so a hot-disabled plugin's link can't linger before restart. - base.html: render plugin links under the Infrastructure group, with the same request.path active-state treatment as core links. - docker/snmp/unifi/traefik: each exports get_nav() → its /plugins/<name>/ view. - Tests: collector behavior (collect, missing hook, reload-replace, malformed item, raising hook, sorted output). With docker enabled, "Docker" now appears under Infrastructure → its fleet view. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC |
||
|
bvandeusenandClaude Opus 4.8
|
0940dc6972 |
feat(crypto): fail loudly on unpersistable secret key; flag undecryptable secrets
Follow-up to the incident where an unwritable /data made Steward silently mint a fresh ephemeral secret key on every boot, orphaning all encrypted secrets — only discovered when an Ansible run failed. Make both failure modes loud and visible. - config._resolve_secret_key: if a new key must be generated but can't be persisted (no STEWARD_SECRET_KEY, unwritable /data), raise RuntimeError and refuse to start, with an actionable fix (set STEWARD_SECRET_KEY, or make /data writable by uid 1000). Also raises a clear error if an existing key file can't be read. First-run on a writable volume still generates + persists normally. - core.settings: detect secrets stored as ciphertext that won't decrypt with the current key (scan_undecryptable_secrets at startup; _is_undecryptable helper). Cached in memory; set_setting discards a key on a fresh write so the banner clears without a restart. - app.py: run the scan after migrate_plaintext_secrets, log a warning, and inject undecryptable_secrets into the template context. - base.html: admin banner naming which secrets to re-enter, each linked to its settings tab. - compose.deploy.yml: document the uid-1000 /data write requirement and the STEWARD_SECRET_KEY option for multi-node Swarm. - Tests: secret-key resolver (reuse existing file, generate when writable, raise when unpersistable) and the undecryptable-secret detection helper. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC |
||
|
bvandeusenandClaude Opus 4.8
|
7b80552a7d |
feat(docker): per-host collection via the host agent; drop central scrape
Docker collection moves off the central single-socket scrape onto the host
agent, giving Docker a real per-host dimension. The Steward host now reports
its own containers like any other host, and same-named containers on different
hosts no longer collide.
- agent: stdlib UDS Docker client (AF_UNIX HTTP/1.1, Connection: close,
chunked-aware), collect_docker() ports the cpu%/mem math; sample["docker"]
added best-effort (silent-skip on absent/unreadable socket). AGENT_VERSION
1.2.0 → 1.3.0; optional docker_socket config key.
- ingest: host_agent ingest hands per-host container snapshots to the docker
plugin via a new "docker.persist_host_samples" capability (no hard import,
no-op when docker disabled), inside a SAVEPOINT so a docker failure never
sinks the host metrics. Resource names are host-scoped ("<host>/<name>").
- schema: docker_containers re-keyed (host_id, name); docker_metrics gains
host_id; docker_002 migration DROP+recreates (dev-only, rule 122).
- ui: Docker page + widgets grouped by host with host links; new per-host
Docker panel embedded on the Hosts hub (gated on docker enabled via a new
enabled_plugins template context). Replaces the SQLite-only strftime
bucketing with DB-agnostic Python bucketing.
- provisioning: install/provision playbooks add steward-agent to the docker
group (best-effort) so the agent can read the socket.
- removed central scrape: docker scheduler.py + scraper.py deleted; plugin.yaml
socket_path/scrape_interval_seconds/include_stopped dropped (plugin 2.0.0).
- tests: agent docker collector units (math, chunked decode, silent-skip,
sample shape, config) + integration (host-scoped schema + persistence).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
|
||
|
bvandeusenandClaude Opus 4.8
|
35f658b573 |
feat(monitors): unify ping/dns/http into one Monitor entity + custom targets
Collapse the three former check types into a single core `Monitor` entity
with one management surface (/monitors), one result table (monitor_results),
and a single scheduled task. Every type can now watch a free-standing custom
destination (optional host_id) — not just a registered Host.
- models: Monitor + MonitorResult replace PingResult/DnsResult; Host loses its
ping/dns facet columns (now Monitor rows linked by host_id).
- checks: monitors/{ping,dns,http}.py pure probes + runner.run_monitor
dispatcher; one monitor_check scheduler with a per-monitor due-filter.
- status: single monitor_status_source replaces the three sources.
- UI: /monitors blueprint (type-aware add/edit/list/widget); host hub shows a
host's linked monitors + "add monitor for this host"; nav + widget registry
+ alert metric catalog rewired. http plugin folded into core and removed.
- migration 0022 merges the http branch, data-migrates host facets +
http_monitors + all three result histories, drops the old tables/columns.
Resolves the per-host ping/dns auto-attach issue (#275): monitors are now
explicit, never auto-added to every host.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
|
||
|
bvandeusenandClaude Opus 4.8
|
591706bd39 |
feat(settings): configurable monitoring thresholds
Move the hardcoded warn/crit cutoffs into Settings -> Thresholds (DB-backed, live, no restart). New thresholds.* keys + to_thresholds_cfg() + a threshold_style(value, kind) jinja global that reads them; latency reuses the existing ping good/warn keys, uptime is direction-aware (floors). Replace the _macros metric_style/uptime_style macros (now removed) with the global across Hosts-Overview, host_agent fleet + panel, Uptime/SLA widget, and the ping page uptime column — all now honor the configured cutoffs. Uptime keeps its green 'good' look when not degraded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
6e91bdc82b |
feat(security): encrypt sensitive settings at rest (Fernet)
Secrets (smtp.password, oidc.client_secret, ldap.bind_password, ansible ssh_private_key/become_password/vault_password) were stored plaintext in app_settings. Add transparent encryption-at-rest: - steward/core/crypto.py: Fernet keyed off the app secret (/data/secret.key), enc:v1: prefix marks ciphertext; passthrough for plaintext/empty/no-key, never reveals plaintext on a wrong key. - settings.py: SECRET_KEYS registry; set_setting encrypts on write; all read paths (get_setting / get_all_settings / load_settings_sync) decrypt transparently; migrate_plaintext_secrets() converts legacy rows in place. - app.py startup: init_crypto(SECRET_KEY) + one-time legacy-secret migration before settings load. - Add cryptography dependency. UI masking is unchanged (it checks decrypted truthiness). Key-loss caveat documented: secrets are unrecoverable if the app secret key is lost. Unit tests cover round-trip, empty/plaintext passthrough, and wrong-key safety. Task #580. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
88857be24e |
feat(ansible): runner robustness — cancel, concurrency, structured results, retention
Closes #550 (all four): - Cancellation: track live subprocesses; POST /ansible/runs/<id>/cancel (operator) SIGTERMs then SIGKILLs after a grace; new 'cancelled' status (+ migration 0019, ALTER TYPE in autocommit). Queued runs cancel cleanly before launch. Cancel button on run detail. - Concurrency: global semaphore (ansible.max_concurrent_runs, default 3, Settings→Ansible) caps simultaneous runs; excess show 'queued' (new status) until a slot frees. Semaphore bound lazily per running loop. - Structured results: parse PLAY RECAP into per-host ok/changed/unreachable/ failed/skipped + capture failed-task lines, stored in new results JSON column (migration 0020); rendered as a host-summary table on run detail. Keeps live streaming (no json-callback swap). - Retention: full output written to a persistent log artifact (/data/ansible/runs/<id>.log, env-overridable) beyond the 1 MB DB cap and across restarts; in-memory replay buffer bounded + GC'd after completion; Download-log route. Boot reconciliation now also sweeps stale 'queued'. Unit tests for recap parsing + cancel flagging. Status colors updated across run list / detail / schedules. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
f3e919892d |
feat(plugins): plugin capability registry + host_agent→Ansible deploy synergy
Implements #253's framework: a small core capability registry (steward/core/capabilities.py) where a module/plugin publishes a named, role-gated action and a consumer discovers it via has_capability() and runs it via invoke_capability() — no hard import, graceful degradation, permission propagation (actor role checked against the capability's required_role). Core publishes "ansible.run_playbook" (operator) wrapping ansible.runner. trigger_run (extended to accept a caller-built inventory). First consumer: the host_agent plugin gains "Deploy via Ansible" on its settings page — pick an inventory target/group and it installs/updates the agent via the bundled host_agent/install.yml, minting a fresh token per host and injecting it as an inventory hostvar (turning per-host curl|sh into one run). Exposed role ordering as middleware.role_meets. Unit tests for the registry + role checks. Task #253 (milestone #37). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
4a0a3ee46e |
feat(ansible): scheduled recurring playbook runs
Adds cron-like recurring runs (the engine the maintenance-automation work needs). New AnsibleSchedule model + migration 0018; a core ScheduledTask (ansible_scheduled_runs, 60s) fires due schedules, each creating a system-triggered AnsibleRun (triggered_by=None). Centralises the resolve-inventory → create-run → launch flow in ansible/runner.trigger_run, shared by the manual route (refactored to use it) and the scheduler. Schedules UI under /ansible/schedules: create/edit/pause/delete/run-now, with interval presets, scope targeting (all / group / target), extra-vars / limit / tags / dry-run, and last-run status (resolved via last_run_id) + next-run. Unit test for the due-check. Task #549 (milestone #37). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
3b6e005ed8 |
feat(status): unified Status page + widget across ping/DNS/HTTP monitors
Adds a Kuma-style "is everything up?" surface that aggregates heterogeneous monitor types via a status-source registry (steward/core/status.py): each type registers an async source(db) -> [StatusEntry]. Core registers ping/DNS; the http plugin registers its own from setup() so core never imports plugin tables. Per entry: current up/down, last-30 heartbeat bar, uptime % (24h/7d/30d), latest latency + response sparkline, and TLS expiry countdown (HTTP). New /status page (live htmx refresh) + a status_overview dashboard widget + nav link. Pure-function unit tests for registry + sparkline. First deliverable of milestone #68 (task #866). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
935cbc105c | feat(ansible): inventory groups CRUD routes + templates + blueprint registration | ||
|
bvandeusenandClaude Opus 4.8
|
a7a281cb11 |
feat(plugins): fold first-party plugins in-tree; bundled + external roots
First-party plugins (host_agent, http, snmp, traefik, unifi, docker) are now tracked under plugins/ and baked into the image, so they version atomically with core — ending the cross-repo import drift the roundtable->steward rename exposed. History for these files is preserved in the archived Roundtable-plugins repo. Plugin discovery becomes multi-root: PLUGIN_DIR (single) -> PLUGIN_DIRS (bundled first, then external) + PLUGIN_INSTALL_DIR. Bundled ships in the image; third-party plugins still mount at runtime into the external root (STEWARD_PLUGIN_DIR, default /data/plugins) and downloads/installs land there. Bundled shadows external on a name collision. - config.py: load_bootstrap returns plugin_dirs + plugin_install_dir - app.py: iterate PLUGIN_DIRS at the migration + load sites - migration_runner.py: discover_all_in() unions every plugin root - plugin_manager.py: resolve_plugin_path() (pure, first-root-wins); load / install / hot-reload span all roots; installs target the external root - settings/routes.py: _discover_plugins scans all roots, dedup bundled-first - Dockerfile: COPY plugins/ ; docker-compose: drop host bind, document external - tests/test_plugin_dirs.py: resolution, multi-root discovery, bootstrap split Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.7
|
88ab5b917e |
chore: rename project Roundtable → Steward
Renames the Python package directory, CLI command, env var prefix, docker-compose service/container/image, Postgres role/db, and all visible branding. Marketing form is "Fabled Steward". Clean break from the previous rebrand: drops the fabledscryer→roundtable import shim in __init__.py and the FABLEDSCRYER_* env var fallback in config.py and migrations/env.py. Env vars are now STEWARD_* only. Heads-up for existing deployments: - Postgres user/db renamed fabledscryer → steward in docker-compose.yml. Existing volumes need the role/db renamed inside Postgres, or override POSTGRES_USER/POSTGRES_DB to keep the old names. - Host-agent systemd unit is now steward-agent.service. Existing agents keep running under the old name; reinstall to switch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |