The schedule and host run forms opened their Source/Playbook <select>s on a
"— choose … —" placeholder, forcing an extra click. Default them to the first
item and cascade the dependent fields so the initial state is real:
- Remove the placeholder options (schedules source+playbook, host source+
playbook, and the shared _playbook_options fragment).
- schedules(): pass sel_source (first source, or the edited schedule's) and
that source's pre-rendered playbooks so the playbook dropdown starts populated
(create mode too, no longer the all-sources union).
- Cascade on load/change: the source <select> fires playbook-options then
(hx-on::after-settle) re-triggers the playbook <select>, which loads that
playbook's variable fields. Host form drives the whole chain from a load
trigger; schedule create mode loads vars via the playbook <select>'s own load
trigger, while edit mode keeps its server-rendered prefill untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Ansible Schedules form only offered a free-form extra-vars textarea,
unlike the browse/host run forms which auto-populate a field per declared
playbook variable (vars:/vars_prompt:) plus the playbook description. Wire
the schedule form to the same shared `_playbook_vars.html` partial:
- Playbook <select> now loads /ansible/playbook-vars?schedule=1 on change
into a #sch-playbook-vars container (mirrors the host detail form).
- Edit mode pre-renders the saved playbook's variables server-side with
their stored values; the extra-vars textarea keeps only leftover
(non-declared) vars.
- Partial gains optional `values` (prefill) and `schedule` flags. In the
schedule context secret vars render disabled ("secrets can't be
scheduled") since they're dropped downstream, and the destructive-confirm
gate is shown but not required.
Backend already parsed var__* fields for schedules, so this is the
front-end/partial parity fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Guardrails so the fd-leak class of bug (Errno 24 lockup; recent SNMP + UniFi
fixes) surfaces early or can't compound, instead of silently killing the app.
- Self-fd watchdog (steward/core/self_monitor.py): records open_fds and
open_fds_pct (% of soft RLIMIT_NOFILE) as "steward"/"process" metrics each
minute through the normal alert pipeline, so the operator can alert on them
via the existing alert-rules UI. Built-in WARNING floor at 80% gives a
zero-config early signal. Stdlib-only (/proc + resource); degrades to a no-op
off Linux. Registered as a core ScheduledTask in app.py.
- Poll-overlap guard (steward/core/scheduler.py): extract a pure _DueTracker
that skips a tick while a task's prior run is still in flight, so a hung poll
can't stack overlapping runs (which amplify per-poll resource/fd use). A
skipped task isn't penalised — it retries the next tick after it completes.
- fd-stability tests (tests/core/): _DueTracker overlap policy, the watchdog
metric/warning/degradation paths, and a real-fd canary that hammers tcp_check
and asserts /proc/self/fd doesn't grow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SNMP devices are config-defined by IP/hostname, not Steward Host records, so
they had no presence on a host's page. Map them by address and embed a fragment
(mirrors the Docker per-host fragment).
- _devices_for_host(devices_cfg, address, name): case-insensitive match of a
device's configured host to the Steward host's address or name (tolerates
non-dict / host-less entries).
- Route /plugins/snmp/host/<id>: renders the matched device(s) + latest readings,
or nothing when none map (so hosts without SNMP carry no empty card).
- snmp/host_panel.html: per-device card (name · address · reachability) with a
readings grid (K/M scaling) and a History link to the full device page.
- hosts/detail.html embeds it after the Docker fragment, gated on snmp enabled.
- Unit test for the matching helper (by address, by name, no-match, blank).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
plugin_metrics grows by (sources × resources × ~30s cadence); keeping 90d of raw
is a large table. Add a raw→hourly rollup (mirroring the Docker plugin) so only a
short raw window is kept at full resolution, with hourly averages archived longer.
- PluginMetricHourly model + core migration 0024 (plugin_metrics_hourly: avg/max/
count per source/resource/metric/hour, unique bucket constraint + lookup index).
- steward/core/metrics_retention.rollup_plugin_metrics: date_trunc('hour') agg of
raw older than the hour-aligned raw window, idempotent pg upsert into hourly,
delete the rolled raw, prune hourly beyond the rollup window.
- cleanup.py: plugin_metrics is no longer blanket-deleted at data.retention_days;
_run_metrics_retention drives the rollup with windows read live from settings.
- Settings: metrics.retention.raw_days (7) + rollup_days (90), tunable on the
Thresholds & Retention page (new "Host metrics retention" card).
- Chart read: _history_for_host merges the hourly rollup (older part of the range)
with raw date_bin (recent part, capped ≤1h), so 30d charts keep working —
recent at full resolution, older at hourly. Route passes raw_days from settings.
- Tests: unit (cutoff helpers) + integration (rollup aggregates/prunes; history
merges hourly + raw) against Postgres.
Speed was already handled by the indexes + SQL aggregation; this is the storage
lever (raw window ~10x smaller).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
Realizes the chosen host-summary layout: a thin live vitals bar at the very top,
separate from the agent management panel (no more duplicated CPU/MEM/DISK/LOAD).
- New fragment _host_vitals.html + route /plugins/host_agent/vitals/<id>: compact
CPU / Memory / Disk(/) / Load-per-core (threshold-coloured + sparkline) +
Pressure + live/stale·version·last-seen. Polled every 15s; renders nothing
until the agent reports.
- The metric computation (latest snapshot + 6h sparkline query + load/core + PSI)
moves from host_panel into host_vitals. host_panel slims to management only
(reg/target/reporting/stale/ansible) and no longer queries metrics; panel.html
drops the gauge row + pressure block, keeping status + lifecycle actions.
- hosts/detail.html: vitals strip on top (full width, live), then a 2-col
[Monitors | Agent] grid, Ansible full width below, docker fragment last.
UI only. Templates parse; plugin-template parse test covers the new fragment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
- Only the nav list scrolls when long (min-height:0 + overflow on .side-nav);
the brand and user/logout stay pinned top/bottom.
- Keyboard focus rings (:focus-visible) on every interactive sidebar element
(brand, nav links, user/logout) and the mobile ☰ toggle.
- Mobile toggle is now a real toggle: aria-controls/aria-expanded kept in sync,
Escape closes the off-canvas sidebar, scrim close stays in sync too.
- prefers-reduced-motion disables the sidebar slide + link transitions.
- Confirmed no dead top-nav CSS/markup remnants from the old top bar.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
The host detail page stacked three full-width cards (Monitors, Agent, Ansible)
left-aligned, using only ~40% of the width with lots of vertical whitespace.
Rework toward the operator-chosen "vitals on top + 2-column" layout:
- The agent panel (CPU/MEM/DISK/LOAD vitals + sparklines + lifecycle) moves to
the top, full width, so a host's vitals lead the page.
- Monitors and Ansible sit side by side in a responsive 2-column grid
(auto-fit, stacks under ~420px), filling the width.
- The Docker per-host fragment moves below, full width.
Note: the vitals live in the host_agent fragment (loaded across the plugin
boundary) and uptime in the host-detail context, so a single literal top "strip"
mixing both isn't clean — the agent card on top is the faithful, contained
realization. UI-only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
plugin_metrics had only a PK on id, so every host-detail, full-metrics, and
dashboard-widget load sequentially scanned the entire time-series table — which
grows by (sources × resources × sample cadence), so the host views got slower
over time (operator-reported "blocked/slow" loads). monitor_results was already
indexed, so the uptime aggregation wasn't the bottleneck.
Add two composite indexes matching the hot query shapes:
- (source_module, resource_name, recorded_at) — history range scans, fleet/
widget queries, and the 'host:%' sub-resource prefix.
- (source_module, resource_name, metric_name, recorded_at) — the latest-value-
per-metric group-by/self-join.
Model __table_args__ + core migration 0023. Integration lane validates the
migration via `alembic upgrade head`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
Plugin UIs had no nav home (only dashboard widgets / typed URLs). Add an
optional get_nav() plugin export and surface it in the sidebar.
- plugin_manager: _PLUGIN_NAV registry + get_plugin_nav() getter + a tolerant
_collect_plugin_nav() (missing hook = fine; raising/malformed = logged &
skipped; idempotent per plugin so hot-reload re-runs cleanly). Collected in
both the startup load path and the hot-reload path.
- app.py: inject plugin_nav into the template context, filtered to enabled
plugins so a hot-disabled plugin's link can't linger before restart.
- base.html: render plugin links under the Infrastructure group, with the same
request.path active-state treatment as core links.
- docker/snmp/unifi/traefik: each exports get_nav() → its /plugins/<name>/ view.
- Tests: collector behavior (collect, missing hook, reload-replace, malformed
item, raising hook, sorted output).
With docker enabled, "Docker" now appears under Infrastructure → its fleet view.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
The flat top bar was a set of ungrouped peers and gave plugin data no home.
Move to a persistent left sidebar with grouped sections, per the navigation
redesign (operator-chosen shell).
- base.html: top <nav> → left <aside class="sidebar"> + content column.
Groups: Overview (Dashboard, Status), Infrastructure (Hosts; plugin links
arrive in slice 2), Monitoring (Monitors, Alerts), Automation (Ansible),
Admin (Settings, Audit; admin-only). Brand on top, user/logout at the bottom.
- Active link highlighted by request.path prefix (aria-current).
- Responsive: sidebar slides off-canvas under 900px via a ☰ toggle + scrim
(inline class toggle, no new JS deps). Candle-glow preserved.
- Logged-out pages (login/setup) render without the sidebar (gated on
session.user_id), content area full-width and centered as before.
- Add tests/test_templates_parse.py: syntax-parses every steward template so a
broken tag fails the unit lane (only the login page renders there today).
Plugin nav links (Docker/SNMP/UniFi/Traefik) come next in slice 2 via a get_nav
hook. UI-only; no behavior/route changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
Follow-up to the incident where an unwritable /data made Steward silently mint a
fresh ephemeral secret key on every boot, orphaning all encrypted secrets — only
discovered when an Ansible run failed. Make both failure modes loud and visible.
- config._resolve_secret_key: if a new key must be generated but can't be
persisted (no STEWARD_SECRET_KEY, unwritable /data), raise RuntimeError and
refuse to start, with an actionable fix (set STEWARD_SECRET_KEY, or make /data
writable by uid 1000). Also raises a clear error if an existing key file can't
be read. First-run on a writable volume still generates + persists normally.
- core.settings: detect secrets stored as ciphertext that won't decrypt with the
current key (scan_undecryptable_secrets at startup; _is_undecryptable helper).
Cached in memory; set_setting discards a key on a fresh write so the banner
clears without a restart.
- app.py: run the scan after migrate_plaintext_secrets, log a warning, and inject
undecryptable_secrets into the template context.
- base.html: admin banner naming which secrets to re-enter, each linked to its
settings tab.
- compose.deploy.yml: document the uid-1000 /data write requirement and the
STEWARD_SECRET_KEY option for multi-node Swarm.
- Tests: secret-key resolver (reuse existing file, generate when writable, raise
when unpersistable) and the undecryptable-secret detection helper.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
The page-top chrome was inconsistent: most nested views used the breadcrumb
kicker + page-title pattern, but plugin sub-pages used ad-hoc "← back" links,
~11 pages stacked a breadcrumb AND a redundant ancestor back button, and two
(monitors/edit, hosts/uptime) had a back button but no breadcrumb. The
settings/plugin_detail page stacked all of it at once.
Unify on the breadcrumb-led model:
- settings/_tabs.html: drop the hardcoded "Settings" h1; the breadcrumb
("Settings › …") plus the tab strip is the header.
- settings/plugin_detail: drop the "← Plugins" back button.
- docker container_detail/swarm/disk + snmp/device: replace ad-hoc back links
with the standard crumbs() breadcrumb.
- host_agent, ansible/*, alerts/maintenance: remove redundant ancestor back
buttons (the breadcrumb's parent crumbs already link there); keep lateral
shortcuts (Inventory/Schedules/Browse/Targets/Groups/New).
- monitors/edit, hosts/uptime: add the missing breadcrumb, drop the back link.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
Bounds Docker time-series growth (the main scaling concern). New
docker_metrics_hourly table + docker_006 migration; a plugin retention module
(docker.run_retention capability) rolls raw docker_metrics older than the raw
window into hourly averages (idempotent upsert), deletes the rolled raw rows,
then prunes stale rollups + lifecycle events. Core cleanup.py drives it each
hourly run via the capability (no plugin-model import), reading the three
retention windows fresh from settings so changes apply without restart (rule 25).
Settings → "Thresholds & Retention" gains a Docker retention card (raw /
rolled-up / events windows, working defaults 7/90/30 days). Unit tests cover the
hour-aligned cutoff/bucketing helpers; integration test exercises the real
rollup-average + prune across both windows.
Milestone 77 task #941.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
Wires the agent's enriched + swarm payloads through the docker.persist_host_
samples capability:
* Swarm topology — persist sample["swarm"] into docker_swarm_services /
docker_swarm_nodes (upsert + prune stale, host-scoped so two managers don't
clobber). Migration docker_005 adds services.placement_json for the
task→node placement the agent now reports.
* Lifecycle events — _derive_events (pure, unit-tested) diffs the newest
snapshot against stored per-container state: start / stop / die (non-zero
exit) / oom / health_change → docker_events rows. Skipped on a host's first
snapshot so the baseline doesn't emit a start per existing container.
* Alerts — record restart_count (always) and is_healthy (1.0/0.0, only when a
HEALTHCHECK exists) alongside cpu/mem, under host-scoped resource names;
METRIC_CATALOG[docker] gains restart_count + is_healthy so they're alertable.
host_agent ingest captures the newest sample's swarm object and threads it to
the capability (now persist_host_docker(session, host, snapshots, swarm=None));
invoked when containers OR swarm are present, under the same SAVEPOINT. Unit
tests cover the event-diff matrix; integration tests cover event derivation
across two snapshots and swarm topology round-trip (incl. placement).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
Docker collection moves off the central single-socket scrape onto the host
agent, giving Docker a real per-host dimension. The Steward host now reports
its own containers like any other host, and same-named containers on different
hosts no longer collide.
- agent: stdlib UDS Docker client (AF_UNIX HTTP/1.1, Connection: close,
chunked-aware), collect_docker() ports the cpu%/mem math; sample["docker"]
added best-effort (silent-skip on absent/unreadable socket). AGENT_VERSION
1.2.0 → 1.3.0; optional docker_socket config key.
- ingest: host_agent ingest hands per-host container snapshots to the docker
plugin via a new "docker.persist_host_samples" capability (no hard import,
no-op when docker disabled), inside a SAVEPOINT so a docker failure never
sinks the host metrics. Resource names are host-scoped ("<host>/<name>").
- schema: docker_containers re-keyed (host_id, name); docker_metrics gains
host_id; docker_002 migration DROP+recreates (dev-only, rule 122).
- ui: Docker page + widgets grouped by host with host links; new per-host
Docker panel embedded on the Hosts hub (gated on docker enabled via a new
enabled_plugins template context). Replaces the SQLite-only strftime
bucketing with DB-agnostic Python bucketing.
- provisioning: install/provision playbooks add steward-agent to the docker
group (best-effort) so the agent can read the socket.
- removed central scrape: docker scheduler.py + scraper.py deleted; plugin.yaml
socket_path/scrape_interval_seconds/include_stopped dropped (plugin 2.0.0).
- tests: agent docker collector units (math, chunked decode, silent-skip,
sample shape, config) + integration (host-scoped schema + persistence).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
Collapse the three former check types into a single core `Monitor` entity
with one management surface (/monitors), one result table (monitor_results),
and a single scheduled task. Every type can now watch a free-standing custom
destination (optional host_id) — not just a registered Host.
- models: Monitor + MonitorResult replace PingResult/DnsResult; Host loses its
ping/dns facet columns (now Monitor rows linked by host_id).
- checks: monitors/{ping,dns,http}.py pure probes + runner.run_monitor
dispatcher; one monitor_check scheduler with a per-monitor due-filter.
- status: single monitor_status_source replaces the three sources.
- UI: /monitors blueprint (type-aware add/edit/list/widget); host hub shows a
host's linked monitors + "add monitor for this host"; nav + widget registry
+ alert metric catalog rewired. http plugin folded into core and removed.
- migration 0022 merges the http branch, data-migrates host facets +
http_monitors + all three result histories, drops the old tables/columns.
Resolves the per-host ping/dns auto-attach issue (#275): monitors are now
explicit, never auto-added to every host.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
Move the hardcoded warn/crit cutoffs into Settings -> Thresholds (DB-backed,
live, no restart). New thresholds.* keys + to_thresholds_cfg() + a
threshold_style(value, kind) jinja global that reads them; latency reuses the
existing ping good/warn keys, uptime is direction-aware (floors).
Replace the _macros metric_style/uptime_style macros (now removed) with the
global across Hosts-Overview, host_agent fleet + panel, Uptime/SLA widget, and
the ping page uptime column — all now honor the configured cutoffs. Uptime keeps
its green 'good' look when not degraded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an inverted uptime_style macro (low-is-bad mirror of metric_style) and
color the inline ping latency (warn 100ms / crit 250ms) and 24h uptime values
in the Hosts-Overview widget, which were the remaining uncolored metrics.
(Uptime/SLA + Ping widgets already colored degraded values.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Operator chose a single canonical summary. Repurpose the status_overview
widget into 'Overview' (host count + monitor up/down/pending + Alerts link;
key kept so existing dashboards upgrade in place) and remove the redundant
fixed top strip from the dashboard view. Drops _get_summary_stats and its
now-unused PingResult/DnsResult imports (rule 22).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Hosts-Overview: move the CPU sparkline inline next to the cpu % value it
represents (was floated far right on the name line, reading as unrelated).
- Host Agent history widget: caption the chart with the host it represents
(the panel title is generic) — links to the host hub.
- History widget: snap the y-axis to a stable 0..next-10%-band ceiling instead
of auto-scaling to the exact peak, so the 'zoom' no longer jumps each poll.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address graphical issues raised from the dashboard screenshot:
- No more truncated host names. Hosts-Overview and Host-Agent-Resources put the
host name on its own line (full, wraps if needed) with its data grouped beneath
it; ping/dns (.ping-name) and uptime widget names wrap instead of ellipsis.
Prefer vertical overflow over cramming/truncating a row.
- History graph fills the panel: drop the fixed 0–100 y-axis ceiling (beginAtZero
+ 8% grace) so the lines use the vertical space instead of hugging the bottom;
axis labels still show real %.
- Container-query breakpoints: the widget body is now a query container, so
fragments restyle to their OWN panel width. Host-list widgets flow into 2 cols
≥520px and 3 cols ≥900px (.host-blocks) — use the width, remove vertical
deadspace — and collapse to one column when narrow. Mirrored into the share view.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Operator ask: show the graphs next to their fields in the fleet widget, like the
host page's AGENT panel. Each row now renders cpu/mem/disk/load as a value with a
trend sparkline beneath it (1h window), instead of bare numbers.
- _fleet_rows fetches a per-host recent series (cpu/mem/load host-level, disk from
the root mount) in one 1h query and attaches a sparkline per metric to each row.
- widget_table.html lays out a metric cell (label + threshold-coloured value +
sparkline) per field, mirroring panel.html. Threshold colour is computed in the
loop and passed into the cell macro (keeps Jinja macro scope clean).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Milestone 72 phase D — the dashboard view IS the edit surface (operator ask):
- /d/<id> renders the grid via Gridstack in STATIC mode — positioned exactly like
before, live HTMX widget bodies keep polling. An "Edit" button flips the same
grid interactive (grid.setStatic(false)) in place, so you drag/resize over real
data. "Done" flips it back. Layout autosaves on change.
- The widget picker is now a bottom drawer (position:fixed overlay) revealed by
body.dash-editing — so the dashboard width is identical entering/leaving edit.
- Add: POST returns a single grid item; JS inserts it + grid.makeWidget +
htmx.process so it loads live data. Remove: POST 204 + grid.removeWidget.
Per-panel drag handle + remove ✕ are in the DOM for editors, shown only while
editing.
- Gridstack loads for everyone (viewers get the static grid); edit wiring is
gated on can_edit. Mobile collapses to one column.
- Removed the separate /d/<id>/edit route + edit.html + _edit_panels.html
(rule 22). Dashboard-list Edit and new-dashboard create now deep-link
/d/<id>?edit=1 which opens edit mode on load.
Browser-only behaviour — CI can't exercise it; needs an operator visual check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Milestone 72 phase C — bring the host-view graphs onto the dashboard:
- host_resource_history widget reworked into a real host-view chart: epoch-ms
linear axis (no Chart.js date adapter), themed like the host-detail charts,
maintainAspectRatio:false so it fills the resized panel, unique canvas per
widget instance (wid), and empty states ("pick a host" / "no metrics yet").
Was previously unusable — it had a broken time axis and no way to choose a host.
- Add a "host" param type: the edit form renders a live dropdown of hosts
(dashboard routes now pass the host list to the editor); the chosen host_id is
stored in config and fed to the widget.
- Hosts-overview widget gains a per-row CPU sparkline (last hour) via the shared
sparkline_svg helper — the host-view at-a-glance trend, on the main widget.
Charts/sparklines render only in the browser, so CI can't exercise them — needs
an operator visual check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Milestone 72 phase B — Grafana-style drag-resize grid for dashboard widgets:
- DashboardWidget: replace `position` with a 12-col grid placement
(grid_x/grid_y/grid_w/grid_h). Migration 0021 backfills the old position
order into a 3-up grid (4 cols x 4 cells each) and drops position.
- View + share render a static CSS grid from x/y/w/h: fixed cell height
(h * 70px) with the body scrolling, so the arranged layout is what's shown;
collapses to a single column under 820px.
- Edit view: Gridstack.js 12.6.0 (vanilla, CDN, pinned) — drag the title bar
to move, drag a corner/edge to resize; every change autosaves to a new
/d/<id>/edit/layout endpoint. Replaces the SortableJS position reorder.
- add_widget appends at the bottom-left; remove no longer renumbers.
_get_widgets now orders by grid position (drives DOM + mobile fallback order).
Note: Gridstack drag-resize is browser-only, so CI can't exercise it — needs an
operator visual check of the edit experience.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Milestone 72 phase A (clarity wins, no schema change):
- Add shared metric_style() macro in _macros.html: colours a numeric metric
amber (>=warn) / red (>=crit) on the value ITSELF, not just the status dot.
Defaults 80/90 for percentage gauges; load /core uses warn 80 / crit 100.
Applied to CPU/mem/disk across host_agent panel + fleet widget + the unified
hosts-overview widget.
- Link every host reference to the host hub (/hosts/<id>) in ping, dns,
hosts-overview, host_agent fleet, and uptime widgets; status widget entries
link to their own detail_url. All guarded on session.user_id so the public
share view degrades to plain text (the bare href carries no share token).
- Fix truncation: title= tooltips on all names that ellipsis, plus a hover
underline affordance on the now-clickable ping-name links.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New maintenance/system_update.yml: cross-distro OS package upgrade (apt +
dnf/yum) with two run-form flags:
- restart_services: restart every service that needs it after the upgrade
(Debian via needrestart -r a, installed if missing; RHEL via
needs-restarting -s → systemctl try-restart).
- reboot_if_required: reboot the host only when a reboot is actually pending
(Debian /var/run/reboot-required; RHEL needs-restarting -r). Never reboots
otherwise.
Tagged steward:category=maintenance and steward:confirm=true (significant /
reboot-capable), so it shows in the run UI with the confirm gate and both flags
as fill-in fields. No code changes — auto-discovered from the builtin source.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Host agent panel (reporting state) gains a "Re-provision" collapsible (admin +
linked target + managed key): bootstrap user/password → provision.yml, which
reinstalls the steward account + managed key + agent. This is the missing path
after regenerating the managed key — Update alone can't fix a broken key.
- Remove the Ansible sections (run-playbook + target link) from the host EDIT
page — they were the stale free-text version and the wrong place. They live on
the host detail hub (dropdown + discovered variables). Edit page now links to
the host page; edit_host route simplified (no ansible fetch).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a session has expired, require_role now bounces to /login?next=<path> and
sends the user back there after re-login.
- middleware: safe_next_url() (same-site relative path/query only; rejects
off-site, protocol-relative, javascript:, and the auth pages — no open
redirect). _login_redirect() builds the next param; for HTMX requests it uses
HX-Current-URL (the page, not the fragment) and HX-Redirect so the whole
browser navigates instead of swapping the login page into a fragment.
- login GET/POST carry `next` (hidden field), validated, used on success for
local + LDAP; OIDC stashes it in session across the IdP round-trip.
- login.html: hidden next field + next on the SSO link.
- tests for safe_next_url.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When ansible.ssh_private_key can't be decrypted (app secret key changed),
decrypt_secret returns the ciphertext unchanged; the executor was writing that
enc:v1: blob as the SSH key file → cryptic "Load key ...: error in libcrypto"
→ Permission denied. Now:
- build_credentials skips a still-encrypted key value (never writes ciphertext
as a key file).
- start_run broadcasts a plain-language run error: "The managed SSH key could
not be decrypted (the app secret key changed). Regenerate it in Settings →
Ansible and re-provision the host(s)."
The run still fails (no usable key), but the reason is now obvious instead of a
libcrypto error. Operator remedy: regenerate the managed key + re-provision.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Run UX: when start_run throws before/around launch (e.g. ENOSPC creating the
temp dir — the box is out of disk), the run was marked "failed" with empty
output. Now the exception is broadcast + written to the run output/results so
the run view shows e.g. "[run error] OSError: [Errno 28] No space left on
device" instead of a blank failure.
Widget audit follow-ups (no broken links were found; these are consistency):
- host_resource_history widget now charts root (/) disk, consistent with the
host panel (was the opaque "disk worst").
- host_resources widget: tooltip on the health dot explaining it warns on the
worst mount while the number shows root.
- status_overview widget detail_url /status → /status/ (avoid redirect).
- Normalize ad-hoc widget empty-states to the shared .empty style (wording,
which distinguishes "configured" vs "data yet", preserved).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bring the dashboard in line with the unified host IA + improved displays.
- New core "Hosts — Overview" widget (/hosts/overview/widget): one row per host
combining monitor status (ping dot + latency, uptime 24h) with the agent
glance (CPU / memory / disk root + stale flag), each row linking to the host
hub. Reads agent data from the generic PluginMetric table via a core-safe
_agent_overview_by_host helper (no host_agent import); freshness vs the
plugin's stale window. The granular Ping/DNS/Uptime/Agent widgets stay.
- Group the add-widget picker into Core monitors / Monitoring capabilities /
Integrations (a `group` field on every WIDGET_REGISTRY entry + section
headings in _edit_panels.html), matching the Settings → Plugins taxonomy.
- Fix the agent fleet widget rows to link to the host hub (/hosts/<id>) instead
of the old /plugins/host_agent/<id>/ page.
Scribe #903.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extend the playbook metadata convention with a namespaced `# steward:<key>:`
comment block:
- steward:category — free-text grouping label, shown as a badge in the browse
list and on the run form.
- steward:confirm — true/yes/1/on marks a playbook destructive; the run form
then requires a confirmation tick (required checkbox in the shared vars
fragment) before it can launch.
sources.discover_playbook_meta() parses description + category + confirm (first
match per key; `# description:` still primary, `# steward:description:` alias).
discover_playbook_description() now delegates to it. The browse list reads
per-playbook meta to show category badges + descriptions; the run-form and
playbook-vars fragments render the badge + confirm gate.
Bundled playbooks tagged: docker_prune → category maintenance + confirm true;
provision/install/update → category host-agent.
Docs: docs/reference/playbook-authoring.md updated (keys now implemented) and a
quick reference added next to the code at steward/ansible/PLAYBOOK_CONVENTIONS.md.
Tests added for category/confirm/alias parsing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Playbooks can ship a human description Steward reads and shows when one is
selected. Convention: a `# description: <text>` magic comment (Ansible rejects
unknown play keys, so a comment is the portable place — works for third-party
playbooks too); falls back to the first play's name:. sources
.discover_playbook_description().
Surfaced at the top of the shared _playbook_vars.html partial, which loads on
playbook selection in the host run form, schedules form, and browse run form.
All four bundled playbooks (provision/install/update/docker_prune) now carry a
description line. Unit tests added.
Scribe #900.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A fresh install had no prompt for general.public_base_url, so first-run agent
installs/share links silently used the request Host header. Add a "Steward URL"
field to the first-run /setup page (create-admin), pre-filled with the current
address, with help text. setup_post saves general.public_base_url and applies
it to app.config immediately (no restart). Editable later in Settings → General.
Scribe #896.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The assertion failed ("Pass steward_url/token/pubkey") because those were
injected as inventory HOST vars, but the playbooks declared them in the play
`vars:` block — and play vars OUTRANK inventory host vars, so the empty
defaults won and the injected values never reached the play.
- Pass globals (steward_url, steward_pubkey, steward_user, agent_interval) as
extra-vars via the JSON -e @file (highest precedence, space-safe). Keep only
the per-host steward_token as an inventory host var.
- provision.yml / install.yml: drop steward_token from `vars:` so the host var
isn't shadowed; assertions use `| default('')` for the un-defaulted token.
This was the next layer under the inventory-format fix — first the inventory
wouldn't parse, now the injected vars actually apply.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stop making operators type playbook paths and guess extra-var names.
- Reusable infra: shared ansible/_playbook_vars.html (the discovered-variable
fields) + ansible/_playbook_options.html; two HTMX endpoints —
/ansible/playbook-options (a source's playbooks, optional ?selected for edit)
and /ansible/playbook-vars (a playbook's vars:/vars_prompt: as fill-in
fields). browse _run_form.html refactored to include the shared partial.
- Host "Run a playbook against this host": source dropdown → playbook dropdown
→ variable fields, all chained via HTMX. Handler reuses _parse_run_params so
var__/secret__ fields flow through extra_vars_map + the unpersisted
secret_vars channel.
- Schedules: playbook free-text+datalist → source-dependent dropdown; fixed the
extra-vars edit pre-fill to read extra_vars_map (stale list key after the
earlier JSON-extra-vars change).
Scribe #895.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A failed provision looked successful in two ways:
1. The host panel showed the agent as deployed (metrics + Update/Rotate/Remove)
because provision/deploy mint the registration row BEFORE the playbook runs
and the panel keyed "installed" on that row. Now gated on the agent actually
checking in (reg.last_seen_at). Three states: reporting (metrics + lifecycle),
pending (token minted but no check-in → "no metrics yet, deploy may be
running/failed" banner + retry + Clear pending registration), and none
(install path).
2. The run reported success though nothing ran — ansible-playbook exits 0 on
"no hosts matched"/empty inventory. The executor now treats an empty PLAY
RECAP (returncode 0 but no hosts executed) as failed, with a clear failure
note. Non-zero exits and recap failed/unreachable were already caught.
Scribe issue #887.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
generate_inventory() emits Ansible's dynamic --list shape (all.hosts is a
LIST, vars under _meta) — valid only as an executable inventory script's
stdout. We were writing it to a static file and passing -i, so Ansible's yaml
plugin rejected it ("Invalid 'hosts' entry for 'all' group, requires a
dictionary, found ...list...") and fell back to implicit localhost → "no hosts
matched". Affected every steward:* scope run; surfaced on the first real
provision.
- New inventory_to_yaml(inv): convert the --list dict → a valid static YAML
inventory (all.hosts dict keyed by host, groups under all.children, group
vars preserved, injected per-host vars like steward_token retained).
- Wire it into runner.trigger_run, host_agent deploy + provision.
- executor writes the file as inventory.yml so the yaml plugin's extension
check reliably claims it.
- generate_inventory unchanged (still the --list dict); conversion happens at
write time. Unit tests added.
Scribe issue #885.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "could not decrypt a stored secret" warning was generic, so an operator
couldn't tell which of the six secret settings was encrypted under an old key.
Thread the setting key through _decode → decrypt_secret(context=...) so the log
now reads e.g. "Could not decrypt stored secret smtp.password (wrong/rotated
key — re-enter it)". Pure diagnostic; decrypt behaviour unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dev-only instance, no bookmarks — per family rule 22, fully remove old paths
instead of shimming them.
- Delete the /plugins/host_agent/ (index) and /plugins/host_agent/settings/
redirect routes; delete the now-dead host_list.html fleet template.
- Move the remaining management POST routes off /settings/ to /fleet/
(add-host, rotate-token, delete) — single canonical prefix.
- Repoint real callers to canonical URLs: dashboard widgets (host resources →
/hosts/, history → /plugins/host_agent/fleet/), the full-metrics page
breadcrumb + back link (→ the host hub), Settings→Ansible link, and the
agent panel's curl-install link.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Final phase of the host-IA unification (milestone 70).
- Settings → Plugins split into two tiers: "Monitoring capabilities"
(host_agent, http, snmp, docker — built-in host facets, surfaced via
Hosts/Status, on by default) and "Integrations" (traefik, unifi — external
systems, off until configured). Presentation only: a CAPABILITY_PLUGINS set
(overridable by plugin.yaml `kind:`) tags each plugin; module loading,
optional deps, and migrations are untouched.
- Drop the "default-enable a plugin" framing in the UI copy — capabilities are
described as built-in, not optional add-ons.
- Nav: remove the standalone "Uptime" item (folded into Hosts; still reachable
via the SLA button on the Hosts list).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Hosts list: new Agent column (latest CPU/mem read from the generic
PluginMetric table — no host_agent import) + an admin "Agent fleet" button.
- /plugins/host_agent/ (old fleet page) now redirects to /hosts/ (folded into
the hub; kept as a redirect so widgets/links don't 404).
- Agent management moved off the "settings" URL: the management page is now
/plugins/host_agent/fleet/ ("Agent fleet" — bulk provision/install/update +
registrations + curl install), reachable from the Hosts list. Old
/plugins/host_agent/settings/ redirects there. Per-host management lives on
the host detail page; this page is now explicitly the bulk/fleet view.
Milestone 70 phases 2-3. Phase 4 (plugins capability/integration split + nav
cleanup) next.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make Hosts the front-and-center hub. A host now has a real detail page at
/hosts/<id> that pulls its facets into one view, instead of management being
scattered across a nav-less Host-Agents area and the edit form.
- hosts: new GET /hosts/<id> detail route + hosts/detail.html. Shows the
monitors summary (ping/DNS status + latency + uptime 24h/7d/30d), an Ansible
section (linked target, link/create, run-playbook), and an embedded Agent
panel. Hosts list name links here; ansible-link redirects here.
- host_agent: GET /plugins/host_agent/panel/<host_id> — a self-contained HTMX
fragment embedded into the core hub across the plugin boundary (core never
imports plugin models). Shows live agent metrics + Update/Rotate/Remove when
installed, or the provisioning path when not: inline "generate managed key"
warning, a prompt to link an Ansible target first, then Provision (bootstrap
password) / Install (managed key) tied to the host's target scope.
Part of milestone 70 (Hosts hub). Phase 2+ will enrich the list, redirect the
old fleet/settings pages, and re-taxonomize plugins into capabilities vs
integrations.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Provisioning review corrections + the matching frontend, plus breadcrumb
header integration.
- provision.yml: authorize the managed pubkey with a regexp match on the
' steward-managed' comment so rotating the key REPLACES the host's steward
key in place instead of stacking a second authorized entry. Hand-added keys
(other comments) are untouched.
- update.yml (new): refresh agent.py + restart only. Does NOT rotate the token
or rewrite /etc/steward-agent.conf — the host keeps its identity. Asserts the
agent is already installed and fails clearly otherwise.
- host_agent /update route: runs update.yml as the managed steward user (no
token minting). Token rotation stays a deliberate action.
- settings/ansible/generate-key honors a safe relative `next` redirect, so an
inline trigger elsewhere returns to its page.
- Host Agents settings: reworked into a clear lifecycle — an intro card that
explains it runs Ansible to deploy the agent (+ inline "generate managed key"
warning/trigger when none exists), then three labelled cards: 1 Provision,
2 Install/enroll, 3 Update. Each explains what it does.
- base.html: breadcrumb now renders as a kicker line directly above the page
title (moved below alerts, tightened margin) so nested and top-level views
share one consistent header.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When you click Run, Steward now parses the playbook and renders a field
for each declared variable instead of a blank extra-vars textarea. The
form loads on demand via HTMX (/ansible/run-form/<source>/<playbook>).
- sources.discover_playbook_variables: parse vars: defaults + vars_prompt:
(vars_prompt wins on name collision; non-scalar vars skipped; role/include
vars not traversed). Flags secret-looking names + vars_prompt private.
- Run-time values flow through a JSON extra-vars file (-e @file), which is
space/quote-safe — fixes a latent shlex-split bug in the old -e key=value
textarea path. executor.build_extra_vars_file (pure) + start_run merge.
- Secret-flagged fields are masked AND routed through an unpersisted
secret_vars channel (runner.trigger_run → start_run), so passwords entered
at run time never land in the DB / run history.
- Defaults shown as placeholders (not prefilled): an untouched field falls
through to the inventory/play default instead of overriding it.
- routes: run_form HTMX endpoint; _parse_run_params now returns
(params, secret_vars, err) and reads var__/secret__ fields. Schedules drop
secret vars (can't prompt unattended).
- templates: ansible/_run_form.html fragment; browse.html rewired to HTMX,
static JS run-form removed. Advanced section keeps limit/tags/check + a
free-form extra-vars escape hatch.
- tests: test_playbook_variables.py (discovery + extra-vars file).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Turn the agent-install playbook into a full provisioning + maintenance
path. Solves the bootstrap chicken-and-egg: first contact uses an
operator-supplied password (one run, never stored), which creates a
dedicated `steward` login account with NOPASSWD sudo + Steward's managed
public key. Every run thereafter connects as `steward` with the managed
key — fully unattended (scheduled prune, agent updates).
- core/crypto: generate_ssh_keypair() — ed25519, OpenSSH formats.
- settings: ansible.ssh_public_key (non-secret, displayed) + ansible.ssh_user
(default steward); to_ansible_cfg extended.
- settings UI + route: "Generate managed key" (private encrypted, public
shown to copy) + SSH-user field.
- executor: build_bootstrap() writes a 0600 vars file (-e @file) for the
per-run user/password — never argv, never DB, never logged; drops the
managed key when a bootstrap password is given; --user floor from the
global ssh_user when no override.
- runner.trigger_run: pass-through `connection` kwarg, deliberately NOT
persisted on AnsibleRun.params (password stays out of the DB).
- bundled/host_agent/provision.yml: create steward user + authorized_keys
+ /etc/sudoers.d/steward (visudo-validated) + agent install.
- host_agent: /provision route + "Provision a fresh host" card (bootstrap
user/password; injects pubkey + user + token as space-safe JSON hostvars).
- Dockerfile: add sshpass (Ansible shells out to it for password SSH).
- tests: keypair generation + build_bootstrap (secret stays off argv).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes#550 (all four):
- Cancellation: track live subprocesses; POST /ansible/runs/<id>/cancel
(operator) SIGTERMs then SIGKILLs after a grace; new 'cancelled' status
(+ migration 0019, ALTER TYPE in autocommit). Queued runs cancel cleanly
before launch. Cancel button on run detail.
- Concurrency: global semaphore (ansible.max_concurrent_runs, default 3,
Settings→Ansible) caps simultaneous runs; excess show 'queued' (new status)
until a slot frees. Semaphore bound lazily per running loop.
- Structured results: parse PLAY RECAP into per-host ok/changed/unreachable/
failed/skipped + capture failed-task lines, stored in new results JSON
column (migration 0020); rendered as a host-summary table on run detail.
Keeps live streaming (no json-callback swap).
- Retention: full output written to a persistent log artifact
(/data/ansible/runs/<id>.log, env-overridable) beyond the 1 MB DB cap and
across restarts; in-memory replay buffer bounded + GC'd after completion;
Download-log route. Boot reconciliation now also sweeps stale 'queued'.
Unit tests for recap parsing + cancel flagging. Status colors updated across
run list / detail / schedules.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
