Commit Graph

180 Commits

Author SHA1 Message Date
bvandeusen 71715c38d8 feat(auth): return to the original view after an auth-expiry redirect
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 45s
When a session has expired, require_role now bounces to /login?next=<path> and
sends the user back there after re-login.

- middleware: safe_next_url() (same-site relative path/query only; rejects
  off-site, protocol-relative, javascript:, and the auth pages — no open
  redirect). _login_redirect() builds the next param; for HTMX requests it uses
  HX-Current-URL (the page, not the fragment) and HX-Redirect so the whole
  browser navigates instead of swapping the login page into a fragment.
- login GET/POST carry `next` (hidden field), validated, used on success for
  local + LDAP; OIDC stashes it in session across the IdP round-trip.
- login.html: hidden next field + next on the SSO link.
- tests for safe_next_url.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 13:05:37 -04:00
bvandeusen 2594ca517d fix(ansible): clear error when the managed SSH key can't be decrypted
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 6s
CI / integration (push) Successful in 2m17s
CI / publish (push) Successful in 54s
When ansible.ssh_private_key can't be decrypted (app secret key changed),
decrypt_secret returns the ciphertext unchanged; the executor was writing that
enc:v1: blob as the SSH key file → cryptic "Load key ...: error in libcrypto"
→ Permission denied. Now:
- build_credentials skips a still-encrypted key value (never writes ciphertext
  as a key file).
- start_run broadcasts a plain-language run error: "The managed SSH key could
  not be decrypted (the app secret key changed). Regenerate it in Settings →
  Ansible and re-provision the host(s)."

The run still fails (no usable key), but the reason is now obvious instead of a
libcrypto error. Operator remedy: regenerate the managed key + re-provision.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 12:52:24 -04:00
bvandeusen e0253fba48 fix(ansible): surface run failure reason; widget audit cleanup
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m17s
CI / publish (push) Successful in 1m1s
Run UX: when start_run throws before/around launch (e.g. ENOSPC creating the
temp dir — the box is out of disk), the run was marked "failed" with empty
output. Now the exception is broadcast + written to the run output/results so
the run view shows e.g. "[run error] OSError: [Errno 28] No space left on
device" instead of a blank failure.

Widget audit follow-ups (no broken links were found; these are consistency):
- host_resource_history widget now charts root (/) disk, consistent with the
  host panel (was the opaque "disk worst").
- host_resources widget: tooltip on the health dot explaining it warns on the
  worst mount while the number shows root.
- status_overview widget detail_url /status → /status/ (avoid redirect).
- Normalize ad-hoc widget empty-states to the shared .empty style (wording,
  which distinguishes "configured" vs "data yet", preserved).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 12:46:08 -04:00
bvandeusen 609bd78af2 feat(dashboard): unified Hosts widget + grouped widget picker
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m15s
CI / publish (push) Successful in 56s
Bring the dashboard in line with the unified host IA + improved displays.

- New core "Hosts — Overview" widget (/hosts/overview/widget): one row per host
  combining monitor status (ping dot + latency, uptime 24h) with the agent
  glance (CPU / memory / disk root + stale flag), each row linking to the host
  hub. Reads agent data from the generic PluginMetric table via a core-safe
  _agent_overview_by_host helper (no host_agent import); freshness vs the
  plugin's stale window. The granular Ping/DNS/Uptime/Agent widgets stay.
- Group the add-widget picker into Core monitors / Monitoring capabilities /
  Integrations (a `group` field on every WIDGET_REGISTRY entry + section
  headings in _edit_panels.html), matching the Settings → Plugins taxonomy.
- Fix the agent fleet widget rows to link to the host hub (/hosts/<id>) instead
  of the old /plugins/host_agent/<id>/ page.

Scribe #903.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 12:00:12 -04:00
bvandeusen 42f7840c26 feat(ansible): steward:category + steward:confirm playbook metadata
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m14s
CI / publish (push) Successful in 59s
Extend the playbook metadata convention with a namespaced `# steward:<key>:`
comment block:

- steward:category — free-text grouping label, shown as a badge in the browse
  list and on the run form.
- steward:confirm — true/yes/1/on marks a playbook destructive; the run form
  then requires a confirmation tick (required checkbox in the shared vars
  fragment) before it can launch.

sources.discover_playbook_meta() parses description + category + confirm (first
match per key; `# description:` still primary, `# steward:description:` alias).
discover_playbook_description() now delegates to it. The browse list reads
per-playbook meta to show category badges + descriptions; the run-form and
playbook-vars fragments render the badge + confirm gate.

Bundled playbooks tagged: docker_prune → category maintenance + confirm true;
provision/install/update → category host-agent.

Docs: docs/reference/playbook-authoring.md updated (keys now implemented) and a
quick reference added next to the code at steward/ansible/PLAYBOOK_CONVENTIONS.md.
Tests added for category/confirm/alias parsing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 11:35:35 -04:00
bvandeusen b32fce1d74 docs: Steward playbook-authoring conventions reference
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m22s
CI / publish (push) Successful in 4s
A self-contained guide (docs/reference/playbook-authoring.md) to the contract
between a playbook and Steward's run UI — the `# description:` comment,
vars/vars_prompt → fill-in fields, secret naming, hosts: all targeting,
managed credentials, and idempotency. Includes a "what Steward reads" summary
table, the metadata/extensibility note (only `# description:` today; reserved
`# steward:<key>:` namespace for future keys), and an annotated example.
Meant to be fed to another session authoring Steward-friendly playbooks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 11:26:56 -04:00
bvandeusen e5f6a11f94 feat(ansible): playbooks self-describe via "# description:" comment
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m15s
CI / publish (push) Successful in 52s
Playbooks can ship a human description Steward reads and shows when one is
selected. Convention: a `# description: <text>` magic comment (Ansible rejects
unknown play keys, so a comment is the portable place — works for third-party
playbooks too); falls back to the first play's name:. sources
.discover_playbook_description().

Surfaced at the top of the shared _playbook_vars.html partial, which loads on
playbook selection in the host run form, schedules form, and browse run form.
All four bundled playbooks (provision/install/update/docker_prune) now carry a
description line. Unit tests added.

Scribe #900.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 11:25:25 -04:00
bvandeusen a0d1c5f07c feat(host_agent): sparklines + load/core + PSI on the host panel
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m23s
CI / publish (push) Successful in 7s
- Each at-a-glance metric (CPU, Memory, Disk /, Load) now shows a 6h sparkline
  (reused core.status.sparkline_svg + a recent-series query; disk uses the root
  mount sub-resource) so trend/consistency is visible, not just the instant.
- Load is now normalized: "Load /core" = 1m load ÷ CPU cores as % (100% = run
  queue matches capacity), comparable across different hardware. Cores derived
  from the per-core CPU metrics already collected — no agent change. Raw load
  in the tooltip.
- Added a "Pressure 10s" line: PSI cpu/mem/io (some, avg10), the
  hardware-independent saturation signal already collected by the agent.

Scribe #898.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 11:10:46 -04:00
bvandeusen 36212dc58b feat(host_agent): at-a-glance disk = root (/) not "worst"; metric tooltips
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m19s
CI / publish (push) Successful in 6s
The "Disk (worst)" number was opaque at a glance. Show the root filesystem (/)
usage instead — what people actually care about — on the host panel and the
dashboard widget. "Worst" is kept only for the widget's health dot (so a full
/var or /data still warns) and on the full-metrics page (per-mount + worst
trend). Added hover tooltips defining CPU / Memory / Disk / Load.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 11:05:43 -04:00
bvandeusen f80f6c87e8 feat(auth): capture Steward URL during first-run admin setup (OOBE)
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m17s
CI / publish (push) Successful in 56s
A fresh install had no prompt for general.public_base_url, so first-run agent
installs/share links silently used the request Host header. Add a "Steward URL"
field to the first-run /setup page (create-admin), pre-filled with the current
address, with help text. setup_post saves general.public_base_url and applies
it to app.config immediately (no restart). Editable later in Settings → General.

Scribe #896.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 10:15:28 -04:00
bvandeusen 7e6e63521b fix(host_agent): provision/deploy vars shadowed by play-var precedence
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m15s
CI / publish (push) Successful in 54s
The assertion failed ("Pass steward_url/token/pubkey") because those were
injected as inventory HOST vars, but the playbooks declared them in the play
`vars:` block — and play vars OUTRANK inventory host vars, so the empty
defaults won and the injected values never reached the play.

- Pass globals (steward_url, steward_pubkey, steward_user, agent_interval) as
  extra-vars via the JSON -e @file (highest precedence, space-safe). Keep only
  the per-host steward_token as an inventory host var.
- provision.yml / install.yml: drop steward_token from `vars:` so the host var
  isn't shadowed; assertions use `| default('')` for the un-defaulted token.

This was the next layer under the inventory-format fix — first the inventory
wouldn't parse, now the injected vars actually apply.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 10:11:30 -04:00
bvandeusen ad726e65f3 feat(ansible): dropdown playbook selection + auto-populated variable fields
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m17s
CI / publish (push) Successful in 43s
Stop making operators type playbook paths and guess extra-var names.

- Reusable infra: shared ansible/_playbook_vars.html (the discovered-variable
  fields) + ansible/_playbook_options.html; two HTMX endpoints —
  /ansible/playbook-options (a source's playbooks, optional ?selected for edit)
  and /ansible/playbook-vars (a playbook's vars:/vars_prompt: as fill-in
  fields). browse _run_form.html refactored to include the shared partial.
- Host "Run a playbook against this host": source dropdown → playbook dropdown
  → variable fields, all chained via HTMX. Handler reuses _parse_run_params so
  var__/secret__ fields flow through extra_vars_map + the unpersisted
  secret_vars channel.
- Schedules: playbook free-text+datalist → source-dependent dropdown; fixed the
  extra-vars edit pre-fill to read extra_vars_map (stale list key after the
  earlier JSON-extra-vars change).

Scribe #895.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 10:02:15 -04:00
bvandeusen bb90411f00 fix(host_agent): gate "deployed" on agent check-in; fail no-op runs
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 9s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 58s
A failed provision looked successful in two ways:

1. The host panel showed the agent as deployed (metrics + Update/Rotate/Remove)
   because provision/deploy mint the registration row BEFORE the playbook runs
   and the panel keyed "installed" on that row. Now gated on the agent actually
   checking in (reg.last_seen_at). Three states: reporting (metrics + lifecycle),
   pending (token minted but no check-in → "no metrics yet, deploy may be
   running/failed" banner + retry + Clear pending registration), and none
   (install path).

2. The run reported success though nothing ran — ansible-playbook exits 0 on
   "no hosts matched"/empty inventory. The executor now treats an empty PLAY
   RECAP (returncode 0 but no hosts executed) as failed, with a clear failure
   note. Non-zero exits and recap failed/unreachable were already caught.

Scribe issue #887.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:50:11 -04:00
bvandeusen a92d1995d5 fix(ansible): write DB inventory as static YAML, not dynamic --list JSON
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m20s
CI / publish (push) Successful in 1m6s
generate_inventory() emits Ansible's dynamic --list shape (all.hosts is a
LIST, vars under _meta) — valid only as an executable inventory script's
stdout. We were writing it to a static file and passing -i, so Ansible's yaml
plugin rejected it ("Invalid 'hosts' entry for 'all' group, requires a
dictionary, found ...list...") and fell back to implicit localhost → "no hosts
matched". Affected every steward:* scope run; surfaced on the first real
provision.

- New inventory_to_yaml(inv): convert the --list dict → a valid static YAML
  inventory (all.hosts dict keyed by host, groups under all.children, group
  vars preserved, injected per-host vars like steward_token retained).
- Wire it into runner.trigger_run, host_agent deploy + provision.
- executor writes the file as inventory.yml so the yaml plugin's extension
  check reliably claims it.
- generate_inventory unchanged (still the --list dict); conversion happens at
  write time. Unit tests added.

Scribe issue #885.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:28:31 -04:00
bvandeusen 9ce4cce5c5 feat(crypto): name the failing setting in wrong-key decrypt log
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m21s
CI / publish (push) Successful in 58s
The "could not decrypt a stored secret" warning was generic, so an operator
couldn't tell which of the six secret settings was encrypted under an old key.
Thread the setting key through _decode → decrypt_secret(context=...) so the log
now reads e.g. "Could not decrypt stored secret smtp.password (wrong/rotated
key — re-enter it)". Pure diagnostic; decrypt behaviour unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 08:05:47 -04:00
bvandeusen cfe6b4c25f fix(host_agent): deploy/provision 500 — double-begin on session
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 9s
CI / integration (push) Successful in 2m21s
CI / publish (push) Successful in 8s
fetch_scope_targets() runs a SELECT that autobegins the session transaction
(SQLAlchemy 2.0), so the following `async with db.begin()` raised
"A transaction is already begun on this Session" → 500 on both
/plugins/host_agent/deploy and /provision.

Mint registrations directly into the autobegun transaction, build the
inventory while the ORM objects are still live, then await db.commit().
Escaped CI because unit tests use a mock db_sessionmaker; no integration
test exercises these routes (follow-up noted in Scribe issue #884).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 22:53:13 -04:00
bvandeusen 17c9c875e4 refactor(hosts): remove legacy host_agent redirects/paths (no back-compat)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m25s
CI / publish (push) Successful in 39s
Dev-only instance, no bookmarks — per family rule 22, fully remove old paths
instead of shimming them.

- Delete the /plugins/host_agent/ (index) and /plugins/host_agent/settings/
  redirect routes; delete the now-dead host_list.html fleet template.
- Move the remaining management POST routes off /settings/ to /fleet/
  (add-host, rotate-token, delete) — single canonical prefix.
- Repoint real callers to canonical URLs: dashboard widgets (host resources →
  /hosts/, history → /plugins/host_agent/fleet/), the full-metrics page
  breadcrumb + back link (→ the host hub), Settings→Ansible link, and the
  agent panel's curl-install link.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 21:20:11 -04:00
bvandeusen 7ef1af2184 feat(settings): Phase 4 — capabilities vs integrations + nav cleanup
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 53s
Final phase of the host-IA unification (milestone 70).

- Settings → Plugins split into two tiers: "Monitoring capabilities"
  (host_agent, http, snmp, docker — built-in host facets, surfaced via
  Hosts/Status, on by default) and "Integrations" (traefik, unifi — external
  systems, off until configured). Presentation only: a CAPABILITY_PLUGINS set
  (overridable by plugin.yaml `kind:`) tags each plugin; module loading,
  optional deps, and migrations are untouched.
- Drop the "default-enable a plugin" framing in the UI copy — capabilities are
  described as built-in, not optional add-ons.
- Nav: remove the standalone "Uptime" item (folded into Hosts; still reachable
  via the SLA button on the Hosts list).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:57:10 -04:00
bvandeusen f29255039d feat(hosts): Phase 2+3 — agent column, fold fleet + management into Hosts
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 9s
CI / integration (push) Successful in 2m16s
CI / publish (push) Successful in 54s
- Hosts list: new Agent column (latest CPU/mem read from the generic
  PluginMetric table — no host_agent import) + an admin "Agent fleet" button.
- /plugins/host_agent/ (old fleet page) now redirects to /hosts/ (folded into
  the hub; kept as a redirect so widgets/links don't 404).
- Agent management moved off the "settings" URL: the management page is now
  /plugins/host_agent/fleet/ ("Agent fleet" — bulk provision/install/update +
  registrations + curl install), reachable from the Hosts list. Old
  /plugins/host_agent/settings/ redirects there. Per-host management lives on
  the host detail page; this page is now explicitly the bulk/fleet view.

Milestone 70 phases 2-3. Phase 4 (plugins capability/integration split + nav
cleanup) next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:51:33 -04:00
bvandeusen 8bdf07f709 feat(hosts): Phase 1 — host detail hub page (unify host IA)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m16s
CI / publish (push) Successful in 48s
Make Hosts the front-and-center hub. A host now has a real detail page at
/hosts/<id> that pulls its facets into one view, instead of management being
scattered across a nav-less Host-Agents area and the edit form.

- hosts: new GET /hosts/<id> detail route + hosts/detail.html. Shows the
  monitors summary (ping/DNS status + latency + uptime 24h/7d/30d), an Ansible
  section (linked target, link/create, run-playbook), and an embedded Agent
  panel. Hosts list name links here; ansible-link redirects here.
- host_agent: GET /plugins/host_agent/panel/<host_id> — a self-contained HTMX
  fragment embedded into the core hub across the plugin boundary (core never
  imports plugin models). Shows live agent metrics + Update/Rotate/Remove when
  installed, or the provisioning path when not: inline "generate managed key"
  warning, a prompt to link an Ansible target first, then Provision (bootstrap
  password) / Install (managed key) tied to the host's target scope.

Part of milestone 70 (Hosts hub). Phase 2+ will enrich the list, redirect the
old fleet/settings pages, and re-taxonomize plugins into capabilities vs
integrations.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:29:34 -04:00
bvandeusen 6a8146b544 fix(ansible): replace steward key on reprovision + distinct agent update path
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m19s
CI / publish (push) Successful in 54s
Provisioning review corrections + the matching frontend, plus breadcrumb
header integration.

- provision.yml: authorize the managed pubkey with a regexp match on the
  ' steward-managed' comment so rotating the key REPLACES the host's steward
  key in place instead of stacking a second authorized entry. Hand-added keys
  (other comments) are untouched.
- update.yml (new): refresh agent.py + restart only. Does NOT rotate the token
  or rewrite /etc/steward-agent.conf — the host keeps its identity. Asserts the
  agent is already installed and fails clearly otherwise.
- host_agent /update route: runs update.yml as the managed steward user (no
  token minting). Token rotation stays a deliberate action.
- settings/ansible/generate-key honors a safe relative `next` redirect, so an
  inline trigger elsewhere returns to its page.
- Host Agents settings: reworked into a clear lifecycle — an intro card that
  explains it runs Ansible to deploy the agent (+ inline "generate managed key"
  warning/trigger when none exists), then three labelled cards: 1 Provision,
  2 Install/enroll, 3 Update. Each explains what it does.
- base.html: breadcrumb now renders as a kicker line directly above the page
  title (moved below alerts, tightened margin) so nested and top-level views
  share one consistent header.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 19:22:12 -04:00
bvandeusen a996cc6908 feat(ansible): per-variable fields in the playbook run form
CI / lint (push) Successful in 9s
CI / unit (push) Successful in 14s
CI / integration (push) Successful in 2m19s
CI / publish (push) Successful in 55s
When you click Run, Steward now parses the playbook and renders a field
for each declared variable instead of a blank extra-vars textarea. The
form loads on demand via HTMX (/ansible/run-form/<source>/<playbook>).

- sources.discover_playbook_variables: parse vars: defaults + vars_prompt:
  (vars_prompt wins on name collision; non-scalar vars skipped; role/include
  vars not traversed). Flags secret-looking names + vars_prompt private.
- Run-time values flow through a JSON extra-vars file (-e @file), which is
  space/quote-safe — fixes a latent shlex-split bug in the old -e key=value
  textarea path. executor.build_extra_vars_file (pure) + start_run merge.
- Secret-flagged fields are masked AND routed through an unpersisted
  secret_vars channel (runner.trigger_run → start_run), so passwords entered
  at run time never land in the DB / run history.
- Defaults shown as placeholders (not prefilled): an untouched field falls
  through to the inventory/play default instead of overriding it.
- routes: run_form HTMX endpoint; _parse_run_params now returns
  (params, secret_vars, err) and reads var__/secret__ fields. Schedules drop
  secret vars (can't prompt unattended).
- templates: ansible/_run_form.html fragment; browse.html rewired to HTMX,
  static JS run-form removed. Advanced section keeps limit/tags/check + a
  free-form extra-vars escape hatch.
- tests: test_playbook_variables.py (discovery + extra-vars file).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 17:54:01 -04:00
bvandeusen 0318f6423f feat(ansible): host provisioning via steward managed SSH identity
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 1m3s
Turn the agent-install playbook into a full provisioning + maintenance
path. Solves the bootstrap chicken-and-egg: first contact uses an
operator-supplied password (one run, never stored), which creates a
dedicated `steward` login account with NOPASSWD sudo + Steward's managed
public key. Every run thereafter connects as `steward` with the managed
key — fully unattended (scheduled prune, agent updates).

- core/crypto: generate_ssh_keypair() — ed25519, OpenSSH formats.
- settings: ansible.ssh_public_key (non-secret, displayed) + ansible.ssh_user
  (default steward); to_ansible_cfg extended.
- settings UI + route: "Generate managed key" (private encrypted, public
  shown to copy) + SSH-user field.
- executor: build_bootstrap() writes a 0600 vars file (-e @file) for the
  per-run user/password — never argv, never DB, never logged; drops the
  managed key when a bootstrap password is given; --user floor from the
  global ssh_user when no override.
- runner.trigger_run: pass-through `connection` kwarg, deliberately NOT
  persisted on AnsibleRun.params (password stays out of the DB).
- bundled/host_agent/provision.yml: create steward user + authorized_keys
  + /etc/sudoers.d/steward (visudo-validated) + agent install.
- host_agent: /provision route + "Provision a fresh host" card (bootstrap
  user/password; injects pubkey + user + token as space-safe JSON hostvars).
- Dockerfile: add sshpass (Ansible shells out to it for password SSH).
- tests: keypair generation + build_bootstrap (secret stays off argv).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 17:17:38 -04:00
bvandeusen 6e91bdc82b feat(security): encrypt sensitive settings at rest (Fernet)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m17s
CI / publish (push) Successful in 45s
Secrets (smtp.password, oidc.client_secret, ldap.bind_password, ansible
ssh_private_key/become_password/vault_password) were stored plaintext in
app_settings. Add transparent encryption-at-rest:

- steward/core/crypto.py: Fernet keyed off the app secret (/data/secret.key),
  enc:v1: prefix marks ciphertext; passthrough for plaintext/empty/no-key,
  never reveals plaintext on a wrong key.
- settings.py: SECRET_KEYS registry; set_setting encrypts on write; all read
  paths (get_setting / get_all_settings / load_settings_sync) decrypt
  transparently; migrate_plaintext_secrets() converts legacy rows in place.
- app.py startup: init_crypto(SECRET_KEY) + one-time legacy-secret migration
  before settings load.
- Add cryptography dependency.

UI masking is unchanged (it checks decrypted truthiness). Key-loss caveat
documented: secrets are unrecoverable if the app secret key is lost. Unit
tests cover round-trip, empty/plaintext passthrough, and wrong-key safety.

Task #580.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 15:22:43 -04:00
bvandeusen 88857be24e feat(ansible): runner robustness — cancel, concurrency, structured results, retention
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 55s
Closes #550 (all four):

- Cancellation: track live subprocesses; POST /ansible/runs/<id>/cancel
  (operator) SIGTERMs then SIGKILLs after a grace; new 'cancelled' status
  (+ migration 0019, ALTER TYPE in autocommit). Queued runs cancel cleanly
  before launch. Cancel button on run detail.
- Concurrency: global semaphore (ansible.max_concurrent_runs, default 3,
  Settings→Ansible) caps simultaneous runs; excess show 'queued' (new status)
  until a slot frees. Semaphore bound lazily per running loop.
- Structured results: parse PLAY RECAP into per-host ok/changed/unreachable/
  failed/skipped + capture failed-task lines, stored in new results JSON
  column (migration 0020); rendered as a host-summary table on run detail.
  Keeps live streaming (no json-callback swap).
- Retention: full output written to a persistent log artifact
  (/data/ansible/runs/<id>.log, env-overridable) beyond the 1 MB DB cap and
  across restarts; in-memory replay buffer bounded + GC'd after completion;
  Download-log route. Boot reconciliation now also sweeps stale 'queued'.

Unit tests for recap parsing + cancel flagging. Status colors updated across
run list / detail / schedules.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 14:51:04 -04:00
bvandeusen 389002fc6f feat(ui): breadcrumb navigation across nested pages
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 1m2s
Adds a breadcrumb trail to nested views for orientation. Mechanism: a
{% block breadcrumb %} slot in base.html (+ styling) and a shared crumbs()
macro in templates/_macros.html; each nested page fills the block with its
trail (root→current, last item is the current page). Pages without the block
render no bar, so top-level nav roots stay clean.

Applied to: Ansible (browse, schedules, playbook editor, run detail) +
inventory (targets/groups + detail), host_agent (fleet, host detail,
settings), hosts form, settings tabs (ansible/auth/notifications/plugins/
reports + plugin detail), dashboard (list, edit), and alerts (rule form,
maintenance + new). Dynamic labels (host/target/run/dashboard names) come
from the page context — no route changes. All 60 templates Jinja-compile.

Task #873.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 14:22:19 -04:00
bvandeusen 71e4724286 feat(ansible): cross-link inventory/schedules/browse for discoverability
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m21s
CI / publish (push) Successful in 56s
The inventory CRUD UI (/ansible/inventory/targets + /groups) existed but was
unreachable from the main Ansible pages — only a buried text hint pointed to
it. Add an "Inventory" button to the Runs, Browse, and Schedules headers, and
"← Ansible" back-links on the inventory target/group pages, so the targeting
features (manual runs, schedules, Deploy-via-Ansible) are findable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 14:09:58 -04:00
bvandeusen c10eae1c74 feat(ansible): in-app playbook authoring/editor
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m17s
CI / publish (push) Successful in 1m6s
Adds create/edit/delete of playbooks from the UI (admin only), so a homelab
user without a git workflow can author automation in-app. A new always-present
writable local source "steward-local" (/data/ansible/playbooks, env-overridable,
created on first save) is editable alongside operator local-dir sources; the
bundled and git sources stay read-only (git is GitOps, clobbered on pull).

sources.py: write_playbook / delete_playbook (traversal-guarded, .yml/.yaml
only) + validate_playbook_yaml (YAML + play-list check) + is_editable_source.
routes.py: /playbooks/new, /edit, /save, /delete (admin). Browse gains a
"New playbook" button and per-playbook + view-page Edit/Delete for editable
sources. Plain textarea editor with save-time YAML validation. Unit tests for
write/delete/guard/validate.

Task #579 — completes milestone #37 (Ansible automation).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 11:39:47 -04:00
bvandeusen f3e919892d feat(plugins): plugin capability registry + host_agent→Ansible deploy synergy
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 1m10s
Implements #253's framework: a small core capability registry
(steward/core/capabilities.py) where a module/plugin publishes a named,
role-gated action and a consumer discovers it via has_capability() and runs it
via invoke_capability() — no hard import, graceful degradation, permission
propagation (actor role checked against the capability's required_role).

Core publishes "ansible.run_playbook" (operator) wrapping ansible.runner.
trigger_run (extended to accept a caller-built inventory). First consumer: the
host_agent plugin gains "Deploy via Ansible" on its settings page — pick an
inventory target/group and it installs/updates the agent via the bundled
host_agent/install.yml, minting a fresh token per host and injecting it as an
inventory hostvar (turning per-host curl|sh into one run). Exposed role
ordering as middleware.role_meets. Unit tests for the registry + role checks.

Task #253 (milestone #37).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 11:28:29 -04:00
bvandeusen 656bda2e3d feat(ansible): bundled first-party playbooks + built-in source
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m16s
CI / publish (push) Successful in 55s
Ships a read-only "steward-builtin" source (steward/ansible/bundled/) that
always appears alongside operator-configured sources, with two playbooks:
- maintenance/docker_prune.yml — docker system prune for swarm/standalone
  nodes (safe by default; prune_all_images / prune_volumes extra-vars to
  widen). Schedule it against a swarm-node group for recurring cleanup (#869).
- host_agent/install.yml — installs/updates the host agent (mirrors
  install.sh: user, agent.py, config, hardened systemd unit), parameterised
  with steward_url + steward_token extra-vars.

get_sources() now prepends the builtin source. Tests updated to find the
configured git source by name; added coverage that the bundled playbooks are
discoverable.

Tasks #869 + agent-install (milestone #37).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 11:07:34 -04:00
bvandeusen 4a0a3ee46e feat(ansible): scheduled recurring playbook runs
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m20s
CI / publish (push) Successful in 49s
Adds cron-like recurring runs (the engine the maintenance-automation work
needs). New AnsibleSchedule model + migration 0018; a core ScheduledTask
(ansible_scheduled_runs, 60s) fires due schedules, each creating a
system-triggered AnsibleRun (triggered_by=None). Centralises the
resolve-inventory → create-run → launch flow in ansible/runner.trigger_run,
shared by the manual route (refactored to use it) and the scheduler.

Schedules UI under /ansible/schedules: create/edit/pause/delete/run-now,
with interval presets, scope targeting (all / group / target), extra-vars /
limit / tags / dry-run, and last-run status (resolved via last_run_id) +
next-run. Unit test for the due-check.

Task #549 (milestone #37).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 11:04:02 -04:00
bvandeusen bca1b92cc6 feat(host_agent): Netdata-style fleet overview + per-host detail view
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m24s
CI / publish (push) Successful in 58s
Surfaces the metrics the agent now collects (task #868). Adds a viewer-facing
fleet page (/plugins/host_agent/) with per-host cards (CPU/mem/disk/load/temp,
stale flag) and a per-host detail page (/plugins/host_agent/<id>/) — the link
the fleet widget already pointed at but had no route for.

Detail page shows current gauges (CPU, memory incl. swap/cache, load, network
rx/tx, disk I/O, temp max, memory/cpu/io PSI), per-core CPU bars, per-mount
filesystem bars, and per-interface/disk/sensor breakdowns, plus history charts
(utilization %, throughput B/s, load & pressure) over a selectable range.
Charts use a linear epoch-ms x-axis so no Chart.js date adapter is needed.
Stale state uses the plugin's stale_after_seconds threshold. Repoints the
host_resources widget detail link from admin settings to the new fleet page.

Completes milestone #68 (task #867).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 10:27:57 -04:00
bvandeusen f037b69c58 test(host_agent): update existing collector tests for new agent API
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m15s
CI / publish (push) Successful in 8s
collect_cpu now returns (aggregate, [per_core]) and build_sample takes a
rate-state dict, so the pre-existing tests that asserted the old float
return / no-state signature needed updating. Mocks the new collectors
(temps/psi/net/diskio) for determinism.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 10:15:43 -04:00
bvandeusen a9e7baee6a feat(host_agent): collect network, disk I/O, per-core CPU, temps, memory PSI
CI / lint (push) Successful in 3s
CI / unit (push) Failing after 8s
CI / integration (push) Successful in 2m17s
CI / publish (push) Has been skipped
Extends the push agent (v1.2.0) with the Netdata-style signals the operator
wants: per-core CPU, per-interface network throughput and per-disk I/O
(rates derived in-agent from monotonic /proc counter deltas, with
counter-reset clamping), hardware temperatures (/sys/class/hwmon),
memory-pressure PSI (/proc/pressure), and cached/buffers memory breakdown.
All stdlib-only.

Server side needs no migration — these land as additional rows in the shared
PluginMetric table via _expand_sample_to_metrics. Per-resource series use a
':' in resource_name (host:net:eth0, host:core0, host:temp:Package) so the
existing fleet widget's ':' filter ignores them; host-level totals/max are
emitted at the bare host resource. New sample keys are optional, so older
agents keep ingesting unchanged.

Unit tests for the new parsers, rate/reset logic, and the expander contract.
Data foundation for the Netdata-style host view (task #867).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 10:11:30 -04:00
bvandeusen 3b6e005ed8 feat(status): unified Status page + widget across ping/DNS/HTTP monitors
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m15s
CI / publish (push) Successful in 56s
Adds a Kuma-style "is everything up?" surface that aggregates heterogeneous
monitor types via a status-source registry (steward/core/status.py): each
type registers an async source(db) -> [StatusEntry]. Core registers ping/DNS;
the http plugin registers its own from setup() so core never imports plugin
tables. Per entry: current up/down, last-30 heartbeat bar, uptime %
(24h/7d/30d), latest latency + response sparkline, and TLS expiry countdown
(HTTP). New /status page (live htmx refresh) + a status_overview dashboard
widget + nav link. Pure-function unit tests for registry + sparkline.

First deliverable of milestone #68 (task #866).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 22:18:29 -04:00
bvandeusen 67a1bc740d fix(test): call sync create_app directly in ansible inventory integration test
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 1m7s
create_app is synchronous (returns Quart) and runs its own internal
asyncio.run for settings/migrations. _make_app wrapped it in asyncio.run,
which nested event loops and raised "a coroutine is required" — the other
integration tests already call create_app() directly. This test was added
in an unpushed commit, so CI never exercised it until now.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 21:39:41 -04:00
bvandeusen b49496b57b feat(plugins): default-enable generic bundled plugins on fresh install
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Failing after 2m16s
CI / publish (push) Has been skipped
A fresh install enabled zero plugins — settings.py DEFAULTS had no plugin.*
keys, so to_plugins_cfg returned {} and every plugin had to be flipped on
by hand. Seed docker, host_agent, http, snmp as default-on (generic, non-
vendor-specific); traefik and unifi stay opt-in. Stored choices override
the default, so disabling persists.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 21:24:21 -04:00
bvandeusen 80613d6310 chore(lint): remove unused imports flagged by ruff
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 18:55:21 -04:00
bvandeusen 923905e72d test(ansible): integration tests for inventory tables + fetch_scope_targets 2026-06-05 18:52:10 -04:00
bvandeusen dd0acaf623 feat(ansible): host edit page Ansible target link/create/unlink section 2026-06-05 18:51:33 -04:00
bvandeusen 37d9ca8a8a feat(ansible): inventory targets CRUD templates 2026-06-05 18:50:40 -04:00
bvandeusen 935cbc105c feat(ansible): inventory groups CRUD routes + templates + blueprint registration 2026-06-05 18:50:09 -04:00
bvandeusen 2c991eabd2 feat(ansible): scope picker in run form + create_run() DB inventory generation 2026-06-05 18:49:02 -04:00
bvandeusen d906232eb2 feat(ansible): GIT_ASKPASS per-source HTTP token auth + settings UI 2026-06-05 18:48:00 -04:00
bvandeusen ea47169049 feat(ansible): generate_inventory() + fetch_scope_targets() + unit tests 2026-06-05 18:46:22 -04:00
bvandeusen 10df05c13e feat(ansible): add inventory_scope to AnsibleRun — 0017_ansible_run_scope migration 2026-06-05 18:45:38 -04:00
bvandeusen fa32488fdf feat(ansible): AnsibleGroup + join table — 0016_ansible_inventory_groups migration 2026-06-05 18:45:16 -04:00
bvandeusen c11b3f01ed feat(ansible): AnsibleTarget model + 0015_ansible_inventory_targets migration 2026-06-05 18:45:00 -04:00
bvandeusen eb319d715e docs: Ansible inventory infrastructure implementation plan 2026-06-05 18:19:17 -04:00
bvandeusen 06dc5073ca fix(deploy): explicit backend network for steward↔db traffic
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 6s
CI / integration (push) Successful in 2m13s
CI / publish (push) Successful in 5s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 11:37:10 -04:00