#5212 "Plain labels: no raw ids, scopes, metric keys or unexplained jargon",
batch 2:
- A run records where it came from: AnsibleRun.origin ("schedule", "alert"
or "mcp") and origin_id, set by the scheduler, Run now, alert actions and
the MCP tools. Migration 0031 backfills past runs from alert events,
schedules' last runs, and MCP audit entries.
- The runs list has a Started by column: "schedule nightly", "alert rule
nas down", "alice via MCP" or "alice", linked where there is a page.
The raw scope tooltip is gone.
- Run detail: no run id in the subtitle or body, Started by in words, and
the host summary lists failed and unreachable hosts first. The live
output only follows new lines while the reader is at the bottom, and
when the run finishes the page reloads once so the badge and summary
are final.
- list_ansible_runs and get_ansible_run report origin and origin_id.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5211 "Lists load as you scroll: audit, alert history, Ansible runs"
(milestone 477), per preference 172.
- core/paging.py: keyset paging over (time, id), newest first, with an
opaque "<iso>|<id>" cursor; a page reads one row past itself to know
whether another follows. No OFFSET, so rows added while someone scrolls
never shift the pages after them.
- _macros.html scroll_more / scroll_end and static/scroll-more.js, written
once: an IntersectionObserver with a 600px margin fetches the next page
before the reader reaches the end; "Loading more…" while in flight,
"End of the list." at the end, and a failed page says so with a Retry
button instead of re-firing on every scroll.
- Audit log: no "Load more" link and no 500-row cap.
- Alert history: no 200-row stop with "pick a shorter range";
alert_svc.history takes the cursor, so the MCP tool and the page share
one query.
- Ansible runs: no silent cap of 50.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5210 "Problems first, with since-time, across every list" (milestone 477),
first batch.
- Alerts: the Active list reads firing, no data, acknowledged, pending, the
longest-running first, and re-polls itself. An acknowledged alert says how
long it has been breaching (from the alert's events; acknowledging moves
last_transition_at) and, apart from that, who acknowledged it and when.
Rules show what each is doing now; disabled rules sort last. The current
value reads the latest-value table (#5208) instead of plugin_metrics.
- Monitors: a down monitor says "Down 12m · connection refused" in the list.
- Hosts: sorted down, agent gone quiet, pending, healthy. The agent column
shows "—" for a missing reading (was 0%) and a Stale badge with its age.
agent_metrics_by_host is gone; the list reads agent_overview_by_host.
- Schedules: failing first (failed or interrupted run, or a firing that could
not start one), then running, then the rest, paused last; status badge
from the shared states.
- Maintenance windows: active (ending soonest), then upcoming (starting
soonest), with expired folded away; each says when it ends/starts/ended.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The "[queued — waiting for run X on the same host]" and run-slot lines were
broadcast to live viewers and never stored, so a run that sat queued had no
record of why once the live view was gone (CI 8439: the overlap test found
the order right and the explanation missing). _claim_hosts now returns the
line it waited on and start_run seeds the run's output with every wait.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- One run per host: start_run claims every host it touches (addresses read
from the inventory Steward wrote; a source's own inventory file is claimed
as that file) before it takes a run slot. The claim is all-or-nothing. A run
waiting on a host shows queued and names the run it waits for, honours a
cancel, and fails after HOST_WAIT_SECONDS (1 h) with which host and run.
- A due schedule whose previous run is still queued or running skips the
firing: last_run_at moves on, last_run_id still names the run going, and
last_error says "Skipped: the previous run was still going". The Schedules
page shows that beside the run (muted), and a launch error in red.
- An alert rule that fires while the action run from an earlier firing is
still going starts no new run; the event says so and names the run. An
in-process set closes the gap between the check and the run being linked.
- docs/core/ansible.md: queued/cancelled statuses and the overlap rules.
Tests: tests/core/test_ansible_overlap.py (claim keys, waiting, all-or-nothing,
cancel, deadline) and tests/integration/test_ansible_overlap.py (two real
runs on one host take turns; schedule skip then fire; alert action skip then
start).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- git ships in the image; checkouts live under /data/ansible/sources
(STEWARD_ANSIBLE_SOURCE_DIR), on the persistent volume.
- One task, ansible_git_pulls, reads the sources every 30 s and pulls each
as its interval passes: a source added in Settings is pulled without a
restart, a removed one is forgotten.
- git_pull: a 120 s timeout ending the process group, GIT_TERMINAL_PROMPT=0,
pull --ff-only, a failed clone removed; it raises GitPullError with git's
message, so the Sync button stops saying Synced after a failure (and the
message is now escaped).
- An alert action's vars and the firing's steward_alert_* vars reach
ansible-playbook as one JSON extra-vars file, not -e key=value, which
Ansible split on whitespace (a rule named 'High CPU on web').
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Output is read in 64 KiB chunks and split into lines here; a line past
64 KiB is cut and marked instead of raising out of readline's limit.
- ansible-playbook runs in its own session; whatever ends the read (finish,
an error, the task being cancelled) SIGTERMs then SIGKILLs its whole group,
so no ssh or worker it forked outlives the run.
- A run whose task is cancelled (shutdown) records 'cancelled' with
finished_at before re-raising, instead of staying 'running'.
- A cancel that lands after the process exited 0 no longer turns a
successful run into 'cancelled'.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Runs read "update.yml on web-1" (run_title over the scope label, beside
describe_scope) on the runs list, the run page's title and breadcrumb, and
the run-started panel. The list shows started |ago (full time on hover),
|duration (counting "so far" while a run is going) and a dry-run tag.
Run statuses join STATES, so the list and the run page use status_badge:
queued dim, running info (blue, so work in progress doesn't read as a
problem), success ok "Succeeded", failed crit, interrupted warn, cancelled
dim. run_started.html leaves its hard-coded colours for tokens; the run
page's facts move to the .kv list and its failures to var(--red). Template
hex colours: 54 -> 47.
Also fixes#5124's sidebar count, which CI run 8389 failed on: the stylesheet
guard allows no weight above 500, and .nav-count asked for 600.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5037 "Secrets at rest and in run output: webhook URL and git token
encrypted, askpass via env, secret values scrubbed from logs".
- webhook.url joins SECRET_KEYS; existing values are sealed at startup by
migrate_plaintext_secrets. Settings -> Notifications no longer echoes the
URL: it names the saved host, keeps it when left blank, and has a Remove
box.
- Git source http_token is sealed inside ansible.sources on save and at
startup, opened in get_sources, and counted by the undecryptable-secrets
scan as ansible.sources[<name>].http_token.
- The askpass script holds no token: git runs it with STEWARD_GIT_TOKEN in
its environment and printf prints it as data.
- The executor masks every secret the run is handed (become/vault
passwords, bootstrap passwords, sensitive run vars, and the per-host agent
tokens the host_agent plugin now passes as redact) in each line before it
is broadcast, logged or stored.
- Bundled "Write agent config" tasks are no_log; a test fails if any bundled
task templating steward_token lacks it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5034 "Playbook runs: path and inventory checks in trigger_run, connection
vars admin-only, confirm enforced server-side, web actions audited".
Ansible runs inside the Steward container, so an operator who could name
any playbook path, set ansible_connection=local or ProxyCommand ssh args,
or put {{ lookup('pipe', …) }} in a value could run commands on Steward.
- steward/ansible/guards.py: check_playbook (the source must list it),
check_inventory_scope (steward:* or an inventory file the source has),
check_vars_for_role (ansible_* beyond host/port/user/password/become,
and any Jinja in a value, are admin-only), changed_vars (an edit is
held to the role only for what it changes).
- trigger_run applies the path checks, so every run path — web, MCP,
alerts, schedules, plugins — gets them.
- Variables are checked by role on the web run form, host-page runs,
schedule create/update, inventory group/target create/update, and MCP
run_playbook / create_schedule / update_schedule.
- A steward:confirm playbook needs the confirmation ticked on the server
side for a real (non-check) web run, as MCP already required.
- Audited: runs, cancel, schedule create/update/enable/disable/delete/
run-now, playbook create/update/delete, inventory group and target
create/update/delete (variable names only, never values). One
audit_request helper serves the web routes.
- The CSRF form guard and the route guard now each test that they fail
on a synthetic broken input (rule: a guard must be able to fail).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5014 "Ansible run scope is shown as a raw id"
- inventory_gen.describe_scope / scope_labels turn "steward:target:<uuid>"
into "target web-1", "steward:group:<uuid>" into "group Docker Swarm",
and a deleted one into "target abcdef01 (deleted)".
- MCP run and schedule rows carry scope_label beside the stored scope; the
ansible.run.started audit detail records it too.
- The web run page and schedules table show the name, with the stored
scope as the tooltip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
M78 step 1. The bundled maintenance/docker_prune.yml was a single
`docker system prune -f`; the upcoming per-host prune buttons need a
granular split. Add a `prune_target` extra-var:
- containers → docker container prune -f
- images → docker image prune -f (+ -a when prune_all_images)
- system → docker system prune -f (+ -a / --volumes) — the DEFAULT,
so existing manual/scheduled callers (which set no var)
keep today's behavior.
Reuses the existing bundled playbook rather than adding parallel files;
keeps the description/category/confirm meta so it still self-describes as
a confirm-gated maintenance run. Adds unit coverage: the playbook stays
valid YAML, its meta still discovers, and all three targets are guarded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CAGR73DUowdVFVvYzLXC5C
The schedule and host run forms opened their Source/Playbook <select>s on a
"— choose … —" placeholder, forcing an extra click. Default them to the first
item and cascade the dependent fields so the initial state is real:
- Remove the placeholder options (schedules source+playbook, host source+
playbook, and the shared _playbook_options fragment).
- schedules(): pass sel_source (first source, or the edited schedule's) and
that source's pre-rendered playbooks so the playbook dropdown starts populated
(create mode too, no longer the all-sources union).
- Cascade on load/change: the source <select> fires playbook-options then
(hx-on::after-settle) re-triggers the playbook <select>, which loads that
playbook's variable fields. Host form drives the whole chain from a load
trigger; schedule create mode loads vars via the playbook <select>'s own load
trigger, while edit mode keeps its server-rendered prefill untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Ansible Schedules form only offered a free-form extra-vars textarea,
unlike the browse/host run forms which auto-populate a field per declared
playbook variable (vars:/vars_prompt:) plus the playbook description. Wire
the schedule form to the same shared `_playbook_vars.html` partial:
- Playbook <select> now loads /ansible/playbook-vars?schedule=1 on change
into a #sch-playbook-vars container (mirrors the host detail form).
- Edit mode pre-renders the saved playbook's variables server-side with
their stored values; the extra-vars textarea keeps only leftover
(non-declared) vars.
- Partial gains optional `values` (prefill) and `schedule` flags. In the
schedule context secret vars render disabled ("secrets can't be
scheduled") since they're dropped downstream, and the destructive-confirm
gate is shown but not required.
Backend already parsed var__* fields for schedules, so this is the
front-end/partial parity fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Docker collection moves off the central single-socket scrape onto the host
agent, giving Docker a real per-host dimension. The Steward host now reports
its own containers like any other host, and same-named containers on different
hosts no longer collide.
- agent: stdlib UDS Docker client (AF_UNIX HTTP/1.1, Connection: close,
chunked-aware), collect_docker() ports the cpu%/mem math; sample["docker"]
added best-effort (silent-skip on absent/unreadable socket). AGENT_VERSION
1.2.0 → 1.3.0; optional docker_socket config key.
- ingest: host_agent ingest hands per-host container snapshots to the docker
plugin via a new "docker.persist_host_samples" capability (no hard import,
no-op when docker disabled), inside a SAVEPOINT so a docker failure never
sinks the host metrics. Resource names are host-scoped ("<host>/<name>").
- schema: docker_containers re-keyed (host_id, name); docker_metrics gains
host_id; docker_002 migration DROP+recreates (dev-only, rule 122).
- ui: Docker page + widgets grouped by host with host links; new per-host
Docker panel embedded on the Hosts hub (gated on docker enabled via a new
enabled_plugins template context). Replaces the SQLite-only strftime
bucketing with DB-agnostic Python bucketing.
- provisioning: install/provision playbooks add steward-agent to the docker
group (best-effort) so the agent can read the socket.
- removed central scrape: docker scheduler.py + scraper.py deleted; plugin.yaml
socket_path/scrape_interval_seconds/include_stopped dropped (plugin 2.0.0).
- tests: agent docker collector units (math, chunked decode, silent-skip,
sample shape, config) + integration (host-scoped schema + persistence).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
New maintenance/system_update.yml: cross-distro OS package upgrade (apt +
dnf/yum) with two run-form flags:
- restart_services: restart every service that needs it after the upgrade
(Debian via needrestart -r a, installed if missing; RHEL via
needs-restarting -s → systemctl try-restart).
- reboot_if_required: reboot the host only when a reboot is actually pending
(Debian /var/run/reboot-required; RHEL needs-restarting -r). Never reboots
otherwise.
Tagged steward:category=maintenance and steward:confirm=true (significant /
reboot-capable), so it shows in the run UI with the confirm gate and both flags
as fill-in fields. No code changes — auto-discovered from the builtin source.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When ansible.ssh_private_key can't be decrypted (app secret key changed),
decrypt_secret returns the ciphertext unchanged; the executor was writing that
enc:v1: blob as the SSH key file → cryptic "Load key ...: error in libcrypto"
→ Permission denied. Now:
- build_credentials skips a still-encrypted key value (never writes ciphertext
as a key file).
- start_run broadcasts a plain-language run error: "The managed SSH key could
not be decrypted (the app secret key changed). Regenerate it in Settings →
Ansible and re-provision the host(s)."
The run still fails (no usable key), but the reason is now obvious instead of a
libcrypto error. Operator remedy: regenerate the managed key + re-provision.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Run UX: when start_run throws before/around launch (e.g. ENOSPC creating the
temp dir — the box is out of disk), the run was marked "failed" with empty
output. Now the exception is broadcast + written to the run output/results so
the run view shows e.g. "[run error] OSError: [Errno 28] No space left on
device" instead of a blank failure.
Widget audit follow-ups (no broken links were found; these are consistency):
- host_resource_history widget now charts root (/) disk, consistent with the
host panel (was the opaque "disk worst").
- host_resources widget: tooltip on the health dot explaining it warns on the
worst mount while the number shows root.
- status_overview widget detail_url /status → /status/ (avoid redirect).
- Normalize ad-hoc widget empty-states to the shared .empty style (wording,
which distinguishes "configured" vs "data yet", preserved).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extend the playbook metadata convention with a namespaced `# steward:<key>:`
comment block:
- steward:category — free-text grouping label, shown as a badge in the browse
list and on the run form.
- steward:confirm — true/yes/1/on marks a playbook destructive; the run form
then requires a confirmation tick (required checkbox in the shared vars
fragment) before it can launch.
sources.discover_playbook_meta() parses description + category + confirm (first
match per key; `# description:` still primary, `# steward:description:` alias).
discover_playbook_description() now delegates to it. The browse list reads
per-playbook meta to show category badges + descriptions; the run-form and
playbook-vars fragments render the badge + confirm gate.
Bundled playbooks tagged: docker_prune → category maintenance + confirm true;
provision/install/update → category host-agent.
Docs: docs/reference/playbook-authoring.md updated (keys now implemented) and a
quick reference added next to the code at steward/ansible/PLAYBOOK_CONVENTIONS.md.
Tests added for category/confirm/alias parsing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Playbooks can ship a human description Steward reads and shows when one is
selected. Convention: a `# description: <text>` magic comment (Ansible rejects
unknown play keys, so a comment is the portable place — works for third-party
playbooks too); falls back to the first play's name:. sources
.discover_playbook_description().
Surfaced at the top of the shared _playbook_vars.html partial, which loads on
playbook selection in the host run form, schedules form, and browse run form.
All four bundled playbooks (provision/install/update/docker_prune) now carry a
description line. Unit tests added.
Scribe #900.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The assertion failed ("Pass steward_url/token/pubkey") because those were
injected as inventory HOST vars, but the playbooks declared them in the play
`vars:` block — and play vars OUTRANK inventory host vars, so the empty
defaults won and the injected values never reached the play.
- Pass globals (steward_url, steward_pubkey, steward_user, agent_interval) as
extra-vars via the JSON -e @file (highest precedence, space-safe). Keep only
the per-host steward_token as an inventory host var.
- provision.yml / install.yml: drop steward_token from `vars:` so the host var
isn't shadowed; assertions use `| default('')` for the un-defaulted token.
This was the next layer under the inventory-format fix — first the inventory
wouldn't parse, now the injected vars actually apply.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stop making operators type playbook paths and guess extra-var names.
- Reusable infra: shared ansible/_playbook_vars.html (the discovered-variable
fields) + ansible/_playbook_options.html; two HTMX endpoints —
/ansible/playbook-options (a source's playbooks, optional ?selected for edit)
and /ansible/playbook-vars (a playbook's vars:/vars_prompt: as fill-in
fields). browse _run_form.html refactored to include the shared partial.
- Host "Run a playbook against this host": source dropdown → playbook dropdown
→ variable fields, all chained via HTMX. Handler reuses _parse_run_params so
var__/secret__ fields flow through extra_vars_map + the unpersisted
secret_vars channel.
- Schedules: playbook free-text+datalist → source-dependent dropdown; fixed the
extra-vars edit pre-fill to read extra_vars_map (stale list key after the
earlier JSON-extra-vars change).
Scribe #895.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A failed provision looked successful in two ways:
1. The host panel showed the agent as deployed (metrics + Update/Rotate/Remove)
because provision/deploy mint the registration row BEFORE the playbook runs
and the panel keyed "installed" on that row. Now gated on the agent actually
checking in (reg.last_seen_at). Three states: reporting (metrics + lifecycle),
pending (token minted but no check-in → "no metrics yet, deploy may be
running/failed" banner + retry + Clear pending registration), and none
(install path).
2. The run reported success though nothing ran — ansible-playbook exits 0 on
"no hosts matched"/empty inventory. The executor now treats an empty PLAY
RECAP (returncode 0 but no hosts executed) as failed, with a clear failure
note. Non-zero exits and recap failed/unreachable were already caught.
Scribe issue #887.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
generate_inventory() emits Ansible's dynamic --list shape (all.hosts is a
LIST, vars under _meta) — valid only as an executable inventory script's
stdout. We were writing it to a static file and passing -i, so Ansible's yaml
plugin rejected it ("Invalid 'hosts' entry for 'all' group, requires a
dictionary, found ...list...") and fell back to implicit localhost → "no hosts
matched". Affected every steward:* scope run; surfaced on the first real
provision.
- New inventory_to_yaml(inv): convert the --list dict → a valid static YAML
inventory (all.hosts dict keyed by host, groups under all.children, group
vars preserved, injected per-host vars like steward_token retained).
- Wire it into runner.trigger_run, host_agent deploy + provision.
- executor writes the file as inventory.yml so the yaml plugin's extension
check reliably claims it.
- generate_inventory unchanged (still the --list dict); conversion happens at
write time. Unit tests added.
Scribe issue #885.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Provisioning review corrections + the matching frontend, plus breadcrumb
header integration.
- provision.yml: authorize the managed pubkey with a regexp match on the
' steward-managed' comment so rotating the key REPLACES the host's steward
key in place instead of stacking a second authorized entry. Hand-added keys
(other comments) are untouched.
- update.yml (new): refresh agent.py + restart only. Does NOT rotate the token
or rewrite /etc/steward-agent.conf — the host keeps its identity. Asserts the
agent is already installed and fails clearly otherwise.
- host_agent /update route: runs update.yml as the managed steward user (no
token minting). Token rotation stays a deliberate action.
- settings/ansible/generate-key honors a safe relative `next` redirect, so an
inline trigger elsewhere returns to its page.
- Host Agents settings: reworked into a clear lifecycle — an intro card that
explains it runs Ansible to deploy the agent (+ inline "generate managed key"
warning/trigger when none exists), then three labelled cards: 1 Provision,
2 Install/enroll, 3 Update. Each explains what it does.
- base.html: breadcrumb now renders as a kicker line directly above the page
title (moved below alerts, tightened margin) so nested and top-level views
share one consistent header.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When you click Run, Steward now parses the playbook and renders a field
for each declared variable instead of a blank extra-vars textarea. The
form loads on demand via HTMX (/ansible/run-form/<source>/<playbook>).
- sources.discover_playbook_variables: parse vars: defaults + vars_prompt:
(vars_prompt wins on name collision; non-scalar vars skipped; role/include
vars not traversed). Flags secret-looking names + vars_prompt private.
- Run-time values flow through a JSON extra-vars file (-e @file), which is
space/quote-safe — fixes a latent shlex-split bug in the old -e key=value
textarea path. executor.build_extra_vars_file (pure) + start_run merge.
- Secret-flagged fields are masked AND routed through an unpersisted
secret_vars channel (runner.trigger_run → start_run), so passwords entered
at run time never land in the DB / run history.
- Defaults shown as placeholders (not prefilled): an untouched field falls
through to the inventory/play default instead of overriding it.
- routes: run_form HTMX endpoint; _parse_run_params now returns
(params, secret_vars, err) and reads var__/secret__ fields. Schedules drop
secret vars (can't prompt unattended).
- templates: ansible/_run_form.html fragment; browse.html rewired to HTMX,
static JS run-form removed. Advanced section keeps limit/tags/check + a
free-form extra-vars escape hatch.
- tests: test_playbook_variables.py (discovery + extra-vars file).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Turn the agent-install playbook into a full provisioning + maintenance
path. Solves the bootstrap chicken-and-egg: first contact uses an
operator-supplied password (one run, never stored), which creates a
dedicated `steward` login account with NOPASSWD sudo + Steward's managed
public key. Every run thereafter connects as `steward` with the managed
key — fully unattended (scheduled prune, agent updates).
- core/crypto: generate_ssh_keypair() — ed25519, OpenSSH formats.
- settings: ansible.ssh_public_key (non-secret, displayed) + ansible.ssh_user
(default steward); to_ansible_cfg extended.
- settings UI + route: "Generate managed key" (private encrypted, public
shown to copy) + SSH-user field.
- executor: build_bootstrap() writes a 0600 vars file (-e @file) for the
per-run user/password — never argv, never DB, never logged; drops the
managed key when a bootstrap password is given; --user floor from the
global ssh_user when no override.
- runner.trigger_run: pass-through `connection` kwarg, deliberately NOT
persisted on AnsibleRun.params (password stays out of the DB).
- bundled/host_agent/provision.yml: create steward user + authorized_keys
+ /etc/sudoers.d/steward (visudo-validated) + agent install.
- host_agent: /provision route + "Provision a fresh host" card (bootstrap
user/password; injects pubkey + user + token as space-safe JSON hostvars).
- Dockerfile: add sshpass (Ansible shells out to it for password SSH).
- tests: keypair generation + build_bootstrap (secret stays off argv).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes#550 (all four):
- Cancellation: track live subprocesses; POST /ansible/runs/<id>/cancel
(operator) SIGTERMs then SIGKILLs after a grace; new 'cancelled' status
(+ migration 0019, ALTER TYPE in autocommit). Queued runs cancel cleanly
before launch. Cancel button on run detail.
- Concurrency: global semaphore (ansible.max_concurrent_runs, default 3,
Settings→Ansible) caps simultaneous runs; excess show 'queued' (new status)
until a slot frees. Semaphore bound lazily per running loop.
- Structured results: parse PLAY RECAP into per-host ok/changed/unreachable/
failed/skipped + capture failed-task lines, stored in new results JSON
column (migration 0020); rendered as a host-summary table on run detail.
Keeps live streaming (no json-callback swap).
- Retention: full output written to a persistent log artifact
(/data/ansible/runs/<id>.log, env-overridable) beyond the 1 MB DB cap and
across restarts; in-memory replay buffer bounded + GC'd after completion;
Download-log route. Boot reconciliation now also sweeps stale 'queued'.
Unit tests for recap parsing + cancel flagging. Status colors updated across
run list / detail / schedules.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds create/edit/delete of playbooks from the UI (admin only), so a homelab
user without a git workflow can author automation in-app. A new always-present
writable local source "steward-local" (/data/ansible/playbooks, env-overridable,
created on first save) is editable alongside operator local-dir sources; the
bundled and git sources stay read-only (git is GitOps, clobbered on pull).
sources.py: write_playbook / delete_playbook (traversal-guarded, .yml/.yaml
only) + validate_playbook_yaml (YAML + play-list check) + is_editable_source.
routes.py: /playbooks/new, /edit, /save, /delete (admin). Browse gains a
"New playbook" button and per-playbook + view-page Edit/Delete for editable
sources. Plain textarea editor with save-time YAML validation. Unit tests for
write/delete/guard/validate.
Task #579 — completes milestone #37 (Ansible automation).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements #253's framework: a small core capability registry
(steward/core/capabilities.py) where a module/plugin publishes a named,
role-gated action and a consumer discovers it via has_capability() and runs it
via invoke_capability() — no hard import, graceful degradation, permission
propagation (actor role checked against the capability's required_role).
Core publishes "ansible.run_playbook" (operator) wrapping ansible.runner.
trigger_run (extended to accept a caller-built inventory). First consumer: the
host_agent plugin gains "Deploy via Ansible" on its settings page — pick an
inventory target/group and it installs/updates the agent via the bundled
host_agent/install.yml, minting a fresh token per host and injecting it as an
inventory hostvar (turning per-host curl|sh into one run). Exposed role
ordering as middleware.role_meets. Unit tests for the registry + role checks.
Task #253 (milestone #37).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ships a read-only "steward-builtin" source (steward/ansible/bundled/) that
always appears alongside operator-configured sources, with two playbooks:
- maintenance/docker_prune.yml — docker system prune for swarm/standalone
nodes (safe by default; prune_all_images / prune_volumes extra-vars to
widen). Schedule it against a swarm-node group for recurring cleanup (#869).
- host_agent/install.yml — installs/updates the host agent (mirrors
install.sh: user, agent.py, config, hardened systemd unit), parameterised
with steward_url + steward_token extra-vars.
get_sources() now prepends the builtin source. Tests updated to find the
configured git source by name; added coverage that the bundled playbooks are
discoverable.
Tasks #869 + agent-install (milestone #37).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds cron-like recurring runs (the engine the maintenance-automation work
needs). New AnsibleSchedule model + migration 0018; a core ScheduledTask
(ansible_scheduled_runs, 60s) fires due schedules, each creating a
system-triggered AnsibleRun (triggered_by=None). Centralises the
resolve-inventory → create-run → launch flow in ansible/runner.trigger_run,
shared by the manual route (refactored to use it) and the scheduler.
Schedules UI under /ansible/schedules: create/edit/pause/delete/run-now,
with interval presets, scope targeting (all / group / target), extra-vars /
limit / tags / dry-run, and last-run status (resolved via last_run_id) +
next-run. Unit test for the due-check.
Task #549 (milestone #37).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
From a host's edit page, an operator can run a playbook against just that host
via an ephemeral one-host inventory — no need for the host to exist in a source
inventory.
- ansible/sources.py: pure host_inventory_content(host) -> '<name> ansible_host=<addr>'
- executor.start_run: optional inventory_content written to the temp dir and used
as -i (overrides inventory_path); cleaned up with the creds dir
- hosts/routes.py: POST /hosts/<id>/run-playbook (operator) — validates source +
playbook, builds the ephemeral inventory, starts a user-triggered run, redirects
to the live run detail; edit page gets the ansible source list
- hosts/form.html: 'Run Ansible playbook against this host' panel (shown when
editing an existing host and sources exist) — source + playbook + extra-vars/
tags/dry-run (no --limit; single host)
- tests: unit host_inventory_content; integration runs a hosts:all/connection:local
playbook against the ephemeral inventory and asserts inventory_hostname
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Global Ansible credentials, applied to every run (manual + alert-triggered):
- core/settings.py: ansible.ssh_private_key / become_password / vault_password
(plaintext at rest, masked in UI — encryption tracked in #580) + host_key_checking
(default off); surfaced via to_ansible_cfg into app.config[ANSIBLE]
- executor.py: pure build_credentials() materializes creds into a 0600 temp dir
(--private-key, --vault-password-file, become via -e @vars-file so the password
never hits argv) cleaned up in finally; pure ansible_env() sets
ANSIBLE_HOST_KEY_CHECKING. build_ansible_command stays param-only
- settings/routes.py + ansible.html: admin-only Credentials section, masked-update
(blank keeps current, explicit Clear checkbox), reload app config on save
- tests: unit (build_credentials per cred + none; ansible_env toggle); integration
vault round-trip (ansible-vault encrypt a vars file, run via executor with the
vault password, assert it decrypted)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Run form + executor now support optional params, passed as argv (never
shell-interpolated):
- extra_vars: one key=value per line -> repeated -e
- limit -> --limit; tags -> --tags
- dry-run checkbox -> --check --diff
- executor.py: pure build_ansible_command(playbook, inventory, params); start_run
gains an optional params arg (backward-compatible)
- models/ansible.py + migration 0013: nullable params JSON column
- routes.py: create_run parses + validates (extra-var lines need '='), stores
params on the run, passes to the executor
- browse.html run form: Extra vars / Limit / Tags / Dry-run fields
- run_detail.html: shows the params used
- tests/test_ansible_command.py: unit coverage of the builder; integration test
runs a real playbook with extra-var + limit + check and asserts substitution
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Foundation for the Ansible automation milestone (#37) — makes the existing
manual playbook runner actually executable and the schema automation-ready.
- pyproject: [ansible] extra (full ansible package, batteries-included, pinned)
- Dockerfile: pip install .[ansible]; add openssh-client for remote runs
- models/ansible.py + migration 0012: AnsibleRun.triggered_by now nullable so
automated (alert/schedule) runs need no human actor
- ansible/routes.py + run_detail.html: show 'Triggered by' (username or 'system')
- CI: integration lane installs .[dev,ansible]; new tests/integration/
test_ansible_foundation.py runs a real connection:local playbook end-to-end,
asserts success+output, and round-trips a NULL-triggered run
No extra-vars/limit/credentials/scheduling here — those are their own #37 tasks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The fabledscryer->steward rename had only ever reached host_agent. The other
five bundled plugins (http, snmp, traefik, unifi, docker) still imported
`from fabledscryer.*` (package no longer exists) and read FABLEDSCRYER_* env
vars — so every one of them was broken at import since the original rebrand.
CI stayed green only because none are enabled by default and migrations don't
import plugin modules. Now that they version in-tree, complete the rename:
- fabledscryer.* -> steward.* imports across all five plugins
- FABLEDSCRYER_* -> STEWARD_* in plugin migration env.py files
- author/repository/homepage + user-facing 'Fabled Scryer' strings -> Steward
- snmp/scheduler.py: also drop dead `now`/datetime; record_metric from steward
Adds tests/test_no_legacy_names.py — fails if 'scryer'/'roundtable' ever
reappear in shipped code (the drift bit twice; this stops a third time).
Also clears pre-existing ruff lint debt (unused imports, semicolon statements,
mid-file import) surfaced by the new lint lane.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Renames the Python package directory, CLI command, env var prefix,
docker-compose service/container/image, Postgres role/db, and all
visible branding. Marketing form is "Fabled Steward".
Clean break from the previous rebrand: drops the fabledscryer→roundtable
import shim in __init__.py and the FABLEDSCRYER_* env var fallback in
config.py and migrations/env.py. Env vars are now STEWARD_* only.
Heads-up for existing deployments:
- Postgres user/db renamed fabledscryer → steward in docker-compose.yml.
Existing volumes need the role/db renamed inside Postgres, or override
POSTGRES_USER/POSTGRES_DB to keep the old names.
- Host-agent systemd unit is now steward-agent.service. Existing agents
keep running under the old name; reinstall to switch.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
