feat(ansible): runner robustness — cancel, concurrency, structured results, retention
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 55s

Closes #550 (all four):

- Cancellation: track live subprocesses; POST /ansible/runs/<id>/cancel
  (operator) SIGTERMs then SIGKILLs after a grace; new 'cancelled' status
  (+ migration 0019, ALTER TYPE in autocommit). Queued runs cancel cleanly
  before launch. Cancel button on run detail.
- Concurrency: global semaphore (ansible.max_concurrent_runs, default 3,
  Settings→Ansible) caps simultaneous runs; excess show 'queued' (new status)
  until a slot frees. Semaphore bound lazily per running loop.
- Structured results: parse PLAY RECAP into per-host ok/changed/unreachable/
  failed/skipped + capture failed-task lines, stored in new results JSON
  column (migration 0020); rendered as a host-summary table on run detail.
  Keeps live streaming (no json-callback swap).
- Retention: full output written to a persistent log artifact
  (/data/ansible/runs/<id>.log, env-overridable) beyond the 1 MB DB cap and
  across restarts; in-memory replay buffer bounded + GC'd after completion;
  Download-log route. Boot reconciliation now also sweeps stale 'queued'.

Unit tests for recap parsing + cancel flagging. Status colors updated across
run list / detail / schedules.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-16 14:51:04 -04:00
parent 389002fc6f
commit 88857be24e
13 changed files with 406 additions and 61 deletions
+39 -1
View File
@@ -166,7 +166,7 @@ async def stream_run(run_id: str):
return "Not found", 404
async def generate():
if run.status != AnsibleRunStatus.running:
if run.status not in (AnsibleRunStatus.running, AnsibleRunStatus.queued):
yield f"event: done\ndata: {run.status.value}\n\n"
return
@@ -208,6 +208,44 @@ async def run_detail(run_id: str):
"ansible/run_detail.html", run=run, triggered_label=triggered_label)
@ansible_bp.post("/runs/<run_id>/cancel")
@require_role(UserRole.operator)
async def cancel_run(run_id: str):
async with current_app.db_sessionmaker() as db:
run = (await db.execute(
select(AnsibleRun).where(AnsibleRun.id == run_id))).scalar_one_or_none()
if run is None:
return "Not found", 404
if run.status in (AnsibleRunStatus.running, AnsibleRunStatus.queued):
executor.cancel_run(run_id)
return redirect(url_for("ansible.run_detail", run_id=run_id))
@ansible_bp.get("/runs/<run_id>/log")
@require_role(UserRole.viewer)
async def run_log(run_id: str):
"""Download the full run log (persistent artifact; falls back to DB output)."""
import os
from steward.ansible.executor import artifact_path
async with current_app.db_sessionmaker() as db:
run = (await db.execute(
select(AnsibleRun).where(AnsibleRun.id == run_id))).scalar_one_or_none()
if run is None:
return "Not found", 404
path = artifact_path(run_id)
if os.path.exists(path):
with open(path, encoding="utf-8", errors="replace") as f:
body = f.read()
else:
body = run.output or ""
return Response(
body, mimetype="text/plain",
headers={"Content-Disposition": f'attachment; filename="ansible-run-{run_id[:8]}.log"'},
)
# ── Scheduled recurring runs ──────────────────────────────────────────────────
@ansible_bp.get("/schedules")