bvandeusenandClaude Opus 5.5
22b0be2b6c
#5767 "Steward hands a host a maintenance job and follows it to the end, through its own restarts", milestone 514. A host_maintenance_jobs row per job: queued → updating → [draining →] rebooting → [undraining →] done, or failed / timed_out. Steward's answer to an ingest post carries the step that host's agent should carry out now; the agent (1.18.0) leaves each step id for the root helper once, and the helper refuses an id it already ran, so repeats are harmless. Results come back as the #5766 maintenance events, matched by step id; the reboot completes when the host's `started` event has a boot_at after the reboot was handed out. The job reboots only when the update says one is needed (or never, if asked). A swarm node is drained first by itself if it's a manager or by a manager with the helper, and made active after; with no manager to drain it the job is refused. A minute tick times out a job past its state's deadline. A maintenance window over host_agent/<host> lasts the rest of the job. Everything is on the row, so a Steward restart, or the reboot of the host Steward runs on, resumes from the next post. Ingest runs it in its own savepoint so a job going wrong never costs the samples. The Agent panel leads with a running job and says what a finished one did; Manage agent has Update now (with "never reboot"), or says why a job can't start. Integration test: hand off, Steward restarts, the post-boot events close the job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
223 lines
13 KiB
HTML
223 lines
13 KiB
HTML
{# Per-host agent panel — embedded into /hosts/<id> via HTMX. Self-contained. #}
|
|
{% set scope = "steward:target:" ~ target.id if target else "" %}
|
|
<div class="card">
|
|
<div style="display:flex;align-items:baseline;justify-content:space-between;margin-bottom:0.75rem;gap:1rem;flex-wrap:wrap;">
|
|
<h3 class="section-title" style="margin-bottom:0;">Agent</h3>
|
|
{% if reporting %}
|
|
<span style="font-size:0.78rem;" class="tone-{{ 'warn' if stale else 'ok' }}">
|
|
{{ 'stale' if stale else 'reporting' }}{% if reg.agent_version %} · v{{ reg.agent_version }}{% endif %}
|
|
· last seen {{ reg.last_seen_at|ago }}
|
|
</span>
|
|
{% elif reg %}
|
|
<span style="font-size:0.78rem;color:var(--text-muted);">pending — no check-in yet</span>
|
|
{% endif %}
|
|
</div>
|
|
|
|
{% if reporting %}
|
|
{# Reporting: live vitals are shown by the vitals strip above; this panel is
|
|
the way to the full metrics, with the lifecycle actions folded under
|
|
"Manage agent" (#5278). #}
|
|
<a href="/plugins/host_agent/{{ host.id }}/" class="btn btn-sm btn-ghost">Full metrics →</a>
|
|
|
|
{# What the host has waiting, from the agent's hourly check (#5765): a
|
|
needed reboot first, then the updates. #}
|
|
{% if reg.updates_checked_at or reg.reboot_required or maint_job %}
|
|
<div style="margin-top:0.8rem;font-size:0.82rem;display:flex;flex-direction:column;gap:0.2rem;">
|
|
{# Steward's own maintenance job on this host (#5767): a running one first. #}
|
|
{% if maint_job %}
|
|
{% set running = maint_job.state not in ('done', 'failed', 'timed_out') %}
|
|
<div class="{{ 'tone-warn' if running or maint_job.state != 'done' else '' }}" title="{{ maint_job.detail or '' }}">
|
|
Maintenance: {{ maint_summary }}
|
|
<span style="color:var(--text-muted);">· {% if running %}since {{ maint_job.state_at|ago }}{% else %}{{ maint_job.finished_at|ago }}{% endif %}{% if maint_job.state in ('failed', 'timed_out') and maint_job.detail %} · {{ maint_job.detail|truncate(90) }}{% endif %}</span>
|
|
</div>
|
|
{% endif %}
|
|
{% if reg.reboot_required %}
|
|
<div class="tone-warn" title="{{ reg.reboot_packages or '' }}">
|
|
Reboot required{% if reg.reboot_required_since %} since {{ reg.reboot_required_since|ago }}{% endif %}{% if reg.reboot_packages %}
|
|
<span style="color:var(--text-muted);">· {{ reg.reboot_packages|truncate(80) }}</span>{% endif %}
|
|
</div>
|
|
{% endif %}
|
|
{% if reg.updates_checked_at %}
|
|
<div>
|
|
{% if reg.updates_error %}<span class="tone-warn">Update check failed: {{ reg.updates_error }}</span>
|
|
{% elif reg.updates_pending %}<span class="{{ 'tone-warn' if reg.updates_security else '' }}">{{ reg.updates_pending }} update{{ 's' if reg.updates_pending != 1 }} waiting{% if reg.updates_security %} ({{ reg.updates_security }} security){% endif %}</span>
|
|
{% else %}Up to date{% endif %}
|
|
<span style="color:var(--text-muted);">· checked {{ reg.updates_checked_at|ago }}</span>
|
|
</div>
|
|
{% endif %}
|
|
</div>
|
|
{% endif %}
|
|
|
|
{# What the agent said happened while it was away, last 7 days (#5756). #}
|
|
{% if downtime %}
|
|
<div style="margin-top:0.8rem;">
|
|
<div style="font-size:0.78rem;color:var(--text-muted);margin-bottom:0.3rem;">Downtime, last 7 days</div>
|
|
{% for d in downtime %}
|
|
<div style="display:flex;gap:0.6rem;align-items:baseline;font-size:0.82rem;padding:0.15rem 0;">
|
|
<span class="{{ 'tone-warn' if d.ongoing or d.kind == 'unclean' else '' }}" style="min-width:12rem;">{{ d.label }}</span>
|
|
<span style="color:var(--text-muted);">{{ d.started_at|when }}</span>
|
|
<span style="color:var(--text-muted);margin-left:auto;">{{ 'going on, ' if d.ongoing }}{{ d.seconds|duration }}</span>
|
|
</div>
|
|
{% endfor %}
|
|
</div>
|
|
{% endif %}
|
|
|
|
{% if session.user_role == 'admin' %}
|
|
<details style="margin-top:0.7rem;">
|
|
<summary style="cursor:pointer;font-size:0.82rem;color:var(--text-muted);">Manage agent</summary>
|
|
<div style="display:flex;gap:0.5rem;flex-wrap:wrap;align-items:center;margin-top:0.6rem;">
|
|
{% if ansible_available and target %}
|
|
<form method="post" action="/plugins/host_agent/update" style="margin:0;">{{ csrf_field() }}
|
|
<input type="hidden" name="inventory_scope" value="{{ scope }}">
|
|
<button type="submit" class="btn btn-sm" title="Refresh agent.py and restart it; the token is kept">Update agent</button>
|
|
</form>
|
|
{% endif %}
|
|
<form method="post" action="/plugins/host_agent/fleet/{{ host.id }}/rotate-token" style="margin:0;"
|
|
data-confirm="Rotate token? The agent stops reporting until reinstalled/updated with the new token.">{{ csrf_field() }}
|
|
<button type="submit" class="btn btn-sm btn-ghost">Rotate token</button>
|
|
</form>
|
|
<form method="post" action="/plugins/host_agent/fleet/{{ host.id }}/delete" style="margin:0;"
|
|
data-confirm="Unregister this host's agent? Its token stops working and metrics stop until it is registered again. The agent itself stays installed on the host.">{{ csrf_field() }}
|
|
<button type="submit" class="btn btn-sm btn-danger">Unregister agent</button>
|
|
</form>
|
|
</div>
|
|
{% if ansible_available and not target %}
|
|
<p style="color:var(--text-dim);font-size:0.8rem;margin:0.6rem 0 0;">
|
|
Link an Ansible target (below) to update the agent from here.
|
|
</p>
|
|
{% endif %}
|
|
{# Update now (#5767): the agent's helper updates the host and reboots it
|
|
only if the update needs it, draining its swarm node first. #}
|
|
{% if maint_blocker %}
|
|
<p style="font-size:0.8rem;margin:0.6rem 0 0;color:var(--text-dim);">Can't update from here: {{ maint_blocker }}.</p>
|
|
{% else %}
|
|
<form method="post" action="/plugins/host_agent/fleet/{{ host.id }}/maintain"
|
|
style="display:flex;gap:0.6rem;align-items:center;flex-wrap:wrap;margin:0.6rem 0 0;"
|
|
data-confirm="Update {{ host.name }} now? It reboots only if the update needs it, after its swarm node is drained.">{{ csrf_field() }}
|
|
<button type="submit" class="btn btn-sm">Update now</button>
|
|
<label style="font-size:0.8rem;color:var(--text-muted);display:flex;gap:0.3rem;align-items:center;">
|
|
<input type="checkbox" name="reboot_policy" value="never"> never reboot
|
|
</label>
|
|
</form>
|
|
{% endif %}
|
|
{# The root helper that carries out scheduled updates and reboots (#5766). #}
|
|
<p style="font-size:0.8rem;margin:0.6rem 0 0;color:var(--text-dim);">
|
|
{% if reg.maint_helper %}Maintenance helper installed: Steward can update and reboot this host.
|
|
{% else %}No maintenance helper yet: Update agent installs it, so Steward can update and reboot this host.{% endif %}
|
|
</p>
|
|
|
|
{% if ansible_available and target and managed_key_set %}
|
|
<details style="margin-top:0.7rem;">
|
|
<summary style="cursor:pointer;font-size:0.8rem;color:var(--text-muted);">Re-provision (reinstall the steward account + managed key, then the agent)</summary>
|
|
<p style="font-size:0.78rem;color:var(--text-dim);margin:0.5rem 0;">
|
|
Use after regenerating the managed key, or if SSH auth as <code>steward</code> breaks.
|
|
Connects over a one-time bootstrap user + password (not stored).
|
|
</p>
|
|
<form method="post" action="/plugins/host_agent/provision"
|
|
style="display:flex;gap:0.6rem;align-items:flex-end;flex-wrap:wrap;">{{ csrf_field() }}
|
|
<input type="hidden" name="inventory_scope" value="{{ scope }}">
|
|
<div class="form-group" style="margin:0;">
|
|
<label>Bootstrap user</label>
|
|
<input type="text" name="bootstrap_user" required placeholder="root" style="width:8rem;" autocomplete="off">
|
|
</div>
|
|
<div class="form-group" style="margin:0;">
|
|
<label>Bootstrap password</label>
|
|
<input type="password" name="bootstrap_password" required style="width:10rem;" autocomplete="new-password">
|
|
</div>
|
|
<button type="submit" class="btn btn-sm">Re-provision</button>
|
|
</form>
|
|
</details>
|
|
{% endif %}
|
|
</details>
|
|
{% endif %}
|
|
|
|
{% else %}
|
|
{# ── Not reporting: pending (token minted, no check-in) or never installed ── #}
|
|
{% if reg %}
|
|
<div style="background:color-mix(in srgb,var(--yellow) 10%,var(--bg-elevated));
|
|
border:1px solid color-mix(in srgb,var(--yellow) 30%,var(--border));
|
|
border-radius:6px;padding:0.7rem 0.9rem;margin-bottom:0.75rem;font-size:0.84rem;">
|
|
A token was minted for this host but <strong>no metrics have arrived yet</strong> — the
|
|
deploy may still be running, or it failed. Open the latest
|
|
<a href="/ansible/">Ansible run</a> to check, then retry below. Live metrics appear here
|
|
once the agent checks in.
|
|
</div>
|
|
{% else %}
|
|
<p style="color:var(--text-muted);font-size:0.88rem;margin-bottom:0.75rem;">
|
|
No agent installed. The agent reports CPU, memory, disk, network and more back to Steward.
|
|
</p>
|
|
{% endif %}
|
|
|
|
{% if session.user_role != 'admin' %}
|
|
<p style="color:var(--text-dim);font-size:0.85rem;">An admin can install the agent here.</p>
|
|
|
|
{% elif not ansible_available %}
|
|
<p style="color:var(--text-dim);font-size:0.85rem;">
|
|
Ansible isn't available, so the agent can't be deployed from here. Install it manually instead (below).
|
|
</p>
|
|
|
|
{% elif not managed_key_set %}
|
|
<div style="background:color-mix(in srgb,var(--yellow) 12%,var(--bg-elevated));
|
|
border:1px solid color-mix(in srgb,var(--yellow) 35%,var(--border));
|
|
border-radius:6px;padding:0.7rem 0.9rem;display:flex;align-items:center;gap:0.8rem;flex-wrap:wrap;">
|
|
<span style="color:var(--yellow);">⚠</span>
|
|
<span style="font-size:0.84rem;flex:1;min-width:13rem;">No managed SSH key yet — Steward needs one to log into hosts.</span>
|
|
<form method="post" action="/settings/ansible/generate-key" style="margin:0;">{{ csrf_field() }}
|
|
<input type="hidden" name="next" value="/hosts/{{ host.id }}">
|
|
<button type="submit" class="btn btn-sm">Generate managed key</button>
|
|
</form>
|
|
</div>
|
|
|
|
{% elif not target %}
|
|
<p style="color:var(--text-dim);font-size:0.85rem;">
|
|
Link or create an Ansible target for this host (in the Ansible section below) first — provisioning
|
|
needs an SSH connection.
|
|
</p>
|
|
|
|
{% else %}
|
|
<p style="font-size:0.82rem;color:var(--text-muted);margin-bottom:0.75rem;">
|
|
Deploys the agent to <code>{{ target.name }}</code> by running an Ansible playbook. Steward mints
|
|
the host's API token automatically; you'll watch the run live.
|
|
</p>
|
|
{# Provision: brand-new host (creates steward account + key over a one-time password) #}
|
|
<form method="post" action="/plugins/host_agent/provision"
|
|
style="display:flex;gap:0.6rem;align-items:flex-end;flex-wrap:wrap;margin-bottom:0.75rem;">{{ csrf_field() }}
|
|
<input type="hidden" name="inventory_scope" value="{{ scope }}">
|
|
<div style="font-size:0.78rem;color:var(--text-muted);width:100%;">Provision (first contact — fresh host):</div>
|
|
<div class="form-group" style="margin:0;">
|
|
<label>Bootstrap user</label>
|
|
<input type="text" name="bootstrap_user" required placeholder="root" style="width:8rem;" autocomplete="off">
|
|
</div>
|
|
<div class="form-group" style="margin:0;">
|
|
<label>Bootstrap password</label>
|
|
<input type="password" name="bootstrap_password" required style="width:10rem;" autocomplete="new-password">
|
|
</div>
|
|
<button type="submit" class="btn btn-sm">Provision</button>
|
|
</form>
|
|
{# Install: host already has the steward account (managed key works) #}
|
|
<form method="post" action="/plugins/host_agent/deploy" style="display:flex;gap:0.6rem;align-items:center;">{{ csrf_field() }}
|
|
<input type="hidden" name="inventory_scope" value="{{ scope }}">
|
|
<span style="font-size:0.78rem;color:var(--text-muted);">Already provisioned?</span>
|
|
<button type="submit" class="btn btn-sm btn-ghost">Install agent (managed key)</button>
|
|
</form>
|
|
{% endif %}
|
|
|
|
{% if not reg and session.user_role == 'admin' %}
|
|
{# A manual install: no Ansible, no SSH. Issues this host's token and shows
|
|
the one-line install command on the fleet page, once (#5282). #}
|
|
<form method="post" action="/plugins/host_agent/fleet/{{ host.id }}/register"
|
|
style="display:flex;gap:0.6rem;align-items:center;flex-wrap:wrap;margin:0.75rem 0 0;">{{ csrf_field() }}
|
|
<span style="font-size:0.78rem;color:var(--text-muted);">Or by hand on the host:</span>
|
|
<button type="submit" class="btn btn-sm btn-ghost">Install manually</button>
|
|
</form>
|
|
{% endif %}
|
|
|
|
{% if reg and session.user_role == 'admin' %}
|
|
<form method="post" action="/plugins/host_agent/fleet/{{ host.id }}/delete" style="margin:0.75rem 0 0;"
|
|
data-confirm="Clear this pending registration? Its token is discarded.">{{ csrf_field() }}
|
|
<button type="submit" class="btn btn-sm btn-ghost">Clear pending registration</button>
|
|
</form>
|
|
{% endif %}
|
|
{% endif %}
|
|
</div>
|