CI 8616: the integration test asserted Vary == 'HX-Request', but the session
layer appends 'Cookie'. resp.vary.add keeps both; the test asserts membership.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5294 "Hosts list: search, group and status filters; sortable columns; a Groups column".
Ansible inventory groups are Steward's grouping system (operator's ruling): a host's
groups are its linked target's, and membership overlaps, so the list filters by group
rather than sectioning by it.
- Filter bar: search on name or address, a group (or "No group"), a status (Down,
Agent silent, Up, No monitors). The table swaps in alone and the URL keeps the filters.
- Name, Status, Latency, 30d uptime, CPU and Memory headers sort (sort_th in _macros);
problems-first stays the default, and a host with no value goes last either way.
- A Groups column of chips, each filtering to its group; shown when any group exists.
- A swap is keyed on HX-Target, so Back to a pushed URL still gets the page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stylesheet's reset (* { margin: 0 }) removed the browser's margin: auto
that centres a modal <dialog>; .modal puts it back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5290 "Host page: the health tiles span the card's width; even spacing above Ansible":
the four tiles take equal quarters (halves on a phone) and their sparklines stretch
across them; the Monitors + Agent row drops the Agent card's own margin, so the gap
above Ansible is 1rem like every other.
#5291 "Add host opens as a dialog over the Hosts list": a native <dialog> on /hosts/,
opened by Add host or /hosts/?add=1. A name clash re-renders the list with it open,
the error and what was typed. GET /hosts/new is gone; hosts/form.html is Edit only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hosts list:
- CPU and Memory are their own labelled columns (#5279); a silent
agent's last readings are greyed, and Status says "agent silent for 7m".
- One uptime figure, 30 days, with 24h and 7d on hover. The SLA tab
(/hosts/uptime) repeated it and is removed (#5280; one home per fact).
- Status is a word with how long ("Down for 12m"); host_status now carries
down_since. Edit and Delete live on the host's page only (#5281).
- Figures are right-aligned with tabular digits (.table .num), not a
monospace face.
Agent fleet (#5282): its add-host form is gone, so hosts are added only on
Hosts -> Add host. That form was also the only way to get a token for a
manual curl install, so that moves to the host's Agent card as "Install
manually" (POST /fleet/<host_id>/register), shown on the fleet page as
before. "Delete" there reads "Unregister".
Add host (#5283): laid out like the add-monitor form; a name clash comes
back inline with what was typed instead of a bare text 400; the address
is optional, as the host page already allowed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docker panel (#5276): problems first, then the 8 busiest running
containers by CPU; the other running and the stopped fold behind one
"+N more running · M stopped" line. Containers that are not swarm tasks
read the way the Docker page reads them: CI jobs as "CI: workflow · job",
buildx builders as "BuildKit builder · name".
Monitors card (#5277): the edit/delete icon buttons and the inline add
form are gone. Each row links to its monitor's page; "Add monitor" opens
the add form filed under this host, starting from its address, with the
host in its breadcrumbs and Cancel back to it.
Agent card (#5278): Full metrics stays in view; Update, Rotate token,
Unregister agent (was "Remove") and Re-provision sit under "Manage agent".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DNS monitors take an optional Resolver, a server's IP. Set, the check asks
that server directly for A and AAAA over UDP and records how long it took
to answer; empty keeps today's lookup through the container's resolver.
"Does my local DNS still answer for public names" and "does 1.1.1.1" are
now separate monitors. No new dependency.
On a host's page the Ansible card moves above the Docker list, which can
run to dozens of containers and hid it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Monitor page: a response-time chart for the last day with failed checks
marked; Edit / Pause / Delete as labelled buttons; the state tile says
"for 29m" instead of "for at least 29m 25s ago"; the check strip fills
its card; no failures is one line, not an empty box; Settings names the
type, target and the actual global interval, with readable field names.
Add monitor: a line under the fields says what the selected type checks
and when it counts as up; Name says it defaults to the target.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Status repeated the Monitors page's counts, check strip and uptime. Monitors
now has one header row and every column Status had: the last 30 checks,
latest response with its trend, uptime over 24h / 7d / 30d (the range picker
goes) and TLS days on every row. /status and /status/rows and the nav link
are removed outright; the dashboard Overview (/status/widget) stays and its
Details link goes to Monitors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A chart built while its cell was still being laid out could stay blank until
a window resize; it now measures and draws again after layout settles. Each
refresh rebuilds the chart (the flash), and its buckets are minutes wide, so
the widget refreshes every 5 minutes instead of every poll.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each figure after its bars takes a fixed width, so memory figures of
different lengths no longer shift their bars. CPU and Memory headers say
"last 6h" and the note gives the slice length.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fill bars were a share of the host's capacity, a sliver on hosts with
80 cores. Each figure now sits beside 24 bars of the last 6 hours (1h/24h
per widget), each row scaled to its own peak but never below 1 core or
1 GiB (configurable per widget), so a 0.05-core container reads flat and
a bar's height means real load. One grouped date_bin query feeds them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
View mode splits each saved row into seven, so a widget no longer carries up
to 69px of empty space below its content (Overview, SNMP). Edit mode and the
saved layout keep 70px rows. The Busiest Containers host column aligns left
like its header, its note fits one line, and the Hosts column reads Disk.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI 8551 integration: "integer out of range" for a 2099 timestamp.
The SQL in the log showed a second problem: `CAST(epoch AS INTEGER) / 60`
compiled to `/ CAST(60 AS NUMERIC)`, i.e. decimal division. Every scrape got
its own "bucket", so the request chart and the Traefik page's sparklines
never averaged anything. The int4 cast would also overflow in 2038.
Both now use date_bin from the epoch, as host_agent/metrics_query.py does.
The integration test adds a second scrape 30 s later in the same minute and
checks that the bucket averages the totals and weights the 5xx share over
both scrapes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5256 "Traefik — Request chart: blank on the live dashboard; time axis and honest labels".
- "Req/s" was the average of the routers, not the total. Each scrape is now
summed over routers, then averaged per bucket (request_history).
- The 5xx share is now weighted by each router's traffic. The axis is no
longer pinned at 0–100 (suggestedMax 5).
- The x axis shows times, in the viewer's local time. The tooltip shows
both series at that time.
- A verdict line comes first: now / peak req/s and the peak 5xx share. The
figures read even if the chart doesn't.
- The blank live chart's cause is still unknown. Chart.js and
chart-colors.js both load on the dashboard. A missing library or a thrown
error is now written into the widget instead of leaving it empty, so the
cause will show on the page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5255 "SNMP — Devices: status in words and key readings".
- Verdict first: "All N devices answering", "ups: no reply since 14m ago",
"N of M devices not answering", or "N not polled yet".
- Devices that stopped answering come first, then those never polled, then
the rest.
- One header row: Device / Readings / Polled.
- The name is the link; the separate "detail →" is gone.
- The hollow dot is replaced by words: "no data yet", or "no reply since …;
readings are from then".
- Key readings: the first two in config order, gauges ahead of counters.
Units come from the label suffix: _centisecs as a duration, _pct, _dV as
volts, _octets as bytes total.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI 8548: '<th' also matched '<thead', and the certificate test's clock was
read before its certificates were built, so 10 days came out as 10.00…
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5254 "Traefik — Routers: current routers only, header row, certs on the router they serve".
- Router hosts come from Traefik's API (/api/http|tcp/routers), read
best-effort on each scrape. An optional api_url overrides it; the default
is /api on the metrics host. A certificate's expiry then shows in a TLS
column on the router whose host it covers. An expiring one makes that
router a problem, graded by Settings → Thresholds.
- Only current: routers and certificates must have appeared in the last
three scrapes. Certificates are upserted by serial and never deleted, so
renewed or retired ones lingered on the widget.
- Widget order: verdict, problem routers with the reason, then the busiest
healthy routers under one header (Router / Req/s / Bandwidth / p95 / TLS).
A missing value shows "—".
- If the API is unreachable, expiring certificates are listed after the
routers, not above them.
- The page splits certificates into served (with the routers using each,
plus SANs) and "No longer served".
- Latest row per router: an index walk (recursive CTE + LATERAL) replaces
DISTINCT plus a query per router.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The widget names the busiest container in a verdict ("Busiest: tdarr on
VDNT-Docker03, 1.6 of 8 cores."), then a fixed-layout table (container,
host, CPU in cores, memory in bytes) with each figure's share drawn as
shading behind it: of the host's cores, and of the memory limit or the
host's memory.
The numbers were already rendered, but each row grew as wide as its longest
name. A grid item won't shrink below its content, so Gitea CI container
names pushed the figures out of view and the cell scrolled sideways. Names
now come from display_name(): a swarm task by its service (no slot number),
an Actions job container as "CI: <workflow> · <job>". They cut off with …
and show whole on hover.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A verdict line (services healthy · other containers running · failed in
24h), then "Needs a look": troubled services with why (crash-looping, N of
M running), since when and the last error, and troubled containers
(unhealthy, restarting, exited with an error, out of memory). Then "Failed
in the last 24h" by name, with how often, when and where. Then the healthy
rest one line each (name · hosts · replicas) in as many columns as the
widget is wide, collapsing after 40 to "+N more healthy".
Per-container CPU is gone from this widget: Busiest Containers owns it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Hosts — Overview widget opens with a verdict ("VDNT-Util01: CPU high."
or "All 3 hosts healthy."), then one row per host under one header: CPU,
memory, disk / and load. Rows sort by host_glance: a down monitor, a
reading past critical, a quiet agent, a warning, then the healthy. Load
reads "16.8 / 8 cores", graded per core as Settings → Thresholds intends
(cores counted from the per-core CPU sub-resources, as on the host page).
A quiet agent's last figures are shown uncoloured, with when it last reported.
Ping and 24h uptime are gone from this widget: they belong to Monitors. So
are the unlabelled CPU sparkline and the 30-day uptime query it no longer
needs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Monitors widget opens with a verdict ("1 of 5 monitors down."), then one
row per monitor under a single header: latency and uptime over 24h, 7d and
30d, down monitors first with why and since when, the name linking to the
monitor. Counts follow the type filter. Problems always show; the limit
trims only the healthy rest ("+N more up").
Uptime / SLA (uptime_summary) repeated those numbers per host; migration
0033 moves dashboards onto Monitors. Shared widget styles (.w-verdict,
.w-table, .w-sub, .w-more) carry the widget-shape ruling for the widgets
that follow.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each widget is as tall as its content, never taller than its saved height
(where it scrolls). The dashboard is capped at 2400px instead of 1400px.
Past 12 columns' worth of width it adds columns (one per 110px, up to 24)
and places widgets again in reading order at their saved widths, so more sit
side by side instead of stretching. Edit mode shows and saves the
12-column layout, so a layout doesn't depend on the screen it was edited on.
The share view uses the same layout code (static/dash-layout.js).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New installs start from widgets.RECOMMENDED, packed row by row for the
enabled plugins; the edit drawer can reset a dashboard to it.
- Host Agent — Resources folds into Hosts — Overview (which gains load 1m),
UniFi — Devices into UniFi — Overview; migration 0032 moves existing
dashboards onto the survivors.
- The Overview labels what it counts (hosts, monitors up/down/pending,
alerts firing) and lists firing alerts and down monitors with since-times.
- Uptime polls every 5 minutes (poll_seconds) instead of every cycle.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5212 "Plain labels: no raw ids, scopes, metric keys or unexplained jargon",
batch 3:
- Disk page: each tile says what it holds. "Writable layers" becomes
"Container files" and Build cache says it is safe to clear. Shared is
"Shared with other images" with a tooltip, and "dangling" is gone.
Untagged images show by id instead of "<none>", a capped list says
"The 50 largest of N images", and each host shows "as of" its report.
- Traefik: an expired certificate reads "expired 12 days ago", not "-12
days" (cert_left). The Traffic empty state knows whether the access log
is on: off points to Settings → Plugins → Traefik; on says to check the
mounted log path.
- Moments use |ago with an exact UTC tooltip: UniFi last seen, alarms,
events, speed test and DPI; AI token last used.
- Settings: the Auth tab is "Authentication" like its breadcrumb, plugin
fields read "Metrics URL" not metrics_url (field_label), a source's pull
interval is "every 1h", and the last report time drops the ISO "T".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5212 "Plain labels: no raw ids, scopes, metric keys or unexplained jargon",
batch 2:
- A run records where it came from: AnsibleRun.origin ("schedule", "alert"
or "mcp") and origin_id, set by the scheduler, Run now, alert actions and
the MCP tools. Migration 0031 backfills past runs from alert events,
schedules' last runs, and MCP audit entries.
- The runs list has a Started by column: "schedule nightly", "alert rule
nas down", "alice via MCP" or "alice", linked where there is a page.
The raw scope tooltip is gone.
- Run detail: no run id in the subtitle or body, Started by in words, and
the host summary lists failed and unreachable hosts first. The live
output only follows new lines while the reader is at the bottom, and
when the run finishes the page reloads once so the badge and summary
are final.
- list_ansible_runs and get_ansible_run report origin and origin_id.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run 8504 failed unit: tests/core/test_alert_history_row.py built a rule
with no metric_name, which _history_row now reads to say the reading in
words. The rule now names cpu_pct, and the value and condition asserts
expect "CPU 98%", "No CPU reading" and "CPU > 90%", plus an up/down case.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5212 "Plain labels: no raw ids, scopes, metric keys or unexplained jargon",
batch 1:
- Audit log: each action reads as a sentence ("paused monitor"), with the
code as its tooltip. Detail shows flags as yes/no, lists joined, a run's
scope by name, spans as spans, and no ids the entity column already gives.
The filter is now a picker of areas.
- Alert rules, history and Status say the condition in words: "is down",
"Fullest disk >= 85%". Readings use metric_value, and history times use
|ago.
- Rule form: sources and metrics by name, the threshold's unit, "fires
when it is down/up" for an up/down metric instead of 0.5, and a metric
reference built from the catalog.
- Schedules and Maintenance: viewers no longer see Run now, Pause, Edit,
Delete or New. Intervals and next runs read as "every 2h" and "in 3h";
a scope reads "Every alert on Host agent".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5211 "Lists load as you scroll: audit, alert history, Ansible runs"
(milestone 477), per preference 172.
- core/paging.py: keyset paging over (time, id), newest first, with an
opaque "<iso>|<id>" cursor; a page reads one row past itself to know
whether another follows. No OFFSET, so rows added while someone scrolls
never shift the pages after them.
- _macros.html scroll_more / scroll_end and static/scroll-more.js, written
once: an IntersectionObserver with a 600px margin fetches the next page
before the reader reaches the end; "Loading more…" while in flight,
"End of the list." at the end, and a failed page says so with a Retry
button instead of re-firing on every scroll.
- Audit log: no "Load more" link and no 500-row cap.
- Alert history: no 200-row stop with "pick a shorter range";
alert_svc.history takes the cursor, so the MCP tool and the page share
one query.
- Ansible runs: no silent cap of 50.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5210 "Problems first, with since-time, across every list" (milestone 477),
second batch.
- Host page: an "Alerts on this host" strip first (rules on its agent
series, its containers or its monitors: alert_svc.watches_host); its
monitors problems first with "Down 12m · reason"; Docker above Ansible.
- Uptime page and widget: worst 24h uptime first, then 7d and 30d; hosts
with no uptime yet last.
- Agent panel and fleet widget: last seen as "3m ago", not a clock time.
- SNMP: a device is "Reachable" only while it answered within the last 3
polls; older readings say "No answer" with their age, and every device
says when it was last read (was: reachable whenever any reading existed).
- Traefik: a router missing from 3 scrapes in a row is listed apart as no
longer reported, by when it was last seen, instead of keeping its last
state (a removed router stayed Failing).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5210 "Problems first, with since-time, across every list" (milestone 477),
first batch.
- Alerts: the Active list reads firing, no data, acknowledged, pending, the
longest-running first, and re-polls itself. An acknowledged alert says how
long it has been breaching (from the alert's events; acknowledging moves
last_transition_at) and, apart from that, who acknowledged it and when.
Rules show what each is doing now; disabled rules sort last. The current
value reads the latest-value table (#5208) instead of plugin_metrics.
- Monitors: a down monitor says "Down 12m · connection refused" in the list.
- Hosts: sorted down, agent gone quiet, pending, healthy. The agent column
shows "—" for a missing reading (was 0%) and a Stale badge with its age.
agent_metrics_by_host is gone; the list reads agent_overview_by_host.
- Schedules: failing first (failed or interrupted run, or a firing that could
not start one), then running, then the rest, paused last; status badge
from the shared states.
- Maintenance windows: active (ending soonest), then upcoming (starting
soonest), with expired folded away; each says when it ends/starts/ended.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
failing_services filtered on `(service_trouble(s) or 9) <= 2`; a crash loop
ranks 0, which is falsy, so the worst kind of trouble was left out. Test the
rank against None. The page test's task error also gains the age_s the
_swarm_failures include reads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5209 "Docker views: readable names, CPU/memory in cores and bytes, failures
listed with since-time" (milestone 477 "UI problems first").
- A swarm task reads as its service and slot ("codex_db · 1", or the node
for a global service), with the raw name in the title: Docker page, host
panel, busiest widget, container detail.
- CPU reads in cores (Docker's 100% = one core), toned by its share of the
host's CPUs from the agent's per-core series. Memory reads in bytes, against
a limit only when one is really set (Docker reports host RAM otherwise).
- Exit 137/143 without an OOM kill is a stop, not a failure: no
exited_nonzero, no crash-loop death, no "failed (24h)", dim exit code.
- "Failed (24h)" is a list (Docker page #failures, linked from the widget
and the tile): plain containers by their deaths, swarm services by Swarm's
failed tasks; a second manager's copy of a task death counts once.
- Failing swarm services lead Needs attention; services sort problems first
with since-times on the page, the widget and the Swarm page; nodes that
are down or drained sort first.
- Busiest widget orders by a 5-minute CPU average so the list holds still.
- Host panel: a table, problems first with what is wrong and since when;
replaced swarm tasks hidden and counted.
- Container detail: node hostname, I/O labelled "since start", Restarts
hidden for swarm tasks with a note on where task failures are.
- Widget names: "Docker — Services & failures", "Docker — Busiest containers".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Migration 0030 seeded plugin_metric_latest from the whole history: two index
probes per distinct metric key. On the live instance that held the first
boot for 7+ minutes with Steward down. The migration now only creates the
table. Live sources fill it on their next sample, and backfill_latest runs
after startup for resources that have history but no latest rows. Per
resource it does one backward probe for the last sample, then takes the
newest value per metric within the hour before it. Resources that already
have rows are skipped, so later boots only walk the resource names.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_last_n_results (Status, the monitors list, host monitors), host_status
(hosts overview, list_hosts) and the report's down-monitor list each
ranked every result a monitor ever recorded to keep the newest few. Each
monitor now gets ORDER BY checked_at DESC LIMIT n in a LATERAL subquery,
which walks ix_monitor_results_monitor_checked backwards and stops.
Also fixes the latest-table test, which read between two begin() blocks
and so autobegan a transaction the second block then refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dashboard's host widgets took 18-60 s. Every "latest reading" query
(hosts overview, host-agent fleet, host page, vitals, SNMP, MCP get_host)
searched plugin_metrics for the newest row per metric. Postgres has no skip
scan, so each one read every raw sample in the 7-day window.
- plugin_metric_latest: one row per (source, resource, metric).
record_metrics upserts it in the same transaction, moving forward only and
in sorted key order. Migration 0030 seeds it by stepping the existing index.
- Retention drops a latest row once its sample leaves the raw window, so a
resource that stopped reporting disappears when its history does, as before.
Renames carry it along.
- A quiet host keeps its last values, marked stale. A time-bounded query
would have shown it as missing.
- Vitals sparklines span the 6 h they claim, averaged in SQL to ~120 points,
instead of fetching 6 h raw and drawing the last 120 samples.
- Dashboard and share-view cells: "Loading…" until the first answer; a 20 s
deadline per fetch; "Couldn't load (why)" with Retry now and an automatic
retry for cells that don't poll; "as of HH:MM" in the header, and
"couldn't refresh · as of" when a refresh fails over shown data.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The operator's concept, redrawn as a single-colour SVG in the accent gold:
a key whose bow is a cloverleaf holding three connected nodes, hanging on a
loop threaded through a keyring and swinging into the ring's middle, with a
small sparkle.
- static/logo/steward-logo.svg: the full mark (node graph in the bow).
- static/logo/steward-mark.svg: the small mark for 16-32 px (heavier lines,
one centre node, two teeth); the node graph turns to mush below ~40 px.
Used in the sidebar brand (was an inline crown) and as the favicon on every
page and on the public share view.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run 8447 on b307f53: the boot check failed at the forced assertion, Push was
skipped, clean-up ran, :dev stayed on sha256:718d2ca (98854d9) and no
:b307f53 image exists in the registry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Dockerfile: python:3.14-slim, the Python CI tests on (was 3.13). tini from
apt is PID 1 (ENTRYPOINT tini -- entrypoint.sh): it reaps the zombies ssh
and ansible leave and forwards SIGTERM.
- cli: Hypercorn's serve() with one worker (run, stream and scheduler state is
in-process), a 10 s graceful timeout, and SIGTERM/SIGINT as the shutdown
trigger. Access and error lines go through Steward's logging. --debug keeps
Quart's reloading dev server.
- Logging: steward/core/logging_setup.py installs one root handler before
anything logs (until now a library's basicConfig on import was the only
configuration). Its level is the `log.level` setting (DEBUG/INFO/WARNING/
ERROR, default INFO), applied at boot and on save in Settings → General.
- CI publish: build, then "The image boots with no internet" (rule 164,
snippet #3552), then push, so a broken image never reaches a tag (rule 177).
On an --internal network with its own Postgres, the image must migrate and
answer /health, run tini as PID 1, serve first-run setup with its assets
from /static and no third-party host, and exit 0 on SIGTERM. Logs and
clean-up run always.
Tests: tests/core/test_logging_setup.py; the smoke step itself.
Not in this commit: the dependency lockfile (it conflicts with rule 154 and
is waiting on the operator).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The "[queued — waiting for run X on the same host]" and run-slot lines were
broadcast to live viewers and never stored, so a run that sat queued had no
record of why once the live view was gone (CI 8439: the overlap test found
the order right and the explanation missing). _claim_hosts now returns the
line it waited on and start_run seeds the run's output with every wait.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- One run per host: start_run claims every host it touches (addresses read
from the inventory Steward wrote; a source's own inventory file is claimed
as that file) before it takes a run slot. The claim is all-or-nothing. A run
waiting on a host shows queued and names the run it waits for, honours a
cancel, and fails after HOST_WAIT_SECONDS (1 h) with which host and run.
- A due schedule whose previous run is still queued or running skips the
firing: last_run_at moves on, last_run_id still names the run going, and
last_error says "Skipped: the previous run was still going". The Schedules
page shows that beside the run (muted), and a launch error in red.
- An alert rule that fires while the action run from an earlier firing is
still going starts no new run; the event says so and names the run. An
in-process set closes the gap between the check and the run being linked.
- docs/core/ansible.md: queued/cancelled statuses and the overlap rules.
Tests: tests/core/test_ansible_overlap.py (claim keys, waiting, all-or-nothing,
cancel, deadline) and tests/integration/test_ansible_overlap.py (two real
runs on one host take turns; schedule skip then fire; alert action skip then
start).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- git ships in the image; checkouts live under /data/ansible/sources
(STEWARD_ANSIBLE_SOURCE_DIR), on the persistent volume.
- One task, ansible_git_pulls, reads the sources every 30 s and pulls each
as its interval passes: a source added in Settings is pulled without a
restart, a removed one is forgotten.
- git_pull: a 120 s timeout ending the process group, GIT_TERMINAL_PROMPT=0,
pull --ff-only, a failed clone removed; it raises GitPullError with git's
message, so the Sync button stops saying Synced after a failure (and the
message is now escaped).
- An alert action's vars and the firing's steward_alert_* vars reach
ansible-playbook as one JSON extra-vars file, not -e key=value, which
Ansible split on whitespace (a rule named 'High CPU on web').
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Output is read in 64 KiB chunks and split into lines here; a line past
64 KiB is cut and marked instead of raising out of readline's limit.
- ansible-playbook runs in its own session; whatever ends the read (finish,
an error, the task being cancelled) SIGTERMs then SIGKILLs its whole group,
so no ssh or worker it forked outlives the run.
- A run whose task is cancelled (shutdown) records 'cancelled' with
finished_at before re-raising, instead of staying 'running'.
- A cancel that lands after the process exited 0 no longer turns a
successful run into 'cancelled'.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- prune_by_age(column) registers a time-series column with the hourly cleanup;
Traefik (metrics, access summaries, global stats) and UniFi (WAN, client,
WLAN, DPI, events, alarms, speedtests) register theirs in setup().
- Alert events join monitor results and Ansible runs on data.retention_days,
read from Settings each run; Ansible run log files older than it are deleted.
- audit.retention_days (365) keeps the audit log longer, set on Settings > General.
- metric_series.points reads metrics.retention.raw_days, so the chart's
raw/rollup seam is where the rollup put it (RAW_DAYS removed).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
