Commit Graph

46 Commits

Author SHA1 Message Date
bvandeusen fee654b53a feat(docker): schema for lifecycle events + swarm topology
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 47s
CI / integration (push) Successful in 2m20s
CI / publish (push) Successful in 58s
Adds the milestone-77 storage that doesn't fit on the per-container row:

  * docker_events — lifecycle (start/stop/die/oom/health_change), to be
    derived by diffing consecutive host snapshots; host-scoped, indexed for
    timeline lookups (host_id, container_name, at) and retention pruning (at).
  * docker_swarm_services / docker_swarm_nodes — manager-reported Swarm
    topology (desired-vs-running replicas, node role/availability/status).

Migration docker_004 extends the docker branch (down_revision docker_003);
purely additive, no DROP+recreate. event/mode/role are plain strings (no
CHECK whitelist), matching how docker_containers models status. Integration
guard asserts the three new host-scoped tables exist.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
2026-06-18 20:47:46 -04:00
bvandeusen 82c3d2cf36 feat(docker): per-container enrichment — health, restarts, exit code, I/O, grouping
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 44s
CI / integration (push) Successful in 2m20s
CI / publish (push) Successful in 1m10s
First slice of milestone 77 (Docker monitoring depth). Surfaces real per-container
stats beyond basic state, all read-only on the existing push model.

- agent (→1.4.0): collect_docker now inspects each container (health, restart
  count, exit code, OOM) and reads net + block I/O from the stats payload; pulls
  compose project + swarm service/task/node from container labels. Per-container
  inspect+stats calls run over a small bounded ThreadPool so the ~1s-per-stats
  blocking doesn't stretch the sample on a busy host.
- schema (docker_003): additive columns on docker_containers — health, exit_code,
  oom_killed, compose_project, service_name, task_id, node_id, and BigInteger
  net/blk byte counters.
- ingest: persists the enrichment + restart_count (.get keeps older agents working).
- ui: Docker page rows now show health badge, uptime ("up 3d 4h"), restart count,
  exit code (+OOM) for stopped containers, and compose/service grouping label.
- tests: agent helpers (grouping, inspect fields, net/IO sum) + collect_docker
  assembly incl. inspect; integration asserts enrichment round-trips.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
2026-06-18 20:37:40 -04:00
bvandeusen 7b80552a7d feat(docker): per-host collection via the host agent; drop central scrape
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 2m16s
CI / unit (push) Successful in 4m28s
CI / publish (push) Successful in 1m4s
Docker collection moves off the central single-socket scrape onto the host
agent, giving Docker a real per-host dimension. The Steward host now reports
its own containers like any other host, and same-named containers on different
hosts no longer collide.

- agent: stdlib UDS Docker client (AF_UNIX HTTP/1.1, Connection: close,
  chunked-aware), collect_docker() ports the cpu%/mem math; sample["docker"]
  added best-effort (silent-skip on absent/unreadable socket). AGENT_VERSION
  1.2.0 → 1.3.0; optional docker_socket config key.
- ingest: host_agent ingest hands per-host container snapshots to the docker
  plugin via a new "docker.persist_host_samples" capability (no hard import,
  no-op when docker disabled), inside a SAVEPOINT so a docker failure never
  sinks the host metrics. Resource names are host-scoped ("<host>/<name>").
- schema: docker_containers re-keyed (host_id, name); docker_metrics gains
  host_id; docker_002 migration DROP+recreates (dev-only, rule 122).
- ui: Docker page + widgets grouped by host with host links; new per-host
  Docker panel embedded on the Hosts hub (gated on docker enabled via a new
  enabled_plugins template context). Replaces the SQLite-only strftime
  bucketing with DB-agnostic Python bucketing.
- provisioning: install/provision playbooks add steward-agent to the docker
  group (best-effort) so the agent can read the socket.
- removed central scrape: docker scheduler.py + scraper.py deleted; plugin.yaml
  socket_path/scrape_interval_seconds/include_stopped dropped (plugin 2.0.0).
- tests: agent docker collector units (math, chunked decode, silent-skip,
  sample shape, config) + integration (host-scoped schema + persistence).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
2026-06-18 17:54:36 -04:00
bvandeusen 35f658b573 feat(monitors): unify ping/dns/http into one Monitor entity + custom targets
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m17s
CI / publish (push) Successful in 1m10s
Collapse the three former check types into a single core `Monitor` entity
with one management surface (/monitors), one result table (monitor_results),
and a single scheduled task. Every type can now watch a free-standing custom
destination (optional host_id) — not just a registered Host.

- models: Monitor + MonitorResult replace PingResult/DnsResult; Host loses its
  ping/dns facet columns (now Monitor rows linked by host_id).
- checks: monitors/{ping,dns,http}.py pure probes + runner.run_monitor
  dispatcher; one monitor_check scheduler with a per-monitor due-filter.
- status: single monitor_status_source replaces the three sources.
- UI: /monitors blueprint (type-aware add/edit/list/widget); host hub shows a
  host's linked monitors + "add monitor for this host"; nav + widget registry
  + alert metric catalog rewired. http plugin folded into core and removed.
- migration 0022 merges the http branch, data-migrates host facets +
  http_monitors + all three result histories, drops the old tables/columns.

Resolves the per-host ping/dns auto-attach issue (#275): monitors are now
explicit, never auto-added to every host.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
2026-06-18 08:56:13 -04:00
bvandeusen 591706bd39 feat(settings): configurable monitoring thresholds
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m19s
CI / publish (push) Successful in 1m3s
Move the hardcoded warn/crit cutoffs into Settings -> Thresholds (DB-backed,
live, no restart). New thresholds.* keys + to_thresholds_cfg() + a
threshold_style(value, kind) jinja global that reads them; latency reuses the
existing ping good/warn keys, uptime is direction-aware (floors).

Replace the _macros metric_style/uptime_style macros (now removed) with the
global across Hosts-Overview, host_agent fleet + panel, Uptime/SLA widget, and
the ping page uptime column — all now honor the configured cutoffs. Uptime keeps
its green 'good' look when not degraded.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 20:52:58 -04:00
bvandeusen eefa38cc75 feat(host_agent): in-widget 1h/6h/24h time-range toggle on history graph
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m20s
CI / publish (push) Successful in 8s
Add a live range switch to the history widget that re-requests the fragment
without entering edit mode. It rewrites the parent cell's hx-get so the choice
survives polling (htmx re-reads the attribute each poll), then fetches at once.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 20:44:56 -04:00
bvandeusen e7b96fbfa7 feat(host_agent): surface network, disk I/O, and temperature in fleet widget
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m15s
CI / publish (push) Successful in 10s
The agent already collects + ingests net throughput, disk I/O, and temps;
add them to the fleet-glance rows (fmt_bps + two-line io cells + temp with
70/85C threshold color), each shown only when the host reports it so VMs/
containers without sensors don't show blank cells.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 20:43:33 -04:00
bvandeusen 10808c1c5d fix(dashboard): inline cpu sparkline, history graph host label + stable y-axis
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m16s
CI / publish (push) Successful in 54s
- Hosts-Overview: move the CPU sparkline inline next to the cpu % value it
  represents (was floated far right on the name line, reading as unrelated).
- Host Agent history widget: caption the chart with the host it represents
  (the panel title is generic) — links to the host hub.
- History widget: snap the y-axis to a stable 0..next-10%-band ceiling instead
  of auto-scaling to the exact peak, so the 'zoom' no longer jumps each poll.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 16:44:31 -04:00
bvandeusen ebc67723d2 feat(host_agent): horizontal zebra-striped fleet widget rows
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m14s
CI / publish (push) Successful in 7s
Restore the host_agent fleet-glance widget to one horizontal row per host
(name left, metric cells + sparklines right) instead of the multi-column
block grid, and zebra-stripe odd rows so adjacent hosts read as distinct.
Names still wrap rather than hard-truncate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 16:31:27 -04:00
bvandeusen fed9973899 fix(dashboard): downsample history graphs + readable tooltip time
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m15s
CI / publish (push) Successful in 6s
The agent reports every few seconds, so multi-hour history series were hundreds–
thousands of points — a dense, noisy line (esp. CPU). Bucket-average server-side
to ~120 points (keeps the shape, drops the noise) for both the history-graph
widget and the host-detail charts. Also fix the chart tooltip title showing the
raw epoch-ms (e.g. 1,781,720,471,459) — format it as HH:MM.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 16:18:50 -04:00
bvandeusen e58c86cf01 feat(dashboard): widget readability — name-per-line, fill graphs, container breakpoints
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m19s
CI / publish (push) Successful in 53s
Address graphical issues raised from the dashboard screenshot:
- No more truncated host names. Hosts-Overview and Host-Agent-Resources put the
  host name on its own line (full, wraps if needed) with its data grouped beneath
  it; ping/dns (.ping-name) and uptime widget names wrap instead of ellipsis.
  Prefer vertical overflow over cramming/truncating a row.
- History graph fills the panel: drop the fixed 0–100 y-axis ceiling (beginAtZero
  + 8% grace) so the lines use the vertical space instead of hugging the bottom;
  axis labels still show real %.
- Container-query breakpoints: the widget body is now a query container, so
  fragments restyle to their OWN panel width. Host-list widgets flow into 2 cols
  ≥520px and 3 cols ≥900px (.host-blocks) — use the width, remove vertical
  deadspace — and collapse to one column when narrow. Mirrored into the share view.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 16:14:59 -04:00
bvandeusen cb47b5e977 feat(dashboard): per-metric sparklines in the Host Agent — Resources widget
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 9s
CI / integration (push) Successful in 2m17s
CI / publish (push) Successful in 47s
Operator ask: show the graphs next to their fields in the fleet widget, like the
host page's AGENT panel. Each row now renders cpu/mem/disk/load as a value with a
trend sparkline beneath it (1h window), instead of bare numbers.

- _fleet_rows fetches a per-host recent series (cpu/mem/load host-level, disk from
  the root mount) in one 1h query and attaches a sparkline per metric to each row.
- widget_table.html lays out a metric cell (label + threshold-coloured value +
  sparkline) per field, mirroring panel.html. Threshold colour is computed in the
  loop and passed into the cell macro (keeps Jinja macro scope clean).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 16:06:05 -04:00
bvandeusen 988c13d51f feat(dashboard): phase C richer panels — host time-series graph + cpu sparklines
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m25s
CI / publish (push) Successful in 56s
Milestone 72 phase C — bring the host-view graphs onto the dashboard:
- host_resource_history widget reworked into a real host-view chart: epoch-ms
  linear axis (no Chart.js date adapter), themed like the host-detail charts,
  maintainAspectRatio:false so it fills the resized panel, unique canvas per
  widget instance (wid), and empty states ("pick a host" / "no metrics yet").
  Was previously unusable — it had a broken time axis and no way to choose a host.
- Add a "host" param type: the edit form renders a live dropdown of hosts
  (dashboard routes now pass the host list to the editor); the chosen host_id is
  stored in config and fed to the widget.
- Hosts-overview widget gains a per-row CPU sparkline (last hour) via the shared
  sparkline_svg helper — the host-view at-a-glance trend, on the main widget.

Charts/sparklines render only in the browser, so CI can't exercise them — needs
an operator visual check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 15:18:50 -04:00
bvandeusen 5f92340c0c feat(dashboard): phase A widget clarity — threshold colours, host links, tooltips
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m20s
CI / publish (push) Successful in 56s
Milestone 72 phase A (clarity wins, no schema change):
- Add shared metric_style() macro in _macros.html: colours a numeric metric
  amber (>=warn) / red (>=crit) on the value ITSELF, not just the status dot.
  Defaults 80/90 for percentage gauges; load /core uses warn 80 / crit 100.
  Applied to CPU/mem/disk across host_agent panel + fleet widget + the unified
  hosts-overview widget.
- Link every host reference to the host hub (/hosts/<id>) in ping, dns,
  hosts-overview, host_agent fleet, and uptime widgets; status widget entries
  link to their own detail_url. All guarded on session.user_id so the public
  share view degrades to plain text (the bare href carries no share token).
- Fix truncation: title= tooltips on all names that ellipsis, plus a hover
  underline affordance on the now-clickable ping-name links.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 14:49:49 -04:00
bvandeusen 88afad9de4 feat(hosts): re-provision action on reporting hosts; Ansible off the edit page
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 54s
- Host agent panel (reporting state) gains a "Re-provision" collapsible (admin +
  linked target + managed key): bootstrap user/password → provision.yml, which
  reinstalls the steward account + managed key + agent. This is the missing path
  after regenerating the managed key — Update alone can't fix a broken key.
- Remove the Ansible sections (run-playbook + target link) from the host EDIT
  page — they were the stale free-text version and the wrong place. They live on
  the host detail hub (dropdown + discovered variables). Edit page now links to
  the host page; edit_host route simplified (no ansible fetch).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 13:08:35 -04:00
bvandeusen e0253fba48 fix(ansible): surface run failure reason; widget audit cleanup
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m17s
CI / publish (push) Successful in 1m1s
Run UX: when start_run throws before/around launch (e.g. ENOSPC creating the
temp dir — the box is out of disk), the run was marked "failed" with empty
output. Now the exception is broadcast + written to the run output/results so
the run view shows e.g. "[run error] OSError: [Errno 28] No space left on
device" instead of a blank failure.

Widget audit follow-ups (no broken links were found; these are consistency):
- host_resource_history widget now charts root (/) disk, consistent with the
  host panel (was the opaque "disk worst").
- host_resources widget: tooltip on the health dot explaining it warns on the
  worst mount while the number shows root.
- status_overview widget detail_url /status → /status/ (avoid redirect).
- Normalize ad-hoc widget empty-states to the shared .empty style (wording,
  which distinguishes "configured" vs "data yet", preserved).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 12:46:08 -04:00
bvandeusen 609bd78af2 feat(dashboard): unified Hosts widget + grouped widget picker
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m15s
CI / publish (push) Successful in 56s
Bring the dashboard in line with the unified host IA + improved displays.

- New core "Hosts — Overview" widget (/hosts/overview/widget): one row per host
  combining monitor status (ping dot + latency, uptime 24h) with the agent
  glance (CPU / memory / disk root + stale flag), each row linking to the host
  hub. Reads agent data from the generic PluginMetric table via a core-safe
  _agent_overview_by_host helper (no host_agent import); freshness vs the
  plugin's stale window. The granular Ping/DNS/Uptime/Agent widgets stay.
- Group the add-widget picker into Core monitors / Monitoring capabilities /
  Integrations (a `group` field on every WIDGET_REGISTRY entry + section
  headings in _edit_panels.html), matching the Settings → Plugins taxonomy.
- Fix the agent fleet widget rows to link to the host hub (/hosts/<id>) instead
  of the old /plugins/host_agent/<id>/ page.

Scribe #903.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 12:00:12 -04:00
bvandeusen a0d1c5f07c feat(host_agent): sparklines + load/core + PSI on the host panel
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m23s
CI / publish (push) Successful in 7s
- Each at-a-glance metric (CPU, Memory, Disk /, Load) now shows a 6h sparkline
  (reused core.status.sparkline_svg + a recent-series query; disk uses the root
  mount sub-resource) so trend/consistency is visible, not just the instant.
- Load is now normalized: "Load /core" = 1m load ÷ CPU cores as % (100% = run
  queue matches capacity), comparable across different hardware. Cores derived
  from the per-core CPU metrics already collected — no agent change. Raw load
  in the tooltip.
- Added a "Pressure 10s" line: PSI cpu/mem/io (some, avg10), the
  hardware-independent saturation signal already collected by the agent.

Scribe #898.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 11:10:46 -04:00
bvandeusen 36212dc58b feat(host_agent): at-a-glance disk = root (/) not "worst"; metric tooltips
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m19s
CI / publish (push) Successful in 6s
The "Disk (worst)" number was opaque at a glance. Show the root filesystem (/)
usage instead — what people actually care about — on the host panel and the
dashboard widget. "Worst" is kept only for the widget's health dot (so a full
/var or /data still warns) and on the full-metrics page (per-mount + worst
trend). Added hover tooltips defining CPU / Memory / Disk / Load.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 11:05:43 -04:00
bvandeusen 7e6e63521b fix(host_agent): provision/deploy vars shadowed by play-var precedence
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m15s
CI / publish (push) Successful in 54s
The assertion failed ("Pass steward_url/token/pubkey") because those were
injected as inventory HOST vars, but the playbooks declared them in the play
`vars:` block — and play vars OUTRANK inventory host vars, so the empty
defaults won and the injected values never reached the play.

- Pass globals (steward_url, steward_pubkey, steward_user, agent_interval) as
  extra-vars via the JSON -e @file (highest precedence, space-safe). Keep only
  the per-host steward_token as an inventory host var.
- provision.yml / install.yml: drop steward_token from `vars:` so the host var
  isn't shadowed; assertions use `| default('')` for the un-defaulted token.

This was the next layer under the inventory-format fix — first the inventory
wouldn't parse, now the injected vars actually apply.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 10:11:30 -04:00
bvandeusen bb90411f00 fix(host_agent): gate "deployed" on agent check-in; fail no-op runs
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 9s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 58s
A failed provision looked successful in two ways:

1. The host panel showed the agent as deployed (metrics + Update/Rotate/Remove)
   because provision/deploy mint the registration row BEFORE the playbook runs
   and the panel keyed "installed" on that row. Now gated on the agent actually
   checking in (reg.last_seen_at). Three states: reporting (metrics + lifecycle),
   pending (token minted but no check-in → "no metrics yet, deploy may be
   running/failed" banner + retry + Clear pending registration), and none
   (install path).

2. The run reported success though nothing ran — ansible-playbook exits 0 on
   "no hosts matched"/empty inventory. The executor now treats an empty PLAY
   RECAP (returncode 0 but no hosts executed) as failed, with a clear failure
   note. Non-zero exits and recap failed/unreachable were already caught.

Scribe issue #887.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:50:11 -04:00
bvandeusen a92d1995d5 fix(ansible): write DB inventory as static YAML, not dynamic --list JSON
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m20s
CI / publish (push) Successful in 1m6s
generate_inventory() emits Ansible's dynamic --list shape (all.hosts is a
LIST, vars under _meta) — valid only as an executable inventory script's
stdout. We were writing it to a static file and passing -i, so Ansible's yaml
plugin rejected it ("Invalid 'hosts' entry for 'all' group, requires a
dictionary, found ...list...") and fell back to implicit localhost → "no hosts
matched". Affected every steward:* scope run; surfaced on the first real
provision.

- New inventory_to_yaml(inv): convert the --list dict → a valid static YAML
  inventory (all.hosts dict keyed by host, groups under all.children, group
  vars preserved, injected per-host vars like steward_token retained).
- Wire it into runner.trigger_run, host_agent deploy + provision.
- executor writes the file as inventory.yml so the yaml plugin's extension
  check reliably claims it.
- generate_inventory unchanged (still the --list dict); conversion happens at
  write time. Unit tests added.

Scribe issue #885.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:28:31 -04:00
bvandeusen cfe6b4c25f fix(host_agent): deploy/provision 500 — double-begin on session
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 9s
CI / integration (push) Successful in 2m21s
CI / publish (push) Successful in 8s
fetch_scope_targets() runs a SELECT that autobegins the session transaction
(SQLAlchemy 2.0), so the following `async with db.begin()` raised
"A transaction is already begun on this Session" → 500 on both
/plugins/host_agent/deploy and /provision.

Mint registrations directly into the autobegun transaction, build the
inventory while the ORM objects are still live, then await db.commit().
Escaped CI because unit tests use a mock db_sessionmaker; no integration
test exercises these routes (follow-up noted in Scribe issue #884).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 22:53:13 -04:00
bvandeusen 17c9c875e4 refactor(hosts): remove legacy host_agent redirects/paths (no back-compat)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m25s
CI / publish (push) Successful in 39s
Dev-only instance, no bookmarks — per family rule 22, fully remove old paths
instead of shimming them.

- Delete the /plugins/host_agent/ (index) and /plugins/host_agent/settings/
  redirect routes; delete the now-dead host_list.html fleet template.
- Move the remaining management POST routes off /settings/ to /fleet/
  (add-host, rotate-token, delete) — single canonical prefix.
- Repoint real callers to canonical URLs: dashboard widgets (host resources →
  /hosts/, history → /plugins/host_agent/fleet/), the full-metrics page
  breadcrumb + back link (→ the host hub), Settings→Ansible link, and the
  agent panel's curl-install link.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 21:20:11 -04:00
bvandeusen f29255039d feat(hosts): Phase 2+3 — agent column, fold fleet + management into Hosts
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 9s
CI / integration (push) Successful in 2m16s
CI / publish (push) Successful in 54s
- Hosts list: new Agent column (latest CPU/mem read from the generic
  PluginMetric table — no host_agent import) + an admin "Agent fleet" button.
- /plugins/host_agent/ (old fleet page) now redirects to /hosts/ (folded into
  the hub; kept as a redirect so widgets/links don't 404).
- Agent management moved off the "settings" URL: the management page is now
  /plugins/host_agent/fleet/ ("Agent fleet" — bulk provision/install/update +
  registrations + curl install), reachable from the Hosts list. Old
  /plugins/host_agent/settings/ redirects there. Per-host management lives on
  the host detail page; this page is now explicitly the bulk/fleet view.

Milestone 70 phases 2-3. Phase 4 (plugins capability/integration split + nav
cleanup) next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:51:33 -04:00
bvandeusen 8bdf07f709 feat(hosts): Phase 1 — host detail hub page (unify host IA)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m16s
CI / publish (push) Successful in 48s
Make Hosts the front-and-center hub. A host now has a real detail page at
/hosts/<id> that pulls its facets into one view, instead of management being
scattered across a nav-less Host-Agents area and the edit form.

- hosts: new GET /hosts/<id> detail route + hosts/detail.html. Shows the
  monitors summary (ping/DNS status + latency + uptime 24h/7d/30d), an Ansible
  section (linked target, link/create, run-playbook), and an embedded Agent
  panel. Hosts list name links here; ansible-link redirects here.
- host_agent: GET /plugins/host_agent/panel/<host_id> — a self-contained HTMX
  fragment embedded into the core hub across the plugin boundary (core never
  imports plugin models). Shows live agent metrics + Update/Rotate/Remove when
  installed, or the provisioning path when not: inline "generate managed key"
  warning, a prompt to link an Ansible target first, then Provision (bootstrap
  password) / Install (managed key) tied to the host's target scope.

Part of milestone 70 (Hosts hub). Phase 2+ will enrich the list, redirect the
old fleet/settings pages, and re-taxonomize plugins into capabilities vs
integrations.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:29:34 -04:00
bvandeusen 6a8146b544 fix(ansible): replace steward key on reprovision + distinct agent update path
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m19s
CI / publish (push) Successful in 54s
Provisioning review corrections + the matching frontend, plus breadcrumb
header integration.

- provision.yml: authorize the managed pubkey with a regexp match on the
  ' steward-managed' comment so rotating the key REPLACES the host's steward
  key in place instead of stacking a second authorized entry. Hand-added keys
  (other comments) are untouched.
- update.yml (new): refresh agent.py + restart only. Does NOT rotate the token
  or rewrite /etc/steward-agent.conf — the host keeps its identity. Asserts the
  agent is already installed and fails clearly otherwise.
- host_agent /update route: runs update.yml as the managed steward user (no
  token minting). Token rotation stays a deliberate action.
- settings/ansible/generate-key honors a safe relative `next` redirect, so an
  inline trigger elsewhere returns to its page.
- Host Agents settings: reworked into a clear lifecycle — an intro card that
  explains it runs Ansible to deploy the agent (+ inline "generate managed key"
  warning/trigger when none exists), then three labelled cards: 1 Provision,
  2 Install/enroll, 3 Update. Each explains what it does.
- base.html: breadcrumb now renders as a kicker line directly above the page
  title (moved below alerts, tightened margin) so nested and top-level views
  share one consistent header.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 19:22:12 -04:00
bvandeusen 0318f6423f feat(ansible): host provisioning via steward managed SSH identity
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 1m3s
Turn the agent-install playbook into a full provisioning + maintenance
path. Solves the bootstrap chicken-and-egg: first contact uses an
operator-supplied password (one run, never stored), which creates a
dedicated `steward` login account with NOPASSWD sudo + Steward's managed
public key. Every run thereafter connects as `steward` with the managed
key — fully unattended (scheduled prune, agent updates).

- core/crypto: generate_ssh_keypair() — ed25519, OpenSSH formats.
- settings: ansible.ssh_public_key (non-secret, displayed) + ansible.ssh_user
  (default steward); to_ansible_cfg extended.
- settings UI + route: "Generate managed key" (private encrypted, public
  shown to copy) + SSH-user field.
- executor: build_bootstrap() writes a 0600 vars file (-e @file) for the
  per-run user/password — never argv, never DB, never logged; drops the
  managed key when a bootstrap password is given; --user floor from the
  global ssh_user when no override.
- runner.trigger_run: pass-through `connection` kwarg, deliberately NOT
  persisted on AnsibleRun.params (password stays out of the DB).
- bundled/host_agent/provision.yml: create steward user + authorized_keys
  + /etc/sudoers.d/steward (visudo-validated) + agent install.
- host_agent: /provision route + "Provision a fresh host" card (bootstrap
  user/password; injects pubkey + user + token as space-safe JSON hostvars).
- Dockerfile: add sshpass (Ansible shells out to it for password SSH).
- tests: keypair generation + build_bootstrap (secret stays off argv).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 17:17:38 -04:00
bvandeusen 389002fc6f feat(ui): breadcrumb navigation across nested pages
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 1m2s
Adds a breadcrumb trail to nested views for orientation. Mechanism: a
{% block breadcrumb %} slot in base.html (+ styling) and a shared crumbs()
macro in templates/_macros.html; each nested page fills the block with its
trail (root→current, last item is the current page). Pages without the block
render no bar, so top-level nav roots stay clean.

Applied to: Ansible (browse, schedules, playbook editor, run detail) +
inventory (targets/groups + detail), host_agent (fleet, host detail,
settings), hosts form, settings tabs (ansible/auth/notifications/plugins/
reports + plugin detail), dashboard (list, edit), and alerts (rule form,
maintenance + new). Dynamic labels (host/target/run/dashboard names) come
from the page context — no route changes. All 60 templates Jinja-compile.

Task #873.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 14:22:19 -04:00
bvandeusen f3e919892d feat(plugins): plugin capability registry + host_agent→Ansible deploy synergy
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m18s
CI / publish (push) Successful in 1m10s
Implements #253's framework: a small core capability registry
(steward/core/capabilities.py) where a module/plugin publishes a named,
role-gated action and a consumer discovers it via has_capability() and runs it
via invoke_capability() — no hard import, graceful degradation, permission
propagation (actor role checked against the capability's required_role).

Core publishes "ansible.run_playbook" (operator) wrapping ansible.runner.
trigger_run (extended to accept a caller-built inventory). First consumer: the
host_agent plugin gains "Deploy via Ansible" on its settings page — pick an
inventory target/group and it installs/updates the agent via the bundled
host_agent/install.yml, minting a fresh token per host and injecting it as an
inventory hostvar (turning per-host curl|sh into one run). Exposed role
ordering as middleware.role_meets. Unit tests for the registry + role checks.

Task #253 (milestone #37).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 11:28:29 -04:00
bvandeusen bca1b92cc6 feat(host_agent): Netdata-style fleet overview + per-host detail view
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m24s
CI / publish (push) Successful in 58s
Surfaces the metrics the agent now collects (task #868). Adds a viewer-facing
fleet page (/plugins/host_agent/) with per-host cards (CPU/mem/disk/load/temp,
stale flag) and a per-host detail page (/plugins/host_agent/<id>/) — the link
the fleet widget already pointed at but had no route for.

Detail page shows current gauges (CPU, memory incl. swap/cache, load, network
rx/tx, disk I/O, temp max, memory/cpu/io PSI), per-core CPU bars, per-mount
filesystem bars, and per-interface/disk/sensor breakdowns, plus history charts
(utilization %, throughput B/s, load & pressure) over a selectable range.
Charts use a linear epoch-ms x-axis so no Chart.js date adapter is needed.
Stale state uses the plugin's stale_after_seconds threshold. Repoints the
host_resources widget detail link from admin settings to the new fleet page.

Completes milestone #68 (task #867).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 10:27:57 -04:00
bvandeusen a9e7baee6a feat(host_agent): collect network, disk I/O, per-core CPU, temps, memory PSI
CI / lint (push) Successful in 3s
CI / unit (push) Failing after 8s
CI / integration (push) Successful in 2m17s
CI / publish (push) Has been skipped
Extends the push agent (v1.2.0) with the Netdata-style signals the operator
wants: per-core CPU, per-interface network throughput and per-disk I/O
(rates derived in-agent from monotonic /proc counter deltas, with
counter-reset clamping), hardware temperatures (/sys/class/hwmon),
memory-pressure PSI (/proc/pressure), and cached/buffers memory breakdown.
All stdlib-only.

Server side needs no migration — these land as additional rows in the shared
PluginMetric table via _expand_sample_to_metrics. Per-resource series use a
':' in resource_name (host:net:eth0, host:core0, host:temp:Package) so the
existing fleet widget's ':' filter ignores them; host-level totals/max are
emitted at the bare host resource. New sample keys are optional, so older
agents keep ingesting unchanged.

Unit tests for the new parsers, rate/reset logic, and the expander contract.
Data foundation for the Netdata-style host view (task #867).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 10:11:30 -04:00
bvandeusen 3b6e005ed8 feat(status): unified Status page + widget across ping/DNS/HTTP monitors
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m15s
CI / publish (push) Successful in 56s
Adds a Kuma-style "is everything up?" surface that aggregates heterogeneous
monitor types via a status-source registry (steward/core/status.py): each
type registers an async source(db) -> [StatusEntry]. Core registers ping/DNS;
the http plugin registers its own from setup() so core never imports plugin
tables. Per entry: current up/down, last-30 heartbeat bar, uptime %
(24h/7d/30d), latest latency + response sparkline, and TLS expiry countdown
(HTTP). New /status page (live htmx refresh) + a status_overview dashboard
widget + nav link. Pure-function unit tests for registry + sparkline.

First deliverable of milestone #68 (task #866).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 22:18:29 -04:00
bvandeusen 3b2146bc7a feat(host_agent): agent reports its primary IP; mirror into Host.address (task 274)
CI / lint (push) Successful in 3s
CI / unit (push) Failing after 8s
CI / integration (push) Successful in 2m9s
The agent now detects its own primary non-loopback IP (UDP-socket trick, no
packets sent) and sends it in the metadata bag; bumps AGENT_VERSION 1.0.0->1.1.0
and re-detects on SIGHUP. The server persists it on HostAgentRegistration.host_ip
and mirrors it into Host.address ONLY when that field is blank — an admin-typed
address is never overwritten. The reported IP shows in the settings table so
admins can see drift regardless.

- agent.py: detect_primary_ip(); host_ip in collect_metadata(); refresh on reload
- routes.py: pure pick_host_address() helper; ingest persists host_ip + mirrors
- models.py + migration host_agent_002_host_ip: new String(45) column
- settings_list.html: Reported IP column
- plugin.yaml: 1.0.0 -> 1.1.0
- tests: pick_host_address (fill-when-blank / never-clobber / no-op),
  detect_primary_ip (never raises, non-loopback), metadata carries host_ip

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 12:11:41 -04:00
bvandeusen af60ca446d fix(plugins): complete steward rename across bundled plugins; clear lint debt
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m8s
The fabledscryer->steward rename had only ever reached host_agent. The other
five bundled plugins (http, snmp, traefik, unifi, docker) still imported
`from fabledscryer.*` (package no longer exists) and read FABLEDSCRYER_* env
vars — so every one of them was broken at import since the original rebrand.
CI stayed green only because none are enabled by default and migrations don't
import plugin modules. Now that they version in-tree, complete the rename:
- fabledscryer.* -> steward.* imports across all five plugins
- FABLEDSCRYER_* -> STEWARD_* in plugin migration env.py files
- author/repository/homepage + user-facing 'Fabled Scryer' strings -> Steward
- snmp/scheduler.py: also drop dead `now`/datetime; record_metric from steward

Adds tests/test_no_legacy_names.py — fails if 'scryer'/'roundtable' ever
reappear in shipped code (the drift bit twice; this stops a third time).

Also clears pre-existing ruff lint debt (unused imports, semicolon statements,
mid-file import) surfaced by the new lint lane.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 11:22:20 -04:00
bvandeusen a7a281cb11 feat(plugins): fold first-party plugins in-tree; bundled + external roots
First-party plugins (host_agent, http, snmp, traefik, unifi, docker) are now
tracked under plugins/ and baked into the image, so they version atomically
with core — ending the cross-repo import drift the roundtable->steward rename
exposed. History for these files is preserved in the archived Roundtable-plugins
repo.

Plugin discovery becomes multi-root: PLUGIN_DIR (single) -> PLUGIN_DIRS
(bundled first, then external) + PLUGIN_INSTALL_DIR. Bundled ships in the image;
third-party plugins still mount at runtime into the external root
(STEWARD_PLUGIN_DIR, default /data/plugins) and downloads/installs land there.
Bundled shadows external on a name collision.

- config.py: load_bootstrap returns plugin_dirs + plugin_install_dir
- app.py: iterate PLUGIN_DIRS at the migration + load sites
- migration_runner.py: discover_all_in() unions every plugin root
- plugin_manager.py: resolve_plugin_path() (pure, first-root-wins); load /
  install / hot-reload span all roots; installs target the external root
- settings/routes.py: _discover_plugins scans all roots, dedup bundled-first
- Dockerfile: COPY plugins/ ; docker-compose: drop host bind, document external
- tests/test_plugin_dirs.py: resolution, multi-root discovery, bootstrap split

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 08:37:24 -04:00
bvandeusen fa318073f1 chore: move plugins/ to separate repo, gitignore the directory
plugins/ is now tracked at bvandeusen/fabledscryer-plugins.
Removed tracked traefik files from this repo's index (files remain on disk).
Added plugins/ to .gitignore so the directory is ignored going forward.

PLUGIN_DIR in config.yaml still points to plugins/ for local dev and Docker.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-22 18:33:22 -04:00
bvandeusen 165a202ba4 feat: UI redesign (CSS system, DNS widget, Traefik dashboard widget)
- Replace base.html style block with full CSS design system using custom properties (dark theme, cards, tables, badges, dots, widgets)
- Add DNS blueprint (fablednetmon/dns/) with /dns/ and /dns/rows routes + templates
- Add Traefik /widget route and widget.html fragment for dashboard
- Update dashboard: DNS + Traefik widgets, traefik_enabled config detection
- Update all templates to use new CSS classes (page-title, section-title, card-flush, table, badge-*, dot-*, btn-*)
- Register dns_bp in app.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 23:37:46 -04:00
bvandeusen e6fa8baed1 feat: AppSetting model and migration 0004
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 20:22:23 -04:00
bvandeusen 4cc83c8a19 fix: use relative imports in traefik plugin to prevent double-registration; resolve PLUGIN_DIR path 2026-03-18 18:46:02 -04:00
bvandeusen 2164ea28f2 feat: traefik UI (per-router dashboard with SVG sparklines) 2026-03-18 18:43:05 -04:00
bvandeusen aacfcd420e feat: traefik scrape task (delta tracking, DB write, record_metric) 2026-03-18 18:38:54 -04:00
bvandeusen 8ea2b09d05 fix: cap percentile at last finite bucket when target falls in open +Inf bucket 2026-03-18 18:38:03 -04:00
bvandeusen c336f1f774 feat: traefik prometheus scraper (parse, compute rates and percentiles) 2026-03-17 21:37:34 -04:00
bvandeusen 0af38016cd feat: traefik plugin migration (traefik_001_initial, depends_on core head)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 21:34:40 -04:00
bvandeusen 6fde57f63e feat: traefik plugin scaffold (plugin.yaml, model, __init__ stub) 2026-03-17 21:32:14 -04:00