41 Commits

Author SHA1 Message Date
bvandeusen b49496b57b feat(plugins): default-enable generic bundled plugins on fresh install
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Failing after 2m16s
CI / publish (push) Has been skipped
A fresh install enabled zero plugins — settings.py DEFAULTS had no plugin.*
keys, so to_plugins_cfg returned {} and every plugin had to be flipped on
by hand. Seed docker, host_agent, http, snmp as default-on (generic, non-
vendor-specific); traefik and unifi stay opt-in. Stored choices override
the default, so disabling persists.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 21:24:21 -04:00
bvandeusen 80613d6310 chore(lint): remove unused imports flagged by ruff
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 18:55:21 -04:00
bvandeusen 923905e72d test(ansible): integration tests for inventory tables + fetch_scope_targets 2026-06-05 18:52:10 -04:00
bvandeusen dd0acaf623 feat(ansible): host edit page Ansible target link/create/unlink section 2026-06-05 18:51:33 -04:00
bvandeusen 37d9ca8a8a feat(ansible): inventory targets CRUD templates 2026-06-05 18:50:40 -04:00
bvandeusen 935cbc105c feat(ansible): inventory groups CRUD routes + templates + blueprint registration 2026-06-05 18:50:09 -04:00
bvandeusen 2c991eabd2 feat(ansible): scope picker in run form + create_run() DB inventory generation 2026-06-05 18:49:02 -04:00
bvandeusen d906232eb2 feat(ansible): GIT_ASKPASS per-source HTTP token auth + settings UI 2026-06-05 18:48:00 -04:00
bvandeusen ea47169049 feat(ansible): generate_inventory() + fetch_scope_targets() + unit tests 2026-06-05 18:46:22 -04:00
bvandeusen 10df05c13e feat(ansible): add inventory_scope to AnsibleRun — 0017_ansible_run_scope migration 2026-06-05 18:45:38 -04:00
bvandeusen fa32488fdf feat(ansible): AnsibleGroup + join table — 0016_ansible_inventory_groups migration 2026-06-05 18:45:16 -04:00
bvandeusen c11b3f01ed feat(ansible): AnsibleTarget model + 0015_ansible_inventory_targets migration 2026-06-05 18:45:00 -04:00
bvandeusen eb319d715e docs: Ansible inventory infrastructure implementation plan 2026-06-05 18:19:17 -04:00
bvandeusen 06dc5073ca fix(deploy): explicit backend network for steward↔db traffic
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 6s
CI / integration (push) Successful in 2m13s
CI / publish (push) Successful in 5s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 11:37:10 -04:00
bvandeusen 98edb12abb fix(deploy): let app own secret key — remove STEWARD_SECRET_KEY from deploy compose
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m13s
CI / publish (push) Successful in 4s
The app auto-generates a key on first boot and persists it to /data/secret.key,
which the app_data volume keeps across image updates. No reason to surface this
as a user-facing env var in the deploy compose. Removed from .env.example too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 11:28:53 -04:00
bvandeusen ef6c90c022 Revert "fix(deploy): move POSTGRES_PASSWORD default into .env.example"
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m14s
CI / publish (push) Successful in 5s
This reverts commit 42d9a7ba07.
2026-06-05 11:25:31 -04:00
bvandeusen 42d9a7ba07 fix(deploy): move POSTGRES_PASSWORD default into .env.example
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m13s
CI / publish (push) Successful in 4s
Plain ${POSTGRES_PASSWORD} in compose; default 'steward' lives in .env.example
where it's visible and editable rather than buried in compose YAML.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 11:24:29 -04:00
bvandeusen fe57dde57b feat(ci): publish steward image to the registry + remote-deploy compose
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m13s
CI / publish (push) Successful in 1m12s
Add the rule-46 image-publish lane: a `publish` job gated on
[lint, unit, integration] that builds the runtime Dockerfile in ci-builder
and pushes git.fabledsword.com/bvandeusen/steward:<sha> always, :dev on dev
and :latest on main. Authenticates with the REGISTRY_TOKEN repo secret
(the injected GITHUB_TOKEN lacks write:package). Steward is the first family
app to implement the publish lane; mechanics mirror CI-runner's
build-ci-python.yml.

Add compose.deploy.yml for a persistent remote instance that pulls :dev
(no source bind-mounts, persistent app_data+pgdata volumes, bundled Postgres),
plus a README "Remote dev instance" runbook and the POSTGRES_PASSWORD env key.
Fix .env.example's database URL var to the canonical STEWARD_DATABASE_URL
(single underscore) and document the publish lane in ci-requirements.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 11:08:43 -04:00
bvandeusen 38f61b71c1 feat(ansible): run a playbook against a single Steward host (task 547)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Successful in 2m19s
From a host's edit page, an operator can run a playbook against just that host
via an ephemeral one-host inventory — no need for the host to exist in a source
inventory.

- ansible/sources.py: pure host_inventory_content(host) -> '<name> ansible_host=<addr>'
- executor.start_run: optional inventory_content written to the temp dir and used
  as -i (overrides inventory_path); cleaned up with the creds dir
- hosts/routes.py: POST /hosts/<id>/run-playbook (operator) — validates source +
  playbook, builds the ephemeral inventory, starts a user-triggered run, redirects
  to the live run detail; edit page gets the ansible source list
- hosts/form.html: 'Run Ansible playbook against this host' panel (shown when
  editing an existing host and sources exist) — source + playbook + extra-vars/
  tags/dry-run (no --limit; single host)
- tests: unit host_inventory_content; integration runs a hosts:all/connection:local
  playbook against the ephemeral inventory and asserts inventory_hostname

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 19:26:58 -04:00
bvandeusen 0bf007173b feat(ansible): credentials — SSH key, become, vault, host-key checking (task 548)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m24s
Global Ansible credentials, applied to every run (manual + alert-triggered):
- core/settings.py: ansible.ssh_private_key / become_password / vault_password
  (plaintext at rest, masked in UI — encryption tracked in #580) + host_key_checking
  (default off); surfaced via to_ansible_cfg into app.config[ANSIBLE]
- executor.py: pure build_credentials() materializes creds into a 0600 temp dir
  (--private-key, --vault-password-file, become via -e @vars-file so the password
  never hits argv) cleaned up in finally; pure ansible_env() sets
  ANSIBLE_HOST_KEY_CHECKING. build_ansible_command stays param-only
- settings/routes.py + ansible.html: admin-only Credentials section, masked-update
  (blank keeps current, explicit Clear checkbox), reload app config on save
- tests: unit (build_credentials per cred + none; ansible_env toggle); integration
  vault round-trip (ansible-vault encrypt a vars file, run via executor with the
  vault password, assert it decrypted)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 18:41:58 -04:00
bvandeusen 0a36a57901 chore(dev): mount ./ansible-dev as a local Ansible source at /srv/ansible
Dev convenience so you can author playbooks on the host and have Steward's
local-source discovery pick them up. ./ansible-dev/ is gitignored (operator
content); a safe demo playbook lives there locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 17:58:18 -04:00
bvandeusen 534ed030b8 test(alerts): commit the test user before the rule's created_by FK
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m14s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 14:31:25 -04:00
bvandeusen 43a5325e69 fix(alerts): persist AlertOperator by value to match the DB enum
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Failing after 2m16s
The alertoperator Postgres enum (migration 0002) was created with the operator
VALUES as labels ('>', '<', …), but the model's Enum(AlertOperator) defaulted to
persisting enum-member NAMES ('gt', 'lt', …) — so inserting any AlertRule against
real Postgres raised 'invalid input value for enum alertoperator: "gt"'. Never
caught because unit tests mock the DB and no integration test inserted a rule
until task 250's. Add values_callable so the ORM round-trips the values.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 14:27:25 -04:00
bvandeusen 4771d17f6d feat(alerts): run an Ansible playbook on alert firing (task 250)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 8s
CI / integration (push) Failing after 2m16s
Rebuilds the deleted NUT/UPS automation as a general alert action: any metric
can drive a playbook run on transition-to-firing.

- models/alerts.py + migration 0014: AlertRule.ansible_action (JSON, admin-only,
  reuses the #546 param shape); AlertEvent.ansible_run_id links a firing event to
  the run it triggered
- core/alerts.py: pure alert_extra_vars() injects steward_alert_* context; on
  ('firing', event) with an action set, schedule _run_ansible_action (deferred
  after commit, same pattern as notifications) — fires once per transition,
  consecutive_failures_required is the debounce; system-triggered AnsibleRun
- alerts/routes.py: admin-only parse/validate of the action (source must be
  configured, playbook must exist); operators keep editing rules, action preserved
- rules_form.html: admin-only 'On firing -> run a playbook' section
- tests: unit for alert_extra_vars; integration drives record_metric to firing
  and asserts a system AnsibleRun ran the playbook with the injected var and the
  event linked to it

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 14:22:29 -04:00
bvandeusen 8b62eb2ca3 feat(ansible): parameterized runs — extra-vars, limit, tags, dry-run (task 546)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m16s
Run form + executor now support optional params, passed as argv (never
shell-interpolated):
- extra_vars: one key=value per line -> repeated -e
- limit -> --limit; tags -> --tags
- dry-run checkbox -> --check --diff

- executor.py: pure build_ansible_command(playbook, inventory, params); start_run
  gains an optional params arg (backward-compatible)
- models/ansible.py + migration 0013: nullable params JSON column
- routes.py: create_run parses + validates (extra-var lines need '='), stores
  params on the run, passes to the executor
- browse.html run form: Extra vars / Limit / Tags / Dry-run fields
- run_detail.html: shows the params used
- tests/test_ansible_command.py: unit coverage of the builder; integration test
  runs a real playbook with extra-var + limit + check and asserts substitution

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 13:31:16 -04:00
bvandeusen 5af7312a72 fix(ansible): shorten migration 0012 revision id to fit alembic_version(32)
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m21s
The id 0012_ansible_run_triggered_by_nullable was 38 chars; alembic_version.
version_num is VARCHAR(32), so stamping it raised StringDataRightTruncationError
and the boot/migrate integration lane failed. Renamed to 0012_ansible_run_nullable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 11:35:55 -04:00
bvandeusen fb579bcf97 feat(ansible): ship ansible in image + allow system-triggered runs (task 545)
CI / lint (push) Successful in 4s
CI / unit (push) Successful in 9s
CI / integration (push) Failing after 2m22s
Foundation for the Ansible automation milestone (#37) — makes the existing
manual playbook runner actually executable and the schema automation-ready.

- pyproject: [ansible] extra (full ansible package, batteries-included, pinned)
- Dockerfile: pip install .[ansible]; add openssh-client for remote runs
- models/ansible.py + migration 0012: AnsibleRun.triggered_by now nullable so
  automated (alert/schedule) runs need no human actor
- ansible/routes.py + run_detail.html: show 'Triggered by' (username or 'system')
- CI: integration lane installs .[dev,ansible]; new tests/integration/
  test_ansible_foundation.py runs a real connection:local playbook end-to-end,
  asserts success+output, and round-trips a NULL-triggered run

No extra-vars/limit/credentials/scheduling here — those are their own #37 tasks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 11:31:43 -04:00
bvandeusen ccc7601182 fix(docker): stop ignoring plugins/ in image build; dev-friendly compose
.dockerignore excluded plugins/ (leftover from the separate-repo era), so the
Dockerfile's COPY plugins/ found nothing and the build failed. Track it now that
first-party plugins ship in the image.

docker-compose.yml reworked for local dev: live-mounted ./steward + ./plugins
with PYTHONPATH=/app, --debug auto-reload, fixed dev SECRET_KEY, Postgres port
exposed, and the host-specific traefik/docker.sock mounts commented out.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 10:35:58 -04:00
bvandeusen 712aec60c1 test(host_agent): assert parsed agent version matches AGENT_VERSION constant
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m11s
The on-disk-version test hardcoded '1.0.0' and broke on the 1.1.0 bump. Compare
against agent.AGENT_VERSION so future bumps don't require touching the test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 12:14:53 -04:00
bvandeusen 3b2146bc7a feat(host_agent): agent reports its primary IP; mirror into Host.address (task 274)
CI / lint (push) Successful in 3s
CI / unit (push) Failing after 8s
CI / integration (push) Successful in 2m9s
The agent now detects its own primary non-loopback IP (UDP-socket trick, no
packets sent) and sends it in the metadata bag; bumps AGENT_VERSION 1.0.0->1.1.0
and re-detects on SIGHUP. The server persists it on HostAgentRegistration.host_ip
and mirrors it into Host.address ONLY when that field is blank — an admin-typed
address is never overwritten. The reported IP shows in the settings table so
admins can see drift regardless.

- agent.py: detect_primary_ip(); host_ip in collect_metadata(); refresh on reload
- routes.py: pure pick_host_address() helper; ingest persists host_ip + mirrors
- models.py + migration host_agent_002_host_ip: new String(45) column
- settings_list.html: Reported IP column
- plugin.yaml: 1.0.0 -> 1.1.0
- tests: pick_host_address (fill-when-blank / never-clobber / no-op),
  detect_primary_ip (never raises, non-loopback), metadata carries host_ip

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 12:11:41 -04:00
bvandeusen af60ca446d fix(plugins): complete steward rename across bundled plugins; clear lint debt
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m8s
The fabledscryer->steward rename had only ever reached host_agent. The other
five bundled plugins (http, snmp, traefik, unifi, docker) still imported
`from fabledscryer.*` (package no longer exists) and read FABLEDSCRYER_* env
vars — so every one of them was broken at import since the original rebrand.
CI stayed green only because none are enabled by default and migrations don't
import plugin modules. Now that they version in-tree, complete the rename:
- fabledscryer.* -> steward.* imports across all five plugins
- FABLEDSCRYER_* -> STEWARD_* in plugin migration env.py files
- author/repository/homepage + user-facing 'Fabled Scryer' strings -> Steward
- snmp/scheduler.py: also drop dead `now`/datetime; record_metric from steward

Adds tests/test_no_legacy_names.py — fails if 'scryer'/'roundtable' ever
reappear in shipped code (the drift bit twice; this stops a third time).

Also clears pre-existing ruff lint debt (unused imports, semicolon statements,
mid-file import) surfaced by the new lint lane.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 11:22:20 -04:00
bvandeusen d925709c77 ci: add Forgejo CI (lint + unit + Postgres integration) + ci-requirements.md
CI / lint (push) Failing after 2s
CI / unit (push) Successful in 7s
CI / integration (push) Successful in 2m8s
Single-repo CI now that plugins are bundled in-tree. Three lanes on push to
dev/main, modeled on FabledCurator's canonical ci.yml (minus frontend/Redis):
- lint: ruff check steward/ plugins/ tests/ (no dep install)
- unit: pytest -m 'not integration' — whole current suite (testing=True mocks DB)
- integration: postgres:16-alpine service via socket-discovered bridge IP; a
  boot-and-migrate test creates the real app, running core + all bundled-plugin
  migrations, then round-trips the DB — guards the folded-in migration graph.

Registers the 'integration' pytest marker; adds tests/integration/test_boot.py.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 08:37:32 -04:00
bvandeusen a7a281cb11 feat(plugins): fold first-party plugins in-tree; bundled + external roots
First-party plugins (host_agent, http, snmp, traefik, unifi, docker) are now
tracked under plugins/ and baked into the image, so they version atomically
with core — ending the cross-repo import drift the roundtable->steward rename
exposed. History for these files is preserved in the archived Roundtable-plugins
repo.

Plugin discovery becomes multi-root: PLUGIN_DIR (single) -> PLUGIN_DIRS
(bundled first, then external) + PLUGIN_INSTALL_DIR. Bundled ships in the image;
third-party plugins still mount at runtime into the external root
(STEWARD_PLUGIN_DIR, default /data/plugins) and downloads/installs land there.
Bundled shadows external on a name collision.

- config.py: load_bootstrap returns plugin_dirs + plugin_install_dir
- app.py: iterate PLUGIN_DIRS at the migration + load sites
- migration_runner.py: discover_all_in() unions every plugin root
- plugin_manager.py: resolve_plugin_path() (pure, first-root-wins); load /
  install / hot-reload span all roots; installs target the external root
- settings/routes.py: _discover_plugins scans all roots, dedup bundled-first
- Dockerfile: COPY plugins/ ; docker-compose: drop host bind, document external
- tests/test_plugin_dirs.py: resolution, multi-root discovery, bootstrap split

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 08:37:24 -04:00
bvandeusen 88ab5b917e chore: rename project Roundtable → Steward
Renames the Python package directory, CLI command, env var prefix,
docker-compose service/container/image, Postgres role/db, and all
visible branding. Marketing form is "Fabled Steward".

Clean break from the previous rebrand: drops the fabledscryer→roundtable
import shim in __init__.py and the FABLEDSCRYER_* env var fallback in
config.py and migrations/env.py. Env vars are now STEWARD_* only.

Heads-up for existing deployments:
- Postgres user/db renamed fabledscryer → steward in docker-compose.yml.
  Existing volumes need the role/db renamed inside Postgres, or override
  POSTGRES_USER/POSTGRES_DB to keep the old names.
- Host-agent systemd unit is now steward-agent.service. Existing agents
  keep running under the old name; reinstall to switch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 16:20:14 -04:00
bvandeusen e118543b2e feat(dashboard): use public_base_url helper for share link generation 2026-04-16 00:18:45 -04:00
bvandeusen b7293588be feat(settings): public_base_url field in general settings UI 2026-04-15 20:30:12 -04:00
bvandeusen 5bc8ef2566 feat(settings): add public_base_url helper with request.host_url fallback 2026-04-15 20:29:49 -04:00
bvandeusen 3a3d094a2a test(settings): failing tests for public_base_url helper 2026-04-15 20:29:28 -04:00
bvandeusen 2c68ba5094 feat(settings): cache PUBLIC_BASE_URL into app.config on reload 2026-04-15 20:29:13 -04:00
bvandeusen fdbcc58385 feat(settings): add general.public_base_url default 2026-04-15 20:29:02 -04:00
bvandeusen dbe50794d3 Merge pull request 'feat: host_agent plugin — push-model host resource monitoring' (#1) from feat/host-agent into main 2026-04-15 13:43:57 +00:00
279 changed files with 14015 additions and 824 deletions
+3 -2
View File
@@ -26,8 +26,9 @@ config.yaml
# Runtime data (mounted at runtime via volume)
playbook_cache/
# Plugins (mounted at runtime via volume)
plugins/
# NOTE: first-party plugins under plugins/ now ship IN the image
# (Dockerfile `COPY plugins/`), so plugins/ must NOT be ignored here.
# Third-party plugins are mounted at runtime into /data/plugins instead.
# Deployment files not needed in image
docker-compose.yml
+8 -3
View File
@@ -1,6 +1,11 @@
# Copy to .env for Docker / local development secrets
# These override values in config.yaml
ROUNDTABLE_SECRET_KEY=change-me
ROUNDTABLE_DATABASE__URL=postgresql+asyncpg://roundtable:password@localhost/roundtable
ROUNDTABLE_SMTP__PASSWORD=
STEWARD_DATABASE_URL=postgresql+asyncpg://steward:password@localhost/steward
STEWARD_SMTP__PASSWORD=
# --- compose.deploy.yml (remote instance) only ---
# Password for the bundled Postgres; the steward service builds its
# STEWARD_DATABASE_URL from it. Leave STEWARD_SECRET_KEY above unset to let the
# app generate + persist one on the /data volume.
POSTGRES_PASSWORD=change-me
+129
View File
@@ -0,0 +1,129 @@
name: CI
# CI lanes per FabledRulebook forgejo.md "CI philosophy":
# - lint: ruff only, no dep install — fast-fail for the common lint bounce.
# - unit: pytest -m "not integration"; no service containers, no DB.
# - integration: postgres service container; boots the real app + runs all
# (core + bundled-plugin) migrations.
# Single repo: first-party plugins are bundled in-tree under plugins/, so the
# unit lane imports them directly — no cross-repo checkout.
on:
push:
branches: [dev, main]
# pull_request intentionally absent — push on [dev, main] already fires CI for
# every dev commit and dev→main PRs. Single-operator repo, no fork PRs.
jobs:
# Fast-fail lint lane. ruff is pre-installed in the ci-python image, so this
# runs with NO dependency install and surfaces lint bounces in seconds.
lint:
runs-on: python-ci
container:
image: git.fabledsword.com/bvandeusen/ci-python:3.14
steps:
- uses: actions/checkout@v4
- name: Ruff lint
run: ruff check steward/ plugins/ tests/
unit:
runs-on: python-ci
container:
image: git.fabledsword.com/bvandeusen/ci-python:3.14
steps:
- uses: actions/checkout@v4
- name: Install Steward + test deps
# uv: 5-10x faster wheel resolve than pip on cold caches. Falls back to
# pip on uv-missing runners. Steward declares its deps in pyproject; the
# [dev] extra adds pytest + pytest-asyncio.
run: |
if command -v uv >/dev/null 2>&1; then
uv pip install --system -e '.[dev]'
else
pip install -e '.[dev]'
fi
- name: Pytest (unit only — integration runs in its own lane)
run: pytest tests/ -v -m "not integration"
integration:
runs-on: python-ci
container:
image: git.fabledsword.com/bvandeusen/ci-python:3.14
env:
STEWARD_SECRET_KEY: ci_integration_placeholder
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: steward
POSTGRES_PASSWORD: steward
POSTGRES_DB: steward
options: >-
--health-cmd "pg_isready -U steward"
--health-interval 10s
--health-timeout 5s
--health-retries 10
steps:
- uses: actions/checkout@v4
- name: Boot + migrate against Postgres
# act_runner (swarm-runner) puts service containers on the default bridge
# with no embedded DNS, so the hostname `postgres` won't resolve — we
# discover the service container's bridge IP via the docker socket. The
# job name `integration` (no underscore) is the docker-ps name filter.
run: |
set -euxo pipefail
echo "=== container landscape (diagnostic for filter scoping) ==="
docker ps -a --format '{{.ID}} {{.Image}} -> {{.Names}}'
echo "=== end landscape ==="
PG=$(docker ps --filter "name=integration" --filter "ancestor=postgres:16-alpine" -q | head -n1)
test -n "$PG"
PG_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$PG")
test -n "$PG_IP"
export STEWARD_DATABASE_URL="postgresql+asyncpg://steward:steward@$PG_IP:5432/steward"
for i in $(seq 1 60); do
(echo > "/dev/tcp/$PG_IP/5432") >/dev/null 2>&1 && break
sleep 2
done
if command -v uv >/dev/null 2>&1; then
uv pip install --system -e '.[dev,ansible]'
else
pip install -e '.[dev,ansible]'
fi
pytest tests/integration -v -m integration --durations=10
# Image-publish lane — FabledRulebook rule 46: every push to dev/main publishes
# an immutable commit-SHA image (the rollback unit); dev moves :dev, main moves
# :latest. Gated on the three test lanes via `needs:` so a red commit never
# produces a :dev image. Mechanics mirror CI-runner's build-ci-python.yml.
publish:
needs: [lint, unit, integration]
runs-on: python-ci
container:
# ci-builder carries docker + buildx + git. The docker socket is
# auto-mounted by act_runner (valid_volumes) — do NOT add a
# container.options "-v /var/run/docker.sock:..." mount; that duplicate
# mount fails the job at "Set up job".
image: git.fabledsword.com/bvandeusen/ci-builder:latest
steps:
- uses: actions/checkout@v4
- name: Registry login
# The injected GITHUB_TOKEN lacks write:package scope, so docker push
# 401s with it. REGISTRY_TOKEN is a repo Actions secret holding a
# Forgejo token scoped read:package + write:package.
run: echo "${{ secrets.REGISTRY_TOKEN }}" | docker login git.fabledsword.com -u bvandeusen --password-stdin
- name: Build + push (commit-SHA always; :dev on dev, :latest on main)
run: |
set -euxo pipefail
IMAGE=git.fabledsword.com/bvandeusen/steward
docker build -t "$IMAGE:${{ github.sha }}" .
docker push "$IMAGE:${{ github.sha }}"
if [ "${{ github.ref }}" = "refs/heads/dev" ]; then
docker tag "$IMAGE:${{ github.sha }}" "$IMAGE:dev"
docker push "$IMAGE:dev"
elif [ "${{ github.ref }}" = "refs/heads/main" ]; then
docker tag "$IMAGE:${{ github.sha }}" "$IMAGE:latest"
docker push "$IMAGE:latest"
fi
- name: Prune dangling layers
if: always()
run: docker image prune -f
+10 -3
View File
@@ -1,9 +1,10 @@
# Planning files
docs/superpowers/
# Plugin directory — managed as a separate repo (bvandeusen/roundtable-plugins)
# PLUGIN_DIR in config.yaml points here at runtime
/plugins/
# First-party plugins are tracked in-tree under plugins/ and ship in the image.
# Third-party plugins are mounted at runtime into the external dir
# (STEWARD_PLUGIN_DIR, default /data/plugins) and are not tracked here.
/plugins/**/*.zip
# Python
__pycache__/
@@ -50,3 +51,9 @@ Thumbs.db
# Playbook cache (runtime data)
playbook_cache/
# Superpowers brainstorm scratch (visual companion mockups, state)
.superpowers/
# Local dev Ansible playbooks (operator content, mounted into the container)
/ansible-dev/
+11 -4
View File
@@ -5,6 +5,8 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update \
iputils-ping \
# gosu — minimal privilege-drop tool used by entrypoint.sh
gosu \
# openssh-client — ansible-playbook reaches remote hosts over ssh
openssh-client \
&& rm -rf /var/lib/apt/lists/*
# Upgrade pip to suppress the version notice
@@ -13,16 +15,21 @@ RUN pip install --upgrade pip
WORKDIR /app
COPY pyproject.toml .
COPY roundtable/ roundtable/
RUN pip install --no-cache-dir .
COPY steward/ steward/
# .[ansible] pulls the full Ansible package so the playbook runner works in-image.
RUN pip install --no-cache-dir '.[ansible]'
COPY alembic.ini .
# First-party plugins ship inside the image (bundled root at /app/plugins).
# Third-party plugins are mounted at runtime into the external root
# (STEWARD_PLUGIN_DIR, default /data/plugins).
COPY plugins/ plugins/
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
# Runtime directories — owned by app user. The container starts as root
# so the entrypoint can fix up /data permissions if needed, then drops to
# 'app' (uid 1000) via gosu before launching roundtable.
# 'app' (uid 1000) via gosu before launching steward.
RUN useradd -m -u 1000 app \
&& mkdir -p /data/playbook_cache /data/plugins \
&& chown -R app:app /data /app
@@ -30,4 +37,4 @@ RUN useradd -m -u 1000 app \
EXPOSE 5000
ENTRYPOINT ["/entrypoint.sh"]
CMD ["roundtable", "--host", "0.0.0.0", "--port", "5000"]
CMD ["steward", "--host", "0.0.0.0", "--port", "5000"]
+41 -5
View File
@@ -1,6 +1,6 @@
# Roundtable
# Steward
A self-hosted network monitoring and infrastructure management hub for home servers. Roundtable gives you a single pane of glass over your hosts, services, and automation — with live-updating dashboards, alerting, and Ansible integration.
A self-hosted network monitoring and infrastructure management hub for home servers. Steward gives you a single pane of glass over your hosts, services, and automation — with live-updating dashboards, alerting, and Ansible integration.
---
@@ -21,13 +21,49 @@ No JavaScript framework. No build step. No external workers or message brokers.
```bash
cp .env.example .env
# Set ROUNDTABLE_DATABASE_URL in .env
# Set STEWARD_DATABASE_URL in .env
docker compose up -d
```
Open `http://localhost:5000`. On first run you'll be prompted to create the admin account.
This compose file builds the image locally and bind-mounts the source for live editing. To run a persistent instance off the published CI image instead, see **Remote dev instance** below.
---
## Remote dev instance
For a long-lived instance on a remote server that tracks the `dev` CI line, use `compose.deploy.yml`. It pulls the published image (`git.fabledsword.com/bvandeusen/steward:dev`) rather than building, runs no source bind-mounts, and persists data in named volumes.
**One-time prerequisite — registry push token (on the Git host).** CI publishes the image, so the FabledSteward repo needs a push credential. The injected `GITHUB_TOKEN` lacks `write:package`, so create a Forgejo token scoped **`read:package` + `write:package`** and add it as the repo Actions secret **`REGISTRY_TOKEN`**. Until this exists, the `publish` CI job 401s and no `:dev` image is produced.
**Standup (on the server):**
```bash
# 1. Authenticate to the registry (read:package token is enough to pull)
docker login git.fabledsword.com -u <user>
# 2. Configure secrets
cp .env.example .env
# Set POSTGRES_PASSWORD. Leave STEWARD_SECRET_KEY unset to let the app
# generate + persist one on the /data volume.
# 3. Pull + boot
docker compose -f compose.deploy.yml up -d
```
Open `http://<server>:5000` and create the admin account on first run.
**Update to the latest dev build:**
```bash
docker compose -f compose.deploy.yml pull
docker compose -f compose.deploy.yml up -d
```
Every push to `dev` that goes CI-green publishes a fresh `:dev` (and an immutable `:<commit-sha>` for rollback). To pin a specific commit, change the `image:` tag in `compose.deploy.yml` to `:<sha>`.
---
## Quick Start — Bare Metal
@@ -41,7 +77,7 @@ pip install -e .
cp config.example.yaml config.yaml
# Edit config.yaml — set database.url at minimum
roundtable --host 0.0.0.0 --port 5000
steward --host 0.0.0.0 --port 5000
```
ICMP ping requires `CAP_NET_RAW` or the `setuid` ping binary. TCP mode (the default) needs no elevated privileges.
@@ -54,7 +90,7 @@ Only two things must be set to run the app:
| What | How |
|---|---|
| Database URL | `ROUNDTABLE_DATABASE_URL` env var or `database.url` in `config.yaml` |
| Database URL | `STEWARD_DATABASE_URL` env var or `database.url` in `config.yaml` |
| Secret key | Auto-generated on first run and saved to `/data/secret.key` |
Everything else — SMTP, webhooks, monitor intervals, Ansible sources, plugin settings — is configured through the web UI at `/settings/`.
+1 -1
View File
@@ -1,5 +1,5 @@
[alembic]
script_location = roundtable/migrations
script_location = steward/migrations
prepend_sys_path = .
version_path_separator = os
+58
View File
@@ -0,0 +1,58 @@
# CI Requirements — Steward
> Spec: https://git.fabledsword.com/bvandeusen/CI-runner/src/branch/main/docs/process.md
## Runtime image
git.fabledsword.com/bvandeusen/ci-python:3.14
## Image deps used
- python 3.14
- ruff (analyzer for `steward/`, `plugins/`, `tests/`)
- docker CLI (integration lane: bridge-IP discovery of the Postgres service container)
## Per-job tool installs
- `uv pip install --system -e '.[dev]'` (pip fallback) — in the `unit` job.
- `uv pip install --system -e '.[dev,ansible]'` (pip fallback) — in the
`integration` job, which boots the real app and exercises the Ansible runner,
so it needs the `ansible` extra (`ansible>=10,<13`) at import time.
Steward declares its deps in `pyproject.toml`; the `[dev]` extra adds
`pytest` + `pytest-asyncio`. No `requirements.txt` is tracked.
## Lanes
- **lint** — `ruff check steward/ plugins/ tests/`, no dep install.
- **unit** — `pytest -m "not integration"`. The whole current suite runs here:
it builds the app with `testing=True`, which mocks `db_sessionmaker`, so no DB
is touched. First-party plugins are bundled in-tree under `plugins/`, so plugin
unit tests import them directly — no cross-repo checkout.
- **integration** — `pytest tests/integration -m integration` against a
`postgres:16-alpine` service. The single test boots the real app
(`create_app(testing=False)`), which runs core + every bundled plugin's
migrations, then round-trips the DB. This is the guard that the folded-in
plugin migration graph resolves.
- **publish** — builds the runtime `Dockerfile` and pushes to the package
registry. Runs in `ci-builder:latest` (docker + buildx, socket auto-mounted),
gated on `needs: [lint, unit, integration]` so a red commit never ships an
image. Publishes `git.fabledsword.com/bvandeusen/steward:<sha>` always, plus
`:dev` on `dev` and `:latest` on `main` (FabledRulebook rule 46). Needs the
`REGISTRY_TOKEN` repo Actions secret (`read:package` + `write:package`) —
the injected `GITHUB_TOKEN` can't push packages.
## Notes
- Steward has **no frontend and no Redis/Celery** (no external workers), so there
is no frontend-build lane and the only service container is Postgres.
- Integration uses Forgejo Actions `services:` + a socket-discovered bridge IP
because `act_runner` (swarm-runner v0.6+) puts services on the default bridge
with no embedded DNS. FabledCurator's `ci.yml` is the canonical example of the
pattern; Steward's is the same shape minus Redis and minus sharding.
- The job name `integration` has no underscore — `act_runner` strips underscores
from job names when building service-container labels, so the `docker ps`
name filter uses the bare job name.
- Migrations run via the app itself (`create_app``run_core_migrations`, async
through `asyncpg`), so the integration lane needs no separate `alembic upgrade`
step and no psycopg/sync driver.
+59
View File
@@ -0,0 +1,59 @@
# Remote deployment — pulls the published :dev image instead of building locally.
#
# This is NOT the local-dev compose. `docker-compose.yml` bind-mounts ./steward
# + ./plugins and runs --debug for live editing; THIS file runs the image as the
# source of truth (no bind-mounts, no reloader) for a persistent remote instance
# that tracks the dev CI line.
#
# Standup + update runbook: see README → "Remote dev instance".
services:
steward:
image: git.fabledsword.com/bvandeusen/steward:dev
container_name: steward
# Re-pull :dev on every `up` so restarting the stack picks up the latest
# green dev build without a manual `docker pull`.
pull_policy: always
ports:
- "5000:5000"
networks:
- backend
volumes:
# /data holds the auto-generated secret key, third-party plugins, and the
# Ansible playbook cache — all of which must survive image updates. No
# source bind-mounts: core + first-party plugins live inside the image.
- app_data:/data
environment:
- STEWARD_DATABASE_URL=postgresql+asyncpg://steward:${POSTGRES_PASSWORD:-steward}@db/steward
# STEWARD_SECRET_KEY is intentionally absent — the app generates a random
# key on first boot and persists it to /data/secret.key, which the app_data
# volume keeps across image updates.
depends_on:
db:
condition: service_healthy
restart: unless-stopped
db:
image: postgres:16-alpine
networks:
- backend
environment:
POSTGRES_USER: steward
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-steward}
POSTGRES_DB: steward
volumes:
- pgdata:/var/lib/postgresql/data
# No host port published — db is reachable only on the backend network.
healthcheck:
test: ["CMD-SHELL", "pg_isready -U steward"]
interval: 5s
timeout: 5s
retries: 5
restart: unless-stopped
networks:
backend:
volumes:
pgdata:
app_data:
+3 -3
View File
@@ -1,9 +1,9 @@
# Roundtable — Bootstrap Configuration Example
# Steward — Bootstrap Configuration Example
#
# The only REQUIRED setting is the database URL.
# Set it via env var (recommended for Docker):
#
# ROUNDTABLE_DATABASE_URL=postgresql+asyncpg://user:pass@host/db
# STEWARD_DATABASE_URL=postgresql+asyncpg://user:pass@host/db
#
# All other settings (SMTP, webhook, ansible, plugins, retention)
# are stored in the database and managed via the Settings UI at /settings/.
@@ -11,7 +11,7 @@
# config.yaml is optional. Use it only if you prefer file-based bootstrap.
database:
url: "postgresql+asyncpg://roundtable:password@localhost/roundtable"
url: "postgresql+asyncpg://steward:password@localhost/steward"
# Optional: override the auto-generated secret key.
# If not set, a key is auto-generated on first run and saved to /data/secret.key.
+28 -11
View File
@@ -1,17 +1,31 @@
services:
roundtable:
steward:
build: .
container_name: roundtable
image: roundtable:latest
container_name: steward
image: steward:latest
# Dev: --debug turns on Quart's auto-reloader so edits to the mounted source
# below take effect without a rebuild.
command: ["steward", "--host", "0.0.0.0", "--port", "5000", "--debug"]
ports:
- "5000:5000"
volumes:
- app_data:/data
- ./plugins:/app/plugins:ro
- /mnt/Data/traefik/log:/var/log/traefik:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
# Live-edit core + first-party plugins without rebuilding. PYTHONPATH=/app
# (below) makes this mounted source win over the image-installed package.
- ./steward:/app/steward
- ./plugins:/app/plugins
# Dev playbooks: edit on the host under ./ansible-dev/, register a *local*
# Ansible source pointing at /srv/ansible in Settings → Ansible.
- ./ansible-dev:/srv/ansible
# Optional host mounts — enable the matching plugin in Settings first, then
# uncomment (the traefik path must exist on this host):
# - /mnt/Data/traefik/log:/var/log/traefik:ro # traefik plugin
# - /var/run/docker.sock:/var/run/docker.sock:ro # docker plugin
environment:
- ROUNDTABLE_DATABASE_URL=postgresql+asyncpg://fabledscryer:fabledscryer@db/fabledscryer
- STEWARD_DATABASE_URL=postgresql+asyncpg://steward:steward@db/steward
# Dev-only fixed key so sessions survive restarts. Replace in production.
- STEWARD_SECRET_KEY=dev-secret-key-change-me
- PYTHONPATH=/app
depends_on:
db:
condition: service_healthy
@@ -20,13 +34,16 @@ services:
db:
image: postgres:16-alpine
environment:
POSTGRES_USER: fabledscryer
POSTGRES_PASSWORD: fabledscryer
POSTGRES_DB: fabledscryer
POSTGRES_USER: steward
POSTGRES_PASSWORD: steward
POSTGRES_DB: steward
ports:
# Exposed for dev convenience (psql from the host). Drop for prod.
- "5432:5432"
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U fabledscryer"]
test: ["CMD-SHELL", "pg_isready -U steward"]
interval: 5s
timeout: 5s
retries: 5
+13 -13
View File
@@ -1,12 +1,12 @@
# Architecture
Roundtable is a single Quart (async Python) process. There are no separate worker processes, no message brokers, and no build pipeline for the frontend. All monitoring, scheduling, and request handling runs on a single asyncio event loop.
Steward is a single Quart (async Python) process. There are no separate worker processes, no message brokers, and no build pipeline for the frontend. All monitoring, scheduling, and request handling runs on a single asyncio event loop.
---
## Startup Sequence
`create_app()` in `roundtable/app.py` runs these steps synchronously before the event loop starts:
`create_app()` in `steward/app.py` runs these steps synchronously before the event loop starts:
| Step | What happens |
|---|---|
@@ -32,14 +32,14 @@ All routes are Quart Blueprints registered in `app.py`:
| Prefix | Blueprint | Module |
|---|---|---|
| `/auth/` | `auth_bp` | `roundtable/auth/routes.py` |
| `/` | `dashboard_bp` | `roundtable/dashboard/routes.py` |
| `/hosts/` | `hosts_bp` | `roundtable/hosts/routes.py` |
| `/ping/` | `ping_bp` | `roundtable/ping/routes.py` |
| `/dns/` | `dns_bp` | `roundtable/dns/routes.py` |
| `/alerts/` | `alerts_bp` | `roundtable/alerts/routes.py` |
| `/ansible/` | `ansible_bp` | `roundtable/ansible/routes.py` |
| `/settings/` | `settings_bp` | `roundtable/settings/routes.py` |
| `/auth/` | `auth_bp` | `steward/auth/routes.py` |
| `/` | `dashboard_bp` | `steward/dashboard/routes.py` |
| `/hosts/` | `hosts_bp` | `steward/hosts/routes.py` |
| `/ping/` | `ping_bp` | `steward/ping/routes.py` |
| `/dns/` | `dns_bp` | `steward/dns/routes.py` |
| `/alerts/` | `alerts_bp` | `steward/alerts/routes.py` |
| `/ansible/` | `ansible_bp` | `steward/ansible/routes.py` |
| `/settings/` | `settings_bp` | `steward/settings/routes.py` |
| `/plugins/<name>/` | plugin blueprint | `plugins/<name>/routes.py` |
Plugin blueprints are mounted automatically by `load_plugins()` using the plugin directory name as the URL prefix.
@@ -48,7 +48,7 @@ Plugin blueprints are mounted automatically by `load_plugins()` using the plugin
## Scheduler
`roundtable/core/scheduler.py` exports two things:
`steward/core/scheduler.py` exports two things:
- `ScheduledTask` dataclass — holds a `name`, `coro_factory` (zero-argument callable returning a coroutine), `interval_seconds`, and optional `run_on_startup` flag
- `start_scheduler(tasks)` — async function that loops every second, calling `asyncio.create_task()` for each task whose interval has elapsed
@@ -93,9 +93,9 @@ This means the only file you must touch to get the app running is the database U
No JavaScript framework, no build step. The frontend is:
- **Jinja2 templates** rendered server-side (`roundtable/templates/`)
- **Jinja2 templates** rendered server-side (`steward/templates/`)
- **HTMX** for live-updating fragments (dashboard widgets, ping pills, DNS status)
- A single CSS design system in `roundtable/templates/base.html` using CSS custom properties
- A single CSS design system in `steward/templates/base.html` using CSS custom properties
Live-updating widgets use HTMX polling:
```html
+2 -2
View File
@@ -14,7 +14,7 @@ When any monitor or plugin calls `record_metric()`:
4. If a state transition occurs, an `AlertEvent` row is written
5. Notification I/O is deferred outside the transaction via `asyncio.create_task()`
**Key function:** `roundtable/core/alerts.py``record_metric(session, source_module, resource_name, metric_name, value)`
**Key function:** `steward/core/alerts.py``record_metric(session, source_module, resource_name, metric_name, value)`
`record_metric()` must always be called inside an active transaction:
@@ -130,7 +130,7 @@ Webhook is skipped if `webhook.url` is empty.
## Data Models
Defined in `roundtable/models/alerts.py`:
Defined in `steward/models/alerts.py`:
- **`alert_rules`** — one row per configured rule
- **`alert_states`** — one row per rule, tracks current state and consecutive failure count
+6 -6
View File
@@ -1,6 +1,6 @@
# Ansible Integration
Roundtable can browse, trigger, and stream output from Ansible playbooks directly from the web UI. Runs execute as asyncio tasks inside the same process — no Celery, no external workers.
Steward can browse, trigger, and stream output from Ansible playbooks directly from the web UI. Runs execute as asyncio tasks inside the same process — no Celery, no external workers.
---
@@ -115,7 +115,7 @@ Output stored in the DB is capped at 1 MB. If truncated, `[output truncated]` is
## Data Model
`ansible_runs` table (defined in `roundtable/models/ansible.py`):
`ansible_runs` table (defined in `steward/models/ansible.py`):
| Column | Type | Description |
|---|---|---|
@@ -137,7 +137,7 @@ Runs older than `data.retention_days` (default 90) are pruned by the `data_clean
| File | Purpose |
|---|---|
| `roundtable/ansible/sources.py` | Source discovery, git pull logic |
| `roundtable/ansible/executor.py` | Subprocess execution and output streaming |
| `roundtable/ansible/routes.py` | HTTP routes (browse, trigger, stream, history) |
| `roundtable/models/ansible.py` | `AnsibleRun` model |
| `steward/ansible/sources.py` | Source discovery, git pull logic |
| `steward/ansible/executor.py` | Subprocess execution and output streaming |
| `steward/ansible/routes.py` | HTTP routes (browse, trigger, stream, history) |
| `steward/models/ansible.py` | `AnsibleRun` model |
+10 -12
View File
@@ -1,6 +1,6 @@
# Configuration
Roundtable uses a two-layer configuration system. Only the bare minimum needed to boot lives in files or environment variables. Everything else is stored in the database and managed through the Settings UI.
Steward uses a two-layer configuration system. Only the bare minimum needed to boot lives in files or environment variables. Everything else is stored in the database and managed through the Settings UI.
---
@@ -10,31 +10,29 @@ These three values are read at startup from `config.yaml` and/or environment var
| Key | Env var | Default | Description |
|---|---|---|---|
| `database.url` | `ROUNDTABLE_DATABASE_URL` | — | PostgreSQL async URL. **Required.** |
| `secret_key` | `ROUNDTABLE_SECRET_KEY` | auto-generated | Flask/Quart session signing key. Auto-generated and saved to `/data/secret.key` if not set. |
| `plugin_dir` | `ROUNDTABLE_PLUGIN_DIR` | `plugins` | Path to the plugins directory. |
| `database.url` | `STEWARD_DATABASE_URL` | — | PostgreSQL async URL. **Required.** |
| `secret_key` | `STEWARD_SECRET_KEY` | auto-generated | Flask/Quart session signing key. Auto-generated and saved to `/data/secret.key` if not set. |
| `plugin_dir` | `STEWARD_PLUGIN_DIR` | `plugins` | Path to the plugins directory. |
**Resolution order for `database_url`:** env var `ROUNDTABLE_DATABASE_URL` → env var `ROUNDTABLE_DATABASE__URL` (legacy double-underscore) → `database.url` in `config.yaml`.
**Resolution order for `database_url`:** env var `STEWARD_DATABASE_URL` → env var `STEWARD_DATABASE__URL` (legacy double-underscore) → `database.url` in `config.yaml`.
**Resolution order for `secret_key`:** env var `ROUNDTABLE_SECRET_KEY``secret_key` in `config.yaml``/data/secret.key` file → auto-generate and write to `/data/secret.key`.
**Resolution order for `secret_key`:** env var `STEWARD_SECRET_KEY``secret_key` in `config.yaml``/data/secret.key` file → auto-generate and write to `/data/secret.key`.
### Minimal config.yaml
```yaml
database:
url: "postgresql+asyncpg://user:password@localhost/roundtable"
url: "postgresql+asyncpg://user:password@localhost/steward"
```
### Minimal env-only setup (Docker)
```bash
ROUNDTABLE_DATABASE_URL=postgresql+asyncpg://user:password@db/roundtable
STEWARD_DATABASE_URL=postgresql+asyncpg://user:password@db/steward
```
A `.env` file is loaded automatically if present.
> **Legacy env vars:** `FABLEDSCRYER_*` env vars are still accepted as a fallback for existing deployments. They will be removed in a future release — migrate to `ROUNDTABLE_*` when convenient.
---
## App Settings (Database-backed)
@@ -68,7 +66,7 @@ Default webhook template:
### Reading and Writing Settings in Code
```python
from roundtable.core.settings import get_setting, set_setting
from steward.core.settings import get_setting, set_setting
# Read
async with current_app.db_sessionmaker() as session:
@@ -84,7 +82,7 @@ async with current_app.db_sessionmaker() as session:
### The DEFAULTS Dict
`roundtable/core/settings.py` contains the `DEFAULTS` dict — the canonical list of all recognised settings and their default values. Add new settings here to make them recognised by `get_all_settings()` and the settings UI.
`steward/core/settings.py` contains the `DEFAULTS` dict — the canonical list of all recognised settings and their default values. Add new settings here to make them recognised by `get_all_settings()` and the settings UI.
---
+7 -7
View File
@@ -1,13 +1,13 @@
# Core Monitors
Roundtable ships two built-in monitors: Ping and DNS. Both run as asyncio scheduled tasks on the same event loop as the web server, with no separate processes.
Steward ships two built-in monitors: Ping and DNS. Both run as asyncio scheduled tasks on the same event loop as the web server, with no separate processes.
---
## Ping Monitor
**Source:** `roundtable/monitors/ping.py`
**Scheduler task:** `ping_monitor` in `roundtable/app.py`
**Source:** `steward/monitors/ping.py`
**Scheduler task:** `ping_monitor` in `steward/app.py`
**Interval:** `monitors.poll_interval_seconds` (default 60s), runs on startup
### How It Works
@@ -29,7 +29,7 @@ Each probe writes a `PingResult` row and calls `record_metric()`:
### Data Model
`ping_results` table (defined in `roundtable/models/monitors.py`):
`ping_results` table (defined in `steward/models/monitors.py`):
| Column | Type | Description |
|---|---|---|
@@ -51,8 +51,8 @@ Old rows are pruned by the `data_cleanup` task (default: 90 days).
## DNS Monitor
**Source:** `roundtable/monitors/dns.py`
**Scheduler task:** `dns_monitor` in `roundtable/app.py`
**Source:** `steward/monitors/dns.py`
**Scheduler task:** `dns_monitor` in `steward/app.py`
**Interval:** `monitors.poll_interval_seconds` (default 60s), runs on startup
### How It Works
@@ -70,7 +70,7 @@ Each check writes a `DnsResult` row and calls `record_metric()`:
### Data Model
`dns_results` table (defined in `roundtable/models/monitors.py`):
`dns_results` table (defined in `steward/models/monitors.py`):
| Column | Type | Description |
|---|---|---|
+36 -36
View File
@@ -2,13 +2,13 @@
**Date:** 2026-04-14
**Status:** Approved, ready for implementation planning
**Scope:** Roundtable plugin for remote host resource monitoring (CPU, memory, storage, load, uptime) via a lightweight Python push agent.
**Scope:** Steward plugin for remote host resource monitoring (CPU, memory, storage, load, uptime) via a lightweight Python push agent.
---
## Goal
Give Roundtable a fleet-glance view of remote Linux hosts — current resource usage, history, and alert integration — without requiring agentless SSH/Ansible round-trips, a new toolchain, or SNMP gymnastics on every target.
Give Steward a fleet-glance view of remote Linux hosts — current resource usage, history, and alert integration — without requiring agentless SSH/Ansible round-trips, a new toolchain, or SNMP gymnastics on every target.
## Non-goals
@@ -24,7 +24,7 @@ Give Roundtable a fleet-glance view of remote Linux hosts — current resource u
```
┌──────────────┐ POST /plugins/host_agent/ingest ┌────────────────────────┐
│ Agent │ ──────────────────────────────────────────→ │ Roundtable
│ Agent │ ──────────────────────────────────────────→ │ Steward
│ (Python) │ Authorization: Bearer <per-host-token> │ host_agent plugin │
│ on target │ JSON body: metrics snapshot │ │
│ host │ │ routes.py (ingest + │
@@ -48,8 +48,8 @@ Give Roundtable a fleet-glance view of remote Linux hosts — current resource u
- **Agent** is a single Python file. No external dependencies beyond Python 3.8+ stdlib.
- **Plugin** is self-contained under `plugins/host_agent/`. Writes to the core `PluginMetric` time-series bus (the designed cross-plugin integration point) and its own private `host_agent_registrations` table. Writes to the core `Host` model for identity, but adds no new columns to it.
- **Auth** is per-host bearer tokens, minted on "Add host" in the plugin's settings page.
- **Install** is a one-line `curl | sh` command rendered per-host by Roundtable with the token already baked in.
- **Failure** is handled at the agent (in-memory ring buffer + exponential backoff) so Roundtable outages don't lose brief-window data.
- **Install** is a one-line `curl | sh` command rendered per-host by Steward with the token already baked in.
- **Failure** is handled at the agent (in-memory ring buffer + exponential backoff) so Steward outages don't lose brief-window data.
---
@@ -58,10 +58,10 @@ Give Roundtable a fleet-glance view of remote Linux hosts — current resource u
### File layout on the target host
```
/usr/local/lib/roundtable-agent/agent.py # the script, target ~300 lines
/etc/roundtable-agent.conf # key=value config, 0640 root:roundtable-agent
/etc/systemd/system/roundtable-agent.service # unit file
# dedicated system user: roundtable-agent
/usr/local/lib/steward-agent/agent.py # the script, target ~300 lines
/etc/steward-agent.conf # key=value config, 0640 root:steward-agent
/etc/systemd/system/steward-agent.service # unit file
# dedicated system user: steward-agent
```
### Config file format
@@ -69,7 +69,7 @@ Give Roundtable a fleet-glance view of remote Linux hosts — current resource u
Flat `key = value`, parsed by a ~20-line homegrown parser (no TOML/YAML dependency):
```
url = https://roundtable.home.lan
url = https://steward.home.lan
token = a1b2c3d4...
interval_seconds = 30
hostname = myhost # optional; defaults to uname -n
@@ -111,7 +111,7 @@ To stderr only (systemd captures to journal). No file logging, no log rotation.
### Identity
The agent's self-reported hostname in the payload is advisory. The real identity is the bearer token — Roundtable looks up the `Host` row via `HostAgentRegistration.token_hash`, not via hostname. Changing a host's hostname does not break its identity, and two hosts accidentally sharing a hostname can't collide.
The agent's self-reported hostname in the payload is advisory. The real identity is the bearer token — Steward looks up the `Host` row via `HostAgentRegistration.token_hash`, not via hostname. Changing a host's hostname does not break its identity, and two hosts accidentally sharing a hostname can't collide.
### Failure behavior
@@ -194,7 +194,7 @@ One `ScheduledTask` running every 60 seconds that flags `HostAgentRegistration`
### `METRIC_CATALOG` registration
Add to `roundtable/alerts/routes.py`:
Add to `steward/alerts/routes.py`:
```python
"host_agent": ["cpu_pct", "mem_used_pct", "mem_available_bytes",
@@ -202,7 +202,7 @@ Add to `roundtable/alerts/routes.py`:
"load_5m", "load_15m", "uptime_secs"],
```
This is the one edit outside the plugin directory, matching the pattern every other plugin already follows. Not ideal from a pure-plugin-independence standpoint but unavoidable until Roundtable core grows a plugin-registered catalog API — deferred as future core work.
This is the one edit outside the plugin directory, matching the pattern every other plugin already follows. Not ideal from a pure-plugin-independence standpoint but unavoidable until Steward core grows a plugin-registered catalog API — deferred as future core work.
---
@@ -212,7 +212,7 @@ This is the one edit outside the plugin directory, matching the pattern every ot
```http
POST /plugins/host_agent/ingest HTTP/1.1
Host: roundtable.home.lan
Host: steward.home.lan
Authorization: Bearer a1b2c3d4e5f6...
Content-Type: application/json
```
@@ -311,7 +311,7 @@ Agent treats anything non-2xx as failure → ring buffer. Agent treats 401 speci
### The one-liner (what the UI shows)
```
curl -sSL 'https://roundtable.home.lan/plugins/host_agent/install.sh?token=a1b2c3...' | sudo sh
curl -sSL 'https://steward.home.lan/plugins/host_agent/install.sh?token=a1b2c3...' | sudo sh
```
The UI also offers a "review script before running" link that opens a modal showing the two-step form:
@@ -326,19 +326,19 @@ sudo sh install.sh
```sh
#!/bin/sh
# Roundtable host agent installer
# Steward host agent installer
# Generated for: {{ host_name }} ({{ host_address }})
# Roundtable URL: {{ url }}
# Steward URL: {{ url }}
set -e
ROUNDTABLE_URL="{{ url }}"
STEWARD_URL="{{ url }}"
AGENT_TOKEN="{{ token }}"
AGENT_VERSION="{{ agent_version }}"
AGENT_USER="roundtable-agent"
AGENT_DIR="/usr/local/lib/roundtable-agent"
CONF_FILE="/etc/roundtable-agent.conf"
UNIT_FILE="/etc/systemd/system/roundtable-agent.service"
AGENT_USER="steward-agent"
AGENT_DIR="/usr/local/lib/steward-agent"
CONF_FILE="/etc/steward-agent.conf"
UNIT_FILE="/etc/systemd/system/steward-agent.service"
# ── preflight ────────────────────────────────────────────────────────────────
[ "$(id -u)" = "0" ] || { echo "must run as root (use sudo)"; exit 1; }
@@ -347,7 +347,7 @@ command -v python3 >/dev/null 2>&1 || { echo "python3 not found — install py
# Handle --uninstall
if [ "${1:-}" = "--uninstall" ]; then
systemctl disable --now roundtable-agent.service 2>/dev/null || true
systemctl disable --now steward-agent.service 2>/dev/null || true
rm -f "$UNIT_FILE" "$CONF_FILE"
rm -rf "$AGENT_DIR"
systemctl daemon-reload
@@ -363,13 +363,13 @@ fi
# ── drop the agent file ──────────────────────────────────────────────────────
mkdir -p "$AGENT_DIR"
curl -sSL "${ROUNDTABLE_URL}/plugins/host_agent/agent.py" -o "$AGENT_DIR/agent.py"
curl -sSL "${STEWARD_URL}/plugins/host_agent/agent.py" -o "$AGENT_DIR/agent.py"
chmod 0755 "$AGENT_DIR/agent.py"
chown root:root "$AGENT_DIR/agent.py"
# ── write config ─────────────────────────────────────────────────────────────
cat > "$CONF_FILE" <<EOF
url = $ROUNDTABLE_URL
url = $STEWARD_URL
token = $AGENT_TOKEN
interval_seconds = 30
EOF
@@ -379,7 +379,7 @@ chmod 0640 "$CONF_FILE"
# ── write systemd unit ───────────────────────────────────────────────────────
cat > "$UNIT_FILE" <<EOF
[Unit]
Description=Roundtable host agent
Description=Steward host agent
After=network-online.target
Wants=network-online.target
@@ -400,17 +400,17 @@ WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable --now roundtable-agent.service
systemctl enable --now steward-agent.service
echo
echo "Roundtable host agent $AGENT_VERSION installed and running."
echo "Check status: systemctl status roundtable-agent"
echo "Logs: journalctl -u roundtable-agent -f"
echo "Steward host agent $AGENT_VERSION installed and running."
echo "Check status: systemctl status steward-agent"
echo "Logs: journalctl -u steward-agent -f"
```
### Design points
- **Agent binary served by Roundtable itself.** `GET /plugins/host_agent/agent.py` serves the script bundled with the plugin, so version drift between install-script and agent is impossible — re-running the installer picks up whatever agent the currently-running Roundtable ships.
- **Agent binary served by Steward itself.** `GET /plugins/host_agent/agent.py` serves the script bundled with the plugin, so version drift between install-script and agent is impossible — re-running the installer picks up whatever agent the currently-running Steward ships.
- **Systemd hardening is cheap and correct.** `NoNewPrivileges`, `ProtectSystem=strict`, `ProtectHome`, `PrivateTmp`, `ReadOnlyPaths=/proc /sys`. The agent only needs read access to `/proc`, `/sys`, and mount points; denying everything else narrows blast radius.
- **Uninstall is a first-class flag**, not a separate script. Same one-liner with `--uninstall` appended.
- **Fail-fast on preflight.** Missing systemd or python3 → clear error, exit. No half-installed agent.
@@ -421,7 +421,7 @@ echo "Logs: journalctl -u roundtable-agent -f"
## UI surfaces
### Dashboard widgets (`roundtable/core/widgets.py`)
### Dashboard widgets (`steward/core/widgets.py`)
- **`host_resources`** — table widget. One row per monitored host: name, CPU %, mem %, disk % (worst mount), load 1m, "last seen Xs ago" with red/yellow/green coloring. Fleet glance.
- **`host_resource_history`** — chart widget for one host. CPU / mem / disk over a selectable time range (1h, 6h, 24h, 7d). Same pattern as every other history widget — Chart.js.
@@ -447,9 +447,9 @@ List of registered hosts with their enable flags, an "Add host" button that open
| Failure | Who handles it | Response |
|---|---|---|
| Agent can't reach Roundtable | Agent | Push to ring buffer, exponential backoff (30→60→120→300s cap), log WARN. Resets on success. |
| Roundtable rejects with 401 | Agent | Log ERROR with remediation hint, continue retry loop. Admin fixes via UI + conf edit; no restart needed. |
| Roundtable rejects with 400 | Agent | Log ERROR, drop the sample (don't re-buffer), continue. Indicates agent/server version skew. |
| Agent can't reach Steward | Agent | Push to ring buffer, exponential backoff (30→60→120→300s cap), log WARN. Resets on success. |
| Steward rejects with 401 | Agent | Log ERROR with remediation hint, continue retry loop. Admin fixes via UI + conf edit; no restart needed. |
| Steward rejects with 400 | Agent | Log ERROR, drop the sample (don't re-buffer), continue. Indicates agent/server version skew. |
| Ring buffer full | Agent | Drop oldest, keep newest. DEBUG log (expected during outages). |
| Config file missing or malformed | Agent | Log ERROR, exit non-zero. Systemd restarts after 10s. Repeated restarts are visible in journal. |
| `/proc/stat` read fails between samples | Agent | Skip CPU for this cycle, still POST the rest. Partial samples allowed. |
@@ -528,7 +528,7 @@ These are not blockers; they are feature boundaries inherent to the design.
6. Settings page: list, add-host flow, rotate-token, delete.
7. Dashboard widgets: table + history chart. Register in `core/widgets.py`.
8. Per-host detail page.
9. `METRIC_CATALOG` entry in `roundtable/alerts/routes.py`.
9. `METRIC_CATALOG` entry in `steward/alerts/routes.py`.
10. Scheduler: stale-agent marker.
11. Tests at every stage; final integration test to tie it together.
+52 -52
View File
@@ -2,7 +2,7 @@
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Ship a `host_agent` Roundtable plugin plus a stdlib-only Python push agent that reports CPU/memory/storage/load/uptime from remote Linux hosts to Roundtable.
**Goal:** Ship a `host_agent` Steward plugin plus a stdlib-only Python push agent that reports CPU/memory/storage/load/uptime from remote Linux hosts to Steward.
**Architecture:** Plugin lives under `plugins/host_agent/` (mirrors `plugins/http/`). It owns a private `host_agent_registrations` table, writes time-series data into the core `PluginMetric` bus, serves its own agent source at `GET /plugins/host_agent/agent.py`, and renders a per-host install script at `GET /plugins/host_agent/install.sh`. The agent is a single ~300-line Python script with a 30s collect→POST loop, in-memory ring buffer, exponential backoff, and systemd unit installed by a curl one-liner.
@@ -14,7 +14,7 @@
## Execution note — path 1 (no DB-backed tests)
Mid-execution discovery: the Roundtable test harness runs `create_app(testing=True)`, which mocks `db_sessionmaker` and skips all migrations. There is no existing DB-backed test fixture in this codebase, and no existing plugin ships with tests. The first execution attempt tried to invent a SQLite-backed override fixture, which cascaded into modifying a production migration (`0007_dashboard_ownership.py`) to remove an inline FK — unacceptable. That commit was reverted.
Mid-execution discovery: the Steward test harness runs `create_app(testing=True)`, which mocks `db_sessionmaker` and skips all migrations. There is no existing DB-backed test fixture in this codebase, and no existing plugin ships with tests. The first execution attempt tried to invent a SQLite-backed override fixture, which cascaded into modifying a production migration (`0007_dashboard_ownership.py`) to remove an inline FK — unacceptable. That commit was reverted.
**Revised test strategy:** test the agent exhaustively (everything that lives in `plugins/host_agent/agent.py` — pure Python, stdlib only, trivially unit-testable). Test the server-side metric-expansion function as a pure function that takes a sample dict + host name and returns a list of `(metric_name, resource_name, value)` tuples. Everything else — ingest route, install route, settings routes, widgets, detail page, scheduler — is verified manually against the dev server.
@@ -43,7 +43,7 @@ Where code blocks in tasks below reference DB-backed tests, treat them as guidan
- `plugins/host_agent/scheduler.py` — stale-agent marker task.
- `plugins/host_agent/agent.py` — the Python agent script, served to targets.
- `plugins/host_agent/migrations/__init__.py`
- `plugins/host_agent/migrations/env.py` — alembic env (copy of http plugin's, but imports `roundtable.models.base`).
- `plugins/host_agent/migrations/env.py` — alembic env (copy of http plugin's, but imports `steward.models.base`).
- `plugins/host_agent/migrations/versions/__init__.py`
- `plugins/host_agent/migrations/versions/host_agent_001_initial.py` — creates `host_agent_registrations`.
- `plugins/host_agent/templates/install.sh.j2` — Jinja install script template.
@@ -68,8 +68,8 @@ Where code blocks in tasks below reference DB-backed tests, treat them as guidan
**Modified files (core, small edits):**
- `roundtable/alerts/routes.py` — add `"host_agent"` entry to `METRIC_CATALOG`.
- `roundtable/core/widgets.py` — add `host_resources` and `host_resource_history` entries.
- `steward/alerts/routes.py` — add `"host_agent"` entry to `METRIC_CATALOG`.
- `steward/core/widgets.py` — add `host_resources` and `host_resource_history` entries.
- `docs/plugins/index.yaml.example` — add catalog entry.
---
@@ -102,7 +102,7 @@ pytestmark = pytest.mark.asyncio
async def test_host_agent_migration_creates_table(app):
# The app fixture runs all plugin migrations on startup.
from roundtable.core.db import get_engine
from steward.core.db import get_engine
engine = get_engine()
async with engine.connect() as conn:
def _check(sync_conn):
@@ -127,11 +127,11 @@ Expected: FAIL (plugin not loaded, table missing).
name: host_agent
version: "1.0.0"
description: "Remote Linux host resource monitoring via a lightweight Python push agent (CPU, memory, storage, load, uptime)"
author: "Roundtable"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/src/branch/main/host_agent"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/host_agent"
tags:
- host
- monitoring
@@ -180,7 +180,7 @@ from __future__ import annotations
import uuid
from datetime import datetime, timezone
from sqlalchemy import Column, String, DateTime, ForeignKey
from roundtable.models.base import Base
from steward.models.base import Base
def _uuid() -> str:
@@ -269,8 +269,8 @@ from alembic import context
sys.path.insert(0, str(Path(__file__).parent.parent.parent.parent))
from roundtable.models.base import Base
import roundtable.models # noqa: F401
from steward.models.base import Base
import steward.models # noqa: F401
from plugins.host_agent.models import HostAgentRegistration # noqa: F401
config = context.config
@@ -282,14 +282,14 @@ target_metadata = Base.metadata
def _get_url() -> str:
import yaml
cfg_path = os.environ.get("ROUNDTABLE_CONFIG", "config.yaml")
cfg_path = os.environ.get("STEWARD_CONFIG", "config.yaml")
try:
with open(cfg_path) as f:
cfg = yaml.safe_load(f) or {}
url = cfg.get("database", {}).get("url", "")
except FileNotFoundError:
url = ""
return os.environ.get("ROUNDTABLE_DATABASE__URL", url)
return os.environ.get("STEWARD_DATABASE__URL", url)
def run_migrations_offline() -> None:
@@ -413,12 +413,12 @@ from plugins.host_agent.agent import read_config, ConfigError
def test_parses_flat_key_value(tmp_path):
p = tmp_path / "agent.conf"
p.write_text(
"url = https://roundtable.example\n"
"url = https://steward.example\n"
"token = abc123\n"
"interval_seconds = 45\n"
)
cfg = read_config(str(p))
assert cfg["url"] == "https://roundtable.example"
assert cfg["url"] == "https://steward.example"
assert cfg["token"] == "abc123"
assert cfg["interval_seconds"] == 45
@@ -466,7 +466,7 @@ Expected: FAIL (`read_config` does not exist).
```python
# plugins/host_agent/agent.py
"""Roundtable host agent — pushes resource metrics to a Roundtable instance.
"""Steward host agent — pushes resource metrics to a Steward instance.
Python 3.8+ stdlib only. Target ~300 lines. Served to targets at
GET /plugins/host_agent/agent.py.
@@ -1060,7 +1060,7 @@ def post_payload(url: str, token: str, payload: dict) -> tuple[bool, int | None]
headers={
"Content-Type": "application/json",
"Authorization": f"Bearer {token}",
"User-Agent": f"roundtable-host-agent/{AGENT_VERSION}",
"User-Agent": f"steward-host-agent/{AGENT_VERSION}",
},
method="POST",
)
@@ -1112,7 +1112,7 @@ def main_loop(conf_path: str) -> int:
buffer = RingBuffer(maxlen=20)
backoff = 0
_log("INFO", f"roundtable-host-agent {AGENT_VERSION} starting "
_log("INFO", f"steward-host-agent {AGENT_VERSION} starting "
f"(url={cfg['url']}, interval={cfg['interval_seconds']}s)")
while not _shutdown_requested:
@@ -1171,7 +1171,7 @@ def main_loop(conf_path: str) -> int:
if __name__ == "__main__":
conf = os.environ.get("ROUNDTABLE_AGENT_CONFIG", "/etc/roundtable-agent.conf")
conf = os.environ.get("STEWARD_AGENT_CONFIG", "/etc/steward-agent.conf")
sys.exit(main_loop(conf))
```
@@ -1209,8 +1209,8 @@ git commit -m "feat(host_agent): agent POST, backoff, and main loop"
import hashlib
import pytest_asyncio
from roundtable.models.hosts import Host
from roundtable.core.db import get_session
from steward.models.hosts import Host
from steward.core.db import get_session
from plugins.host_agent.models import HostAgentRegistration
@@ -1233,7 +1233,7 @@ async def registered_host(app):
return {"host": host, "registration": reg, "token": raw_token}
```
> **Note:** Verify `Host.__init__` accepts `name=` and `address=` kwargs before relying on this. If the real Host model requires more fields (e.g., `probe_type`), pass them here. Read `roundtable/models/hosts.py` first to confirm.
> **Note:** Verify `Host.__init__` accepts `name=` and `address=` kwargs before relying on this. If the real Host model requires more fields (e.g., `probe_type`), pass them here. Read `steward/models/hosts.py` first to confirm.
- [ ] **Step 2: Write failing ingest test**
@@ -1244,8 +1244,8 @@ from datetime import datetime, timezone
import pytest
from sqlalchemy import select
from roundtable.models.metrics import PluginMetric
from roundtable.core.db import get_session
from steward.models.metrics import PluginMetric
from steward.core.db import get_session
from plugins.host_agent.models import HostAgentRegistration
pytestmark = pytest.mark.asyncio
@@ -1352,9 +1352,9 @@ from typing import Any
from quart import Blueprint, jsonify, request
from sqlalchemy import select
from roundtable.core.db import get_session
from roundtable.models.hosts import Host
from roundtable.models.metrics import PluginMetric
from steward.core.db import get_session
from steward.models.hosts import Host
from steward.models.metrics import PluginMetric
from .models import HostAgentRegistration
host_agent_bp = Blueprint("host_agent", __name__, template_folder="templates")
@@ -1628,7 +1628,7 @@ async def test_install_sh_renders_with_token(client, registered_host):
assert resp.status_code == 200
assert resp.content_type.startswith("text/plain")
text = (await resp.get_data()).decode()
assert "roundtable-agent" in text
assert "steward-agent" in text
assert registered_host["token"] in text
assert "systemctl enable --now" in text
assert "NoNewPrivileges=yes" in text
@@ -1758,11 +1758,11 @@ git commit -m "feat(host_agent): install.sh and agent.py serving routes"
- Create: `plugins/host_agent/templates/settings_list.html`
- Create: `tests/plugins/host_agent/test_settings_routes.py`
Auth note: the existing Roundtable admin decorator pattern needs to be used here. Read `roundtable/settings/routes.py` (or similar) to find the decorator name — likely something like `@require_admin` or a session check. Use whatever the rest of the codebase uses. **Do not invent a new auth mechanism.**
Auth note: the existing Steward admin decorator pattern needs to be used here. Read `steward/settings/routes.py` (or similar) to find the decorator name — likely something like `@require_admin` or a session check. Use whatever the rest of the codebase uses. **Do not invent a new auth mechanism.**
- [ ] **Step 1: Find the admin auth decorator**
Run: `grep -rn "require_admin\|@admin_required\|def admin" roundtable/settings/ roundtable/core/auth.py 2>/dev/null | head -20`
Run: `grep -rn "require_admin\|@admin_required\|def admin" steward/settings/ steward/core/auth.py 2>/dev/null | head -20`
Note the decorator name and import path for use in Step 3.
- [ ] **Step 2: Write failing test**
@@ -1772,8 +1772,8 @@ Note the decorator name and import path for use in Step 3.
import pytest
from sqlalchemy import select
from roundtable.core.db import get_session
from roundtable.models.hosts import Host
from steward.core.db import get_session
from steward.models.hosts import Host
from plugins.host_agent.models import HostAgentRegistration
pytestmark = pytest.mark.asyncio
@@ -1839,7 +1839,7 @@ from quart import redirect, url_for
# TODO: replace _require_admin with the project-wide decorator found in Step 1.
# Placeholder below mirrors the shape; swap for real admin auth.
from roundtable.core.auth import require_admin # adjust import to actual path
from steward.core.auth import require_admin # adjust import to actual path
def _new_token_pair() -> tuple[str, str]:
@@ -1938,7 +1938,7 @@ async def settings_list():
```html
{% extends "base.html" %}
{% block title %}Host Agent — Roundtable{% endblock %}
{% block title %}Host Agent — Steward{% endblock %}
{% block content %}
<h1 class="page-title">Host Agent — Registered Hosts</h1>
@@ -2003,7 +2003,7 @@ async def settings_list():
- [ ] **Step 6: Run tests — iterate on auth decorator if needed**
Run: `pytest tests/plugins/host_agent/test_settings_routes.py -v`
Expected: PASS. If admin auth fails, the actual import path from Step 1 is wrong — fix the `from roundtable.core.auth import require_admin` line.
Expected: PASS. If admin auth fails, the actual import path from Step 1 is wrong — fix the `from steward.core.auth import require_admin` line.
- [ ] **Step 7: Commit**
@@ -2020,8 +2020,8 @@ git commit -m "feat(host_agent): plugin settings page — add, rotate, delete"
- Modify: `plugins/host_agent/routes.py` — add `/widget` and `/widget/history` partials.
- Create: `plugins/host_agent/templates/widget_table.html`
- Create: `plugins/host_agent/templates/widget_history.html`
- Modify: `roundtable/core/widgets.py`
- Modify: `roundtable/alerts/routes.py`
- Modify: `steward/core/widgets.py`
- Modify: `steward/alerts/routes.py`
- [ ] **Step 1: Add widget partial routes to `plugins/host_agent/routes.py`**
@@ -2165,7 +2165,7 @@ async def widget_history():
</div>
```
- [ ] **Step 4: Register widgets in `roundtable/core/widgets.py`**
- [ ] **Step 4: Register widgets in `steward/core/widgets.py`**
Add to `WIDGET_REGISTRY` (alphabetically by existing convention, or at the end — check the file first):
@@ -2206,7 +2206,7 @@ Add to `WIDGET_REGISTRY` (alphabetically by existing convention, or at the end
- [ ] **Step 5: Register `host_agent` in `METRIC_CATALOG`**
Read `roundtable/alerts/routes.py`. Locate the `METRIC_CATALOG` dict. Add:
Read `steward/alerts/routes.py`. Locate the `METRIC_CATALOG` dict. Add:
```python
"host_agent": [
@@ -2244,7 +2244,7 @@ Expected: PASS.
- [ ] **Step 7: Commit**
```bash
git add plugins/host_agent/routes.py plugins/host_agent/templates/widget_table.html plugins/host_agent/templates/widget_history.html roundtable/core/widgets.py roundtable/alerts/routes.py tests/plugins/host_agent/test_ingest_route.py
git add plugins/host_agent/routes.py plugins/host_agent/templates/widget_table.html plugins/host_agent/templates/widget_history.html steward/core/widgets.py steward/alerts/routes.py tests/plugins/host_agent/test_ingest_route.py
git commit -m "feat(host_agent): dashboard widgets and alert metric catalog entry"
```
@@ -2409,8 +2409,8 @@ from datetime import datetime, timedelta, timezone
import pytest
from sqlalchemy import select
from roundtable.core.db import get_session
from roundtable.models.hosts import Host
from steward.core.db import get_session
from steward.models.hosts import Host
from plugins.host_agent.models import HostAgentRegistration
from plugins.host_agent.scheduler import find_stale_registrations
@@ -2459,8 +2459,8 @@ from datetime import datetime, timedelta, timezone
from sqlalchemy import select
from roundtable.core.db import get_session
from roundtable.models.hosts import Host
from steward.core.db import get_session
from steward.models.hosts import Host
from .models import HostAgentRegistration
@@ -2531,8 +2531,8 @@ from unittest.mock import patch
import pytest
from sqlalchemy import select
from roundtable.core.db import get_session
from roundtable.models.metrics import PluginMetric
from steward.core.db import get_session
from steward.models.metrics import PluginMetric
from plugins.host_agent import agent as a
pytestmark = pytest.mark.asyncio
@@ -2609,12 +2609,12 @@ Append to the `plugins:` list in `docs/plugins/index.yaml.example`:
- name: host_agent
version: "1.0.0"
description: "Remote Linux host resource monitoring via a lightweight Python push agent"
author: "Roundtable"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/src/branch/main/host_agent"
download_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/releases/download/host_agent-v1.0.0/host_agent.zip"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/host_agent"
download_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins/releases/download/host_agent-v1.0.0/host_agent.zip"
checksum_sha256: ""
tags:
- host
@@ -2643,7 +2643,7 @@ Use `fable_update_task` to set status=done on task 252 ("Implement host_agent pl
- [ ] **Step 2: Add a Fable note summarizing what shipped**
One-paragraph `fable_create_note` attached to Roundtable project (id 6) with:
One-paragraph `fable_create_note` attached to Steward project (id 6) with:
- Link to spec: `docs/plugins/host-agent-design.md`
- Link to plan: `docs/plugins/host-agent-plan.md`
- Note any deviations from the spec discovered during implementation (e.g., auth decorator name, fixture adjustments).
+25 -25
View File
@@ -1,10 +1,10 @@
# roundtable-plugins / index.yaml
# steward-plugins / index.yaml
#
# This file is the catalog index for the roundtable plugin repository.
# This file is the catalog index for the steward plugin repository.
# It is fetched by the app's Settings → Plugins → Plugin Catalog UI.
#
# Roundtable reads this file from:
# https://git.fabledsword.com/bvandeusen/Roundtable-plugins/raw/branch/main/index.yaml
# Steward reads this file from:
# https://git.fabledsword.com/bvandeusen/Steward-plugins/raw/branch/main/index.yaml
#
# After adding or updating a plugin entry, commit and push — the change is
# live immediately for anyone whose app fetches the catalog (cache TTL: 5 min).
@@ -25,7 +25,7 @@
#
# Download URL conventions:
# Gitea release assets (recommended):
# https://git.fabledsword.com/bvandeusen/Roundtable-plugins/releases/download/traefik-v1.0.0/traefik.zip
# https://git.fabledsword.com/bvandeusen/Steward-plugins/releases/download/traefik-v1.0.0/traefik.zip
# Upload the zip as a release attachment in Gitea; paste the URL here.
# Source archive tarballs work too but release assets are preferred (smaller, plugin-only).
@@ -37,12 +37,12 @@ plugins:
- name: http
version: "1.0.0"
description: "Synthetic HTTP endpoint monitoring — status code, response time, content match, TLS expiry"
author: "Roundtable"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/src/branch/main/http"
download_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/releases/download/http-v1.0.0/http.zip"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/http"
download_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins/releases/download/http-v1.0.0/http.zip"
checksum_sha256: ""
tags:
- monitoring
@@ -53,12 +53,12 @@ plugins:
- name: docker
version: "1.0.0"
description: "Docker container status, resource usage, and restart tracking via Docker socket"
author: "Roundtable"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/src/branch/main/docker"
download_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/releases/download/docker-v1.0.0/docker.zip"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/docker"
download_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins/releases/download/docker-v1.0.0/docker.zip"
checksum_sha256: ""
tags:
- containers
@@ -68,12 +68,12 @@ plugins:
- name: traefik
version: "1.0.0"
description: "Traefik reverse proxy metrics and access log integration"
author: "Roundtable"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/src/branch/main/traefik"
download_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/releases/download/traefik-v1.0.0/traefik.zip"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/traefik"
download_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins/releases/download/traefik-v1.0.0/traefik.zip"
checksum_sha256: "" # fill in after running: sha256sum traefik.zip
tags:
- proxy
@@ -83,12 +83,12 @@ plugins:
- name: unifi
version: "1.0.0"
description: "UniFi Network controller integration — WAN health, devices, clients, DPI"
author: "Roundtable"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/src/branch/main/unifi"
download_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/releases/download/unifi-v1.0.0/unifi.zip"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/unifi"
download_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins/releases/download/unifi-v1.0.0/unifi.zip"
checksum_sha256: ""
tags:
- network
@@ -98,12 +98,12 @@ plugins:
- name: host_agent
version: "1.0.0"
description: "Remote Linux host resource monitoring via a lightweight Python push agent"
author: "Roundtable"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/src/branch/main/host_agent"
download_url: "https://git.fabledsword.com/bvandeusen/Roundtable-plugins/releases/download/host_agent-v1.0.0/host_agent.zip"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/host_agent"
download_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins/releases/download/host_agent-v1.0.0/host_agent.zip"
checksum_sha256: ""
tags:
- host
+5 -5
View File
@@ -1,12 +1,12 @@
# Plugin System Overview
Plugins extend Roundtable with new data sources, UI pages, scheduled tasks, and dashboard widgets. Only enabled plugins are imported — disabled or unlisted plugins have zero runtime overhead.
Plugins extend Steward with new data sources, UI pages, scheduled tasks, and dashboard widgets. Only enabled plugins are imported — disabled or unlisted plugins have zero runtime overhead.
---
## How Plugins Are Loaded
Plugin loading happens in step 9 of `create_app()`, after all core blueprints and tasks are registered. The entrypoint is `load_plugins(app)` in `roundtable/core/plugin_manager.py`.
Plugin loading happens in step 9 of `create_app()`, after all core blueprints and tasks are registered. The entrypoint is `load_plugins(app)` in `steward/core/plugin_manager.py`.
For each plugin listed as `enabled: true` in the `PLUGINS` config:
@@ -57,7 +57,7 @@ description: "Does a thing"
# Optional
author: "Your Name"
min_app_version: "0.1.0" # Minimum Roundtable version required
min_app_version: "0.1.0" # Minimum Steward version required
# Default config — merged with user overrides at runtime
# Access at runtime via: app.config["PLUGINS"]["myplugin"]["my_setting"]
@@ -90,7 +90,7 @@ def setup(app):
Returns a list of `ScheduledTask` objects. Return `[]` if the plugin has no background tasks. Called after `setup()`, so any app references set in `setup()` are available.
```python
from roundtable.core.scheduler import ScheduledTask
from steward.core.scheduler import ScheduledTask
def get_scheduled_tasks():
app = _app
@@ -154,7 +154,7 @@ A plugin can contribute a dashboard widget by:
1. Adding a `GET /widget` route to its blueprint that returns an HTMX HTML fragment
2. The dashboard template polling that endpoint with HTMX
The dashboard (`roundtable/templates/dashboard/index.html`) currently includes the Traefik widget conditionally based on `traefik_enabled`. To add a new plugin widget, the dashboard route and template both need to be updated to detect and render the new widget. See the Traefik widget implementation as a reference.
The dashboard (`steward/templates/dashboard/index.html`) currently includes the Traefik widget conditionally based on `traefik_enabled`. To add a new plugin widget, the dashboard route and template both need to be updated to detect and render the new widget. See the Traefik widget implementation as a reference.
---
+16 -16
View File
@@ -46,7 +46,7 @@ config:
## Step 3: Define Models (if needed)
If your plugin stores data, define SQLAlchemy models using the shared `Base` from `roundtable.models.base`.
If your plugin stores data, define SQLAlchemy models using the shared `Base` from `steward.models.base`.
```python
# plugins/myplugin/models.py
@@ -55,7 +55,7 @@ import uuid
from datetime import datetime
from sqlalchemy import String, Float, DateTime
from sqlalchemy.orm import Mapped, mapped_column
from roundtable.models.base import Base
from steward.models.base import Base
class MyPluginMetric(Base):
@@ -79,7 +79,7 @@ Generate the initial migration:
# From the project root
alembic --config alembic.ini revision \
--autogenerate \
--head=roundtable@head \
--head=steward@head \
--branch-label=myplugin \
-m "myplugin initial"
```
@@ -106,8 +106,8 @@ Keep task logic in a separate file so `__init__.py` stays clean.
# plugins/myplugin/scheduler.py
from __future__ import annotations
import logging
from roundtable.core.scheduler import ScheduledTask
from roundtable.core.alerts import record_metric
from steward.core.scheduler import ScheduledTask
from steward.core.alerts import record_metric
logger = logging.getLogger(__name__)
@@ -174,8 +174,8 @@ async def _fetch_value(url: str) -> float:
```python
# plugins/myplugin/routes.py
from quart import Blueprint, current_app, render_template
from roundtable.auth.middleware import require_role
from roundtable.models.users import UserRole
from steward.auth.middleware import require_role
from steward.models.users import UserRole
from .models import MyPluginMetric
myplugin_bp = Blueprint("myplugin", __name__, template_folder="templates")
@@ -215,7 +215,7 @@ Templates live in `plugins/myplugin/templates/myplugin/` (the extra nesting avoi
```html
{# plugins/myplugin/templates/myplugin/index.html #}
{% extends "base.html" %}
{% block title %}My Plugin — Roundtable{% endblock %}
{% block title %}My Plugin — Steward{% endblock %}
{% block content %}
<div class="page-title">My Plugin</div>
{% for row in rows %}
@@ -283,7 +283,7 @@ On next startup, the plugin will be loaded, its migrations applied, and its blue
`record_metric()` is how plugins feed data into the alert pipeline. Any metric you write here can be the target of an alert rule created in the UI.
```python
from roundtable.core.alerts import record_metric
from steward.core.alerts import record_metric
# Must be inside an active transaction
async with session.begin():
@@ -302,11 +302,11 @@ async with session.begin():
## Auth in Routes
Use the `@require_role` decorator from `roundtable.auth.middleware`:
Use the `@require_role` decorator from `steward.auth.middleware`:
```python
from roundtable.auth.middleware import require_role
from roundtable.models.users import UserRole
from steward.auth.middleware import require_role
from steward.models.users import UserRole
@myplugin_bp.get("/admin-only")
@require_role(UserRole.admin)
@@ -325,13 +325,13 @@ Role hierarchy: `admin > operator > viewer`. Requiring `viewer` grants access to
## Publishing to the Catalog
The official plugin catalog is hosted at `https://git.fabledsword.com/bvandeusen/Roundtable-plugins`. Anyone can submit a plugin by opening a pull request — first-party and third-party plugins are treated identically by the catalog system.
The official plugin catalog is hosted at `https://git.fabledsword.com/bvandeusen/Steward-plugins`. Anyone can submit a plugin by opening a pull request — first-party and third-party plugins are treated identically by the catalog system.
### Repo layout
```
roundtable-plugins/
├── index.yaml ← catalog index — the only file Roundtable fetches
steward-plugins/
├── index.yaml ← catalog index — the only file Steward fetches
├── myplugin/
│ ├── plugin.yaml
│ ├── __init__.py
@@ -381,7 +381,7 @@ myplugin.zip
Generate the zip and its checksum:
```bash
cd roundtable-plugins
cd steward-plugins
zip -r myplugin.zip myplugin/
sha256sum myplugin.zip # paste this into index.yaml checksum_sha256
```
+59 -59
View File
@@ -8,15 +8,15 @@ Quick reference for where key functions, models, and entry points live in the co
| What | File | Function / Class |
|---|---|---|
| App factory | `roundtable/app.py` | `create_app()` |
| CLI entry point | `roundtable/cli.py` | `main()` |
| Bootstrap config loading | `roundtable/config.py` | `load_bootstrap()` |
| Secret key resolution | `roundtable/config.py` | `_resolve_secret_key()` |
| DB engine init | `roundtable/database.py` | `init_db()` |
| Core migrations | `roundtable/core/migration_runner.py` | `run_core_migrations()` |
| Plugin migrations | `roundtable/core/migration_runner.py` | `run_plugin_migrations()` |
| Core task registration | `roundtable/app.py` | `_register_core_tasks()` |
| Scheduler loop | `roundtable/core/scheduler.py` | `start_scheduler()` |
| App factory | `steward/app.py` | `create_app()` |
| CLI entry point | `steward/cli.py` | `main()` |
| Bootstrap config loading | `steward/config.py` | `load_bootstrap()` |
| Secret key resolution | `steward/config.py` | `_resolve_secret_key()` |
| DB engine init | `steward/database.py` | `init_db()` |
| Core migrations | `steward/core/migration_runner.py` | `run_core_migrations()` |
| Plugin migrations | `steward/core/migration_runner.py` | `run_plugin_migrations()` |
| Core task registration | `steward/app.py` | `_register_core_tasks()` |
| Scheduler loop | `steward/core/scheduler.py` | `start_scheduler()` |
---
@@ -24,15 +24,15 @@ Quick reference for where key functions, models, and entry points live in the co
| What | File | Function |
|---|---|---|
| All defaults | `roundtable/core/settings.py` | `DEFAULTS` dict |
| Read a setting | `roundtable/core/settings.py` | `get_setting(session, key)` |
| Write a setting | `roundtable/core/settings.py` | `set_setting(session, key, value)` |
| Read all settings | `roundtable/core/settings.py` | `get_all_settings(session)` |
| Sync load at startup | `roundtable/core/settings.py` | `load_settings_sync(db_url)` |
| Extract SMTP dict | `roundtable/core/settings.py` | `to_smtp_cfg(settings)` |
| Extract webhook dict | `roundtable/core/settings.py` | `to_webhook_cfg(settings)` |
| Extract Ansible dict | `roundtable/core/settings.py` | `to_ansible_cfg(settings)` |
| Extract plugins dict | `roundtable/core/settings.py` | `to_plugins_cfg(settings)` |
| All defaults | `steward/core/settings.py` | `DEFAULTS` dict |
| Read a setting | `steward/core/settings.py` | `get_setting(session, key)` |
| Write a setting | `steward/core/settings.py` | `set_setting(session, key, value)` |
| Read all settings | `steward/core/settings.py` | `get_all_settings(session)` |
| Sync load at startup | `steward/core/settings.py` | `load_settings_sync(db_url)` |
| Extract SMTP dict | `steward/core/settings.py` | `to_smtp_cfg(settings)` |
| Extract webhook dict | `steward/core/settings.py` | `to_webhook_cfg(settings)` |
| Extract Ansible dict | `steward/core/settings.py` | `to_ansible_cfg(settings)` |
| Extract plugins dict | `steward/core/settings.py` | `to_plugins_cfg(settings)` |
---
@@ -40,9 +40,9 @@ Quick reference for where key functions, models, and entry points live in the co
| What | File | Function |
|---|---|---|
| Plugin loading | `roundtable/core/plugin_manager.py` | `load_plugins(app)` |
| ScheduledTask dataclass | `roundtable/core/scheduler.py` | `ScheduledTask` |
| Task runner | `roundtable/core/scheduler.py` | `start_scheduler(tasks)` |
| Plugin loading | `steward/core/plugin_manager.py` | `load_plugins(app)` |
| ScheduledTask dataclass | `steward/core/scheduler.py` | `ScheduledTask` |
| Task runner | `steward/core/scheduler.py` | `start_scheduler(tasks)` |
---
@@ -50,11 +50,11 @@ Quick reference for where key functions, models, and entry points live in the co
| What | File | Function |
|---|---|---|
| Write metric + evaluate alerts | `roundtable/core/alerts.py` | `record_metric(session, source_module, resource_name, metric_name, value)` |
| Init alert pipeline | `roundtable/core/alerts.py` | `init_alerts(app)` |
| Rule evaluation | `roundtable/core/alerts.py` | `_evaluate_rule()` (internal) |
| Notification dispatch | `roundtable/core/alerts.py` | `_dispatch_notification()` (internal) |
| Email + webhook send | `roundtable/core/notifications.py` | `dispatch_notifications()` |
| Write metric + evaluate alerts | `steward/core/alerts.py` | `record_metric(session, source_module, resource_name, metric_name, value)` |
| Init alert pipeline | `steward/core/alerts.py` | `init_alerts(app)` |
| Rule evaluation | `steward/core/alerts.py` | `_evaluate_rule()` (internal) |
| Notification dispatch | `steward/core/alerts.py` | `_dispatch_notification()` (internal) |
| Email + webhook send | `steward/core/notifications.py` | `dispatch_notifications()` |
---
@@ -62,9 +62,9 @@ Quick reference for where key functions, models, and entry points live in the co
| What | File | Function |
|---|---|---|
| Ping a host | `roundtable/monitors/ping.py` | `ping_check(host, session)` |
| DNS check a host | `roundtable/monitors/dns.py` | `dns_check(host, session)` |
| Data cleanup | `roundtable/core/cleanup.py` | `run_cleanup(app)` |
| Ping a host | `steward/monitors/ping.py` | `ping_check(host, session)` |
| DNS check a host | `steward/monitors/dns.py` | `dns_check(host, session)` |
| Data cleanup | `steward/core/cleanup.py` | `run_cleanup(app)` |
---
@@ -72,9 +72,9 @@ Quick reference for where key functions, models, and entry points live in the co
| What | File | Function / Class |
|---|---|---|
| Role-based access decorator | `roundtable/auth/middleware.py` | `@require_role(UserRole.X)` |
| Login / session handling | `roundtable/auth/middleware.py` | `login_user()`, `logout_user()` |
| User count (for first-run) | `roundtable/auth/middleware.py` | `get_user_count(app)` |
| Role-based access decorator | `steward/auth/middleware.py` | `@require_role(UserRole.X)` |
| Login / session handling | `steward/auth/middleware.py` | `login_user()`, `logout_user()` |
| User count (for first-run) | `steward/auth/middleware.py` | `get_user_count(app)` |
---
@@ -82,18 +82,18 @@ Quick reference for where key functions, models, and entry points live in the co
| Model | File | Table |
|---|---|---|
| `Host` | `roundtable/models/hosts.py` | `hosts` |
| `PingResult` | `roundtable/models/monitors.py` | `ping_results` |
| `DnsResult` | `roundtable/models/monitors.py` | `dns_results` |
| `AlertRule` | `roundtable/models/alerts.py` | `alert_rules` |
| `AlertState` | `roundtable/models/alerts.py` | `alert_states` |
| `AlertEvent` | `roundtable/models/alerts.py` | `alert_events` |
| `PluginMetric` | `roundtable/models/metrics.py` | `plugin_metrics` |
| `AnsibleRun` | `roundtable/models/ansible.py` | `ansible_runs` |
| `User` | `roundtable/models/users.py` | `users` |
| `AppSetting` | `roundtable/models/settings.py` | `app_settings` |
| `Host` | `steward/models/hosts.py` | `hosts` |
| `PingResult` | `steward/models/monitors.py` | `ping_results` |
| `DnsResult` | `steward/models/monitors.py` | `dns_results` |
| `AlertRule` | `steward/models/alerts.py` | `alert_rules` |
| `AlertState` | `steward/models/alerts.py` | `alert_states` |
| `AlertEvent` | `steward/models/alerts.py` | `alert_events` |
| `PluginMetric` | `steward/models/metrics.py` | `plugin_metrics` |
| `AnsibleRun` | `steward/models/ansible.py` | `ansible_runs` |
| `User` | `steward/models/users.py` | `users` |
| `AppSetting` | `steward/models/settings.py` | `app_settings` |
| `TraefikMetric` | `plugins/traefik/models.py` | `traefik_metrics` |
| SQLAlchemy `Base` | `roundtable/models/base.py` | (shared declarative base) |
| SQLAlchemy `Base` | `steward/models/base.py` | (shared declarative base) |
---
@@ -101,16 +101,16 @@ Quick reference for where key functions, models, and entry points live in the co
| URL pattern | Blueprint | File |
|---|---|---|
| `/` (dashboard) | `dashboard_bp` | `roundtable/dashboard/routes.py` |
| `/auth/login`, `/auth/logout` | `auth_bp` | `roundtable/auth/routes.py` |
| `/hosts/` | `hosts_bp` | `roundtable/hosts/routes.py` |
| `/ping/`, `/ping/rows`, `/ping/settings` | `ping_bp` | `roundtable/ping/routes.py` |
| `/dns/`, `/dns/rows` | `dns_bp` | `roundtable/dns/routes.py` |
| `/alerts/` | `alerts_bp` | `roundtable/alerts/routes.py` |
| `/ansible/` | `ansible_bp` | `roundtable/ansible/routes.py` |
| `/settings/` | `settings_bp` | `roundtable/settings/routes.py` |
| `/` (dashboard) | `dashboard_bp` | `steward/dashboard/routes.py` |
| `/auth/login`, `/auth/logout` | `auth_bp` | `steward/auth/routes.py` |
| `/hosts/` | `hosts_bp` | `steward/hosts/routes.py` |
| `/ping/`, `/ping/rows`, `/ping/settings` | `ping_bp` | `steward/ping/routes.py` |
| `/dns/`, `/dns/rows` | `dns_bp` | `steward/dns/routes.py` |
| `/alerts/` | `alerts_bp` | `steward/alerts/routes.py` |
| `/ansible/` | `ansible_bp` | `steward/ansible/routes.py` |
| `/settings/` | `settings_bp` | `steward/settings/routes.py` |
| `/plugins/traefik/`, `/plugins/traefik/widget` | `traefik_bp` | `plugins/traefik/routes.py` |
| `/health` | (inline) | `roundtable/app.py` |
| `/health` | (inline) | `steward/app.py` |
---
@@ -118,12 +118,12 @@ Quick reference for where key functions, models, and entry points live in the co
| Template | Purpose |
|---|---|
| `roundtable/templates/base.html` | Layout, navigation, full CSS design system |
| `roundtable/templates/dashboard/index.html` | Dashboard with stat strip and widget grid |
| `roundtable/templates/ping/rows.html` | HTMX fragment: ping pill rows (shared by dashboard and /ping/) |
| `roundtable/templates/ping/index.html` | Full /ping/ page |
| `roundtable/templates/dns/rows.html` | HTMX fragment: DNS status rows |
| `roundtable/templates/dns/index.html` | Full /dns/ page |
| `steward/templates/base.html` | Layout, navigation, full CSS design system |
| `steward/templates/dashboard/index.html` | Dashboard with stat strip and widget grid |
| `steward/templates/ping/rows.html` | HTMX fragment: ping pill rows (shared by dashboard and /ping/) |
| `steward/templates/ping/index.html` | Full /ping/ page |
| `steward/templates/dns/rows.html` | HTMX fragment: DNS status rows |
| `steward/templates/dns/index.html` | Full /dns/ page |
| `plugins/traefik/templates/traefik/widget.html` | HTMX fragment: Traefik dashboard widget |
| `plugins/traefik/templates/traefik/index.html` | Full Traefik detail page |
@@ -133,6 +133,6 @@ Quick reference for where key functions, models, and entry points live in the co
| Location | Covers |
|---|---|
| `roundtable/migrations/versions/` | Core schema (hosts, users, monitors, alerts, app_settings) |
| `steward/migrations/versions/` | Core schema (hosts, users, monitors, alerts, app_settings) |
| `plugins/traefik/migrations/versions/` | `traefik_metrics` table |
| `alembic.ini` | Alembic config; `version_locations` lists all migration directories |
+95 -95
View File
@@ -1,10 +1,10 @@
# Roundtable Rebrand Implementation Plan
# Steward Rebrand Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
>
> **Testing:** Per project feedback, skip all test authoring and test runs during this work. Verify manually by running the app.
**Goal:** Rebrand FabledScryer to Roundtable: full visual reskin plus full rename of package, config, containers, and docs.
**Goal:** Rebrand FabledScryer to Steward: full visual reskin plus full rename of package, config, containers, and docs.
**Architecture:** Staged as six sequential PRs. PR 1 is a pure visual/copy reskin that still ships under the FabledScryer name. PRs 24 perform the mechanical rename in order (package → config → container + user-visible strings) with a fallback shim in PR 3 so self-hosters don't break mid-upgrade. PR 5 updates docs. PR 6 removes the shim after one release cycle.
@@ -25,29 +25,29 @@
- Any template with an empty-state message — themed line
**Renamed (PR 2 — package):**
- `fabledscryer/``roundtable/` (directory + every `from fabledscryer` / `import fabledscryer` reference)
- `fabledscryer/``steward/` (directory + every `from fabledscryer` / `import fabledscryer` reference)
- `pyproject.toml` — name, entry point, hatch packages
- `alembic.ini``script_location`
- `tests/**` — imports (tests not executed, but must not break collection; update imports mechanically)
**Modified (PR 3 — config & env):**
- `fabledscryer/config.py``roundtable/config.py` (already moved in PR 2) — read both `ROUNDTABLE_*` and `FABLEDSCRYER_*` with deprecation warning
- `fabledscryer/config.py``steward/config.py` (already moved in PR 2) — read both `STEWARD_*` and `FABLEDSCRYER_*` with deprecation warning
- `.env.example`, `config.example.yaml` — new names
- First-boot migration of `~/.config/fabledscryer``~/.config/roundtable` (if the app uses it — verify during task)
- First-boot migration of `~/.config/fabledscryer``~/.config/steward` (if the app uses it — verify during task)
**Modified (PR 4 — container + user strings):**
- `Dockerfile` — COPY paths, CMD, image labels
- `docker-compose.yml` — service name, image, container_name, volumes, env
- `entrypoint.sh` — package path for `nut_setup.py` invocation
- `roundtable/templates/base.html` — wordmark text "Fabled Scryer" → "Roundtable", `<title>`
- `roundtable/templates/auth/login.html` — any residual "Fabled Scryer" text
- `steward/templates/base.html` — wordmark text "Fabled Scryer" → "Steward", `<title>`
- `steward/templates/auth/login.html` — any residual "Fabled Scryer" text
- `README.md` — first-page references (full doc sweep is PR 5)
**Modified (PR 5 — docs):**
- `README.md`, `docs/**/*.md`
**Modified (PR 6 — cleanup):**
- `roundtable/config.py` — remove fallback shim
- `steward/config.py` — remove fallback shim
---
@@ -197,7 +197,7 @@ git commit -m "style: candlelit glow background replaces star field"
- [ ] **Step 1: Change the title block**
```html
<title>{% block title %}Roundtable{% endblock %}</title>
<title>{% block title %}Steward{% endblock %}</title>
```
Note: the visible wordmark text still reads "Fabled Scryer" — that flips in PR 4. This only changes the browser tab title, which is acceptable to flip early since it's not visible inside the UI.
@@ -263,7 +263,7 @@ Content for `fabledscryer/templates/errors/404.html`:
```html
{% extends "base.html" %}
{% block title %}Not Found — Roundtable{% endblock %}
{% block title %}Not Found — Steward{% endblock %}
{% block content %}
<div style="text-align:center; padding: 4rem 2rem; font-family: var(--font-serif);">
<h1 style="color: var(--gold); font-size: 2.5rem; margin-bottom: 0.5rem;">404</h1>
@@ -278,7 +278,7 @@ Content for `fabledscryer/templates/errors/500.html`:
```html
{% extends "base.html" %}
{% block title %}Error — Roundtable{% endblock %}
{% block title %}Error — Steward{% endblock %}
{% block content %}
<div style="text-align:center; padding: 4rem 2rem; font-family: var(--font-serif);">
<h1 style="color: var(--red); font-size: 2.5rem; margin-bottom: 0.5rem;">500</h1>
@@ -361,7 +361,7 @@ git add -A
git commit -m "style: visual polish after manual review"
```
**PR 1 done.** Open PR with title `feat: roundtable visual reskin (pewter & gold)`.
**PR 1 done.** Open PR with title `feat: steward visual reskin (pewter & gold)`.
---
@@ -370,39 +370,39 @@ git commit -m "style: visual polish after manual review"
### Task 10: Rename the package directory
**Files:**
- Rename: `fabledscryer/``roundtable/`
- Rename: `fabledscryer/``steward/`
- [ ] **Step 1: Rename the directory**
```bash
git mv fabledscryer roundtable
git mv fabledscryer steward
```
- [ ] **Step 2: Confirm the move**
```bash
ls roundtable/ | head
ls steward/ | head
git status | head
```
Expected: see `__init__.py`, `app.py`, `cli.py`, etc. inside `roundtable/`.
Expected: see `__init__.py`, `app.py`, `cli.py`, etc. inside `steward/`.
- [ ] **Step 3: Commit**
```bash
git commit -m "refactor: rename package directory fabledscryer → roundtable"
git commit -m "refactor: rename package directory fabledscryer → steward"
```
### Task 11: Rewrite all `fabledscryer` imports to `roundtable`
### Task 11: Rewrite all `fabledscryer` imports to `steward`
**Files:**
- Modify: every file under `roundtable/`, `tests/`, `alembic.ini`, `entrypoint.sh`, `Dockerfile` that imports from the old package
- Modify: every file under `steward/`, `tests/`, `alembic.ini`, `entrypoint.sh`, `Dockerfile` that imports from the old package
- [ ] **Step 1: Find every remaining `fabledscryer` reference in Python code**
Run:
```bash
grep -rn "fabledscryer" roundtable/ tests/ alembic.ini entrypoint.sh Dockerfile pyproject.toml
grep -rn "fabledscryer" steward/ tests/ alembic.ini entrypoint.sh Dockerfile pyproject.toml
```
Keep the output visible as a checklist.
@@ -410,24 +410,24 @@ Keep the output visible as a checklist.
- [ ] **Step 2: Run a safe codemod across Python files**
```bash
grep -rl "fabledscryer" roundtable/ tests/ | xargs sed -i 's/fabledscryer/roundtable/g'
grep -rl "fabledscryer" steward/ tests/ | xargs sed -i 's/fabledscryer/steward/g'
```
This touches only files under `roundtable/` and `tests/`. Do NOT run it across the whole repo yet — `docs/`, `.env.example`, `docker-compose.yml`, `config.example.yaml` and `README.md` are handled in later PRs.
This touches only files under `steward/` and `tests/`. Do NOT run it across the whole repo yet — `docs/`, `.env.example`, `docker-compose.yml`, `config.example.yaml` and `README.md` are handled in later PRs.
- [ ] **Step 3: Spot-check a few critical files**
```bash
grep -n "roundtable\|fabledscryer" roundtable/app.py roundtable/cli.py roundtable/config.py roundtable/migrations/env.py
grep -n "steward\|fabledscryer" steward/app.py steward/cli.py steward/config.py steward/migrations/env.py
```
Expected: only `roundtable` references remain; no stray `fabledscryer`.
Expected: only `steward` references remain; no stray `fabledscryer`.
- [ ] **Step 4: Commit**
```bash
git add -A
git commit -m "refactor: update imports fabledscryer → roundtable"
git commit -m "refactor: update imports fabledscryer → steward"
```
### Task 12: Update `pyproject.toml`
@@ -439,15 +439,15 @@ git commit -m "refactor: update imports fabledscryer → roundtable"
```toml
[project]
name = "roundtable"
name = "steward"
version = "0.1.0"
# ... dependencies unchanged ...
[project.scripts]
roundtable = "roundtable.cli:main"
steward = "steward.cli:main"
[tool.hatch.build.targets.wheel]
packages = ["roundtable"]
packages = ["steward"]
```
Leave `[project.optional-dependencies]`, `[tool.pytest.ini_options]`, and all dependency pins untouched.
@@ -456,7 +456,7 @@ Leave `[project.optional-dependencies]`, `[tool.pytest.ini_options]`, and all de
```bash
git add pyproject.toml
git commit -m "build: rename package to roundtable in pyproject"
git commit -m "build: rename package to steward in pyproject"
```
### Task 13: Update `alembic.ini`
@@ -468,7 +468,7 @@ git commit -m "build: rename package to roundtable in pyproject"
```ini
[alembic]
script_location = roundtable/migrations
script_location = steward/migrations
prepend_sys_path = .
version_path_separator = os
```
@@ -477,7 +477,7 @@ version_path_separator = os
```bash
git add alembic.ini
git commit -m "build: point alembic at roundtable/migrations"
git commit -m "build: point alembic at steward/migrations"
```
### Task 14: Verify the package installs and imports cleanly
@@ -487,8 +487,8 @@ git commit -m "build: point alembic at roundtable/migrations"
```bash
python -m venv /tmp/rt-verify
/tmp/rt-verify/bin/pip install -e .
/tmp/rt-verify/bin/python -c "import roundtable; import roundtable.app; import roundtable.cli; print('ok')"
/tmp/rt-verify/bin/roundtable --help
/tmp/rt-verify/bin/python -c "import steward; import steward.app; import steward.cli; print('ok')"
/tmp/rt-verify/bin/steward --help
```
Expected: `ok` and CLI help output. No ImportError.
@@ -501,20 +501,20 @@ rm -rf /tmp/rt-verify
- [ ] **Step 3: No commit needed (verification only).**
**PR 2 done.** Open PR with title `refactor: rename python package fabledscryer → roundtable`.
**PR 2 done.** Open PR with title `refactor: rename python package fabledscryer → steward`.
---
## PR 3 — Config & Environment Variables
### Task 15: Add env-var fallback shim in `roundtable/config.py`
### Task 15: Add env-var fallback shim in `steward/config.py`
**Files:**
- Modify: `roundtable/config.py`
- Modify: `steward/config.py`
- [ ] **Step 1: Replace the env var lookups with a fallback helper**
Open `roundtable/config.py` and replace the existing `load_bootstrap` body so it reads `ROUNDTABLE_*` first and falls back to `FABLEDSCRYER_*` with a deprecation warning.
Open `steward/config.py` and replace the existing `load_bootstrap` body so it reads `STEWARD_*` first and falls back to `FABLEDSCRYER_*` with a deprecation warning.
```python
def _env_with_fallback(new_name: str, old_name: str) -> str | None:
@@ -536,19 +536,19 @@ Then inside `load_bootstrap`, replace the existing env lookups with:
```python
database_url = (
_env_with_fallback("ROUNDTABLE_DATABASE_URL", "FABLEDSCRYER_DATABASE_URL")
or _env_with_fallback("ROUNDTABLE_DATABASE__URL", "FABLEDSCRYER_DATABASE__URL")
_env_with_fallback("STEWARD_DATABASE_URL", "FABLEDSCRYER_DATABASE_URL")
or _env_with_fallback("STEWARD_DATABASE__URL", "FABLEDSCRYER_DATABASE__URL")
or raw.get("database", {}).get("url")
)
if not database_url:
raise ValueError(
"Database URL is required. Set ROUNDTABLE_DATABASE_URL env var "
"Database URL is required. Set STEWARD_DATABASE_URL env var "
"or add 'database.url' to config.yaml."
)
# ...
plugin_dir = (
_env_with_fallback("ROUNDTABLE_PLUGIN_DIR", "FABLEDSCRYER_PLUGIN_DIR")
_env_with_fallback("STEWARD_PLUGIN_DIR", "FABLEDSCRYER_PLUGIN_DIR")
or raw.get("plugin_dir", "plugins")
)
```
@@ -557,15 +557,15 @@ And in `_resolve_secret_key`:
```python
from_env = (
_env_with_fallback("ROUNDTABLE_SECRET_KEY", "FABLEDSCRYER_SECRET_KEY")
_env_with_fallback("STEWARD_SECRET_KEY", "FABLEDSCRYER_SECRET_KEY")
or raw.get("secret_key")
)
```
- [ ] **Step 2: Grep for any other `FABLEDSCRYER_` or `FABLEDNETMON_` env var reads across `roundtable/`**
- [ ] **Step 2: Grep for any other `FABLEDSCRYER_` or `FABLEDNETMON_` env var reads across `steward/`**
```bash
grep -rn "FABLEDSCRYER_\|FABLEDNETMON_" roundtable/
grep -rn "FABLEDSCRYER_\|FABLEDNETMON_" steward/
```
For each hit, apply the same pattern (new name first, old as fallback with warning). If `FABLEDNETMON_` is pure residue with no current consumer, delete the reference.
@@ -573,8 +573,8 @@ For each hit, apply the same pattern (new name first, old as fallback with warni
- [ ] **Step 3: Commit**
```bash
git add roundtable/config.py
git commit -m "feat: ROUNDTABLE_* env vars with FABLEDSCRYER_* fallback"
git add steward/config.py
git commit -m "feat: STEWARD_* env vars with FABLEDSCRYER_* fallback"
```
### Task 16: Update `.env.example` and `config.example.yaml`
@@ -588,11 +588,11 @@ git commit -m "feat: ROUNDTABLE_* env vars with FABLEDSCRYER_* fallback"
grep -n "FABLEDSCRYER\|FABLEDNETMON" .env.example
```
Replace every occurrence with `ROUNDTABLE_*`. Add a short comment at top:
Replace every occurrence with `STEWARD_*`. Add a short comment at top:
```
# Roundtable environment configuration.
# Only ROUNDTABLE_DATABASE_URL is required. Other settings live in the DB.
# Steward environment configuration.
# Only STEWARD_DATABASE_URL is required. Other settings live in the DB.
```
- [ ] **Step 2: Rewrite `config.example.yaml`**
@@ -600,25 +600,25 @@ Replace every occurrence with `ROUNDTABLE_*`. Add a short comment at top:
Replace the header and example strings:
```yaml
# Roundtable — Bootstrap Configuration Example
# Steward — Bootstrap Configuration Example
#
# The only REQUIRED setting is the database URL.
# Set it via env var (recommended for Docker):
#
# ROUNDTABLE_DATABASE_URL=postgresql+asyncpg://user:pass@host/db
# STEWARD_DATABASE_URL=postgresql+asyncpg://user:pass@host/db
#
# All other settings are stored in the database and managed via
# the Settings UI at /settings/.
database:
url: "postgresql+asyncpg://roundtable:password@localhost/roundtable"
url: "postgresql+asyncpg://steward:password@localhost/steward"
```
- [ ] **Step 3: Commit**
```bash
git add .env.example config.example.yaml
git commit -m "docs: ROUNDTABLE_* env var examples"
git commit -m "docs: STEWARD_* env var examples"
```
### Task 17: Manual config verification
@@ -626,8 +626,8 @@ git commit -m "docs: ROUNDTABLE_* env var examples"
- [ ] **Step 1: Run the app with only the new env var**
```bash
ROUNDTABLE_DATABASE_URL=postgresql+asyncpg://fabledscryer:fabledscryer@localhost/fabledscryer \
python -m roundtable.cli --host 127.0.0.1 --port 5001
STEWARD_DATABASE_URL=postgresql+asyncpg://fabledscryer:fabledscryer@localhost/fabledscryer \
python -m steward.cli --host 127.0.0.1 --port 5001
```
(Use whatever local DB you have; the old db name is fine — data isn't being renamed.)
@@ -638,14 +638,14 @@ Expected: app starts, no deprecation warning.
```bash
FABLEDSCRYER_DATABASE_URL=postgresql+asyncpg://fabledscryer:fabledscryer@localhost/fabledscryer \
python -m roundtable.cli --host 127.0.0.1 --port 5001
python -m steward.cli --host 127.0.0.1 --port 5001
```
Expected: app starts, deprecation warning logged once.
- [ ] **Step 3: Kill processes.** No commit.
**PR 3 done.** Open PR with title `feat: ROUNDTABLE_* env vars (FABLEDSCRYER_* fallback)`.
**PR 3 done.** Open PR with title `feat: STEWARD_* env vars (FABLEDSCRYER_* fallback)`.
---
@@ -664,12 +664,12 @@ COPY fabledscryer/ fabledscryer/
```
to
```dockerfile
COPY roundtable/ roundtable/
COPY steward/ steward/
```
Change the CMD:
```dockerfile
CMD ["roundtable", "--host", "0.0.0.0", "--port", "5000"]
CMD ["steward", "--host", "0.0.0.0", "--port", "5000"]
```
Leave everything else (apt packages, gosu, app user, EXPOSE 5000) untouched.
@@ -678,7 +678,7 @@ Leave everything else (apt packages, gosu, app user, EXPOSE 5000) untouched.
```bash
git add Dockerfile
git commit -m "build: update Dockerfile for roundtable package"
git commit -m "build: update Dockerfile for steward package"
```
### Task 19: Update `entrypoint.sh`
@@ -688,15 +688,15 @@ git commit -m "build: update Dockerfile for roundtable package"
- [ ] **Step 1: Replace the `nut_setup.py` invocation path**
Change every `/app/fabledscryer/nut_setup.py` to `/app/roundtable/nut_setup.py`.
Change every `/app/fabledscryer/nut_setup.py` to `/app/steward/nut_setup.py`.
Update any comment that says "FabledScryer container entrypoint." to "Roundtable container entrypoint."
Update any comment that says "FabledScryer container entrypoint." to "Steward container entrypoint."
- [ ] **Step 2: Commit**
```bash
git add entrypoint.sh
git commit -m "build: entrypoint.sh uses roundtable package path"
git commit -m "build: entrypoint.sh uses steward package path"
```
### Task 20: Update `docker-compose.yml`
@@ -708,10 +708,10 @@ git commit -m "build: entrypoint.sh uses roundtable package path"
```yaml
services:
roundtable:
steward:
build: .
container_name: roundtable
image: roundtable:latest
container_name: steward
image: steward:latest
ports:
- "5000:5000"
volumes:
@@ -720,7 +720,7 @@ services:
- /mnt/Data/traefik/log:/var/log/traefik:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- ROUNDTABLE_DATABASE_URL=postgresql+asyncpg://roundtable:roundtable@db/roundtable
- STEWARD_DATABASE_URL=postgresql+asyncpg://steward:steward@db/steward
depends_on:
db:
condition: service_healthy
@@ -729,13 +729,13 @@ services:
db:
image: postgres:16-alpine
environment:
POSTGRES_USER: roundtable
POSTGRES_PASSWORD: roundtable
POSTGRES_DB: roundtable
POSTGRES_USER: steward
POSTGRES_PASSWORD: steward
POSTGRES_DB: steward
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U roundtable"]
test: ["CMD-SHELL", "pg_isready -U steward"]
interval: 5s
timeout: 5s
retries: 5
@@ -752,37 +752,37 @@ volumes:
```bash
git add docker-compose.yml
git commit -m "build: docker-compose service → roundtable"
git commit -m "build: docker-compose service → steward"
```
### Task 21: Flip user-visible wordmark and title
**Files:**
- Modify: `roundtable/templates/base.html`
- Modify: `steward/templates/base.html`
- [ ] **Step 1: Update the `<title>` block (around line 6)**
```html
<title>{% block title %}Roundtable{% endblock %}</title>
<title>{% block title %}Steward{% endblock %}</title>
```
- [ ] **Step 2: Update the nav wordmark (around line 214)**
Change `Fabled Scryer` inside `nav .brand` to `Roundtable`.
Change `Fabled Scryer` inside `nav .brand` to `Steward`.
- [ ] **Step 3: Grep for any other user-visible "Fabled Scryer" strings in templates**
```bash
grep -rn "Fabled Scryer\|FabledScryer" roundtable/templates/
grep -rn "Fabled Scryer\|FabledScryer" steward/templates/
```
Replace each with "Roundtable".
Replace each with "Steward".
- [ ] **Step 4: Commit**
```bash
git add roundtable/templates/
git commit -m "copy: flip visible wordmark → Roundtable"
git add steward/templates/
git commit -m "copy: flip visible wordmark → Steward"
```
### Task 22: Manual container verification
@@ -795,8 +795,8 @@ docker compose up --build
```
- [ ] **Step 2: Verify**
- Container named `roundtable` runs
- Browser shows "Roundtable" in tab title and nav
- Container named `steward` runs
- Browser shows "Steward" in tab title and nav
- Login, dashboard, and error pages all render with the new palette and wordmark
- Logs show no import errors and no unhandled deprecation warnings
@@ -806,7 +806,7 @@ docker compose up --build
docker compose down
```
**PR 4 done.** Open PR with title `feat: roundtable container & wordmark flip`.
**PR 4 done.** Open PR with title `feat: steward container & wordmark flip`.
---
@@ -828,9 +828,9 @@ grep -rn "fabledscryer\|FabledScryer\|Fabled Scryer" README.md docs/
```bash
grep -rl "fabledscryer\|FabledScryer\|Fabled Scryer" README.md docs/ \
| xargs sed -i \
-e 's/FabledScryer/Roundtable/g' \
-e 's/fabledscryer/roundtable/g' \
-e 's/Fabled Scryer/Roundtable/g'
-e 's/FabledScryer/Steward/g' \
-e 's/fabledscryer/steward/g' \
-e 's/Fabled Scryer/Steward/g'
```
Then read each changed file top-to-bottom and fix any mangled sentences (e.g. capitalization at the start of sentences, code blocks that now reference a directory that still needs a `./` prefix, etc.). Pay extra attention to:
@@ -843,12 +843,12 @@ Then read each changed file top-to-bottom and fix any mangled sentences (e.g. ca
```bash
git add README.md docs/
git commit -m "docs: roundtable rename sweep"
git commit -m "docs: steward rename sweep"
```
### Task 24: Rename the Forgejo repo
- [ ] **Step 1: Via Forgejo UI or API, rename the repo from `FabledScryer` to `Roundtable`.** The old URL will redirect for one grace period.
- [ ] **Step 1: Via Forgejo UI or API, rename the repo from `FabledScryer` to `Steward`.** The old URL will redirect for one grace period.
- [ ] **Step 2: Update your local remote URL**
@@ -862,13 +862,13 @@ git remote -v
```bash
# from the parent dir
cd ..
mv FabledScryer Roundtable
cd Roundtable
mv FabledScryer Steward
cd Steward
```
- [ ] **Step 4: No commit.**
**PR 5 done.** Open PR with title `docs: roundtable rename sweep`.
**PR 5 done.** Open PR with title `docs: steward rename sweep`.
---
@@ -877,11 +877,11 @@ cd Roundtable
### Task 25: Remove env var fallback shim
**Files:**
- Modify: `roundtable/config.py`
- Modify: `steward/config.py`
- [ ] **Step 1: Delete `_env_with_fallback` and inline direct `os.environ.get("ROUNDTABLE_*")` lookups**
- [ ] **Step 1: Delete `_env_with_fallback` and inline direct `os.environ.get("STEWARD_*")` lookups**
Replace each `_env_with_fallback("ROUNDTABLE_X", "FABLEDSCRYER_X")` call with `os.environ.get("ROUNDTABLE_X")`. Delete the helper.
Replace each `_env_with_fallback("STEWARD_X", "FABLEDSCRYER_X")` call with `os.environ.get("STEWARD_X")`. Delete the helper.
- [ ] **Step 2: Grep to confirm no `FABLEDSCRYER_` references remain**
@@ -894,7 +894,7 @@ Expected: zero hits (or only historical commit messages, which don't show up in
- [ ] **Step 3: Commit**
```bash
git add roundtable/config.py
git add steward/config.py
git commit -m "refactor: remove FABLEDSCRYER_* env var fallback shim"
```
@@ -923,5 +923,5 @@ git commit -m "refactor: final rename sweep"
- Spec coverage: all 5 spec sections mapped to PRs 15; PR 6 handles the cleanup implied by the shim in PR 3.
- No placeholders — every template replacement includes concrete code; every grep includes the actual pattern.
- Type consistency: config helper is `_env_with_fallback` in both Task 15 and Task 25; env var names are `ROUNDTABLE_DATABASE_URL`, `ROUNDTABLE_PLUGIN_DIR`, `ROUNDTABLE_SECRET_KEY` consistently.
- Type consistency: config helper is `_env_with_fallback` in both Task 15 and Task 25; env var names are `STEWARD_DATABASE_URL`, `STEWARD_PLUGIN_DIR`, `STEWARD_SECRET_KEY` consistently.
- Tests intentionally skipped per project feedback; verification is manual (browser + container).
File diff suppressed because it is too large Load Diff
@@ -1,21 +1,21 @@
# Roundtable Rebrand — Design
# Steward Rebrand — Design
**Date:** 2026-04-13
**Status:** Approved for planning
## 1. Identity
**Name:** Roundtable (dropping the "Fabled" prefix).
**Name:** Steward (dropping the "Fabled" prefix).
**Why the change:** The old name `FabledScryer` framed the app as a single seer peering into one crystal ball. The actual product is a gathering point — hosts, metrics, alerts, plugins, and dashboards from across the homelab converge here. "Roundtable" captures that: a place where the realm's tools and information meet.
**Why the change:** The old name `FabledScryer` framed the app as a single seer peering into one crystal ball. The actual product is a gathering point — hosts, metrics, alerts, plugins, and dashboards from across the homelab converge here. "Steward" captures that: a place where the realm's tools and information meet.
**Name conflict check:**
- *Roundtable Software* — a Progress ABL tooling vendor. Different demographic, no overlap.
- *Steward Software* — a Progress ABL tooling vendor. Different demographic, no overlap.
- *Fabled.gg* — a virtual tabletop product. Closest concern, and the reason the "Fabled" prefix is dropped entirely to avoid confusion inside the tabletop/fantasy space.
**Umbrella:** `fabledsword.com` remains the family umbrella domain; `git.fabledsword.com` remains the plugin catalog host. Roundtable becomes one product under that umbrella.
**Umbrella:** `fabledsword.com` remains the family umbrella domain; `git.fabledsword.com` remains the plugin catalog host. Steward becomes one product under that umbrella.
**Tone:** Heraldic / Arthurian register. The roundtable is the gathering place of those who keep watch over the realm.
**Tone:** Heraldic / Arthurian register. The steward is the gathering place of those who keep watch over the realm.
## 2. Visual System
@@ -78,7 +78,7 @@ Candlelit glow — a soft, warm radial wash behind the main content area. Replac
### Wordmark
"Roundtable" in EB Garamond, gold (`#c8a840`), paired with the seal mark to the left in the nav header.
"Steward" in EB Garamond, gold (`#c8a840`), paired with the seal mark to the left in the nav header.
## 3. Voice & Copy
@@ -100,21 +100,21 @@ Candlelit glow — a soft, warm radial wash behind the main content area. Replac
## 4. Rename Mechanics
**Code & packaging**
- `fabledscryer/` package directory → `roundtable/`
- All `from fabledscryer...` / `import fabledscryer...``roundtable`
- `pyproject.toml`: project name, CLI entry point (`fabledscryer = "fabledscryer.cli:main"``roundtable = "roundtable.cli:main"`), tool sections
- `fabledscryer/` package directory → `steward/`
- All `from fabledscryer...` / `import fabledscryer...``steward`
- `pyproject.toml`: project name, CLI entry point (`fabledscryer = "fabledscryer.cli:main"``steward = "steward.cli:main"`), tool sections
- `__init__.py` package metadata
**Runtime config**
- Env vars: `FABLEDSCRYER_*``ROUNDTABLE_*`
- Env vars: `FABLEDSCRYER_*``STEWARD_*`
- Residual `FABLEDNETMON_*` env vars removed
- Config dir: `~/.config/fabledscryer``~/.config/roundtable` (one-shot copy on first boot)
- Config dir: `~/.config/fabledscryer``~/.config/steward` (one-shot copy on first boot)
- Log file names, systemd unit names if any
**Container & deploy**
- `Dockerfile` labels, workdir
- `docker-compose.yml` service name, image tag, volume names, `container_name`
- Published image: `fabledscryer:latest``roundtable:latest`
- Published image: `fabledscryer:latest``steward:latest`
**Database**
- Alembic `script_location` and any customized version table
@@ -144,10 +144,10 @@ Staged so `main` stays runnable between PRs.
New palette tokens, locked logo SVG, EB Garamond font swap, candlelit glow background, themed edge copy (login, empty states, 404/500, dashboard hero "Under Watch, Under Care"). Still named FabledScryer internally. Ships independently, low risk.
**PR 2 — Python package rename**
`fabledscryer/``roundtable/`, all imports, `pyproject.toml` name + entry point, `__init__.py` metadata. Docker/config untouched. Package installable as `roundtable` locally.
`fabledscryer/``steward/`, all imports, `pyproject.toml` name + entry point, `__init__.py` metadata. Docker/config untouched. Package installable as `steward` locally.
**PR 3 — Config & env vars**
`FABLEDSCRYER_*``ROUNDTABLE_*` with a short-lived fallback reader (accept both, warn on old) so self-hosters don't break mid-upgrade. Config dir migration on first boot. Drop `FABLEDNETMON_*` residue.
`FABLEDSCRYER_*``STEWARD_*` with a short-lived fallback reader (accept both, warn on old) so self-hosters don't break mid-upgrade. Config dir migration on first boot. Drop `FABLEDNETMON_*` residue.
**PR 4 — Container & deploy**
`Dockerfile`, `docker-compose.yml` service/image/volume/container names. Published image tag switch. Template wordmark + `<title>` + meta (the last user-visible string flip).
+1 -1
View File
@@ -1,5 +1,5 @@
#!/bin/sh
# Roundtable container entrypoint.
# Steward container entrypoint.
#
# Drops from root to the 'app' user via gosu, then runs the command.
# The container starts as root only so /data permissions can be fixed up
+1
View File
@@ -0,0 +1 @@
# plugins/__init__.py
+24
View File
@@ -0,0 +1,24 @@
# plugins/docker/__init__.py
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from quart import Quart
_app: "Quart | None" = None
def setup(app: "Quart") -> None:
global _app
_app = app
from .models import DockerContainer, DockerMetric # noqa: F401 — register with Base
def get_scheduled_tasks() -> list:
from .scheduler import make_task
return [make_task(_app)]
def get_blueprint():
from .routes import docker_bp
return docker_bp
+70
View File
@@ -0,0 +1,70 @@
# plugins/docker/migrations/env.py
"""Alembic env.py for the Docker plugin."""
import asyncio
import os
import sys
from pathlib import Path
from logging.config import fileConfig
from sqlalchemy import pool
from sqlalchemy.engine import Connection
from sqlalchemy.ext.asyncio import async_engine_from_config
from alembic import context
sys.path.insert(0, str(Path(__file__).parent.parent.parent.parent))
from steward.models.base import Base
import steward.models # noqa: F401
from plugins.docker.models import DockerContainer, DockerMetric # noqa: F401
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
target_metadata = Base.metadata
def _get_url() -> str:
import yaml
cfg_path = os.environ.get("STEWARD_CONFIG", "config.yaml")
try:
with open(cfg_path) as f:
cfg = yaml.safe_load(f) or {}
url = cfg.get("database", {}).get("url", "")
except FileNotFoundError:
url = ""
return os.environ.get("STEWARD_DATABASE__URL", url)
def run_migrations_offline() -> None:
context.configure(
url=_get_url(), target_metadata=target_metadata,
literal_binds=True, dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def do_run_migrations(connection: Connection) -> None:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
async def run_async_migrations() -> None:
cfg = config.get_section(config.config_ini_section, {})
cfg["sqlalchemy.url"] = _get_url()
connectable = async_engine_from_config(cfg, prefix="sqlalchemy.", poolclass=pool.NullPool)
async with connectable.connect() as connection:
await connection.run_sync(do_run_migrations)
await connectable.dispose()
def run_migrations_online() -> None:
asyncio.run(run_async_migrations())
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
@@ -0,0 +1,47 @@
"""Docker plugin initial tables
Revision ID: docker_001_initial
Revises: (none — branch off core via depends_on)
Create Date: 2026-03-22
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "docker_001_initial"
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = "docker"
depends_on: Union[str, Sequence[str], None] = ("0004_app_settings",)
def upgrade() -> None:
op.create_table(
"docker_containers",
sa.Column("name", sa.String(255), primary_key=True),
sa.Column("container_id", sa.String(64), nullable=False, server_default=""),
sa.Column("image", sa.String(512), nullable=False, server_default=""),
sa.Column("status", sa.String(32), nullable=False, server_default="unknown"),
sa.Column("cpu_pct", sa.Float, nullable=True),
sa.Column("mem_usage_bytes", sa.Integer, nullable=True),
sa.Column("mem_limit_bytes", sa.Integer, nullable=True),
sa.Column("mem_pct", sa.Float, nullable=True),
sa.Column("restart_count", sa.Integer, nullable=False, server_default="0"),
sa.Column("ports_json", sa.Text, nullable=False, server_default="[]"),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("scraped_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_table(
"docker_metrics",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("container_name", sa.String(255), nullable=False, index=True),
sa.Column("scraped_at", sa.DateTime(timezone=True), nullable=False, index=True),
sa.Column("cpu_pct", sa.Float, nullable=False, server_default="0"),
sa.Column("mem_pct", sa.Float, nullable=False, server_default="0"),
sa.Column("mem_usage_bytes", sa.Integer, nullable=False, server_default="0"),
)
def downgrade() -> None:
op.drop_table("docker_metrics")
op.drop_table("docker_containers")
+49
View File
@@ -0,0 +1,49 @@
# plugins/docker/models.py
from __future__ import annotations
import uuid
from datetime import datetime, timezone
from sqlalchemy import DateTime, Float, Integer, String, Text
from sqlalchemy.orm import Mapped, mapped_column
from steward.models.base import Base
class DockerContainer(Base):
"""Latest known state per container — upserted by name on each scrape."""
__tablename__ = "docker_containers"
# Container name is the stable natural key; container_id changes on recreation
name: Mapped[str] = mapped_column(String(255), primary_key=True)
container_id: Mapped[str] = mapped_column(String(64), nullable=False, default="")
image: Mapped[str] = mapped_column(String(512), nullable=False, default="")
status: Mapped[str] = mapped_column(String(32), nullable=False, default="unknown")
# running | stopped | paused | exited | dead
cpu_pct: Mapped[float | None] = mapped_column(Float, nullable=True)
mem_usage_bytes: Mapped[int | None] = mapped_column(Integer, nullable=True)
mem_limit_bytes: Mapped[int | None] = mapped_column(Integer, nullable=True)
mem_pct: Mapped[float | None] = mapped_column(Float, nullable=True)
restart_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
ports_json: Mapped[str] = mapped_column(Text, nullable=False, default="[]")
# JSON: [{"host_port": 8080, "container_port": 80, "protocol": "tcp"}]
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
scraped_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False,
default=lambda: datetime.now(timezone.utc),
)
class DockerMetric(Base):
"""Time-series CPU/memory per container — one row per scrape per running container."""
__tablename__ = "docker_metrics"
id: Mapped[str] = mapped_column(
String(36), primary_key=True, default=lambda: str(uuid.uuid4())
)
container_name: Mapped[str] = mapped_column(String(255), nullable=False, index=True)
scraped_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False,
default=lambda: datetime.now(timezone.utc),
index=True,
)
cpu_pct: Mapped[float] = mapped_column(Float, nullable=False, default=0.0)
mem_pct: Mapped[float] = mapped_column(Float, nullable=False, default=0.0)
mem_usage_bytes: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
+17
View File
@@ -0,0 +1,17 @@
name: docker
version: "1.0.0"
description: "Docker container status, resource usage, restart tracking via Docker socket"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/docker"
tags:
- containers
- docker
- infrastructure
config:
socket_path: /var/run/docker.sock
scrape_interval_seconds: 60
include_stopped: false
+160
View File
@@ -0,0 +1,160 @@
# plugins/docker/routes.py
from __future__ import annotations
import json
from quart import Blueprint, current_app, render_template, request
from sqlalchemy import Integer, cast, func, select
from steward.auth.middleware import require_role
from steward.models.users import UserRole
from steward.core.time_range import parse_range, DEFAULT_RANGE, bucket_seconds
from .models import DockerContainer, DockerMetric
docker_bp = Blueprint("docker", __name__, template_folder="templates")
def _sparkline(values: list[float], width: int = 80, height: int = 20) -> str:
if len(values) < 2:
return f'<svg width="{width}" height="{height}"></svg>'
mn, mx = min(values), max(values)
if mx == mn:
mx = mn + 1.0
step = width / (len(values) - 1)
pts = []
for i, v in enumerate(values):
x = i * step
y = height - (v - mn) / (mx - mn) * (height - 2) - 1
pts.append(f"{x:.1f},{y:.1f}")
poly = " ".join(pts)
return (
f'<svg width="{width}" height="{height}" viewBox="0 0 {width} {height}" '
f'style="vertical-align:middle;">'
f'<polyline points="{poly}" fill="none" stroke="#6060c0" stroke-width="1.5"/>'
f'</svg>'
)
@docker_bp.get("/")
@require_role(UserRole.viewer)
async def index():
poll_interval = current_app.config.get("MONITORS_POLL_INTERVAL", 60)
current_range = request.args.get("range", DEFAULT_RANGE)
return await render_template(
"docker/index.html",
poll_interval=poll_interval,
current_range=current_range,
)
@docker_bp.get("/rows")
@require_role(UserRole.viewer)
async def rows():
"""HTMX fragment: container list with status and resource sparklines."""
since, range_key = parse_range(request.args.get("range"))
b_secs = bucket_seconds(since)
bucket_col = (
cast(func.strftime('%s', DockerMetric.scraped_at), Integer) / b_secs
).label("bucket")
async with current_app.db_sessionmaker() as db:
# All known containers ordered by running first, then name
result = await db.execute(
select(DockerContainer)
.order_by(
# running first
(DockerContainer.status == "running").desc(),
DockerContainer.name,
)
)
containers = list(result.scalars())
# Build per-container sparkline histories
histories: dict[str, list] = {}
for c in containers:
result = await db.execute(
select(
func.avg(DockerMetric.cpu_pct).label("cpu_pct"),
func.avg(DockerMetric.mem_pct).label("mem_pct"),
bucket_col,
)
.where(DockerMetric.container_name == c.name)
.where(DockerMetric.scraped_at >= since)
.group_by(bucket_col)
.order_by(bucket_col)
)
histories[c.name] = result.all()
running = sum(1 for c in containers if c.status == "running")
stopped = len(containers) - running
container_data = []
for c in containers:
hist = histories.get(c.name, [])
ports = json.loads(c.ports_json) if c.ports_json else []
container_data.append({
"container": c,
"ports": ports,
"sparkline_cpu": _sparkline([r.cpu_pct or 0 for r in hist]),
"sparkline_mem": _sparkline([r.mem_pct or 0 for r in hist]),
})
return await render_template(
"docker/rows.html",
container_data=container_data,
running=running,
stopped=stopped,
range_key=range_key,
)
@docker_bp.get("/widget")
@require_role(UserRole.viewer)
async def widget():
"""HTMX dashboard widget: container status overview."""
show_stopped = request.args.get("show_stopped", "no") == "yes"
widget_id = request.args.get("wid", "0")
async with current_app.db_sessionmaker() as db:
result = await db.execute(
select(DockerContainer)
.order_by(
(DockerContainer.status == "running").desc(),
DockerContainer.name,
)
)
all_containers = list(result.scalars())
running = [c for c in all_containers if c.status == "running"]
stopped = [c for c in all_containers if c.status != "running"]
display = all_containers if show_stopped else running
return await render_template(
"docker/widget.html",
containers=display,
running_count=len(running),
stopped_count=len(stopped),
show_stopped=show_stopped,
widget_id=widget_id,
)
@docker_bp.get("/widget/resources")
@require_role(UserRole.viewer)
async def widget_resources():
"""HTMX dashboard widget: CPU + memory usage for running containers."""
limit = max(1, min(20, int(request.args.get("limit", 10) or 10)))
widget_id = request.args.get("wid", "0")
async with current_app.db_sessionmaker() as db:
result = await db.execute(
select(DockerContainer)
.where(DockerContainer.status == "running")
.order_by(DockerContainer.cpu_pct.desc().nullslast())
)
containers = list(result.scalars())[:limit]
return await render_template(
"docker/widget_resources.html",
containers=containers,
widget_id=widget_id,
)
+93
View File
@@ -0,0 +1,93 @@
# plugins/docker/scheduler.py
from __future__ import annotations
import json
import logging
from datetime import datetime, timezone
from steward.core.scheduler import ScheduledTask
from steward.core.alerts import record_metric
logger = logging.getLogger(__name__)
def make_task(app) -> ScheduledTask:
interval = int(
app.config["PLUGINS"]["docker"].get("scrape_interval_seconds", 60)
)
async def scrape():
await _do_scrape(app)
return ScheduledTask(
name="docker_scrape",
coro_factory=scrape,
interval_seconds=interval,
run_on_startup=True,
)
async def _do_scrape(app) -> None:
from .scraper import scrape_docker
from .models import DockerContainer, DockerMetric
cfg = app.config["PLUGINS"]["docker"]
socket_path = cfg.get("socket_path", "/var/run/docker.sock")
include_stopped = bool(cfg.get("include_stopped", False))
try:
containers = await scrape_docker(socket_path, include_stopped)
except ConnectionError as exc:
logger.error("Docker scrape failed: %s", exc)
return
except Exception:
logger.exception("Docker scrape error")
return
now = datetime.now(timezone.utc)
async with app.db_sessionmaker() as session:
async with session.begin():
for c in containers:
# Upsert container state
existing = await session.get(DockerContainer, c["name"])
if existing is None:
existing = DockerContainer(name=c["name"])
session.add(existing)
existing.container_id = c["container_id"]
existing.image = c["image"]
existing.status = c["status"]
existing.cpu_pct = c["cpu_pct"]
existing.mem_usage_bytes = c["mem_usage_bytes"]
existing.mem_limit_bytes = c["mem_limit_bytes"]
existing.mem_pct = c["mem_pct"]
existing.restart_count = c["restart_count"]
existing.ports_json = json.dumps(c["ports"])
existing.started_at = c["started_at"]
existing.scraped_at = now
# Time-series metric (running containers only)
if c["status"] == "running" and c["cpu_pct"] is not None:
session.add(DockerMetric(
container_name=c["name"],
scraped_at=now,
cpu_pct=c["cpu_pct"],
mem_pct=c["mem_pct"] or 0.0,
mem_usage_bytes=c["mem_usage_bytes"] or 0,
))
# Feed alert pipeline
await record_metric(
session=session,
source_module="docker",
resource_name=c["name"],
metric_name="cpu_pct",
value=c["cpu_pct"],
)
if c["mem_pct"] is not None:
await record_metric(
session=session,
source_module="docker",
resource_name=c["name"],
metric_name="mem_pct",
value=c["mem_pct"],
)
+139
View File
@@ -0,0 +1,139 @@
# plugins/docker/scraper.py
"""Docker API client via Unix socket."""
from __future__ import annotations
import asyncio
import logging
from datetime import datetime, timezone
import httpx
logger = logging.getLogger(__name__)
def _short_id(container_id: str) -> str:
return container_id[:12] if container_id else ""
def _calc_cpu_pct(stats: dict) -> float:
"""Calculate CPU % from a Docker stats snapshot (one-shot)."""
try:
cpu = stats.get("cpu_stats", {})
precpu = stats.get("precpu_stats", {})
cpu_delta = (
cpu["cpu_usage"]["total_usage"] - precpu["cpu_usage"]["total_usage"]
)
sys_delta = cpu.get("system_cpu_usage", 0) - precpu.get("system_cpu_usage", 0)
num_cpus = cpu.get("online_cpus") or len(
cpu["cpu_usage"].get("percpu_usage") or [None]
)
if sys_delta <= 0 or cpu_delta < 0:
return 0.0
return round((cpu_delta / sys_delta) * num_cpus * 100.0, 2)
except (KeyError, TypeError, ZeroDivisionError):
return 0.0
def _calc_mem(stats: dict) -> tuple[int, int, float]:
"""Return (usage_bytes, limit_bytes, mem_pct) from Docker stats."""
try:
mem = stats.get("memory_stats", {})
usage = mem.get("usage", 0)
limit = mem.get("limit", 0)
# Subtract page cache for working set (cgroup v1: "cache", v2: "inactive_file")
cache = mem.get("stats", {}).get("cache", 0) or \
mem.get("stats", {}).get("inactive_file", 0)
actual = max(0, usage - cache)
pct = round(actual / limit * 100.0, 2) if limit > 0 else 0.0
return actual, limit, pct
except (KeyError, TypeError):
return 0, 0, 0.0
def _parse_ports(ports: list) -> list[dict]:
"""Normalise Docker port bindings to a compact list."""
result = []
for p in (ports or []):
if p.get("PublicPort"):
result.append({
"host_port": p["PublicPort"],
"container_port": p["PrivatePort"],
"protocol": p.get("Type", "tcp"),
})
return result
async def scrape_docker(socket_path: str, include_stopped: bool) -> list[dict]:
"""
Fetch all containers + resource stats from the Docker daemon.
Returns a list of normalised dicts ready for the scheduler to persist.
Raises ConnectionError if the socket is unreachable.
"""
transport = httpx.AsyncHTTPTransport(uds=socket_path)
try:
async with httpx.AsyncClient(
transport=transport,
base_url="http://docker",
timeout=10.0,
) as client:
resp = await client.get(
"/containers/json",
params={"all": "true" if include_stopped else "false"},
)
resp.raise_for_status()
containers = resp.json()
async def _get_stats(c: dict) -> tuple[dict, dict | None]:
if c.get("State") != "running":
return c, None
try:
r = await client.get(
f"/containers/{c['Id']}/stats",
params={"stream": "false", "one-shot": "true"},
timeout=5.0,
)
return c, r.json()
except Exception as exc:
logger.debug(
"Stats unavailable for %s: %s", _short_id(c.get("Id", "")), exc
)
return c, None
pairs = await asyncio.gather(*[_get_stats(c) for c in containers])
except httpx.ConnectError as exc:
raise ConnectionError(
f"Cannot connect to Docker socket at {socket_path}: {exc}"
) from exc
results = []
for container, stats in pairs:
names = container.get("Names") or []
name = names[0].lstrip("/") if names else _short_id(container.get("Id", ""))
cpu_pct = mem_usage = mem_limit = mem_pct = None
if stats is not None:
cpu_pct = _calc_cpu_pct(stats)
mem_usage, mem_limit, mem_pct = _calc_mem(stats)
# Created is a Unix epoch int from /containers/json
created_ts = container.get("Created")
started_at = (
datetime.fromtimestamp(created_ts, tz=timezone.utc)
if isinstance(created_ts, (int, float)) else None
)
results.append({
"name": name,
"container_id": _short_id(container.get("Id", "")),
"image": container.get("Image", ""),
"status": container.get("State", "unknown"),
"cpu_pct": cpu_pct,
"mem_usage_bytes": mem_usage,
"mem_limit_bytes": mem_limit,
"mem_pct": mem_pct,
"restart_count": 0, # would require /inspect; left as 0 for now
"ports": _parse_ports(container.get("Ports", [])),
"started_at": started_at,
})
return results
@@ -0,0 +1,16 @@
{% extends "base.html" %}
{% block title %}Docker — Steward{% endblock %}
{% block content %}
<div style="display:flex;align-items:center;justify-content:space-between;margin-bottom:1.5rem;gap:1rem;flex-wrap:wrap;">
<h1 class="page-title" style="margin-bottom:0;">Docker</h1>
{% include "_time_range.html" %}
</div>
<div id="docker-rows"
hx-get="/plugins/docker/rows"
hx-trigger="load, rangeChange from:body"
hx-include="#time-range"
hx-swap="innerHTML">
<div style="color:var(--text-muted);font-size:0.9rem;padding:2rem;">Loading...</div>
</div>
{% endblock %}
+86
View File
@@ -0,0 +1,86 @@
{# docker/rows.html — HTMX fragment for the Docker main page #}
{# ── Summary strip ─────────────────────────────────────────────────────────── #}
<div style="display:grid;grid-template-columns:repeat(auto-fill,minmax(140px,1fr));gap:0.75rem;margin-bottom:1.5rem;">
<div class="card" style="margin-bottom:0;">
<div class="section-title" style="margin-bottom:0.35rem;">Running</div>
<span class="stat-val" style="color:var(--green);">{{ running }}</span>
</div>
<div class="card" style="margin-bottom:0;">
<div class="section-title" style="margin-bottom:0.35rem;">Stopped</div>
<span class="stat-val" style="{% if stopped %}color:var(--text-muted){% endif %};">{{ stopped }}</span>
</div>
<div class="card" style="margin-bottom:0;">
<div class="section-title" style="margin-bottom:0.35rem;">Total</div>
<span class="stat-val">{{ container_data | length }}</span>
</div>
</div>
{# ── Container table ─────────────────────────────────────────────────────── #}
{% if container_data %}
<div class="card-flush">
<table class="table">
<thead>
<tr>
<th>Container</th>
<th>Image</th>
<th>Ports</th>
<th>CPU %</th>
<th style="min-width:100px;">CPU history</th>
<th>Mem %</th>
<th style="min-width:100px;">Mem history</th>
</tr>
</thead>
<tbody>
{% for item in container_data %}
{% set c = item.container %}
<tr>
<td>
<div style="display:flex;align-items:center;gap:0.5rem;">
<span class="dot {% if c.status == 'running' %}dot-up{% elif c.status == 'paused' %}dot-warn{% else %}dot-down{% endif %}"></span>
<div>
<div style="font-weight:500;font-size:0.9rem;">{{ c.name }}</div>
<div style="font-size:0.73rem;color:var(--text-muted);">{{ c.status }}</div>
</div>
</div>
</td>
<td style="font-size:0.82rem;color:var(--text-muted);max-width:200px;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;">
{{ c.image }}
</td>
<td style="font-size:0.78rem;font-family:ui-monospace,monospace;">
{% for p in item.ports %}
<div style="color:var(--text-muted);">{{ p.host_port }}:{{ p.container_port }}/{{ p.protocol }}</div>
{% endfor %}
</td>
<td style="font-family:ui-monospace,monospace;font-size:0.88rem;">
{% if c.cpu_pct is not none %}
<span style="color:{% if c.cpu_pct > 80 %}var(--red){% elif c.cpu_pct > 50 %}var(--orange){% else %}var(--text){% endif %};">
{{ "%.1f" | format(c.cpu_pct) }}%
</span>
{% else %}
<span style="color:var(--text-dim);"></span>
{% endif %}
</td>
<td>{{ item.sparkline_cpu | safe }}</td>
<td style="font-family:ui-monospace,monospace;font-size:0.88rem;">
{% if c.mem_pct is not none %}
<span style="color:{% if c.mem_pct > 90 %}var(--red){% elif c.mem_pct > 70 %}var(--orange){% else %}var(--text){% endif %};">
{{ "%.1f" | format(c.mem_pct) }}%
</span>
{% else %}
<span style="color:var(--text-dim);"></span>
{% endif %}
</td>
<td>{{ item.sparkline_mem | safe }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<div class="card" style="text-align:center;padding:3rem;">
<div style="color:var(--text-muted);font-size:0.9rem;">
No containers found. Make sure the Docker socket is accessible and the plugin is configured correctly.
</div>
</div>
{% endif %}
@@ -0,0 +1,35 @@
{# docker/widget.html — dashboard widget: container status overview #}
{% if not containers and running_count == 0 and stopped_count == 0 %}
<div style="color:var(--text-muted);font-size:0.85rem;padding:0.5rem 0;">No containers found.</div>
{% else %}
<div style="display:flex;gap:1rem;margin-bottom:0.65rem;">
<div>
<span style="font-size:1.4rem;font-weight:700;line-height:1;color:var(--green);">{{ running_count }}</span>
<span style="font-size:0.78rem;color:var(--text-muted);margin-left:0.25rem;">running</span>
</div>
{% if stopped_count %}
<div>
<span style="font-size:1.4rem;font-weight:700;line-height:1;color:var(--text-muted);">{{ stopped_count }}</span>
<span style="font-size:0.78rem;color:var(--text-muted);margin-left:0.25rem;">stopped</span>
</div>
{% endif %}
</div>
<div style="display:grid;gap:2px;">
{% for c in containers %}
<div style="display:flex;align-items:center;gap:0.5rem;font-size:0.82rem;padding:0.15rem 0;">
<span class="dot {% if c.status == 'running' %}dot-up{% elif c.status == 'paused' %}dot-warn{% else %}dot-down{% endif %}"></span>
<span style="flex:1;min-width:0;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;">
{{ c.name }}
</span>
{% if c.cpu_pct is not none %}
<span style="font-size:0.72rem;color:var(--text-muted);font-family:ui-monospace,monospace;flex-shrink:0;">
{{ "%.1f" | format(c.cpu_pct) }}%
</span>
{% endif %}
</div>
{% endfor %}
</div>
{% endif %}
@@ -0,0 +1,36 @@
{# docker/widget_resources.html — dashboard widget: CPU + memory bars for running containers #}
{% if not containers %}
<div style="color:var(--text-muted);font-size:0.85rem;padding:0.5rem 0;">No running containers.</div>
{% else %}
<div style="display:grid;gap:0.5rem;">
{% for c in containers %}
<div>
<div style="display:flex;justify-content:space-between;align-items:baseline;margin-bottom:0.15rem;">
<span style="font-size:0.8rem;font-weight:500;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;max-width:55%;">
{{ c.name }}
</span>
<span style="font-size:0.72rem;color:var(--text-muted);font-family:ui-monospace,monospace;flex-shrink:0;margin-left:0.5rem;">
{% if c.cpu_pct is not none %}CPU {{ "%.1f" | format(c.cpu_pct) }}%{% endif %}
{% if c.mem_pct is not none %} · Mem {{ "%.1f" | format(c.mem_pct) }}%{% endif %}
</span>
</div>
{% if c.cpu_pct is not none %}
<div style="height:3px;background:var(--bg-elevated);border-radius:2px;margin-bottom:2px;">
<div style="height:100%;border-radius:2px;width:{{ [c.cpu_pct, 100] | min }}%;
background:{% if c.cpu_pct > 80 %}var(--red){% elif c.cpu_pct > 50 %}var(--orange){% else %}var(--accent){% endif %};
transition:width 0.4s;">
</div>
</div>
{% endif %}
{% if c.mem_pct is not none %}
<div style="height:3px;background:var(--bg-elevated);border-radius:2px;">
<div style="height:100%;border-radius:2px;width:{{ [c.mem_pct, 100] | min }}%;
background:{% if c.mem_pct > 90 %}var(--red){% elif c.mem_pct > 70 %}var(--orange){% else %}var(--green){% endif %};
transition:width 0.4s;">
</div>
</div>
{% endif %}
</div>
{% endfor %}
</div>
{% endif %}
+23
View File
@@ -0,0 +1,23 @@
# plugins/host_agent/__init__.py
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from quart import Quart
_app: "Quart | None" = None
def setup(app: "Quart") -> None:
global _app
_app = app
def get_scheduled_tasks() -> list:
from .scheduler import make_task
return [make_task(_app)]
def get_blueprint():
from .routes import host_agent_bp
return host_agent_bp
+399
View File
@@ -0,0 +1,399 @@
# plugins/host_agent/agent.py
"""Steward host agent — pushes resource metrics to a Steward instance.
Python 3.8+ stdlib only. Target ~300 lines. Served to targets at
GET /plugins/host_agent/agent.py.
"""
from __future__ import annotations
import json
import os
import shutil
import signal
import socket
import sys
import time
import urllib.error
import urllib.request
from collections import deque
from datetime import datetime, timezone
AGENT_VERSION = "1.1.0"
class ConfigError(Exception):
pass
REQUIRED_KEYS = ("url", "token")
INT_KEYS = ("interval_seconds",)
LIST_KEYS = ("mounts",)
def read_config(path: str) -> dict:
"""Parse a flat `key = value` config file."""
cfg: dict = {}
try:
with open(path, "r", encoding="utf-8") as f:
for lineno, raw in enumerate(f, 1):
line = raw.strip()
if not line or line.startswith("#"):
continue
if "=" not in line:
raise ConfigError(f"{path}:{lineno}: expected 'key = value'")
key, _, value = line.partition("=")
key = key.strip()
value = value.strip()
if key in INT_KEYS:
try:
cfg[key] = int(value)
except ValueError:
raise ConfigError(f"{path}:{lineno}: {key} must be int")
elif key in LIST_KEYS:
cfg[key] = [v.strip() for v in value.split(",") if v.strip()]
else:
cfg[key] = value
except FileNotFoundError:
raise ConfigError(f"{path}: not found")
missing = [k for k in REQUIRED_KEYS if k not in cfg]
if missing:
raise ConfigError(f"{path}: missing required key(s): {', '.join(missing)}")
cfg.setdefault("interval_seconds", 30)
return cfg
# ─── collectors ──────────────────────────────────────────────────────────────
STAT_PATH = "/proc/stat"
MEMINFO_PATH = "/proc/meminfo"
LOADAVG_PATH = "/proc/loadavg"
UPTIME_PATH = "/proc/uptime"
OS_RELEASE_PATH = "/etc/os-release"
def _read_file(path: str) -> str:
with open(path, "r", encoding="utf-8") as f:
return f.read()
def _parse_cpu_line(stat_text: str) -> tuple[int, int]:
"""Return (total_jiffies, idle_jiffies) for the aggregate cpu line."""
for line in stat_text.splitlines():
if line.startswith("cpu "):
parts = line.split()
fields = [int(x) for x in parts[1:]]
idle = fields[3] + (fields[4] if len(fields) > 4 else 0) # idle + iowait
total = sum(fields)
return total, idle
raise RuntimeError("no aggregate cpu line in /proc/stat")
def collect_cpu(sample_window: float = 0.2) -> float:
"""Return CPU utilization % over sample_window seconds."""
t1, i1 = _parse_cpu_line(_read_file(STAT_PATH))
time.sleep(sample_window)
t2, i2 = _parse_cpu_line(_read_file(STAT_PATH))
dt = t2 - t1
di = i2 - i1
if dt <= 0:
return 0.0
return round(100.0 * (dt - di) / dt, 2)
def collect_memory() -> dict:
info: dict[str, int] = {}
for line in _read_file(MEMINFO_PATH).splitlines():
if ":" not in line:
continue
key, _, rest = line.partition(":")
parts = rest.strip().split()
if not parts:
continue
try:
value_kb = int(parts[0])
except ValueError:
continue
info[key] = value_kb * 1024 # bytes
total = info.get("MemTotal", 0)
available = info.get("MemAvailable", 0)
swap_total = info.get("SwapTotal", 0)
swap_free = info.get("SwapFree", 0)
return {
"total_bytes": total,
"used_bytes": max(total - available, 0),
"available_bytes": available,
"swap_used_bytes": max(swap_total - swap_free, 0),
}
def collect_storage(mounts: list[str]) -> list[dict]:
out: list[dict] = []
for m in mounts:
try:
u = shutil.disk_usage(m)
except (FileNotFoundError, PermissionError):
continue
out.append({"mount": m, "total_bytes": u.total, "used_bytes": u.used})
return out
def collect_load() -> dict:
parts = _read_file(LOADAVG_PATH).split()
return {"1m": float(parts[0]), "5m": float(parts[1]), "15m": float(parts[2])}
def collect_uptime() -> int:
return int(float(_read_file(UPTIME_PATH).split()[0]))
def detect_primary_ip() -> str | None:
"""Best-effort primary non-loopback IPv4 of this host.
connect() on a SOCK_DGRAM socket only sets the default peer and selects the
outbound interface — no packets are sent — so getsockname() reveals the
source IP the kernel would use to reach the internet. This is the host's own
view of its address, so it survives reverse proxies / NAT (unlike the
server reading request.remote_addr). Returns None on any error (offline / no
route) and skips loopback.
"""
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
s.connect(("8.8.8.8", 80))
ip = s.getsockname()[0]
except OSError:
return None
finally:
s.close()
if not ip or ip.startswith("127."):
return None
return ip
def collect_metadata() -> dict:
u = os.uname()
distro = "unknown"
try:
for line in _read_file(OS_RELEASE_PATH).splitlines():
if line.startswith("PRETTY_NAME="):
distro = line.split("=", 1)[1].strip().strip('"')
break
except (OSError, FileNotFoundError):
pass
return {
"kernel": u.release,
"distro": distro,
"arch": u.machine,
"host_ip": detect_primary_ip(),
}
def default_mounts() -> list[str]:
"""Return real mount points from /proc/mounts, skipping pseudo filesystems."""
skip_types = {"tmpfs", "devtmpfs", "proc", "sysfs", "cgroup", "cgroup2",
"overlay", "squashfs", "ramfs", "devpts", "mqueue", "pstore",
"securityfs", "debugfs", "tracefs", "hugetlbfs", "fusectl",
"configfs", "bpf", "autofs", "efivarfs", "binfmt_misc"}
mounts: list[str] = []
try:
with open("/proc/mounts", "r") as f:
for line in f:
parts = line.split()
if len(parts) < 3:
continue
if parts[2] in skip_types:
continue
mounts.append(parts[1])
except OSError:
return ["/"]
return mounts or ["/"]
# ─── ring buffer ─────────────────────────────────────────────────────────────
class RingBuffer:
def __init__(self, maxlen: int = 20) -> None:
self._dq: deque = deque(maxlen=maxlen)
def push(self, item) -> None:
self._dq.append(item)
def drain(self) -> list:
out = list(self._dq)
self._dq.clear()
return out
def __len__(self) -> int:
return len(self._dq)
# ─── payload ─────────────────────────────────────────────────────────────────
def build_sample(mounts: list[str]) -> dict:
"""Collect one full sample. Partial samples allowed if a collector fails."""
sample: dict = {"ts": datetime.now(timezone.utc).isoformat()}
try:
sample["cpu_pct"] = collect_cpu()
except Exception:
sample["cpu_pct"] = None
try:
sample["mem"] = collect_memory()
except Exception:
sample["mem"] = None
try:
sample["load"] = collect_load()
except Exception:
sample["load"] = None
try:
sample["uptime_secs"] = collect_uptime()
except Exception:
sample["uptime_secs"] = None
try:
sample["storage"] = collect_storage(mounts)
except Exception:
sample["storage"] = []
return sample
def build_payload(samples: list[dict], hostname: str, metadata: dict) -> dict:
return {
"agent_version": AGENT_VERSION,
"hostname": hostname,
"metadata": metadata,
"samples": samples,
}
# ─── POST + backoff ──────────────────────────────────────────────────────────
BACKOFF_CAP = 300
def next_backoff(current: int) -> int:
if current <= 0:
return 30
return min(current * 2, BACKOFF_CAP)
def post_payload(url: str, token: str, payload: dict) -> tuple[bool, int | None]:
body = json.dumps(payload).encode("utf-8")
req = urllib.request.Request(
url.rstrip("/") + "/plugins/host_agent/ingest",
data=body,
headers={
"Content-Type": "application/json",
"Authorization": f"Bearer {token}",
"User-Agent": f"steward-host-agent/{AGENT_VERSION}",
},
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
return (200 <= resp.status < 300, resp.status)
except urllib.error.HTTPError as e:
return (False, e.code)
except (urllib.error.URLError, TimeoutError, socket.timeout, OSError):
return (False, None)
# ─── main loop ───────────────────────────────────────────────────────────────
_reload_requested = False
_shutdown_requested = False
def _handle_hup(_signum, _frame):
global _reload_requested
_reload_requested = True
def _handle_term(_signum, _frame):
global _shutdown_requested
_shutdown_requested = True
def _log(level: str, msg: str) -> None:
sys.stderr.write(f"[{level}] {msg}\n")
sys.stderr.flush()
def main_loop(conf_path: str) -> int:
global _reload_requested, _shutdown_requested
signal.signal(signal.SIGHUP, _handle_hup)
signal.signal(signal.SIGTERM, _handle_term)
try:
cfg = read_config(conf_path)
except ConfigError as e:
_log("ERROR", str(e))
return 2
metadata = collect_metadata()
hostname = cfg.get("hostname") or socket.gethostname()
mounts = cfg.get("mounts") or default_mounts()
buffer = RingBuffer(maxlen=20)
backoff = 0
_log("INFO", f"steward-host-agent {AGENT_VERSION} starting "
f"(url={cfg['url']}, interval={cfg['interval_seconds']}s)")
while not _shutdown_requested:
if _reload_requested:
try:
cfg = read_config(conf_path)
mounts = cfg.get("mounts") or default_mounts()
metadata = collect_metadata() # refresh host_ip/distro on reload
_log("INFO", "config reloaded")
except ConfigError as e:
_log("ERROR", f"reload failed: {e}")
_reload_requested = False
sample = build_sample(mounts)
buffered = buffer.drain()
payload = build_payload(
samples=buffered + [sample],
hostname=hostname,
metadata=metadata,
)
ok, status = post_payload(cfg["url"], cfg["token"], payload)
if ok:
if buffered:
_log("INFO", f"flushed {len(buffered)} buffered samples")
backoff = 0
sleep_for = cfg["interval_seconds"]
else:
if status == 400:
_log("ERROR", "server rejected payload (400) — dropping sample")
elif status == 401:
_log("ERROR", "token rejected (401) — check config + UI")
buffer.push(sample)
else:
_log("WARN", f"POST failed (status={status}); buffering")
for s in buffered:
buffer.push(s)
buffer.push(sample)
backoff = next_backoff(backoff)
sleep_for = backoff
slept = 0.0
while slept < sleep_for and not _shutdown_requested and not _reload_requested:
time.sleep(min(1.0, sleep_for - slept))
slept += 1.0
_log("INFO", "SIGTERM — flushing and exiting")
final = buffer.drain()
if final:
post_payload(cfg["url"], cfg["token"],
build_payload(final, hostname, metadata))
return 0
if __name__ == "__main__":
conf = os.environ.get("STEWARD_AGENT_CONFIG", "/etc/steward-agent.conf")
sys.exit(main_loop(conf))
+70
View File
@@ -0,0 +1,70 @@
# plugins/host_agent/migrations/env.py
"""Alembic env.py for the host_agent plugin."""
import asyncio
import os
import sys
from pathlib import Path
from logging.config import fileConfig
from sqlalchemy import pool
from sqlalchemy.engine import Connection
from sqlalchemy.ext.asyncio import async_engine_from_config
from alembic import context
sys.path.insert(0, str(Path(__file__).parent.parent.parent.parent))
from steward.models.base import Base
import steward.models # noqa: F401
from plugins.host_agent.models import HostAgentRegistration # noqa: F401
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
target_metadata = Base.metadata
def _get_url() -> str:
import yaml
cfg_path = os.environ.get("STEWARD_CONFIG", "config.yaml")
try:
with open(cfg_path) as f:
cfg = yaml.safe_load(f) or {}
url = cfg.get("database", {}).get("url", "")
except FileNotFoundError:
url = ""
return os.environ.get("STEWARD_DATABASE__URL", url)
def run_migrations_offline() -> None:
context.configure(
url=_get_url(), target_metadata=target_metadata,
literal_binds=True, dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def do_run_migrations(connection: Connection) -> None:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
async def run_async_migrations() -> None:
cfg = config.get_section(config.config_ini_section, {})
cfg["sqlalchemy.url"] = _get_url()
connectable = async_engine_from_config(cfg, prefix="sqlalchemy.", poolclass=pool.NullPool)
async with connectable.connect() as connection:
await connection.run_sync(do_run_migrations)
await connectable.dispose()
def run_migrations_online() -> None:
asyncio.run(run_async_migrations())
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
@@ -0,0 +1,36 @@
# plugins/host_agent/migrations/versions/host_agent_001_initial.py
"""host_agent plugin initial tables
Revision ID: host_agent_001_initial
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "host_agent_001_initial"
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = "host_agent"
depends_on: Union[str, Sequence[str], None] = ("0004_app_settings",)
def upgrade() -> None:
op.create_table(
"host_agent_registrations",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("host_id", sa.String(36),
sa.ForeignKey("hosts.id", ondelete="CASCADE"),
nullable=False, unique=True),
sa.Column("token_hash", sa.String(64), nullable=False, unique=True, index=True),
sa.Column("token_created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("agent_version", sa.String(32), nullable=True),
sa.Column("kernel", sa.String(128), nullable=True),
sa.Column("distro", sa.String(128), nullable=True),
sa.Column("arch", sa.String(32), nullable=True),
sa.Column("last_seen_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
)
def downgrade() -> None:
op.drop_table("host_agent_registrations")
@@ -0,0 +1,24 @@
# plugins/host_agent/migrations/versions/host_agent_002_host_ip.py
"""host_agent: add agent-reported host_ip column
Revision ID: host_agent_002_host_ip
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "host_agent_002_host_ip"
down_revision: Union[str, None] = "host_agent_001_initial"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"host_agent_registrations",
sa.Column("host_ip", sa.String(45), nullable=True),
)
def downgrade() -> None:
op.drop_column("host_agent_registrations", "host_ip")
+34
View File
@@ -0,0 +1,34 @@
# plugins/host_agent/models.py
from __future__ import annotations
import uuid
from datetime import datetime, timezone
from sqlalchemy import Column, String, DateTime, ForeignKey
from steward.models.base import Base
def _uuid() -> str:
return str(uuid.uuid4())
def _utcnow() -> datetime:
return datetime.now(timezone.utc)
class HostAgentRegistration(Base):
__tablename__ = "host_agent_registrations"
id = Column(String(36), primary_key=True, default=_uuid)
host_id = Column(String(36), ForeignKey("hosts.id", ondelete="CASCADE"),
unique=True, nullable=False)
token_hash = Column(String(64), nullable=False, unique=True, index=True)
token_created_at = Column(DateTime(timezone=True), nullable=False, default=_utcnow)
agent_version = Column(String(32), nullable=True)
kernel = Column(String(128), nullable=True)
distro = Column(String(128), nullable=True)
arch = Column(String(32), nullable=True)
# Agent-reported primary IP. 45 chars = max textual IPv6 (no zone suffix).
host_ip = Column(String(45), nullable=True)
last_seen_at = Column(DateTime(timezone=True), nullable=True)
created_at = Column(DateTime(timezone=True), nullable=False, default=_utcnow)
updated_at = Column(DateTime(timezone=True), nullable=False,
default=_utcnow, onupdate=_utcnow)
+19
View File
@@ -0,0 +1,19 @@
# plugins/host_agent/plugin.yaml
name: host_agent
version: "1.1.0"
description: "Remote Linux host resource monitoring via a lightweight Python push agent (CPU, memory, storage, load, uptime)"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/host_agent"
tags:
- host
- monitoring
- cpu
- memory
- storage
config:
stale_after_seconds: 180
default_interval_seconds: 30
+419
View File
@@ -0,0 +1,419 @@
# plugins/host_agent/routes.py
from __future__ import annotations
import hashlib
from datetime import datetime, timezone
from typing import Any
from pathlib import Path
import secrets
from quart import Blueprint, current_app, jsonify, request, Response, render_template, redirect, url_for
from steward.auth.middleware import require_role
from steward.models.users import UserRole
from sqlalchemy import select, func
from datetime import timedelta
from steward.core.settings import public_base_url
from steward.models.hosts import Host
from steward.models.metrics import PluginMetric
from .models import HostAgentRegistration
host_agent_bp = Blueprint("host_agent", __name__, template_folder="templates")
SOURCE_MODULE = "host_agent"
def _hash_token(raw: str) -> str:
return hashlib.sha256(raw.encode("utf-8")).hexdigest()
def _parse_ts(ts: str) -> datetime:
if ts.endswith("Z"):
ts = ts[:-1] + "+00:00"
return datetime.fromisoformat(ts)
def pick_host_address(current: str | None, reported: str | None) -> str | None:
"""Return the address to store on the Host, or None for no change.
The agent-reported IP fills Host.address only when the current value is
blank — an admin-typed address (DNS name, management IP) is never
overwritten. The reported IP is always kept on the registration regardless,
so admins can still see drift.
"""
if reported and not (current or "").strip():
return reported
return None
def _expand_sample_to_metrics(
sample: dict, host_name: str, recorded_at: datetime
) -> list[PluginMetric]:
rows: list[PluginMetric] = []
def row(metric: str, resource: str, value: float) -> None:
rows.append(PluginMetric(
source_module=SOURCE_MODULE,
resource_name=resource,
metric_name=metric,
value=float(value),
recorded_at=recorded_at,
))
if sample.get("cpu_pct") is not None:
row("cpu_pct", host_name, sample["cpu_pct"])
mem = sample.get("mem") or {}
if mem.get("total_bytes"):
total = mem["total_bytes"]
available = mem.get("available_bytes", 0)
used_pct = 100.0 * (total - available) / total if total else 0.0
row("mem_used_pct", host_name, used_pct)
row("mem_available_bytes", host_name, available)
row("swap_used_bytes", host_name, mem.get("swap_used_bytes", 0))
load = sample.get("load") or {}
for key in ("1m", "5m", "15m"):
if key in load:
row(f"load_{key}", host_name, load[key])
if sample.get("uptime_secs") is not None:
row("uptime_secs", host_name, sample["uptime_secs"])
worst_pct = 0.0
for disk in sample.get("storage") or []:
total = disk.get("total_bytes", 0)
used = disk.get("used_bytes", 0)
pct = 100.0 * used / total if total else 0.0
mount_res = f"{host_name}:{disk['mount']}"
row("disk_used_pct", mount_res, pct)
row("disk_used_bytes", mount_res, used)
row("disk_total_bytes", mount_res, total)
if pct > worst_pct:
worst_pct = pct
if sample.get("storage"):
row("disk_used_pct_worst", host_name, worst_pct)
return rows
def _error(status: int, code: str, detail: str | None = None) -> tuple[Any, int]:
body: dict = {"ok": False, "error": code}
if detail:
body["detail"] = detail
return jsonify(body), status
@host_agent_bp.post("/ingest")
async def ingest():
auth = request.headers.get("Authorization", "")
if not auth.startswith("Bearer "):
return _error(401, "invalid_token")
raw = auth[len("Bearer "):].strip()
if not raw:
return _error(401, "invalid_token")
try:
payload = await request.get_json(force=True)
except Exception:
return _error(400, "malformed_payload", "invalid JSON")
if not isinstance(payload, dict) or "samples" not in payload:
return _error(400, "malformed_payload", "missing 'samples'")
samples = payload.get("samples") or []
if not isinstance(samples, list) or not samples:
return _error(400, "malformed_payload", "'samples' must be a non-empty list")
token_hash = _hash_token(raw)
async with current_app.db_sessionmaker() as session:
reg = (await session.execute(
select(HostAgentRegistration).where(
HostAgentRegistration.token_hash == token_hash)
)).scalar_one_or_none()
if reg is None:
return _error(401, "invalid_token")
host = (await session.execute(
select(Host).where(Host.id == reg.host_id)
)).scalar_one_or_none()
if host is None:
return _error(401, "invalid_token")
accepted = 0
latest_ts: datetime | None = None
for sample in samples:
try:
recorded_at = _parse_ts(sample["ts"])
except (KeyError, ValueError):
continue
metrics = _expand_sample_to_metrics(sample, host.name, recorded_at)
for m in metrics:
session.add(m)
accepted += 1
if latest_ts is None or recorded_at > latest_ts:
latest_ts = recorded_at
if accepted == 0:
await session.rollback()
return _error(400, "malformed_payload", "no valid samples")
md = payload.get("metadata") or {}
changed = False
if latest_ts and (reg.last_seen_at is None or latest_ts > reg.last_seen_at):
reg.last_seen_at = latest_ts
changed = True
version = payload.get("agent_version")
if version and reg.agent_version != version:
reg.agent_version = version
changed = True
for field in ("kernel", "distro", "arch"):
if field in md and getattr(reg, field) != md[field]:
setattr(reg, field, md[field])
changed = True
host_ip = md.get("host_ip")
if host_ip and reg.host_ip != host_ip:
reg.host_ip = host_ip
changed = True
# Mirror the reported IP into Host.address only when it's blank.
new_address = pick_host_address(host.address, host_ip)
if new_address is not None:
host.address = new_address
if changed:
reg.updated_at = datetime.now(timezone.utc)
if latest_ts:
skew = abs((datetime.now(timezone.utc) - latest_ts).total_seconds())
if skew > 300:
current_app.logger.warning(
"host_agent ingest: clock skew %.0fs for host=%s", skew, host.name)
await session.commit()
return jsonify({"ok": True, "samples_accepted": accepted}), 200
AGENT_SOURCE_PATH = Path(__file__).parent / "agent.py"
def _agent_version() -> str:
try:
for line in AGENT_SOURCE_PATH.read_text().splitlines():
if line.startswith("AGENT_VERSION"):
return line.split("=", 1)[1].strip().strip('"').strip("'")
except OSError:
pass
return "unknown"
@host_agent_bp.get("/install.sh")
async def install_script():
token = request.args.get("token", "")
if not token:
return _error(401, "invalid_token")
async with current_app.db_sessionmaker() as session:
reg = (await session.execute(
select(HostAgentRegistration).where(
HostAgentRegistration.token_hash == _hash_token(token))
)).scalar_one_or_none()
if reg is None:
return _error(401, "invalid_token")
host = (await session.execute(
select(Host).where(Host.id == reg.host_id)
)).scalar_one_or_none()
if host is None:
return _error(401, "invalid_token")
url = public_base_url(request)
rendered = await render_template(
"install.sh.j2",
url=url,
token=token,
agent_version=_agent_version(),
host_name=host.name,
host_address=host.address,
)
return Response(rendered, mimetype="text/plain")
@host_agent_bp.get("/agent.py")
async def agent_source():
try:
body = AGENT_SOURCE_PATH.read_text(encoding="utf-8")
except OSError:
return _error(500, "agent_missing")
return Response(body, mimetype="text/x-python")
@host_agent_bp.get("/widget")
async def widget_table():
"""Fleet-glance table: one row per monitored host, latest metrics."""
async with current_app.db_sessionmaker() as session:
regs = (await session.execute(select(HostAgentRegistration))).scalars().all()
host_ids = [r.host_id for r in regs]
hosts = {
h.id: h for h in (await session.execute(
select(Host).where(Host.id.in_(host_ids))
)).scalars().all()
} if host_ids else {}
subq = (
select(
PluginMetric.resource_name,
PluginMetric.metric_name,
func.max(PluginMetric.recorded_at).label("max_ts"),
)
.where(PluginMetric.source_module == SOURCE_MODULE)
.group_by(PluginMetric.resource_name, PluginMetric.metric_name)
).subquery()
latest_rows = (await session.execute(
select(PluginMetric).join(
subq,
(PluginMetric.resource_name == subq.c.resource_name) &
(PluginMetric.metric_name == subq.c.metric_name) &
(PluginMetric.recorded_at == subq.c.max_ts),
).where(PluginMetric.source_module == SOURCE_MODULE)
)).scalars().all()
latest: dict[str, dict[str, float]] = {}
for row in latest_rows:
if ":" in row.resource_name:
continue
latest.setdefault(row.resource_name, {})[row.metric_name] = row.value
rows = []
for reg in regs:
host = hosts.get(reg.host_id)
if host is None:
continue
m = latest.get(host.name, {})
rows.append({
"host": host,
"reg": reg,
"cpu_pct": m.get("cpu_pct"),
"mem_used_pct": m.get("mem_used_pct"),
"disk_worst": m.get("disk_used_pct_worst"),
"load_1m": m.get("load_1m"),
})
return await render_template("widget_table.html", rows=rows)
@host_agent_bp.get("/widget/history")
async def widget_history():
host_id = request.args.get("host_id", "")
hours = int(request.args.get("hours", "6"))
cutoff = datetime.now(timezone.utc) - timedelta(hours=hours)
async with current_app.db_sessionmaker() as session:
host = (await session.execute(
select(Host).where(Host.id == host_id))).scalar_one_or_none()
if host is None:
return _error(404, "not_found")
points = (await session.execute(
select(PluginMetric).where(
PluginMetric.source_module == SOURCE_MODULE,
PluginMetric.resource_name == host.name,
PluginMetric.metric_name.in_(
("cpu_pct", "mem_used_pct", "disk_used_pct_worst")),
PluginMetric.recorded_at >= cutoff,
).order_by(PluginMetric.recorded_at)
)).scalars().all()
series: dict[str, list[dict]] = {"cpu_pct": [], "mem_used_pct": [], "disk_used_pct_worst": []}
for p in points:
series[p.metric_name].append({"t": p.recorded_at.isoformat(), "v": p.value})
return await render_template("widget_history.html", host=host, series=series, hours=hours)
def _new_token_pair() -> tuple[str, str]:
raw = secrets.token_urlsafe(32)
return raw, _hash_token(raw)
@host_agent_bp.get("/settings/")
@require_role(UserRole.admin)
async def settings_list():
async with current_app.db_sessionmaker() as session:
regs = (await session.execute(select(HostAgentRegistration))).scalars().all()
hosts_by_id = {
h.id: h for h in (await session.execute(
select(Host).where(Host.id.in_([r.host_id for r in regs])))).scalars().all()
} if regs else {}
new_token = request.args.get("new_token")
new_host_id = request.args.get("host_id")
install_url = None
if new_token and new_host_id:
install_url = f"{public_base_url(request)}/plugins/host_agent/install.sh?token={new_token}"
return await render_template(
"settings_list.html",
registrations=[
{"reg": r, "host": hosts_by_id.get(r.host_id)} for r in regs
],
new_token=new_token,
install_url=install_url,
)
@host_agent_bp.post("/settings/add-host")
@require_role(UserRole.admin)
async def add_host():
form = await request.form
name = (form.get("name") or "").strip()
address = (form.get("address") or "").strip()
if not name:
return _error(400, "missing_name")
async with current_app.db_sessionmaker() as session:
host = (await session.execute(
select(Host).where(Host.name == name))).scalar_one_or_none()
if host is None:
host = Host(name=name, address=address)
session.add(host)
await session.flush()
existing = (await session.execute(
select(HostAgentRegistration).where(
HostAgentRegistration.host_id == host.id))).scalar_one_or_none()
if existing is not None:
await session.rollback()
return _error(400, "already_registered")
raw, hashed = _new_token_pair()
reg = HostAgentRegistration(host_id=host.id, token_hash=hashed)
session.add(reg)
await session.commit()
host_id = host.id
return redirect(url_for("host_agent.settings_list") + f"?new_token={raw}&host_id={host_id}")
@host_agent_bp.post("/settings/<host_id>/rotate-token")
@require_role(UserRole.admin)
async def rotate_token(host_id: str):
async with current_app.db_sessionmaker() as session:
reg = (await session.execute(
select(HostAgentRegistration).where(
HostAgentRegistration.host_id == host_id))).scalar_one_or_none()
if reg is None:
return _error(404, "not_found")
raw, hashed = _new_token_pair()
reg.token_hash = hashed
reg.token_created_at = datetime.now(timezone.utc)
await session.commit()
return redirect(url_for("host_agent.settings_list") + f"?new_token={raw}&host_id={host_id}")
@host_agent_bp.post("/settings/<host_id>/delete")
@require_role(UserRole.admin)
async def delete_registration(host_id: str):
async with current_app.db_sessionmaker() as session:
reg = (await session.execute(
select(HostAgentRegistration).where(
HostAgentRegistration.host_id == host_id))).scalar_one_or_none()
if reg is not None:
await session.delete(reg)
await session.commit()
return redirect(url_for("host_agent.settings_list"))
+78
View File
@@ -0,0 +1,78 @@
# plugins/host_agent/scheduler.py
from __future__ import annotations
import logging
from datetime import datetime, timedelta, timezone
from typing import Iterable
from sqlalchemy import select
from steward.core.scheduler import ScheduledTask
from steward.models.hosts import Host
from .models import HostAgentRegistration
logger = logging.getLogger(__name__)
def _filter_stale(
regs: Iterable,
*,
now: datetime,
stale_after_seconds: int,
) -> list:
"""Pure staleness filter: returns the subset with last_seen_at strictly
older than (now - stale_after_seconds). Rows with last_seen_at=None are
never stale (they are unregistered-in-practice)."""
cutoff = now - timedelta(seconds=stale_after_seconds)
return [
r for r in regs
if r.last_seen_at is not None and r.last_seen_at < cutoff
]
async def find_stale_registrations(app, stale_after_seconds: int = 180) -> list[dict]:
async with app.db_sessionmaker() as session:
all_regs = (await session.execute(
select(HostAgentRegistration)
)).scalars().all()
stale_rows = _filter_stale(
all_regs,
now=datetime.now(timezone.utc),
stale_after_seconds=stale_after_seconds,
)
if not stale_rows:
return []
hosts = {
h.id: h for h in (await session.execute(
select(Host).where(Host.id.in_([r.host_id for r in stale_rows]))
)).scalars().all()
}
return [
{
"host_id": r.host_id,
"host_name": hosts[r.host_id].name if r.host_id in hosts else "?",
"last_seen_at": r.last_seen_at,
}
for r in stale_rows
]
def make_task(app) -> ScheduledTask:
async def _check_stale():
try:
stale = await find_stale_registrations(app)
except Exception:
logger.exception("host_agent stale check failed")
return
if stale:
logger.info(
"host_agent: %d stale agent(s): %s",
len(stale),
[s["host_name"] for s in stale],
)
return ScheduledTask(
name="host_agent_stale_check",
coro_factory=_check_stale,
interval_seconds=60,
run_on_startup=False,
)
@@ -0,0 +1,81 @@
#!/bin/sh
# Steward host agent installer
# Generated for: {{ host_name }} ({{ host_address }})
# Steward URL: {{ url }}
set -e
STEWARD_URL="{{ url }}"
AGENT_TOKEN="{{ token }}"
AGENT_VERSION="{{ agent_version }}"
AGENT_USER="steward-agent"
AGENT_DIR="/usr/local/lib/steward-agent"
CONF_FILE="/etc/steward-agent.conf"
UNIT_FILE="/etc/systemd/system/steward-agent.service"
# ── preflight ────────────────────────────────────────────────────────────────
[ "$(id -u)" = "0" ] || { echo "must run as root (use sudo)"; exit 1; }
command -v systemctl >/dev/null 2>&1 || { echo "systemd not found — unsupported init system"; exit 1; }
command -v python3 >/dev/null 2>&1 || { echo "python3 not found — install python3 first"; exit 1; }
# Handle --uninstall
if [ "${1:-}" = "--uninstall" ]; then
systemctl disable --now steward-agent.service 2>/dev/null || true
rm -f "$UNIT_FILE" "$CONF_FILE"
rm -rf "$AGENT_DIR"
systemctl daemon-reload
# keep the system user — removing it risks orphaned files elsewhere
echo "uninstalled"
exit 0
fi
# ── create system user ───────────────────────────────────────────────────────
if ! id "$AGENT_USER" >/dev/null 2>&1; then
useradd --system --no-create-home --shell /usr/sbin/nologin "$AGENT_USER"
fi
# ── drop the agent file ──────────────────────────────────────────────────────
mkdir -p "$AGENT_DIR"
curl -sSL "${STEWARD_URL}/plugins/host_agent/agent.py" -o "$AGENT_DIR/agent.py"
chmod 0755 "$AGENT_DIR/agent.py"
chown root:root "$AGENT_DIR/agent.py"
# ── write config ─────────────────────────────────────────────────────────────
cat > "$CONF_FILE" <<EOF
url = $STEWARD_URL
token = $AGENT_TOKEN
interval_seconds = 30
EOF
chown "root:$AGENT_USER" "$CONF_FILE"
chmod 0640 "$CONF_FILE"
# ── write systemd unit ───────────────────────────────────────────────────────
cat > "$UNIT_FILE" <<EOF
[Unit]
Description=Steward host agent
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=$AGENT_USER
ExecStart=/usr/bin/python3 $AGENT_DIR/agent.py
Restart=on-failure
RestartSec=10
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
ReadOnlyPaths=/proc /sys
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable --now steward-agent.service
echo
echo "Steward host agent $AGENT_VERSION installed and running."
echo "Check status: systemctl status steward-agent"
echo "Logs: journalctl -u steward-agent -f"
@@ -0,0 +1,66 @@
{% extends "base.html" %}
{% block title %}Host Agent — Steward{% endblock %}
{% block content %}
<h1 class="page-title">Host Agent — Registered Hosts</h1>
{% if new_token %}
<div class="card" style="background:var(--accent-bg);">
<h3>New token — copy this install command now</h3>
<p style="font-size:0.82rem;color:var(--text-muted);">
This is the only time the raw token is shown. Rotate if you lose it.
</p>
<pre style="user-select:all;word-break:break-all;">curl -sSL '{{ install_url }}' | sudo sh</pre>
</div>
{% endif %}
<div class="card">
<form method="post" action="/plugins/host_agent/settings/add-host"
style="display:flex;gap:0.75rem;align-items:flex-end;flex-wrap:wrap;">
<div class="form-group" style="margin-bottom:0;">
<label>Host name</label>
<input type="text" name="name" required>
</div>
<div class="form-group" style="margin-bottom:0;">
<label>Address (optional)</label>
<input type="text" name="address">
</div>
<button type="submit" class="btn">Add host</button>
</form>
</div>
<div class="card-flush">
<table class="table">
<thead>
<tr>
<th>Host</th><th>Reported IP</th><th>Agent version</th><th>Distro</th>
<th>Last seen</th><th class="td-actions">Actions</th>
</tr>
</thead>
<tbody>
{% for item in registrations %}
<tr>
<td>{{ item.host.name if item.host else item.reg.host_id }}</td>
<td>{{ item.reg.host_ip or "—" }}</td>
<td>{{ item.reg.agent_version or "—" }}</td>
<td>{{ item.reg.distro or "—" }}</td>
<td>{{ item.reg.last_seen_at.strftime("%Y-%m-%d %H:%M:%S") if item.reg.last_seen_at else "never" }}</td>
<td class="td-actions">
<form method="post" action="/plugins/host_agent/settings/{{ item.reg.host_id }}/rotate-token"
style="display:inline;"
onsubmit="return confirm('Rotate the token for this host? The existing agent will stop working until reinstalled with the new token.');">
<button type="submit" class="btn btn-ghost btn-sm">Rotate token</button>
</form>
<form method="post" action="/plugins/host_agent/settings/{{ item.reg.host_id }}/delete"
style="display:inline;"
onsubmit="return confirm('Delete this host registration? The agent will be unable to report metrics until re-registered.');">
<button type="submit" class="btn btn-danger btn-sm">Delete</button>
</form>
</td>
</tr>
{% else %}
<tr><td colspan="6" class="empty">No hosts registered. Add one above.</td></tr>
{% endfor %}
</tbody>
</table>
</div>
{% endblock %}
@@ -0,0 +1,24 @@
<div class="widget-history">
<h3>{{ host.name }} — last {{ hours }}h</h3>
<canvas id="host-agent-chart-{{ host.id }}" width="600" height="200"></canvas>
<script>
(function() {
const ctx = document.getElementById("host-agent-chart-{{ host.id }}").getContext("2d");
const series = {{ series|tojson }};
new Chart(ctx, {
type: "line",
data: {
datasets: [
{ label: "CPU %", data: series.cpu_pct.map(p => ({x: p.t, y: p.v})) },
{ label: "Mem %", data: series.mem_used_pct.map(p => ({x: p.t, y: p.v})) },
{ label: "Disk % worst", data: series.disk_used_pct_worst.map(p => ({x: p.t, y: p.v})) },
],
},
options: {
responsive: true,
scales: { x: { type: "time" }, y: { min: 0, max: 100 } },
},
});
})();
</script>
</div>
@@ -0,0 +1,34 @@
{# host_agent widget — fleet glance, flex rows (no header wrap) #}
{% if rows %}
<div style="display:grid;gap:0.1rem;">
{% for r in rows %}
{% set stale = not r.reg.last_seen_at %}
<div style="display:flex;align-items:center;gap:0.6rem;font-size:0.82rem;padding:0.3rem 0;border-bottom:1px solid var(--border);">
<span class="dot {% if stale %}dot-dim{% elif (r.cpu_pct or 0) >= 90 or (r.mem_used_pct or 0) >= 90 or (r.disk_worst or 0) >= 90 %}dot-warn{% else %}dot-up{% endif %}"></span>
<a href="/plugins/host_agent/{{ r.host.id }}/"
style="flex:1;min-width:0;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;font-weight:500;">
{{ r.host.name }}
</a>
<span style="display:flex;gap:0.75rem;font-variant-numeric:tabular-nums;color:var(--text-muted);font-size:0.75rem;flex-shrink:0;">
{% if r.cpu_pct is not none %}
<span title="CPU"><span style="color:var(--text-dim);">cpu</span> {{ "%.0f"|format(r.cpu_pct) }}%</span>
{% endif %}
{% if r.mem_used_pct is not none %}
<span title="Memory"><span style="color:var(--text-dim);">mem</span> {{ "%.0f"|format(r.mem_used_pct) }}%</span>
{% endif %}
{% if r.disk_worst is not none %}
<span title="Worst disk mount"><span style="color:var(--text-dim);">disk</span> {{ "%.0f"|format(r.disk_worst) }}%</span>
{% endif %}
{% if r.load_1m is not none %}
<span title="Load average 1m"><span style="color:var(--text-dim);">load</span> {{ "%.2f"|format(r.load_1m) }}</span>
{% endif %}
</span>
<span style="min-width:60px;text-align:right;font-size:0.72rem;color:var(--text-dim);flex-shrink:0;">
{{ r.reg.last_seen_at.strftime("%H:%M:%S") if r.reg.last_seen_at else "never" }}
</span>
</div>
{% endfor %}
</div>
{% else %}
<p class="empty" style="padding:0.5rem 0;font-size:0.85rem;">No hosts with agent data yet.</p>
{% endif %}
+24
View File
@@ -0,0 +1,24 @@
# plugins/http/__init__.py
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from quart import Quart
_app: "Quart | None" = None
def setup(app: "Quart") -> None:
global _app
_app = app
from .models import HttpMonitor, HttpResult # noqa: F401
def get_scheduled_tasks() -> list:
from .scheduler import make_task
return [make_task(_app)]
def get_blueprint():
from .routes import http_bp
return http_bp
+104
View File
@@ -0,0 +1,104 @@
# plugins/http/checker.py
"""Core HTTP check logic — runs a single monitor check and returns a result dict."""
from __future__ import annotations
import asyncio
import logging
import ssl
import time
from datetime import datetime, timezone
from urllib.parse import urlparse
import httpx
logger = logging.getLogger(__name__)
async def _get_tls_expiry(hostname: str, port: int) -> datetime | None:
"""Attempt a bare TLS handshake to extract the certificate expiry date."""
ctx = ssl.create_default_context()
try:
reader, writer = await asyncio.wait_for(
asyncio.open_connection(hostname, port, ssl=ctx),
timeout=5.0,
)
cert = writer.get_extra_info("ssl_object").getpeercert()
writer.close()
try:
await writer.wait_closed()
except Exception:
pass
exp_str = cert.get("notAfter", "")
if not exp_str:
return None
# Format: "Mar 22 12:00:00 2027 GMT"
return datetime.strptime(exp_str, "%b %d %H:%M:%S %Y %Z").replace(
tzinfo=timezone.utc
)
except Exception as exc:
logger.debug("TLS check failed for %s:%d%s", hostname, port, exc)
return None
async def run_check(
url: str,
method: str = "GET",
expected_status: int = 200,
content_match: str = "",
headers: dict | None = None,
timeout_seconds: int = 10,
follow_redirects: bool = True,
verify_ssl: bool = True,
) -> dict:
"""
Perform one HTTP check. Returns a dict with:
is_up, status_code, response_ms, content_matched, error_msg, tls_expires_at
"""
result: dict = {
"is_up": False,
"status_code": None,
"response_ms": None,
"content_matched": None,
"error_msg": None,
"tls_expires_at": None,
}
parsed = urlparse(url)
is_https = parsed.scheme.lower() == "https"
t0 = time.monotonic()
try:
async with httpx.AsyncClient(
follow_redirects=follow_redirects,
verify=verify_ssl,
timeout=timeout_seconds,
) as client:
response = await client.request(
method,
url,
headers=headers or {},
)
result["response_ms"] = round((time.monotonic() - t0) * 1000, 1)
result["status_code"] = response.status_code
status_ok = response.status_code == expected_status
if content_match:
matched = content_match in response.text
result["content_matched"] = matched
result["is_up"] = status_ok and matched
else:
result["is_up"] = status_ok
except httpx.TimeoutException:
result["response_ms"] = round((time.monotonic() - t0) * 1000, 1)
result["error_msg"] = "Timeout"
except httpx.ConnectError as exc:
result["error_msg"] = f"Connection error: {exc}"
except Exception as exc:
result["error_msg"] = str(exc)[:512]
# TLS expiry — only for HTTPS, and only when the check succeeded or we got a response
if is_https and result["status_code"] is not None:
port = parsed.port or 443
result["tls_expires_at"] = await _get_tls_expiry(parsed.hostname, port)
return result
+70
View File
@@ -0,0 +1,70 @@
# plugins/http/migrations/env.py
"""Alembic env.py for the HTTP monitoring plugin."""
import asyncio
import os
import sys
from pathlib import Path
from logging.config import fileConfig
from sqlalchemy import pool
from sqlalchemy.engine import Connection
from sqlalchemy.ext.asyncio import async_engine_from_config
from alembic import context
sys.path.insert(0, str(Path(__file__).parent.parent.parent.parent))
from steward.models.base import Base
import steward.models # noqa: F401
from plugins.http.models import HttpMonitor, HttpResult # noqa: F401
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
target_metadata = Base.metadata
def _get_url() -> str:
import yaml
cfg_path = os.environ.get("STEWARD_CONFIG", "config.yaml")
try:
with open(cfg_path) as f:
cfg = yaml.safe_load(f) or {}
url = cfg.get("database", {}).get("url", "")
except FileNotFoundError:
url = ""
return os.environ.get("STEWARD_DATABASE__URL", url)
def run_migrations_offline() -> None:
context.configure(
url=_get_url(), target_metadata=target_metadata,
literal_binds=True, dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def do_run_migrations(connection: Connection) -> None:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
async def run_async_migrations() -> None:
cfg = config.get_section(config.config_ini_section, {})
cfg["sqlalchemy.url"] = _get_url()
connectable = async_engine_from_config(cfg, prefix="sqlalchemy.", poolclass=pool.NullPool)
async with connectable.connect() as connection:
await connection.run_sync(do_run_migrations)
await connectable.dispose()
def run_migrations_online() -> None:
asyncio.run(run_async_migrations())
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
@@ -0,0 +1,52 @@
"""HTTP monitoring plugin initial tables
Revision ID: http_001_initial
Revises: (none — branch off core via depends_on)
Create Date: 2026-03-22
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "http_001_initial"
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = "http"
depends_on: Union[str, Sequence[str], None] = ("0004_app_settings",)
def upgrade() -> None:
op.create_table(
"http_monitors",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("name", sa.String(128), nullable=False),
sa.Column("url", sa.String(2048), nullable=False),
sa.Column("method", sa.String(8), nullable=False, server_default="GET"),
sa.Column("expected_status", sa.Integer, nullable=False, server_default="200"),
sa.Column("content_match", sa.String(512), nullable=False, server_default=""),
sa.Column("headers_json", sa.Text, nullable=False, server_default="{}"),
sa.Column("timeout_seconds", sa.Integer, nullable=False, server_default="10"),
sa.Column("check_interval_seconds", sa.Integer, nullable=False, server_default="0"),
sa.Column("follow_redirects", sa.Boolean, nullable=False, server_default="1"),
sa.Column("verify_ssl", sa.Boolean, nullable=False, server_default="1"),
sa.Column("enabled", sa.Boolean, nullable=False, server_default="1"),
sa.Column("last_checked_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_table(
"http_results",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("monitor_id", sa.String(36), nullable=False, index=True),
sa.Column("checked_at", sa.DateTime(timezone=True), nullable=False, index=True),
sa.Column("status_code", sa.Integer, nullable=True),
sa.Column("response_ms", sa.Float, nullable=True),
sa.Column("is_up", sa.Boolean, nullable=False, server_default="0"),
sa.Column("content_matched", sa.Boolean, nullable=True),
sa.Column("error_msg", sa.String(512), nullable=True),
sa.Column("tls_expires_at", sa.DateTime(timezone=True), nullable=True),
)
def downgrade() -> None:
op.drop_table("http_results")
op.drop_table("http_monitors")
+63
View File
@@ -0,0 +1,63 @@
# plugins/http/models.py
from __future__ import annotations
import uuid
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, Float, Integer, String, Text
from sqlalchemy.orm import Mapped, mapped_column
from steward.models.base import Base
class HttpMonitor(Base):
"""A configured HTTP endpoint to check periodically."""
__tablename__ = "http_monitors"
id: Mapped[str] = mapped_column(
String(36), primary_key=True, default=lambda: str(uuid.uuid4())
)
name: Mapped[str] = mapped_column(String(128), nullable=False)
url: Mapped[str] = mapped_column(String(2048), nullable=False)
method: Mapped[str] = mapped_column(String(8), nullable=False, default="GET")
expected_status: Mapped[int] = mapped_column(Integer, nullable=False, default=200)
# Optional substring to find in the response body (empty = skip check)
content_match: Mapped[str] = mapped_column(String(512), nullable=False, default="")
# JSON dict of extra request headers, e.g. {"Authorization": "Bearer ..."}
headers_json: Mapped[str] = mapped_column(Text, nullable=False, default="{}")
timeout_seconds: Mapped[int] = mapped_column(Integer, nullable=False, default=10)
# 0 = use global plugin check_interval_seconds
check_interval_seconds: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
follow_redirects: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
verify_ssl: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
# Updated after each check so the scheduler can compute next-run time efficiently
last_checked_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False,
default=lambda: datetime.now(timezone.utc),
)
class HttpResult(Base):
"""One check result for an HttpMonitor."""
__tablename__ = "http_results"
id: Mapped[str] = mapped_column(
String(36), primary_key=True, default=lambda: str(uuid.uuid4())
)
monitor_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
checked_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False,
default=lambda: datetime.now(timezone.utc),
index=True,
)
status_code: Mapped[int | None] = mapped_column(Integer, nullable=True)
response_ms: Mapped[float | None] = mapped_column(Float, nullable=True)
is_up: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
# None when no content_match is configured; True/False when it is
content_matched: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
error_msg: Mapped[str | None] = mapped_column(String(512), nullable=True)
# TLS expiry if the endpoint is HTTPS; None for HTTP or on error
tls_expires_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
+19
View File
@@ -0,0 +1,19 @@
name: http
version: "1.0.0"
description: "Synthetic HTTP endpoint monitoring — status code, response time, content match, TLS expiry"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/http"
tags:
- monitoring
- http
- synthetic
- uptime
config:
check_interval_seconds: 60
default_timeout_seconds: 10
follow_redirects: true
verify_ssl: true
+228
View File
@@ -0,0 +1,228 @@
# plugins/http/routes.py
from __future__ import annotations
import uuid
from datetime import datetime, timezone
from quart import Blueprint, current_app, render_template, request, redirect, url_for
from sqlalchemy import Integer, cast, func, select
from steward.auth.middleware import require_role
from steward.models.users import UserRole
from steward.core.time_range import parse_range, DEFAULT_RANGE, bucket_seconds
from .models import HttpMonitor, HttpResult
http_bp = Blueprint("http", __name__, template_folder="templates")
def _sparkline(values: list[float], width: int = 80, height: int = 20) -> str:
if len(values) < 2:
return f'<svg width="{width}" height="{height}"></svg>'
mn, mx = min(values), max(values)
if mx == mn:
mx = mn + 1.0
step = width / (len(values) - 1)
pts = []
for i, v in enumerate(values):
x = i * step
y = height - (v - mn) / (mx - mn) * (height - 2) - 1
pts.append(f"{x:.1f},{y:.1f}")
poly = " ".join(pts)
return (
f'<svg width="{width}" height="{height}" viewBox="0 0 {width} {height}" '
f'style="vertical-align:middle;">'
f'<polyline points="{poly}" fill="none" stroke="#6060c0" stroke-width="1.5"/>'
f'</svg>'
)
async def _render_rows(range_key: str = DEFAULT_RANGE):
since, range_key = parse_range(range_key)
b_secs = bucket_seconds(since)
bucket_col = (
cast(func.extract('epoch', HttpResult.checked_at), Integer) / b_secs
).label("bucket")
async with current_app.db_sessionmaker() as db:
result = await db.execute(
select(HttpMonitor).order_by(HttpMonitor.created_at)
)
monitors = list(result.scalars())
# Latest result per monitor
latest_map: dict[str, HttpResult] = {}
histories: dict[str, list] = {}
for m in monitors:
r = await db.execute(
select(HttpResult)
.where(HttpResult.monitor_id == m.id)
.order_by(HttpResult.checked_at.desc())
.limit(1)
)
latest = r.scalar_one_or_none()
if latest:
latest_map[m.id] = latest
r2 = await db.execute(
select(
func.avg(HttpResult.response_ms).label("response_ms"),
func.min(HttpResult.is_up.cast(Integer)).label("had_down"),
bucket_col,
)
.where(HttpResult.monitor_id == m.id)
.where(HttpResult.checked_at >= since)
.group_by(bucket_col)
.order_by(bucket_col)
)
histories[m.id] = r2.all()
monitor_data = []
for m in monitors:
hist = histories.get(m.id, [])
latest = latest_map.get(m.id)
now = datetime.now(timezone.utc)
tls_days = None
if latest and latest.tls_expires_at:
tls_days = (latest.tls_expires_at - now).total_seconds() / 86400.0
monitor_data.append({
"monitor": m,
"latest": latest,
"tls_days": tls_days,
"sparkline_ms": _sparkline([r.response_ms or 0 for r in hist]),
})
up = sum(1 for d in monitor_data if d["latest"] and d["latest"].is_up)
down = sum(1 for d in monitor_data if d["latest"] and not d["latest"].is_up)
return monitor_data, up, down, range_key
@http_bp.get("/")
@require_role(UserRole.viewer)
async def index():
poll_interval = current_app.config.get("MONITORS_POLL_INTERVAL", 60)
current_range = request.args.get("range", DEFAULT_RANGE)
return await render_template(
"http/index.html",
poll_interval=poll_interval,
current_range=current_range,
)
@http_bp.get("/rows")
@require_role(UserRole.viewer)
async def rows():
"""HTMX fragment: monitor list with status + sparklines."""
monitor_data, up, down, range_key = await _render_rows(
request.args.get("range", DEFAULT_RANGE)
)
return await render_template(
"http/rows.html",
monitor_data=monitor_data,
up=up,
down=down,
range_key=range_key,
)
@http_bp.post("/add")
@require_role(UserRole.operator)
async def add_monitor():
"""Add a new HTTP monitor."""
form = await request.form
url = form.get("url", "").strip()
if not url:
return redirect(url_for("http.index"))
if not url.startswith(("http://", "https://")):
url = "https://" + url
monitor = HttpMonitor(
id=str(uuid.uuid4()),
name=form.get("name", "").strip() or url,
url=url,
method=form.get("method", "GET").upper(),
expected_status=int(form.get("expected_status", 200) or 200),
content_match=form.get("content_match", "").strip(),
headers_json="{}",
timeout_seconds=int(form.get("timeout_seconds", 10) or 10),
check_interval_seconds=int(form.get("check_interval_seconds", 0) or 0),
follow_redirects="follow_redirects" in form,
verify_ssl="verify_ssl" in form,
enabled=True,
created_at=datetime.now(timezone.utc),
)
async with current_app.db_sessionmaker() as db:
async with db.begin():
db.add(monitor)
return redirect(url_for("http.index"))
@http_bp.post("/<monitor_id>/delete")
@require_role(UserRole.operator)
async def delete_monitor(monitor_id: str):
async with current_app.db_sessionmaker() as db:
async with db.begin():
m = await db.get(HttpMonitor, monitor_id)
if m:
await db.delete(m)
return redirect(url_for("http.index"))
@http_bp.post("/<monitor_id>/toggle")
@require_role(UserRole.operator)
async def toggle_monitor(monitor_id: str):
async with current_app.db_sessionmaker() as db:
async with db.begin():
m = await db.get(HttpMonitor, monitor_id)
if m:
m.enabled = not m.enabled
return redirect(url_for("http.index"))
@http_bp.get("/widget")
@require_role(UserRole.viewer)
async def widget():
"""HTMX dashboard widget: up/down counts + monitor list."""
show_down_only = request.args.get("show_down_only", "no") == "yes"
limit = max(1, min(20, int(request.args.get("limit", 10) or 10)))
widget_id = request.args.get("wid", "0")
async with current_app.db_sessionmaker() as db:
result = await db.execute(
select(HttpMonitor)
.where(HttpMonitor.enabled == True) # noqa: E712
.order_by(HttpMonitor.created_at)
)
monitors = list(result.scalars())
latest_map: dict[str, HttpResult] = {}
for m in monitors:
r = await db.execute(
select(HttpResult)
.where(HttpResult.monitor_id == m.id)
.order_by(HttpResult.checked_at.desc())
.limit(1)
)
latest = r.scalar_one_or_none()
if latest:
latest_map[m.id] = latest
monitor_data = [
{"monitor": m, "latest": latest_map.get(m.id)}
for m in monitors
]
up = sum(1 for d in monitor_data if d["latest"] and d["latest"].is_up)
down = sum(1 for d in monitor_data if d["latest"] and not d["latest"].is_up)
pending = sum(1 for d in monitor_data if not d["latest"])
if show_down_only:
monitor_data = [d for d in monitor_data if not (d["latest"] and d["latest"].is_up)]
return await render_template(
"http/widget.html",
monitor_data=monitor_data[:limit],
up=up,
down=down,
pending=pending,
show_down_only=show_down_only,
widget_id=widget_id,
)
+121
View File
@@ -0,0 +1,121 @@
# plugins/http/scheduler.py
from __future__ import annotations
import asyncio
import json
import logging
from datetime import datetime, timezone
from steward.core.scheduler import ScheduledTask
from steward.core.alerts import record_metric
logger = logging.getLogger(__name__)
def make_task(app) -> ScheduledTask:
interval = int(
app.config["PLUGINS"]["http"].get("check_interval_seconds", 60)
)
async def check():
await _do_checks(app)
return ScheduledTask(
name="http_check",
coro_factory=check,
interval_seconds=interval,
run_on_startup=True,
)
async def _do_checks(app) -> None:
from sqlalchemy import select
from .models import HttpMonitor, HttpResult
from .checker import run_check
cfg = app.config["PLUGINS"]["http"]
global_interval = int(cfg.get("check_interval_seconds", 60))
global_timeout = int(cfg.get("default_timeout_seconds", 10))
global_follow = bool(cfg.get("follow_redirects", True))
global_verify = bool(cfg.get("verify_ssl", True))
now = datetime.now(timezone.utc)
async with app.db_sessionmaker() as session:
result = await session.execute(
select(HttpMonitor).where(HttpMonitor.enabled == True) # noqa: E712
)
monitors = list(result.scalars())
# Filter to monitors whose next check time has passed
due: list[HttpMonitor] = []
for m in monitors:
effective_interval = m.check_interval_seconds or global_interval
if m.last_checked_at is None:
due.append(m)
elif (now - m.last_checked_at).total_seconds() >= effective_interval:
due.append(m)
if not due:
return
async def _check_one(monitor: HttpMonitor) -> tuple[HttpMonitor, dict]:
try:
headers = json.loads(monitor.headers_json or "{}")
except (ValueError, TypeError):
headers = {}
result = await run_check(
url=monitor.url,
method=monitor.method,
expected_status=monitor.expected_status,
content_match=monitor.content_match,
headers=headers,
timeout_seconds=monitor.timeout_seconds or global_timeout,
follow_redirects=monitor.follow_redirects if monitor.follow_redirects is not None else global_follow,
verify_ssl=monitor.verify_ssl if monitor.verify_ssl is not None else global_verify,
)
return monitor, result
pairs = await asyncio.gather(*[_check_one(m) for m in due], return_exceptions=True)
check_time = datetime.now(timezone.utc)
async with app.db_sessionmaker() as session:
async with session.begin():
for item in pairs:
if isinstance(item, Exception):
logger.error("HTTP check task error: %s", item)
continue
monitor, res = item
session.add(HttpResult(
monitor_id=monitor.id,
checked_at=check_time,
status_code=res["status_code"],
response_ms=res["response_ms"],
is_up=res["is_up"],
content_matched=res["content_matched"],
error_msg=res["error_msg"],
tls_expires_at=res["tls_expires_at"],
))
# Update monitor's last_checked_at for next-run scheduling
db_monitor = await session.get(HttpMonitor, monitor.id)
if db_monitor:
db_monitor.last_checked_at = check_time
# Alert pipeline
if res["response_ms"] is not None:
await record_metric(
session=session,
source_module="http",
resource_name=monitor.name,
metric_name="response_ms",
value=res["response_ms"],
)
await record_metric(
session=session,
source_module="http",
resource_name=monitor.name,
metric_name="is_up",
value=1.0 if res["is_up"] else 0.0,
)
+77
View File
@@ -0,0 +1,77 @@
{% extends "base.html" %}
{% block title %}HTTP Monitors — Steward{% endblock %}
{% block content %}
<div style="display:flex;align-items:center;justify-content:space-between;margin-bottom:1.5rem;gap:1rem;flex-wrap:wrap;">
<h1 class="page-title" style="margin-bottom:0;">HTTP Monitors</h1>
{% include "_time_range.html" %}
</div>
{# ── Add monitor form ─────────────────────────────────────────────────────── #}
<div class="card" style="margin-bottom:1.25rem;">
<div class="section-title" style="margin-bottom:0.75rem;">Add Monitor</div>
<form method="post" action="/plugins/http/add" style="display:grid;gap:0.6rem;">
<div style="display:grid;grid-template-columns:1fr 1fr;gap:0.6rem;">
<div>
<label style="font-size:0.75rem;color:var(--text-muted);display:block;margin-bottom:0.2rem;">Name</label>
<input type="text" name="name" placeholder="My Service" autocomplete="off"
style="width:100%;padding:0.4rem 0.65rem;background:var(--bg);border:1px solid var(--border-mid);border-radius:4px;color:var(--text);font-size:0.88rem;">
</div>
<div>
<label style="font-size:0.75rem;color:var(--text-muted);display:block;margin-bottom:0.2rem;">URL <span style="color:var(--red);">*</span></label>
<input type="text" name="url" placeholder="https://example.com" autocomplete="off" required
style="width:100%;padding:0.4rem 0.65rem;background:var(--bg);border:1px solid var(--border-mid);border-radius:4px;color:var(--text);font-size:0.88rem;">
</div>
</div>
<div style="display:grid;grid-template-columns:repeat(auto-fill,minmax(140px,1fr));gap:0.6rem;align-items:end;">
<div>
<label style="font-size:0.75rem;color:var(--text-muted);display:block;margin-bottom:0.2rem;">Method</label>
<select name="method" style="width:100%;padding:0.4rem 0.65rem;background:var(--bg);border:1px solid var(--border-mid);border-radius:4px;color:var(--text);font-size:0.88rem;">
<option value="GET">GET</option>
<option value="HEAD">HEAD</option>
<option value="POST">POST</option>
</select>
</div>
<div>
<label style="font-size:0.75rem;color:var(--text-muted);display:block;margin-bottom:0.2rem;">Expected status</label>
<input type="number" name="expected_status" value="200" min="100" max="599"
style="width:100%;padding:0.4rem 0.65rem;background:var(--bg);border:1px solid var(--border-mid);border-radius:4px;color:var(--text);font-size:0.88rem;">
</div>
<div>
<label style="font-size:0.75rem;color:var(--text-muted);display:block;margin-bottom:0.2rem;">Timeout (s)</label>
<input type="number" name="timeout_seconds" value="10" min="1" max="60"
style="width:100%;padding:0.4rem 0.65rem;background:var(--bg);border:1px solid var(--border-mid);border-radius:4px;color:var(--text);font-size:0.88rem;">
</div>
<div>
<label style="font-size:0.75rem;color:var(--text-muted);display:block;margin-bottom:0.2rem;">Interval (s, 0=global)</label>
<input type="number" name="check_interval_seconds" value="0" min="0"
style="width:100%;padding:0.4rem 0.65rem;background:var(--bg);border:1px solid var(--border-mid);border-radius:4px;color:var(--text);font-size:0.88rem;">
</div>
<div>
<label style="font-size:0.75rem;color:var(--text-muted);display:block;margin-bottom:0.2rem;">Content match (substring)</label>
<input type="text" name="content_match" placeholder="optional"
style="width:100%;padding:0.4rem 0.65rem;background:var(--bg);border:1px solid var(--border-mid);border-radius:4px;color:var(--text);font-size:0.88rem;">
</div>
<div style="display:flex;flex-direction:column;gap:0.3rem;padding-top:0.8rem;">
<label style="display:flex;align-items:center;gap:0.4rem;font-size:0.82rem;cursor:pointer;">
<input type="checkbox" name="follow_redirects" checked> Follow redirects
</label>
<label style="display:flex;align-items:center;gap:0.4rem;font-size:0.82rem;cursor:pointer;">
<input type="checkbox" name="verify_ssl" checked> Verify SSL
</label>
</div>
</div>
<div>
<button type="submit" class="btn btn-sm">Add Monitor</button>
</div>
</form>
</div>
{# ── Monitor list ─────────────────────────────────────────────────────────── #}
<div id="http-rows"
hx-get="/plugins/http/rows"
hx-trigger="load, every {{ poll_interval }}s, rangeChange from:body"
hx-include="#time-range"
hx-swap="innerHTML">
<div style="color:var(--text-muted);font-size:0.9rem;padding:2rem;">Loading...</div>
</div>
{% endblock %}
+109
View File
@@ -0,0 +1,109 @@
{# http/rows.html — HTMX fragment for HTTP monitor status list #}
{# ── Summary ─────────────────────────────────────────────────────────────── #}
<div style="display:grid;grid-template-columns:repeat(auto-fill,minmax(130px,1fr));gap:0.75rem;margin-bottom:1.25rem;">
<div class="card" style="margin-bottom:0;">
<div class="section-title" style="margin-bottom:0.3rem;">Up</div>
<span class="stat-val" style="color:var(--green);">{{ up }}</span>
</div>
<div class="card" style="margin-bottom:0;">
<div class="section-title" style="margin-bottom:0.3rem;">Down</div>
<span class="stat-val" style="color:{% if down %}var(--red){% else %}var(--text-muted){% endif %};">{{ down }}</span>
</div>
<div class="card" style="margin-bottom:0;">
<div class="section-title" style="margin-bottom:0.3rem;">Monitors</div>
<span class="stat-val">{{ monitor_data | length }}</span>
</div>
</div>
{# ── Monitor list ─────────────────────────────────────────────────────────── #}
{% if monitor_data %}
<div class="card-flush">
<table class="table">
<thead>
<tr>
<th>Monitor</th>
<th>Status</th>
<th>Response</th>
<th style="min-width:90px;">History</th>
<th>TLS expiry</th>
<th style="width:1%;"></th>
</tr>
</thead>
<tbody>
{% for item in monitor_data %}
{% set m = item.monitor %}
{% set latest = item.latest %}
<tr>
<td>
<div style="font-weight:500;font-size:0.9rem;">{{ m.name }}</div>
<div style="font-size:0.75rem;color:var(--text-muted);font-family:ui-monospace,monospace;word-break:break-all;">
<span style="color:var(--text-dim);">{{ m.method }}</span> {{ m.url }}
</div>
{% if not m.enabled %}
<span style="font-size:0.72rem;color:var(--text-dim);">paused</span>
{% endif %}
</td>
<td>
{% if not latest %}
<span style="color:var(--text-dim);font-size:0.82rem;">pending</span>
{% elif latest.is_up %}
<div style="display:flex;align-items:center;gap:0.4rem;">
<span class="dot dot-up"></span>
<span style="font-size:0.82rem;color:var(--green);">{{ latest.status_code }}</span>
</div>
{% else %}
<div style="display:flex;align-items:center;gap:0.4rem;">
<span class="dot dot-down"></span>
<span style="font-size:0.82rem;color:var(--red);">
{{ latest.status_code or latest.error_msg or "error" }}
</span>
</div>
{% if latest.content_matched == false %}
<div style="font-size:0.72rem;color:var(--orange);">content mismatch</div>
{% endif %}
{% endif %}
</td>
<td style="font-family:ui-monospace,monospace;font-size:0.88rem;">
{% if latest and latest.response_ms is not none %}
<span style="color:{% if latest.response_ms > 2000 %}var(--red){% elif latest.response_ms > 500 %}var(--orange){% else %}var(--text){% endif %};">
{{ "%.0f" | format(latest.response_ms) }}ms
</span>
{% else %}
<span style="color:var(--text-dim);"></span>
{% endif %}
</td>
<td>{{ item.sparkline_ms | safe }}</td>
<td style="font-size:0.82rem;">
{% if item.tls_days is not none %}
<span style="color:{% if item.tls_days < 14 %}var(--red){% elif item.tls_days < 30 %}var(--orange){% else %}var(--text-muted){% endif %};">
{{ item.tls_days | int }}d
</span>
{% elif m.url.startswith('https://') %}
<span style="color:var(--text-dim);"></span>
{% endif %}
</td>
<td style="white-space:nowrap;">
<form method="post" action="/plugins/http/{{ m.id }}/toggle" style="display:inline;margin:0;">
<button type="submit" class="btn btn-ghost btn-sm" style="font-size:0.72rem;">
{% if m.enabled %}Pause{% else %}Resume{% endif %}
</button>
</form>
<form method="post" action="/plugins/http/{{ m.id }}/delete"
style="display:inline;margin:0;"
onsubmit="return confirm('Delete {{ m.name }}?')">
<button type="submit" class="btn btn-danger btn-sm" style="font-size:0.72rem;">Del</button>
</form>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<div class="card" style="text-align:center;padding:3rem;">
<div style="color:var(--text-muted);font-size:0.9rem;">
No monitors configured yet. Add one using the form above.
</div>
</div>
{% endif %}
+56
View File
@@ -0,0 +1,56 @@
{# http/widget.html — dashboard widget: HTTP monitor summary #}
{% if not monitor_data and up == 0 and down == 0 and pending == 0 %}
<div style="color:var(--text-muted);font-size:0.85rem;padding:0.5rem 0;">
No monitors configured. <a href="/plugins/http/">Add monitors →</a>
</div>
{% else %}
<div style="display:flex;gap:1rem;margin-bottom:0.65rem;flex-wrap:wrap;">
{% if up %}
<div>
<span style="font-size:1.4rem;font-weight:700;line-height:1;color:var(--green);">{{ up }}</span>
<span style="font-size:0.78rem;color:var(--text-muted);margin-left:0.25rem;">up</span>
</div>
{% endif %}
{% if down %}
<div>
<span style="font-size:1.4rem;font-weight:700;line-height:1;color:var(--red);">{{ down }}</span>
<span style="font-size:0.78rem;color:var(--text-muted);margin-left:0.25rem;">down</span>
</div>
{% endif %}
{% if pending %}
<div>
<span style="font-size:1.4rem;font-weight:700;line-height:1;color:var(--text-dim);">{{ pending }}</span>
<span style="font-size:0.78rem;color:var(--text-muted);margin-left:0.25rem;">pending</span>
</div>
{% endif %}
</div>
<div style="display:grid;gap:3px;">
{% for item in monitor_data %}
{% set latest = item.latest %}
<div style="display:flex;align-items:center;gap:0.5rem;font-size:0.82rem;padding:0.15rem 0;">
{% if not latest %}
<span class="dot dot-dim"></span>
{% elif latest.is_up %}
<span class="dot dot-up"></span>
{% else %}
<span class="dot dot-down"></span>
{% endif %}
<span style="flex:1;min-width:0;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;">
{{ item.monitor.name }}
</span>
{% if latest and latest.response_ms is not none %}
<span style="font-size:0.72rem;font-family:ui-monospace,monospace;color:var(--text-muted);flex-shrink:0;">
{{ "%.0f" | format(latest.response_ms) }}ms
</span>
{% elif latest and not latest.is_up %}
<span style="font-size:0.72rem;color:var(--red);flex-shrink:0;">
{{ latest.status_code or "err" }}
</span>
{% endif %}
</div>
{% endfor %}
</div>
{% endif %}
+90
View File
@@ -0,0 +1,90 @@
# steward-plugins / index.yaml
#
# Plugin catalog for Steward.
# Fetched by the app at: Settings → Plugins → Plugin Catalog
#
# After updating an entry, commit and push — changes are live within 5 minutes
# (the app caches this file in-process for 300 seconds).
#
# See docs/plugins/index.yaml.example in the main app repo for the full schema.
version: 1
updated: "2026-03-22"
plugins:
- name: http
version: "1.0.0"
description: "Synthetic HTTP endpoint monitoring — status code, response time, content match, TLS expiry"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/http"
download_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins/releases/download/http-v1.0.0/http.zip"
checksum_sha256: ""
tags:
- monitoring
- http
- synthetic
- uptime
- name: docker
version: "1.0.0"
description: "Docker container status, resource usage, and restart tracking via Docker socket"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/docker"
download_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins/releases/download/docker-v1.0.0/docker.zip"
checksum_sha256: ""
tags:
- containers
- docker
- infrastructure
- name: traefik
version: "1.0.0"
description: "Traefik reverse proxy metrics and access log integration"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/traefik"
download_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins/releases/download/traefik-v1.0.0/traefik.zip"
checksum_sha256: ""
tags:
- proxy
- metrics
- access-log
- name: unifi
version: "1.0.0"
description: "UniFi Network controller integration — WAN health, devices, clients, DPI"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/unifi"
download_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins/releases/download/unifi-v1.0.0/unifi.zip"
checksum_sha256: ""
tags:
- network
- unifi
- ubiquiti
- name: ups
version: "1.0.0"
description: "UPS monitoring via NUT (Network UPS Tools) with Ansible shutdown automation"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/ups"
download_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins/releases/download/ups-v1.0.0/ups.zip"
checksum_sha256: ""
tags:
- ups
- power
- nut
+23
View File
@@ -0,0 +1,23 @@
# plugins/snmp/__init__.py
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from quart import Quart
_app: "Quart | None" = None
def setup(app: "Quart") -> None:
global _app
_app = app
def get_scheduled_tasks() -> list:
from .scheduler import make_poll_task
return [make_poll_task(_app)]
def get_blueprint():
from .routes import snmp_bp
return snmp_bp
+49
View File
@@ -0,0 +1,49 @@
# plugins/snmp/plugin.yaml
name: snmp
version: "1.0.0"
description: "SNMP polling for network devices — switches, routers, printers, UPSes, etc."
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/snmp"
tags:
- snmp
- network
- monitoring
config:
poll_interval_seconds: 60
# Each device is polled independently. OIDs must resolve to numeric values.
devices:
- name: "core-switch"
host: "192.168.1.1"
port: 161
community: "public"
version: "2c" # "1", "2c", or "3"
oids:
- oid: "1.3.6.1.2.1.1.3.0"
label: "uptime_centisecs"
- oid: "1.3.6.1.2.1.2.2.1.10.1"
label: "if1_in_octets"
- oid: "1.3.6.1.2.1.2.2.1.16.1"
label: "if1_out_octets"
# Example: network UPS with SNMP management card (RFC 1628 UPS-MIB).
# upsBatteryStatus values: 1=unknown, 2=batteryNormal, 3=batteryLow, 4=batteryDepleted.
# Uncomment and point at your UPS to enable.
# - name: "ups"
# host: "192.168.1.10"
# port: 161
# community: "public"
# version: "2c"
# oids:
# - oid: "1.3.6.1.2.1.33.1.2.1.0"
# label: "battery_status"
# - oid: "1.3.6.1.2.1.33.1.2.2.0"
# label: "seconds_on_battery"
# - oid: "1.3.6.1.2.1.33.1.2.3.0"
# label: "runtime_minutes_remaining"
# - oid: "1.3.6.1.2.1.33.1.2.4.0"
# label: "battery_charge_pct"
# - oid: "1.3.6.1.2.1.33.1.3.3.1.3.1"
# label: "input_voltage_dV"
+94
View File
@@ -0,0 +1,94 @@
# plugins/snmp/poller.py
"""
Synchronous SNMP GET helper, run via executor.
Requires pysnmp-lextudio (maintained pysnmp fork):
pip install 'steward[snmp]'
If pysnmp is not installed, poll_device() returns an empty dict and logs a warning.
"""
from __future__ import annotations
import logging
logger = logging.getLogger(__name__)
def _pysnmp_available() -> bool:
try:
import pysnmp # noqa: F401
return True
except ImportError:
return False
def _mp_model(version: str) -> int:
"""Map version string to pysnmp mpModel integer."""
return 0 if version == "1" else 1
def poll_device_sync(
host: str,
port: int,
community: str,
version: str,
oids: list[dict],
) -> dict[str, float]:
"""
Perform SNMP GET for each OID and return {label: float_value}.
Non-numeric OIDs (strings, etc.) are skipped.
Returns empty dict on any error.
"""
if not _pysnmp_available():
logger.warning("pysnmp not installed — SNMP polling disabled. "
"Install with: pip install 'steward[snmp]'")
return {}
from pysnmp.hlapi import (
CommunityData,
ContextData,
ObjectIdentity,
ObjectType,
SnmpEngine,
UdpTransportTarget,
getCmd,
)
results: dict[str, float] = {}
engine = SnmpEngine()
for oid_cfg in oids:
oid = oid_cfg["oid"]
label = oid_cfg.get("label") or oid
scale = float(oid_cfg.get("scale", 1.0))
try:
error_indication, error_status, error_index, var_binds = next(
getCmd(
engine,
CommunityData(community, mpModel=_mp_model(version)),
UdpTransportTarget((host, port), timeout=5, retries=1),
ContextData(),
ObjectType(ObjectIdentity(oid)),
)
)
except Exception as exc:
logger.debug("SNMP GET %s@%s OID %s failed: %s", host, port, oid, exc)
continue
if error_indication:
logger.debug("SNMP error %s@%s OID %s: %s", host, port, oid, error_indication)
continue
if error_status:
logger.debug("SNMP status %s@%s OID %s: %s at %s",
host, port, oid, error_status.prettyPrint(),
error_index and var_binds[int(error_index) - 1][0] or "?")
continue
for _, val in var_binds:
try:
results[label] = float(val) * scale
except (TypeError, ValueError):
# Non-numeric type (e.g. OctetString description) — skip
logger.debug("SNMP non-numeric value for %s label=%s: %r", oid, label, val)
return results
+146
View File
@@ -0,0 +1,146 @@
# plugins/snmp/routes.py
from __future__ import annotations
from datetime import datetime, timedelta, timezone
from quart import Blueprint, current_app, render_template, request
from sqlalchemy import func, select
from steward.auth.middleware import require_role
from steward.models.users import UserRole
from steward.models.metrics import PluginMetric
snmp_bp = Blueprint("snmp", __name__, template_folder="templates")
async def _latest_readings(db, device_names: list[str]) -> dict[str, dict[str, float]]:
"""Return {device_name: {label: latest_value}} for all configured devices."""
if not device_names:
return {}
# Latest value per (resource_name, metric_name) pair
subq = (
select(
PluginMetric.resource_name,
PluginMetric.metric_name,
func.max(PluginMetric.recorded_at).label("latest_at"),
)
.where(PluginMetric.source_module == "snmp")
.where(PluginMetric.resource_name.in_(device_names))
.group_by(PluginMetric.resource_name, PluginMetric.metric_name)
.subquery()
)
result = await db.execute(
select(PluginMetric).join(
subq,
(PluginMetric.resource_name == subq.c.resource_name)
& (PluginMetric.metric_name == subq.c.metric_name)
& (PluginMetric.recorded_at == subq.c.latest_at),
).where(PluginMetric.source_module == "snmp")
)
rows = result.scalars().all()
out: dict[str, dict[str, float]] = {}
for row in rows:
out.setdefault(row.resource_name, {})[row.metric_name] = row.value
return out
async def _history(db, device_name: str, hours: int = 24) -> dict[str, list]:
"""Return {label: [(recorded_at, value), ...]} for a single device."""
since = datetime.now(timezone.utc) - timedelta(hours=hours)
result = await db.execute(
select(PluginMetric)
.where(
PluginMetric.source_module == "snmp",
PluginMetric.resource_name == device_name,
PluginMetric.recorded_at >= since,
)
.order_by(PluginMetric.recorded_at)
)
rows = result.scalars().all()
out: dict[str, list] = {}
for row in rows:
out.setdefault(row.metric_name, []).append(
(row.recorded_at.isoformat(), row.value)
)
return out
@snmp_bp.get("/")
@require_role(UserRole.viewer)
async def index():
devices_cfg: list[dict] = current_app.config["PLUGINS"]["snmp"].get("devices", [])
device_names = [d.get("name") or d.get("host", "?") for d in devices_cfg]
async with current_app.db_sessionmaker() as db:
latest = await _latest_readings(db, device_names)
# Merge config + latest readings for the template
devices = []
for cfg in devices_cfg:
name = cfg.get("name") or cfg.get("host", "?")
devices.append({
"name": name,
"host": cfg.get("host", ""),
"oids": cfg.get("oids", []),
"readings": latest.get(name, {}),
})
poll_interval = current_app.config.get("MONITORS_POLL_INTERVAL", 60)
return await render_template("snmp/index.html",
devices=devices,
poll_interval=poll_interval)
@snmp_bp.get("/widget")
@require_role(UserRole.viewer)
async def widget():
devices_cfg: list[dict] = current_app.config["PLUGINS"]["snmp"].get("devices", [])
device_names = [d.get("name") or d.get("host", "?") for d in devices_cfg]
async with current_app.db_sessionmaker() as db:
latest = await _latest_readings(db, device_names[:10])
devices = []
for cfg in devices_cfg[:10]:
name = cfg.get("name") or cfg.get("host", "?")
devices.append({
"name": name,
"oids": cfg.get("oids", []),
"readings": latest.get(name, {}),
})
return await render_template("snmp/widget.html",
devices=devices,
total=len(devices_cfg))
@snmp_bp.get("/device/<device_name>")
@require_role(UserRole.viewer)
async def device_detail(device_name: str):
devices_cfg: list[dict] = current_app.config["PLUGINS"]["snmp"].get("devices", [])
device_cfg = next(
(d for d in devices_cfg if (d.get("name") or d.get("host")) == device_name),
None,
)
if device_cfg is None:
from quart import abort
abort(404)
hours = int(request.args.get("hours", 24))
async with current_app.db_sessionmaker() as db:
hist = await _history(db, device_name, hours=hours)
import json
history_json = json.dumps(hist)
oid_labels = [o.get("label") or o.get("oid") for o in device_cfg.get("oids", [])]
return await render_template(
"snmp/device.html",
device=device_cfg,
device_name=device_name,
oid_labels=oid_labels,
history_json=history_json,
hours=hours,
)
+76
View File
@@ -0,0 +1,76 @@
# plugins/snmp/scheduler.py
from __future__ import annotations
import asyncio
import logging
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from quart import Quart
from steward.core.scheduler import ScheduledTask
logger = logging.getLogger(__name__)
def make_poll_task(app: "Quart") -> ScheduledTask:
interval = app.config["PLUGINS"]["snmp"].get("poll_interval_seconds", 60)
async def poll() -> None:
await _do_poll(app)
return ScheduledTask(
name="snmp_poll",
coro_factory=poll,
interval_seconds=int(interval),
run_on_startup=True,
)
async def _do_poll(app: "Quart") -> None:
from .poller import poll_device_sync
from steward.core.alerts import record_metric
devices: list[dict] = app.config["PLUGINS"]["snmp"].get("devices", [])
if not devices:
return
loop = asyncio.get_event_loop()
async with app.db_sessionmaker() as session:
async with session.begin():
for device in devices:
if not isinstance(device, dict):
continue
name = device.get("name") or device.get("host", "unknown")
host = device.get("host", "")
port = int(device.get("port", 161))
community = device.get("community", "public")
version = str(device.get("version", "2c"))
oids = device.get("oids", [])
if not host or not oids:
continue
try:
readings = await loop.run_in_executor(
None,
poll_device_sync,
host, port, community, version, oids,
)
except Exception:
logger.exception("SNMP poll failed for device %s (%s)", name, host)
continue
for label, value in readings.items():
await record_metric(
session=session,
source_module="snmp",
resource_name=name,
metric_name=label,
value=value,
)
if readings:
logger.debug("SNMP polled %s (%s): %d OID(s)", name, host, len(readings))
else:
logger.debug("SNMP polled %s (%s): no readings", name, host)
+84
View File
@@ -0,0 +1,84 @@
{# plugins/snmp/templates/snmp/device.html #}
{% extends "base.html" %}
{% block title %}SNMP — {{ device_name }} — Steward{% endblock %}
{% block content %}
<div style="display:flex;align-items:center;gap:1rem;margin-bottom:1.5rem;flex-wrap:wrap;">
<a href="/plugins/snmp/" style="color:var(--text-muted);font-size:0.85rem;">&#8592; SNMP</a>
<h1 class="page-title" style="margin:0;">{{ device_name }}</h1>
<span style="font-size:0.8rem;color:var(--text-dim);">{{ device.host }}</span>
<div style="margin-left:auto;display:flex;gap:0.5rem;">
{% for h in [1, 6, 24, 168] %}
<a href="?hours={{ h }}"
class="btn btn-ghost"
style="font-size:0.78rem;padding:0.2rem 0.6rem;{% if hours == h %}background:var(--accent);color:#fff;border-color:var(--accent);{% endif %}">
{% if h < 24 %}{{ h }}h{% elif h == 24 %}24h{% else %}7d{% endif %}
</a>
{% endfor %}
</div>
</div>
{% if oid_labels %}
<div class="card">
<canvas id="snmp-chart" style="width:100%;max-height:320px;"></canvas>
</div>
<script>
(function(){
const raw = {{ history_json | safe }};
const labels = {{ oid_labels | tojson }};
const palette = ["#6060c0","#e07040","#40b080","#c040c0","#4080e0","#e0c040","#c08040"];
// Collect all unique timestamps, sorted
const tsSet = new Set();
for (const pts of Object.values(raw)) {
for (const [ts] of pts) tsSet.add(ts);
}
const allTs = Array.from(tsSet).sort();
if (allTs.length < 2) {
document.getElementById("snmp-chart").parentElement.innerHTML =
'<p style="color:var(--text-muted);padding:1rem;">Not enough data for a chart yet.</p>';
return;
}
const datasets = [];
labels.forEach((label, i) => {
const pts = raw[label] || [];
const map = Object.fromEntries(pts.map(([ts, v]) => [ts, v]));
datasets.push({
label,
data: allTs.map(ts => map[ts] ?? null),
borderColor: palette[i % palette.length],
backgroundColor: "transparent",
borderWidth: 1.5,
pointRadius: 0,
tension: 0.2,
spanGaps: true,
});
});
const ctx = document.getElementById("snmp-chart").getContext("2d");
new Chart(ctx, {
type: "line",
data: { labels: allTs.map(ts => ts.substring(11, 16)), datasets },
options: {
responsive: true,
interaction: { mode: "index", intersect: false },
plugins: {
legend: { position: "top", labels: { color: "#aaa", boxWidth: 12 } },
tooltip: { backgroundColor: "#1e1e2e", titleColor: "#ccc", bodyColor: "#aaa" },
},
scales: {
x: { ticks: { color: "#888", maxTicksLimit: 12 }, grid: { color: "#2a2a3a" } },
y: { ticks: { color: "#888" }, grid: { color: "#2a2a3a" } },
},
},
});
})();
</script>
{% else %}
<div class="card" style="color:var(--text-muted);text-align:center;padding:2rem;">
No OIDs configured for this device.
</div>
{% endif %}
{% endblock %}
+57
View File
@@ -0,0 +1,57 @@
{# plugins/snmp/templates/snmp/index.html #}
{% extends "base.html" %}
{% block title %}SNMP — Steward{% endblock %}
{% block content %}
<div style="display:flex;align-items:center;gap:1rem;margin-bottom:1.5rem;">
<h1 class="page-title" style="margin:0;">SNMP Devices</h1>
<span style="font-size:0.8rem;color:var(--text-dim);">{{ devices|length }} device(s) configured</span>
</div>
{% if not devices %}
<div class="card" style="color:var(--text-muted);text-align:center;padding:2rem;">
No SNMP devices configured. Add devices to <code>plugin.yaml</code>.
</div>
{% else %}
{% for device in devices %}
<div class="card" style="margin-bottom:1rem;">
<div style="display:flex;align-items:center;gap:1rem;margin-bottom:1rem;">
<h2 style="margin:0;font-size:1rem;font-weight:600;">{{ device.name }}</h2>
<span style="font-size:0.78rem;color:var(--text-dim);">{{ device.host }}</span>
{% if device.readings %}
<span style="font-size:0.75rem;color:var(--green);margin-left:auto;">&#9679; reachable</span>
{% else %}
<span style="font-size:0.75rem;color:var(--text-dim);margin-left:auto;">&#9675; no data yet</span>
{% endif %}
<a href="/plugins/snmp/device/{{ device.name }}" class="btn btn-ghost" style="font-size:0.78rem;padding:0.2rem 0.6rem;">History</a>
</div>
{% if device.readings %}
<div style="display:grid;grid-template-columns:repeat(auto-fill,minmax(200px,1fr));gap:0.75rem;">
{% for oid_cfg in device.oids %}
{% set label = oid_cfg.label if oid_cfg.label else oid_cfg.oid %}
{% set val = device.readings.get(label) %}
<div style="background:var(--bg-alt);border-radius:6px;padding:0.6rem 0.9rem;">
<div style="font-size:0.72rem;color:var(--text-dim);margin-bottom:0.2rem;font-family:monospace;">{{ label }}</div>
{% if val is not none %}
<div style="font-size:1.1rem;font-weight:600;font-variant-numeric:tabular-nums;">
{% if val >= 1000000 %}{{ "%.2f"|format(val / 1000000) }}<span style="font-size:0.75rem;color:var(--text-muted);">M</span>
{% elif val >= 1000 %}{{ "%.1f"|format(val / 1000) }}<span style="font-size:0.75rem;color:var(--text-muted);">K</span>
{% else %}{{ "%.4g"|format(val) }}{% endif %}
</div>
{% else %}
<div style="font-size:0.85rem;color:var(--text-dim);"></div>
{% endif %}
</div>
{% endfor %}
</div>
{% else %}
<p style="font-size:0.85rem;color:var(--text-muted);margin:0;">
No readings yet — waiting for next poll cycle.
</p>
{% endif %}
</div>
{% endfor %}
{% endif %}
{% endblock %}
+49
View File
@@ -0,0 +1,49 @@
{# plugins/snmp/templates/snmp/widget.html #}
{% if devices %}
{% for device in devices %}
<div class="ping-row" style="align-items:flex-start;flex-direction:column;gap:0.3rem;padding:0.4rem 0;">
<div style="display:flex;align-items:center;gap:0.6rem;width:100%;">
<span style="font-size:0.82rem;font-weight:500;flex:1;min-width:0;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;">
{{ device.name }}
</span>
{% if device.readings %}
<span style="font-size:0.7rem;color:var(--green);">&#9679;</span>
{% else %}
<span style="font-size:0.7rem;color:var(--text-dim);">&#9675;</span>
{% endif %}
<a href="/plugins/snmp/device/{{ device.name }}" style="font-size:0.72rem;color:var(--text-muted);">detail →</a>
</div>
{% if device.readings %}
<div style="display:flex;flex-wrap:wrap;gap:0.5rem 1rem;padding-left:0.1rem;">
{% for oid_cfg in device.oids[:4] %}
{% set label = oid_cfg.label if oid_cfg.label else oid_cfg.oid %}
{% set val = device.readings.get(label) %}
{% if val is not none %}
<span style="font-size:0.78rem;font-variant-numeric:tabular-nums;">
<span style="color:var(--text-muted);">{{ label }}</span>
<span style="margin-left:0.25rem;">
{% if val >= 1000000 %}{{ "%.1f"|format(val / 1000000) }}M
{% elif val >= 1000 %}{{ "%.0f"|format(val / 1000) }}K
{% else %}{{ "%.4g"|format(val) }}{% endif %}
</span>
</span>
{% endif %}
{% endfor %}
{% if device.oids|length > 4 %}
<span style="font-size:0.72rem;color:var(--text-dim);">+{{ device.oids|length - 4 }} more</span>
{% endif %}
</div>
{% endif %}
</div>
{% if not loop.last %}
<div style="height:1px;background:var(--border-low);margin:0.1rem 0;"></div>
{% endif %}
{% endfor %}
{% if total > devices|length %}
<div style="color:var(--text-dim);font-size:0.75rem;padding:0.25rem 0;">
+{{ total - devices|length }} more — <a href="/plugins/snmp/">view all →</a>
</div>
{% endif %}
{% else %}
<p class="empty" style="padding:0.5rem 0;font-size:0.85rem;">No SNMP devices configured.</p>
{% endif %}
+30
View File
@@ -0,0 +1,30 @@
# plugins/traefik/__init__.py
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from quart import Quart
_app: "Quart | None" = None
def setup(app: "Quart") -> None:
global _app
_app = app
from .models import TraefikMetric, TraefikCert, TraefikGlobalStat, TraefikAccessSummary # noqa: F401
def get_scheduled_tasks() -> list:
from .scheduler import make_scrape_task, make_access_log_task
tasks = [make_scrape_task(_app)]
access_log_cfg = _app.config["PLUGINS"]["traefik"].get("access_log", {})
if not isinstance(access_log_cfg, dict):
access_log_cfg = {}
if access_log_cfg.get("enabled", False):
tasks.append(make_access_log_task(_app))
return tasks
def get_blueprint():
from .routes import traefik_bp
return traefik_bp
+199
View File
@@ -0,0 +1,199 @@
# plugins/traefik/access_log.py
"""Traefik JSON access log tailer and traffic-origin aggregator.
Reads new lines from Traefik's JSON access log on each scheduler tick,
classifies source IPs as internal (RFC1918) or external, optionally
looks up country codes via MaxMind GeoLite2, and returns an aggregated
summary ready to persist in TraefikAccessSummary.
"""
from __future__ import annotations
import ipaddress
import json
from collections import defaultdict
from pathlib import Path
# ── Module-level file-position state ─────────────────────────────────────────
_log_position: int = 0 # byte offset of last read
_log_inode: int = 0 # detect rotation by inode change
# ── Private IP ranges (RFC1918 + loopback + link-local) ──────────────────────
_PRIVATE_NETS = [
ipaddress.ip_network("10.0.0.0/8"),
ipaddress.ip_network("172.16.0.0/12"),
ipaddress.ip_network("192.168.0.0/16"),
ipaddress.ip_network("127.0.0.0/8"),
ipaddress.ip_network("169.254.0.0/16"),
ipaddress.ip_network("::1/128"),
ipaddress.ip_network("fc00::/7"),
ipaddress.ip_network("fe80::/10"),
]
def _strip_port(raw: str) -> str:
"""Strip trailing :port from an IPv4 address string. Leaves IPv6 untouched."""
if not raw:
return raw
# Bracketed IPv6 with port: [::1]:12345
if raw.startswith("["):
return raw[1:raw.index("]")] if "]" in raw else raw
# IPv4:port has exactly one colon
if raw.count(":") == 1:
return raw.rsplit(":", 1)[0]
return raw
def is_internal(raw_ip: str) -> bool:
"""Return True if the address falls within a private/loopback range."""
ip_str = _strip_port(raw_ip)
try:
addr = ipaddress.ip_address(ip_str)
return any(addr in net for net in _PRIVATE_NETS)
except ValueError:
return False
def _country(ip_str: str, geoip_db) -> str | None:
"""Return ISO country code via an open maxminddb reader, or None."""
if geoip_db is None:
return None
clean = _strip_port(ip_str)
try:
record = geoip_db.get(clean)
if record:
return record.get("country", {}).get("iso_code")
except Exception:
pass
return None
def read_new_lines(log_path: str) -> list[dict]:
"""Return all new JSON-parsed access log entries since the last call.
Handles log rotation: if the file shrinks or its inode changes, we
reset to the beginning of the new file.
"""
global _log_position, _log_inode
path = Path(log_path)
if not path.exists():
return []
try:
stat = path.stat()
except OSError:
return []
current_inode = stat.st_ino
current_size = stat.st_size
# Rotation detected
if current_inode != _log_inode or current_size < _log_position:
_log_position = 0
_log_inode = current_inode
if current_size == _log_position:
return []
entries: list[dict] = []
try:
with open(log_path, "rb") as f:
f.seek(_log_position)
raw = f.read(current_size - _log_position)
_log_position = current_size
except OSError:
return []
for raw_line in raw.split(b"\n"):
raw_line = raw_line.strip()
if not raw_line:
continue
try:
entries.append(json.loads(raw_line))
except json.JSONDecodeError:
continue
return entries
def aggregate(entries: list[dict], geoip_db=None, top_n: int = 10) -> dict | None:
"""Aggregate a batch of access log entries into a summary dict.
Returns None if there are no entries.
Result keys map directly to TraefikAccessSummary columns (excluding
id and period_start/period_end, which the caller provides).
"""
if not entries:
return None
total = 0
internal = 0
external = 0
total_bytes = 0.0
by_router: dict[str, int] = defaultdict(int)
ip_counts: dict[str, int] = defaultdict(int)
ip_internal: dict[str, bool] = {}
ip_country: dict[str, str | None] = {}
country_counts: dict[str, int] = defaultdict(int)
for entry in entries:
total += 1
raw_ip = entry.get("ClientHost", "")
router = entry.get("RouterName") or entry.get("ServiceName") or "unknown"
total_bytes += entry.get("DownstreamContentSize") or 0
by_router[router] += 1
internal_flag = is_internal(raw_ip)
if internal_flag:
internal += 1
else:
external += 1
if raw_ip not in ip_country:
ip_country[raw_ip] = _country(raw_ip, geoip_db)
c = ip_country[raw_ip]
if c:
country_counts[c] += 1
ip_counts[raw_ip] += 1
ip_internal[raw_ip] = internal_flag
top_ips = [
{
"ip": ip,
"count": cnt,
"internal": ip_internal[ip],
"country": ip_country.get(ip),
}
for ip, cnt in sorted(ip_counts.items(), key=lambda x: x[1], reverse=True)[:top_n]
]
top_countries = [
{"country": c, "count": n}
for c, n in sorted(country_counts.items(), key=lambda x: x[1], reverse=True)[:top_n]
]
top_routers = [
{"router": r, "count": n}
for r, n in sorted(by_router.items(), key=lambda x: x[1], reverse=True)[:top_n]
]
return {
"total_requests": total,
"internal_requests": internal,
"external_requests": external,
"total_bytes": total_bytes,
"top_ips_json": json.dumps(top_ips),
"top_countries_json": json.dumps(top_countries),
"top_routers_json": json.dumps(top_routers),
}
def open_geoip(db_path: str | None):
"""Open a MaxMind mmdb file if available. Returns None if not configured or missing."""
if not db_path:
return None
try:
import maxminddb # type: ignore[import]
return maxminddb.open_database(db_path)
except Exception:
return None
+75
View File
@@ -0,0 +1,75 @@
# plugins/traefik/migrations/env.py
"""Alembic env.py for the Traefik plugin.
For standalone use during plugin development.
At app startup, migration_runner.py uses the core env.py with version_locations.
"""
import asyncio
import os
import sys
from pathlib import Path
from logging.config import fileConfig
from sqlalchemy import pool
from sqlalchemy.engine import Connection
from sqlalchemy.ext.asyncio import async_engine_from_config
from alembic import context
# Make steward importable
sys.path.insert(0, str(Path(__file__).parent.parent.parent.parent))
from steward.models.base import Base
import steward.models # noqa: F401 — core models
from plugins.traefik.models import TraefikMetric # noqa: F401 — plugin model
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
target_metadata = Base.metadata
def _get_url() -> str:
import yaml
cfg_path = os.environ.get("STEWARD_CONFIG", "config.yaml")
try:
with open(cfg_path) as f:
cfg = yaml.safe_load(f) or {}
url = cfg.get("database", {}).get("url", "")
except FileNotFoundError:
url = ""
return os.environ.get("STEWARD_DATABASE__URL", url)
def run_migrations_offline() -> None:
context.configure(
url=_get_url(), target_metadata=target_metadata,
literal_binds=True, dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def do_run_migrations(connection: Connection) -> None:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
async def run_async_migrations() -> None:
cfg = config.get_section(config.config_ini_section, {})
cfg["sqlalchemy.url"] = _get_url()
connectable = async_engine_from_config(cfg, prefix="sqlalchemy.", poolclass=pool.NullPool)
async with connectable.connect() as connection:
await connection.run_sync(do_run_migrations)
await connectable.dispose()
def run_migrations_online() -> None:
asyncio.run(run_async_migrations())
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
@@ -0,0 +1,37 @@
# plugins/traefik/migrations/versions/traefik_001_initial.py
"""Traefik metrics table
Revision ID: traefik_001_initial
Revises: (none — this is a branch off the core head via depends_on)
Create Date: 2026-03-17
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "traefik_001_initial"
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = "traefik"
depends_on: Union[str, Sequence[str], None] = ("0004_app_settings",)
def upgrade() -> None:
op.create_table(
"traefik_metrics",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("router_name", sa.String(255), nullable=False),
sa.Column("scraped_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("request_rate", sa.Float, nullable=False),
sa.Column("error_rate_4xx_pct", sa.Float, nullable=False),
sa.Column("error_rate_5xx_pct", sa.Float, nullable=False),
sa.Column("latency_p50_ms", sa.Float, nullable=False),
sa.Column("latency_p95_ms", sa.Float, nullable=False),
sa.Column("latency_p99_ms", sa.Float, nullable=False),
)
op.create_index("ix_traefik_metrics_router_scraped", "traefik_metrics",
["router_name", "scraped_at"])
def downgrade() -> None:
op.drop_index("ix_traefik_metrics_router_scraped", "traefik_metrics")
op.drop_table("traefik_metrics")
@@ -0,0 +1,48 @@
"""Traefik extended stats: bandwidth, certs, global stats
Revision ID: traefik_002_stats
Revises: traefik_001_initial
Create Date: 2026-03-19
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "traefik_002_stats"
down_revision: Union[str, None] = "traefik_001_initial"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"traefik_metrics",
sa.Column("response_bytes_rate", sa.Float, nullable=False, server_default="0"),
)
op.create_table(
"traefik_certs",
sa.Column("serial", sa.String(255), primary_key=True),
sa.Column("cn", sa.String(512), nullable=True),
sa.Column("sans", sa.Text, nullable=True),
sa.Column("not_after", sa.DateTime(timezone=True), nullable=False),
sa.Column("scraped_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_table(
"traefik_global_stats",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("scraped_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("open_conns_total", sa.Float, nullable=True),
sa.Column("config_reloads_total", sa.Float, nullable=True),
sa.Column("config_last_reload_success", sa.Float, nullable=True),
sa.Column("process_memory_bytes", sa.Float, nullable=True),
)
op.create_index("ix_traefik_global_stats_scraped", "traefik_global_stats", ["scraped_at"])
def downgrade() -> None:
op.drop_index("ix_traefik_global_stats_scraped", "traefik_global_stats")
op.drop_table("traefik_global_stats")
op.drop_table("traefik_certs")
op.drop_column("traefik_metrics", "response_bytes_rate")
@@ -0,0 +1,40 @@
"""Traefik access log traffic summaries
Revision ID: traefik_003_access_log
Revises: traefik_002_stats
Create Date: 2026-03-20
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "traefik_003_access_log"
down_revision: Union[str, None] = "traefik_002_stats"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"traefik_access_summaries",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("period_start", sa.DateTime(timezone=True), nullable=False),
sa.Column("period_end", sa.DateTime(timezone=True), nullable=False),
sa.Column("total_requests", sa.Integer, nullable=False, server_default="0"),
sa.Column("internal_requests", sa.Integer, nullable=False, server_default="0"),
sa.Column("external_requests", sa.Integer, nullable=False, server_default="0"),
sa.Column("total_bytes", sa.Float, nullable=False, server_default="0"),
sa.Column("top_ips_json", sa.Text, nullable=False, server_default="[]"),
sa.Column("top_countries_json", sa.Text, nullable=False, server_default="[]"),
sa.Column("top_routers_json", sa.Text, nullable=False, server_default="[]"),
)
op.create_index(
"ix_traefik_access_summaries_period_end",
"traefik_access_summaries",
["period_end"],
)
def downgrade() -> None:
op.drop_index("ix_traefik_access_summaries_period_end", "traefik_access_summaries")
op.drop_table("traefik_access_summaries")
+67
View File
@@ -0,0 +1,67 @@
# plugins/traefik/models.py
from __future__ import annotations
import uuid
from datetime import datetime, timezone
from sqlalchemy import DateTime, Float, Integer, String, Text
from sqlalchemy.orm import Mapped, mapped_column
from steward.models.base import Base
class TraefikMetric(Base):
__tablename__ = "traefik_metrics"
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4()))
router_name: Mapped[str] = mapped_column(String(255), nullable=False)
scraped_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, default=lambda: datetime.now(timezone.utc)
)
request_rate: Mapped[float] = mapped_column(Float, nullable=False)
error_rate_4xx_pct: Mapped[float] = mapped_column(Float, nullable=False)
error_rate_5xx_pct: Mapped[float] = mapped_column(Float, nullable=False)
latency_p50_ms: Mapped[float] = mapped_column(Float, nullable=False)
latency_p95_ms: Mapped[float] = mapped_column(Float, nullable=False)
latency_p99_ms: Mapped[float] = mapped_column(Float, nullable=False)
response_bytes_rate: Mapped[float] = mapped_column(Float, nullable=False, default=0.0)
class TraefikCert(Base):
"""TLS certificate expiry tracking — upserted by serial each scrape."""
__tablename__ = "traefik_certs"
serial: Mapped[str] = mapped_column(String(255), primary_key=True)
cn: Mapped[str | None] = mapped_column(String(512), nullable=True)
sans: Mapped[str | None] = mapped_column(Text, nullable=True)
not_after: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
scraped_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, default=lambda: datetime.now(timezone.utc)
)
class TraefikAccessSummary(Base):
"""Aggregated traffic-origin summary — one row per scheduler tick."""
__tablename__ = "traefik_access_summaries"
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4()))
period_start: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
period_end: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
total_requests: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
internal_requests: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
external_requests: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
total_bytes: Mapped[float] = mapped_column(Float, nullable=False, default=0.0)
top_ips_json: Mapped[str] = mapped_column(Text, nullable=False, default="[]")
top_countries_json: Mapped[str] = mapped_column(Text, nullable=False, default="[]")
top_routers_json: Mapped[str] = mapped_column(Text, nullable=False, default="[]")
class TraefikGlobalStat(Base):
"""One row per scrape — process-level and config stats."""
__tablename__ = "traefik_global_stats"
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4()))
scraped_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, default=lambda: datetime.now(timezone.utc)
)
open_conns_total: Mapped[float | None] = mapped_column(Float, nullable=True)
config_reloads_total: Mapped[float | None] = mapped_column(Float, nullable=True)
config_last_reload_success: Mapped[float | None] = mapped_column(Float, nullable=True)
process_memory_bytes: Mapped[float | None] = mapped_column(Float, nullable=True)
+21
View File
@@ -0,0 +1,21 @@
# plugins/traefik/plugin.yaml
name: traefik
version: "1.0.0"
description: "Traefik reverse proxy metrics and access log integration"
author: "Steward"
license: "MIT"
min_app_version: "0.1.0"
repository_url: "https://git.fabledsword.com/bvandeusen/Steward-plugins"
homepage: "https://git.fabledsword.com/bvandeusen/Steward-plugins/src/branch/main/traefik"
tags:
- proxy
- metrics
- access-log
config:
metrics_url: "http://localhost:8080/metrics"
scrape_interval_seconds: 60
access_log:
enabled: false
path: "/var/log/traefik/access.log"
geoip_db: "" # optional: path to MaxMind GeoLite2-Country.mmdb
+373
View File
@@ -0,0 +1,373 @@
# plugins/traefik/routes.py
from __future__ import annotations
from collections import defaultdict
from datetime import datetime, timezone
from quart import Blueprint, current_app, render_template, request
from sqlalchemy import Integer, cast, func, select
import json
from steward.auth.middleware import require_role
from steward.models.users import UserRole
from steward.core.time_range import parse_range, DEFAULT_RANGE, bucket_seconds, subsample
from .models import TraefikAccessSummary, TraefikCert, TraefikGlobalStat, TraefikMetric
traefik_bp = Blueprint("traefik", __name__, template_folder="templates")
def _sparkline(values: list[float], width: int = 80, height: int = 20) -> str:
"""Generate an inline SVG sparkline from a list of float values."""
if len(values) < 2:
return f'<svg width="{width}" height="{height}"></svg>'
mn, mx = min(values), max(values)
if mx == mn:
mx = mn + 1.0
step = width / (len(values) - 1)
pts = []
for i, v in enumerate(values):
x = i * step
y = height - (v - mn) / (mx - mn) * (height - 2) - 1
pts.append(f"{x:.1f},{y:.1f}")
poly = " ".join(pts)
return (
f'<svg width="{width}" height="{height}" viewBox="0 0 {width} {height}" '
f'style="vertical-align:middle;">'
f'<polyline points="{poly}" fill="none" stroke="#6060c0" stroke-width="1.5"/>'
f'</svg>'
)
@traefik_bp.get("/")
@require_role(UserRole.viewer)
async def index():
poll_interval = current_app.config.get("MONITORS_POLL_INTERVAL", 60)
_al_cfg = current_app.config["PLUGINS"]["traefik"].get("access_log", {})
if not isinstance(_al_cfg, dict):
_al_cfg = {}
access_log_enabled = _al_cfg.get("enabled", False)
current_range = request.args.get("range", DEFAULT_RANGE)
return await render_template(
"traefik/index.html",
poll_interval=poll_interval,
access_log_enabled=access_log_enabled,
current_range=current_range,
)
@traefik_bp.get("/rows")
@require_role(UserRole.viewer)
async def rows():
"""HTMX fragment: service cards with sparklines scoped to selected time range."""
since, range_key = parse_range(request.args.get("range"))
now = datetime.now(timezone.utc)
async with current_app.db_sessionmaker() as db:
# All known routers
result = await db.execute(
select(TraefikMetric.router_name)
.distinct()
.order_by(TraefikMetric.router_name)
)
routers = [row[0] for row in result.all()]
b_secs = bucket_seconds(since)
bucket_col = (
cast(func.extract('epoch', TraefikMetric.scraped_at), Integer) / b_secs
).label("bucket")
router_data = []
for router in routers:
# Bucketed history for sparklines — always ~80 points regardless of range
result = await db.execute(
select(
func.avg(TraefikMetric.request_rate).label("request_rate"),
func.avg(TraefikMetric.latency_p95_ms).label("latency_p95_ms"),
func.avg(TraefikMetric.error_rate_5xx_pct).label("error_rate_5xx_pct"),
func.min(TraefikMetric.scraped_at).label("scraped_at"),
bucket_col,
)
.where(TraefikMetric.router_name == router)
.where(TraefikMetric.scraped_at >= since)
.group_by(bucket_col)
.order_by(bucket_col)
)
history = result.all()
# Latest value — always the most recent raw row
result = await db.execute(
select(TraefikMetric)
.where(TraefikMetric.router_name == router)
.order_by(TraefikMetric.scraped_at.desc())
.limit(1)
)
latest = result.scalar_one_or_none()
if not latest:
continue
router_data.append({
"name": router,
"latest": latest,
"sparkline_req": _sparkline([r.request_rate for r in history]),
"sparkline_p95": _sparkline([r.latency_p95_ms for r in history]),
"sparkline_5xx": _sparkline([r.error_rate_5xx_pct for r in history]),
})
# Latest global stat (always most recent, not range-filtered)
result = await db.execute(
select(TraefikGlobalStat)
.order_by(TraefikGlobalStat.scraped_at.desc())
.limit(1)
)
global_stat = result.scalar_one_or_none()
# All certs, sorted by soonest expiry
result = await db.execute(
select(TraefikCert).order_by(TraefikCert.not_after)
)
raw_certs = result.scalars().all()
certs = []
for c in raw_certs:
days = (c.not_after - now).total_seconds() / 86400.0
certs.append({
"serial": c.serial,
"cn": c.cn or c.serial,
"sans": c.sans,
"not_after": c.not_after,
"days_remaining": days,
})
return await render_template(
"traefik/rows.html",
router_data=router_data,
global_stat=global_stat,
certs=certs,
range_key=range_key,
)
@traefik_bp.get("/widget")
@require_role(UserRole.viewer)
async def widget():
"""HTMX dashboard widget fragment."""
now = datetime.now(timezone.utc)
async with current_app.db_sessionmaker() as db:
result = await db.execute(
select(TraefikMetric.router_name).distinct().order_by(TraefikMetric.router_name)
)
routers = [row[0] for row in result.all()]
router_data = []
for router in routers:
result = await db.execute(
select(TraefikMetric)
.where(TraefikMetric.router_name == router)
.order_by(TraefikMetric.scraped_at.desc())
.limit(1)
)
latest = result.scalar_one_or_none()
if latest:
router_data.append({"name": router, "latest": latest})
router_data.sort(key=lambda r: (
-(1 if r["latest"].error_rate_5xx_pct > 0.1 else 0),
-(1 if r["latest"].error_rate_4xx_pct > 5.0 else 0),
-r["latest"].request_rate,
))
total_routers = len(router_data)
router_data = router_data[:5]
result = await db.execute(
select(TraefikCert).order_by(TraefikCert.not_after)
)
raw_certs = result.scalars().all()
expiring_certs = []
for c in raw_certs:
days = (c.not_after - now).total_seconds() / 86400.0
if days < 30:
expiring_certs.append({"cn": c.cn or c.serial, "days_remaining": days})
return await render_template(
"traefik/widget.html",
router_data=router_data,
expiring_certs=expiring_certs,
total_routers=total_routers,
)
@traefik_bp.get("/widget/access_log")
@require_role(UserRole.viewer)
async def widget_access_log():
"""HTMX dashboard widget: traffic origin summary with IP filter."""
from collections import defaultdict
ip_filter = request.args.get("ip_filter", "all")
limit = max(1, min(50, int(request.args.get("limit", 10) or 10)))
widget_id = request.args.get("wid", "0")
async with current_app.db_sessionmaker() as db:
result = await db.execute(
select(TraefikAccessSummary)
.order_by(TraefikAccessSummary.period_end.desc())
.limit(12) # last ~1 hour of 5-min windows
)
rows = result.scalars().all()
if not rows:
return await render_template("traefik/widget_access_log.html",
summary=None, widget_id=widget_id)
total = sum(r.total_requests for r in rows)
internal = sum(r.internal_requests for r in rows)
external = sum(r.external_requests for r in rows)
ip_counts: dict = defaultdict(lambda: {"count": 0, "internal": True})
for row in rows:
for entry in json.loads(row.top_ips_json):
k = entry["ip"]
ip_counts[k]["count"] += entry["count"]
ip_counts[k]["internal"] = entry.get("internal", True)
all_ips = sorted(
[{"ip": k, **v} for k, v in ip_counts.items()],
key=lambda x: x["count"], reverse=True,
)
if ip_filter == "internal":
all_ips = [e for e in all_ips if e["internal"]]
elif ip_filter == "external":
all_ips = [e for e in all_ips if not e["internal"]]
summary = {
"total": total,
"internal": internal,
"external": external,
"external_pct": (external / total * 100.0) if total else 0.0,
"top_ips": all_ips[:limit],
"ip_filter": ip_filter,
}
return await render_template("traefik/widget_access_log.html",
summary=summary, widget_id=widget_id)
@traefik_bp.get("/widget/request_chart")
@require_role(UserRole.viewer)
async def widget_request_chart():
"""HTMX dashboard widget: Chart.js request rate + error % over time."""
from datetime import timedelta
hours = max(1, min(24, int(request.args.get("hours", 6) or 6)))
widget_id = request.args.get("wid", "0")
since = datetime.now(timezone.utc) - timedelta(hours=hours)
b_secs = max(60, (hours * 3600) // 80) # ~80 buckets
bucket_col = (
cast(func.extract('epoch', TraefikMetric.scraped_at), Integer) / b_secs
).label("bucket")
async with current_app.db_sessionmaker() as db:
result = await db.execute(
select(
func.avg(TraefikMetric.request_rate).label("req_rate"),
func.avg(TraefikMetric.error_rate_5xx_pct).label("err_pct"),
func.min(TraefikMetric.scraped_at).label("scraped_at"),
bucket_col,
)
.where(TraefikMetric.scraped_at >= since)
.group_by(bucket_col)
.order_by(bucket_col)
)
history = result.all()
labels = list(range(len(history)))
req_rates = [round(r.req_rate or 0, 3) for r in history]
err_pcts = [round(r.err_pct or 0, 2) for r in history]
return await render_template(
"traefik/widget_request_chart.html",
labels_json=json.dumps(labels),
req_rate_json=json.dumps(req_rates),
error_rate_json=json.dumps(err_pcts),
widget_id=widget_id,
hours=hours,
)
@traefik_bp.get("/traffic")
@require_role(UserRole.viewer)
async def traffic():
"""HTMX fragment: traffic origin summary scoped to selected time range."""
since, range_key = parse_range(request.args.get("range"))
async with current_app.db_sessionmaker() as db:
result = await db.execute(
select(TraefikAccessSummary)
.where(TraefikAccessSummary.period_end >= since)
.order_by(TraefikAccessSummary.period_end.asc())
# No LIMIT — all summaries in range needed for accurate totals;
# sparkline subsamples to 80 points afterwards
)
rows = result.scalars().all()
if not rows:
return await render_template(
"traefik/traffic.html", summary=None, history=[], range_key=range_key
)
total = sum(r.total_requests for r in rows)
internal = sum(r.internal_requests for r in rows)
external = sum(r.external_requests for r in rows)
total_bytes = sum(r.total_bytes for r in rows)
ip_counts: dict = defaultdict(lambda: {"count": 0, "internal": True, "country": None})
country_counts: dict[str, int] = defaultdict(int)
router_counts: dict[str, int] = defaultdict(int)
for row in rows:
for entry in json.loads(row.top_ips_json):
k = entry["ip"]
ip_counts[k]["count"] += entry["count"]
ip_counts[k]["internal"] = entry["internal"]
ip_counts[k]["country"] = entry.get("country")
for entry in json.loads(row.top_countries_json):
country_counts[entry["country"]] += entry["count"]
for entry in json.loads(row.top_routers_json):
router_counts[entry["router"]] += entry["count"]
top_ips = sorted(
[{"ip": k, **v} for k, v in ip_counts.items()],
key=lambda x: x["count"], reverse=True
)[:10]
top_countries = sorted(
[{"country": k, "count": v} for k, v in country_counts.items()],
key=lambda x: x["count"], reverse=True
)[:10]
top_routers = sorted(
[{"router": k, "count": v} for k, v in router_counts.items()],
key=lambda x: x["count"], reverse=True
)[:10]
history = [
{
"period_end": r.period_end,
"external_pct": (r.external_requests / r.total_requests * 100.0)
if r.total_requests else 0.0,
}
for r in rows
]
summary = {
"total": total,
"internal": internal,
"external": external,
"total_bytes": total_bytes,
"external_pct": (external / total * 100.0) if total else 0.0,
"top_ips": top_ips,
"top_countries": top_countries,
"top_routers": top_routers,
"sparkline_external": _sparkline(
[h["external_pct"] for h in subsample(history, 80)]
),
}
return await render_template(
"traefik/traffic.html", summary=summary, history=history, range_key=range_key
)
+203
View File
@@ -0,0 +1,203 @@
# plugins/traefik/scheduler.py
from __future__ import annotations
import logging
import time
from typing import TYPE_CHECKING
from datetime import datetime
from steward.core.scheduler import ScheduledTask
if TYPE_CHECKING:
from quart import Quart
logger = logging.getLogger(__name__)
# Module-level state: previous scrape data for delta computation
_prev_metrics: dict = {}
_prev_time: float = 0.0
# Access log state
_geoip_db = None
_access_log_last_tick: "datetime | None" = None
def make_scrape_task(app: "Quart") -> ScheduledTask:
"""Return a ScheduledTask that scrapes Traefik metrics on the configured interval."""
interval = app.config["PLUGINS"]["traefik"]["scrape_interval_seconds"]
async def scrape() -> None:
await _do_scrape(app)
return ScheduledTask(
name="traefik_scrape",
coro_factory=scrape,
interval_seconds=int(interval),
run_on_startup=True,
)
async def _do_scrape(app: "Quart") -> None:
"""Fetch Traefik metrics, write to DB, and emit plugin_metrics."""
global _prev_metrics, _prev_time
from datetime import datetime, timezone
from .models import TraefikCert, TraefikGlobalStat, TraefikMetric
from .scraper import (
compute_global_stats,
compute_router_metrics,
extract_certs,
fetch_metrics,
)
from steward.core.alerts import record_metric
metrics_url: str = app.config["PLUGINS"]["traefik"]["metrics_url"]
try:
current = await fetch_metrics(metrics_url)
except Exception:
logger.exception("Traefik scrape failed (url=%s)", metrics_url)
return
now = time.monotonic()
elapsed = now - _prev_time if _prev_time else 60.0
router_metrics = compute_router_metrics(current, _prev_metrics or None, elapsed)
global_stats = compute_global_stats(current, _prev_metrics or None)
cert_list = extract_certs(current)
_prev_metrics = current
_prev_time = now
scraped_at = datetime.now(timezone.utc)
async with app.db_sessionmaker() as session:
async with session.begin():
# ── Per-service metrics ───────────────────────────────────────────
for router_name, metrics in router_metrics.items():
row = TraefikMetric(
router_name=router_name,
scraped_at=scraped_at,
request_rate=metrics["request_rate"],
error_rate_4xx_pct=metrics["error_rate_4xx_pct"],
error_rate_5xx_pct=metrics["error_rate_5xx_pct"],
latency_p50_ms=metrics["latency_p50_ms"],
latency_p95_ms=metrics["latency_p95_ms"],
latency_p99_ms=metrics["latency_p99_ms"],
response_bytes_rate=metrics["response_bytes_rate"],
)
session.add(row)
for metric_name, value in [
("request_rate", metrics["request_rate"]),
("error_rate_4xx_pct", metrics["error_rate_4xx_pct"]),
("error_rate_5xx_pct", metrics["error_rate_5xx_pct"]),
("latency_p50_ms", metrics["latency_p50_ms"]),
("latency_p95_ms", metrics["latency_p95_ms"]),
("latency_p99_ms", metrics["latency_p99_ms"]),
("response_bytes_rate", metrics["response_bytes_rate"]),
]:
await record_metric(
session=session,
source_module="traefik",
resource_name=router_name,
metric_name=metric_name,
value=value,
)
# ── Global stats ──────────────────────────────────────────────────
session.add(TraefikGlobalStat(scraped_at=scraped_at, **global_stats))
# ── TLS certs (upsert by serial) ──────────────────────────────────
for cert in cert_list:
existing = await session.get(TraefikCert, cert["serial"])
if existing:
existing.cn = cert["cn"]
existing.sans = cert["sans"]
existing.not_after = cert["not_after"]
existing.scraped_at = scraped_at
else:
session.add(TraefikCert(
serial=cert["serial"],
cn=cert["cn"],
sans=cert["sans"],
not_after=cert["not_after"],
scraped_at=scraped_at,
))
# Emit cert_expiry_days for alert rules
days_remaining = (cert["not_after"] - scraped_at).total_seconds() / 86400.0
await record_metric(
session=session,
source_module="traefik",
resource_name=cert["cn"] or cert["serial"],
metric_name="cert_expiry_days",
value=max(0.0, days_remaining),
)
logger.debug(
"Traefik scrape: %d router(s), %d cert(s)",
len(router_metrics),
len(cert_list),
)
def make_access_log_task(app: "Quart") -> ScheduledTask:
"""Return a ScheduledTask that processes new Traefik access log lines."""
global _geoip_db
cfg = app.config["PLUGINS"]["traefik"].get("access_log", {})
if not isinstance(cfg, dict):
cfg = {}
log_path: str = cfg.get("path", "/var/log/traefik/access.log")
geoip_path: str = cfg.get("geoip_db", "")
from .access_log import open_geoip
_geoip_db = open_geoip(geoip_path or None)
if _geoip_db:
logger.info("Traefik access log: GeoIP database loaded from %s", geoip_path)
else:
logger.info("Traefik access log: GeoIP not configured, country lookup disabled")
interval = app.config["PLUGINS"]["traefik"]["scrape_interval_seconds"]
async def process() -> None:
await _do_access_log(app, log_path)
return ScheduledTask(
name="traefik_access_log",
coro_factory=process,
interval_seconds=int(interval),
run_on_startup=True,
)
async def _do_access_log(app: "Quart", log_path: str) -> None:
"""Read new access log lines, aggregate, and persist a summary."""
global _access_log_last_tick
from datetime import timezone
from .access_log import read_new_lines, aggregate
from .models import TraefikAccessSummary
now = datetime.now(timezone.utc)
period_start = _access_log_last_tick or now
_access_log_last_tick = now
entries = read_new_lines(log_path)
summary = aggregate(entries, geoip_db=_geoip_db)
if summary is None:
return
async with app.db_sessionmaker() as session:
async with session.begin():
session.add(TraefikAccessSummary(
period_start=period_start,
period_end=now,
**summary,
))
logger.debug(
"Traefik access log: %d requests (%d internal, %d external)",
summary["total_requests"],
summary["internal_requests"],
summary["external_requests"],
)
+276
View File
@@ -0,0 +1,276 @@
# plugins/traefik/scraper.py
"""Prometheus text-format scraper and per-service metrics calculator for Traefik.
Traefik exposes Prometheus metrics at /metrics. This module:
1. Fetches the endpoint via httpx
2. Parses the Prometheus text format (counters + histograms)
3. Computes per-service request rate (delta/elapsed), error rates (% 4xx, % 5xx),
latency percentiles (p50, p95, p99) via linear histogram interpolation,
and bandwidth (bytes/sec from response bytes counter delta)
4. Extracts TLS certificate expiry info
5. Computes process-level and config health stats
"""
from __future__ import annotations
import math
import re
from collections import defaultdict
from datetime import datetime, timezone
import httpx
# ──────────────────────────────────────────────────────────────────────────────
# Prometheus text format parser
# ──────────────────────────────────────────────────────────────────────────────
# Parsed sample: {metric_name: {frozenset(label_items): float}}
ParsedMetrics = dict[str, dict[frozenset, float]]
_SAMPLE_RE = re.compile(
r'^([a-zA-Z_:][a-zA-Z0-9_:]*)(?:\{([^}]*)\})?\s+'
r'([+-]?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?|[+-]?Inf|NaN)'
)
_LABEL_RE = re.compile(r'([a-zA-Z_][a-zA-Z0-9_]*)="([^"\\]*(?:\\.[^"\\]*)*)"')
def parse_prometheus(text: str) -> ParsedMetrics:
"""Parse Prometheus text exposition format into nested dicts.
Returns: {metric_name: {frozenset({(label, value), ...}): float}}
"""
metrics: ParsedMetrics = {}
for line in text.splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
m = _SAMPLE_RE.match(line)
if not m:
continue
name, labels_str, value_str = m.groups()
try:
value = float(value_str)
except ValueError:
continue
labels = frozenset(
(lm.group(1), lm.group(2))
for lm in _LABEL_RE.finditer(labels_str or "")
)
metrics.setdefault(name, {})[labels] = value
return metrics
# ──────────────────────────────────────────────────────────────────────────────
# Per-router metric computation
# ──────────────────────────────────────────────────────────────────────────────
def compute_router_metrics(
current: ParsedMetrics,
previous: ParsedMetrics | None,
elapsed_seconds: float,
) -> dict[str, dict[str, float]]:
"""Compute per-service derived metrics from two consecutive scrapes.
Uses traefik_service_* metrics (available by default).
Returns:
{service_name: {
"request_rate": float, # requests/sec
"error_rate_4xx_pct": float, # % 4xx of total
"error_rate_5xx_pct": float, # % 5xx of total
"latency_p50_ms": float,
"latency_p95_ms": float,
"latency_p99_ms": float,
"response_bytes_rate": float, # bytes/sec
}}
"""
prev = previous or {}
elapsed = max(elapsed_seconds, 1.0)
# ── Request counters ──────────────────────────────────────────────────────
req_samples = current.get("traefik_service_requests_total", {})
prev_req = prev.get("traefik_service_requests_total", {})
service_total: dict[str, float] = defaultdict(float)
service_4xx: dict[str, float] = defaultdict(float)
service_5xx: dict[str, float] = defaultdict(float)
for labels, value in req_samples.items():
label_dict = dict(labels)
service = label_dict.get("service", "")
if not service:
continue
code = label_dict.get("code", "")
prev_value = prev_req.get(labels, value) # no delta on first scrape
delta = max(0.0, value - prev_value)
service_total[service] += delta
if code.startswith("4"):
service_4xx[service] += delta
elif code.startswith("5"):
service_5xx[service] += delta
# ── Response bytes counters ───────────────────────────────────────────────
bytes_samples = current.get("traefik_service_responses_bytes_total", {})
prev_bytes = prev.get("traefik_service_responses_bytes_total", {})
service_bytes: dict[str, float] = defaultdict(float)
for labels, value in bytes_samples.items():
service = dict(labels).get("service", "")
if not service:
continue
prev_value = prev_bytes.get(labels, value)
service_bytes[service] += max(0.0, value - prev_value)
# ── Histogram buckets ─────────────────────────────────────────────────────
hist_buckets = current.get("traefik_service_request_duration_seconds_bucket", {})
# Collect all known services
all_services: set[str] = set(service_total.keys()) | set(service_bytes.keys())
for labels in hist_buckets:
service = dict(labels).get("service", "")
if service:
all_services.add(service)
result: dict[str, dict[str, float]] = {}
for service in all_services:
total = service_total.get(service, 0.0)
result[service] = {
"request_rate": total / elapsed,
"error_rate_4xx_pct": (service_4xx.get(service, 0.0) / total * 100.0)
if total > 0 else 0.0,
"error_rate_5xx_pct": (service_5xx.get(service, 0.0) / total * 100.0)
if total > 0 else 0.0,
"latency_p50_ms": _percentile_ms(hist_buckets, service, 0.50),
"latency_p95_ms": _percentile_ms(hist_buckets, service, 0.95),
"latency_p99_ms": _percentile_ms(hist_buckets, service, 0.99),
"response_bytes_rate": service_bytes.get(service, 0.0) / elapsed,
}
return result
def compute_global_stats(
current: ParsedMetrics,
previous: ParsedMetrics | None,
) -> dict[str, float | None]:
"""Extract process-level and config health metrics from a scrape.
Returns:
{
"open_conns_total": float | None,
"config_reloads_total": float | None,
"config_last_reload_success": float | None, # 1.0 = success, 0.0 = failure
"process_memory_bytes": float | None,
}
"""
stats: dict[str, float | None] = {
"open_conns_total": None,
"config_reloads_total": None,
"config_last_reload_success": None,
"process_memory_bytes": None,
}
# Open connections — sum across all entrypoints/protocols
open_conns = current.get("traefik_open_connections", {})
if open_conns:
stats["open_conns_total"] = sum(open_conns.values())
# Config reloads — sum across result labels (success + failure)
reloads = current.get("traefik_config_reloads_total", {})
if reloads:
stats["config_reloads_total"] = sum(reloads.values())
# Last reload success gauge
last_success = current.get("traefik_config_last_reload_success", {})
if last_success:
# Usually a single sample with no labels
stats["config_last_reload_success"] = next(iter(last_success.values()))
# Process RSS memory
mem = current.get("process_resident_memory_bytes", {})
if mem:
stats["process_memory_bytes"] = next(iter(mem.values()))
return stats
def extract_certs(current: ParsedMetrics) -> list[dict]:
"""Extract TLS certificate expiry info from traefik_tls_certs_not_after.
Returns a list of dicts:
[{"serial": str, "cn": str, "sans": str, "not_after": datetime}, ...]
"""
certs = []
samples = current.get("traefik_tls_certs_not_after", {})
for labels, ts_value in samples.items():
label_dict = dict(labels)
serial = label_dict.get("serial", "")
if not serial:
continue
try:
not_after = datetime.fromtimestamp(ts_value, tz=timezone.utc)
except (ValueError, OSError, OverflowError):
continue
certs.append({
"serial": serial,
"cn": label_dict.get("cn") or None,
"sans": label_dict.get("sans") or None,
"not_after": not_after,
})
return certs
def _percentile_ms(
buckets: dict[frozenset, float],
service: str,
pct: float,
) -> float:
"""Linear interpolation of a Prometheus histogram percentile, returned in ms."""
router_buckets: list[tuple[float, float]] = []
for labels, count in buckets.items():
label_dict = dict(labels)
if label_dict.get("service") != service:
continue
le_str = label_dict.get("le", "")
try:
le = float("inf") if le_str == "+Inf" else float(le_str)
router_buckets.append((le, count))
except ValueError:
continue
if not router_buckets:
return 0.0
router_buckets.sort(key=lambda t: t[0])
total = router_buckets[-1][1] # +Inf bucket count == total requests
if total == 0.0:
return 0.0
target = total * pct
for i, (le, count) in enumerate(router_buckets):
if count < target:
continue
if i == 0:
ratio = target / count if count > 0 else 0.0
return le * ratio * 1000.0
prev_le, prev_count = router_buckets[i - 1]
if count == prev_count:
return prev_le * 1000.0
if math.isinf(le):
return prev_le * 1000.0
fraction = (target - prev_count) / (count - prev_count)
return (prev_le + fraction * (le - prev_le)) * 1000.0
return 0.0
# ──────────────────────────────────────────────────────────────────────────────
# HTTP fetch
# ──────────────────────────────────────────────────────────────────────────────
async def fetch_metrics(metrics_url: str) -> ParsedMetrics:
"""Fetch and parse Prometheus metrics from the given URL."""
async with httpx.AsyncClient(timeout=10.0) as client:
response = await client.get(metrics_url)
response.raise_for_status()
return parse_prometheus(response.text)
@@ -0,0 +1,31 @@
{# plugins/traefik/templates/traefik/index.html #}
{% extends "base.html" %}
{% block title %}Traefik — Steward{% endblock %}
{% block content %}
<div style="display:flex;align-items:center;justify-content:space-between;flex-wrap:wrap;gap:0.75rem;margin-bottom:1.5rem;">
<div style="display:flex;align-items:baseline;gap:1rem;">
<h1 class="page-title" style="margin-bottom:0;">Traefik Services</h1>
<span style="color:var(--text-muted);font-size:0.85rem;">polling every {{ poll_interval }}s</span>
</div>
{% include "_time_range.html" %}
</div>
<div id="traefik-rows"
hx-get="/plugins/traefik/rows"
hx-trigger="load, every {{ poll_interval }}s, rangeChange from:body"
hx-include="#time-range"
hx-swap="innerHTML">
</div>
{% if access_log_enabled %}
<div style="margin-top:1.5rem;">
<h2 class="section-title" style="margin-bottom:1rem;">Traffic Origin</h2>
<div id="traefik-traffic"
hx-get="/plugins/traefik/traffic"
hx-trigger="load, every {{ poll_interval }}s, rangeChange from:body"
hx-include="#time-range"
hx-swap="innerHTML">
</div>
</div>
{% endif %}
{% endblock %}
+127
View File
@@ -0,0 +1,127 @@
{# plugins/traefik/templates/traefik/rows.html #}
{# ── Global stats bar ──────────────────────────────────────────────────────── #}
{% if global_stat %}
<div style="display:flex;flex-wrap:wrap;gap:1.5rem;margin-bottom:1.25rem;padding:0.75rem 1rem;background:var(--card-bg);border:1px solid var(--border);border-radius:6px;font-size:0.85rem;font-variant-numeric:tabular-nums;">
{% if global_stat.open_conns_total is not none %}
<span>
<span style="color:var(--text-muted);">Open conns</span>
<span style="margin-left:0.4rem;font-weight:600;">{{ global_stat.open_conns_total | int }}</span>
</span>
{% endif %}
{% if global_stat.config_last_reload_success is not none %}
<span>
<span style="color:var(--text-muted);">Config</span>
{% if global_stat.config_last_reload_success == 1.0 %}
<span style="margin-left:0.4rem;color:var(--green);">&#10003; OK</span>
{% else %}
<span style="margin-left:0.4rem;color:var(--red);">&#10007; failed</span>
{% endif %}
{% if global_stat.config_reloads_total is not none %}
<span style="color:var(--text-dim);margin-left:0.3rem;">({{ global_stat.config_reloads_total | int }} reloads)</span>
{% endif %}
</span>
{% endif %}
{% if global_stat.process_memory_bytes is not none %}
<span>
<span style="color:var(--text-muted);">Memory</span>
{% set mb = (global_stat.process_memory_bytes / 1048576) %}
<span style="margin-left:0.4rem;">
{% if mb >= 1024 %}{{ "%.1f"|format(mb / 1024) }} GB{% else %}{{ "%.0f"|format(mb) }} MB{% endif %}
</span>
</span>
{% endif %}
<span style="margin-left:auto;color:var(--text-dim);font-size:0.78rem;">
{{ global_stat.scraped_at.strftime("%H:%M:%S") }} UTC
</span>
</div>
{% endif %}
{# ── Per-service cards + cert card in shared columns layout ───────────────── #}
{% if router_data or certs %}
<div style="column-width:340px;column-count:3;column-gap:1rem;">
{% for r in router_data %}
<div class="card" style="break-inside:avoid;margin-bottom:1rem;">
<h3 class="section-title" style="margin-bottom:0.75rem;word-break:break-all;">{{ r.name }}</h3>
<table class="table" style="font-size:0.85rem;">
<tr>
<td style="color:var(--text-muted);padding:0.2rem 0;">Req/s</td>
<td style="padding:0.2rem 0.5rem;">{{ "%.2f"|format(r.latest.request_rate) }}</td>
<td style="padding:0.2rem 0;">{{ r.sparkline_req | safe }}</td>
</tr>
<tr>
<td style="color:var(--text-muted);padding:0.2rem 0;">Bandwidth</td>
<td style="padding:0.2rem 0.5rem;">
{% set bps = r.latest.response_bytes_rate %}
{% if bps >= 1048576 %}{{ "%.1f"|format(bps / 1048576) }} MB/s
{% elif bps >= 1024 %}{{ "%.1f"|format(bps / 1024) }} KB/s
{% else %}{{ "%.0f"|format(bps) }} B/s{% endif %}
</td>
<td style="padding:0.2rem 0;"></td>
</tr>
<tr>
<td style="color:var(--text-muted);padding:0.2rem 0;">p95 ms</td>
<td style="padding:0.2rem 0.5rem;">{{ "%.1f"|format(r.latest.latency_p95_ms) }}</td>
<td style="padding:0.2rem 0;">{{ r.sparkline_p95 | safe }}</td>
</tr>
<tr>
<td style="color:var(--text-muted);padding:0.2rem 0;">p99 ms</td>
<td style="padding:0.2rem 0.5rem;">{{ "%.1f"|format(r.latest.latency_p99_ms) }}</td>
<td style="padding:0.2rem 0;"></td>
</tr>
<tr>
<td style="color:var(--text-muted);padding:0.2rem 0;">4xx %</td>
<td style="color:{% if r.latest.error_rate_4xx_pct > 1 %}var(--yellow){% else %}var(--green){% endif %};padding:0.2rem 0.5rem;">
{{ "%.1f"|format(r.latest.error_rate_4xx_pct) }}%
</td>
<td style="padding:0.2rem 0;"></td>
</tr>
<tr>
<td style="color:var(--text-muted);padding:0.2rem 0;">5xx %</td>
<td style="color:{% if r.latest.error_rate_5xx_pct > 0.1 %}var(--red){% else %}var(--green){% endif %};padding:0.2rem 0.5rem;">
{{ "%.1f"|format(r.latest.error_rate_5xx_pct) }}%
</td>
<td style="padding:0.2rem 0;">{{ r.sparkline_5xx | safe }}</td>
</tr>
</table>
<div style="color:var(--text-dim);font-size:0.75rem;margin-top:0.5rem;">
Last scraped {{ r.latest.scraped_at.strftime("%H:%M:%S") }} UTC
</div>
</div>
{% endfor %}
{% if certs %}
<div class="card" style="break-inside:avoid;margin-bottom:1rem;">
<div class="section-title" style="margin-bottom:0.75rem;">TLS Certificates</div>
<table class="table" style="font-size:0.83rem;width:100%;">
{% for cert in certs %}
<tr>
<td style="padding:0.2rem 0 0.2rem 0;word-break:break-all;">{{ cert.cn }}</td>
<td style="text-align:right;padding:0.2rem 0;white-space:nowrap;font-weight:600;
{% if cert.days_remaining < 7 %}color:var(--red);
{% elif cert.days_remaining < 30 %}color:var(--yellow);
{% else %}color:var(--green);{% endif %}">
{{ cert.days_remaining | int }}d
</td>
</tr>
<tr>
<td colspan="2" style="padding:0 0 0.4rem 0;color:var(--text-dim);font-size:0.75rem;">
{{ cert.not_after.strftime("%Y-%m-%d") }}
</td>
</tr>
{% endfor %}
</table>
</div>
{% endif %}
</div>
{% else %}
<div class="card">
<p style="color:#606080;">No Traefik metrics yet. Configure <code>metrics_url</code> in <code>config.yaml</code> under <code>plugins.traefik</code>.</p>
</div>
{% endif %}
@@ -0,0 +1,101 @@
{# plugins/traefik/templates/traefik/traffic.html #}
{% if summary %}
{# ── Overview bar ─────────────────────────────────────────────────────────── #}
<div style="display:flex;flex-wrap:wrap;gap:1.5rem;margin-bottom:1.25rem;padding:0.75rem 1rem;background:var(--card-bg);border:1px solid var(--border);border-radius:6px;font-size:0.85rem;font-variant-numeric:tabular-nums;align-items:center;">
<span>
<span style="color:var(--text-muted);">Total</span>
<span style="margin-left:0.4rem;font-weight:600;">{{ summary.total }}</span>
</span>
<span>
<span style="color:var(--text-muted);">Internal</span>
<span style="margin-left:0.4rem;color:var(--green);font-weight:600;">{{ summary.internal }}</span>
</span>
<span>
<span style="color:var(--text-muted);">External</span>
<span style="margin-left:0.4rem;color:{% if summary.external_pct > 50 %}var(--yellow){% else %}var(--green){% endif %};font-weight:600;">
{{ summary.external }} ({{ "%.1f"|format(summary.external_pct) }}%)
</span>
</span>
<span>
<span style="color:var(--text-muted);">Bytes</span>
<span style="margin-left:0.4rem;">
{% set mb = summary.total_bytes / 1048576 %}
{% if mb >= 1024 %}{{ "%.1f"|format(mb / 1024) }} GB{% else %}{{ "%.1f"|format(mb) }} MB{% endif %}
</span>
</span>
{# External % sparkline #}
<span style="margin-left:auto;">
<span style="color:var(--text-muted);font-size:0.78rem;margin-right:0.4rem;">ext % over time</span>
{{ summary.sparkline_external | safe }}
</span>
</div>
{# ── Internal/external fill bar ───────────────────────────────────────────── #}
{% set ext_pct = summary.external_pct %}
<div style="height:6px;border-radius:3px;background:var(--border);margin-bottom:1.25rem;overflow:hidden;">
<div style="height:100%;width:{{ ext_pct | round(1) }}%;background:var(--yellow);border-radius:3px;transition:width 0.4s;"></div>
</div>
{# ── Detail cards in columns ──────────────────────────────────────────────── #}
<div style="column-width:300px;column-count:3;column-gap:1rem;">
{# Top source IPs #}
<div class="card" style="break-inside:avoid;margin-bottom:1rem;">
<div class="section-title" style="margin-bottom:0.75rem;">Top Source IPs</div>
<table class="table" style="font-size:0.82rem;width:100%;">
{% for entry in summary.top_ips %}
<tr>
<td style="padding:0.2rem 0.5rem 0.2rem 0;font-family:monospace;">{{ entry.ip }}</td>
{% if entry.country %}
<td style="padding:0.2rem 0.3rem;color:var(--text-muted);">{{ entry.country }}</td>
{% endif %}
<td style="text-align:right;padding:0.2rem 0;">
<span style="color:{% if entry.internal %}var(--text-muted){% else %}var(--yellow){% endif %};">
{{ entry.count }}
</span>
</td>
</tr>
{% endfor %}
</table>
</div>
{# Top routers #}
<div class="card" style="break-inside:avoid;margin-bottom:1rem;">
<div class="section-title" style="margin-bottom:0.75rem;">Top Services Hit</div>
<table class="table" style="font-size:0.82rem;width:100%;">
{% for entry in summary.top_routers %}
<tr>
<td style="padding:0.2rem 0.5rem 0.2rem 0;word-break:break-all;">{{ entry.router }}</td>
<td style="text-align:right;padding:0.2rem 0;">{{ entry.count }}</td>
</tr>
{% endfor %}
</table>
</div>
{# Top countries (only shown if GeoIP is active) #}
{% if summary.top_countries %}
<div class="card" style="break-inside:avoid;margin-bottom:1rem;">
<div class="section-title" style="margin-bottom:0.75rem;">External Traffic by Country</div>
<table class="table" style="font-size:0.82rem;width:100%;">
{% for entry in summary.top_countries %}
<tr>
<td style="padding:0.2rem 0.5rem 0.2rem 0;">{{ entry.country }}</td>
<td style="text-align:right;padding:0.2rem 0;">{{ entry.count }}</td>
</tr>
{% endfor %}
</table>
</div>
{% endif %}
</div>
{% else %}
<div class="card" style="color:var(--text-muted);font-size:0.85rem;">
No access log data yet.
{% if not access_log_enabled %}
Enable <code>access_log.enabled: true</code> in your plugin config and bind-mount the Traefik log directory.
{% endif %}
</div>
{% endif %}
@@ -0,0 +1,56 @@
{# plugins/traefik/templates/traefik/widget.html #}
{% if expiring_certs %}
<div style="margin-bottom:0.6rem;">
{% for cert in expiring_certs %}
<div style="display:flex;justify-content:space-between;font-size:0.8rem;padding:0.15rem 0;
{% if cert.days_remaining < 7 %}color:var(--red);{% else %}color:var(--yellow);{% endif %}">
<span>&#9888; {{ cert.cn }}</span>
<span>{{ cert.days_remaining | int }}d</span>
</div>
{% endfor %}
</div>
{% endif %}
{% if router_data %}
{% for r in router_data %}
<div class="ping-row">
<div class="ping-meta">
<span class="ping-name" style="font-size:0.8rem;word-break:break-all;">{{ r.name }}</span>
</div>
<div style="flex:1;display:flex;gap:1.5rem;font-size:0.82rem;font-variant-numeric:tabular-nums;">
<span title="Requests/s">
<span style="color:var(--text-muted);">req/s</span>
<span style="margin-left:0.3rem;">{{ "%.2f"|format(r.latest.request_rate) }}</span>
</span>
<span title="Bandwidth">
<span style="color:var(--text-muted);">bw</span>
<span style="margin-left:0.3rem;">
{% set bps = r.latest.response_bytes_rate %}
{% if bps >= 1048576 %}{{ "%.1f"|format(bps / 1048576) }}MB/s
{% elif bps >= 1024 %}{{ "%.0f"|format(bps / 1024) }}KB/s
{% else %}{{ "%.0f"|format(bps) }}B/s{% endif %}
</span>
</span>
<span title="p95 latency">
<span style="color:var(--text-muted);">p95</span>
<span style="margin-left:0.3rem;
{% if r.latest.latency_p95_ms > 500 %}color:var(--red){% elif r.latest.latency_p95_ms > 200 %}color:var(--yellow){% else %}color:var(--green){% endif %};">
{{ "%.0f"|format(r.latest.latency_p95_ms) }}ms
</span>
</span>
{% if r.latest.error_rate_5xx_pct > 0 %}
<span title="5xx error rate">
<span style="color:var(--red);">{{ "%.1f"|format(r.latest.error_rate_5xx_pct) }}% 5xx</span>
</span>
{% endif %}
</div>
</div>
{% endfor %}
{% if total_routers > 5 %}
<div style="color:var(--text-dim);font-size:0.75rem;padding:0.25rem 0;">
+{{ total_routers - 5 }} more — <a href="/plugins/traefik/">view all →</a>
</div>
{% endif %}
{% else %}
<p class="empty" style="padding:0.5rem 0;font-size:0.85rem;">No Traefik data yet.</p>
{% endif %}
@@ -0,0 +1,40 @@
{# traefik/widget_access_log.html — dashboard widget: traffic origin summary #}
{% if not summary %}
<div style="color:var(--text-muted);font-size:0.85rem;padding:0.5rem 0;">No access log data yet.</div>
{% else %}
<div style="display:flex;gap:1rem;margin-bottom:0.75rem;flex-wrap:wrap;">
<div>
<span style="font-size:1.4rem;font-weight:700;line-height:1;">{{ summary.total | int }}</span>
<span style="font-size:0.78rem;color:var(--text-muted);margin-left:0.25rem;">total req</span>
</div>
<div>
<span style="font-size:1.4rem;font-weight:700;line-height:1;color:var(--orange);">
{{ "%.1f" | format(summary.external_pct) }}%
</span>
<span style="font-size:0.78rem;color:var(--text-muted);margin-left:0.25rem;">external</span>
</div>
</div>
{% if summary.top_ips %}
<div style="font-size:0.72rem;color:var(--text-muted);text-transform:uppercase;letter-spacing:0.05em;margin-bottom:0.4rem;">
Top IPs
{% if summary.ip_filter != "all" %}
<span style="color:var(--accent);margin-left:0.4rem;">{{ summary.ip_filter }}</span>
{% endif %}
</div>
<div style="display:grid;gap:2px;">
{% for ip in summary.top_ips %}
<div style="display:flex;align-items:center;gap:0.5rem;font-size:0.82rem;padding:0.2rem 0;">
<span class="dot {% if ip.internal %}dot-dim{% else %}dot-warn{% endif %}"></span>
<span style="flex:1;font-family:ui-monospace,monospace;font-size:0.78rem;
color:{% if ip.internal %}var(--text-muted){% else %}var(--text){% endif %};">
{{ ip.ip }}
</span>
<span style="color:var(--text-muted);font-size:0.78rem;">{{ ip.count }}</span>
</div>
{% endfor %}
</div>
{% endif %}
{% endif %}
@@ -0,0 +1,75 @@
{# traefik/widget_request_chart.html — Chart.js request rate + error % over time #}
{% if not labels_json or labels_json == '[]' %}
<div style="color:var(--text-muted);font-size:0.85rem;padding:0.5rem 0;">No data for last {{ hours }}h.</div>
{% else %}
<div style="position:relative;height:160px;">
<canvas id="chart-traefik-req-{{ widget_id }}"></canvas>
</div>
<script>
(function() {
var ctx = document.getElementById("chart-traefik-req-{{ widget_id }}");
if (!ctx || !window.Chart) return;
var existing = Chart.getChart(ctx);
if (existing) existing.destroy();
new Chart(ctx.getContext("2d"), {
type: "line",
data: {
labels: {{ labels_json | safe }},
datasets: [
{
label: "Req/s",
data: {{ req_rate_json | safe }},
borderColor: "#7c5ef4",
backgroundColor: "rgba(124,94,244,0.08)",
fill: true,
tension: 0.3,
pointRadius: 0,
borderWidth: 1.5,
yAxisID: "y",
},
{
label: "5xx %",
data: {{ error_rate_json | safe }},
borderColor: "#e83848",
fill: false,
tension: 0.3,
pointRadius: 0,
borderWidth: 1.5,
yAxisID: "y2",
},
]
},
options: {
responsive: true,
maintainAspectRatio: false,
animation: false,
plugins: {
legend: {
labels: { color: "#6858a0", boxWidth: 10, font: { size: 10 } }
}
},
scales: {
x: { display: false },
y: {
min: 0,
grid: { color: "rgba(30,30,62,0.8)" },
ticks: { color: "#6858a0", font: { size: 10 } },
title: { display: true, text: "req/s", color: "#6858a0", font: { size: 9 } },
},
y2: {
position: "right",
min: 0,
max: 100,
grid: { drawOnChartArea: false },
ticks: { color: "#e83848", font: { size: 10 } },
title: { display: true, text: "5xx%", color: "#e83848", font: { size: 9 } },
},
}
}
});
})();
</script>
<div style="font-size:0.72rem;color:var(--text-muted);margin-top:0.35rem;text-align:right;">
last {{ hours }}h
</div>
{% endif %}
+28
View File
@@ -0,0 +1,28 @@
# plugins/unifi/__init__.py
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from quart import Quart
_app: "Quart | None" = None
def setup(app: "Quart") -> None:
global _app
_app = app
from .models import ( # noqa: F401
UnifiWanStat, UnifiClientSnapshot, UnifiWlanStat, UnifiDpiSnapshot,
UnifiDevice, UnifiKnownClient, UnifiEvent, UnifiAlarm,
UnifiSpeedtestResult, UnifiNetwork, UnifiPortForward, UnifiFwRule,
)
def get_scheduled_tasks() -> list:
from .scheduler import make_poll_task
return [make_poll_task(_app)]
def get_blueprint():
from .routes import unifi_bp
return unifi_bp
+170
View File
@@ -0,0 +1,170 @@
# plugins/unifi/client.py
"""Async UniFi Network controller API client.
Supports UDM/UDM-Pro (firmware 2+) using the /proxy/network/ path prefix
and Bearer-token auth via the /api/auth/login endpoint.
On 401, re-authenticates automatically once before raising.
"""
from __future__ import annotations
import logging
import httpx
logger = logging.getLogger(__name__)
class UnifiClient:
def __init__(
self,
host: str,
username: str,
password: str,
site: str = "default",
verify_ssl: bool = False,
) -> None:
self._host = host.rstrip("/")
self._username = username
self._password = password
self._site = site
self._verify_ssl = verify_ssl
self._http: httpx.AsyncClient | None = None
self._csrf_token: str = ""
# ── Lifecycle ─────────────────────────────────────────────────────────────
async def _ensure_http(self) -> None:
if self._http is None:
self._http = httpx.AsyncClient(
verify=self._verify_ssl,
timeout=15.0,
follow_redirects=True,
)
async def login(self) -> None:
await self._ensure_http()
resp = await self._http.post(
f"{self._host}/api/auth/login",
json={"username": self._username, "password": self._password, "rememberMe": False},
)
resp.raise_for_status()
self._csrf_token = resp.headers.get("X-CSRF-Token", "")
logger.debug("UniFi login OK (csrf=%s…)", self._csrf_token[:8] if self._csrf_token else "none")
async def close(self) -> None:
if self._http:
await self._http.aclose()
self._http = None
# ── API helpers ───────────────────────────────────────────────────────────
def _api_url(self, path: str) -> str:
return f"{self._host}/proxy/network/api/s/{self._site}{path}"
def _headers(self) -> dict:
return {"X-CSRF-Token": self._csrf_token} if self._csrf_token else {}
async def _get(self, path: str, params: dict | None = None) -> list | dict:
await self._ensure_http()
resp = await self._http.get(self._api_url(path), headers=self._headers(), params=params)
if resp.status_code == 401:
await self.login()
resp = await self._http.get(self._api_url(path), headers=self._headers(), params=params)
resp.raise_for_status()
data = resp.json()
return data.get("data", data)
# ── Core monitoring ───────────────────────────────────────────────────────
async def get_health(self) -> list[dict]:
"""Subsystem health (wan, wlan, lan, www)."""
result = await self._get("/stat/health")
return result if isinstance(result, list) else []
async def get_active_clients(self) -> list[dict]:
"""Currently connected clients."""
result = await self._get("/stat/sta")
return result if isinstance(result, list) else []
async def get_devices(self) -> list[dict]:
"""Managed network devices (APs, switches, gateways) with full detail."""
result = await self._get("/stat/device")
return result if isinstance(result, list) else []
# ── Events & alarms ───────────────────────────────────────────────────────
async def get_events(self, limit: int = 200) -> list[dict]:
"""Recent site events (client connects, IDS alerts, device state changes, etc.)."""
result = await self._get("/stat/event", params={"_limit": limit, "_sort": "-time"})
return result if isinstance(result, list) else []
async def get_alarms(self, archived: bool = False) -> list[dict]:
"""Active (or archived) alarms."""
result = await self._get("/stat/alarm")
if not isinstance(result, list):
return []
return [a for a in result if a.get("archived", False) == archived]
# ── Traffic & DPI ─────────────────────────────────────────────────────────
async def get_dpi_stats(self) -> list[dict]:
"""DPI per-client application/category traffic stats (cumulative counters)."""
result = await self._get("/stat/dpi")
return result if isinstance(result, list) else []
async def get_dpi_site_stats(self) -> list[dict]:
"""DPI site-level category totals (not per-client)."""
result = await self._get("/dpi")
return result if isinstance(result, list) else []
async def get_speedtest_status(self) -> dict | None:
"""Most recent WAN speed test result."""
result = await self._get("/stat/speedtest-status")
if isinstance(result, list) and result:
return result[0]
if isinstance(result, dict):
return result
return None
# ── Network inventory ─────────────────────────────────────────────────────
async def get_known_clients(self) -> list[dict]:
"""All historically seen clients (not just currently connected)."""
result = await self._get("/rest/user")
return result if isinstance(result, list) else []
async def get_networks(self) -> list[dict]:
"""Network/VLAN configurations."""
result = await self._get("/rest/networkconf")
return result if isinstance(result, list) else []
async def get_wlan_configs(self) -> list[dict]:
"""WLAN (SSID) configurations."""
result = await self._get("/rest/wlanconf")
return result if isinstance(result, list) else []
async def get_port_forwards(self) -> list[dict]:
"""Port forwarding rules."""
result = await self._get("/rest/portforwarding")
return result if isinstance(result, list) else []
async def get_firewall_rules(self) -> list[dict]:
"""Firewall rules."""
result = await self._get("/rest/firewallrule")
return result if isinstance(result, list) else []
async def get_firewall_groups(self) -> list[dict]:
"""Firewall groups (IP sets / port sets referenced by rules)."""
result = await self._get("/rest/firewallgroup")
return result if isinstance(result, list) else []
# ── System ────────────────────────────────────────────────────────────────
async def get_sysinfo(self) -> dict | None:
"""Controller system information (version, uptime, hostname)."""
result = await self._get("/stat/sysinfo")
if isinstance(result, list) and result:
return result[0]
if isinstance(result, dict):
return result
return None

Some files were not shown because too many files have changed in this diff Show More