Commit Graph
9 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 fcd58224de feat(syslog): a device through Docker's address is placed by its name, or a device address set on it (#6096)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 1m32s
CI / integration (push) Successful in 1m42s
CI / publish (push) Successful in 20s
The SNMP poll asks each device's sysName in the same GET and keeps it
(snmp_003_sys_name). A syslog source told apart by name finds its own
address from the SNMP device (reported or given name) or host of that
name, so its lines reach the SNMP device's and host's pages again; the
edit page gains Device address, which always wins (syslog_004).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 12:24:58 -04:00
bvandeusenandClaude Opus 5.5 ad8bbd8d69 feat(snmp): a NAS's disk and volume I/O, CPU waiting on disk, available memory, interface errors (#5997)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m31s
CI / unit (push) Failing after 1m32s
CI / publish (push) Skipped
- Synology: what each disk and volume reads and writes, bytes and requests
  a second and the load DSM gives it (SYNOLOGY-STORAGEIO- and -SPACEIO-MIB,
  64-bit byte counters), and each disk's reconnects and sectors not found.
- Linux server: CPU waiting on disk, the wait tick counter's share of the
  CPU's, and memory available (free + buffers + cache). A reading may now be
  worked out from others: the poll asks for its inputs, rates each counter,
  and stores only the result.
- Interface traffic: errors and discards in and out, per second.
- byte_count unit: a byte counter shown as bytes a second.

Asustor's MIB has nothing more for this; its Linux server readings gain the
same CPU wait and available memory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 09:50:52 -04:00
bvandeusenandClaude Opus 5.5 41fc8073db feat: readings kept through a storage stall; an agent's backlog arrives in order (#5995)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 1m39s
CI / integration (push) Successful in 1m49s
CI / publish (push) Successful in 1m49s
On 2026-10-17 the NFS under Postgres stalled for ~15 minutes. Two things
went wrong for the readings that would have explained it:

- The SNMP poll stored before it polled again, so while the database hung
  it stopped polling. It now collects on schedule and hands each round to a
  Holding (steward/core/holding.py): one writer stores what is held, oldest
  first, each reading at the time it was collected. Bounded at two hours of
  polls, with a 30-minute deadline on a store; the device list read falls
  back to the last one read after 10 s.
- "Mount not answering" (10 checks in a row) went pending on Docker02 and
  Docker03 and never fired: the agent sent each post's backlog with its
  newest, healthy sample, and that sample ended every post's run. Agent
  1.22.0 queues the new sample behind the backlog, so Steward receives a
  host's samples in the order they were taken.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 09:36:47 -04:00
bvandeusenandClaude Opus 5.5 d065cafd11 feat(snmp): NAS presets for Synology and Asustor, reading every disk and volume (#5985, #5986)
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 1m38s
CI / integration (push) Successful in 1m41s
CI / publish (push) Successful in 1m31s
A preset can now read a vendor's tables: one row per disk, volume, fan or
sensor. The scheduled poll discovers the rows by walking the device: on its
first poll, after a save, and then hourly, and only while it answers. Each row
becomes readings named by the device's own labels ("Disk 1 status"). Only the
columns a row answered are asked for.

A vendor status, whether a code (Synology) or a word (Asustor), is a state
reading. It is stored as a level (0 OK, 1 busy, 2 warning, 3 failed), so one
alert rule covers any vendor, and shown in the device's own word.

- Synology: system, power and fan status and temperature; per disk: status,
  health, temperature and bad sectors; per volume: RAID state (scrubbing,
  parity check, rebuild, degraded), free space and size.
- Asustor: per disk: status, SMART and temperature; per volume: status, free
  space and size; CPU and system temperature, free memory and fans.
  From ADM's ASUSTOR-DISK-MIB and ASUSTOR-SYSTEM-MIB.
- Linux server: plus each physical disk's 1-minute busy % (UCD-DISKIO-MIB).

Migration snmp_002_table_rows adds tables, tables_at and texts to
snmp_devices. Milestone 530.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 22:40:14 -04:00
bvandeusenandClaude Opus 5.5 9ca2e38fa1 feat(snmp): devices you add, test and read on the SNMP page (milestone 491)
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 55s
CI / unit (push) Successful in 1m24s
CI / publish (push) Skipped
The operator, reviewing SNMP on #5214: "this system looks incomplete like
there's something missing from it". The one device shown was plugin.yaml's
example (core-switch 192.168.1.1, community public), polled every minute;
devices could only be defined by editing YAML in the image; a failed poll was
logged at DEBUG and stored nowhere, so the page said "waiting for the next
poll" forever; v3 ran as v2c; counters showed as lifetime totals.

- Devices are rows (snmp_devices, migration snmp_001): address, port,
  v1/v2c community or a v3 user with auth/priv protocols and keys (secrets
  encrypted), presets plus custom readings, an optional host binding, and
  the newest poll's outcome. The YAML device list and its example are gone.
- The SNMP page adds, edits and deletes them (admin, audited, with
  get_audit_entities so the audit log names and links a device). Readings
  come from presets (system uptime, interface traffic by number, UPS-MIB,
  Linux net-snmp) and custom "OID name [unit]" lines.
- One GET per device with every OID, devices polled at once. The outcome is
  written to the row in words ("No reply from … within 5 s. Check the address
  and port, that SNMP is on, and the community"), with readings it lacks.
  v1's whole-request error drops that OID and asks again. SNMPv3 signs in
  with USM (MD5/SHA-1/SHA-2, DES/3DES/AES).
- Test asks the unsaved form's device for its name, description, uptime and
  interfaces by number. It is an admin's explicit action; every page reads
  only stored rows (System 178's ruling).
- Each reading has a unit; a counter is stored as a per-second rate (bytes as
  bit/s), a reset or wrap skipped. The page, widget and host panel say values
  in units and why a device is silent; the device page draws a chart per
  reading on a dated axis.

Tests: readings and presets, the form and its secrets, v3 checks, rates,
the poller against a stand-in pysnmp (one GET, v1 retry, timeout wording,
USM), the pages, and an integration test of the poll writing outcomes and
rates on Postgres.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:54:11 -04:00
bvandeusenandClaude Opus 5.5 58bfc7c361 feat(alerts): one evaluation path; host-agent ingest fires rules (#5060)
CI / lint (push) Successful in 2s
CI / integration (push) Failing after 45s
CI / unit (push) Successful in 1m7s
CI / publish (push) Skipped
#5060 "One evaluation path: record_metrics batch API; host-agent ingest
evaluates rules".

- record_metrics(session, rows) writes metric rows and evaluates the rules
  that watch them, reading rules, states and maintenance windows once per
  batch. Each rule sees its key's samples oldest first, so an agent catching
  up after an outage counts consecutive checks as they happened.
- Host-agent ingest now goes through it. Before, it wrote plugin_metrics
  directly and no host_agent rule could ever fire. The catalog gains the
  per-mount disk_used_pct ("<host>:<mount>").
- MetricBatch gathers a transaction's samples; the docker ingest, traefik,
  unifi, snmp, monitor and self-monitor writers record one batch each instead
  of a rule query per sample.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 12:53:40 -04:00
bvandeusenandClaude Opus 4.8 4fc8c96c41 fix(snmp): bundle pysnmp in image and port poller to the asyncio HLAPI
CI / unit (push) Successful in 46s
CI / integration (push) Successful in 2m21s
CI / lint (push) Successful in 3s
CI / publish (push) Successful in 1m16s
The SNMP plugin ships in the image but logged "pysnmp not installed — SNMP
polling disabled" on every poll, so polling never worked. Two coupled defects:

1. The Dockerfile installed only `.[ansible]`, so the `snmp` extra (pysnmp)
   was never bundled even though the plugin is first-party and shipped.
2. poller.py used the synchronous pysnmp HLAPI (`next(getCmd(...))`), which
   pysnmp-lextudio 6.x removed — it's asyncio-only now — so even with the dep
   present, polling would have thrown and silently returned nothing. The 5.x
   line that still has the sync API isn't safe on the image's Python 3.13.

Fix:
- Dockerfile: install `.[ansible,snmp]`.
- poller.py: `poll_device_sync` → `async def poll_device` on the asyncio HLAPI,
  with a dual-version import (pysnmp 7.x `pysnmp.hlapi.v3arch.asyncio`/`get_cmd`
  + async `UdpTransportTarget.create`; pysnmp-lextudio 6.2.x
  `pysnmp.hlapi.asyncio`/`getCmd` + direct `UdpTransportTarget`) so a dependency
  bump can't silently re-break it.
- scheduler.py: await poll_device directly; drop the run_in_executor wrapper
  and the now-unused asyncio import.
- Add tests/plugins/snmp/test_poller.py covering the version→mpModel mapping,
  that the poller is a coroutine, and the graceful no-pysnmp path.

Note: CI confirms import/load and the no-pysnmp path, but has no SNMP target —
live polling against real devices is verified after deploy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
2026-06-19 13:28:07 -04:00
bvandeusenandClaude Opus 4.8 af60ca446d fix(plugins): complete steward rename across bundled plugins; clear lint debt
CI / integration (push) Successful in 2m8s
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 7s
The fabledscryer->steward rename had only ever reached host_agent. The other
five bundled plugins (http, snmp, traefik, unifi, docker) still imported
`from fabledscryer.*` (package no longer exists) and read FABLEDSCRYER_* env
vars — so every one of them was broken at import since the original rebrand.
CI stayed green only because none are enabled by default and migrations don't
import plugin modules. Now that they version in-tree, complete the rename:
- fabledscryer.* -> steward.* imports across all five plugins
- FABLEDSCRYER_* -> STEWARD_* in plugin migration env.py files
- author/repository/homepage + user-facing 'Fabled Scryer' strings -> Steward
- snmp/scheduler.py: also drop dead `now`/datetime; record_metric from steward

Adds tests/test_no_legacy_names.py — fails if 'scryer'/'roundtable' ever
reappear in shipped code (the drift bit twice; this stops a third time).

Also clears pre-existing ruff lint debt (unused imports, semicolon statements,
mid-file import) surfaced by the new lint lane.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 11:22:20 -04:00
bvandeusenandClaude Opus 4.8 a7a281cb11 feat(plugins): fold first-party plugins in-tree; bundled + external roots
First-party plugins (host_agent, http, snmp, traefik, unifi, docker) are now
tracked under plugins/ and baked into the image, so they version atomically
with core — ending the cross-repo import drift the roundtable->steward rename
exposed. History for these files is preserved in the archived Roundtable-plugins
repo.

Plugin discovery becomes multi-root: PLUGIN_DIR (single) -> PLUGIN_DIRS
(bundled first, then external) + PLUGIN_INSTALL_DIR. Bundled ships in the image;
third-party plugins still mount at runtime into the external root
(STEWARD_PLUGIN_DIR, default /data/plugins) and downloads/installs land there.
Bundled shadows external on a name collision.

- config.py: load_bootstrap returns plugin_dirs + plugin_install_dir
- app.py: iterate PLUGIN_DIRS at the migration + load sites
- migration_runner.py: discover_all_in() unions every plugin root
- plugin_manager.py: resolve_plugin_path() (pure, first-root-wins); load /
  install / hot-reload span all roots; installs target the external root
- settings/routes.py: _discover_plugins scans all roots, dedup bundled-first
- Dockerfile: COPY plugins/ ; docker-compose: drop host bind, document external
- tests/test_plugin_dirs.py: resolution, multi-root discovery, bootstrap split

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 08:37:24 -04:00