bvandeusen
88ab5b917e
Renames the Python package directory, CLI command, env var prefix, docker-compose service/container/image, Postgres role/db, and all visible branding. Marketing form is "Fabled Steward". Clean break from the previous rebrand: drops the fabledscryer→roundtable import shim in __init__.py and the FABLEDSCRYER_* env var fallback in config.py and migrations/env.py. Env vars are now STEWARD_* only. Heads-up for existing deployments: - Postgres user/db renamed fabledscryer → steward in docker-compose.yml. Existing volumes need the role/db renamed inside Postgres, or override POSTGRES_USER/POSTGRES_DB to keep the old names. - Host-agent systemd unit is now steward-agent.service. Existing agents keep running under the old name; reinstall to switch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
105 lines
4.1 KiB
Markdown
105 lines
4.1 KiB
Markdown
# Core Monitors
|
|
|
|
Steward ships two built-in monitors: Ping and DNS. Both run as asyncio scheduled tasks on the same event loop as the web server, with no separate processes.
|
|
|
|
---
|
|
|
|
## Ping Monitor
|
|
|
|
**Source:** `steward/monitors/ping.py`
|
|
**Scheduler task:** `ping_monitor` in `steward/app.py`
|
|
**Interval:** `monitors.poll_interval_seconds` (default 60s), runs on startup
|
|
|
|
### How It Works
|
|
|
|
On each tick, the scheduler fetches all hosts with `ping_enabled = true` and calls `ping_check(host, session)` for each.
|
|
|
|
`ping_check()` probes the host using:
|
|
- **ICMP** if `host.probe_type == "icmp"` — uses the system `ping` binary (`/bin/ping` or equivalent). Requires `iputils-ping` in Docker.
|
|
- **TCP** if `host.probe_type == "tcp"` (default) — attempts an async TCP connection to `host.address:host.probe_port` (default port 80).
|
|
|
|
Each probe writes a `PingResult` row and calls `record_metric()`:
|
|
|
|
| `source_module` | `resource_name` | `metric_name` | `value` |
|
|
|---|---|---|---|
|
|
| `ping` | `host.name` | `response_time_ms` | measured latency, or `0.0` if down |
|
|
| `ping` | `host.name` | `up` | `1.0` if up, `0.0` if down |
|
|
|
|
**Alert rule note:** Because `response_time_ms` is recorded as `0.0` when a host is down, a latency rule (e.g. `response_time_ms > 500`) will not fire on complete outages. Use a separate rule on `up == 0.0` to detect host down events.
|
|
|
|
### Data Model
|
|
|
|
`ping_results` table (defined in `steward/models/monitors.py`):
|
|
|
|
| Column | Type | Description |
|
|
|---|---|---|
|
|
| `id` | UUID | Primary key |
|
|
| `host_id` | FK → hosts | |
|
|
| `probed_at` | timestamp UTC | When the probe ran |
|
|
| `status` | enum `up`/`down` | Result |
|
|
| `response_time_ms` | float | Null if down |
|
|
|
|
Old rows are pruned by the `data_cleanup` task (default: 90 days).
|
|
|
|
### UI
|
|
|
|
- **Dashboard widget** — live-updating via HTMX polling (`/ping/rows`)
|
|
- **`/ping/` page** — full page with 30-pill history per host and threshold settings form
|
|
- **Hosts list** — shows latest ping status dot and latency
|
|
|
|
---
|
|
|
|
## DNS Monitor
|
|
|
|
**Source:** `steward/monitors/dns.py`
|
|
**Scheduler task:** `dns_monitor` in `steward/app.py`
|
|
**Interval:** `monitors.poll_interval_seconds` (default 60s), runs on startup
|
|
|
|
### How It Works
|
|
|
|
On each tick, the scheduler fetches all hosts with `dns_enabled = true` and calls `dns_check(host, session)` for each.
|
|
|
|
`dns_check()` resolves `host.address` using the system resolver. If `host.dns_expected_ip` is set, the check passes only if at least one returned A/AAAA record exactly matches that string. If `dns_expected_ip` is null, any successful resolution counts as a pass.
|
|
|
|
Each check writes a `DnsResult` row and calls `record_metric()`:
|
|
|
|
| `source_module` | `resource_name` | `metric_name` | `value` |
|
|
|---|---|---|---|
|
|
| `dns` | `host.name` | `resolved` | `1.0` if resolved, `0.0` if failed |
|
|
| `dns` | `host.name` | `ip_changed` | `1.0` if IP changed from last successful result, `0.0` otherwise |
|
|
|
|
### Data Model
|
|
|
|
`dns_results` table (defined in `steward/models/monitors.py`):
|
|
|
|
| Column | Type | Description |
|
|
|---|---|---|
|
|
| `id` | UUID | Primary key |
|
|
| `host_id` | FK → hosts | |
|
|
| `resolved_at` | timestamp UTC | When the check ran |
|
|
| `status` | enum `resolved`/`failed` | Result |
|
|
| `resolved_ip` | str | First returned A/AAAA record; null if failed |
|
|
|
|
`ip_changed` is computed by comparing `resolved_ip` of the current result against the most recent prior `resolved` result for the same host.
|
|
|
|
### UI
|
|
|
|
- **Dashboard widget** — live-updating via HTMX polling (`/dns/rows`)
|
|
- **`/dns/` page** — full page showing all DNS-enabled hosts with status, resolved IP, and timestamp
|
|
- **Hosts list** — shows latest DNS status dot
|
|
|
|
---
|
|
|
|
## Host Configuration
|
|
|
|
Hosts are managed at `/hosts/`. Both monitors are configured per-host:
|
|
|
|
| Field | Description |
|
|
|---|---|
|
|
| `ping_enabled` | Enable ping probing for this host |
|
|
| `probe_type` | `tcp` (default) or `icmp` |
|
|
| `probe_port` | TCP port to connect to (default 80; ignored for ICMP) |
|
|
| `dns_enabled` | Enable DNS resolution checks |
|
|
| `dns_expected_ip` | If set, the resolved IP must match this string exactly |
|
|
| `poll_interval_seconds` | Per-host override for the global poll interval; null uses global |
|