bvandeusenandClaude Opus 5.5
f3433fe3ce
Host CPU, memory and load per core, container CPU and memory, and every monitor's response time learn a normal per hour (milestone 513, System 182). A reading past its fixed cutoff but within its normal reads healthy and says what is usual at this hour; one well out of it keeps its grade and is a metric anomaly. Disk, a monitor down and an agent offline keep their cutoffs. - Each normal's floor, the smallest departure that counts, is a setting: Settings -> Thresholds -> Normals (norms.floor.*). Defaults: host CPU 10 points, host memory 5, load per core 10, container CPU 10 points of one core, container memory 5, monitor response 100 ms; Traefik's 2-point 4xx and 50 ms p95 floors move there from constants. learn_normal takes a floor function (floor_setting) read at each pass. - learn_normal(parts=False): a host learns, its cores don't. - Ingest records load_pct (load_1m / cores x 100), the figure pages grade. - Shared grading: anomalies.against / graded, worst_reading(normals=), hosts' host_grades, docker usage(thresholds=, norms=), monitors' latency_grade. Applied to host vitals, the Hosts widget, MCP overview, Docker usage, and the monitor list, widget and page. - The monitor and container pages list their anomalies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
56 lines
2.0 KiB
Python
56 lines
2.0 KiB
Python
# plugins/host_agent/__init__.py
|
|
from __future__ import annotations
|
|
from typing import TYPE_CHECKING
|
|
|
|
if TYPE_CHECKING:
|
|
from quart import Quart
|
|
|
|
_app: "Quart | None" = None
|
|
|
|
|
|
def setup(app: "Quart") -> None:
|
|
global _app
|
|
_app = app
|
|
# A host's CPU, memory and load per core learn their normal (#5742), each
|
|
# with its floor from Settings → Thresholds. Parts don't: a host's CPU has
|
|
# a normal, each of its cores doesn't. Disks stay on their fixed cutoffs:
|
|
# a nearly full disk is a failure whatever its history (System 182).
|
|
from steward.core.norms import floor_setting, learn_normal
|
|
learn_normal("host_agent", "cpu_pct", floor_setting(app, "host_cpu_pct"), parts=False)
|
|
learn_normal("host_agent", "mem_used_pct", floor_setting(app, "host_mem_pct"), parts=False)
|
|
learn_normal("host_agent", "load_pct", floor_setting(app, "host_load_pct"), parts=False)
|
|
|
|
|
|
def get_scheduled_tasks() -> list:
|
|
from .scheduler import make_task
|
|
return [make_task(_app)]
|
|
|
|
|
|
def get_blueprint():
|
|
from .routes import host_agent_bp
|
|
return host_agent_bp
|
|
|
|
|
|
def register_mcp_tools(register) -> None:
|
|
from .mcp_tools import register_tools
|
|
register_tools(register)
|
|
|
|
|
|
async def mcp_overview(db) -> list[dict]:
|
|
from .mcp_tools import mcp_overview as findings
|
|
return await findings(db)
|
|
|
|
|
|
def get_alert_metrics() -> dict[str, list[str]]:
|
|
"""Host metrics ingest records for alert rules (resource = the host name;
|
|
disk_used_pct per mount, resource "<host>:<mount>"). seconds_since_seen
|
|
comes from the scheduler, not the agent: "agent offline" is > 180.
|
|
clock_off_secs is how far the host's clock is from Steward's, either way
|
|
(agent ≥ 1.12.0): "clock drifting" is > 30. load_pct is load_1m as a %
|
|
of the host's cores."""
|
|
return {"host_agent": [
|
|
"cpu_pct", "mem_used_pct", "mem_available_bytes", "swap_used_bytes",
|
|
"disk_used_pct_worst", "disk_used_pct", "load_1m", "load_5m", "load_15m", "load_pct",
|
|
"uptime_secs", "seconds_since_seen", "clock_off_secs",
|
|
]}
|