fix(anomalies): one slow reading no longer opens an anomaly (#6793)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 1m38s
CI / integration (push) Successful in 2m3s
CI / publish (push) Successful in 1m30s

A single tall reading lifted a 15-minute average past the usual height
on its own, so one slow request opened a row that stayed open until the
reading rolled out of the window. Four of the seven rows that occurred
after the 10-30 detection change were this: HA web (one 6.7 s check),
error-pages, codex and mailcow-svc latency.

detect() now opens by height only when the window is still past the
usual height, and still departs, with its highest reading left out. A
window that fails that falls through to the held test, which now needs
at least HELD_FEWEST (3) readings behind its median, so a lone reading
on a quiet router opens nothing either way.

The median and the topless average share one per-key read (_per_key).
The architecture doc and get_metric_normal's description state the
rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-02 22:41:29 -05:00
co-authored by Claude Opus 5.5
parent 19f405e55a
commit c3072312a4
4 changed files with 104 additions and 22 deletions
+3 -2
View File
@@ -293,8 +293,9 @@ async def get_metric_normal(source_module: str, resource: str, metric: str) -> d
departed there (usual hold ×1.25, and at least `floor` above it). A
departure opens an anomaly once it passes both `usual_peak` and
`usual_anywhere` (how high the metric has gone in most hours of the day,
one figure for the whole key), or when the median of the last hour is over
`flag_above` too.
one figure for the whole key) even without the window's highest reading,
or when the median of the last hour (three readings or more) is over
`flag_above` too. A lone slow reading opens nothing.
`floor` is the smallest departure worth calling an anomaly, and
`floor_setting` the Settings → Thresholds floor it comes from. Hours an
anomaly covers are learned only once someone marks it normal.