fix(anomalies): a burst within what a metric reaches in most hours opens nothing (#6727)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 1m36s
CI / integration (push) Successful in 1m50s
CI / publish (push) Successful in 1m11s

#6727 "A metric that bursts now and then opens anomalies at hours whose usual
peak reads 0". usual_peak is the p90 of a slot's hourly max, so a metric that
bursts in under about 1 hour in 10 reads 0 there however often it bursts across
the day. #6723 "An anomaly opens on a burst of minutes inside the metric's usual
peak, so most rows aren't outside the norm" therefore still opened on those.

usual_anywhere is the median, across the hours of the day, of each hour's
highest peak. Height is now judged against the greater of it and the slot's
usual peak. Bursts at any hour lift the median; a job at a time of its own
raises a few hours and leaves it low, so the same level at another time of day
is still an anomaly, as System 182 rules. A held level still opens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-30 19:45:16 -04:00
co-authored by Claude Opus 5.5
parent db39fc2604
commit 177ea12c9b
7 changed files with 122 additions and 26 deletions
+6 -3
View File
@@ -53,7 +53,7 @@ async def list_anomalies(review: str = "to_review", limit: int = 50) -> dict:
"""Metric anomalies: a learned metric whose 15-minute level went well
above what is normal for it at that hour (its usual hold ×1.25, and by
more than the metric's floor), and either past how high it usually goes
(`usual_peak`) or held there for most of an hour.
(see get_metric_normal) or held there for most of an hour.
review: "to_review" (default) — waiting for someone to mark them; "normal"
— accepted, their hours are learned; "real" — a real problem, not learned;
@@ -291,8 +291,10 @@ async def get_metric_normal(source_module: str, resource: str, metric: str) -> d
90th percentile across past days), `usual_peak` (how high they usually
went), `days_seen`, and `flag_above`: the 15-minute average that has
departed there (usual hold ×1.25, and at least `floor` above it). A
departure opens an anomaly once it passes `usual_peak`, or when the median
of the last hour is over `flag_above` too.
departure opens an anomaly once it passes both `usual_peak` and
`usual_anywhere` (how high the metric has gone in most hours of the day,
one figure for the whole key), or when the median of the last hour is over
`flag_above` too.
`floor` is the smallest departure worth calling an anomaly, and
`floor_setting` the Settings → Thresholds floor it comes from. Hours an
anomaly covers are learned only once someone marks it normal.
@@ -319,6 +321,7 @@ async def get_metric_normal(source_module: str, resource: str, metric: str) -> d
out = {
"source_module": source_module, "resource": resource, "metric": metric,
"learned": bool(slots), "kind": kind,
"usual_anywhere": rnd(rows[0].usual_anywhere, 3) if rows else None,
"floor": rnd(floor, 3), "floor_setting": learned.floor_name,
"only_while": ({"metric": learned.gate[0],
"at_least": rnd(learned.gate[1]() if callable(learned.gate[1]) else learned.gate[1], 3),