fix(anomalies): a burst within what a metric reaches in most hours opens nothing (#6727)
#6727 "A metric that bursts now and then opens anomalies at hours whose usual peak reads 0". usual_peak is the p90 of a slot's hourly max, so a metric that bursts in under about 1 hour in 10 reads 0 there however often it bursts across the day. #6723 "An anomaly opens on a burst of minutes inside the metric's usual peak, so most rows aren't outside the norm" therefore still opened on those. usual_anywhere is the median, across the hours of the day, of each hour's highest peak. Height is now judged against the greater of it and the slot's usual peak. Bursts at any hour lift the median; a job at a time of its own raises a few hours and leaves it low, so the same level at another time of day is still an anomaly, as System 182 rules. A held level still opens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -53,7 +53,7 @@ async def list_anomalies(review: str = "to_review", limit: int = 50) -> dict:
|
||||
"""Metric anomalies: a learned metric whose 15-minute level went well
|
||||
above what is normal for it at that hour (its usual hold ×1.25, and by
|
||||
more than the metric's floor), and either past how high it usually goes
|
||||
(`usual_peak`) or held there for most of an hour.
|
||||
(see get_metric_normal) or held there for most of an hour.
|
||||
|
||||
review: "to_review" (default) — waiting for someone to mark them; "normal"
|
||||
— accepted, their hours are learned; "real" — a real problem, not learned;
|
||||
@@ -291,8 +291,10 @@ async def get_metric_normal(source_module: str, resource: str, metric: str) -> d
|
||||
90th percentile across past days), `usual_peak` (how high they usually
|
||||
went), `days_seen`, and `flag_above`: the 15-minute average that has
|
||||
departed there (usual hold ×1.25, and at least `floor` above it). A
|
||||
departure opens an anomaly once it passes `usual_peak`, or when the median
|
||||
of the last hour is over `flag_above` too.
|
||||
departure opens an anomaly once it passes both `usual_peak` and
|
||||
`usual_anywhere` (how high the metric has gone in most hours of the day,
|
||||
one figure for the whole key), or when the median of the last hour is over
|
||||
`flag_above` too.
|
||||
`floor` is the smallest departure worth calling an anomaly, and
|
||||
`floor_setting` the Settings → Thresholds floor it comes from. Hours an
|
||||
anomaly covers are learned only once someone marks it normal.
|
||||
@@ -319,6 +321,7 @@ async def get_metric_normal(source_module: str, resource: str, metric: str) -> d
|
||||
out = {
|
||||
"source_module": source_module, "resource": resource, "metric": metric,
|
||||
"learned": bool(slots), "kind": kind,
|
||||
"usual_anywhere": rnd(rows[0].usual_anywhere, 3) if rows else None,
|
||||
"floor": rnd(floor, 3), "floor_setting": learned.floor_name,
|
||||
"only_while": ({"metric": learned.gate[0],
|
||||
"at_least": rnd(learned.gate[1]() if callable(learned.gate[1]) else learned.gate[1], 3),
|
||||
|
||||
Reference in New Issue
Block a user