fix(alerts): acknowledgements hold until resolve; clear after N OK; rule changes reset (#5059)
#5059 "Alert state machine: acknowledgement holds until resolve; N consecutive OK to resolve; editing, disabling or deleting a rule resets it". - The state machine is now next_step(), a pure function of one sample. An acknowledged alert stays acknowledged while breached; before, the next breaching sample re-fired it. Firing and acknowledged alerts resolve after consecutive_ok_required OK checks in a row (new rule column, default 1; rule form, rules list and MCP create/update_alert_rule clear_after_checks). - Changing what a rule watches (source, resource, metric, operator, threshold) or disabling it resets its alert to inactive; a firing or acknowledged one records a resolved event whose new note says why, and nobody is notified. Web and MCP edits share alerts.service.apply_rule_change. - Migration 0028: consecutive_ok_required, consecutive_ok_count, alert_events.note, and metric_value nullable (a reset has no sample). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -41,6 +41,9 @@ class AlertRule(Base):
|
||||
)
|
||||
threshold: Mapped[float] = mapped_column(Float, nullable=False)
|
||||
consecutive_failures_required: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
|
||||
# Non-breaching checks in a row before a firing (or acknowledged) alert resolves.
|
||||
consecutive_ok_required: Mapped[int] = mapped_column(Integer, nullable=False, default=1,
|
||||
server_default="1")
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
|
||||
# Optional admin-only action: run an Ansible playbook on transition-to-firing.
|
||||
# Shape: {source, playbook, inventory, extra_vars, limit, tags, check}. NULL = none.
|
||||
@@ -62,6 +65,8 @@ class AlertState(Base):
|
||||
Enum(AlertStateEnum), nullable=False, default=AlertStateEnum.inactive
|
||||
)
|
||||
consecutive_failure_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
consecutive_ok_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0,
|
||||
server_default="0")
|
||||
last_transition_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, default=lambda: datetime.now(timezone.utc)
|
||||
)
|
||||
@@ -98,7 +103,10 @@ class AlertEvent(Base):
|
||||
)
|
||||
from_state: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||
to_state: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||
metric_value: Mapped[float] = mapped_column(Float, nullable=False)
|
||||
# None when the transition came from a change to the rule, not a sample.
|
||||
metric_value: Mapped[float | None] = mapped_column(Float, nullable=True)
|
||||
# Why a transition happened when the value alone doesn't say (a rule edit).
|
||||
note: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
transitioned_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, default=lambda: datetime.now(timezone.utc)
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user