feat(alerts): dispatch after commit; maintenance mutes, not skips; actions via trigger_run (#5063)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 1m11s
CI / integration (push) Failing after 2m56s
CI / publish (push) Skipped

Notifications and alert actions now start only once the evaluating
transaction commits, and never for one that rolls back. record_metrics
queues them on the session (_after_commit), and after_commit/after_rollback
listeners on the sync session spawn or drop them. Spawned tasks are held in a
module set so they cannot be garbage-collected before they run.

A maintenance window no longer skips evaluation. Alerts in its scope still
change state, but their notifications and actions are held, and each event
says so ("notification muted: maintenance window <name>"; also on no-data
and resume events). notify_after_maintenance, run every minute with the
silence check as the new core task alert_reconcile, sends one notification
for an alert still firing after its window ends and runs its action. The
event's note records the release.

Alert actions go through runner.trigger_run like every other run path, so
the playbook and the inventory must be ones the source lists (#5034 guards).
The firing event links to the run, or its note says why the action did not
start. The maintenance page and form wording now say what a window does.

Tests:
- integration: a rolled-back evaluation notifies nobody, and dispatch waits
  for the commit; maintenance mutes while the state moves and notifies once
  when the window ends; an action whose playbook is outside its source is
  refused, with the note set;
- the alert-action test uses a listed inventory name and waits, with a
  deadline, for the run the action starts;
- unit: _muted_by.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-06 13:45:20 -04:00
co-authored by Claude Opus 5.5
parent b009a8dd30
commit 49607a2441
7 changed files with 371 additions and 82 deletions
+11
View File
@@ -96,3 +96,14 @@ def test_a_sample_after_silence_clears_it_and_records_the_resume():
assert value == 42.0 and note == "data resumed after 15m silent"
[event] = session.added
assert (event.id, event.from_state, event.to_state) == (event_id, "no_data", "firing")
def test_muted_by_names_the_window_that_covers_the_rule():
from steward.core.alerts import _muted_by
rule = NS(source_module="host_agent", resource_name="nas")
windows = [NS(name="other", scope_source="docker", scope_resource=None),
NS(name="nas work", scope_source="host_agent", scope_resource="nas")]
assert _muted_by(windows, rule) == "notification muted: maintenance window nas work"
assert _muted_by(windows[:1], rule) is None
assert _muted_by([NS(name="all", scope_source=None, scope_resource=None)], rule) == (
"notification muted: maintenance window all")