feat(alerts): dispatch after commit; maintenance mutes, not skips; actions via trigger_run (#5063)
CI / lint (push) Successful in 2s
CI / unit (push) Successful in 1m11s
CI / integration (push) Failing after 2m56s
CI / publish (push) Skipped

Notifications and alert actions now start only once the evaluating
transaction commits, and never for one that rolls back. record_metrics
queues them on the session (_after_commit), and after_commit/after_rollback
listeners on the sync session spawn or drop them. Spawned tasks are held in a
module set so they cannot be garbage-collected before they run.

A maintenance window no longer skips evaluation. Alerts in its scope still
change state, but their notifications and actions are held, and each event
says so ("notification muted: maintenance window <name>"; also on no-data
and resume events). notify_after_maintenance, run every minute with the
silence check as the new core task alert_reconcile, sends one notification
for an alert still firing after its window ends and runs its action. The
event's note records the release.

Alert actions go through runner.trigger_run like every other run path, so
the playbook and the inventory must be ones the source lists (#5034 guards).
The firing event links to the run, or its note says why the action did not
start. The maintenance page and form wording now say what a window does.

Tests:
- integration: a rolled-back evaluation notifies nobody, and dispatch waits
  for the commit; maintenance mutes while the state moves and notifies once
  when the window ends; an action whose playbook is outside its source is
  refused, with the note set;
- the alert-action test uses a listed inventory name and waits, with a
  deadline, for the run the action starts;
- unit: _muted_by.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-06 13:45:20 -04:00
co-authored by Claude Opus 5.5
parent b009a8dd30
commit 49607a2441
7 changed files with 371 additions and 82 deletions
+3 -2
View File
@@ -11,8 +11,9 @@
</div>
<p style="color:var(--text-muted);font-size:0.88rem;margin-bottom:1.5rem;">
During an active window, alert rule evaluation is suppressed for the matching scope.
Metrics are still recorded — only notifications are silenced.
During an active window, alerts in its scope still change state and show on the Alerts page,
but send no notifications and run no Ansible actions. Each event a window mutes says so in
the alert's history. An alert that is still firing when the window ends notifies then, and runs its action.
</p>
{% if windows %}
@@ -50,7 +50,7 @@
<input type="text" name="scope_resource" id="scope-resource"
placeholder="Exact resource name (e.g. my-server)">
<p style="font-size:0.78rem;color:var(--text-muted);margin-top:0.25rem;">
Leave blank to suppress all resources within the selected source.
Leave blank to mute all resources within the selected source.
</p>
</div>
@@ -64,9 +64,9 @@
<div class="card" style="margin-top:1rem;">
<div class="section-title" style="margin-bottom:0.5rem;">How scope works</div>
<div style="display:grid;gap:0.4rem;font-size:0.82rem;color:var(--text-muted);">
<div><strong style="color:var(--text);">All alerts</strong> — every alert rule is suppressed</div>
<div><strong style="color:var(--text);">By source</strong> — e.g. source=ping suppresses all ping alerts</div>
<div><strong style="color:var(--text);">By source + resource</strong> — e.g. ping/my-server suppresses only that host's ping alerts</div>
<div><strong style="color:var(--text);">All alerts</strong> — every alert rule's notifications are muted</div>
<div><strong style="color:var(--text);">By source</strong> — e.g. source=icmp mutes all ping alerts</div>
<div><strong style="color:var(--text);">By source + resource</strong> — e.g. icmp/my-server mutes only that host's ping alerts</div>
</div>
</div>
</div>