feat(alerts): dispatch after commit; maintenance mutes, not skips; actions via trigger_run (#5063)
Notifications and alert actions now start only once the evaluating
transaction commits, and never for one that rolls back. record_metrics
queues them on the session (_after_commit), and after_commit/after_rollback
listeners on the sync session spawn or drop them. Spawned tasks are held in a
module set so they cannot be garbage-collected before they run.
A maintenance window no longer skips evaluation. Alerts in its scope still
change state, but their notifications and actions are held, and each event
says so ("notification muted: maintenance window <name>"; also on no-data
and resume events). notify_after_maintenance, run every minute with the
silence check as the new core task alert_reconcile, sends one notification
for an alert still firing after its window ends and runs its action. The
event's note records the release.
Alert actions go through runner.trigger_run like every other run path, so
the playbook and the inventory must be ones the source lists (#5034 guards).
The firing event links to the run, or its note says why the action did not
start. The maintenance page and form wording now say what a window does.
Tests:
- integration: a rolled-back evaluation notifies nobody, and dispatch waits
for the commit; maintenance mutes while the state moves and notifies once
when the window ends; an action whose playbook is outside its source is
refused, with the note set;
- the alert-action test uses a listed inventory name and waits, with a
deadline, for the run the action starts;
- unit: _muted_by.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -11,8 +11,9 @@
|
||||
</div>
|
||||
|
||||
<p style="color:var(--text-muted);font-size:0.88rem;margin-bottom:1.5rem;">
|
||||
During an active window, alert rule evaluation is suppressed for the matching scope.
|
||||
Metrics are still recorded — only notifications are silenced.
|
||||
During an active window, alerts in its scope still change state and show on the Alerts page,
|
||||
but send no notifications and run no Ansible actions. Each event a window mutes says so in
|
||||
the alert's history. An alert that is still firing when the window ends notifies then, and runs its action.
|
||||
</p>
|
||||
|
||||
{% if windows %}
|
||||
|
||||
@@ -50,7 +50,7 @@
|
||||
<input type="text" name="scope_resource" id="scope-resource"
|
||||
placeholder="Exact resource name (e.g. my-server)">
|
||||
<p style="font-size:0.78rem;color:var(--text-muted);margin-top:0.25rem;">
|
||||
Leave blank to suppress all resources within the selected source.
|
||||
Leave blank to mute all resources within the selected source.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
@@ -64,9 +64,9 @@
|
||||
<div class="card" style="margin-top:1rem;">
|
||||
<div class="section-title" style="margin-bottom:0.5rem;">How scope works</div>
|
||||
<div style="display:grid;gap:0.4rem;font-size:0.82rem;color:var(--text-muted);">
|
||||
<div><strong style="color:var(--text);">All alerts</strong> — every alert rule is suppressed</div>
|
||||
<div><strong style="color:var(--text);">By source</strong> — e.g. source=ping suppresses all ping alerts</div>
|
||||
<div><strong style="color:var(--text);">By source + resource</strong> — e.g. ping/my-server suppresses only that host's ping alerts</div>
|
||||
<div><strong style="color:var(--text);">All alerts</strong> — every alert rule's notifications are muted</div>
|
||||
<div><strong style="color:var(--text);">By source</strong> — e.g. source=icmp mutes all ping alerts</div>
|
||||
<div><strong style="color:var(--text);">By source + resource</strong> — e.g. icmp/my-server mutes only that host's ping alerts</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
Reference in New Issue
Block a user