feat(email): delivery you can see: each alert's email in history, outgoing email on Settings -> System (#5509)

#5509 "Delivery you can see: each alert's email state in history, and
outgoing email on Settings -> System" (milestone 501).

- Alerts -> History: an alert's email reads from the outbox (#5506):
  "gathering" with when it sends, "sending", "retrying" with the attempt,
  next try and the server's last answer, "sent", or "failed" naming each
  reader it didn't reach and why (outbox.delivery / email_states). Events
  from before the outbox keep what they recorded. The Notified column
  shows once an alert has a queued email.
- Settings -> System gets "Outgoing email": every email of the last 7
  days not yet sent, failures first, then the longest retrying, with
  recipient, kind, state and last error, and a count of what was sent.
  Failures and anything still retrying after 5 minutes lead the page's
  verdict (outbox.outgoing).
- get_alert_history (MCP) adds `email`: the same delivery state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

steward/templates/alerts/_history_page.html

@@ -48,11 +48,20 @@
           <span class="badge badge-yellow" title="A maintenance window muted this notification">muted · {{ ep.muted_by }}</span>
           {% if ep.released %}<div class="cell-note">notified when the window ended</div>{% endif %}
           {% endif %}
-          {% for label, sent, error in ep.deliveries %}
+          {# Each channel's outcome; an email through the outbox (#5509) also
+             says when it sends, or how its retries stand, or why it failed. #}
+          {% for d in ep.deliveries %}
           <div>
-            {% if sent %}<span class="badge badge-green">{{ label }} sent</span>
-            {% else %}<span class="badge badge-red">{{ label }} failed</span>
-            {% if error %}<div class="cell-note" style="max-width:28rem;overflow-wrap:anywhere;">{{ error }}</div>{% endif %}
+            {% if d.state == "sent" %}<span class="badge badge-green">{{ d.label }} sent</span>
+            {%- if d.at %} <span class="tone-dim">{{ d.at|when }}</span>{% endif %}
+            {% elif d.state == "held" %}<span class="badge badge-yellow">{{ d.label }} gathering</span>
+            <div class="cell-note">waiting for other alerts; sends {{ d.at|when }}</div>
+            {% elif d.state == "sending" %}<span class="badge badge-yellow">{{ d.label }} sending</span>
+            {% elif d.state == "retrying" %}<span class="badge badge-yellow">{{ d.label }} retrying</span>
+            <div class="cell-note" style="max-width:28rem;overflow-wrap:anywhere;">attempt {{ d.attempts }}
+              {%- if d.at %}, next {{ d.at|when }}{% endif %}{% if d.error %}: {{ d.error }}{% endif %}</div>
+            {% else %}<span class="badge badge-red">{{ d.label }} failed</span>
+            {% if d.error %}<div class="cell-note" style="max-width:28rem;overflow-wrap:anywhere;">{{ d.error }}</div>{% endif %}
             {% endif %}
           </div>
           {% endfor %}

steward/templates/settings/system.html

@@ -9,8 +9,8 @@
 {% block breadcrumb %}{{ crumbs([("Settings", "/settings/"), ("System", "")]) }}{% endblock %}
 {% block content %}
 <p class="page-verdict">
-  {%- if not seen %}<span class="tone-dim">No readings yet: the first arrive within a minute of Steward starting.</span>
-  {%- elif problems %}{% set said = problems|join("; ") %}<span class="tone-warn">{{ said[:1]|upper }}{{ said[1:] }}.</span>
+  {%- if problems %}{% set said = problems|join("; ") %}<span class="tone-warn">{{ said[:1]|upper }}{{ said[1:] }}.</span>
+  {%- elif not seen %}<span class="tone-dim">No readings yet: the first arrive within a minute of Steward starting.</span>
   {%- else %}<span class="tone-ok">Steward is running normally.</span>{% endif -%}
 </p>
 
@@ -39,6 +39,46 @@
   Recorded each minute (the cleanup each hour). Any of these can be watched by an
   <a href="/alerts/rules/new">alert rule</a>: pick the source "Steward itself".</p>
 
+{# Outgoing email (#5509): every email waits in the outbox until the mail
+   server takes it. What hasn't gone yet, failures first, then the longest
+   retrying; sent mail is only counted. #}
+<h2 class="section-title" style="margin:2rem 0 0.5rem;">Outgoing email</h2>
+<p style="margin:0 0 1rem;">
+  {%- if not email.rows %}Nothing waiting to send. {% endif -%}
+  {{ "{:,}".format(email.sent) }} sent in the last 7 days.</p>
+{% if email.rows %}
+<div class="card-flush">
+  <table class="table">
+    <thead><tr><th>To</th><th>What</th><th>State</th><th>Queued</th></tr></thead>
+    <tbody>
+      {% for r in email.rows %}
+      <tr>
+        <td class="cell-name"><span title="{{ r.recipient }}">{{ r.recipient }}</span></td>
+        <td class="nowrap">{{ kind_labels.get(r.kind, r.kind) }}</td>
+        <td>
+          {%- if r.state == "failed" %}<span class="badge badge-red">failed</span>
+          {%- elif r.state == "pending" and r.attempts %}<span class="badge badge-yellow">retrying</span>
+            <span class="tone-dim">attempt {{ r.attempts }}{% if r.next_attempt_at %}, next {{ r.next_attempt_at|when }}{% endif %}</span>
+          {%- elif r.state == "held" %}<span class="badge badge-yellow">gathering</span>
+            <span class="tone-dim">sends {{ r.send_after|when }}</span>
+          {%- else %}<span class="badge badge-yellow">sending</span>{% endif %}
+          {%- if r.last_error %}<div class="cell-note" style="max-width:32rem;overflow-wrap:anywhere;">{{ r.last_error }}</div>{% endif -%}
+        </td>
+        <td class="nowrap">{{ r.created_at|ago }}</td>
+      </tr>
+      {% endfor %}
+      {% if email.more %}
+      <tr><td colspan="4" class="tone-dim">+{{ email.more }} more not yet sent</td></tr>
+      {% endif %}
+    </tbody>
+  </table>
+</div>
+{% endif %}
+<p class="field-hint" style="margin-top:0.75rem;">
+  An email the server refuses is failed at once; one it can't take yet is tried
+  again for up to an hour. The mail server is set under
+  <a href="/settings/notifications/">Notifications</a>.</p>
+
 {# Storage (#5469): Postgres's own figures for each table, then what the
    hourly cleanup last removed. Read from the catalog, never by counting. #}
 <h2 class="section-title" style="margin:2rem 0 0.5rem;">Storage</h2>