feat(syslog): set-up help on the page, port 5514 published, Docker-hidden senders called out (#6003)

The Syslog page says where to send (this server, its port and protocols) and
where the setting is on Synology, QNAP, TrueNAS, OpenWrt, UniFi and rsyslog,
open until a device sends. A sender whose address is on Steward's own
container network (Swarm ingress publishing hides the real sender) is
flagged, with the host-mode ports to publish instead. compose files and the
Dockerfile publish 5514/udp and /tcp; docs/plugins/syslog.md describes the
plugin and the deploy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Dockerfile

@@ -52,7 +52,7 @@ RUN useradd -m -u 1000 app \
     && mkdir -p /data/ansible/sources /data/ansible/runs /data/ansible/playbooks /data/plugins \
     && chown -R app:app /data /app
 
-EXPOSE 5000
+EXPOSE 5000 5514/udp 5514/tcp
 
 ENTRYPOINT ["/usr/bin/tini", "--", "/entrypoint.sh"]
 CMD ["steward", "--host", "0.0.0.0", "--port", "5000"]

README.md

@@ -53,7 +53,7 @@ cp .env.example .env
 docker compose -f compose.deploy.yml up -d
 ```
 
-Open `http://<server>:5000` and create the admin account on first run.
+Open `http://<server>:5000` and create the admin account on first run. Devices send syslog to `<server>:5514` (UDP or TCP); see `docs/plugins/syslog.md`.
 
 **Update to the latest dev build:**
 
@@ -124,5 +124,6 @@ See `docs/plugins/` for a full plugin development guide.
 | `docs/plugins/writing-a-plugin.md` | Step-by-step plugin development guide |
 | `docs/plugins/traefik.md` | Traefik plugin: config, metrics, alert rule examples |
 | `docs/plugins/prometheus.md` | Prometheus plugin: scraping any /metrics endpoint, how series are stored, the Fabled Archives recipe |
+| `docs/plugins/syslog.md` | Syslog plugin: sources, patterns and alerts, publishing port 5514 (host mode under Swarm) |
 | `docs/plugins/prometheus-widgets.md` | Prometheus widget templates: writing, sharing and copying one in, every block, the format reference |
 | `docs/reference/code-map.md` | Where every key function, model, and route lives in the codebase |

compose.deploy.yml

@@ -16,6 +16,13 @@ services:
     pull_policy: always
     ports:
       - "5000:5000"
+      # Syslog (plugins/syslog): devices send here. Steward runs unprivileged,
+      # so it listens above 1024; publish 514 to it instead ("514:5514/udp")
+      # if a device can't choose its port. Under Docker Swarm use the long form
+      # with `mode: host`: Swarm's default ingress publishing hands every packet
+      # over from one internal address, and devices are told apart by theirs.
+      - "5514:5514/udp"
+      - "5514:5514/tcp"
     networks:
       - backend
     volumes:

docker-compose.yml

@@ -8,6 +8,8 @@ services:
     command: ["steward", "--host", "0.0.0.0", "--port", "5000", "--debug"]
     ports:
       - "5000:5000"
+      - "5514:5514/udp"     # syslog (plugins/syslog)
+      - "5514:5514/tcp"
     volumes:
       - app_data:/data
       # Live-edit core + first-party plugins without rebuilding. PYTHONPATH=/app

docs/plugins/syslog.md

@@ -0,0 +1,67 @@
+# Syslog plugin
+
+Receives syslog from NASes, routers and other devices over UDP and TCP
+(RFC 3164 and RFC 5424). The plugin keeps the lines of the sources you add and
+puts them beside Steward's other readings.
+
+## What it does
+
+- **Sources.** A device that starts sending is *offered* on the Syslog page.
+  Its lines are counted there but not stored until an admin adds it. Ignored
+  senders are still counted. Each added source keeps its lines in a ring:
+  the age limit (`retention_days`, 14 by default) and the size cap
+  (`max_bytes_per_source`) are set in Settings → Plugins → Syslog.
+- **Reading.** Each source has its own page, with lines filtered by severity,
+  app, text and time. A host's page and an SNMP device's page show the newest
+  lines of the sources mapped to them. A source maps to a host by its binding,
+  or, when it has none, by sharing the host's or device's address.
+- **Patterns.** Syslog → Patterns counts each source's lines matching some
+  phrases (case-insensitive) or a severity over a window. The count is written
+  every minute as the metric `lines_matching_<key>` under source module
+  `syslog`, zeros included, so an ordinary alert rule fires on the first match
+  and clears once the window has passed. Ready-made patterns cover disk I/O
+  errors, disk resets, NFS server trouble, filesystem errors, degraded arrays
+  and "errors and worse". "Alert on it" opens the rule form aimed at the count.
+- **Beside the readings.** `get_anomaly` includes the warnings and worse logged
+  by the anomaly's host or SNMP device. Each anomaly row has "Syslog around
+  it". Host metrics charts and SNMP device charts mark each error with a
+  dashed red line.
+- **MCP.** `list_syslog_sources` and `search_syslog`.
+
+## Deploying
+
+Steward runs as an unprivileged user, so it listens on **5514** (UDP and TCP),
+not 514. Publish that port from the container:
+
+```yaml
+# docker compose
+ports:
+  - "5514:5514/udp"
+  - "5514:5514/tcp"
+```
+
+Under **Docker Swarm**, publish it in **host mode**:
+
+```yaml
+ports:
+  - { target: 5514, published: 5514, protocol: udp, mode: host }
+  - { target: 5514, published: 5514, protocol: tcp, mode: host }
+```
+
+Swarm's default (ingress) publishing hands every packet over from an address on
+the ingress network. Every device would then read as one sender, because
+sources are told apart by the address their lines come from. The Syslog page
+flags a sender whose address is on Steward's own container network.
+
+Host mode publishes the port on the node that runs the task, so point devices
+at that node. Keep the service on one node with a placement constraint, so the
+address you give the devices stays right.
+
+To let devices use 514 instead, publish `514:5514` instead of `5514:5514`.
+
+## Pointing a device at it
+
+The Syslog page's "Point a device here" section gives the address and port,
+and where the setting is on common devices (Synology, QNAP, TrueNAS, OpenWrt,
+UniFi, rsyslog). Choose the RFC 5424 / IETF format where it is offered: those
+lines carry their time zone.

plugins/syslog/network.py

@@ -0,0 +1,60 @@
+# plugins/syslog/network.py
+"""Whether a sender's address is really the sender's (#6003).
+
+A source is told apart by the address its lines come from. Docker can hide
+that address: a port published through Swarm's ingress (the default) hands
+every packet to Steward from an address on the ingress network, so every
+device reads as one sender. Such an address lies on one of the container's
+own networks. The page says so beside it and how to fix it: publish the
+port in host mode."""
+from __future__ import annotations
+
+import ipaddress
+import socket
+import struct
+from functools import lru_cache
+
+ROUTES = "/proc/net/route"
+
+
+def subnets_from(route_table: str) -> list[ipaddress.IPv4Network]:
+    """The networks a host is directly on, from /proc/net/route's text: each
+    route with no gateway, the default route left out."""
+    out = []
+    for line in route_table.splitlines()[1:]:
+        fields = line.split()
+        if len(fields) < 8:
+            continue
+        try:
+            dest, gateway, mask = (int(fields[i], 16) for i in (1, 2, 7))
+        except ValueError:
+            continue
+        if dest == 0 or gateway != 0:
+            continue
+        address = socket.inet_ntoa(struct.pack("<I", dest))
+        netmask = socket.inet_ntoa(struct.pack("<I", mask))
+        try:
+            out.append(ipaddress.IPv4Network(f"{address}/{netmask}", strict=False))
+        except ValueError:
+            continue
+    return out
+
+
+@lru_cache(maxsize=1)
+def own_subnets() -> tuple[ipaddress.IPv4Network, ...]:
+    """This container's own networks, read once; none off Linux."""
+    try:
+        with open(ROUTES) as f:
+            return tuple(subnets_from(f.read()))
+    except OSError:
+        return ()
+
+
+def on_own_network(address: str, subnets=None) -> bool:
+    """Whether a sender's address is on one of Steward's own networks: most
+    likely Docker standing in for the real sender."""
+    try:
+        ip = ipaddress.ip_address(address)
+    except ValueError:
+        return False
+    return any(ip in net for net in (own_subnets() if subnets is None else subnets))

plugins/syslog/routes.py

@@ -19,6 +19,7 @@ from steward.models.metrics import PluginMetricLatest
 from steward.models.users import UserRole
 
 from .models import SyslogLine, SyslogPattern, SyslogSource
+from .network import on_own_network
 from .parse import SEVERITIES, facility_name, severity_name
 from .patterns import (PRESETS, PRESETS_BY_KEY, SOURCE_MODULE, WINDOWS, PatternError, key_for, parse_form,
                        resource_of)
@@ -55,6 +56,7 @@ def listener_view(port: int, udp: bool, tcp: bool) -> dict:
 def source_view(s: SyslogSource, hosts: dict, rate: int | None = None) -> dict:
     host = hosts.get(s.host_id) if s.host_id else None
     return {"id": s.id, "address": s.address, "state": s.state, "rate": rate,
+            "docker_address": on_own_network(s.address),
             "name": s.name or s.hostname_seen or s.address,
             "hostname_seen": s.hostname_seen, "host": host,
             "first_seen_at": s.first_seen_at, "last_seen_at": s.last_seen_at,
@@ -89,6 +91,7 @@ async def index():
     return await render_template("syslog/index.html", listener=listener,
                                  sources=views["accepted"], offered=views["offered"],
                                  ignored=views["ignored"], hosts=hosts,
+                                 server=request.host.rsplit(":", 1)[0] if request.host.count(":") == 1 else request.host,
                                  now=datetime.now(timezone.utc))
 
 

plugins/syslog/templates/syslog/index.html

@@ -34,6 +34,38 @@
   {% if listener.senders_full %}<p class="tone-warn" style="margin:0.4rem 0 0;">Steward knows {{ listener.max_senders }} senders, the most it keeps; a new one is counted, not offered{% if listener.unoffered %} ({{ "{:,}".format(listener.unoffered) }} line{{ "s" if listener.unoffered != 1 }} so far){% endif %}. Remove ones you don't need.</p>{% endif %}
 </section>
 
+{% set hidden_senders = (offered + sources)|selectattr("docker_address")|list %}
+{% if hidden_senders %}
+<p class="tone-warn" role="status">{{ hidden_senders|map(attribute="address")|unique|join(", ") }} {{ "is" if hidden_senders|map(attribute="address")|unique|list|length == 1 else "are" }} on Steward's own container network: Docker is most likely standing in for the real devices, so they all read as one sender. Publish the port in host mode, as <a href="#syslog-setup">Point a device here</a> shows.</p>
+{% endif %}
+
+<details class="card" id="syslog-setup"{% if not sources and not offered %} open{% endif %}>
+  <summary class="section-title" style="cursor:pointer;">Point a device here</summary>
+  {% set port = listener.port %}
+  {% set proto = listener.protocols|join(" or ") or "UDP or TCP" %}
+  <p style="margin:0.75rem 0 0.5rem;">In the device's remote syslog settings, send to <span class="mono">{{ server }}</span>, port <span class="mono">{{ port }}</span>, {{ proto }}. If the device can't resolve that name, use the server's IP address. Choose the RFC 5424 (IETF) format where it is offered: its lines carry their zone. Once a device sends, it appears above as sending, and Add keeps its lines.</p>
+  <table class="table" style="width:100%;font-size:0.82rem;">
+    <thead><tr><th>Device</th><th>Where</th></tr></thead>
+    <tbody>
+      <tr><td>Synology (DSM)</td><td>Log Center → Log Sending: send logs to a syslog server, with the server, port, protocol and the IETF format.</td></tr>
+      <tr><td>QNAP (QTS)</td><td>QuLog Center → Log Sender: add a destination with the server, port, protocol and RFC 5424.</td></tr>
+      <tr><td>TrueNAS</td><td>System → Advanced Settings → Syslog: the remote syslog server as <span class="mono">{{ server }}:{{ port }}</span>, and its transport.</td></tr>
+      <tr><td>OpenWrt</td><td>System → System → Logging: the external system log server, its port and protocol.</td></tr>
+      <tr><td>UniFi</td><td>Network settings, under remote logging (named Activity Logging or SIEM server in newer versions): the server and port.</td></tr>
+      <tr><td>Linux (rsyslog)</td><td><span class="mono">*.* @{{ server }}:{{ port }}</span> (UDP) or <span class="mono">*.* @@{{ server }}:{{ port }}</span> (TCP) in <span class="mono">/etc/rsyslog.d/90-steward.conf</span>, then restart rsyslog.</td></tr>
+    </tbody>
+  </table>
+  <h3 class="section-title" style="margin:1rem 0 0.4rem;font-size:0.9rem;">Nothing arriving?</h3>
+  <ul class="field-hint" style="margin:0;padding-left:1.1rem;">
+    <li>Steward's container must publish port {{ port }}. In Docker Swarm publish it in <strong>host mode</strong>: through Swarm's default ingress every device reaches Steward from one internal address and can't be told apart.</li>
+    <li>A firewall on the server, or between the device's network and the server's, may drop it.</li>
+    <li>Ports below 1024, such as syslog's usual 514, can't be opened by Steward. Map 514 to {{ port }} where the port is published, or point devices at {{ port }}.</li>
+  </ul>
+  <pre class="mono" style="margin:0.6rem 0 0;font-size:0.78rem;white-space:pre-wrap;">ports:
+  - { target: {{ port }}, published: {{ port }}, protocol: udp, mode: host }
+  - { target: {{ port }}, published: {{ port }}, protocol: tcp, mode: host }</pre>
+</details>
+
 {% if offered %}
 <section class="card" aria-labelledby="syslog-offered">
   <h2 id="syslog-offered" class="section-title">Sending, not added</h2>
@@ -43,7 +75,7 @@
     <tbody>
     {% for s in offered %}
     <tr>
-      <td><span class="mono">{{ s.address }}</span>{% if s.hostname_seen %}<div class="field-hint">calls itself {{ s.hostname_seen }}</div>{% endif %}</td>
+      <td><span class="mono">{{ s.address }}</span>{% if s.hostname_seen %}<div class="field-hint">calls itself {{ s.hostname_seen }}</div>{% endif %}{% if s.docker_address %}<div class="tone-warn" style="font-size:0.78rem;">Docker's address, not the device's</div>{% endif %}</td>
       <td>{{ "{:,}".format(s.lines_total) }}</td>
       <td>{{ s.last_seen_at|ago }}</td>
       <td class="mono" style="max-width:28rem;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;" title="{{ s.last_line }}"><span class="{{ tone(s.level) }}">{{ s.severity or "" }}</span> {{ s.last_line or "—" }}</td>
@@ -82,7 +114,7 @@
     <tbody>
     {% for s in sources %}
     <tr>
-      <td><a href="/plugins/syslog/sources/{{ s.id }}">{{ s.name }}</a><div class="field-hint mono">{{ s.address }}</div></td>
+      <td><a href="/plugins/syslog/sources/{{ s.id }}">{{ s.name }}</a><div class="field-hint mono">{{ s.address }}</div>{% if s.docker_address %}<div class="tone-warn" style="font-size:0.78rem;">Docker's address, not the device's</div>{% endif %}</td>
       <td>{% if s.host %}<a href="/hosts/{{ s.host.id }}">{{ s.host.name }}</a>{% else %}—{% endif %}</td>
       <td class="num">{{ "{:,}".format(s.rate or 0) }}</td>
       <td>{{ s.last_seen_at|ago }}</td>

tests/plugins/syslog/test_network.py

@@ -0,0 +1,27 @@
+"""Whether a sender's address is Docker's (#6003): the container's own
+networks read from /proc/net/route."""
+import ipaddress
+
+from plugins.syslog.network import on_own_network, subnets_from
+
+# A swarm task on the ingress network (10.0.0.0/24) and an overlay (10.0.5.0/24),
+# with its default route through the bridge gateway 172.18.0.1.
+ROUTE = """Iface	Destination	Gateway 	Flags	RefCnt	Use	Metric	Mask		MTU	Window	IRTT
+eth1	00000000	010012AC	0003	0	0	0	00000000	0	0	0
+eth0	0000000A	00000000	0001	0	0	0	00FFFFFF	0	0	0
+eth2	0005000A	00000000	0001	0	0	0	00FFFFFF	0	0	0
+eth1	000012AC	00000000	0001	0	0	0	0000FFFF	0	0	0
+eth9	nonsense
+"""
+
+
+def test_the_directly_connected_networks_are_read_the_default_route_left_out():
+    assert subnets_from(ROUTE) == [ipaddress.IPv4Network("10.0.0.0/24"), ipaddress.IPv4Network("10.0.5.0/24"),
+                                   ipaddress.IPv4Network("172.18.0.0/16")]
+
+
+def test_a_sender_on_a_container_network_is_flagged_and_a_lan_one_is_not():
+    nets = subnets_from(ROUTE)
+    assert on_own_network("10.0.0.2", nets) and on_own_network("172.18.0.1", nets)
+    assert not on_own_network("192.168.1.20", nets)
+    assert not on_own_network("not an address", nets) and not on_own_network("10.0.0.2", [])

tests/plugins/syslog/test_syslog_page.py

@@ -27,7 +27,7 @@ async def _render(app, role, **ctx):
     from quart import session
     ctx = {"listener": _listener(), "sources": [], "offered": [], "ignored": [],
            "hosts": [NS(id="h1", name="SynNAS", address="192.168.0.7")],
-           "now": datetime.now(timezone.utc), **ctx}
+           "server": "steward.lan", "now": datetime.now(timezone.utc), **ctx}
     async with app.test_request_context("/plugins/syslog/"):
         session["user_role"] = role
         page = await render_plugin_template(app, "syslog", "syslog/index.html", **ctx)
@@ -62,6 +62,23 @@ async def test_a_port_it_cannot_open_is_said(app):
     assert "No sources yet." in page
 
 
+async def test_setup_help_names_this_server_and_opens_until_a_device_sends(app):
+    page = await _render(app, "viewer")
+    assert '<details class="card" id="syslog-setup" open>' in page
+    assert "send to <span class=\"mono\">steward.lan</span>, port <span class=\"mono\">5514</span>, UDP or TCP" in page
+    assert "*.* @@steward.lan:5514" in page and "mode: host" in page
+    added = await _render(app, "viewer", sources=[_source("SynNAS", "accepted")])
+    assert '<details class="card" id="syslog-setup">' in added                 # closed once one sends
+
+
+async def test_a_sender_on_dockers_own_network_is_called_out(app):
+    hidden = {**_source("ingress", "offered"), "address": "10.0.0.2", "docker_address": True}
+    page = await _render(app, "admin", offered=[hidden, {**hidden, "id": "id-2"}])
+    assert "10.0.0.2 is on Steward's own container network" in page
+    assert page.count("Docker's address, not the device's") == 2
+    assert "own container network" not in await _render(app, "admin", offered=[_source("synnas", "offered")])
+
+
 async def test_the_edit_page_has_its_trail_and_a_remove(app):
     from quart import session
     source = NS(id="s1", name="SynNAS", hostname_seen="synnas", address="192.168.0.7", host_id=None)