diff --git a/README.md b/README.md index 44ad348..e258252 100644 --- a/README.md +++ b/README.md @@ -124,6 +124,6 @@ See `docs/plugins/` for a full plugin development guide. | `docs/plugins/writing-a-plugin.md` | Step-by-step plugin development guide | | `docs/plugins/traefik.md` | Traefik plugin: config, metrics, alert rule examples | | `docs/plugins/prometheus.md` | Prometheus plugin: scraping any /metrics endpoint, how series are stored, the Fabled Archives recipe | -| `docs/plugins/syslog.md` | Syslog plugin: sources, patterns and alerts, publishing port 5514 (host mode under Swarm) | +| `docs/plugins/syslog.md` | Syslog plugin: sources, patterns and alerts, publishing port 5514 (ingress or host mode under Swarm) | | `docs/plugins/prometheus-widgets.md` | Prometheus widget templates: writing, sharing and copying one in, every block, the format reference | | `docs/reference/code-map.md` | Where every key function, model, and route lives in the codebase | diff --git a/docs/plugins/syslog.md b/docs/plugins/syslog.md index 3e9c3ee..1d32777 100644 --- a/docs/plugins/syslog.md +++ b/docs/plugins/syslog.md @@ -40,7 +40,26 @@ ports: - "5514:5514/tcp" ``` -Under **Docker Swarm**, publish it in **host mode**: +Under **Docker Swarm**, either publishing mode works. + +**Ingress** (Swarm's default) lets Steward move between nodes. Devices send to +any node's address, or to an address shared over the nodes (a keepalived VIP): + +```yaml +ports: + - { target: 5514, published: 5514, protocol: udp } + - { target: 5514, published: 5514, protocol: tcp } +``` + +Through ingress every line arrives from Docker's own address on the ingress +network, so the address no longer tells the devices apart. For such an address +Steward keys the source by the hostname each line carries instead (#6096): each +device is its own source, linked to a host by name. A device must put its +hostname in its lines (NASes and routers do); lines with none at Docker's +address all fold into one source, and the page says so. + +**Host mode** keeps each device's own address, but only the node running the +task answers, so devices must be pointed at that node: ```yaml ports: @@ -48,20 +67,14 @@ ports: - { target: 5514, published: 5514, protocol: tcp, mode: host } ``` -Swarm's default (ingress) publishing hands every packet over from an address on -the ingress network. Every device would then read as one sender, because -sources are told apart by the address their lines come from. The Syslog page -flags a sender whose address is on Steward's own container network. - -Host mode publishes the port on the node that runs the task, so point devices -at that node. Keep the service on one node with a placement constraint, so the -address you give the devices stays right. - To let devices use 514 instead, publish `514:5514` instead of `5514:5514`. ## Pointing a device at it The Syslog page's "Point a device here" section gives the address and port, -and where the setting is on common devices (Synology, QNAP, TrueNAS, OpenWrt, -UniFi, rsyslog). Choose the RFC 5424 / IETF format where it is offered: those -lines carry their time zone. +and where the setting is on common devices (Synology, ASUSTOR, QNAP, TrueNAS, +OpenWrt, UniFi, rsyslog). The address is the plugin's `send_to` setting +(Settings → Plugins → Syslog): Steward's web address is often a name only its +proxy answers, so it isn't guessed. Both formats are read, BSD (RFC 3164) and +IETF (RFC 5424); choose IETF where it is offered, as its lines carry their time +zone. ASUSTOR's ADM offers no choice and sends BSD. diff --git a/plugins/syslog/mcp_tools.py b/plugins/syslog/mcp_tools.py index 51db478..505bd71 100644 --- a/plugins/syslog/mcp_tools.py +++ b/plugins/syslog/mcp_tools.py @@ -25,6 +25,10 @@ async def list_syslog_sources(state: str = "") -> dict: "offered" (sending but not added: counted, not kept), "ignored"; empty for all. + A device whose lines reach Steward through Docker's own address (a port + published on Swarm's ingress) is told apart by the name its lines carry: + `through_docker` is true and `address` is Docker's, shared with the others. + Each gives its address, the host it is shown on, lines received in all (`lines_total`) and, for an added source, in the last hour, when it was last heard and its newest line. search_syslog searches the kept lines. @@ -41,6 +45,7 @@ async def list_syslog_sources(state: str = "") -> dict: hosts = {h.id: h.name for h in (await db.execute(select(Host))).scalars()} return {"sources": [{ "source_id": s.id, "name": source_name(s), "address": s.address, "state": s.state, + "through_docker": bool(s.sender), "host_id": s.host_id, "host_name": hosts.get(s.host_id), "calls_itself": s.hostname_seen or None, "lines_total": s.lines_total, "lines_last_hour": per_hour.get(s.id, 0) if s.state == "accepted" else None, "last_heard": iso(s.last_seen_at), "last_heard_age_s": age_s(s.last_seen_at, now), diff --git a/plugins/syslog/migrations/versions/syslog_003_sender.py b/plugins/syslog/migrations/versions/syslog_003_sender.py new file mode 100644 index 0000000..160d051 --- /dev/null +++ b/plugins/syslog/migrations/versions/syslog_003_sender.py @@ -0,0 +1,30 @@ +"""Syslog: devices that reach Steward through Docker's one address, told apart +by the name in their lines + +Revision ID: syslog_003_sender +Revises: syslog_002_patterns +Create Date: 2026-10-19 +""" +from typing import Sequence, Union + +import sqlalchemy as sa +from alembic import op + +revision: str = "syslog_003_sender" +down_revision: Union[str, None] = "syslog_002_patterns" +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + op.add_column("syslog_sources", + sa.Column("sender", sa.String(255), nullable=False, server_default="")) + op.drop_constraint("syslog_sources_address_key", "syslog_sources", type_="unique") + op.create_unique_constraint("uq_syslog_sources_address_sender", "syslog_sources", ["address", "sender"]) + + +def downgrade() -> None: + op.drop_constraint("uq_syslog_sources_address_sender", "syslog_sources", type_="unique") + op.execute("DELETE FROM syslog_sources WHERE sender <> ''") + op.create_unique_constraint("syslog_sources_address_key", "syslog_sources", ["address"]) + op.drop_column("syslog_sources", "sender") diff --git a/plugins/syslog/models.py b/plugins/syslog/models.py index 4b6a3ee..e078ef6 100644 --- a/plugins/syslog/models.py +++ b/plugins/syslog/models.py @@ -3,7 +3,8 @@ from __future__ import annotations from datetime import datetime, timezone -from sqlalchemy import JSON, BigInteger, Boolean, DateTime, ForeignKey, Index, SmallInteger, String, Text +from sqlalchemy import (JSON, BigInteger, Boolean, DateTime, ForeignKey, Index, SmallInteger, String, Text, + UniqueConstraint) from sqlalchemy.orm import Mapped, mapped_column from steward.core.ids import time_ordered_id @@ -13,6 +14,10 @@ from steward.models.base import Base class SyslogSource(Base): """A device that sends syslog to Steward, one row per sending address. + Through a port Swarm publishes on its ingress (the default), every device + arrives from one address of Docker's own; there a row is a device by the + name its lines carry, `sender` (#6096). Everywhere else `sender` is "". + A sender Steward hasn't been told about is a row too, `state="offered"`: its lines are counted and its newest kept as a sample, so the page can offer it ("1,204 lines from 192.168.0.7: add as a source?"), but none is @@ -20,9 +25,11 @@ class SyslogSource(Base): on; `"ignored"` stops the offer and keeps counting. """ __tablename__ = "syslog_sources" + __table_args__ = (UniqueConstraint("address", "sender", name="uq_syslog_sources_address_sender"),) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=time_ordered_id) - address: Mapped[str] = mapped_column(String(64), nullable=False, unique=True) + address: Mapped[str] = mapped_column(String(64), nullable=False) + sender: Mapped[str] = mapped_column(String(255), nullable=False, default="") state: Mapped[str] = mapped_column(String(10), nullable=False, default="offered") name: Mapped[str] = mapped_column(String(100), nullable=False, default="") host_id: Mapped[str | None] = mapped_column( diff --git a/plugins/syslog/plugin.yaml b/plugins/syslog/plugin.yaml index 5886d17..9c17714 100644 --- a/plugins/syslog/plugin.yaml +++ b/plugins/syslog/plugin.yaml @@ -11,8 +11,11 @@ tags: # Sources are added from the Syslog page (/plugins/syslog/), where a device # that starts sending is offered. Steward runs as an unprivileged user, so -# the port is above 1024: point devices at it, or map 514 to it. +# the port is above 1024: point devices at it, or map 514 to it. send_to is +# the address the page's set-up help tells devices to send to (a swarm node's +# IP, or an address shared over the nodes when the port is on Swarm's ingress). config: + send_to: "" port: 5514 udp: true tcp: true diff --git a/plugins/syslog/receiver.py b/plugins/syslog/receiver.py index 3e1315c..2584b66 100644 --- a/plugins/syslog/receiver.py +++ b/plugins/syslog/receiver.py @@ -27,6 +27,7 @@ from sqlalchemy import func, select from steward.core.scheduler import ScheduledTask +from .network import on_own_network from .parse import MAX_MESSAGE, Message, frames, parse if TYPE_CHECKING: @@ -128,6 +129,14 @@ def settings(app: "Quart") -> tuple[int, bool, bool]: return int(cfg.get("port", 5514)), bool(cfg.get("udp", True)), bool(cfg.get("tcp", True)) +def send_to(app: "Quart") -> str: + """The address the set-up help tells devices to send to, as set in the + plugin's settings: the web address Steward is reached by is often a name + only its proxy answers, so it isn't guessed from that.""" + cfg = (app.config.get("PLUGINS") or {}).get("syslog") or {} + return str(cfg.get("send_to") or "").strip() + + async def close() -> None: if LISTENER._udp_transport is not None: LISTENER._udp_transport.close() @@ -218,32 +227,43 @@ async def _store(app: "Quart", rounds: list[list[Received]]) -> None: await store_lines(session, [line for lines in rounds for line in lines]) +def sender_of(line: Received) -> tuple[str, str]: + """(address, sender) a line is filed under. A line from Docker's own + address (a port published on Swarm's ingress) is any device's, so the + name it carries tells the device; elsewhere the address alone does, and + a device renamed stays one source.""" + if on_own_network(line.address): + return line.address, line.message.hostname[:255] + return line.address, "" + + async def store_lines(session, lines: list[Received]) -> None: from .models import SyslogLine, SyslogSource if not lines: return - by_address: dict[str, list[Received]] = {} + by_sender: dict[tuple[str, str], list[Received]] = {} for line in lines: - by_address.setdefault(line.address, []).append(line) - sources = {s.address: s for s in (await session.execute( - select(SyslogSource).where(SyslogSource.address.in_(list(by_address))))).scalars()} - new = [a for a in by_address if a not in sources] + by_sender.setdefault(sender_of(line), []).append(line) + addresses = list({address for address, _ in by_sender}) + sources = {(s.address, s.sender): s for s in (await session.execute( + select(SyslogSource).where(SyslogSource.address.in_(addresses)))).scalars()} + new = [k for k in by_sender if k not in sources] if new: known = (await session.execute(select(func.count()).select_from(SyslogSource))).scalar_one() - for address in new: + for key in new: if known >= MAX_SENDERS: - LISTENER.unoffered += len(by_address.pop(address)) + LISTENER.unoffered += len(by_sender.pop(key)) continue - first = by_address[address][0] - source = SyslogSource(address=address, first_seen_at=first.at, last_seen_at=first.at) + first = by_sender[key][0] + source = SyslogSource(address=key[0], sender=key[1], first_seen_at=first.at, last_seen_at=first.at) session.add(source) - sources[address] = source + sources[key] = source known += 1 await session.flush() - for address, received in by_address.items(): - source = sources[address] + for key, received in by_sender.items(): + source = sources[key] newest = received[-1] source.lines_total = (source.lines_total or 0) + len(received) source.last_seen_at = max(source.last_seen_at, newest.at) if source.last_seen_at else newest.at diff --git a/plugins/syslog/routes.py b/plugins/syslog/routes.py index 143f3a5..ddfc97e 100644 --- a/plugins/syslog/routes.py +++ b/plugins/syslog/routes.py @@ -23,7 +23,7 @@ from .network import on_own_network from .parse import SEVERITIES, facility_name, severity_name from .patterns import (PRESETS, PRESETS_BY_KEY, SOURCE_MODULE, WINDOWS, PatternError, key_for, parse_form, resource_of) -from .receiver import LISTENER, MAX_SENDERS, settings +from .receiver import LISTENER, MAX_SENDERS, send_to, settings from .service import (AROUND_LINES, AROUND_SEVERITY, MARK_SEVERITY, MAX_MARKS, PANEL_LINES, PANEL_SPAN, SEVERITY_CHOICES, apps_of, filtered, line_filter, lines_between, place_for, rates, shown_on, source_name) @@ -55,7 +55,7 @@ def listener_view(port: int, udp: bool, tcp: bool) -> dict: def source_view(s: SyslogSource, hosts: dict, rate: int | None = None) -> dict: host = hosts.get(s.host_id) if s.host_id else None - return {"id": s.id, "address": s.address, "state": s.state, "rate": rate, + return {"id": s.id, "address": s.address, "sender": s.sender, "state": s.state, "rate": rate, "docker_address": on_own_network(s.address), "name": s.name or s.hostname_seen or s.address, "hostname_seen": s.hostname_seen, "host": host, @@ -66,7 +66,11 @@ def source_view(s: SyslogSource, hosts: dict, rate: int | None = None) -> dict: def suggested_host(source: SyslogSource, hosts: list) -> str | None: - """The host whose address is the sender's, if one is.""" + """The host whose address is the sender's, if one is; for a device that + reaches Steward through Docker's address, the host of the name it gives.""" + if source.sender: + name = source.sender.lower() + return next((h.id for h in hosts if name in (h.name.lower(), (h.address or "").lower())), None) return next((h.id for h in hosts if h.address == source.address), None) @@ -91,7 +95,7 @@ async def index(): return await render_template("syslog/index.html", listener=listener, sources=views["accepted"], offered=views["offered"], ignored=views["ignored"], hosts=hosts, - server=request.host.rsplit(":", 1)[0] if request.host.count(":") == 1 else request.host, + server=send_to(current_app), now=datetime.now(timezone.utc)) diff --git a/plugins/syslog/templates/syslog/index.html b/plugins/syslog/templates/syslog/index.html index 954b0dd..2952819 100644 --- a/plugins/syslog/templates/syslog/index.html +++ b/plugins/syslog/templates/syslog/index.html @@ -34,36 +34,43 @@ {% if listener.senders_full %}
Steward knows {{ listener.max_senders }} senders, the most it keeps; a new one is counted, not offered{% if listener.unoffered %} ({{ "{:,}".format(listener.unoffered) }} line{{ "s" if listener.unoffered != 1 }} so far){% endif %}. Remove ones you don't need.
{% endif %} -{% set hidden_senders = (offered + sources)|selectattr("docker_address")|list %} -{% if hidden_senders %} -{{ hidden_senders|map(attribute="address")|unique|join(", ") }} {{ "is" if hidden_senders|map(attribute="address")|unique|list|length == 1 else "are" }} on Steward's own container network: Docker is most likely standing in for the real devices, so they all read as one sender. Publish the port in host mode, as Point a device here shows.
+{% set unnamed = (offered + sources)|selectattr("docker_address")|rejectattr("sender")|list %} +{% if unnamed %} +Lines from {{ unnamed|map(attribute="address")|unique|join(", ") }} come through Docker's own address and carry no device name, so the devices sending them can't be told apart. Have each device put its hostname in its lines, or publish the port in host mode, as Point a device here shows.
{% endif %}In the device's remote syslog settings, send to {{ server }}, port {{ port }}, {{ proto }}. If the device can't resolve that name, use the server's IP address. Choose the RFC 5424 (IETF) format where it is offered: its lines carry their zone. Once a device sends, it appears above as sending, and Add keeps its lines.
+ {% set to = server or "ADDRESS" %} +In the device's remote syslog settings, send to {% if server %}{{ server }}{% else %}the IP address of a Docker host that publishes the port (ADDRESS below; set it under Settings → Plugins → Syslog, "Send to"){% endif %}, port {{ port }}, {{ proto }}. Either syslog format is read: the BSD one (RFC 3164) or the IETF one (RFC 5424), which is better where a device offers the choice, as its lines carry their zone. Once a device sends, it appears above as sending, and Add keeps its lines.
| Device | Where |
|---|---|
| Synology (DSM) | Log Center → Log Sending: send logs to a syslog server, with the server, port, protocol and the IETF format. |
| Synology (DSM) | Log Center → Log Sending: send logs to a syslog server, with the server, port and protocol. Choose the IETF format if it is offered. |
| ASUSTOR (ADM) | In ADM's system log settings, send logs to a syslog server, with the server, port and protocol. ADM offers no format choice; Steward reads the BSD format it sends. |
| QNAP (QTS) | QuLog Center → Log Sender: add a destination with the server, port, protocol and RFC 5424. |
| TrueNAS | System → Advanced Settings → Syslog: the remote syslog server as {{ server }}:{{ port }}, and its transport. |
| TrueNAS | System → Advanced Settings → Syslog: the remote syslog server as {{ to }}:{{ port }}, and its transport. |
| OpenWrt | System → System → Logging: the external system log server, its port and protocol. |
| UniFi | Network settings, under remote logging (named Activity Logging or SIEM server in newer versions): the server and port. |
| Linux (rsyslog) | *.* @{{ server }}:{{ port }} (UDP) or *.* @@{{ server }}:{{ port }} (TCP) in /etc/rsyslog.d/90-steward.conf, then restart rsyslog. |
| Linux (rsyslog) | *.* @{{ to }}:{{ port }} (UDP) or *.* @@{{ to }}:{{ port }} (TCP) in /etc/rsyslog.d/90-steward.conf, then restart rsyslog. |
ports:
- - { target: {{ port }}, published: {{ port }}, protocol: udp, mode: host }
- - { target: {{ port }}, published: {{ port }}, protocol: tcp, mode: host }
+ - { target: {{ port }}, published: {{ port }}, protocol: udp }
+ - { target: {{ port }}, published: {{ port }}, protocol: tcp }