#6718 "A swarm container's metrics, normal and anomalies are keyed by its task
name, so they start over whenever the task moves". Live, the move took
VDNT-Docker02's I/O pressure (psi_io_full_avg60) from about 1% to about 23%,
and passes of 5000 raw samples outran their 15 s slot. Each moved sample
rewrites its row and every index on it.
Batches are 1000 samples, and after a pass the task waits four times as long
as it took, so it holds the disk for about a fifth of the time and the move
slows if the disk does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#6718 "A swarm container's metrics, normal and anomalies are keyed by its task
name, so they start over whenever the task moves". Live, the move found 391,440
keys, about 2,500 of them with raw samples, and took one key per 15 s pass:
about 68 days, in sorted order, so recent history could wait months while
charts and normals read thin.
A pass now runs for up to 2 s, and keys with raw samples go first. Old hourly
keys take a few statements each. A key with no anomalies no longer rewrites the
service's timeline entries. The start logs how many keys each table holds, as
that count is far higher than expected and not explained yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#6718 "A swarm container's metrics, normal and anomalies are keyed by its task
name, so they start over whenever the task moves". The operator: "the metrics
should be keys to the service name not the task name... as that's what's
consistent between updates."
CPU, memory, restart count and health now go under "swarm/<service>", so a
normal, its anomalies and an alert rule carry across a redeploy or a move
between nodes. Network rates stay "<host>/<service>", the task id gone, as they
answer who on a host is sending. Tasks of a multi-task service share the key,
so its level reads as an average across them (of 51 live services, only
authentik-worker and portainer_agent run more than one task).
History under task names moves in a background task, 5000 raw samples every
15 s, not a migration: the raw table holds about 2M such rows and the app
would wait on it at boot. Hourly rows combine where two tasks share an hour,
and rows still to review merge to one per metric.
A service's anomalies no longer reach a host's page, as it has no one host;
get_anomaly reads what happened around one from the service's own events.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>