perf(docker): the hourly log prune ranks only the containers over their cap (#5449)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 1m30s
CI / unit (push) Successful in 1m42s
CI / publish (push) Successful in 1m52s

#5449 "The hourly Docker log prune ranks every log line in the table to find
the few over their cap": the size ring put a running-total window over every
row of docker_logs each hour, sorting the whole table on NFS-backed storage
to find the handful of containers past docker.logs.max_bytes_per_container.

- prune_docker_logs (capability docker.prune_logs) does the age cutoff, then
  a per-container byte total picks the containers at or over the cap, and
  only their rows are ranked and trimmed. Same keep rule as before: the
  newest line always survives.
- run_docker_retention keeps the metrics rollup and the event prune.
- Cleanup runs the log ring in its own transaction after them.

Arose from #5431 "Ingest dropped for ~3 minutes after each deploy while
startup retention runs".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-08 22:31:00 -04:00
co-authored by Claude Opus 5.5
parent 4a33046aef
commit b3a5bbfeff
4 changed files with 111 additions and 57 deletions
+7 -1
View File
@@ -21,7 +21,7 @@ def setup(app: "Quart") -> None:
from steward.core.capabilities import register_capability
from steward.models.users import UserRole
from .ingest import persist_host_docker
from .retention import run_docker_retention
from .retention import prune_docker_logs, run_docker_retention
register_capability(
"docker.persist_host_samples", persist_host_docker,
label="Persist host Docker samples",
@@ -36,6 +36,12 @@ def setup(app: "Quart") -> None:
description="Roll up old docker_metrics to hourly + prune stale metrics/events.",
required_role=UserRole.viewer,
)
register_capability(
"docker.prune_logs", prune_docker_logs,
label="Prune Docker logs",
description="Keep each container's logs within the age window and byte cap.",
required_role=UserRole.viewer,
)
def get_scheduled_tasks() -> list: