Commit Graph

2 Commits

Author SHA1 Message Date
bvandeusen 25e1e098fb fix(activity): record external.* TaskRun.queue as download, not default (#883)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m15s
celery_signals._queue_for is a hand-maintained mirror of task_routes that
stamps TaskRun.queue in the prerun signal. It was missing the
backend.app.tasks.external. prefix, so external fetches recorded
queue='default' even though celery routes external.* → download and runs
them on the download worker. The dashboard's per-queue filters and the
per-queue recovery-sweep threshold therefore missed them — the same
'queue column lies default' gap the 2026-06-02 audit fixed for
backup/admin/library_audit.

Map external.* → download in _queue_for. Composes with the fetch_external_link
task-name sweep override (#883), which wins by precedence regardless of the
recorded queue. Pinned test asserts the mirror agrees with the actual route.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 21:05:26 -04:00
bvandeusen c217009425 feat(maintenance): dedicated maintenance_long lane for long one-shot tasks
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m3s
Even chunked, a single concurrency-1 maintenance lane is fragile — a 30-min DB
backup or a multi-chunk library audit holds the slot and delays the quick
self-healing recovery sweeps / vacuum (operator-flagged 2026-06-07: long runs
must never block quick maintenance).

Route the long one-shots — backup.*, admin.* (normalize/re-extract/cascade-
delete), library_audit.* — to a new `maintenance_long` queue served by a
dedicated worker (concurrency 1), added to docker-compose (+ dev override). The
scheduler keeps the quick `maintenance` lane (sweeps, vacuum, cleanup) for
itself, so a backup can no longer starve a 5-min vacuum. UI queue list +
routing tests updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 09:00:03 -04:00