fix(maintenance): time-box + self-resume the archive re-extract task
reextract_archive_attachments loaded ALL PostAttachments and ran in one pass up to a 30-min soft limit, then died without re-enqueueing — a large archive backlog would only ever partially process. And a naive re-run can't advance: an already-extracted archive is still an archive on disk, so it'd re-extract the same first batch forever. Give it a real cursor + time-box + self-resume (mirrors normalize_tags_task, operator-asked 2026-06-07: reasonable timeout, then re-queue so other work keeps flowing): - service scans attachments with id > after_id in ascending order, time-boxes the chunk, and reports partial=True + resume_after_id (last scanned id). - task passes a 600s budget and re-enqueues itself from the cursor until the scan is exhausted. Routes on the maintenance_long lane. - This is independent of the maintenance_long lane isolation (already shipped) — that stops long tasks starving the quick maintenance queue; this stops the re-extract itself dying on a big backlog. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -57,6 +57,14 @@ def bulk_delete_images_task(self, *, image_ids: list[int]) -> dict:
|
||||
)
|
||||
|
||||
|
||||
# Time-box one chunk well under the soft limit so a large archive back-catalog
|
||||
# can't run the task into the Celery time limit (or hog the maintenance_long
|
||||
# lane). The task re-enqueues itself with the resume cursor until the scan is
|
||||
# exhausted — mirrors normalize_tags_task (operator-asked 2026-06-07: reasonable
|
||||
# timeout, then re-queue so other work keeps flowing).
|
||||
_REEXTRACT_CHUNK_SECONDS = 600
|
||||
|
||||
|
||||
@celery.task(
|
||||
name="backend.app.tasks.admin.reextract_archive_attachments_task",
|
||||
bind=True,
|
||||
@@ -64,15 +72,30 @@ def bulk_delete_images_task(self, *, image_ids: list[int]) -> dict:
|
||||
retry_backoff=15, retry_backoff_max=180, max_retries=1,
|
||||
soft_time_limit=1800, time_limit=2400, # 30 min / 40 min
|
||||
)
|
||||
def reextract_archive_attachments_task(self) -> dict:
|
||||
def reextract_archive_attachments_task(self, after_id: int = 0) -> dict:
|
||||
"""Wraps cleanup_service.reextract_archive_attachments (#713 part 2):
|
||||
re-extract PostAttachments that are actually archives but were filed
|
||||
opaquely before the magic-byte gate, and link their members to the post."""
|
||||
opaquely before the magic-byte gate, and link their members to the post.
|
||||
|
||||
Time-boxed + self-resuming: scans attachments after ``after_id`` and, on a
|
||||
chunk cut, re-enqueues from where it stopped so a big backlog finishes across
|
||||
chunks instead of dying at the soft limit."""
|
||||
SessionLocal = _sync_session_factory()
|
||||
with SessionLocal() as session:
|
||||
return cleanup_service.reextract_archive_attachments(
|
||||
summary = cleanup_service.reextract_archive_attachments(
|
||||
session, images_root=IMAGES_ROOT,
|
||||
time_budget_seconds=_REEXTRACT_CHUNK_SECONDS, after_id=after_id,
|
||||
)
|
||||
# More attachments past this chunk's cursor — continue in the next.
|
||||
if summary.get("partial") and summary.get("resume_after_id", 0) > after_id:
|
||||
log.info(
|
||||
"reextract chunk done (%d scanned, %d archives, resume after id %s) "
|
||||
"— re-enqueuing to continue",
|
||||
summary.get("scanned", 0), summary.get("archives", 0),
|
||||
summary["resume_after_id"],
|
||||
)
|
||||
reextract_archive_attachments_task.delay(summary["resume_after_id"])
|
||||
return summary
|
||||
|
||||
|
||||
# Time-box one chunk well under the soft limit so a large back-catalog (the
|
||||
|
||||
Reference in New Issue
Block a user