feat(import): /api/import/clear-stuck endpoint + Clear stuck UI button — escape hatch for the autoretry-loop case the automatic sweep can't break

Operator hit 3 large PNGs stuck in 'processing' for 2 days 2026-05-25:
the existing recover_interrupted_tasks flips processing > 5min back to
queued + .delay(), but if the underlying file is unfixably broken (e.g.,
PIL OSError, also patched in 68cffce), the loop never terminates and the
'Scanning...' banner sticks at 0/0 forever blocking new scans.

/api/import/clear-stuck:
- Flips every task in pending/queued/processing to 'failed' with a clear
  marker error message
- Finalizes any 'running' ImportBatch that has no remaining active children
- Idempotent + non-destructive: rows survive, can be retried once the
  underlying cause is resolved

UI button 'Clear stuck...' sits next to 'Retry failed' / 'Clear completed'
with a warning-tonal alert in the confirm dialog explaining what it does
and recommending Retry failed once the cause is fixed.

Tests: clears mixed non-terminal states, untouches complete rows,
finalizes orphan batch, no-op when nothing stuck.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-25 12:37:07 -04:00
parent c361032554
commit b6a917ac81
4 changed files with 180 additions and 1 deletions
+77
View File
@@ -120,6 +120,83 @@ async def retry_failed():
return jsonify({"retried": len(failed_ids)})
@import_admin_bp.route("/clear-stuck", methods=["POST"])
async def clear_stuck():
"""Force any non-terminal ImportTask (status in pending/queued/
processing) to 'failed' AND finalize any ImportBatch that ends up
with no active children. Escape hatch for the operator when the
automatic recover_interrupted_tasks sweep keeps re-queueing the
same stuck row forever (e.g., underlying file is genuinely broken
and the import keeps OSError-looping at PIL load).
Idempotent + non-destructive: rows survive as 'failed' so the
Retry-Failed button can re-attempt them once whatever was broken
is fixed. Banked 2026-05-25 — operator hit 3 large PNGs that
autoretry-looped for 2 days after a corrupt-data PIL OSError.
"""
async with get_session() as session:
stuck_ids = (
await session.execute(
select(ImportTask.id).where(
ImportTask.status.in_(["pending", "queued", "processing"])
)
)
).scalars().all()
if stuck_ids:
await session.execute(
update(ImportTask)
.where(ImportTask.id.in_(stuck_ids))
.values(
status="failed",
finished_at=datetime.now(UTC),
error=(
"manually cleared via /api/import/clear-stuck "
"— stuck in non-terminal state; retry once "
"underlying cause (corrupt file, missing model, "
"etc.) is resolved"
),
)
)
# Finalize any 'running' ImportBatch that no longer has any
# active children. The "Scanning..." banner is driven by
# /api/import/status finding a running batch; left untouched,
# it would persist forever after the stuck-task clear.
running_batches = (
await session.execute(
select(ImportBatch.id).where(ImportBatch.status == "running")
)
).scalars().all()
finalized_batches = 0
for batch_id in running_batches:
still_active = (
await session.execute(
select(ImportTask.id)
.where(ImportTask.batch_id == batch_id)
.where(ImportTask.status.in_(
["pending", "queued", "processing"]
))
.limit(1)
)
).scalar_one_or_none()
if still_active is None:
await session.execute(
update(ImportBatch)
.where(ImportBatch.id == batch_id)
.values(
status="complete",
finished_at=datetime.now(UTC),
)
)
finalized_batches += 1
await session.commit()
return jsonify({
"tasks_failed": len(stuck_ids),
"batches_finalized": finalized_batches,
})
@import_admin_bp.route("/clear-completed", methods=["POST"])
async def clear_completed():
body = await request.get_json(silent=True) or {}