feat(import-resilience L2): one-shot re-download for corrupt downloaded files

Layer 2 — remediate a corrupt file by re-fetching a fresh copy from its
source, bounded to a single attempt. Operator-requested 2026-05-28.

New backend/app/services/refetch_service.py:
- resolve_refetch_source: parse the failed file's sidecar → platform,
  derive the artist from the import path, find an ENABLED Source with a
  real feed URL for (artist, platform). Returns None for filesystem-only
  imports, missing sidecars, or `sidecar:<platform>:<slug>` synthetic
  anchors (not pollable).
- attempt_refetch: if not already refetched AND a Source resolves,
  delete the corrupt file (so gallery-dl's skip_existing re-fetches it),
  set ImportTask.refetched=True, and trigger ONE download_source
  re-check. Bounded by `refetched` so source-side corruption can't loop.

Wiring:
- Manual endpoint POST /api/import/tasks/<id>/refetch (only on 'failed'
  tasks). Returns refetch_queued / no_source / already_refetched /
  not_found / not_failed.
- Auto path in recover_interrupted_tasks: for each poison-pill row, if
  env FC_AUTO_REFETCH_CORRUPT=1, attempt_refetch (default OFF — the
  manual button is the primary path; auto is opt-in since re-fetch
  deletes a file + re-runs the downloader).
- Frontend: a cloud-refresh icon button on failed rows in ImportTaskList
  → stores.import.refetchTask → toast keyed on the result status.

Filesystem imports with no upstream return no_source — the operator's
only remediation there is replacing the file on disk, surfaced clearly
in the toast.

Tests: 404 unknown task, 400 non-failed task, no_source when
unresolvable, and the full resolvable-source path (file deleted,
refetched flag set, one download_source dispatched, second call is a
no-op). The resolvable test repoints the migration-seeded
import_settings(id=1) scan path rather than inserting a conflicting row.
This commit is contained in:
2026-05-28 00:08:03 -04:00
parent e3cdd0f92b
commit dcfe55d731
6 changed files with 320 additions and 2 deletions
+37
View File
@@ -128,6 +128,43 @@ async def retry_failed():
return jsonify({"retried": len(failed)})
@import_admin_bp.route("/tasks/<int:task_id>/refetch", methods=["POST"])
async def refetch_task(task_id: int):
"""Layer-2 one-shot re-download: delete the (corrupt) file behind a
failed import task and re-run its source's downloader to fetch a
fresh copy. Only works for files that resolve to an enabled,
real-URL subscription Source; filesystem-only imports return
no_source.
Returns one of: refetch_queued (+source_id) / no_source /
already_refetched / not_found / not_failed.
"""
async with get_session() as session:
result = await session.run_sync(_refetch_task_sync, task_id)
if result["status"] == "not_found":
return jsonify(result), 404
if result["status"] == "not_failed":
return jsonify(result), 400
return jsonify(result)
def _refetch_task_sync(session, task_id: int) -> dict:
from pathlib import Path
from ..models import ImportSettings
from ..services.refetch_service import attempt_refetch
task = session.get(ImportTask, task_id)
if task is None:
return {"status": "not_found"}
if task.status != "failed":
return {"status": "not_failed"}
settings = session.execute(
select(ImportSettings).where(ImportSettings.id == 1)
).scalar_one()
return attempt_refetch(session, task, Path(settings.import_scan_path))
@import_admin_bp.route("/clear-stuck", methods=["POST"])
async def clear_stuck():
"""Force any non-terminal ImportTask (status in pending/queued/