The Dashboard's "Retry All Failed" button used to fetch every non-superseded
failure, then fire one retry HTTP call per row. Two missing guards:
- No per-source dedup — a source with N historical failures enqueued N jobs
for the same URL.
- No "recent success" check — a source that succeeded two hours ago could
still have stale un-superseded failures from last week that got re-queued.
Adds POST /api/downloads/retry-failed-bulk:
• Selects non-superseded failures, ordered (source_id ASC, created_at DESC)
• Per-source dedup keeps only the most recent failure per source
• Skips sources whose last_success is within the configurable recent window
(default 24h, via recent_window_hours body param)
• Resets + enqueues in a single DB transaction, single Celery dispatch loop
• Returns a per-reason skip breakdown: duplicate_source, recent_success,
no_source (orphaned rows)
The Dashboard button now calls the bulk endpoint and surfaces the full
skip breakdown in the toast, so the user knows exactly what was (and
wasn't) queued. Logic is extracted into a pure _plan_bulk_retry helper
and unit-tested against SimpleNamespace fixtures — 7 new tests covering
the dedup, window, and orphan paths.
Backend foundation for the upcoming Downloads modal refresh. Adds three
helpers on GalleryDLService:
_compute_run_stats exit_code + downloaded/skipped/per-item-failure/
warning counts, derived from stdout+stderr
_extract_errors_warnings filters stderr to just [error]/[warning] lines
so the modal can show a concise errors-only view
_truncate_log caps a log string (default 500KB) with head+tail+
elision marker, preventing verbose multi-hour runs
from bloating the download_sessions JSONB
Wires all three into download.metadata_ in the download task. No schema
change — everything nests into the existing metadata JSONB column.
Follow-up PR will consume these fields in the Downloads.vue modal.
gallery-dl logs `[download][error] Failed to download <file>.part` when a
single post attachment's media URL expires or is deleted, then recovers
and proceeds with the rest of the run. Our error categorizer was letting
a urllib3 debug line like `"HEAD /media-u/v3/<id> HTTP/1.1" 404 0` match
the NOT_FOUND_PATTERNS list, flagging the whole source as deleted even
though gallery-dl kept downloading subsequent items successfully.
Adds "failed to download" to the per-item allowlist so the mask flips
when skip activity is present, matching the shape of the campaign-ID
fix. Total-failure runs still classify as failures.
Also renames the Download Details modal's "Error Log" panel to
"Verbose Log (stderr)" — gallery-dl's `-v` flag sends all logging
(including [debug] lines) to stderr, so the original label was
misleading. Drops the blanket red text color for the same reason.
When gallery-dl logs "[patreon][error] Failed to extract campaign ID"
but then recovers via its /cw/<vanity> fallback and skips every file
(all already archived), _categorize_error was flagging the [error] line
as fatal and returning UNKNOWN_ERROR. Add "failed to extract campaign
id" to the per-item error allowlist alongside "not allowed to view" and
"unable to get post" so skip detection proceeds and the run is
classified NO_NEW_CONTENT.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The anyio_backend fixture is only consumed by pytest-anyio, which is not a
dependency of this project (we use pytest-asyncio with asyncio_mode=auto).
Also drop the unused pytest import from test_sanity.py.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
bvandeusen