bvandeusen
aa12a57f97
Operator-flagged: the recovered defective files live DEEP in their artists'
back-catalogues — the normal download cadence (by design, via the seen-gates)
will never re-walk them, so recovery's source re-check alone can't bring them
back. The durable per-post handle is the ExternalLink row, which survives the
image delete:
- services/external_links.refetch_links_for_post: reset settled links to
pending (fresh attempt budget, in-flight left alone) + dispatch their
fetches; sha-dedupe at import discards payload files that still exist, so
only the missing file lands.
- recover_defective_image now captures the image's post ids BEFORE the delete
cascades provenance away and resets those posts' links — future recoveries
are surgical automatically (response gains links_reset; source re-check
stays for gallery-dl-native files within walk reach).
- POST /api/admin/posts/refetch-external {external_post_id, source_id?} — the
manual tool for the three files recovered before this fix existed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
53 lines
2.1 KiB
Python
53 lines
2.1 KiB
Python
"""Surgical re-fetch of a post's external file-host links.
|
|
|
|
The normal download cadence never re-walks deep back-catalogue posts (the
|
|
seen-gates exist precisely to keep old items from resurfacing), so when a
|
|
file that CAME from an ExternalLink is deleted — e.g. the failure-triage
|
|
recovery flow removing a corrupt original — a plain source re-check will
|
|
never bring it back. The link ROW is the durable, per-post handle: resetting
|
|
it to pending and dispatching the fetch re-downloads exactly that link's
|
|
payload, and sha-dedupe at import discards anything that still exists — so
|
|
only the missing file actually lands. (Operator 2026-07-03: recovery must
|
|
not require artist-wide deep scans.)
|
|
"""
|
|
import logging
|
|
|
|
from sqlalchemy import select
|
|
from sqlalchemy.orm import Session
|
|
|
|
from ..models import ExternalLink
|
|
|
|
log = logging.getLogger(__name__)
|
|
|
|
|
|
def refetch_links_for_post(session: Session, post_id: int) -> dict:
|
|
"""Reset every settled ExternalLink on a post to pending (fresh attempt
|
|
budget) and dispatch its fetch. In-flight ('downloading') links are left
|
|
alone. Commits. Returns {links_total, links_reset}."""
|
|
links = session.execute(
|
|
select(ExternalLink).where(ExternalLink.post_id == post_id)
|
|
).scalars().all()
|
|
reset_ids = []
|
|
for link in links:
|
|
if link.status == "downloading":
|
|
continue
|
|
link.status = "pending"
|
|
link.attempts = 0
|
|
link.last_error = None
|
|
link.completed_at = None
|
|
reset_ids.append(link.id)
|
|
session.commit()
|
|
if reset_ids:
|
|
# Lazy import (services -> tasks would cycle at module load). The
|
|
# 10-min extdl sweep would pick pending rows up anyway — dispatching
|
|
# directly just skips the wait.
|
|
from ..tasks.external import fetch_external_link
|
|
|
|
for lid in reset_ids:
|
|
fetch_external_link.delay(lid)
|
|
log.info(
|
|
"external refetch: post %s — %d/%d link(s) reset + dispatched",
|
|
post_id, len(reset_ids), len(links),
|
|
)
|
|
return {"links_total": len(links), "links_reset": len(reset_ids)}
|