feat(recovery): surgical re-fetch for deep posts via ExternalLink reset
CI / lint (push) Successful in 5s
CI / frontend-build (push) Successful in 47s
CI / backend-lint-and-test (push) Successful in 2m0s
CI / integration (push) Successful in 4m50s

Operator-flagged: the recovered defective files live DEEP in their artists'
back-catalogues — the normal download cadence (by design, via the seen-gates)
will never re-walk them, so recovery's source re-check alone can't bring them
back. The durable per-post handle is the ExternalLink row, which survives the
image delete:

- services/external_links.refetch_links_for_post: reset settled links to
  pending (fresh attempt budget, in-flight left alone) + dispatch their
  fetches; sha-dedupe at import discards payload files that still exist, so
  only the missing file lands.
- recover_defective_image now captures the image's post ids BEFORE the delete
  cascades provenance away and resets those posts' links — future recoveries
  are surgical automatically (response gains links_reset; source re-check
  stays for gallery-dl-native files within walk reach).
- POST /api/admin/posts/refetch-external {external_post_id, source_id?} — the
  manual tool for the three files recovered before this fix existed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
This commit is contained in:
2026-07-02 21:07:21 -04:00
parent b1cfbcc06a
commit aa12a57f97
5 changed files with 213 additions and 11 deletions
+66
View File
@@ -390,6 +390,72 @@ async def test_prune_unused_commit_deletes_and_returns_count(client, db):
assert "byebye" in body["sample_names"]
# --- Tier-A: POST /posts/refetch-external ----------------------------
@pytest.mark.asyncio
async def test_refetch_external_resets_and_dispatches(client, db, monkeypatch):
"""Surgical recovery for deep posts (2026-07-03): settled links reset to
pending with a fresh budget + dispatched; in-flight links untouched."""
from sqlalchemy import select
from backend.app.models import ExternalLink, Post, Source
a = Artist(name="Deep", slug="deep")
db.add(a)
await db.flush()
src = Source(artist_id=a.id, platform="patreon",
url="https://www.patreon.com/deep", enabled=True)
db.add(src)
await db.flush()
post = Post(artist_id=a.id, source_id=src.id, external_post_id="96376640")
db.add(post)
await db.flush()
done = ExternalLink(post_id=post.id, artist_id=a.id, host="mega",
url="https://mega.nz/file/a#k1", status="downloaded",
attempts=1)
dead = ExternalLink(post_id=post.id, artist_id=a.id, host="gdrive",
url="https://drive.google.com/x", status="dead",
attempts=5, last_error="quota")
busy = ExternalLink(post_id=post.id, artist_id=a.id, host="mega",
url="https://mega.nz/file/b#k2", status="downloading",
attempts=1)
db.add_all([done, dead, busy])
await db.flush()
ids = (done.id, dead.id, busy.id)
await db.commit()
fetches = []
monkeypatch.setattr(
"backend.app.tasks.external.fetch_external_link.delay",
lambda lid: fetches.append(lid),
)
resp = await client.post(
"/api/admin/posts/refetch-external",
json={"external_post_id": "96376640"},
)
assert resp.status_code == 200
body = await resp.get_json()
assert list(body["posts"].values()) == [{"links_total": 3, "links_reset": 2}]
assert sorted(fetches) == sorted([ids[0], ids[1]])
rows = dict((await db.execute(
select(ExternalLink.id, ExternalLink.status)
.where(ExternalLink.post_id == post.id)
)).all())
assert rows[ids[0]] == "pending"
assert rows[ids[1]] == "pending"
assert rows[ids[2]] == "downloading" # in-flight left alone
# Unknown post → 404, nothing dispatched.
miss = await client.post(
"/api/admin/posts/refetch-external",
json={"external_post_id": "nope"},
)
assert miss.status_code == 404
# --- Tier-A: POST /posts/prune-bare + /posts/reconcile-duplicates ---
# These two routes share _run_dry_run_op with the tag prunes (DRY pass,
# task #753); cover their apply path + reconcile's source_id passthrough so