feat(ingest): Recapture mode — re-grab post bodies/links + localize on-disk inline images (#830)
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 44s
CI / integration (push) Successful in 3m21s
CI / lint (push) Successful in 4s

A plain backfill gates post-body capture on the seen-ledger, so a post whose
media is already on disk AND whose post key is already seen never gets its body
recaptured (operator-flagged: Industrial Lust description missing). Recovery
recaptures unconditionally but re-downloads the whole source.

New 'recapture' walk mode (4th beside tick/backfill/recovery): bypasses the
post-record gate so EVERY post's body + external links are re-captured
(detail-fetching empty bodies) WITHOUT re-downloading on-disk media; and
surfaces already-present media via a separate non-deleting relink channel so the
importer backfills ImageRecord.source_filehash for inline-image localization.

- ingest_core: recapture mode + recapture_records gate bypass + relink collect
- patreon_downloader: recapture surfaces seen-on-disk as skipped_disk(path),
  never refetches seen-missing media, still downloads genuinely-new
- importer.relink_source_filehash: NULL-only sha256 backfill, never unlinks
- download_service: mode derivation + phase-3 relink loop + lifecycle clear
- source_service/api: start_recapture + backfill_recapture field + action
- frontend: Recapture kebab action + 'Recapturing' badge across SourceActions/
  Row/Card/SubscriptionsTab + sources store
- tests across ingester/downloader/importer/source_service/api/download_service

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-14 20:58:40 -04:00
parent 96c29c370b
commit 65ec29ba9b
18 changed files with 467 additions and 29 deletions
+26
View File
@@ -324,6 +324,32 @@ async def test_backfill_endpoint_recover_arms_bypass(client, artist, db):
assert stopped_body["backfill_bypass_seen"] is False
@pytest.mark.asyncio
async def test_backfill_endpoint_recapture_arms_flag(client, artist, db):
"""#830: action='recapture' arms a backfill that re-grabs post bodies/links +
localizes on-disk inline images; the response exposes backfill_recapture=True
(badge 'Recapturing') and NOT backfill_bypass_seen. Stop clears it."""
src = Source(
artist_id=artist.id, platform="patreon",
url="https://patreon.com/alice-recapture", enabled=True,
)
db.add(src)
await db.commit()
sid = src.id
resp = await client.post(f"/api/sources/{sid}/backfill", json={"action": "recapture"})
assert resp.status_code == 200
body = await resp.get_json()
assert body["backfill_state"] == "running"
assert body["backfill_recapture"] is True
assert body["backfill_bypass_seen"] is False
stopped = await client.post(f"/api/sources/{sid}/backfill", json={"action": "stop"})
stopped_body = await stopped.get_json()
assert stopped_body["backfill_state"] is None
assert stopped_body["backfill_recapture"] is False
# --- Plan #703 #2: pre-flight credential verify on backfill/recover arm -----