feat(ingest): Recapture mode — re-grab post bodies/links + localize on-disk inline images (#830)
A plain backfill gates post-body capture on the seen-ledger, so a post whose media is already on disk AND whose post key is already seen never gets its body recaptured (operator-flagged: Industrial Lust description missing). Recovery recaptures unconditionally but re-downloads the whole source. New 'recapture' walk mode (4th beside tick/backfill/recovery): bypasses the post-record gate so EVERY post's body + external links are re-captured (detail-fetching empty bodies) WITHOUT re-downloading on-disk media; and surfaces already-present media via a separate non-deleting relink channel so the importer backfills ImageRecord.source_filehash for inline-image localization. - ingest_core: recapture mode + recapture_records gate bypass + relink collect - patreon_downloader: recapture surfaces seen-on-disk as skipped_disk(path), never refetches seen-missing media, still downloads genuinely-new - importer.relink_source_filehash: NULL-only sha256 backfill, never unlinks - download_service: mode derivation + phase-3 relink loop + lifecycle clear - source_service/api: start_recapture + backfill_recapture field + action - frontend: Recapture kebab action + 'Recapturing' badge across SourceActions/ Row/Card/SubscriptionsTab + sources store - tests across ingester/downloader/importer/source_service/api/download_service Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -324,6 +324,32 @@ async def test_backfill_endpoint_recover_arms_bypass(client, artist, db):
|
||||
assert stopped_body["backfill_bypass_seen"] is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_backfill_endpoint_recapture_arms_flag(client, artist, db):
|
||||
"""#830: action='recapture' arms a backfill that re-grabs post bodies/links +
|
||||
localizes on-disk inline images; the response exposes backfill_recapture=True
|
||||
(badge 'Recapturing') and NOT backfill_bypass_seen. Stop clears it."""
|
||||
src = Source(
|
||||
artist_id=artist.id, platform="patreon",
|
||||
url="https://patreon.com/alice-recapture", enabled=True,
|
||||
)
|
||||
db.add(src)
|
||||
await db.commit()
|
||||
sid = src.id
|
||||
|
||||
resp = await client.post(f"/api/sources/{sid}/backfill", json={"action": "recapture"})
|
||||
assert resp.status_code == 200
|
||||
body = await resp.get_json()
|
||||
assert body["backfill_state"] == "running"
|
||||
assert body["backfill_recapture"] is True
|
||||
assert body["backfill_bypass_seen"] is False
|
||||
|
||||
stopped = await client.post(f"/api/sources/{sid}/backfill", json={"action": "stop"})
|
||||
stopped_body = await stopped.get_json()
|
||||
assert stopped_body["backfill_state"] is None
|
||||
assert stopped_body["backfill_recapture"] is False
|
||||
|
||||
|
||||
# --- Plan #703 #2: pre-flight credential verify on backfill/recover arm -----
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user