feat(ingest): Recapture mode — re-grab post bodies/links + localize on-disk inline images (#830)
A plain backfill gates post-body capture on the seen-ledger, so a post whose media is already on disk AND whose post key is already seen never gets its body recaptured (operator-flagged: Industrial Lust description missing). Recovery recaptures unconditionally but re-downloads the whole source. New 'recapture' walk mode (4th beside tick/backfill/recovery): bypasses the post-record gate so EVERY post's body + external links are re-captured (detail-fetching empty bodies) WITHOUT re-downloading on-disk media; and surfaces already-present media via a separate non-deleting relink channel so the importer backfills ImageRecord.source_filehash for inline-image localization. - ingest_core: recapture mode + recapture_records gate bypass + relink collect - patreon_downloader: recapture surfaces seen-on-disk as skipped_disk(path), never refetches seen-missing media, still downloads genuinely-new - importer.relink_source_filehash: NULL-only sha256 backfill, never unlinks - download_service: mode derivation + phase-3 relink loop + lifecycle clear - source_service/api: start_recapture + backfill_recapture field + action - frontend: Recapture kebab action + 'Recapturing' badge across SourceActions/ Row/Card/SubscriptionsTab + sources store - tests across ingester/downloader/importer/source_service/api/download_service Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -1005,6 +1005,35 @@ class Importer:
|
||||
return sv.strip()
|
||||
return None
|
||||
|
||||
def relink_source_filehash(
|
||||
self, path: Path, source_url: str, *, artist: Artist | None = None,
|
||||
) -> bool:
|
||||
"""#830 recapture: backfill source_url + source_filehash on an EXISTING
|
||||
on-disk image's ImageRecord (matched by sha256) so its post-body inline
|
||||
`<img src=CDN>` remaps to the local copy at render time — WITHOUT
|
||||
re-downloading and WITHOUT unlinking the file (unlike the import path's
|
||||
duplicate-hash branch). NULL-only, mirroring _apply_sidecar: never
|
||||
clobbers an already-set filehash. Returns True only when a row was
|
||||
updated. No-op when the file is gone, the url has no filehash, no record
|
||||
matches the bytes, or the record already carries a filehash.
|
||||
"""
|
||||
fh = filehash_from_url(source_url)
|
||||
if not fh:
|
||||
return False
|
||||
try:
|
||||
sha = _sha256_of(path)
|
||||
except OSError:
|
||||
return False
|
||||
record = self.session.execute(
|
||||
select(ImageRecord).where(ImageRecord.sha256 == sha)
|
||||
).scalar_one_or_none()
|
||||
if record is None or record.source_filehash is not None:
|
||||
return False
|
||||
record.source_url = source_url
|
||||
record.source_filehash = fh
|
||||
self.session.commit()
|
||||
return True
|
||||
|
||||
def _apply_sidecar(
|
||||
self,
|
||||
record: ImageRecord,
|
||||
|
||||
Reference in New Issue
Block a user