Merge dev → main: Recapture mode (#842) — re-grab post bodies/links + localize on-disk inline images #103

2026-06-14T21:21:43-04:00

bvandeusen commented

2026-06-14 21:21:43 -04:00

Ships the #830 follow-up (#842) to main / :latest.

New "recapture" walk mode (4th beside tick/backfill/recovery), Patreon-only, ⋮ action "Recapture post text & links" + "Recapturing" badge:

Re-grabs EVERY post's body + external links (detail-fetches empty bodies) — bypasses the seen-ledger post-record gate — without re-downloading on-disk media.
Localizes already-on-disk inline images: backfills ImageRecord.source_filehash via a separate non-deleting relink channel (importer.relink_source_filehash), closing the Phase 2 hotlink limitation for existing files.
Seen-but-missing media is left alone (recovery's job); genuinely-new media still downloads.

Fixes the operator-flagged gap: a plain backfill gated body capture, so posts whose media was already on disk (e.g. Stickyspoodge "Industrial Lust") never got their description.

No migrations. CI green on dev head 65ec29b (run 1081).

After merge: force a real re-pull of :latest (esp. the download worker), then per-Patreon-source ⋮ → "Recapture post text & links".

Ships the #830 follow-up (#842) to `main` / `:latest`. New **"recapture"** walk mode (4th beside tick/backfill/recovery), Patreon-only, ⋮ action **"Recapture post text & links"** + "Recapturing" badge: - Re-grabs EVERY post's body + external links (detail-fetches empty bodies) — bypasses the seen-ledger post-record gate — **without re-downloading on-disk media**. - Localizes already-on-disk inline images: backfills `ImageRecord.source_filehash` via a separate **non-deleting** relink channel (`importer.relink_source_filehash`), closing the Phase 2 hotlink limitation for existing files. - Seen-but-missing media is left alone (recovery's job); genuinely-new media still downloads. Fixes the operator-flagged gap: a plain backfill gated body capture, so posts whose media was already on disk (e.g. Stickyspoodge "Industrial Lust") never got their description. No migrations. CI green on dev head `65ec29b` (run 1081). After merge: force a real re-pull of `:latest` (esp. the download worker), then per-Patreon-source ⋮ → "Recapture post text & links".

bvandeusen added 1 commit 2026-06-14 21:21:43 -04:00

feat(ingest): Recapture mode — re-grab post bodies/links + localize on-disk inline images (#830 )

CI / lint (push) Successful in 4s

Details

CI / frontend-build (push) Successful in 19s

Details

CI / backend-lint-and-test (push) Successful in 44s

Details

CI / integration (push) Successful in 3m21s

Details

65ec29ba9b

A plain backfill gates post-body capture on the seen-ledger, so a post whose
media is already on disk AND whose post key is already seen never gets its body
recaptured (operator-flagged: Industrial Lust description missing). Recovery
recaptures unconditionally but re-downloads the whole source.

New 'recapture' walk mode (4th beside tick/backfill/recovery): bypasses the
post-record gate so EVERY post's body + external links are re-captured
(detail-fetching empty bodies) WITHOUT re-downloading on-disk media; and
surfaces already-present media via a separate non-deleting relink channel so the
importer backfills ImageRecord.source_filehash for inline-image localization.

- ingest_core: recapture mode + recapture_records gate bypass + relink collect
- patreon_downloader: recapture surfaces seen-on-disk as skipped_disk(path),
  never refetches seen-missing media, still downloads genuinely-new
- importer.relink_source_filehash: NULL-only sha256 backfill, never unlinks
- download_service: mode derivation + phase-3 relink loop + lifecycle clear
- source_service/api: start_recapture + backfill_recapture field + action
- frontend: Recapture kebab action + 'Recapturing' badge across SourceActions/
  Row/Card/SubscriptionsTab + sources store
- tests across ingester/downloader/importer/source_service/api/download_service

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

bvandeusen merged commit f5156bd847 into main

2026-06-14 21:21:49 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#103