feat(ingest): localize inline post-body images to local copies (Phase 2)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m14s

Render a post body faithfully by serving our stored copies of inline
images instead of hotlinking the public CDN. The join key is the CDN
filehash (32-hex MD5) shared between a body <img src> and the media URL
we downloaded (the same identity extract_media dedups by):

- utils.paths.filehash_from_url — one source of truth for the extractor;
  patreon_client._filehash now delegates so capture- and render-time
  hashing cannot drift.
- ImageRecord gains source_url (provenance) + source_filehash (indexed
  match key); migration 0051.
- the per-media sidecar carries the file's source_url; the importer
  persists it (NULL-only) on the ImageRecord via _apply_sidecar.
- post_feed_service.get_post remaps body <img src> -> /images/<path> for
  every inline image whose filehash maps to a stored image of THIS
  artist; unmatched / pre-Phase-2 images keep hotlinking.

Pre-existing on-disk images have no filehash yet, so they fall back to
hotlinking until re-downloaded; localization is forward-looking.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-14 16:39:58 -04:00
parent 5e1655384f
commit 96c29c370b
15 changed files with 350 additions and 22 deletions
+19
View File
@@ -1,9 +1,28 @@
"""Filesystem path helpers — destination derivation, hash-suffixed names."""
import re
from pathlib import Path
_MAX_EXT_LEN = 16
# A Patreon/gallery CDN URL embeds a 32-char hex (MD5) path segment that is the
# file's stable per-file identity — the same role gallery-dl's `_filehash`
# plays. It is the join key between a post body `<img src=CDN>` and the local
# copy we downloaded (extract_media dedups content vs gallery images by it), so
# this ONE extractor must be used for both capture-time persistence and
# render-time matching — they cannot be allowed to drift. Match the FIRST 32-hex
# run anywhere in the URL (path or query); real CDN URLs carry exactly one.
_FILEHASH_RE = re.compile(r"([0-9a-fA-F]{32})")
def filehash_from_url(url: str | None) -> str | None:
"""The 32-char hex (MD5) CDN identity segment of `url`, lowercased, or None
when the URL is empty / carries no such segment."""
if not url:
return None
match = _FILEHASH_RE.search(url)
return match.group(1).lower() if match else None
def safe_ext(name: str | Path) -> str:
"""Conservatively extract a short, alphanumeric file extension.