feat(ingest): localize inline post-body images to local copies (Phase 2)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m14s

Render a post body faithfully by serving our stored copies of inline
images instead of hotlinking the public CDN. The join key is the CDN
filehash (32-hex MD5) shared between a body <img src> and the media URL
we downloaded (the same identity extract_media dedups by):

- utils.paths.filehash_from_url — one source of truth for the extractor;
  patreon_client._filehash now delegates so capture- and render-time
  hashing cannot drift.
- ImageRecord gains source_url (provenance) + source_filehash (indexed
  match key); migration 0051.
- the per-media sidecar carries the file's source_url; the importer
  persists it (NULL-only) on the ImageRecord via _apply_sidecar.
- post_feed_service.get_post remaps body <img src> -> /images/<path> for
  every inline image whose filehash maps to a stored image of THIS
  artist; unmatched / pre-Phase-2 images keep hotlinking.

Pre-existing on-disk images have no filehash yet, so they fall back to
hotlinking until re-downloaded; localization is forward-looking.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-14 16:39:58 -04:00
parent 5e1655384f
commit 96c29c370b
15 changed files with 350 additions and 22 deletions
+38
View File
@@ -6,6 +6,8 @@ server-side to a tight allowlist. nh3 (Rust/ammonia bindings) is used;
bleach is deprecated upstream and intentionally not used.
"""
import re
import nh3
# A faithful-but-safe set: enough to reproduce the SEMANTIC look of a scraped
@@ -50,3 +52,39 @@ def sanitize_post_html(raw: str | None) -> str | None:
link_rel="noopener noreferrer",
clean_content_tags={"script", "style"},
)
# nh3 emits attributes double-quoted, so a single `src="..."` capture is enough
# to read/rewrite inline-image sources on already-sanitized markup (#830 Phase 2).
_IMG_SRC_RE = re.compile(r'(<img\b[^>]*?\bsrc=")([^"]*)(")', re.IGNORECASE)
def extract_img_srcs(html: str | None) -> list[str]:
"""The raw `src` values of every <img> in `html`, in document order, deduped
(first kept). Returned EXACTLY as they appear (still entity-escaped) so a
caller can match them back for verbatim substitution."""
if not html:
return []
out: list[str] = []
seen: set[str] = set()
for _pre, src, _post in _IMG_SRC_RE.findall(html):
if src and src not in seen:
seen.add(src)
out.append(src)
return out
def rewrite_img_srcs(html: str | None, replace: dict[str, str]) -> str | None:
"""Swap each <img src> whose raw (as-in-HTML) value is a key in `replace`
with the mapped value; others are left untouched. Used to point a post body's
inline images at locally-served copies (#830 Phase 2). Operates only on src
values of already-sanitized markup, so the result stays within the allowlist.
Returns `html` unchanged when it or `replace` is empty."""
if not html or not replace:
return html
def _sub(m: re.Match) -> str:
pre, src, post = m.group(1), m.group(2), m.group(3)
return f"{pre}{replace.get(src, src)}{post}"
return _IMG_SRC_RE.sub(_sub, html)