feat(ingest): localize inline post-body images to local copies (Phase 2)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m14s

Render a post body faithfully by serving our stored copies of inline
images instead of hotlinking the public CDN. The join key is the CDN
filehash (32-hex MD5) shared between a body <img src> and the media URL
we downloaded (the same identity extract_media dedups by):

- utils.paths.filehash_from_url — one source of truth for the extractor;
  patreon_client._filehash now delegates so capture- and render-time
  hashing cannot drift.
- ImageRecord gains source_url (provenance) + source_filehash (indexed
  match key); migration 0051.
- the per-media sidecar carries the file's source_url; the importer
  persists it (NULL-only) on the ImageRecord via _apply_sidecar.
- post_feed_service.get_post remaps body <img src> -> /images/<path> for
  every inline image whose filehash maps to a stored image of THIS
  artist; unmatched / pre-Phase-2 images keep hotlinking.

Pre-existing on-disk images have no filehash yet, so they fall back to
hotlinking until re-downloaded; localization is forward-looking.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-14 16:39:58 -04:00
parent 5e1655384f
commit 96c29c370b
15 changed files with 350 additions and 22 deletions
+19 -6
View File
@@ -294,7 +294,7 @@ class PatreonDownloader:
path=quarantine_dest, error=reason,
)
self._write_sidecar(post, out_path)
self._write_sidecar(post, out_path, source_url=media.url)
return MediaOutcome(media=media, status="downloaded", path=out_path, error=None)
# -- download seams ----------------------------------------------------
@@ -490,7 +490,9 @@ class PatreonDownloader:
# -- sidecar -----------------------------------------------------------
def _write_sidecar(self, post: dict, media_path: Path) -> Path:
def _write_sidecar(
self, post: dict, media_path: Path, *, source_url: str | None = None
) -> Path:
"""Write the importer-consumed sidecar next to `media_path`.
Patreon uses base-default sidecar keys (parse_sidecar maps
@@ -498,13 +500,22 @@ class PatreonDownloader:
content->description, published_at->post_date, url->post_url). Patreon
registers no derive_post_url, so `url` is trusted as the permalink — we
pass the post's attributes.url.
"""
return self._write_sidecar_data(post, media_path.with_suffix(".json"))
def _write_sidecar_data(self, post: dict, sidecar_path: Path) -> Path:
`source_url` is THIS file's CDN URL (#830 Phase 2). The importer persists
its filehash on the ImageRecord so the post body's inline `<img src>` can
be remapped to the local copy at render time.
"""
return self._write_sidecar_data(
post, media_path.with_suffix(".json"), source_url=source_url
)
def _write_sidecar_data(
self, post: dict, sidecar_path: Path, *, source_url: str | None = None
) -> Path:
"""Serialize the post's metadata to `sidecar_path`. Shared by the
per-media sidecar (next to each file) and the post-only sidecar
(`write_post_record`, for media-less posts)."""
(`write_post_record`, for media-less posts). `source_url` is set only for
the per-media sidecar — a media-less post has no source file."""
attrs = post.get("attributes") or {}
title = attrs.get("title")
content = attrs.get("content")
@@ -530,6 +541,8 @@ class PatreonDownloader:
"published_at": published if isinstance(published, str) else None,
"url": url if isinstance(url, str) else None,
}
if source_url:
data["source_url"] = source_url
sidecar_path.write_text(json.dumps(data, indent=2))
return sidecar_path