fix: the provenance panel hotlinked the CDN for images already on disk (3965)
CI / lint (push) Successful in 8s
CI / extension-version (push) Successful in 9s
Build images / sign-extension (push) Successful in 9s
Build images / build-agent (push) Successful in 17s
CI / frontend-build (push) Successful in 35s
CI / backend-lint-and-test (push) Successful in 1m18s
Build images / build-web (push) Successful in 1m22s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m19s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m38s
CI / lint (push) Successful in 8s
CI / extension-version (push) Successful in 9s
Build images / sign-extension (push) Successful in 9s
Build images / build-agent (push) Successful in 17s
CI / frontend-build (push) Successful in 35s
CI / backend-lint-and-test (push) Successful in 1m18s
Build images / build-web (push) Successful in 1m22s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m19s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m38s
Two surfaces render a post's HTML body. get_post did _localize_inline_images(sanitize_post_html(...)); provenance_service._post_dict called sanitize_post_html alone. So opening the Provenance panel fetched images from Patreon's CDN for files FC had already downloaded - the archive reaching out to the platform to display what it had archived, which is the thing 830 Phase 2 set out to stop. The same bodies also break when a CDN URL expires or a post is removed, while the identical local copy sits unused. The fix is the shape, not the call. Those were two separately-callable halves and only the first looked mandatory, so a second caller was always going to do half of it. render_post_body in the new services/post_body.py is the whole pipeline in one call, and sanitizing-without-localizing is no longer a reachable operation. _localize_inline_images moves there verbatim; post_feed_service loses five imports that went with it. _post_dict becomes async and takes the session. Both call sites are already inside async methods, so for_image's list comprehension awaits per entry - fine, because localization issues ZERO queries for a body with no inline <img>, which is most of them. Recorded that early exit in the module so the loop isn't "optimized" into a batch without a measurement. Deliberately NOT fixed: provenance still names the same columns url/title/date where the feed says post_url/post_title/post_date, and its description_translated is full text where the feed truncates to DESCRIPTION_LIMIT. That is 3965's wider half - a breaking payload change for ProvenancePanel with no second reason to spend it today. Noted in _post_dict's docstring so the next reader knows it was seen and left. Four regression tests on the provenance path, covering both entry points plus the two refusals the feed already pins: an uncaptured image stays hotlinked (a broken local path is worse than an intact remote one), and a filehash owned by another artist never leaks in. Reverting render_post_body to a bare sanitize fails the first two. Recorded as snippet 3968. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
This commit is contained in:
@@ -270,3 +270,97 @@ async def test_for_post_includes_attachments(db):
|
||||
assert att["download_url"].endswith(
|
||||
f"/api/attachments/{att['id']}/download"
|
||||
)
|
||||
|
||||
|
||||
# --- #3965: the body pipeline is shared with the post detail view ----------
|
||||
#
|
||||
# These are the regression guards for the defect: provenance used to call
|
||||
# `sanitize_post_html` alone, so it served CDN-hotlinking bodies while
|
||||
# `get_post` served local copies of the very same images. Reverting
|
||||
# `render_post_body` back to a bare sanitize fails both.
|
||||
|
||||
|
||||
async def _seed_localizable(db, *, slug, filehash):
|
||||
"""A post whose body inlines a CDN image FC has already downloaded."""
|
||||
cdn = f"https://cdn.test/p/{filehash}/inline.png"
|
||||
artist, source, post = await _seed_post(
|
||||
db, artist_name=slug.title(), slug=slug, platform="patreon",
|
||||
ext_id=f"{slug}-1", title="T", desc=f'<p>x</p><img src="{cdn}">',
|
||||
)
|
||||
db.add(ImageRecord(
|
||||
path=f"/images/{slug}/inline.png", sha256=f"{post.id:064d}",
|
||||
size_bytes=10, mime="image/png", width=10, height=10,
|
||||
origin="downloaded", primary_post_id=post.id, artist_id=artist.id,
|
||||
source_url=cdn, source_filehash=filehash,
|
||||
))
|
||||
await db.flush()
|
||||
return source, post, cdn
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_for_post_serves_our_own_copy_of_an_inline_image(db):
|
||||
"""The archive must not fetch from the platform to show what it archived."""
|
||||
_source, post, cdn = await _seed_localizable(
|
||||
db, slug="localpost", filehash="0123456789abcdef0123456789abcdef",
|
||||
)
|
||||
payload = await ProvenanceService(db).for_post(post.id)
|
||||
html = payload["post"]["description_html"]
|
||||
assert 'src="/images/localpost/inline.png"' in html
|
||||
assert cdn not in html, "provenance body still hotlinks the platform CDN"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_for_image_serves_our_own_copy_of_an_inline_image(db):
|
||||
"""`for_image` shapes N posts, not one — the loop localizes every entry."""
|
||||
rec = await _seed_image(db, sha="b" + "0" * 63)
|
||||
_source, post, cdn = await _seed_localizable(
|
||||
db, slug="localimg", filehash="abcdef0123456789abcdef0123456789",
|
||||
)
|
||||
db.add(ImageProvenance(image_record_id=rec.id, post_id=post.id,
|
||||
source_id=post.source_id))
|
||||
await db.flush()
|
||||
|
||||
payload = await ProvenanceService(db).for_image(rec.id)
|
||||
html = payload["provenance"][0]["post"]["description_html"]
|
||||
assert 'src="/images/localimg/inline.png"' in html
|
||||
assert cdn not in html
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_uncaptured_inline_image_is_left_hotlinking(db):
|
||||
"""Localization only redirects what we actually hold. No stored image means
|
||||
the original src stands — a broken local path would be worse than a
|
||||
hotlink, and this is the behaviour `get_post` already has."""
|
||||
_artist, _source, post = await _seed_post(
|
||||
db, artist_name="Nope", slug="nope", platform="patreon",
|
||||
ext_id="nope-1", desc='<img src="https://cdn.test/p/'
|
||||
'ffffffffffffffffffffffffffffffff/gone.png">',
|
||||
)
|
||||
payload = await ProvenanceService(db).for_post(post.id)
|
||||
assert "cdn.test" in payload["post"]["description_html"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_localization_is_artist_scoped_in_provenance_too(db):
|
||||
"""A filehash owned by a DIFFERENT artist must not leak in — the same
|
||||
scoping `get_post` enforces, asserted on this path so the shared helper
|
||||
cannot be loosened for one caller without failing for the other."""
|
||||
filehash = "cdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcd"
|
||||
cdn = f"https://cdn.test/p/{filehash}/x.png"
|
||||
_a1, _s1, post = await _seed_post(
|
||||
db, artist_name="Owner", slug="owner-prov", platform="patreon",
|
||||
ext_id="own-1", desc=f'<img src="{cdn}">',
|
||||
)
|
||||
other = Artist(name="Other", slug="other-prov")
|
||||
db.add(other)
|
||||
await db.flush()
|
||||
db.add(ImageRecord(
|
||||
path="/images/other-prov/x.png", sha256=f"{post.id:064d}",
|
||||
size_bytes=10, mime="image/png", width=10, height=10,
|
||||
origin="downloaded", artist_id=other.id,
|
||||
source_url=cdn, source_filehash=filehash,
|
||||
))
|
||||
await db.flush()
|
||||
|
||||
payload = await ProvenanceService(db).for_post(post.id)
|
||||
assert cdn in payload["post"]["description_html"]
|
||||
|
||||
Reference in New Issue
Block a user