fix(posts): link duplicate items to every post + prune bare shells
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m20s

The native Patreon backfill flooded the feed with bare 'Post <id>' shells
(1589 for Anduo). Root cause: PostAttachment.sha256 was GLOBALLY unique, so a
non-art file reused across posts only ever linked to the first one, and
_capture_attachment created the Post before that dedup check — leaving later
posts with no image and no attachment. Duplicate IMAGES had the mirror gap:
attach_in_place returned duplicate_hash/duplicate_phash before _apply_sidecar,
so the second post got no provenance row, and the feed only rendered via
primary_post_id (one post per image).

Operator requirement: a duplicate item must show on EVERY post it appears in.
Unify the fix as link-not-suppress:

- importer: on duplicate_hash / duplicate_phash(larger_exists), append an
  image_provenance row for the new post (keep primary on the first). Both the
  download path (attach_in_place) and the filesystem path (_import_media).
- post_feed_service: render thumbnails by image_provenance UNION primary_post_id,
  so a cross-posted image shows on every post (and legacy primary-only images
  still show).
- PostAttachment: per-post uniqueness — drop UNIQUE(sha256), add partial
  UNIQUE(post_id, sha256) + partial UNIQUE(sha256) WHERE post_id IS NULL
  (migration 0043); _capture_attachment dedups per-(post,sha) over the shared
  sha-addressed blob, so no post is left bare.
- cleanup: new prune-bare-posts maintenance action (cleanup_service
  _bare_post_conditions shared by preview/count/delete per preview/apply parity;
  admin endpoint; PostMaintenanceCard). Deletes posts with zero image links
  (primary or provenance) AND zero attachments. Run after the feed fix so a
  hidden provenance link spares the post instead of deleting it.

Tests: dup image shows on both posts; dup attachment shows on both posts; feed
renders provenance-linked duplicates; prune-bare delete-path == preview.

Operator redeploys (migration 0043) then runs the prune to clear the shells.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-08 19:28:33 -04:00
parent df76bc0f58
commit a8f624a0f1
12 changed files with 581 additions and 19 deletions
+37
View File
@@ -11,6 +11,7 @@ from sqlalchemy import select
from backend.app.models import (
Artist,
ImageProvenance,
ImageRecord,
Post,
PostAttachment,
@@ -283,6 +284,42 @@ async def test_scroll_thumbnails_show_first_six_with_overflow_count(db):
assert t["mime"] == "image/jpeg"
@pytest.mark.asyncio
async def test_scroll_thumbnails_include_provenance_linked_duplicates(db):
"""A cross-posted/duplicate image is linked to a second post via
image_provenance (its primary_post_id stays on the first post). The feed
renders by provenance UNION primary, so the image shows on BOTH posts —
not just its primary. Operator requirement 2026-06-08."""
artist = await _seed_artist(db, "alice-prov")
src = await _seed_source(db, artist.id, "patreon", "https://p/alice-prov")
primary_post = await _seed_post(
db, src.id, external_id="PRIMARY", post_date=datetime.now(UTC),
)
other_post = await _seed_post(
db, src.id, external_id="OTHER",
post_date=datetime.now(UTC) + timedelta(minutes=1),
)
img = ImageRecord(
path="/images/dup.jpg", sha256="d" * 64, size_bytes=10,
mime="image/jpeg", width=10, height=10, origin="downloaded",
primary_post_id=primary_post.id, artist_id=artist.id,
)
db.add(img)
await db.flush()
# The duplicate links to the OTHER post via provenance only.
db.add(ImageProvenance(image_record_id=img.id, post_id=other_post.id))
await db.commit()
page = await PostFeedService(db).scroll(cursor=None, limit=10)
by_ext = {it["external_post_id"]: it for it in page["items"]}
assert len(by_ext["PRIMARY"]["thumbnails"]) == 1
assert len(by_ext["OTHER"]["thumbnails"]) == 1 # shown via provenance
assert (
by_ext["PRIMARY"]["thumbnails"][0]["image_id"]
== by_ext["OTHER"]["thumbnails"][0]["image_id"]
)
# --- Attachments composition ---------------------------------------------