fix(posts): link duplicate items to every post + prune bare shells
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m20s

The native Patreon backfill flooded the feed with bare 'Post <id>' shells
(1589 for Anduo). Root cause: PostAttachment.sha256 was GLOBALLY unique, so a
non-art file reused across posts only ever linked to the first one, and
_capture_attachment created the Post before that dedup check — leaving later
posts with no image and no attachment. Duplicate IMAGES had the mirror gap:
attach_in_place returned duplicate_hash/duplicate_phash before _apply_sidecar,
so the second post got no provenance row, and the feed only rendered via
primary_post_id (one post per image).

Operator requirement: a duplicate item must show on EVERY post it appears in.
Unify the fix as link-not-suppress:

- importer: on duplicate_hash / duplicate_phash(larger_exists), append an
  image_provenance row for the new post (keep primary on the first). Both the
  download path (attach_in_place) and the filesystem path (_import_media).
- post_feed_service: render thumbnails by image_provenance UNION primary_post_id,
  so a cross-posted image shows on every post (and legacy primary-only images
  still show).
- PostAttachment: per-post uniqueness — drop UNIQUE(sha256), add partial
  UNIQUE(post_id, sha256) + partial UNIQUE(sha256) WHERE post_id IS NULL
  (migration 0043); _capture_attachment dedups per-(post,sha) over the shared
  sha-addressed blob, so no post is left bare.
- cleanup: new prune-bare-posts maintenance action (cleanup_service
  _bare_post_conditions shared by preview/count/delete per preview/apply parity;
  admin endpoint; PostMaintenanceCard). Deletes posts with zero image links
  (primary or provenance) AND zero attachments. Run after the feed fix so a
  hidden provenance link spares the post instead of deleting it.

Tests: dup image shows on both posts; dup attachment shows on both posts; feed
renders provenance-linked duplicates; prune-bare delete-path == preview.

Operator redeploys (migration 0043) then runs the prune to clear the shells.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-08 19:28:33 -04:00
parent df76bc0f58
commit a8f624a0f1
12 changed files with 581 additions and 19 deletions
+42 -1
View File
@@ -8,7 +8,15 @@ re-read ORM attributes after a service mutates and re-fetches.
import pytest
from sqlalchemy import func, select
from backend.app.models import Artist, ImageRecord, Tag, TagKind
from backend.app.models import (
Artist,
ImageProvenance,
ImageRecord,
Post,
PostAttachment,
Tag,
TagKind,
)
from backend.app.models.series_chapter import SeriesChapter
from backend.app.models.series_page import SeriesPage
from backend.app.models.tag import image_tag
@@ -399,6 +407,39 @@ def test_prune_unused_tags_commit_spares_fandom_and_chaptered_series(
assert "GenuinelyUnused" not in surviving
def test_prune_bare_posts_spares_linked_and_deletes_bare(db_sync, tmp_path):
# The LIVE delete (not just the preview) shares _bare_post_conditions, so a
# post is spared if it has ANY link — a primary image, a provenance (cross-
# posted/duplicate) image, or an attachment — and only the truly-bare shell
# is deleted (the empty-post flood, operator-flagged 2026-06-08).
a = _make_artist(db_sync, slug="pb")
img = _make_image(db_sync, artist=a, path=str(tmp_path / "x.jpg"), sha256="a" * 64)
bare = Post(artist_id=a.id, external_post_id="bare1")
has_primary = Post(artist_id=a.id, external_post_id="prim")
has_prov = Post(artist_id=a.id, external_post_id="prov")
has_att = Post(artist_id=a.id, external_post_id="att")
db_sync.add_all([bare, has_primary, has_prov, has_att])
db_sync.flush()
img.primary_post_id = has_primary.id
db_sync.add(ImageProvenance(image_record_id=img.id, post_id=has_prov.id))
db_sync.add(PostAttachment(
post_id=has_att.id, artist_id=a.id, sha256="b" * 64,
path="/store/b", original_filename="f.pdf", ext=".pdf", size_bytes=1,
))
db_sync.commit()
# dry-run count agrees with the delete: only the 1 truly-bare post.
assert cleanup_service.prune_bare_posts(db_sync, dry_run=True)["count"] == 1
result = cleanup_service.prune_bare_posts(db_sync, dry_run=False)
assert result["deleted"] == 1
surviving = db_sync.execute(select(Post.external_post_id)).scalars().all()
assert "bare1" not in surviving
assert set(surviving) == {"prim", "prov", "att"}
# --- reset_content_tagging ------------------------------------------