bvandeusen/FabledCurator
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 12 Wiki Activity

fix(posts): link duplicate items to every post + prune bare shells #88

Merged
bvandeusen merged 1 commits from dev into main 2026-06-08 19:42:31 -04:00
Conversation 0 Commits 1 Files Changed 12
+581 -19
bvandeusen commented 2026-06-08 19:42:26 -04:00
Owner
Copy Link
Copy Source

Why

The native Patreon backfill flooded the Posts feed with bare "Post <id> / (no description) / No images attached" shells — 1589 for one creator. Two related gaps:

  1. PostAttachment.sha256 was globally unique, so a non-art file reused across posts only ever linked to the first post; _capture_attachment created the Post before that dedup check → bare shells (no image, no attachment).
  2. Duplicate images short-circuited in attach_in_place before _apply_sidecar, so the second post got no image_provenance row; and the feed rendered images only via image_record.primary_post_id (one post per image).

What

Operator requirement — a duplicate item must show on every post it appears in — made this a link-not-suppress fix:

  • Images: on duplicate_hash / duplicate_phash(larger_exists), append an image_provenance row for the new post (primary stays on the first). Both the download path (attach_in_place) and filesystem path (_import_media).
  • Feed: post_feed_service renders thumbnails by image_provenance ∪ primary_post_id, so cross-posted images show on every post (legacy primary-only images still show).
  • Attachments: per-post uniqueness — drop UNIQUE(sha256), add partial UNIQUE(post_id, sha256) + partial UNIQUE(sha256) WHERE post_id IS NULL (migration 0043); _capture_attachment dedups per-(post, sha) over the shared sha-addressed blob.
  • Cleanup: new prune-bare-posts maintenance action — cleanup_service._bare_post_conditions() shared by preview/count/delete (preview/apply parity), /api/admin/posts/prune-bare endpoint, and a Post-maintenance card in the Cleanup view. Deletes only posts with zero image links (primary or provenance) and zero attachments.

Notes

  • Operator redeploys (migration 0043), then runs the prune to clear existing shells. Run order matters: the feed fix resurrects any shell that had a hidden provenance link, so the prune only removes the genuinely empty ones.
  • Tests: dup image on both posts; dup attachment on both posts; feed renders provenance-linked duplicates; prune-bare delete-path == preview.

🤖 Generated with Claude Code

## Why The native Patreon backfill flooded the Posts feed with bare "Post &lt;id&gt; / (no description) / No images attached" shells — 1589 for one creator. Two related gaps: 1. `PostAttachment.sha256` was **globally** unique, so a non-art file reused across posts only ever linked to the first post; `_capture_attachment` created the Post *before* that dedup check → bare shells (no image, no attachment). 2. Duplicate **images** short-circuited in `attach_in_place` before `_apply_sidecar`, so the second post got no `image_provenance` row; and the feed rendered images only via `image_record.primary_post_id` (one post per image). ## What Operator requirement — *a duplicate item must show on every post it appears in* — made this a link-not-suppress fix: - **Images**: on `duplicate_hash` / `duplicate_phash(larger_exists)`, append an `image_provenance` row for the new post (primary stays on the first). Both the download path (`attach_in_place`) and filesystem path (`_import_media`). - **Feed**: `post_feed_service` renders thumbnails by `image_provenance ∪ primary_post_id`, so cross-posted images show on every post (legacy primary-only images still show). - **Attachments**: per-post uniqueness — drop `UNIQUE(sha256)`, add partial `UNIQUE(post_id, sha256)` + partial `UNIQUE(sha256) WHERE post_id IS NULL` (migration 0043); `_capture_attachment` dedups per-(post, sha) over the shared sha-addressed blob. - **Cleanup**: new prune-bare-posts maintenance action — `cleanup_service._bare_post_conditions()` shared by preview/count/delete (preview/apply parity), `/api/admin/posts/prune-bare` endpoint, and a Post-maintenance card in the Cleanup view. Deletes only posts with zero image links (primary or provenance) **and** zero attachments. ## Notes - Operator redeploys (migration 0043), then runs the prune to clear existing shells. Run order matters: the feed fix resurrects any shell that had a hidden provenance link, so the prune only removes the genuinely empty ones. - Tests: dup image on both posts; dup attachment on both posts; feed renders provenance-linked duplicates; prune-bare delete-path == preview. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
bvandeusen added 1 commit 2026-06-08 19:42:26 -04:00
fix(posts): link duplicate items to every post + prune bare shells
CI / lint (push) Successful in 2s
Details
CI / frontend-build (push) Successful in 26s
Details
CI / backend-lint-and-test (push) Successful in 30s
Details
CI / integration (push) Successful in 3m20s
Details
a8f624a0f1 
The native Patreon backfill flooded the feed with bare 'Post <id>' shells
(1589 for Anduo). Root cause: PostAttachment.sha256 was GLOBALLY unique, so a
non-art file reused across posts only ever linked to the first one, and
_capture_attachment created the Post before that dedup check — leaving later
posts with no image and no attachment. Duplicate IMAGES had the mirror gap:
attach_in_place returned duplicate_hash/duplicate_phash before _apply_sidecar,
so the second post got no provenance row, and the feed only rendered via
primary_post_id (one post per image).

Operator requirement: a duplicate item must show on EVERY post it appears in.
Unify the fix as link-not-suppress:

- importer: on duplicate_hash / duplicate_phash(larger_exists), append an
  image_provenance row for the new post (keep primary on the first). Both the
  download path (attach_in_place) and the filesystem path (_import_media).
- post_feed_service: render thumbnails by image_provenance UNION primary_post_id,
  so a cross-posted image shows on every post (and legacy primary-only images
  still show).
- PostAttachment: per-post uniqueness — drop UNIQUE(sha256), add partial
  UNIQUE(post_id, sha256) + partial UNIQUE(sha256) WHERE post_id IS NULL
  (migration 0043); _capture_attachment dedups per-(post,sha) over the shared
  sha-addressed blob, so no post is left bare.
- cleanup: new prune-bare-posts maintenance action (cleanup_service
  _bare_post_conditions shared by preview/count/delete per preview/apply parity;
  admin endpoint; PostMaintenanceCard). Deletes posts with zero image links
  (primary or provenance) AND zero attachments. Run after the feed fix so a
  hidden provenance link spares the post instead of deleting it.

Tests: dup image shows on both posts; dup attachment shows on both posts; feed
renders provenance-linked duplicates; prune-bare delete-path == preview.

Operator redeploys (migration 0043) then runs the prune to clear the shells.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bvandeusen merged commit 4c6406ee18 into main 2026-06-08 19:42:31 -04:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
renovate-bot bvandeusen
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#88