feat(maintenance): reconcile duplicate posts (gallery-dl→native unify)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m17s

An artist first downloaded by gallery-dl gets Post rows keyed by the per-
attachment id; a later native walk keys the SAME real post by the post id. They
never dedup (uq_post_source_external_id is on external_post_id) → duplicate post
rows (cheunart: 943→1109). The real post id is recoverable in-DB from
raw_metadata['post_id'] (both eras store the sidecar there).

reconcile_duplicate_posts (cleanup_service): group posts by (source_id, canonical
post_id = raw_metadata.post_id else external_post_id); for each group >1, keep the
row already keyed by the post id (the format the CURRENT native downloader
produces, so future walks dedup and this can't recur), re-point
ImageRecord.primary_post_id / ImageProvenance / PostAttachment / ExternalLink onto
it conflict-safe (drop the loser's row where the keeper already has the equivalent,
per each table's uniqueness), backfill the keeper's empty date/title/body/raw_meta
from a loser, set external_post_id=post_id + derive post_url, delete losers.
IMAGES ARE NOT TOUCHED (content-addressed/deduped already; operator-confirmed).

Preview/apply share find_duplicate_post_groups (rule 93). API
/api/admin/posts/reconcile-duplicates (dry_run→{groups,posts_to_merge,sample};
apply→{groups,merged,sample}; optional source_id). UI: a second section on
PostMaintenanceCard (preview groups+sample → confirm merge). Tests: merge +
metadata backfill + image move, no-op when unique, provenance-collision dedup.

Design: milestone #73. Forensics: note #917. Out of scope (flagged): cheunart vs
Cheunart case-variant artist dirs/rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-17 21:57:21 -04:00
parent 7f6345dccf
commit eff64275fc
5 changed files with 385 additions and 1 deletions
+17
View File
@@ -129,6 +129,22 @@ export const useAdminStore = defineStore('admin', () => {
}
}
// Tier-A: unify duplicate post rows (gallery-dl attachment-id + native post-id
// for the same real post) onto one post-id-keyed keeper. Images untouched.
// Preview/apply parity on the backend; UI previews then confirms.
async function reconcileDuplicatePosts({ dryRun = true, sourceId = null } = {}) {
lastError.value = null
try {
return await api.post(
'/api/admin/posts/reconcile-duplicates',
{ body: { dry_run: dryRun, source_id: sourceId } },
)
} catch (e) {
lastError.value = e.message
throw e
}
}
async function purgeLegacyTags({ dryRun = true } = {}) {
lastError.value = null
try {
@@ -208,6 +224,7 @@ export const useAdminStore = defineStore('admin', () => {
tagUsageCount,
pruneUnusedTags,
pruneBarePosts,
reconcileDuplicatePosts,
purgeLegacyTags,
resetContentTagging,
normalizeTags,