bvandeusenandClaude Opus 5
2e0f8f8c61
PostAttachment's two FKs are both ON DELETE SET NULL, so a deleted post or artist left the row behind rather than taking it. Nothing ever pruned those rows, and nothing in the repo had ever unlinked a file under the attachment store — so both rows and bytes accumulated permanently, invisible to every existing diagnostic. Why a disk->DB reconciliation rather than a row sweep: the store is sha-addressed and idempotent, so ONE blob backs MANY rows. Deleting a row does not free its blob, and since the artist cascade (#3066) now deletes its attachment rows outright, a freed blob has no DB pointer left to find it by. Walking the store and asking "does any row still reference this sha?" catches orphans from every cause, including ones no future delete path will think to report. Preview and apply share `_orphan_attachment_conditions` (rule 93). The dry-run derives its surviving-sha set by NEGATING that same predicate, so it is honest about blobs the delete would free rather than counting them as still-referenced — the one place this was easy to get backwards, so it has its own parity test. Guards, each with a reason: - A blob is written before its row commits, so a just-stored file legitimately has no referencing row. Files under 6h are never judged — same guard and reasoning as ORPHAN_TEMP_MIN_AGE_HOURS. - `.partial` staging files belong to cleanup_orphaned_temp_files; skipped rather than raced. - The sha is parsed as the first 64 chars, not via Path.stem: store() takes the extension from the source filename, and a URL-encoded basename yields a multi-dot suffix that would make stem eat part of the sha. - A 900s walk budget reports partial=True instead of running to the task's hard limit (rule 89). - TASK_STUCK_THRESHOLD_MINUTES override at 30 (= time_limit 25 + 5). Without it a healthy 20-minute walk is phantom-flagged 'RecoverySweep' at the bare 5-min default — the #883 failure class; its invariant test is mirrored here. Defaults to the safe preview at both the task and the route, unlike the other maintenance triggers: this apply unlinks files. Operator-triggered only, never on a beat. Ships with its UI (rule 27): AttachmentReclaimCard in Cleanup → Duplicates & leftovers, built on the existing useMaintenanceTask/MaintenanceTile shapes, so a run survives navigating away. Surfaces files_failed and partial explicitly, since both change what the numbers mean. Also promotes humanBytes to utils/bytes.js — it was byte-identical in VideoDedupCard and GatedPurgeCard and this card would have been the third copy. The three divergent `formatBytes` helpers are deliberately left alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
108 lines
4.1 KiB
Vue
108 lines
4.1 KiB
Vue
<template>
|
||
<!-- #871: Tier-1 video dedup — collapse same-artist videos that match on
|
||
duration + aspect (the same clip re-encoded / pulled from multiple
|
||
sources). Preview first, then apply (destructive: removes the redundant
|
||
copies, keeping the highest-resolution one). -->
|
||
<MaintenanceTile
|
||
icon="mdi-content-duplicate"
|
||
title="Deduplicate videos"
|
||
blurb="Collapse re-encoded duplicate videos, keeping the best copy."
|
||
destructive
|
||
:open="applying || previewing"
|
||
>
|
||
<p class="text-body-2 mb-3">
|
||
Finds videos of the same artist that are the same content across
|
||
re-encodes (matching duration + aspect ratio) and keeps the
|
||
highest-resolution copy. <strong>Preview</strong> first to see what would
|
||
be removed; <strong>Apply</strong> re-points each post to the kept copy,
|
||
then deletes the redundant videos and their files. The first run also
|
||
computes durations for older videos, so it may take a while.
|
||
</p>
|
||
|
||
<div class="d-flex align-center flex-wrap" style="gap: 12px;">
|
||
<v-btn
|
||
color="primary" variant="tonal" rounded="pill"
|
||
:loading="previewing" :disabled="applying" @click="preview"
|
||
>
|
||
<v-icon start>mdi-magnify</v-icon> Preview
|
||
</v-btn>
|
||
<v-btn
|
||
color="error" rounded="pill"
|
||
:loading="applying"
|
||
:disabled="previewing || !canApply"
|
||
@click="confirmOpen = true"
|
||
>
|
||
<v-icon start>mdi-content-duplicate</v-icon> Apply
|
||
</v-btn>
|
||
</div>
|
||
|
||
<v-alert
|
||
v-if="summary" :type="summaryType" variant="tonal" class="mt-4"
|
||
density="comfortable"
|
||
>
|
||
<span v-if="applied">
|
||
Removed {{ summary.deleted }} redundant video(s) across
|
||
{{ summary.groups }} group(s); re-pointed {{ summary.relinked_posts }}
|
||
post link(s); reclaimed {{ humanBytes(summary.reclaim_bytes) }}.
|
||
</span>
|
||
<span v-else-if="summary.redundant > 0">
|
||
{{ summary.redundant }} redundant video(s) across {{ summary.groups }}
|
||
group(s) — {{ humanBytes(summary.reclaim_bytes) }} reclaimable. Click
|
||
<strong>Apply</strong> to remove them (keeping the best copy).
|
||
</span>
|
||
<span v-else>No duplicate videos found.</span>
|
||
</v-alert>
|
||
|
||
<QueueStatusBar queue="maintenance_long" queue-label="Maintenance" />
|
||
|
||
<v-dialog v-model="confirmOpen" max-width="440">
|
||
<v-card>
|
||
<v-card-title>Remove duplicate videos?</v-card-title>
|
||
<v-card-text class="text-body-2">
|
||
This permanently deletes
|
||
<strong>{{ summary?.redundant ?? 0 }}</strong> redundant video file(s),
|
||
keeping the highest-resolution copy in each group. Posts are re-pointed
|
||
to the kept copy first, so nothing disappears from a post.
|
||
</v-card-text>
|
||
<v-card-actions>
|
||
<v-spacer />
|
||
<v-btn variant="text" @click="confirmOpen = false">Cancel</v-btn>
|
||
<v-btn color="error" @click="apply">Remove duplicates</v-btn>
|
||
</v-card-actions>
|
||
</v-card>
|
||
</v-dialog>
|
||
</MaintenanceTile>
|
||
</template>
|
||
|
||
<script setup>
|
||
import { computed, ref } from 'vue'
|
||
|
||
import { useMaintenanceTask } from '../../composables/useMaintenanceTask.js'
|
||
import { humanBytes } from '../../utils/bytes.js'
|
||
import MaintenanceTile from '../common/MaintenanceTile.vue'
|
||
import QueueStatusBar from './QueueStatusBar.vue'
|
||
|
||
const confirmOpen = ref(false)
|
||
|
||
// Long task (first run re-probes every NULL-duration video) — the lifecycle +
|
||
// resurface-after-navigation live in the shared composable. 150 polls × 2s ≈ 5m.
|
||
const { previewing, applying, summary, applied, preview, apply: applyTask } = useMaintenanceTask({
|
||
endpoint: '/api/admin/maintenance/dedup-videos',
|
||
storageKey: 'fc.maint.dedupVideos',
|
||
appliedToast: 'Duplicate videos removed',
|
||
maxPolls: 150,
|
||
})
|
||
|
||
const canApply = computed(() => !!summary.value && !applied.value && summary.value.redundant > 0)
|
||
const summaryType = computed(() => {
|
||
if (applied.value) return 'success'
|
||
return summary.value && summary.value.redundant > 0 ? 'info' : 'success'
|
||
})
|
||
|
||
// The confirm dialog gates the destructive apply; close it, then run.
|
||
function apply () {
|
||
confirmOpen.value = false
|
||
applyTask()
|
||
}
|
||
</script>
|