revert: remove the placement reconciler — it manufactured the problem it solved
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 59s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m55s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m22s

Milestone #421 built a sweep that compared each image's `artist_id` to the
name of the directory holding its file, and called every mismatch a misplaced
image. It reported 33,789 of 63,605 as wrongly filed. That number described
the comparison, not the library.

What it actually was:

  32,475  (97.1%)  one artist's own folder, spelled differently
                   — Telepurte/ vs telepurte/. Same artist, same art.
     657  ( 2.0%)  loose at the images root
     328  ( 1.0%)  in a folder named after a different artist

And the 1% did not mean what the tool assumed either. `ImageProvenance`
records the post and source every file was downloaded from — the
authoritative answer, which the tool never consulted. Querying it for all 328:

    144  provenance agrees with the record  (move would be right)
     87  provenance agrees with the FOLDER  (the record is wrong; move wrong)
     53  provenance names SEVERAL artists   (no single correct folder)
     41  no provenance at all
      3  agrees with neither

So the sweep would have misfiled or arbitrarily picked for ~41% of the only
set it was really needed for. The system already knew where each file came
from; the tool inferred it from a column and a directory name instead.

Operator, 2026-09-21: *"the current system consistently records where items
are and where they came from this is just complicating something works and
doesn't need fixing."* Correct on both counts.

Removed: the service, the tasks, the model and migration 0099's table, the
/api/cleanup/layout and /placement/* endpoints, the Maintenance card and its
store actions, and the tests. 0100 drops the table (rule #22 — no legacy).

KEPT deliberately, per the operator:
- `utils.paths.canonical_subdir` — new filesystem imports derive their
  directory from the artist's slug, matching what the downloader always did.
  Not part of this tool; removing it would be churn that fixes nothing.
- The 327 files run 1 moved (InsoUwu/ -> insouwu/). Same artist either way,
  and the gallery renders them correctly.
- Everything from #4223 (three-gate dedup, 256-bit pHash) and #4234 (backup
  credential exclusion). Those fixed problems that were actually reported.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
2026-09-21 18:17:13 -04:00
co-authored by Claude Opus 5
parent 2dd9b956d5
commit 11a01a9686
14 changed files with 111 additions and 1850 deletions
@@ -54,7 +54,6 @@
Self-healing and repair: missing files, thumbnails, database upkeep.
</p>
<div class="fc-tile-grid">
<PlacementCard />
<MissingFileRepairCard />
<ThumbnailBackfillCard />
<DbMaintenanceCard />
@@ -81,7 +80,6 @@ import MLBackfillCard from './MLBackfillCard.vue'
import ThumbnailBackfillCard from './ThumbnailBackfillCard.vue'
import ArchiveReextractCard from './ArchiveReextractCard.vue'
import MissingFileRepairCard from './MissingFileRepairCard.vue'
import PlacementCard from './PlacementCard.vue'
import GpuTriageCard from './GpuTriageCard.vue'
import DbMaintenanceCard from './DbMaintenanceCard.vue'
import VideoEmbeddingCard from './VideoEmbeddingCard.vue'
@@ -1,312 +0,0 @@
<template>
<MaintenanceTile
icon="mdi-folder-move"
title="File placement"
blurb="Put every image in its own artist's folder, one artist at a time."
>
<p class="fc-muted text-body-2 mb-4">
The library keeps one folder per artist, named after them. Files written
under older rules can sit in another artist's folder — this moves them
home, updating the record and the file together. Every run can be
reverted, so the safe way to use it is one artist at a time: run it,
look at the gallery, then continue or put it back.
</p>
<v-alert
v-if="error" type="warning" variant="tonal" density="compact" class="mb-3"
>{{ error }}</v-alert>
<!-- Survey -->
<div class="fc-settings-row mb-3">
<v-btn
variant="tonal" rounded="pill" prepend-icon="mdi-magnify"
:loading="surveying" @click="onSurvey"
>Check placement</v-btn>
<span v-if="layout" class="fc-muted text-body-2">
{{ layout.misplaced_rows.toLocaleString() }} of
{{ layout.total_rows.toLocaleString() }} images are in the wrong folder
<template v-if="layout.misplaced_rows">
— across {{ layout.artists.length }} artists
</template>
</span>
</div>
<v-table v-if="layout && layout.artists.length" density="compact" class="mb-2">
<thead>
<tr>
<th>Artist</th>
<th class="text-right">To move</th>
<th>Currently in</th>
<th class="text-right">Plan</th>
</tr>
</thead>
<tbody>
<tr v-for="a in layout.artists" :key="a.artist_id">
<td>{{ a.name }}</td>
<td class="text-right fc-tabular">{{ a.misplaced_rows.toLocaleString() }}</td>
<td class="fc-muted text-body-2">{{ a.stray_dirs.join(', ') }}</td>
<td class="text-right">
<v-btn
size="small" variant="text" :loading="planningId === a.artist_id"
@click="onPlan(a.artist_id)"
>Plan</v-btn>
</td>
</tr>
</tbody>
</v-table>
<v-divider class="my-4" />
<!-- Runs -->
<h3 class="fc-section-title">Runs</h3>
<v-table density="compact">
<thead>
<tr>
<th>When</th>
<th>Scope</th>
<th>Status</th>
<th class="text-right">Planned</th>
<th class="text-right">Moved</th>
<th class="text-right">Refused</th>
<th class="text-right">Actions</th>
</tr>
</thead>
<tbody>
<tr v-for="r in store.placementRuns" :key="r.id">
<td class="fc-tabular" :title="r.started_at">
{{ formatRelative(r.started_at) }}
</td>
<td>{{ artistName(r.artist_id) }}</td>
<td>
<v-icon size="small" :color="statusColor(r.status)">
{{ statusIcon(r.status) }}
</v-icon>
{{ r.status }}
</td>
<td class="text-right fc-tabular">{{ r.planned_count.toLocaleString() }}</td>
<td class="text-right fc-tabular">{{ r.moved_count.toLocaleString() }}</td>
<td class="text-right fc-tabular">
<span :class="r.refused_count ? 'text-warning' : 'fc-muted'">
{{ r.refused_count.toLocaleString() }}
</span>
</td>
<td class="text-right">
<KebabMenu :label="`Actions for placement run ${r.id}`">
<v-list-item title="Review…" @click="onReview(r)" />
<v-list-item
title="Move files" :disabled="r.status !== 'ready' || !r.planned_count"
@click="onApply(r)"
/>
<v-list-item
title="Put back…" :disabled="r.status !== 'applied'"
@click="onRevert(r)"
/>
</KebabMenu>
</td>
</tr>
<tr v-if="!store.placementRuns.length">
<td colspan="7" class="text-center fc-muted py-4">
No runs yet. Check placement above, then plan one artist.
</td>
</tr>
</tbody>
</v-table>
<!-- Review: the plan, which is what gets agreed to -->
<v-dialog v-model="reviewOpen" max-width="900">
<v-card>
<v-card-title>
Run {{ reviewRun?.id }} — {{ reviewRun?.planned_count?.toLocaleString() }} moves
</v-card-title>
<v-card-text>
<p class="fc-muted text-body-2 mb-3">
Showing the first {{ REVIEW_LIMIT }}. Each row moves the file and
its record together; nothing is overwritten.
</p>
<v-table density="compact">
<tbody>
<tr v-for="m in reviewMoves" :key="m.image_id">
<td class="fc-muted text-caption">{{ m.from }}</td>
<td class="text-caption">→ {{ m.to }}</td>
</tr>
</tbody>
</v-table>
<div v-if="reviewRun?.refusals?.length" class="mt-4">
<h4 class="fc-section-title">Refused ({{ reviewRun.refusals.length }})</h4>
<p class="fc-muted text-body-2">
Rows the run declined to touch — the source moved, the
destination was taken, or the record changed since planning.
</p>
<div
v-for="f in reviewRun.refusals.slice(0, REVIEW_LIMIT)" :key="f.image_id"
class="text-caption fc-muted"
>#{{ f.image_id }} — {{ f.reason }}</div>
</div>
</v-card-text>
<v-card-actions>
<v-spacer />
<v-btn variant="text" @click="reviewOpen = false">Close</v-btn>
</v-card-actions>
</v-card>
</v-dialog>
<v-dialog v-model="confirmOpen" max-width="520">
<v-card>
<v-card-title>{{ confirmTitle }}</v-card-title>
<v-card-text class="text-body-2">{{ confirmMessage }}</v-card-text>
<v-card-actions>
<v-spacer />
<v-btn variant="text" @click="confirmOpen = false">Cancel</v-btn>
<v-btn
color="accent" variant="flat" rounded="pill"
@click="confirmOpen = false; onConfirmed()"
>Go ahead</v-btn>
</v-card-actions>
</v-card>
</v-dialog>
</MaintenanceTile>
</template>
<script setup>
import { computed, onMounted, onUnmounted, ref } from 'vue'
import KebabMenu from '../common/KebabMenu.vue'
import MaintenanceTile from '../common/MaintenanceTile.vue'
import { useCleanupStore } from '../../stores/cleanup.js'
import { formatRelative } from '../../utils/date.js'
import { toast } from '../../utils/toast.js'
// A whole-library plan is tens of thousands of moves. The dialog is for
// judging whether the moves look right, which a sample answers — the full
// list is in the run row if anyone needs it.
const REVIEW_LIMIT = 200
const store = useCleanupStore()
const layout = ref(null)
const artists = ref({})
const surveying = ref(false)
const planningId = ref(null)
const error = ref('')
const reviewOpen = ref(false)
const reviewRun = ref(null)
const reviewMoves = computed(() => (reviewRun.value?.moves || []).slice(0, REVIEW_LIMIT))
const confirmOpen = ref(false)
const confirmTitle = ref('')
const confirmMessage = ref('')
let confirmAction = null
let poll = null
function artistName(id) {
if (id === null || id === undefined) return 'Whole library'
return artists.value[id] || `#${id}`
}
function statusIcon(s) {
return {
ready: 'mdi-clipboard-text-outline', applied: 'mdi-check-circle',
reverted: 'mdi-undo-variant', error: 'mdi-close-circle',
running: 'mdi-timer-sand', cancelled: 'mdi-cancel',
}[s] || 'mdi-help-circle'
}
function statusColor(s) {
return {
ready: 'accent', applied: 'success', reverted: 'info',
error: 'error', running: 'warning',
}[s] || 'grey'
}
async function guard(fn) {
error.value = ''
try {
return await fn()
} catch (e) {
error.value = e?.message || String(e)
}
}
async function onSurvey() {
surveying.value = true
try {
await guard(async () => { layout.value = await store.loadLayout() })
} finally {
surveying.value = false
}
}
async function onPlan(artistId) {
planningId.value = artistId
try {
await guard(async () => {
await store.planPlacement(artistId)
toast('Planning the run appears below when it is ready')
await store.loadPlacementRuns()
})
} finally {
planningId.value = null
}
}
async function onReview(run) {
await guard(async () => {
reviewRun.value = await store.getPlacementRun(run.id)
reviewOpen.value = true
})
}
function ask(title, message, action) {
confirmTitle.value = title
confirmMessage.value = message
confirmAction = action
confirmOpen.value = true
}
function onApply(run) {
ask(
'Move these files?',
`${run.planned_count.toLocaleString()} files move into their artist's `
+ 'folder. The record and the file move together, and this run can be '
+ 'put back afterwards.',
async () => {
await store.applyPlacement(run.id)
toast('Moving files — this runs in the background')
await store.loadPlacementRuns()
},
)
}
function onRevert(run) {
ask(
'Put these files back?',
`${run.moved_count.toLocaleString()} files return to where they were `
+ 'before this run.',
async () => {
await store.revertPlacement(run.id)
toast('Putting files back — this runs in the background')
await store.loadPlacementRuns()
},
)
}
async function onConfirmed() {
const action = confirmAction
confirmAction = null
if (action) await guard(action)
}
onMounted(async () => {
await guard(async () => {
artists.value = await store.loadArtistNames()
await store.loadPlacementRuns()
})
// Runs change state in a worker, so the table needs refreshing. The rows
// are server-side, which is why no localStorage resurfacing is needed here
// (useMaintenanceTask's job) — a reload or another machine sees the same
// state because the state IS the row.
poll = setInterval(() => { store.loadPlacementRuns().catch(() => {}) }, 5000)
})
onUnmounted(() => { if (poll) clearInterval(poll) })
</script>
-56
View File
@@ -71,66 +71,10 @@ export const useCleanupStore = defineStore('cleanup', () => {
return await api.post(`/api/cleanup/audit/${id}/cancel`)
}
// --- placement reconciler (milestone #421) --------------------------------
//
// Runs are server-side rows, so the DATABASE is the durable state here —
// no localStorage resurfacing (useMaintenanceTask) is needed. Reload the
// page, open it on another machine, and the run and its status are simply
// there. That also means a plan survives being walked away from for a day.
const placementRuns = ref([])
const layout = ref(null)
// The survey: which rows sit outside their artist's directory. check_disk
// additionally stats every destination (collisions, missing sources) and
// costs one stat per misplaced row over NFS, so it is opt-in.
async function loadLayout(checkDisk = false) {
layout.value = await api.get('/api/cleanup/layout', {
params: checkDisk ? { check_disk: 1 } : {},
})
return layout.value
}
// id -> name, so a run row can say "Conto" instead of "#47". The runs
// endpoint carries artist_id alone: the name belongs to the artist, and
// denormalising it into every run would go stale the moment one is renamed.
async function loadArtistNames() {
const rows = await api.get('/api/artists/names')
return Object.fromEntries((rows || []).map(a => [a.id, a.name]))
}
async function loadPlacementRuns(limit = 25) {
const body = await api.get('/api/cleanup/placement/runs', { params: { limit } })
placementRuns.value = body.runs || []
return placementRuns.value
}
// Detail carries `moves` — the plan the operator reads before agreeing.
async function getPlacementRun(id) {
return await api.get(`/api/cleanup/placement/runs/${id}`)
}
async function planPlacement(artistId = null) {
return await api.post('/api/cleanup/placement/plan', {
body: artistId === null ? {} : { artist_id: artistId },
})
}
async function applyPlacement(id) {
return await api.post(`/api/cleanup/placement/runs/${id}/apply`)
}
async function revertPlacement(id) {
return await api.post(`/api/cleanup/placement/runs/${id}/revert`)
}
return {
defaults, recentRuns,
loadDefaults,
previewMinDim, deleteMinDim,
startAudit, getAudit, loadHistory, latestAuditForRule, applyAudit, cancelAudit,
placementRuns, layout,
loadLayout, loadArtistNames, loadPlacementRuns, getPlacementRun,
planPlacement, applyPlacement, revertPlacement,
}
})