feat(ml): drop image_record.tagger_predictions — image_prediction is sole store (#768 step 3)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m14s

Read cutover verified in prod (suggestions + allowlist read image_prediction;
backfill complete at 908k rows / 51k images). Removes the old JSON column and
everything that fed it:

- ImageRecord.tagger_predictions column removed; migration 0046 DROPs it.
  tagger_model_version kept as the "tagged / current?" signal the backfill
  sweep reads (needs-tagging check switched to tagger_model_version IS NULL).
- tag_and_embed no longer dual-writes the JSON — image_prediction is the only
  write path.
- importer re-import reset drops the JSON line (image_prediction rows are
  already deleted on re-import).
- Retired the one-time #768 backfill task + the #764 prune task, their admin
  endpoints, and their Maintenance cards (Backfill/PrunePredictionsCard).
- Tests seed/assert via image_prediction; stale column refs removed.

Disk reclaim is NOT automatic: DROP COLUMN is a catalog change. Run
`VACUUM FULL image_record` off-hours afterward to return the ~100 GB to the OS
so DB backups go small (#739). image_prediction (~90 MB) stays in pg_dump — it's
the source of truth now.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-11 18:52:33 -04:00
parent 65211a3f2f
commit 3610ba495f
17 changed files with 74 additions and 445 deletions
@@ -1,50 +0,0 @@
<template>
<!-- #768: one-time copy of stored tagger predictions from the
image_record.tagger_predictions JSON into the normalized
image_prediction table. Migration 0045 creates the empty table; this
populates it for the existing library. -->
<v-card>
<v-card-title>Backfill normalized predictions</v-card-title>
<v-card-text>
<p class="text-body-2 mb-3">
Copies each image's stored tagger predictions into the new
<code>image_prediction</code> table (the source the suggestions and
allowlist now read from). Run this <strong>once</strong> after the
upgrade so existing images get their suggestions back — newly tagged
images populate it automatically. Batched, resumable and idempotent;
safe to run more than once and to leave running in the background.
</p>
<v-btn color="primary" rounded="pill" :loading="busy" @click="run">
<v-icon start>mdi-database-import-outline</v-icon> Backfill predictions now
</v-btn>
<span v-if="queued" class="ml-3 text-caption text-success">Queued ✓</span>
<QueueStatusBar queue="maintenance_long" queue-label="Maintenance (long)" />
</v-card-text>
</v-card>
</template>
<script setup>
import { ref } from 'vue'
import { useApi } from '../../composables/useApi.js'
import { toast } from '../../utils/toast.js'
import QueueStatusBar from './QueueStatusBar.vue'
const api = useApi()
const busy = ref(false)
const queued = ref(false)
async function run () {
busy.value = true
queued.value = false
try {
await api.post('/api/admin/maintenance/backfill-predictions')
queued.value = true
toast({ text: 'Prediction backfill queued', type: 'success' })
} catch (e) {
toast({ text: e?.body?.detail || e?.message || 'Failed to queue', type: 'error' })
} finally {
busy.value = false
}
}
</script>
@@ -12,8 +12,6 @@
<ThumbnailBackfillCard />
</div>
<MLThresholdSliders class="mt-4" />
<BackfillPredictionsCard class="mt-4" />
<PrunePredictionsCard class="mt-4" />
<AllowlistTable class="mt-4" />
<AliasTable class="mt-4" />
<DbMaintenanceCard class="mt-6" />
@@ -33,8 +31,6 @@ import MLBackfillCard from './MLBackfillCard.vue'
import CentroidRecomputeCard from './CentroidRecomputeCard.vue'
import ThumbnailBackfillCard from './ThumbnailBackfillCard.vue'
import MLThresholdSliders from './MLThresholdSliders.vue'
import BackfillPredictionsCard from './BackfillPredictionsCard.vue'
import PrunePredictionsCard from './PrunePredictionsCard.vue'
import AllowlistTable from './AllowlistTable.vue'
import AliasTable from './AliasTable.vue'
import DbMaintenanceCard from './DbMaintenanceCard.vue'
@@ -1,58 +0,0 @@
<template>
<!-- #764: drop stored tagger predictions below the store floor to shrink
image_record's TOAST (the sub-0.70 score tail had grown it to ~100 GB). -->
<v-card>
<v-card-title>Prune low-confidence predictions</v-card-title>
<v-card-text>
<p class="text-body-2 mb-3">
Removes stored tagger predictions below the current store floor
(<strong>{{ floorPct }}</strong>) from every image, and clamps any
allowlist threshold below the floor up to it. This is what shrinks the
database — the low-confidence tail was the bulk of its size. Idempotent
and resumable; safe to run more than once. Afterward, reclaim the freed
space with <code>VACUUM FULL</code> / <code>pg_repack</code> on
<code>image_record</code>.
</p>
<v-btn color="primary" rounded="pill" :loading="busy" @click="run">
<v-icon start>mdi-database-minus-outline</v-icon> Prune predictions now
</v-btn>
<span v-if="queued" class="ml-3 text-caption text-success">Queued ✓</span>
<QueueStatusBar queue="maintenance_long" queue-label="Maintenance (long)" />
</v-card-text>
</v-card>
</template>
<script setup>
import { computed, onMounted, ref } from 'vue'
import { useApi } from '../../composables/useApi.js'
import { toast } from '../../utils/toast.js'
import { useMLStore } from '../../stores/ml.js'
import QueueStatusBar from './QueueStatusBar.vue'
const api = useApi()
const ml = useMLStore()
const busy = ref(false)
const queued = ref(false)
const floorPct = computed(() => {
const f = ml.settings?.tagger_store_floor
return f == null ? '' : `${Math.round(f * 100)}%`
})
onMounted(() => { if (!ml.settings) ml.loadSettings() })
async function run () {
busy.value = true
queued.value = false
try {
await api.post('/api/admin/maintenance/prune-predictions')
queued.value = true
toast({ text: 'Prediction prune queued', type: 'success' })
} catch (e) {
toast({ text: e?.body?.detail || e?.message || 'Failed to queue', type: 'error' })
} finally {
busy.value = false
}
}
</script>