bvandeusen
19b962f1a7
The ml-worker's ONLY processing role is now the CPU whole-image embed fallback (tag_and_embed renamed embed_image — Camie tagging was retired #1189 and the name kept implying otherwise; videos were already handled agent-style: frame sampling + mean-pool). Detection/cropping/CCIP stay GPU-agent-only, and their completion is judged per-pipeline: ccip by gpu_job rows, siglip by concept regions at the current model version — never by image_record.siglip_embedding. A CPU embed therefore can NEVER close crop work for the agent (regression test pins this; only the whole-image 'embed' job, the same artifact, is satisfied). Making removal actually safe (operator will drop the container): - GPU-queue coordination (enqueue_gpu_backfill, recover_orphaned_gpu_jobs, reprocess_gpu_jobs) moved verbatim to tasks/gpu_queue.py on the maintenance quick lane — it lived on the 'ml' queue only by module colocation, which made the ml-worker a hard dependency of the whole agent pipeline. - New ml_settings.cpu_embed_enabled (migration 0074, default ON so agent-less installs keep working): OFF stops the four import hooks queueing embed work nothing will consume and no-ops the manual backfill; switch lives on the renamed 'CPU embedding backfill' card. - NB heads training / auto-apply still run on the ml image (sklearn) — a stack that removes the container gives those up too. Deploy note: in-flight messages under the old task names are dropped by the new workers; the 60s orphan sweep + hourly backfill re-fire under the new names immediately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
85 lines
2.9 KiB
Vue
85 lines
2.9 KiB
Vue
<template>
|
|
<MaintenanceTile
|
|
icon="mdi-refresh"
|
|
title="CPU embedding backfill"
|
|
blurb="Whole-image embeddings without a GPU agent — the built-in fallback."
|
|
:open="busy"
|
|
>
|
|
<p class="text-body-2 mb-3">
|
|
Computes the whole-image SigLIP embedding for anything missing one —
|
|
images directly, videos by sampling frames (the same approach as the
|
|
GPU agent). Runs on the ml-worker's CPU, so search, similarity and
|
|
head suggestions work <strong>without</strong> a GPU agent; new imports
|
|
are embedded this way automatically. Detection, cropping and character
|
|
(CCIP) embeddings are GPU-agent-only. Safe to re-run. To re-embed under
|
|
a NEW model, use the GPU agent's "Re-embed library" instead.
|
|
</p>
|
|
<v-switch
|
|
v-model="enabled" color="accent" hide-details density="compact"
|
|
:loading="saving" label="CPU embedding enabled"
|
|
class="mb-1" @update:model-value="onToggle"
|
|
/>
|
|
<p class="fc-muted text-caption mb-3">
|
|
Turn OFF if you run the GPU agent and removed the ml-worker container —
|
|
imports then stop queueing CPU embed work nothing will consume (the
|
|
daily GPU embed backfill covers those images instead).
|
|
</p>
|
|
<v-btn
|
|
color="primary" rounded="pill" :loading="busy" :disabled="!enabled"
|
|
@click="run"
|
|
>
|
|
<v-icon start>mdi-refresh</v-icon> Run backfill now
|
|
</v-btn>
|
|
<span v-if="done" class="ml-3 text-caption">Enqueued.</span>
|
|
<QueueStatusBar queue="ml" queue-label="ML" />
|
|
</MaintenanceTile>
|
|
</template>
|
|
|
|
<script setup>
|
|
import { toast } from '../../utils/toast.js'
|
|
import { onMounted, ref } from 'vue'
|
|
import { useMLStore } from '../../stores/ml.js'
|
|
import MaintenanceTile from '../common/MaintenanceTile.vue'
|
|
import QueueStatusBar from './QueueStatusBar.vue'
|
|
const store = useMLStore()
|
|
const busy = ref(false)
|
|
const done = ref(false)
|
|
const enabled = ref(true)
|
|
const saving = ref(false)
|
|
onMounted(async () => {
|
|
try {
|
|
await store.loadSettings()
|
|
if (store.settings?.cpu_embed_enabled != null) {
|
|
enabled.value = store.settings.cpu_embed_enabled
|
|
}
|
|
} catch { /* non-fatal */ }
|
|
})
|
|
async function onToggle() {
|
|
saving.value = true
|
|
try {
|
|
await store.patchSettings({ cpu_embed_enabled: enabled.value })
|
|
toast({
|
|
text: enabled.value
|
|
? 'CPU embedding on — imports queue embeds for the ml-worker'
|
|
: 'CPU embedding off — the GPU embed backfill owns whole-image embeds',
|
|
type: 'success',
|
|
})
|
|
} catch (e) {
|
|
toast({ text: `Could not save: ${e.message}`, type: 'error' })
|
|
enabled.value = !enabled.value
|
|
} finally {
|
|
saving.value = false
|
|
}
|
|
}
|
|
async function run() {
|
|
busy.value = true
|
|
try { await store.triggerBackfill(); done.value = true }
|
|
catch (e) { toast({ text: e.message, type: 'error' }) }
|
|
finally { busy.value = false }
|
|
}
|
|
</script>
|
|
|
|
<style scoped>
|
|
.fc-muted { color: rgb(var(--v-theme-on-surface-variant)); }
|
|
</style>
|