feat(ml): operator model swap — GPU re-embed + embedder as a setting (#1190)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m33s

Make the SigLIP embedder an operator choice (drop-in to SigLIP 2:
google/siglip2-so400m-patch16-512 is a verified 1152-d model at 512px → no
schema change, better small-cue fidelity). A swap = set model + re-embed +
retrain, all operator-driven; the GPU agent does the re-embed so it's fast.

- settings: embedder_model_name is now a setting (migration 0065) alongside the
  existing embedder_model_version; both editable + validated (non-empty) in the
  ml admin API. The server embedder loads by HF name (AutoImageProcessor/Model,
  model-agnostic), preferring the pre-downloaded local dir for the default so
  existing deploys don't re-download; rebuilds on a name change.
- agent: new 'embed' job = whole-image SigLIP embedding (mean-pool video frames)
  under the lease-announced model → POST /jobs/submit_embedding writes
  image_record.siglip_embedding + siglip_model_version. The lease now announces
  the model FROM THE SETTING (not a constant).
- re-embed routing: enqueue_gpu_backfill('embed') selects unembedded + stale-
  version images; 'siglip' now re-embeds concept crops whose version != current
  (so a swap re-triggers crops, not just the never-embedded back-catalogue). The
  CPU ml-worker backfill no longer re-embeds on a version mismatch (it can't
  churn the library at 512px) — the GPU agent owns version re-embeds. Daily
  'embed' + 'siglip' beats self-heal.
- scoring: score_image only bags embeddings in the CURRENT model's space (whole-
  image gated by siglip_model_version, concept regions by embedding_version) so a
  mid-swap stale vector isn't scored by new-space heads; legacy NULL = current.
- UI: GpuAgentCard "Embedding model (advanced)" — edit name/version, Save, and
  "Re-embed library (GPU)" (queues embed + siglip); points at SigLIP 2.

Tests: lease announces model + submit_embedding round-trip; enqueue 'embed'
selects stale/unembedded; stale-version excluded from scoring; embedder model
settable + empty rejected; siglip gate updated to current-version concept.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
2026-06-30 10:24:30 -04:00
parent 0f472b2f9e
commit 4daa3f2790
15 changed files with 379 additions and 53 deletions
@@ -106,6 +106,37 @@
reversible) so identity tags keep flowing without review. Stricter than
the suggest cut; 0.92 recommended.
</p>
<!-- Embedding model (advanced) -->
<div v-if="ml.settings" class="fc-section-h mt-5 mb-1">Embedding model (advanced)</div>
<div v-if="ml.settings">
<v-text-field
v-model="modelName" label="HF model name" density="compact" hide-details
variant="outlined" class="mb-2"
/>
<v-text-field
v-model="modelVersion" label="Version stamp" density="compact" hide-details
variant="outlined"
/>
<div class="d-flex mt-3" style="gap:8px">
<v-btn
size="small" variant="tonal" rounded="pill" :loading="savingModel"
prepend-icon="mdi-content-save" @click="onSaveModel"
>Save model</v-btn>
<v-btn
size="small" color="accent" variant="flat" rounded="pill"
:loading="reembedding" prepend-icon="mdi-backup-restore" @click="onReembed"
>Re-embed library (GPU)</v-btn>
</div>
<p class="fc-muted text-caption mt-2 mb-0">
Changing the model means a DIFFERENT embedding space. After saving a new
model + version, run <b>Re-embed library</b> (the GPU agent re-embeds
whole images + concept crops), then <b>Retrain heads</b>. Suggestions
degrade until both finish. SigLIP 2 (<code>google/siglip2-so400m-patch16-512</code>,
version <code>siglip2-so400m-patch16-512</code>) is a 1152-d drop-in at
512px no schema change.
</p>
</div>
</MaintenanceTile>
</template>
@@ -131,6 +162,10 @@ const savingThreshold = ref(false)
const autoApply = ref(true)
const autoThreshold = ref(0.92)
const savingAuto = ref(false)
const modelName = ref('')
const modelVersion = ref('')
const savingModel = ref(false)
const reembedding = ref(false)
const queue = ref({ pending: 0, leased: 0, done: 0, error: 0 })
let pollTimer = null
@@ -157,9 +192,42 @@ onMounted(async () => {
autoApply.value = ml.settings.ccip_auto_apply_enabled
autoThreshold.value = ml.settings.ccip_auto_apply_threshold
}
if (ml.settings?.embedder_model_name != null) {
modelName.value = ml.settings.embedder_model_name
modelVersion.value = ml.settings.embedder_model_version
}
} catch { /* non-fatal */ }
})
async function onSaveModel() {
savingModel.value = true
try {
await ml.patchSettings({
embedder_model_name: modelName.value.trim(),
embedder_model_version: modelVersion.value.trim(),
})
toast({ text: 'Embedding model saved — now Re-embed library, then Retrain heads', type: 'success' })
} catch (e) {
toast({ text: `Could not save model: ${e.message}`, type: 'error' })
} finally {
savingModel.value = false
}
}
async function onReembed() {
reembedding.value = true
try {
await store.backfill('embed')
await store.backfill('siglip')
toast({ text: 'Queued whole-image + concept re-embed — run the agent, then Retrain heads', type: 'success' })
await refreshQueue()
} catch (e) {
toast({ text: `Could not queue re-embed: ${e.message}`, type: 'error' })
} finally {
reembedding.value = false
}
}
async function onSaveAuto() {
savingAuto.value = true
try {