feat(ml): operator model swap — GPU re-embed + embedder as a setting (#1190)
Make the SigLIP embedder an operator choice (drop-in to SigLIP 2:
google/siglip2-so400m-patch16-512 is a verified 1152-d model at 512px → no
schema change, better small-cue fidelity). A swap = set model + re-embed +
retrain, all operator-driven; the GPU agent does the re-embed so it's fast.
- settings: embedder_model_name is now a setting (migration 0065) alongside the
existing embedder_model_version; both editable + validated (non-empty) in the
ml admin API. The server embedder loads by HF name (AutoImageProcessor/Model,
model-agnostic), preferring the pre-downloaded local dir for the default so
existing deploys don't re-download; rebuilds on a name change.
- agent: new 'embed' job = whole-image SigLIP embedding (mean-pool video frames)
under the lease-announced model → POST /jobs/submit_embedding writes
image_record.siglip_embedding + siglip_model_version. The lease now announces
the model FROM THE SETTING (not a constant).
- re-embed routing: enqueue_gpu_backfill('embed') selects unembedded + stale-
version images; 'siglip' now re-embeds concept crops whose version != current
(so a swap re-triggers crops, not just the never-embedded back-catalogue). The
CPU ml-worker backfill no longer re-embeds on a version mismatch (it can't
churn the library at 512px) — the GPU agent owns version re-embeds. Daily
'embed' + 'siglip' beats self-heal.
- scoring: score_image only bags embeddings in the CURRENT model's space (whole-
image gated by siglip_model_version, concept regions by embedding_version) so a
mid-swap stale vector isn't scored by new-space heads; legacy NULL = current.
- UI: GpuAgentCard "Embedding model (advanced)" — edit name/version, Save, and
"Re-embed library (GPU)" (queues embed + siglip); points at SigLIP 2.
Tests: lease announces model + submit_embedding round-trip; enqueue 'embed'
selects stale/unembedded; stale-version excluded from scoring; embedder model
settable + empty rejected; siglip gate updated to current-version concept.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
@@ -106,6 +106,37 @@
|
||||
reversible) — so identity tags keep flowing without review. Stricter than
|
||||
the suggest cut; 0.92 recommended.
|
||||
</p>
|
||||
|
||||
<!-- Embedding model (advanced) -->
|
||||
<div v-if="ml.settings" class="fc-section-h mt-5 mb-1">Embedding model (advanced)</div>
|
||||
<div v-if="ml.settings">
|
||||
<v-text-field
|
||||
v-model="modelName" label="HF model name" density="compact" hide-details
|
||||
variant="outlined" class="mb-2"
|
||||
/>
|
||||
<v-text-field
|
||||
v-model="modelVersion" label="Version stamp" density="compact" hide-details
|
||||
variant="outlined"
|
||||
/>
|
||||
<div class="d-flex mt-3" style="gap:8px">
|
||||
<v-btn
|
||||
size="small" variant="tonal" rounded="pill" :loading="savingModel"
|
||||
prepend-icon="mdi-content-save" @click="onSaveModel"
|
||||
>Save model</v-btn>
|
||||
<v-btn
|
||||
size="small" color="accent" variant="flat" rounded="pill"
|
||||
:loading="reembedding" prepend-icon="mdi-backup-restore" @click="onReembed"
|
||||
>Re-embed library (GPU)</v-btn>
|
||||
</div>
|
||||
<p class="fc-muted text-caption mt-2 mb-0">
|
||||
Changing the model means a DIFFERENT embedding space. After saving a new
|
||||
model + version, run <b>Re-embed library</b> (the GPU agent re-embeds
|
||||
whole images + concept crops), then <b>Retrain heads</b>. Suggestions
|
||||
degrade until both finish. SigLIP 2 (<code>google/siglip2-so400m-patch16-512</code>,
|
||||
version <code>siglip2-so400m-patch16-512</code>) is a 1152-d drop-in at
|
||||
512px — no schema change.
|
||||
</p>
|
||||
</div>
|
||||
</MaintenanceTile>
|
||||
</template>
|
||||
|
||||
@@ -131,6 +162,10 @@ const savingThreshold = ref(false)
|
||||
const autoApply = ref(true)
|
||||
const autoThreshold = ref(0.92)
|
||||
const savingAuto = ref(false)
|
||||
const modelName = ref('')
|
||||
const modelVersion = ref('')
|
||||
const savingModel = ref(false)
|
||||
const reembedding = ref(false)
|
||||
const queue = ref({ pending: 0, leased: 0, done: 0, error: 0 })
|
||||
let pollTimer = null
|
||||
|
||||
@@ -157,9 +192,42 @@ onMounted(async () => {
|
||||
autoApply.value = ml.settings.ccip_auto_apply_enabled
|
||||
autoThreshold.value = ml.settings.ccip_auto_apply_threshold
|
||||
}
|
||||
if (ml.settings?.embedder_model_name != null) {
|
||||
modelName.value = ml.settings.embedder_model_name
|
||||
modelVersion.value = ml.settings.embedder_model_version
|
||||
}
|
||||
} catch { /* non-fatal */ }
|
||||
})
|
||||
|
||||
async function onSaveModel() {
|
||||
savingModel.value = true
|
||||
try {
|
||||
await ml.patchSettings({
|
||||
embedder_model_name: modelName.value.trim(),
|
||||
embedder_model_version: modelVersion.value.trim(),
|
||||
})
|
||||
toast({ text: 'Embedding model saved — now Re-embed library, then Retrain heads', type: 'success' })
|
||||
} catch (e) {
|
||||
toast({ text: `Could not save model: ${e.message}`, type: 'error' })
|
||||
} finally {
|
||||
savingModel.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function onReembed() {
|
||||
reembedding.value = true
|
||||
try {
|
||||
await store.backfill('embed')
|
||||
await store.backfill('siglip')
|
||||
toast({ text: 'Queued whole-image + concept re-embed — run the agent, then Retrain heads', type: 'success' })
|
||||
await refreshQueue()
|
||||
} catch (e) {
|
||||
toast({ text: `Could not queue re-embed: ${e.message}`, type: 'error' })
|
||||
} finally {
|
||||
reembedding.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function onSaveAuto() {
|
||||
savingAuto.value = true
|
||||
try {
|
||||
|
||||
Reference in New Issue
Block a user