feat(ml): operator model swap — GPU re-embed + embedder as a setting (#1190)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m33s

Make the SigLIP embedder an operator choice (drop-in to SigLIP 2:
google/siglip2-so400m-patch16-512 is a verified 1152-d model at 512px → no
schema change, better small-cue fidelity). A swap = set model + re-embed +
retrain, all operator-driven; the GPU agent does the re-embed so it's fast.

- settings: embedder_model_name is now a setting (migration 0065) alongside the
  existing embedder_model_version; both editable + validated (non-empty) in the
  ml admin API. The server embedder loads by HF name (AutoImageProcessor/Model,
  model-agnostic), preferring the pre-downloaded local dir for the default so
  existing deploys don't re-download; rebuilds on a name change.
- agent: new 'embed' job = whole-image SigLIP embedding (mean-pool video frames)
  under the lease-announced model → POST /jobs/submit_embedding writes
  image_record.siglip_embedding + siglip_model_version. The lease now announces
  the model FROM THE SETTING (not a constant).
- re-embed routing: enqueue_gpu_backfill('embed') selects unembedded + stale-
  version images; 'siglip' now re-embeds concept crops whose version != current
  (so a swap re-triggers crops, not just the never-embedded back-catalogue). The
  CPU ml-worker backfill no longer re-embeds on a version mismatch (it can't
  churn the library at 512px) — the GPU agent owns version re-embeds. Daily
  'embed' + 'siglip' beats self-heal.
- scoring: score_image only bags embeddings in the CURRENT model's space (whole-
  image gated by siglip_model_version, concept regions by embedding_version) so a
  mid-swap stale vector isn't scored by new-space heads; legacy NULL = current.
- UI: GpuAgentCard "Embedding model (advanced)" — edit name/version, Save, and
  "Re-embed library (GPU)" (queues embed + siglip); points at SigLIP 2.

Tests: lease announces model + submit_embedding round-trip; enqueue 'embed'
selects stale/unembedded; stale-version excluded from scoring; embedder model
settable + empty rejected; siglip gate updated to current-version concept.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
2026-06-30 10:24:30 -04:00
parent 0f472b2f9e
commit 4daa3f2790
15 changed files with 379 additions and 53 deletions
+20
View File
@@ -34,6 +34,26 @@ async def test_get_and_patch_settings(client):
assert (await resp.get_json())["suggestion_threshold_general"] == pytest.approx(0.90)
@pytest.mark.asyncio
async def test_embedder_model_settable_and_empty_rejected(client):
# #1190: the embedder model name + version are operator-settable (a swap),
# and neither may be blanked.
body = await (await client.get("/api/ml/settings")).get_json()
assert body["embedder_model_name"] == "google/siglip-so400m-patch14-384"
ok = await client.patch("/api/ml/settings", json={
"embedder_model_name": "google/siglip2-so400m-patch16-512",
"embedder_model_version": "siglip2-so400m-patch16-512",
})
assert ok.status_code == 200
out = await ok.get_json()
assert out["embedder_model_name"] == "google/siglip2-so400m-patch16-512"
assert out["embedder_model_version"] == "siglip2-so400m-patch16-512"
bad = await client.patch("/api/ml/settings", json={"embedder_model_name": " "})
assert bad.status_code == 400
@pytest.mark.asyncio
async def test_tagger_store_floor_default_and_patch(client):
body = await (await client.get("/api/ml/settings")).get_json()