feat(ml): operator model swap — GPU re-embed + embedder as a setting (#1190)
Make the SigLIP embedder an operator choice (drop-in to SigLIP 2:
google/siglip2-so400m-patch16-512 is a verified 1152-d model at 512px → no
schema change, better small-cue fidelity). A swap = set model + re-embed +
retrain, all operator-driven; the GPU agent does the re-embed so it's fast.
- settings: embedder_model_name is now a setting (migration 0065) alongside the
existing embedder_model_version; both editable + validated (non-empty) in the
ml admin API. The server embedder loads by HF name (AutoImageProcessor/Model,
model-agnostic), preferring the pre-downloaded local dir for the default so
existing deploys don't re-download; rebuilds on a name change.
- agent: new 'embed' job = whole-image SigLIP embedding (mean-pool video frames)
under the lease-announced model → POST /jobs/submit_embedding writes
image_record.siglip_embedding + siglip_model_version. The lease now announces
the model FROM THE SETTING (not a constant).
- re-embed routing: enqueue_gpu_backfill('embed') selects unembedded + stale-
version images; 'siglip' now re-embeds concept crops whose version != current
(so a swap re-triggers crops, not just the never-embedded back-catalogue). The
CPU ml-worker backfill no longer re-embeds on a version mismatch (it can't
churn the library at 512px) — the GPU agent owns version re-embeds. Daily
'embed' + 'siglip' beats self-heal.
- scoring: score_image only bags embeddings in the CURRENT model's space (whole-
image gated by siglip_model_version, concept regions by embedding_version) so a
mid-swap stale vector isn't scored by new-space heads; legacy NULL = current.
- UI: GpuAgentCard "Embedding model (advanced)" — edit name/version, Save, and
"Re-embed library (GPU)" (queues embed + siglip); points at SigLIP 2.
Tests: lease announces model + submit_embedding round-trip; enqueue 'embed'
selects stale/unembedded; stale-version excluded from scoring; embedder model
settable + empty rejected; siglip gate updated to current-version concept.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
@@ -69,6 +69,39 @@ async def test_lease_submit_round_trip(client, db):
|
||||
assert len(regs) == 1 and len(list(regs[0].ccip_embedding)) == 768
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_lease_announces_embed_model_then_submit_embedding(client, db):
|
||||
# Whole-image GPU re-embed (#1190): the lease announces the embedder model so
|
||||
# the agent loads the right one, and submit_embedding writes it back onto
|
||||
# image_record with its version stamp.
|
||||
img = await _img(db, "b" * 64)
|
||||
await GpuJobService(db).enqueue(img.id, "embed")
|
||||
await db.commit()
|
||||
|
||||
token = (await (await client.post("/api/gpu/token/rotate")).get_json())["token"]
|
||||
hdr = {"Authorization": f"Bearer {token}"}
|
||||
|
||||
leased = await client.post(
|
||||
"/api/gpu/jobs/lease", json={"agent_id": "a1", "batch_size": 5}, headers=hdr,
|
||||
)
|
||||
j = (await leased.get_json())["jobs"][0]
|
||||
assert j["task"] == "embed"
|
||||
assert j["embed_model_name"] and j["embed_version"] # server-announced model
|
||||
|
||||
submitted = await client.post("/api/gpu/jobs/submit_embedding", json={
|
||||
"agent_id": "a1", "job_id": j["job_id"],
|
||||
"embedding": [0.2] * 1152, "embedding_version": "siglip2-test-v9",
|
||||
}, headers=hdr)
|
||||
assert submitted.status_code == 200
|
||||
|
||||
st = await (await client.get("/api/gpu/status")).get_json()
|
||||
assert st["done"] == 1 and st["leased"] == 0
|
||||
|
||||
await db.refresh(img)
|
||||
assert img.siglip_model_version == "siglip2-test-v9"
|
||||
assert img.siglip_embedding is not None and len(list(img.siglip_embedding)) == 1152
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_submit_with_stale_lease_is_409(client, db):
|
||||
img = await _img(db, "b" * 64)
|
||||
|
||||
Reference in New Issue
Block a user