perf(agent): batch SigLIP crop embeds per image + load truncated images
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 16s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m26s

Two issues surfaced by the live logs (GPU pegged at ~0% util, 0.5 jobs/s,
truncated-image failures):

- BATCH the SigLIP embeds: collect all of an image's crops (figure + booru_yolo
  components + panels) and embed them in ONE forward pass instead of one
  forward+lock per crop. The per-crop path serialised every crop through the
  inference lock and starved the GPU (≈0% util, autoscaler stuck oscillating);
  batching gives a real GPU-bound workload + far higher throughput. CCIP still
  runs per figure inline.
- LOAD_TRUNCATED_IMAGES in the agent (matches the server embedder): slightly-
  truncated scraped images now load instead of failing the job 3× then erroring
  ("image file is truncated (N bytes not processed)").

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
2026-06-30 18:47:33 -04:00
parent 9eaefac385
commit 2713c3f773
3 changed files with 44 additions and 29 deletions
+11 -3
View File
@@ -58,12 +58,20 @@ class CropEmbedder:
def embed(self, image: Image.Image) -> list[float]:
"""A crop → its embedding as a plain float list, ready to POST."""
return self.embed_batch([image])[0]
def embed_batch(self, images: list) -> list[list[float]]:
"""Embed many crops in ONE forward pass — far better GPU utilisation +
only one lock acquisition than embedding each crop separately (which
starved the GPU and serialised the whole pool)."""
if not images:
return []
self.load()
torch = self._torch
enc = self._processor(images=image, return_tensors="pt")
enc = self._processor(images=images, return_tensors="pt")
pixel_values = enc["pixel_values"].to(self._device, self._dt)
with self._infer_lock, torch.no_grad():
out = self._model.get_image_features(pixel_values=pixel_values)
pooled = out.pooler_output if hasattr(out, "pooler_output") else out
vec = pooled[0].float().cpu().numpy().astype(np.float32).reshape(-1)
return vec.tolist()
arr = pooled.float().cpu().numpy().astype(np.float32)
return [row.reshape(-1).tolist() for row in arr]