perf(agent): batch SigLIP crop embeds per image + load truncated images
Two issues surfaced by the live logs (GPU pegged at ~0% util, 0.5 jobs/s,
truncated-image failures):
- BATCH the SigLIP embeds: collect all of an image's crops (figure + booru_yolo
components + panels) and embed them in ONE forward pass instead of one
forward+lock per crop. The per-crop path serialised every crop through the
inference lock and starved the GPU (≈0% util, autoscaler stuck oscillating);
batching gives a real GPU-bound workload + far higher throughput. CCIP still
runs per figure inline.
- LOAD_TRUNCATED_IMAGES in the agent (matches the server embedder): slightly-
truncated scraped images now load instead of failing the job 3× then erroring
("image file is truncated (N bytes not processed)").
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
@@ -58,12 +58,20 @@ class CropEmbedder:
|
||||
|
||||
def embed(self, image: Image.Image) -> list[float]:
|
||||
"""A crop → its embedding as a plain float list, ready to POST."""
|
||||
return self.embed_batch([image])[0]
|
||||
|
||||
def embed_batch(self, images: list) -> list[list[float]]:
|
||||
"""Embed many crops in ONE forward pass — far better GPU utilisation +
|
||||
only one lock acquisition than embedding each crop separately (which
|
||||
starved the GPU and serialised the whole pool)."""
|
||||
if not images:
|
||||
return []
|
||||
self.load()
|
||||
torch = self._torch
|
||||
enc = self._processor(images=image, return_tensors="pt")
|
||||
enc = self._processor(images=images, return_tensors="pt")
|
||||
pixel_values = enc["pixel_values"].to(self._device, self._dt)
|
||||
with self._infer_lock, torch.no_grad():
|
||||
out = self._model.get_image_features(pixel_values=pixel_values)
|
||||
pooled = out.pooler_output if hasattr(out, "pooler_output") else out
|
||||
vec = pooled[0].float().cpu().numpy().astype(np.float32).reshape(-1)
|
||||
return vec.tolist()
|
||||
arr = pooled.float().cpu().numpy().astype(np.float32)
|
||||
return [row.reshape(-1).tolist() for row in arr]
|
||||
|
||||
Reference in New Issue
Block a user