perf(agent): batch SigLIP crop embeds per image + load truncated images
Two issues surfaced by the live logs (GPU pegged at ~0% util, 0.5 jobs/s,
truncated-image failures):
- BATCH the SigLIP embeds: collect all of an image's crops (figure + booru_yolo
components + panels) and embed them in ONE forward pass instead of one
forward+lock per crop. The per-crop path serialised every crop through the
inference lock and starved the GPU (≈0% util, autoscaler stuck oscillating);
batching gives a real GPU-bound workload + far higher throughput. CCIP still
runs per figure inline.
- LOAD_TRUNCATED_IMAGES in the agent (matches the server embedder): slightly-
truncated scraped images now load instead of failing the job 3× then erroring
("image file is truncated (N bytes not processed)").
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
@@ -6,7 +6,12 @@ import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
from PIL import Image
|
||||
from PIL import Image, ImageFile
|
||||
|
||||
# Load slightly-truncated images (a few missing trailing bytes) instead of
|
||||
# raising — matches the server embedder. These are common in scraped libraries
|
||||
# and would otherwise fail the job 3× then error (operator-flagged 2026-06-30).
|
||||
ImageFile.LOAD_TRUNCATED_IMAGES = True
|
||||
|
||||
|
||||
def is_video(mime: str) -> bool:
|
||||
|
||||
Reference in New Issue
Block a user