bvandeusen
8419ebd761
The last piece: a Dockerised desktop-GPU worker that talks to FC ONLY over HTTP (lease → fetch pixels → detect figures + CCIP-embed → submit), so Redis/Postgres stay private. New top-level agent/ (outside CI scope — verified by running it): - fc_agent/worker.py: the lease/compute/submit loop, concurrency 1, start/pause/ stop (stop frees the card; unprocessed leases expire + re-queue). - fc_agent/models.py: imgutils wrappers — detect_person (figures) + CCIP embed. The two API seams to verify against the installed dghs-imgutils (flagged). - fc_agent/media.py: stills + video frame sampling (ffmpeg) at FC's cadence → per-frame instances (the bag). - fc_agent/crops.py: vendored crop primitive. client.py: the FC HTTP client. - fc_agent/app.py: FastAPI localhost control UI (start/pause/stop + progress + queue depth). Dockerfile (CUDA + onnxruntime-gpu + ffmpeg) + requirements + README (token → build → run --gpus all → Start; CPU-fallback path). This completes the CCIP pipeline end to end: agent produces region CCIP vectors → RegionService stores → matcher suggests characters → rail. Verified by running on the desktop (not CI). README calls out the imgutils API + model-string checks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
49 lines
1.7 KiB
Python
49 lines
1.7 KiB
Python
"""Image + video handling. Stills load directly; videos are sampled into frames
|
|
(ffmpeg) at the cadence FC sends — so a video becomes a bag of per-frame
|
|
instances, each with a timestamp."""
|
|
import io
|
|
import os
|
|
import subprocess
|
|
import tempfile
|
|
|
|
from PIL import Image
|
|
|
|
|
|
def is_video(mime: str) -> bool:
|
|
return bool(mime) and (mime.startswith("video/") or mime in {"image/gif"})
|
|
|
|
|
|
def load_image(data: bytes) -> Image.Image:
|
|
return Image.open(io.BytesIO(data)).convert("RGB")
|
|
|
|
|
|
def sample_frames(
|
|
data: bytes, interval_seconds: float, max_frames: int
|
|
) -> list[tuple[float, Image.Image]]:
|
|
"""Extract up to max_frames frames at one-every-interval_seconds via ffmpeg.
|
|
Returns [(timestamp_seconds, frame)]. Empty on failure (caller falls back)."""
|
|
interval = max(0.5, float(interval_seconds or 4.0))
|
|
cap = max(1, int(max_frames or 64))
|
|
with tempfile.TemporaryDirectory() as tmp:
|
|
src = os.path.join(tmp, "in")
|
|
with open(src, "wb") as fh:
|
|
fh.write(data)
|
|
pattern = os.path.join(tmp, "f_%05d.jpg")
|
|
try:
|
|
subprocess.run(
|
|
[
|
|
"ffmpeg", "-nostdin", "-loglevel", "error", "-i", src,
|
|
"-vf", f"fps=1/{interval}", "-frames:v", str(cap),
|
|
"-q:v", "3", pattern,
|
|
],
|
|
check=True, timeout=600,
|
|
)
|
|
except (subprocess.SubprocessError, FileNotFoundError):
|
|
return []
|
|
out: list[tuple[float, Image.Image]] = []
|
|
names = sorted(n for n in os.listdir(tmp) if n.startswith("f_"))
|
|
for i, name in enumerate(names[:cap]):
|
|
with Image.open(os.path.join(tmp, name)) as im:
|
|
out.append((round(i * interval, 2), im.convert("RGB")))
|
|
return out
|