feat(agent): desktop GPU agent container — CCIP + figure crops over HTTP (#114)
The last piece: a Dockerised desktop-GPU worker that talks to FC ONLY over HTTP (lease → fetch pixels → detect figures + CCIP-embed → submit), so Redis/Postgres stay private. New top-level agent/ (outside CI scope — verified by running it): - fc_agent/worker.py: the lease/compute/submit loop, concurrency 1, start/pause/ stop (stop frees the card; unprocessed leases expire + re-queue). - fc_agent/models.py: imgutils wrappers — detect_person (figures) + CCIP embed. The two API seams to verify against the installed dghs-imgutils (flagged). - fc_agent/media.py: stills + video frame sampling (ffmpeg) at FC's cadence → per-frame instances (the bag). - fc_agent/crops.py: vendored crop primitive. client.py: the FC HTTP client. - fc_agent/app.py: FastAPI localhost control UI (start/pause/stop + progress + queue depth). Dockerfile (CUDA + onnxruntime-gpu + ffmpeg) + requirements + README (token → build → run --gpus all → Start; CPU-fallback path). This completes the CCIP pipeline end to end: agent produces region CCIP vectors → RegionService stores → matcher suggests characters → rail. Verified by running on the desktop (not CI). README calls out the imgutils API + model-string checks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
@@ -0,0 +1,48 @@
|
||||
"""Image + video handling. Stills load directly; videos are sampled into frames
|
||||
(ffmpeg) at the cadence FC sends — so a video becomes a bag of per-frame
|
||||
instances, each with a timestamp."""
|
||||
import io
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
from PIL import Image
|
||||
|
||||
|
||||
def is_video(mime: str) -> bool:
|
||||
return bool(mime) and (mime.startswith("video/") or mime in {"image/gif"})
|
||||
|
||||
|
||||
def load_image(data: bytes) -> Image.Image:
|
||||
return Image.open(io.BytesIO(data)).convert("RGB")
|
||||
|
||||
|
||||
def sample_frames(
|
||||
data: bytes, interval_seconds: float, max_frames: int
|
||||
) -> list[tuple[float, Image.Image]]:
|
||||
"""Extract up to max_frames frames at one-every-interval_seconds via ffmpeg.
|
||||
Returns [(timestamp_seconds, frame)]. Empty on failure (caller falls back)."""
|
||||
interval = max(0.5, float(interval_seconds or 4.0))
|
||||
cap = max(1, int(max_frames or 64))
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
src = os.path.join(tmp, "in")
|
||||
with open(src, "wb") as fh:
|
||||
fh.write(data)
|
||||
pattern = os.path.join(tmp, "f_%05d.jpg")
|
||||
try:
|
||||
subprocess.run(
|
||||
[
|
||||
"ffmpeg", "-nostdin", "-loglevel", "error", "-i", src,
|
||||
"-vf", f"fps=1/{interval}", "-frames:v", str(cap),
|
||||
"-q:v", "3", pattern,
|
||||
],
|
||||
check=True, timeout=600,
|
||||
)
|
||||
except (subprocess.SubprocessError, FileNotFoundError):
|
||||
return []
|
||||
out: list[tuple[float, Image.Image]] = []
|
||||
names = sorted(n for n in os.listdir(tmp) if n.startswith("f_"))
|
||||
for i, name in enumerate(names[:cap]):
|
||||
with Image.open(os.path.join(tmp, name)) as im:
|
||||
out.append((round(i * interval, 2), im.convert("RGB")))
|
||||
return out
|
||||
Reference in New Issue
Block a user