bvandeusen
8419ebd761
The last piece: a Dockerised desktop-GPU worker that talks to FC ONLY over HTTP (lease → fetch pixels → detect figures + CCIP-embed → submit), so Redis/Postgres stay private. New top-level agent/ (outside CI scope — verified by running it): - fc_agent/worker.py: the lease/compute/submit loop, concurrency 1, start/pause/ stop (stop frees the card; unprocessed leases expire + re-queue). - fc_agent/models.py: imgutils wrappers — detect_person (figures) + CCIP embed. The two API seams to verify against the installed dghs-imgutils (flagged). - fc_agent/media.py: stills + video frame sampling (ffmpeg) at FC's cadence → per-frame instances (the bag). - fc_agent/crops.py: vendored crop primitive. client.py: the FC HTTP client. - fc_agent/app.py: FastAPI localhost control UI (start/pause/stop + progress + queue depth). Dockerfile (CUDA + onnxruntime-gpu + ffmpeg) + requirements + README (token → build → run --gpus all → Start; CPU-fallback path). This completes the CCIP pipeline end to end: agent produces region CCIP vectors → RegionService stores → matcher suggests characters → rail. Verified by running on the desktop (not CI). README calls out the imgutils API + model-string checks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
40 lines
1.6 KiB
Python
40 lines
1.6 KiB
Python
"""imgutils model wrappers — the figure DETECTOR + the CCIP EMBEDDER.
|
|
|
|
⚠️ VERIFY ON FIRST RUN: the exact imgutils function names/signatures + the CCIP
|
|
model string can drift between dghs-imgutils releases. These are the two seams to
|
|
check against your installed version (`pip show dghs-imgutils`):
|
|
- detect_person(image, level=...) -> [((x0,y0,x1,y1), label, score), ...]
|
|
- ccip_extract_feature(image, model=...) -> a vector (768-d for caformer)
|
|
imgutils auto-downloads the ONNX models from HuggingFace on first use; GPU is
|
|
used when onnxruntime-gpu is installed.
|
|
"""
|
|
import numpy as np
|
|
from PIL import Image
|
|
|
|
|
|
def detect_figures(image: Image.Image, level: str = "m") -> list[tuple[tuple, float | None]]:
|
|
"""Person/figure bounding boxes, NORMALIZED (x, y, w, h in [0,1]) + score.
|
|
Returns [] if detection finds nothing (caller falls back to whole-image)."""
|
|
from imgutils.detect import detect_person
|
|
|
|
iw, ih = image.size
|
|
out = []
|
|
for (x0, y0, x1, y1), _label, score in detect_person(image, level=level):
|
|
out.append((
|
|
(x0 / iw, y0 / ih, (x1 - x0) / iw, (y1 - y0) / ih),
|
|
float(score),
|
|
))
|
|
return out
|
|
|
|
|
|
def ccip_vector(image: Image.Image, model: str | None = None) -> list[float]:
|
|
"""The CCIP identity embedding of a (cropped) character image, as a plain
|
|
float list ready to POST."""
|
|
from imgutils.metrics import ccip_extract_feature
|
|
|
|
feat = (
|
|
ccip_extract_feature(image, model=model)
|
|
if model else ccip_extract_feature(image)
|
|
)
|
|
return np.asarray(feat, dtype=np.float32).reshape(-1).tolist()
|