feat(agent): desktop GPU agent container — CCIP + figure crops over HTTP (#114)
The last piece: a Dockerised desktop-GPU worker that talks to FC ONLY over HTTP (lease → fetch pixels → detect figures + CCIP-embed → submit), so Redis/Postgres stay private. New top-level agent/ (outside CI scope — verified by running it): - fc_agent/worker.py: the lease/compute/submit loop, concurrency 1, start/pause/ stop (stop frees the card; unprocessed leases expire + re-queue). - fc_agent/models.py: imgutils wrappers — detect_person (figures) + CCIP embed. The two API seams to verify against the installed dghs-imgutils (flagged). - fc_agent/media.py: stills + video frame sampling (ffmpeg) at FC's cadence → per-frame instances (the bag). - fc_agent/crops.py: vendored crop primitive. client.py: the FC HTTP client. - fc_agent/app.py: FastAPI localhost control UI (start/pause/stop + progress + queue depth). Dockerfile (CUDA + onnxruntime-gpu + ffmpeg) + requirements + README (token → build → run --gpus all → Start; CPU-fallback path). This completes the CCIP pipeline end to end: agent produces region CCIP vectors → RegionService stores → matcher suggests characters → rail. Verified by running on the desktop (not CI). README calls out the imgutils API + model-string checks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
@@ -0,0 +1,60 @@
|
||||
# FabledCurator GPU agent
|
||||
|
||||
A desktop-GPU worker that embeds characters (CCIP) + figure crops for
|
||||
FabledCurator. It talks to FC **only over HTTP** — it leases jobs, fetches image
|
||||
pixels, runs the models on your GPU, and posts results back. Your FC database and
|
||||
Redis stay private; the agent never touches them.
|
||||
|
||||
You run it when you want a burst and stop it to reclaim the card.
|
||||
|
||||
## 1. Get a token
|
||||
In FC: **Settings → Tagging → GPU agent → Generate token** (or Rotate). Copy it.
|
||||
|
||||
## 2. Build
|
||||
```sh
|
||||
cd agent
|
||||
docker build -t fc-gpu-agent .
|
||||
```
|
||||
|
||||
## 3. Run (on the machine with the GPU)
|
||||
```sh
|
||||
docker run --rm --gpus all -p 8770:8770 \
|
||||
-e FC_URL=http://curator.traefik.internal \
|
||||
-e FC_TOKEN=<paste-the-token> \
|
||||
-v fc-agent-models:/models \
|
||||
fc-gpu-agent
|
||||
```
|
||||
Then open <http://localhost:8770> — the control page. Click **Start** to begin
|
||||
draining the queue; **Pause**/**Stop** to yield the GPU. The `-v fc-agent-models`
|
||||
volume caches the downloaded ONNX models so restarts are fast.
|
||||
|
||||
Kick off a backfill from FC (**GPU agent card → Queue character embedding**), then
|
||||
watch the queue counts on the control page (or FC's card) drain.
|
||||
|
||||
## Config (env)
|
||||
| var | default | meaning |
|
||||
|---|---|---|
|
||||
| `FC_URL` | `http://localhost:8000` | FC base URL |
|
||||
| `FC_TOKEN` | — | the bearer token (required) |
|
||||
| `AGENT_ID` | `desktop-agent` | identifies this agent's leases |
|
||||
| `BATCH_SIZE` | `4` | jobs leased per round (still processed one at a time) |
|
||||
| `CCIP_MODEL` | imgutils default | CCIP model name |
|
||||
| `DETECTOR_LEVEL` | `m` | person-detector size: `n` < `s` < `m` < `x` |
|
||||
| `POLL_IDLE_SECONDS` | `10` | wait between empty leases |
|
||||
|
||||
## ⚠️ Verify on first run
|
||||
This part can't be CI-tested (no GPU/models in CI), so confirm against your
|
||||
installed `dghs-imgutils` (`pip show dghs-imgutils`) — see `fc_agent/models.py`:
|
||||
- `imgutils.detect.detect_person(image, level=...)` returns
|
||||
`[((x0,y0,x1,y1), label, score), ...]`.
|
||||
- `imgutils.metrics.ccip_extract_feature(image, model=...)` returns a vector
|
||||
(768-d for caformer). If you want the F1-0.94 variant, set
|
||||
`CCIP_MODEL=ccip-caformer_b36-24` (verify the exact string in imgutils).
|
||||
|
||||
If FC's matcher under/over-fires, tune the cosine threshold in
|
||||
`backend/app/services/ml/ccip.py` (`DEFAULT_SIM_THRESHOLD`) and use
|
||||
`GET /api/ccip/overview` + `/api/ccip/images/<id>` to spot-check.
|
||||
|
||||
## CPU fallback
|
||||
Swap `onnxruntime-gpu` → `onnxruntime` in `requirements.txt` and drop `--gpus all`
|
||||
to grind it slowly on the server instead. Same agent, no card.
|
||||
Reference in New Issue
Block a user