The new per-job workload (3 detectors + several SigLIP embeds) is far more
GPU-bound than the old I/O-bound CCIP pass, so the right worker count shifted and
is hard to guess. Add an Auto mode (default ON) that finds it:
- _control_loop samples jobs/sec + GPU util/VRAM every ~6s and hill-climbs the
target: grow while throughput keeps improving and VRAM stays under budget,
revert a step that doesn't help, back off under memory pressure (VRAM >= 90%),
then settle and periodically re-probe (the GPU/IO balance shifts over a run).
- A manual concurrency set is an override → leaves Auto; an "Auto" toggle in the
control UI re-enables it. status() reports `auto`; the dial reflects the
auto-chosen count (read-only) while Auto is on.
- AUTO_SCALE env (default on) + compose doc. Agent py-compiled (outside CI).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Better region PROPOSERS feeding the existing crop→SigLIP→max-over-bag heads (no
change to the learned-tagging approach; no per-tag cost — propose once, embed
each region, all heads in one matmul).
- detectors.py: lazy ultralytics YOLO wrapper, each proposer independently
optional + guarded (a bad weight spec / inference error self-disables that one,
logged, never breaks the worker). Weights resolve from an ultralytics name |
http(s) URL | "hf_repo::file", cached under HF_HOME. NMS merge so a figure two
detectors both find collapses to one crop.
- worker: figure boxes = imgutils detect_person ∪ general COCO person (merged)
→ CCIP + concept (anime + Western/realistic coverage); booru_yolo anatomy
components (head/cat-head/anatomy/…) → concept crops; comic panels → kind=
'panel' concept crops. Capped per frame (MAX_COMPONENTS/MAX_PANELS).
- config + compose: PERSON_WEIGHTS (default yolo11n.pt, works OOB),
ANATOMY_WEIGHTS + PANEL_WEIGHTS (operator sets booru_yolo URL + mosesb panel
hf::file; empty = off). ultralytics added to requirements.
- backend: image_region 'kind' doc notes 'panel'; no migration (free String,
and the bag scorer keys on a non-null siglip_embedding, not the kind, so any
SigLIP region joins the bag automatically).
Agent is outside CI — py-compiled here; operator tests on the GPU and checks
Western-vs-anime crop quality via /api/ccip observability.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Make the SigLIP embedder an operator choice (drop-in to SigLIP 2:
google/siglip2-so400m-patch16-512 is a verified 1152-d model at 512px → no
schema change, better small-cue fidelity). A swap = set model + re-embed +
retrain, all operator-driven; the GPU agent does the re-embed so it's fast.
- settings: embedder_model_name is now a setting (migration 0065) alongside the
existing embedder_model_version; both editable + validated (non-empty) in the
ml admin API. The server embedder loads by HF name (AutoImageProcessor/Model,
model-agnostic), preferring the pre-downloaded local dir for the default so
existing deploys don't re-download; rebuilds on a name change.
- agent: new 'embed' job = whole-image SigLIP embedding (mean-pool video frames)
under the lease-announced model → POST /jobs/submit_embedding writes
image_record.siglip_embedding + siglip_model_version. The lease now announces
the model FROM THE SETTING (not a constant).
- re-embed routing: enqueue_gpu_backfill('embed') selects unembedded + stale-
version images; 'siglip' now re-embeds concept crops whose version != current
(so a swap re-triggers crops, not just the never-embedded back-catalogue). The
CPU ml-worker backfill no longer re-embeds on a version mismatch (it can't
churn the library at 512px) — the GPU agent owns version re-embeds. Daily
'embed' + 'siglip' beats self-heal.
- scoring: score_image only bags embeddings in the CURRENT model's space (whole-
image gated by siglip_model_version, concept regions by embedding_version) so a
mid-swap stale vector isn't scored by new-space heads; legacy NULL = current.
- UI: GpuAgentCard "Embedding model (advanced)" — edit name/version, Save, and
"Re-embed library (GPU)" (queues embed + siglip); points at SigLIP 2.
Tests: lease announces model + submit_embedding round-trip; enqueue 'embed'
selects stale/unembedded; stale-version excluded from scoring; embedder model
settable + empty rejected; siglip gate updated to current-version concept.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
For redeploying curator while away with nobody to restart the agent:
- _process now distinguishes a TRANSPORT error (curator down/redeploying, 5xx,
401/403/408/409/429, or our lease reclaimed mid-flight) from a genuine job
fault. On a transport error it hands the job back (best effort) and signals
the loop to back off — instead of calling fail(), which would burn the job's
server-side attempt budget (MAX_ATTEMPTS=3) and permanently error good jobs
across a redeploy. Job-specific 4xx (404 image gone) still fail so they don't
re-lease forever.
- lease loop retries with capped exponential backoff (poll_idle → 60s) and
resets on the first successful lease, so a long outage is gentle and recovery
is automatic within ≤60s of curator returning. Sleeps are interruptible so
Stop / pool-shrink stays responsive.
- AUTO_START env (default on in compose) resumes the worker on container start,
so a host reboot / crash-restart (restart: unless-stopped) self-heals with
nobody at the desktop.
- control UI shows a "waited out" counter + an "curator unreachable, holding
work" banner so the recovering state reads as recovery, not failure.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Lift recall on small/local concepts (glasses, cum, stomach-bulge, xray,
lactation) that the whole-image SigLIP vector washes out: the GPU agent now
embeds figure crops with SigLIP too, stored as kind='concept' regions, and the
suggestion rail scores each image as a BAG (whole-image + every concept crop),
taking each head's MAX over the bag. The whole-image vector is always in the
bag, so this can never score lower than before.
Model-agnostic by construction: the server ANNOUNCES the embedding model
(HF name + version) in the lease, so the agent loads whatever the heads were
trained in and stays in lock-step — a model swap is a server setting + a
re-embed migration, never an agent change.
- agent: model-agnostic CropEmbedder (torch/transformers get_image_features,
fp16 on CUDA, inference-locked); worker branches on job.task — 'ccip' emits
figure(CCIP)+concept(SigLIP) in one pass, 'siglip' emits concept-only so the
back-catalogue backfill never churns figure/CCIP regions; torch cu124 +
transformers in the image.
- server: lease announces embed_model_name/embed_version; score_image is
max-over-bag (version-filtered region embeddings); enqueue_gpu_backfill
'siglip' gates on a missing concept region (drains the back-catalogue,
retries failures, no double-enqueue); daily siglip-backfill beat; UI button;
/api/ccip/overview reports images_with_concept_siglip.
- v1 scope: suggestion rail only — auto-apply stays whole-image (conservative;
heads' thresholds were calibrated on whole-image). Bulk-apply bag = follow-up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
At 8 workers the GPU sat at ~5% util / <5GB VRAM — the pipeline is I/O-bound
(downloading + decoding images over HTTP), so the GPU starves until many workers
overlap that I/O. Raise MAX_CONCURRENCY 8→32 and make the UI worker control a
number input (reaching 32 by ±1 was tedious); the cap is reported via /status so
the UI clamps to it. Also size the shared requests pool (pool_maxsize=64) — the
default 10 would have throttled 32 workers + spammed "connection pool is full".
Verified by running; watch GPU util/VRAM climb as you dial up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
A naive convert('RGB') on a palette-with-transparency image (common: character
PNGs on a clear background) lets PIL guess the transparent pixels — black-ish
artifacts that bleed into the crop + the CCIP embedding (and the "should be
converted to RGBA" warning). to_rgb() composites over white first for a clean,
consistent background; used by both stills and video frames.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Control UI gains what the operator asked for:
- GPU load (nvidia-smi): util %, VRAM used/total + bar, temp — so you can see how
hard the card is working while you're at the desktop.
- Worker count is now a live − / + control (POST /concurrency), not just an env:
the worker is a pool of independent slots (shared model, so slots add concurrent
inference, not N× VRAM). Dial up for speed, down to free the card. Replaces
pause/resume with Start/Stop + the worker dial.
- Graceful release on stop / pool-shrink: a slot hands its still-leased jobs back
via client.release() so they're re-picked immediately (pairs with the server
recovery sweep).
Not CI-tested (agent/ outside CI) — verified by running.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
onnxruntime-gpu needs cuDNN 9; the plain cuda:12.4.1-runtime image lacks it
(libcudnn.so.9 missing → CUDAExecutionProvider falls back to CPU). Switch to
the -cudnn-runtime variant which bundles cuDNN 9.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
compose file (pull the published image, GPU reservation, model-cache volume,
.env for the token) so the agent runs with `docker compose up -d` instead of a
long docker run. A copy + .env template also placed in ~/Documents/fc-gpu-agent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Build + push fabledcurator-agent alongside web/ml (own CUDA + onnxruntime-gpu
image, context=agent/, same tag cadence: main → :main/:latest/:c-<sha>, tag →
:<version>). So the operator PULLS + runs it on the GPU machine instead of
building locally. README switched to docker pull.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The last piece: a Dockerised desktop-GPU worker that talks to FC ONLY over HTTP
(lease → fetch pixels → detect figures + CCIP-embed → submit), so Redis/Postgres
stay private. New top-level agent/ (outside CI scope — verified by running it):
- fc_agent/worker.py: the lease/compute/submit loop, concurrency 1, start/pause/
stop (stop frees the card; unprocessed leases expire + re-queue).
- fc_agent/models.py: imgutils wrappers — detect_person (figures) + CCIP embed.
The two API seams to verify against the installed dghs-imgutils (flagged).
- fc_agent/media.py: stills + video frame sampling (ffmpeg) at FC's cadence →
per-frame instances (the bag).
- fc_agent/crops.py: vendored crop primitive. client.py: the FC HTTP client.
- fc_agent/app.py: FastAPI localhost control UI (start/pause/stop + progress +
queue depth). Dockerfile (CUDA + onnxruntime-gpu + ffmpeg) + requirements +
README (token → build → run --gpus all → Start; CPU-fallback path).
This completes the CCIP pipeline end to end: agent produces region CCIP vectors →
RegionService stores → matcher suggests characters → rail. Verified by running on
the desktop (not CI). README calls out the imgutils API + model-string checks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
bvandeusen