feat(agent): crop proposers — booru_yolo anatomy + COCO person + comic panels (#1202)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m25s

Better region PROPOSERS feeding the existing crop→SigLIP→max-over-bag heads (no
change to the learned-tagging approach; no per-tag cost — propose once, embed
each region, all heads in one matmul).

- detectors.py: lazy ultralytics YOLO wrapper, each proposer independently
  optional + guarded (a bad weight spec / inference error self-disables that one,
  logged, never breaks the worker). Weights resolve from an ultralytics name |
  http(s) URL | "hf_repo::file", cached under HF_HOME. NMS merge so a figure two
  detectors both find collapses to one crop.
- worker: figure boxes = imgutils detect_person ∪ general COCO person (merged)
  → CCIP + concept (anime + Western/realistic coverage); booru_yolo anatomy
  components (head/cat-head/anatomy/…) → concept crops; comic panels → kind=
  'panel' concept crops. Capped per frame (MAX_COMPONENTS/MAX_PANELS).
- config + compose: PERSON_WEIGHTS (default yolo11n.pt, works OOB),
  ANATOMY_WEIGHTS + PANEL_WEIGHTS (operator sets booru_yolo URL + mosesb panel
  hf::file; empty = off). ultralytics added to requirements.
- backend: image_region 'kind' doc notes 'panel'; no migration (free String,
  and the bag scorer keys on a non-null siglip_embedding, not the kind, so any
  SigLIP region joins the bag automatically).

Agent is outside CI — py-compiled here; operator tests on the GPU and checks
Western-vs-anime crop quality via /api/ccip observability.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
2026-06-30 15:27:26 -04:00
parent 3d7f60a6e3
commit d5f29f7056
6 changed files with 250 additions and 15 deletions
+11
View File
@@ -37,6 +37,17 @@ services:
# Crop embedder (SigLIP concept bag): float16 keeps VRAM low on a shared
# desktop GPU; the model itself is announced by the server.
SIGLIP_DTYPE: ${SIGLIP_DTYPE:-float16}
# Crop PROPOSERS (extra YOLO detectors → more/better concept crops).
# PERSON_WEIGHTS: general COCO person detector (Western/realistic figures),
# merged with the anime person detector. Default works out of the box.
# ANATOMY_WEIGHTS: booru_yolo (anime/furry/NSFW components). Set to the
# weights URL, e.g. the yolov11m_aa22.pt from github.com/aperveyev/booru_yolo.
# PANEL_WEIGHTS: comic-panel detector as "hf_repo::file",
# e.g. mosesb/best-comic-panel-detection::<weights>.pt
# Empty = that proposer is off. Each self-disables if its weights fail to load.
PERSON_WEIGHTS: ${PERSON_WEIGHTS:-yolo11n.pt}
ANATOMY_WEIGHTS: ${ANATOMY_WEIGHTS:-}
PANEL_WEIGHTS: ${PANEL_WEIGHTS:-}
volumes:
# Persist the downloaded ONNX models so restarts are fast.
- fc-agent-models:/models