feat(agent): crop proposers — booru_yolo anatomy + COCO person + comic panels (#1202)
Better region PROPOSERS feeding the existing crop→SigLIP→max-over-bag heads (no change to the learned-tagging approach; no per-tag cost — propose once, embed each region, all heads in one matmul). - detectors.py: lazy ultralytics YOLO wrapper, each proposer independently optional + guarded (a bad weight spec / inference error self-disables that one, logged, never breaks the worker). Weights resolve from an ultralytics name | http(s) URL | "hf_repo::file", cached under HF_HOME. NMS merge so a figure two detectors both find collapses to one crop. - worker: figure boxes = imgutils detect_person ∪ general COCO person (merged) → CCIP + concept (anime + Western/realistic coverage); booru_yolo anatomy components (head/cat-head/anatomy/…) → concept crops; comic panels → kind= 'panel' concept crops. Capped per frame (MAX_COMPONENTS/MAX_PANELS). - config + compose: PERSON_WEIGHTS (default yolo11n.pt, works OOB), ANATOMY_WEIGHTS + PANEL_WEIGHTS (operator sets booru_yolo URL + mosesb panel hf::file; empty = off). ultralytics added to requirements. - backend: image_region 'kind' doc notes 'panel'; no migration (free String, and the bag scorer keys on a non-null siglip_embedding, not the kind, so any SigLIP region joins the bag automatically). Agent is outside CI — py-compiled here; operator tests on the GPU and checks Western-vs-anime crop quality via /api/ccip observability. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
@@ -31,7 +31,10 @@ class ImageRegion(Base):
|
||||
ForeignKey("image_record.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# 'frame' (a whole video frame → SigLIP bag) | 'face' | 'figure' (→ CCIP
|
||||
# character id) | 'concept' (→ SigLIP head bag).
|
||||
# character id) | 'concept' (→ SigLIP head bag) | 'panel' (a comic panel crop,
|
||||
# also SigLIP → the bag). Free String, not an enum — proposers can add kinds
|
||||
# without a migration; the bag scorer keys on a non-null siglip_embedding, not
|
||||
# the kind, so any SigLIP-embedded region joins the bag.
|
||||
kind: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||
# For video/animated media: the source frame's timestamp in SECONDS. NULL for
|
||||
# static images. Lets a video be a BAG of per-frame instances (fixes the
|
||||
|
||||
Reference in New Issue
Block a user