feat(agent): dedupe near-duplicate crops before the SigLIP embed
Figure boxes are already NMS-merged (iou 0.6) and each YOLO detector self-NMSes, but the combined per-frame crop pile (figure→concept ∪ anatomy component→concept ∪ panel) was embedded with no cross-proposer dedup — so genuine near-duplicates slipped through (a figure box ≈ an anatomy component on a solo bust; overlapping booru head classes on one head), embedding the same region twice and burning a slot against max_regions. Add detectors.dedupe_crops(): a greedy, high-IoU (default 0.85), kind-aware pass over the pending (crop, template) list right before embed_batch — drop boxes that overlap ≥ iou within the same kind, keep the highest score. The high threshold is deliberate: it collapses only true near-identical boxes while preserving intentional nested crops across scopes (a whole figure vs a small head component sit well below it) and distinct kinds (concept vs panel). Env-tunable DEDUPE_IOU (≥1.0 disables). Runs on CPU before the GPU work, so it cuts both embed cost and region count. Temporal (cross-frame) dedup deferred. Build marker 2026-07-01.2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
@@ -67,6 +67,37 @@ def nms_merge(boxes, iou_thresh: float = 0.6):
|
||||
return kept
|
||||
|
||||
|
||||
def dedupe_crops(pending, iou_thresh: float = 0.85):
|
||||
"""Greedy high-IoU dedupe over a list of (crop, region_template) pairs, run
|
||||
just before the batched SigLIP embed so we never embed the same region twice.
|
||||
|
||||
Figure boxes are already NMS-merged and each YOLO self-NMSes, but the combined
|
||||
per-frame pile (figure→concept ∪ anatomy component→concept ∪ panel) can still
|
||||
carry genuine near-duplicates across proposers — e.g. a figure box that nearly
|
||||
coincides with an anatomy component on a solo bust, or overlapping booru head
|
||||
classes on one head. Those embed the same region twice, wasting GPU and a slot
|
||||
against max_regions.
|
||||
|
||||
Boxes are compared ONLY within the same output kind and dropped when they
|
||||
overlap at >= iou_thresh, keeping the highest-scoring one. The HIGH default
|
||||
threshold is deliberate: it collapses only true near-identical boxes while
|
||||
preserving intentional nested crops across scopes (a whole figure vs a small
|
||||
head component sit well below it) and distinct kinds (concept vs panel). A
|
||||
value >= 1.0 effectively disables it (nothing but an exact box matches)."""
|
||||
kept = []
|
||||
kept_boxes: dict = {} # kind -> [bbox, ...] already kept
|
||||
for crop, tmpl in sorted(
|
||||
pending, key=lambda p: p[1].get("score") or 0.0, reverse=True
|
||||
):
|
||||
bb = tmpl.get("bbox")
|
||||
prior = kept_boxes.setdefault(tmpl.get("kind"), [])
|
||||
if bb is not None and any(_iou(bb, kb) >= iou_thresh for kb in prior):
|
||||
continue
|
||||
prior.append(bb)
|
||||
kept.append((crop, tmpl))
|
||||
return kept
|
||||
|
||||
|
||||
class YoloProposer:
|
||||
"""One lazily-loaded ultralytics YOLO. detect(image) → [(bbox_norm, score,
|
||||
label)] with bbox normalized (x, y, w, h) in [0,1]. Self-disables on any
|
||||
|
||||
Reference in New Issue
Block a user