feat(ml): argmax grounding in score_image → suggestions carry the winning crop (#133 step 1)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m41s

score_image now keeps the ARGMAX beside the max-over-bag: which bag row won each
head. The region query also selects bbox/kind/detector_version, a parallel
bag_meta maps each row → its region (None for the whole-image vector), and every
hit gains grounding {bbox,kind,detector} (null when the global vector won). Threaded
through SuggestionService (new Suggestion.grounding field) → /api/.../suggestions
payload. This is the data the #1206 hover-overlay draws. CCIP-only hits ground null
for now (figure grounding = step 2). Tests: winning crop grounds the tag with its
bbox+kind; whole-image win → grounding None.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
This commit is contained in:
2026-07-05 23:13:29 -04:00
parent ab362bc79c
commit 409724b981
4 changed files with 57 additions and 5 deletions
+25 -1
View File
@@ -187,7 +187,31 @@ async def test_concept_region_surfaces_via_max_over_bag(db):
))
await db.commit()
general = (await SuggestionService(db).for_image(img.id)).by_category["general"]
assert any(s.canonical_tag_id == tag.id and s.score > 0.7 for s in general)
s = next(x for x in general if x.canonical_tag_id == tag.id)
assert s.score > 0.7
# #1206: the winning crop grounds the tag — hover highlights THIS region
# (the matching-version crop at 0.4,0.4,0.3,0.3), not the whole image.
assert s.grounding is not None
assert s.grounding["bbox"] == pytest.approx([0.4, 0.4, 0.3, 0.3])
assert s.grounding["kind"] == "concept"
@pytest.mark.asyncio
async def test_grounding_none_when_whole_image_wins(db):
# #1206: when the whole-image vector (not a crop) produces the winning score,
# grounding is None — the tag came from the global vector, not a region.
tag = await TagService(db).find_or_create("sky", TagKind.general)
img = await _img(db, "d1" * 32, _emb(0)) # whole-image ALIGNED w/ head
await _head(db, tag.id, slot=0, suggest_threshold=0.5)
db.add(ImageRegion( # an orthogonal crop (0.5)
image_record_id=img.id, kind="concept",
rx=0.1, ry=0.1, rw=0.2, rh=0.2,
siglip_embedding=_emb(5), embedding_version=await _embver(db),
))
await db.commit()
general = (await SuggestionService(db).for_image(img.id)).by_category["general"]
s = next(x for x in general if x.canonical_tag_id == tag.id)
assert s.grounding is None
@pytest.mark.asyncio