Tag-hover → crop-region grounding overlay (#1206, milestone #133) #195

Merged

bvandeusen merged 5 commits from dev into main 2026-07-06 13:29:31 -04:00

Conversation 0 Commits 5 Files Changed 14

+494 -41

5 Commits

Author	SHA1	Message	Date
bvandeusenandClaude Opus 4.8	2638cf1a35	style(tests): hoist test_api_tags grounding imports to module level (ruff I001) ... CI ruff flagged the in-function import block; move them to the top mirroring test_ml_suggestions.py's import line, which ruff already accepts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-06 13:24:55 -04:00
bvandeusenandClaude Opus 4.8	9bb4211722	feat(ui): hover an applied tag chip → highlight its grounding crop (#133 step 4) ... Applied tags aren't scored live, so compute the grounding on demand: run the tag's head over the image's max-over-bag (whole-image + concept crops), argmax → the region that best explains the tag on this image, mirroring what score_image records for live suggestions. - heads.py: extract _image_bag (now shared by score_image) + ground_applied_tag. Returns (grounding, has_head): has_head False = no head to localize with → no overlay; grounding None = the whole-image vector won → whole-image frame. - tags.py: GET /api/images/<id>/tags/<id>/grounding → {grounding, has_head}. - TagChip/TagPanel: applied chips inject fcSuggestionHover and fetch grounding on hover (cached per image+tag, race-guarded), reusing Step 3's overlay in both the modal and Explore. No new frontend overlay code. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-06 13:19:41 -04:00
bvandeusenandClaude Opus 4.8	524a26c618	feat(ui): hover a suggestion → highlight the crop region it came from (#133 step 3) ... The payoff: hover a suggestion in the rail and the exact crop that produced it lights up on the image (a booru:head, a panel, a figure); a null-grounding tag shows a subtle dashed whole-image frame ('global vector won, not a crop'). ImageCanvas gains a grounding overlay that tracks the <img>'s live bounding rect (correct under object-fit letterboxing + pan/zoom) and draws the normalized bbox + a detector/kind label. SuggestionItem sets the hovered grounding via provide/inject (no 4-level event relay through TagPanel/SuggestionsPanel/group); ImageViewer AND ExploreView provide it + pass it to their canvas. Overlay is pointer-events:none so it never blocks pan/zoom/click. Videos out of v1 scope. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-05 23:24:18 -04:00
bvandeusenandClaude Opus 4.8	dfe2fda564	feat(ml): CCIP character matches ground to the matched figure region (#133 step 2) ... match_image now tracks WHICH query figure produced the winning cosine per character (argmax over the per-figure best-reference sim) and attaches its bbox as grounding {bbox,kind:'figure',detector}. SuggestionService carries it: a CCIP-only character hit grounds to its figure; a 'both' hit keeps the head's localized crop if it had one, else falls back to the CCIP figure — so corroborated characters stay grounded. Test: a character match carries the matched figure's bbox+kind. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-05 23:18:41 -04:00
bvandeusenandClaude Opus 4.8	409724b981	feat(ml): argmax grounding in score_image → suggestions carry the winning crop (#133 step 1) ... score_image now keeps the ARGMAX beside the max-over-bag: which bag row won each head. The region query also selects bbox/kind/detector_version, a parallel bag_meta maps each row → its region (None for the whole-image vector), and every hit gains grounding {bbox,kind,detector} (null when the global vector won). Threaded through SuggestionService (new Suggestion.grounding field) → /api/.../suggestions payload. This is the data the #1206 hover-overlay draws. CCIP-only hits ground null for now (figure grounding = step 2). Tests: winning crop grounds the tag with its bbox+kind; whole-image win → grounding None. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-05 23:13:29 -04:00