feat(suggestions): overlay CCIP character matches onto the rail (#114)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m32s

SuggestionService.for_image now merges CCIP character matches with the SigLIP
head suggestions — they're complementary, not exclusive: CCIP is the identity-
specialized signal but needs a detected figure; the heads work whole-image but
conflate identity with style. Merged by tag: 'both' when they corroborate
(higher score wins), 'ccip' / 'head' otherwise. Cheap when no CCIP vectors exist
yet (match_image returns early without a figure vector), so it's a no-op until
the agent runs. Suggestion.source is now 'head' | 'ccip' | 'both'.

Test: a character with a CCIP reference figure surfaces (source='ccip') on a new
image whose figure matches.

NEXT: the agent container (real CCIP/detector models, hands-on) that produces the
vectors this consumes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
2026-06-29 12:52:24 -04:00
parent d57ca847e7
commit 5faf34a3b5
2 changed files with 62 additions and 9 deletions
+33 -1
View File
@@ -4,7 +4,7 @@ scikit-learn, ml image only); scoring is numpy-only (available via pgvector)."""
import pytest
from sqlalchemy import select
from backend.app.models import ImageRecord, MLSettings, TagHead, TagKind
from backend.app.models import ImageRecord, ImageRegion, MLSettings, TagHead, TagKind
from backend.app.models.tag import image_tag
from backend.app.services.ml.allowlist import AllowlistService
from backend.app.services.ml.suggestions import SuggestionService
@@ -131,3 +131,35 @@ async def test_rejected_tag_surfaced_flagged_then_reversible(db):
sl2 = await SuggestionService(db).for_image(img.id)
s2 = next(x for x in sl2.by_category["general"] if x.canonical_tag_id == tag.id)
assert s2.rejected is False
async def _figure(db, image_id, slot):
v = [0.0] * 768
v[slot] = 1.0
db.add(ImageRegion(
image_record_id=image_id, kind="figure",
rx=0.0, ry=0.0, rw=1.0, rh=1.0,
ccip_embedding=v, embedding_version="ccip-test",
))
@pytest.mark.asyncio
async def test_ccip_character_surfaces_in_rail(db):
# A character with a CCIP reference (a tagged figure) is suggested on a new
# image whose figure matches — overlaid into the rail alongside the heads.
raven = await TagService(db).find_or_create("Raven", TagKind.character)
ref = await _img(db, "0" * 64, None) # the operator's tagged example
await _figure(db, ref.id, slot=0)
await db.execute(image_tag.insert().values(
image_record_id=ref.id, tag_id=raven.id, source="manual",
))
query = await _img(db, "1" * 64, None) # untagged, matching figure
await _figure(db, query.id, slot=0)
await db.commit()
sl = await SuggestionService(db).for_image(query.id)
m = next(
c for c in sl.by_category.get("character", [])
if c.canonical_tag_id == raven.id
)
assert m.source == "ccip"