feat(ccip): automation + reference quality — keep identity flowing hands-free (#114)
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m32s
CI / frontend-build (push) Successful in 19s

Works through the optional CCIP ideas + the "keep moving even if I forget" ask:

AUTOMATION (no button needed):
- Hourly beat auto-enqueues CCIP backfill — new images get embedded (and errored
  ones retried) on their own; the queue never goes idle waiting for a click.
- CCIP auto-apply: a daily sweep tags confident matches (source='ccip_auto') so
  identity tags keep flowing. ON by default (opt-out, like head auto-apply);
  ml_settings.ccip_auto_apply_enabled + _threshold (0.92, above the suggest cut),
  migration 0064. Vectorized (one matmul + reduceat per image), reversible, skips
  already-applied/rejected. Switch + threshold in the GPU agent card; GET/PATCH
  /api/ml/settings; auto_applied count in /api/ccip/overview.

REFERENCE QUALITY (the over-fire root cause):
- character_references now draws ONLY from single-character images — on a
  multi-character image the tag is image-level, so every figure would otherwise
  pollute each character's prototypes (a 2-char image tagged 'Velma' made
  Daphne's figure a Velma reference). This is the contamination behind residual
  over-firing.
- Cached on a cheap signature (char-tag count + ccip-region count/max-id) so the
  reference load isn't redone on every modal open.

Tests: multi-character image not used as a reference; auto-apply tags a confident
match as ccip_auto.

NEXT (not done, confirmed): comic-panel cropping + SigLIP concept crops ("spot
interesting content").

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
2026-06-29 22:25:40 -04:00
parent 74b7ceaf47
commit b91a230f12
9 changed files with 324 additions and 3 deletions
+49 -3
View File
@@ -13,7 +13,7 @@ exact CCIP difference metric/threshold gets validated against the model during
the hands-on eval. numpy is imported lazily (API worker has it via pgvector).
"""
from sqlalchemy import select
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from ...models import ImageRegion, MLSettings, Tag, TagKind
@@ -41,10 +41,54 @@ def _l2norm(mat, np):
return mat / n
# Single-shot cache of the (expensive) reference load, keyed on a cheap
# signature that changes exactly when references could: a character tag added/
# removed (n_char_tags) or a figure embedded (max/ n of ccip regions). Shared by
# the live matcher (every modal open) and the auto-apply sweep.
_REF_CACHE: dict = {"sig": None, "refs": None}
def _single_character_images():
"""Subquery of image ids carrying EXACTLY ONE character tag. References come
only from these — on a multi-character image the tag is image-level, so every
figure would otherwise pollute each character's prototype set (a 2-character
image tagged 'Velma' would make Daphne's figure a Velma reference)."""
return (
select(image_tag.c.image_record_id)
.join(Tag, Tag.id == image_tag.c.tag_id)
.where(Tag.kind == TagKind.character)
.group_by(image_tag.c.image_record_id)
.having(func.count() == 1)
)
async def _ref_signature(session: AsyncSession) -> tuple:
n_tags = (
await session.execute(
select(func.count())
.select_from(image_tag)
.join(Tag, Tag.id == image_tag.c.tag_id)
.where(Tag.kind == TagKind.character)
)
).scalar_one()
n_regs, max_id = (
await session.execute(
select(func.count(), func.max(ImageRegion.id)).where(
ImageRegion.kind.in_(_FIGURE_KINDS),
ImageRegion.ccip_embedding.is_not(None),
)
)
).one()
return (n_tags, n_regs, max_id)
async def character_references(session: AsyncSession) -> dict[int, list]:
"""Per character-tag CCIP reference vectors: figure/face-region CCIP
embeddings on images that carry that character tag (the operator's examples).
Multi-prototype — several vectors per character."""
embeddings on UNAMBIGUOUS (single-character) images carrying that tag.
Multi-prototype — several vectors per character. Cached on a cheap signature."""
sig = await _ref_signature(session)
if _REF_CACHE["sig"] == sig and _REF_CACHE["refs"] is not None:
return _REF_CACHE["refs"]
rows = (
await session.execute(
select(image_tag.c.tag_id, ImageRegion.ccip_embedding)
@@ -57,11 +101,13 @@ async def character_references(session: AsyncSession) -> dict[int, list]:
.where(Tag.kind == TagKind.character)
.where(ImageRegion.kind.in_(_FIGURE_KINDS))
.where(ImageRegion.ccip_embedding.is_not(None))
.where(ImageRegion.image_record_id.in_(_single_character_images()))
)
).all()
refs: dict[int, list] = {}
for tag_id, vec in rows:
refs.setdefault(tag_id, []).append(vec)
_REF_CACHE.update(sig=sig, refs=refs)
return refs