feat(tagging): SigLIP concept crops + max-over-bag scoring (#114)
Lift recall on small/local concepts (glasses, cum, stomach-bulge, xray, lactation) that the whole-image SigLIP vector washes out: the GPU agent now embeds figure crops with SigLIP too, stored as kind='concept' regions, and the suggestion rail scores each image as a BAG (whole-image + every concept crop), taking each head's MAX over the bag. The whole-image vector is always in the bag, so this can never score lower than before. Model-agnostic by construction: the server ANNOUNCES the embedding model (HF name + version) in the lease, so the agent loads whatever the heads were trained in and stays in lock-step — a model swap is a server setting + a re-embed migration, never an agent change. - agent: model-agnostic CropEmbedder (torch/transformers get_image_features, fp16 on CUDA, inference-locked); worker branches on job.task — 'ccip' emits figure(CCIP)+concept(SigLIP) in one pass, 'siglip' emits concept-only so the back-catalogue backfill never churns figure/CCIP regions; torch cu124 + transformers in the image. - server: lease announces embed_model_name/embed_version; score_image is max-over-bag (version-filtered region embeddings); enqueue_gpu_backfill 'siglip' gates on a missing concept region (drains the back-catalogue, retries failures, no double-enqueue); daily siglip-backfill beat; UI button; /api/ccip/overview reports images_with_concept_siglip. - v1 scope: suggestion rail only — auto-apply stays whole-image (conservative; heads' thresholds were calibrated on whole-image). Bulk-apply bag = follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
@@ -61,6 +61,16 @@
|
||||
processes until the agent is running.
|
||||
</p>
|
||||
|
||||
<v-btn
|
||||
class="mt-3" color="accent" variant="tonal" rounded="pill" size="small"
|
||||
prepend-icon="mdi-crop" :loading="backfillingSiglip" @click="onBackfillSiglip"
|
||||
>Queue concept crops (SigLIP)</v-btn>
|
||||
<p class="fc-muted text-caption mt-2 mb-0">
|
||||
Enqueues every image that doesn't have concept-crop embeddings yet — the
|
||||
localized vectors that help small/local tags (glasses, etc.) surface. New
|
||||
images get these automatically; this catches the back-catalogue.
|
||||
</p>
|
||||
|
||||
<!-- Match strictness -->
|
||||
<div class="fc-section-h mt-5 mb-1">Character-match strictness</div>
|
||||
<div v-if="ml.settings" class="d-flex align-center" style="gap:12px">
|
||||
@@ -115,6 +125,7 @@ const tokenValue = ref(null)
|
||||
const masked = ref(true)
|
||||
const rotating = ref(false)
|
||||
const backfilling = ref(false)
|
||||
const backfillingSiglip = ref(false)
|
||||
const threshold = ref(0.85)
|
||||
const savingThreshold = ref(false)
|
||||
const autoApply = ref(true)
|
||||
@@ -215,6 +226,19 @@ async function onBackfill() {
|
||||
backfilling.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function onBackfillSiglip() {
|
||||
backfillingSiglip.value = true
|
||||
try {
|
||||
await store.backfill('siglip')
|
||||
toast({ text: 'Queued concept crops — run the agent to process them', type: 'success' })
|
||||
await refreshQueue()
|
||||
} catch (e) {
|
||||
toast({ text: `Could not queue backfill: ${e.message}`, type: 'error' })
|
||||
} finally {
|
||||
backfillingSiglip.value = false
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<style scoped>
|
||||
|
||||
Reference in New Issue
Block a user