Release: Explore diversification, model-swap, Camie/centroid retirement, tag-API, crop proposers #155

Merged

bvandeusen merged 10 commits from dev into main 2026-06-30 16:06:13 -04:00

Conversation 0 Commits 10 Files Changed 65

+1299 -2535

bvandeusen commented 2026-06-30 16:06:06 -04:00

Owner

Copy Link

Copy Source

Promote dev→main (all green on dev). Batch since the last release:

• Explore diversification (#1188) — pHash dedup + MMR so neighbours span clusters.
• Operator model-swap flow (#1190) — embedder is a setting; GPU agent re-embeds; SigLIP 2 (so400m-patch16-512) is a 1152-d drop-in. Migration 0065.
• Centroid removal + Camie/allowlist retirement (#1189) — heads + CCIP are the tag source; accept applies to the image only (no allowlist fan-out). Migrations 0066, 0067.
• Dead-settings hygiene (#1199) — drop inert tagger/suggestion columns. Migration 0068.
• Agent tag-analysis API (#1136) — /api/tags/top, /api/tags/<id>/stats.
• Crop proposers (#1202) — agent gains booru_yolo anatomy + COCO-person merge + comic-panel detectors feeding the SigLIP bag (ultralytics; weights defaulted, self-disabling).
• Agent redeploy-survival (earlier).

Deploy: force re-pull web/ml :latest (runs migrations 0065→0068); then agent docker compose pull && docker compose up -d (new image bundles the detectors). Migrations are additive/drops, no data migration risk.

🤖 Generated with Claude Code

Promote dev→main (all green on dev). Batch since the last release: - **Explore diversification** (#1188) — pHash dedup + MMR so neighbours span clusters. - **Operator model-swap flow** (#1190) — embedder is a setting; GPU agent re-embeds; SigLIP 2 (so400m-patch16-512) is a 1152-d drop-in. **Migration 0065.** - **Centroid removal + Camie/allowlist retirement** (#1189) — heads + CCIP are the tag source; accept applies to the image only (no allowlist fan-out). **Migrations 0066, 0067.** - **Dead-settings hygiene** (#1199) — drop inert tagger/suggestion columns. **Migration 0068.** - **Agent tag-analysis API** (#1136) — `/api/tags/top`, `/api/tags/<id>/stats`. - **Crop proposers** (#1202) — agent gains booru_yolo anatomy + COCO-person merge + comic-panel detectors feeding the SigLIP bag (ultralytics; weights defaulted, self-disabling). - Agent redeploy-survival (earlier). **Deploy:** force re-pull web/ml `:latest` (runs migrations **0065→0068**); then agent `docker compose pull && docker compose up -d` (new image bundles the detectors). Migrations are additive/drops, no data migration risk. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 10 commits 2026-06-30 16:06:06 -04:00

fix(explore): diversify "more like this" so it stops getting stuck (#1188) ... 0f472b2f9e

Pure nearest-cosine piled near-identical images into the neighbour grid — a
reposted banner filled all 24 slots, and once you wandered into a B&W /
comic-panel cluster every neighbour was more of the same with no way back to
colour without the Random button (operator-reported, with screenshot).

similar() now over-fetches a wide candidate pool (5x the requested limit, cap
200), then diversifies down to `limit`:
- pHash near-duplicate collapse: drop candidates within 6 Hamming bits of the
  anchor or an already-kept candidate, so a repost (and the anchor's own clones)
  appears at most once.
- MMR re-rank: greedily pick for closeness-to-anchor minus similarity-to-already
  -picked (lambda 0.55), so the result SPANS clusters instead of returning 40
  variations of one image. Falls back to nearest-order on any failure / small
  pool, so existing nearest-first behaviour is unchanged when there's nothing to
  diversify.

Frontend forwardTarget drops the now-redundant skip-nearest-third hack (the list
is already diversified server-side) — plain random-over-unvisited gives the
variance now.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa

feat(ml): operator model swap — GPU re-embed + embedder as a setting (#1190) ... 4daa3f2790

Make the SigLIP embedder an operator choice (drop-in to SigLIP 2:
google/siglip2-so400m-patch16-512 is a verified 1152-d model at 512px → no
schema change, better small-cue fidelity). A swap = set model + re-embed +
retrain, all operator-driven; the GPU agent does the re-embed so it's fast.

- settings: embedder_model_name is now a setting (migration 0065) alongside the
  existing embedder_model_version; both editable + validated (non-empty) in the
  ml admin API. The server embedder loads by HF name (AutoImageProcessor/Model,
  model-agnostic), preferring the pre-downloaded local dir for the default so
  existing deploys don't re-download; rebuilds on a name change.
- agent: new 'embed' job = whole-image SigLIP embedding (mean-pool video frames)
  under the lease-announced model → POST /jobs/submit_embedding writes
  image_record.siglip_embedding + siglip_model_version. The lease now announces
  the model FROM THE SETTING (not a constant).
- re-embed routing: enqueue_gpu_backfill('embed') selects unembedded + stale-
  version images; 'siglip' now re-embeds concept crops whose version != current
  (so a swap re-triggers crops, not just the never-embedded back-catalogue). The
  CPU ml-worker backfill no longer re-embeds on a version mismatch (it can't
  churn the library at 512px) — the GPU agent owns version re-embeds. Daily
  'embed' + 'siglip' beats self-heal.
- scoring: score_image only bags embeddings in the CURRENT model's space (whole-
  image gated by siglip_model_version, concept regions by embedding_version) so a
  mid-swap stale vector isn't scored by new-space heads; legacy NULL = current.
- UI: GpuAgentCard "Embedding model (advanced)" — edit name/version, Save, and
  "Re-embed library (GPU)" (queues embed + siglip); points at SigLIP 2.

Tests: lease announces model + submit_embedding round-trip; enqueue 'embed'
selects stale/unembedded; stale-version excluded from scoring; embedder model
settable + empty rejected; siglip gate updated to current-version concept.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa

refactor(ml): remove the dead per-tag centroid subsystem (#1189) ... 3d77a38a25

The v2 pivot replaced per-tag SigLIP centroids with learned heads + CCIP.
Centroids were still recomputed (on every tag merge + a daily beat) but NOTHING
read them — suggestions come from heads+CCIP and apply_allowlist_tags applies
via Camie predictions, not centroids. Pure dead wiring; remove it.

Removed: CentroidService, recompute_centroid/recompute_centroids tasks, the
daily beat, POST /api/ml/recompute-centroids, the recompute-on-merge trigger,
the tag_reference_embedding table + model, the centroid_similarity_threshold +
min_reference_images settings (migration 0066), the CentroidRecomputeCard +
its store action + MaintenancePanel tile, and the centroid slider in
MLThresholdSliders. _keep_as_alias drops its vestigial has-centroid branch (the
allowlist branch already covers "could re-emit"); tag merge no longer clears a
table that no longer exists.

NOT touched (still live, parallel to heads): the Camie tagger, ImagePrediction,
and the allowlist bulk-apply — accepting a suggestion still allowlists + applies
it across the library. The tag-eval "centroid" baseline metric is unrelated
(in-memory) and stays. (image_record.centroid_scores JSON column also remains —
separate legacy field, its own micro-cleanup.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa

refactor(ml): retire the Camie tagger + allowlist bulk-apply (#1189) ... 485387ff0b

Heads + CCIP are the tag source and head auto-apply is the earned propagation.
The Camie tagger ran only to feed the allowlist bulk-apply (its ImagePrediction
rows had no other consumer), and the allowlist was a SECOND, un-earned auto-apply
path firing in parallel with heads on every accept — exactly the un-earned spray
the v2 pivot replaced. Retire both.

Behavior change: accepting a suggestion now applies the tag to THAT image only
(source='ml_accepted', a head-training positive) — it no longer allowlists +
fans the tag across the library via Camie. Propagation is heads' earned
auto-apply. (Loses instant cold-start propagation for booru-vocab tags; that was
un-earned and bypassed the precision gate.)

- tag_and_embed is now EMBED-ONLY (no Camie load/infer, no ImagePrediction
  writes); backfill enqueues it for images with no embedding.
- Removed: services/ml/tagger.py, apply_allowlist_tags + helpers + daily beat +
  every enqueue caller (accept/alias/merge/per-image), api/allowlist.py +
  blueprint, ImagePrediction + TagAllowlist models/tables (migration 0067),
  AllowlistTable.vue + allowlist store, the accept coverage-projection payload.
- AllowlistService gutted to accept/dismiss/undismiss/reject (the rejection store
  the rail still needs); accept returns nothing, API returns {accepted, tag_id}.
- tag merge no longer repoints/triggers the allowlist; _keep_as_alias now keys on
  ML-applied image_tag sources (incl. head_auto) instead of the allowlist.
- UI: MLBackfillCard relabelled to embedding-only; accept toast simplified;
  MaintenancePanel drops the allowlist tile.

Left for a follow-up hygiene pass (now-inert, harmless): the dead settings
columns (tagger_store_floor, tagger_model_version, suggestion_threshold_*,
video_min_tag_frames), image_record.tagger_model_version, MLThresholdSliders
trim, and the Camie model download in download_models.py.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa

fix(lint): drop unused select import in tags.py after allowlist removal ... 3d97667f5b

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa

refactor(ml): drop dead tagger/suggestion settings + columns (#1199) ... bc6d43d3f2

Hygiene follow-up to the Camie retirement (#1189) — these were left inert to
bound that change; nothing reads them now. Migration 0068 drops:
- ml_settings: tagger_store_floor, tagger_model_version, suggestion_threshold_
  character/general (already dead pre-retirement — scoring uses per-head
  thresholds), video_min_tag_frames (only the deleted video-prediction
  aggregator used it).
- image_record: tagger_model_version (no writer), centroid_scores (dead JSON
  cache, no reader).

Also: ml_admin _EDITABLE/GET/_validate pruned (dropped the store-floor invariant
+ video_min_tag_frames check); MLThresholdSliders trimmed to a video-embedding
card (interval + max frames only); importer no longer resets the dropped cols;
download_models drops the Camie fetch; stale CASCADE comments in cleanup_service
no longer name the removed tables. Tests updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa

feat(api): agent-friendly tag analysis endpoints — /tags/top + /tags/<id>/stats (#1136) ... 9a3cda007a

Fast, read-only, indexed aggregates shaped for ANALYSIS (not the paged UI
directory, which is alphabetical + builds previews and timed out at 10 min on a
full count sweep).

- GET /api/tags/top — top tags by image count, desc. ?kind, ?limit (cap 500),
  ?min_count, ?source=all|human|manual|accepted|auto (human=manual+ml_accepted,
  auto=head_auto+ccip_auto+ml_auto). One GROUP BY over image_tag (indexed on
  tag_id).
- GET /api/tags/<id>/stats — per-tag dataset health: total + per-source counts
  (manual/accepted/head_auto/ccip_auto), human vs auto rollups, rejection count,
  and whether a trained head exists. Backs concept-readiness + source-split
  analysis.

Plain-HTTP homelab posture, no auth change. Tests cover ranking, source filter,
min_count, the source breakdown, and 404.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa

fix(lint): use dict() not a dict-comprehension in tag_stats (C416) ... 3d7f60a6e3

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa

feat(agent): crop proposers — booru_yolo anatomy + COCO person + comic panels (#1202) ... d5f29f7056

Better region PROPOSERS feeding the existing crop→SigLIP→max-over-bag heads (no
change to the learned-tagging approach; no per-tag cost — propose once, embed
each region, all heads in one matmul).

- detectors.py: lazy ultralytics YOLO wrapper, each proposer independently
  optional + guarded (a bad weight spec / inference error self-disables that one,
  logged, never breaks the worker). Weights resolve from an ultralytics name |
  http(s) URL | "hf_repo::file", cached under HF_HOME. NMS merge so a figure two
  detectors both find collapses to one crop.
- worker: figure boxes = imgutils detect_person ∪ general COCO person (merged)
  → CCIP + concept (anime + Western/realistic coverage); booru_yolo anatomy
  components (head/cat-head/anatomy/…) → concept crops; comic panels → kind=
  'panel' concept crops. Capped per frame (MAX_COMPONENTS/MAX_PANELS).
- config + compose: PERSON_WEIGHTS (default yolo11n.pt, works OOB),
  ANATOMY_WEIGHTS + PANEL_WEIGHTS (operator sets booru_yolo URL + mosesb panel
  hf::file; empty = off). ultralytics added to requirements.
- backend: image_region 'kind' doc notes 'panel'; no migration (free String,
  and the bag scorer keys on a non-null siglip_embedding, not the kind, so any
  SigLIP region joins the bag automatically).

Agent is outside CI — py-compiled here; operator tests on the GPU and checks
Western-vs-anime crop quality via /api/ccip observability.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa

chore(agent): default the anatomy + panel proposer weights to working values ... ce5db5caaf

booru_yolo yolov11m_aa22.pt (40MB) + mosesb/best-comic-panel-detection::best.pt.
Each self-disables if its download fails, so defaulting them on is safe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa

bvandeusen merged commit 9a979ee808 into main 2026-06-30 16:06:13 -04:00

bvandeusen referenced this pull request 2026-06-30 16:13:30 -04:00

Release: GPU re-process trigger (apply new crop detectors to the existing library) #156

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#155