Suggestions perf + UX polish: incremental CCIP + head training, hover label, scroll/toast fixes, dead-preview removal #196

Merged

bvandeusen merged 8 commits from dev into main 2026-07-06 16:41:08 -04:00

Conversation 0 Commits 8 Files Changed 25

+951 -354

bvandeusen commented 2026-07-06 16:41:02 -04:00

Owner

Copy Link

Copy Source

Batch of suggestion-pipeline efficiency + tagging-UX work.

Performance — recompute scales with change, not library size (#1317, milestone #138)

• CCIP references → precomputed incremental prototype store. Was: a per-request rebuild of every character's reference vectors, invalidated by every character accept (~4s stall, confirmed by live timing). Now: character_prototype (capped per character) + ccip_prototype_state (per-character fingerprint); a global gate skips when nothing changed, else only the changed characters rebuild — on a ~15-min beat and after every Retrain/nightly run. The matcher reads the store via a per-character incremental cache (legacy cold-start fallback). Migration 0079.
• Head training → incremental. The Retrain button now refits only the tags whose positives/rejections changed (a tag_head.train_fingerprint); the nightly run does a full=True reconcile for sampled-negative/hygiene drift. Migration 0080.

Tagging UX

• Grounding overlay label leads with the hovered tag; the crop origin (booru:head / panel / person-…) is demoted to a small subline.
• Tag rail no longer scroll-jumps to the top on accept/reject (focus with preventScroll); Explore rail padded so the snackbar can't cover the last suggestions.

Cleanup

• Removed the orphaned backfill dry-run preview (PreviewDialog + route + preview_source + ingester preview()): it was unreachable and low-value (auth check is covered by verify_source_credential, fetching by "Check now").

Two migrations (0079, 0080). CI green on dev head 2cfbb28. After deploy, give the CCIP prototype store ~15 min to populate (or hit Retrain) before judging suggestion speed.

🤖 Generated with Claude Code

Batch of suggestion-pipeline efficiency + tagging-UX work. **Performance — recompute scales with change, not library size (#1317, milestone #138)** - **CCIP references → precomputed incremental prototype store.** Was: a per-request rebuild of every character's reference vectors, invalidated by *every* character accept (~4s stall, confirmed by live timing). Now: `character_prototype` (capped per character) + `ccip_prototype_state` (per-character fingerprint); a global gate skips when nothing changed, else only the changed characters rebuild — on a ~15-min beat and after every Retrain/nightly run. The matcher reads the store via a per-character incremental cache (legacy cold-start fallback). Migration 0079. - **Head training → incremental.** The Retrain button now refits only the tags whose positives/rejections changed (a `tag_head.train_fingerprint`); the nightly run does a `full=True` reconcile for sampled-negative/hygiene drift. Migration 0080. **Tagging UX** - Grounding overlay label leads with the hovered **tag**; the crop origin (booru:head / panel / person-…) is demoted to a small subline. - Tag rail no longer scroll-jumps to the top on accept/reject (focus with `preventScroll`); Explore rail padded so the snackbar can't cover the last suggestions. **Cleanup** - Removed the orphaned backfill dry-run preview (`PreviewDialog` + route + `preview_source` + ingester `preview()`): it was unreachable and low-value (auth check is covered by `verify_source_credential`, fetching by "Check now"). Two migrations (0079, 0080). CI green on dev head `2cfbb28`. After deploy, give the CCIP prototype store ~15 min to populate (or hit Retrain) before judging suggestion speed. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 8 commits 2026-07-06 16:41:02 -04:00

refactor(subscriptions): remove the dead backfill dry-run preview (#1281) ... 8f6547f8d7

PreviewDialog.vue was orphaned — nothing mounted it — so the dry-run preview
was unreachable. Rather than wire it up, remove the whole chain: its unique
value (a capped 3-page count of what a backfill would grab) is low, and its
adjacent needs are already covered — auth validation by verify_source_credential,
and actually fetching recent posts by "Check now". Operator decision 2026-07-06.

Removed:
- frontend: PreviewDialog.vue + sources store previewSource()
- backend: POST /api/sources/<id>/preview route, download_backends.preview_source,
  IngestCore.preview() + its now-unused NativeIngestError import
- tests: the 3 ingester preview tests

Nothing else referenced the chain (verified). Shared campaign-resolution and
ledger helpers stay — they're used by run()/verify_source_credential.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM

feat(ui): grounding overlay leads with the hovered tag, crop origin as subtext (#1206) ... 7939dba9ed

The overlay label showed only the crop origin (booru:head / panel / person-m).
Put the hovered TAG on top as the headline and demote the crop origin to a small,
dimmed, monospace subline — the tag is what you're evaluating; the origin is just
provenance. Threads the tag name through the fcSuggestionHover payload ({g, tag})
from both setters (SuggestionItem for suggestions, TagChip for applied chips).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM

fix(ui): tag-rail no longer scroll-jumps on accept/reject; keep suggestions clear of the toast ... 48be921b8d

- TagAutocomplete: focus the inner input with preventScroll so handing focus
  back after an accept/reject stops yanking the rail up to the field (which sits
  above the suggestions list — you had to scroll back down every time). Applies
  to both the image modal and the Explore rail.
- ExploreView: pad the right rail's scroll bottom (88px) so the bottom-right
  snackbar floats over empty space instead of covering the last suggestions and
  their accept/reject controls.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM

feat(ccip): schema for precomputed incremental character prototypes (#1317, m138 step 1) ... f24dc81764

Foundation for making CCIP character references a precomputed, INCREMENTAL
artifact instead of a request-path rebuild (kills the per-accept ~4s suggestions
stall; cost will scale with change, not library size):

- character_prototype: a character's reference CCIP vectors, capped to
  MLSettings.ccip_prototype_cap so match cost doesn't grow with popularity.
- ccip_prototype_state: per-character fingerprint (ref count + max region id) +
  updated_at → drives per-character incremental rebuilds and the matcher cache's
  reload-only-what-advanced.
- MLSettings.ccip_ref_signature (cheap global change gate) + ccip_prototype_cap.

Migration 0079. Schema + models only — the builder service, refresh task/beat,
and matcher rewrite land in the following steps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM

feat(ccip): incremental character-prototype builder (#1317, m138 step 2) ... 9504870c9a

refresh_character_prototypes (sync, celery ml worker):
- Cheap GLOBAL gate (a few COUNTs) → no-op when nothing that affects references
  changed since the last refresh (the operator's "only recompute if something
  was tagged" trigger).
- Else a per-character fingerprint diff (one GROUP BY: ref count + max region id)
  rebuilds ONLY the characters whose references moved — each capped to
  MLSettings.ccip_prototype_cap — and drops characters that lost all refs.
Cost scales with WHAT changed, not library size. Reuses ccip's reference
predicate (single-character, non-hygiene, figure CCIP) so prototypes match the
legacy matcher exactly. The async matcher (next step) will READ the table.

Tests: gate no-op when idle, only-changed-character rebuild, capping,
single-character exclusion, lost-reference cleanup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM

feat(ccip): refresh task + beat + retrain hook for character prototypes (#1317, m138 step 3) ... a1ed53136e

- refresh_character_prototypes celery task wraps the incremental builder (sync
  ml worker); returns skipped / rebuilt=N removed=N.
- Beat: every ~15 min (cheap global-gate no-op when idle) + a nightly full=True
  reconcile as belt-and-suspenders.
- train_heads enqueues it on success, so the Retrain button AND the nightly head
  retrain refresh CCIP on the SAME trigger — unified lifecycle, as asked.
The initial (cold) full build loads the whole reference set once in the
background, never on a /suggestions request.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM

feat(ccip): matcher reads the incremental prototype store (#1317, m138 step 4) ... a94f6a2789

match_image now sources character references from character_prototype via a
per-character in-process cache (_load_prototypes) that reloads ONLY the
characters whose ccip_prototype_state.updated_at advanced — no request-path
rebuild, so the per-accept ~4s stall is gone once the store is populated. Cold
start (store empty pre-first-refresh) falls back to the legacy on-the-fly
reference build, so character suggestions work immediately post-deploy and the
background refresh populates the store within ~15 min. Match math + grounding
are unchanged; existing tests exercise the legacy fallback, and a new test
covers matching from the populated prototype store.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM

feat(heads): incremental retraining — refit only changed tags (#1317 phase 2, m138) ... 2cfbb284d5

train_all_heads is now incremental by default: a per-tag training-data
fingerprint (positive + rejection count/latest-timestamp, stored on
tag_head.train_fingerprint) means a manual Retrain refits ONLY the tags whose
data changed — O(what you touched), not O(all heads). The nightly
scheduled_train_heads passes full=True to reconcile sampled-negative + hygiene
drift across every head. First incremental run after deploy still refits
everyone (NULL fingerprints), stamping them, then it's incremental.

The refit decision + fingerprint are split into sklearn-free helpers
(_head_fingerprints, _heads_needing_retrain) so the incremental logic is
unit-tested directly (train_head itself needs scikit-learn). Migration 0080.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM

bvandeusen merged commit d68f39ca50 into main 2026-07-06 16:41:08 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#196