feat: ML tag suggestions, character/fandom integrity, underscores, modal polish
Consolidated merge of feat/tag-suggestions branch. Original 64-commit history was lost to git-object corruption in a Nextcloud-synced checkout; this single commit captures the equivalent diff. Includes: - pgvector-backed tag suggestion infra (WD14 + SigLIP centroids, ml-worker container, Celery tasks, suggestion service, accept/reject endpoints + modal UI with green/red chip buttons) - Character/fandom integrity: title-case normalization on every write path, fandom-id backfill, maintenance task + settings button, migrations g26041901 + h26041901 to canonicalize legacy rows with case-only duplicate merging - Tag-underscores + modal polish: WD14 name canonicalization at emit + accept + add/bulk-add paths, migration i26041901 for legacy-row rename-or-merge across character/fandom/NULL kinds, suggestion-accept refresh parity via awaited loadTags, persistent chip tint
This commit is contained in:
@@ -0,0 +1,100 @@
|
||||
# Clickable Post Tag in Gallery Modal
|
||||
|
||||
**Status:** Approved, ready for implementation planning
|
||||
**Date:** 2026-04-18
|
||||
**Scope:** Gallery/showcase image view modal (`_gallery_modal.html`)
|
||||
|
||||
## Problem
|
||||
|
||||
When viewing an image in the gallery/showcase modal, the user frequently wants to revisit the original post (Patreon/SubscribeStar/HentaiFoundry) the image came from, to re-read context. Today the post tag is rendered as a plain, non-clickable `<span>` in the editable tag list — there is no path from the modal to the post.
|
||||
|
||||
Secondary concern: the post tag is system-generated and shouldn't appear inside the editable tag area, where the `×` remove button invites deletion of metadata the user didn't author.
|
||||
|
||||
## Goal
|
||||
|
||||
Make the post tag in the image modal a clickable chip that navigates to the gallery filtered by that post tag. The filtered gallery view already surfaces the post's title, description, and source URL (`main.py:143-157`), so click-to-filter is the shortest path to the original post.
|
||||
|
||||
Move the post tag out of the editable tag list into a dedicated "provenance" area so it reads as metadata, not a user tag.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Opening the external post URL directly from the modal. Navigation goes to the filtered gallery, which is where `source_url` is already presented.
|
||||
- Making `source` (🌐) or `archive` (🗜️) tags clickable in the modal. Will reconsider after the post-tag treatment is validated.
|
||||
- Changes to `tag-editor.js` / the tags-list page's edit modal. Different modal.
|
||||
- New backend endpoints.
|
||||
|
||||
## Design
|
||||
|
||||
### Placement
|
||||
|
||||
A new section `modal-provenance-section` is inserted into the modal sidebar **between** the existing Series section and Tags section.
|
||||
|
||||
- Section name is forward-looking: "provenance" accommodates source/archive later without renaming.
|
||||
- The section is rendered only when the current image has at least one post tag. When empty, it is fully hidden — no placeholder, no heading.
|
||||
|
||||
### Chip content
|
||||
|
||||
One chip per post tag (in practice zero or one). Chip structure:
|
||||
|
||||
- 📌 icon prefix
|
||||
- Display label — see rules below
|
||||
- Trailing `↗` affordance to signal the chip navigates away from the modal
|
||||
|
||||
**Display label rules.** `getTagDisplayName` today only strips prefixes for `artist|character|series|fandom|rating`; post tags fall through and yield `platform:artist:id`, which is ugly. Label resolution for the provenance chip:
|
||||
|
||||
1. **Before metadata resolves:** show the raw tag name split once on `:` (the `platform:artist:id` portion). Ugly but functional placeholder.
|
||||
2. **After metadata resolves:** prefer `pm.artist`; if absent, fall back to `pm.title`; if both absent, keep the placeholder.
|
||||
|
||||
The upgrade from placeholder to metadata-driven label happens in the same render pass that applies the platform accent color.
|
||||
|
||||
### Chip behavior
|
||||
|
||||
- Click target: the whole chip, as an `<a href="/gallery?tag=<encoded post tag name>">`
|
||||
- Navigation: same tab (consistent with other tag chips in the app)
|
||||
- No `×` remove button — system tag, not user-editable
|
||||
|
||||
### Chip styling — platform-tinted ghost chip
|
||||
|
||||
Visually distinct from the standard editable tag chip (which is light-on-light with a `×` on hover). The provenance chip is a ghost chip with a platform-colored accent.
|
||||
|
||||
- **Base:** transparent background, 1px border `rgba(255, 255, 255, 0.15)`, light text
|
||||
- **Platform accent:** a 2px colored left-border driven by `PostMetadata.platform`:
|
||||
- `patreon` → Patreon orange (`#f96854`)
|
||||
- `subscribestar` → teal (`#009cde`)
|
||||
- `hentaifoundry` → red (`#c8232c`)
|
||||
- Unknown / missing metadata → neutral fallback, `rgba(255, 255, 255, 0.4)`
|
||||
- **Hover:** border brightens to `rgba(255, 255, 255, 0.35)`, background `rgba(255, 255, 255, 0.04)`
|
||||
- Platform color is applied via a `data-platform="<platform>"` attribute on the chip plus CSS attribute selectors — no inline styles.
|
||||
|
||||
### Data flow
|
||||
|
||||
1. `view-modal.js` already calls `loadTags(imageId)` when the modal opens and when navigating between images.
|
||||
2. Extend the tag load: partition the returned tags into
|
||||
- `postTags` — `tag.kind === 'post'`
|
||||
- `editableTags` — everything else
|
||||
3. Render `editableTags` into the existing `#modalTagList` container (unchanged behavior, minus the post chip).
|
||||
4. Render `postTags` into a new `#modalProvenanceList` container inside `modal-provenance-section`. Show the section iff `postTags.length > 0`; hide it otherwise.
|
||||
5. For each post tag, resolve its platform by calling the existing endpoint `/api/post-metadata/by-tag-name/<tag_name>`, which returns `{ platform, source_url, ... }`. The chip renders immediately with the neutral fallback accent, then upgrades to the platform-tinted accent when metadata resolves. This keeps the modal responsive even on a slow metadata fetch.
|
||||
6. Clean up provenance rendering on modal close and on image navigation, the same way the tag list is cleaned up today.
|
||||
|
||||
### Components to touch
|
||||
|
||||
- `app/templates/_gallery_modal.html` — add the `modal-provenance-section` with a `#modalProvenanceList` container between the Series and Tags sections.
|
||||
- `app/static/js/view-modal.js` — partition tags on load, render provenance chips, fetch platform metadata per post tag, handle show/hide of the section, reset on close/navigate.
|
||||
- `app/static/style.css` — `.modal-provenance-section`, `.provenance-chip`, platform-specific `[data-platform="..."]` accent rules, hover state.
|
||||
|
||||
### Error handling
|
||||
|
||||
- Metadata endpoint fails, returns 404, or returns no metadata record → chip renders with neutral fallback accent. The chip remains clickable; the filter-by-tag navigation works without platform data.
|
||||
- Post tag exists but its name is malformed → chip still renders with `getTagDisplayName` output and fallback accent; click still navigates to `/gallery?tag=<raw name>`.
|
||||
|
||||
## Risks and considerations
|
||||
|
||||
- Tag-kind drift: if another part of the codebase ever assumes every tag in the modal is user-editable (has a `×` button), splitting the list could surprise it. Mitigation: grep for `#modalTagList` consumers and confirm only view-modal.js reads it.
|
||||
- Duplicate clicks during metadata fetch: chip is clickable before metadata arrives — fine, because click navigation doesn't depend on metadata.
|
||||
- Future extension to source/archive: section and class names are generic (`provenance`), so adding them later is additive and doesn't require renames.
|
||||
|
||||
## Future work (deferred)
|
||||
|
||||
- Consider moving `source` (🌐) and `archive` (🗜️) tags into the same provenance section once the post-tag treatment is polished.
|
||||
- Consider a direct "open original post" action (using `source_url`) as a secondary affordance — only if click-to-filter proves insufficient.
|
||||
@@ -0,0 +1,252 @@
|
||||
# Tag Suggestions — ImageRepo Integration Spec
|
||||
|
||||
**Parent spec:** `docs/superpowers/tag-suggestion-system-spec.md`
|
||||
**Status:** Design approved 2026-04-18. Ready for implementation planning.
|
||||
**Scope this phase:** Scan pipeline + modal suggestion UI. Multi-edit / gallery bulk suggestions deferred.
|
||||
|
||||
This document layers ImageRepo-specific integration decisions on top of the parent spec. Read the parent spec first for model choice, schema rationale, threshold config, and model versioning; this spec assumes all of that is still in effect.
|
||||
|
||||
---
|
||||
|
||||
## 1. System layout
|
||||
|
||||
A new Docker Swarm service joins the existing stack:
|
||||
|
||||
- `ml-worker` — Celery worker on a new `ml` queue. Shares the existing Redis broker and Postgres database. Runs **CPU inference only** (Swarm does not expose GPUs cleanly and this is a single-user system; per-upload latency is async, backfill runs once overnight).
|
||||
|
||||
Existing services are unchanged. The `worker` container stays on its current queues (`import`, `thumbnail`, `sidecar`, `default`); the `scheduler` keeps `maintenance` and `scan`.
|
||||
|
||||
### Image
|
||||
|
||||
`ml-worker` uses its own Dockerfile (`Dockerfile.ml` or similar) so the main `worker` image stays lean:
|
||||
|
||||
- Base: `python:3.12-slim`
|
||||
- System deps: `libgl1`, `libglib2.0-0` (for Pillow / OpenCV if used for preprocessing)
|
||||
- Python deps (new `requirements-ml.txt`): `onnxruntime` (CPU build), `numpy`, `Pillow`, `huggingface_hub` (to pull models at build time), plus everything in the base `requirements.txt` that the Celery app imports
|
||||
- Models baked into the image at build time under `/models/` (WD14 ONNX + SigLIP-SO400M ONNX). Fetching at build time avoids cold-start latency and network dependency at runtime. Image size will be ~2-3 GB; acceptable for a persistent service.
|
||||
|
||||
### Resource sizing
|
||||
|
||||
- CPU: at least 4 cores pinned via `deploy.resources.limits.cpus`
|
||||
- RAM: 6-8 GB (ONNX graph + model weights + image batch preprocessing headroom)
|
||||
- Celery concurrency: **1**. ONNX Runtime will use multiple CPU threads internally via `intra_op_num_threads`; multi-worker concurrency just fights for the same cores.
|
||||
|
||||
---
|
||||
|
||||
## 2. Database
|
||||
|
||||
Follows the parent spec's schema, adapted to ImageRepo conventions:
|
||||
|
||||
- Primary keys use `Integer`, not `BIGSERIAL` (consistent with `ImageRecord.id`, `Tag.id`).
|
||||
- FKs target the actual ImageRepo table names: `image_record.id`, `tag.id`. The parent spec's `images(id)` references become `image_record(id)`.
|
||||
- Table names stay singular per ImageRepo convention: `image_tag_prediction`, `image_embedding`, `character_reference_embedding`, `suggestion_feedback`, `tag_suggestion_config`.
|
||||
- `tag_suggestion_config` could be folded into the existing `AppSettings` key/value table, but the parent spec's dedicated table keeps threshold tuning separate from general settings and makes the thresholds discoverable. Stick with the dedicated table.
|
||||
|
||||
### Alembic migration
|
||||
|
||||
A new revision in `migrations/versions/` does:
|
||||
|
||||
1. `CREATE EXTENSION IF NOT EXISTS vector;`
|
||||
2. Create the five new tables with SQLAlchemy-`pgvector` column types.
|
||||
3. Create the HNSW index on `image_embedding.embedding` using `vector_cosine_ops`.
|
||||
4. Seed `tag_suggestion_config` with the initial threshold rows from the parent spec.
|
||||
|
||||
The `pgvector` Python package must be added to the base `requirements.txt` (not just ml-worker's) so the web process can read embeddings for suggestion generation.
|
||||
|
||||
### Embedding dimension
|
||||
|
||||
Locked at **1152** (SigLIP-SO400M). If the model is swapped later, a new `model_version` coexists in the same tables per the parent spec; the schema does not change.
|
||||
|
||||
---
|
||||
|
||||
## 3. Task flow
|
||||
|
||||
New Celery module `app/tasks/ml.py`, tasks routed to the `ml` queue.
|
||||
|
||||
### Per-upload: chained from the import pipeline
|
||||
|
||||
`app.tasks.import_file.import_media_file` currently ends by committing an `ImageRecord` row. After commit, it enqueues:
|
||||
|
||||
```python
|
||||
from app.tasks.ml import tag_and_embed
|
||||
tag_and_embed.apply_async(args=[image_id], queue='ml')
|
||||
```
|
||||
|
||||
`tag_and_embed(image_id)`:
|
||||
1. Load the image from disk (or from the path stored on `ImageRecord.filepath`)
|
||||
2. Run WD14 preprocessing → WD14 inference → write rows into `image_tag_prediction`
|
||||
3. Run SigLIP preprocessing → SigLIP inference → write row into `image_embedding`
|
||||
4. Commit both in one DB transaction
|
||||
|
||||
If either model fails the task raises and Celery will retry per its default policy. A failure does not block the upload — the image is already in the DB and visible in the gallery; suggestions simply will not appear for it until the task succeeds.
|
||||
|
||||
### Backfill: resumable batch job
|
||||
|
||||
`app.tasks.ml.backfill(batch_size=50)`:
|
||||
|
||||
1. Query images missing predictions **or** embeddings for the current model versions
|
||||
2. For each, enqueue `tag_and_embed.apply_async(args=[image_id], queue='ml')`
|
||||
3. Process in batches so the enqueue side doesn't dump 62k messages into Redis at once — sleep/yield between batches
|
||||
4. Track progress via a row in the existing `ImportBatch` / `Task` pattern (whichever is conventional for long-running scan jobs — follow `scan_directory` for precedent)
|
||||
|
||||
Resumability comes from the query itself: it only enqueues images without current rows. Re-running after a crash picks up exactly where it left off.
|
||||
|
||||
An admin trigger for `backfill` is exposed via a new button on the Settings → Maintenance tab (the tab already exists from the UI/UX redesign). No automated scheduling — the user kicks it off manually once after deploy, and again after a model version change.
|
||||
|
||||
### On accept: centroid recompute
|
||||
|
||||
When a user accepts a `character`-kind suggestion on an image:
|
||||
|
||||
1. Apply the tag as normal (existing tag-add endpoint flow)
|
||||
2. Write a `suggestion_feedback` row with `decision='accepted'`
|
||||
3. Check `character_reference_embedding.reference_count` for this tag. If the current count of confirmed images minus `reference_count` at last computation exceeds `centroid_recompute_delta` (default 3), enqueue `recompute_centroid.apply_async(args=[character_tag_name], queue='ml')`
|
||||
|
||||
`recompute_centroid(tag_name)` fetches all embeddings for images that currently carry that character tag, computes the mean vector, upserts `character_reference_embedding` with the new centroid and the current reference count.
|
||||
|
||||
This runs on the `ml` queue because it touches the pgvector tables; it does **not** need the ONNX models, so it completes in milliseconds.
|
||||
|
||||
### On reject: feedback only
|
||||
|
||||
Rejecting a suggestion writes a `suggestion_feedback` row with `decision='rejected'`. No other side effect. The rejected tag will reappear on the next modal open — intentional, per the parent spec's rationale that feedback is informational and we should not lose tags the user might later want.
|
||||
|
||||
---
|
||||
|
||||
## 4. Web integration
|
||||
|
||||
### Flask endpoints
|
||||
|
||||
All under the existing image blueprint. Permissions inherit from however the modal's existing `/image/<id>/tags` endpoints gate access.
|
||||
|
||||
**`GET /image/<id>/suggestions`**
|
||||
|
||||
Returns the merged suggestion list grouped by category. Body:
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"suggestions": {
|
||||
"character": [
|
||||
{"name": "<tag>", "confidence": 0.91, "source": "embedding_similarity", "exists_in_db": true}
|
||||
],
|
||||
"copyright": [ ... ],
|
||||
"rating": [ ... ],
|
||||
"general": [ ... ]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Suggestion generation logic lives in a new `app/services/tag_suggestions.py` module and is invoked by the route. It:
|
||||
|
||||
1. Fetches this image's `image_tag_prediction` rows for the current WD14 model version, filters by per-category thresholds from `tag_suggestion_config`, drops tags already applied to the image.
|
||||
2. Fetches this image's `image_embedding` for the current SigLIP model version, computes cosine similarity against every `character_reference_embedding` whose `reference_count >= min_reference_images_per_character`, keeps matches above `threshold_embedding_character`, drops characters already applied.
|
||||
3. Merges: WD14-character and embedding-character lists dedupe by tag name; keep the higher normalized confidence and record both sources if desired (for now, single `source` is fine).
|
||||
4. Annotates each suggestion with `exists_in_db: true|false` by looking up each name in the `tag` table.
|
||||
5. Groups by category, sorts by confidence descending within each group.
|
||||
|
||||
Categories returned in the response map 1:1 to WD14's categories. `meta` is omitted from the modal for now (noisy, low value).
|
||||
|
||||
**`POST /image/<id>/suggestions/accept`**
|
||||
|
||||
Body: `{"tag_name": "...", "source": "wd14"|"embedding_similarity", "category": "general"|"character"|"copyright"|"rating"}`.
|
||||
|
||||
Server applies the hybrid vocabulary policy:
|
||||
|
||||
- `character`, `copyright`, `rating` → if the tag doesn't exist in the `tag` table, create it with the appropriate `kind`. Then associate with the image. Kind mapping:
|
||||
- WD14 `character` → `kind='character'`
|
||||
- WD14 `copyright` → `kind='fandom'` (ImageRepo's IP/franchise kind — series is reserved for sequential readers)
|
||||
- WD14 `rating` → `kind='rating'`
|
||||
- `general` → if the tag exists, associate with the image. If not, return `{"ok": false, "error": "unknown_general_tag"}` (the UI renders this branch as a disabled chip in advance; this error is the safety net for race conditions).
|
||||
|
||||
After applying, writes `suggestion_feedback` with `decision='accepted'` and triggers centroid recompute if the accepted tag is a character.
|
||||
|
||||
**`POST /image/<id>/suggestions/reject`**
|
||||
|
||||
Body: `{"tag_name": "...", "source": "..."}`. Writes `suggestion_feedback` with `decision='rejected'`. Returns `{"ok": true}`.
|
||||
|
||||
### Modal UI
|
||||
|
||||
New section in the gallery modal sidebar (`app/templates/_gallery_modal.html`) **below** the existing Tags section, with id `modalSuggestionsSection` and a `display: none` inline style that JavaScript toggles (matches the project's inline-style-over-class convention per commit `e37ce27`).
|
||||
|
||||
Structure:
|
||||
|
||||
```html
|
||||
<div class="modal-sidebar-section modal-suggestions-section" id="modalSuggestionsSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
<span class="section-title">Suggestions</span>
|
||||
<button class="suggestions-refresh" type="button" aria-label="Refresh suggestions">↻</button>
|
||||
</div>
|
||||
<div id="modalSuggestionsList" class="suggestions-list"></div>
|
||||
</div>
|
||||
```
|
||||
|
||||
Frontend logic goes in `app/static/js/view-modal.js` next to the existing `loadTags` pattern. A new `loadSuggestions(imageId)` fetches `/image/<id>/suggestions`, renders a flat DOM structure with subtle category-heading dividers between the groups, and wires up the accept / reject click handlers.
|
||||
|
||||
**Chip rendering:**
|
||||
|
||||
- Layout: `[+ label · XX%]` with a hover-revealed `✕` on the right.
|
||||
- Accept click: disable chip, `POST .../accept`, on success animate-out and call the existing `refreshItemTags(imageId)` hook so the gallery thumbnail overlay updates. Also append the new tag to the editable tag list in the modal.
|
||||
- Reject click: disable chip, `POST .../reject`, animate-out. No other side effect.
|
||||
- `exists_in_db: false` and policy says general-must-exist: render the chip dimmed, accept button disabled, tooltip "create this tag first in Tags admin."
|
||||
- Platform-style accents: not needed here. Keep chips visually distinct from the editable tags (maybe a dashed border or slightly lower opacity) so users don't confuse suggestions with applied tags.
|
||||
|
||||
**Feedback:**
|
||||
|
||||
Reuse the `tagActionFeedback` element the modal already owns for "applied" / "rejected" / error messages. No toast system.
|
||||
|
||||
**Lifecycle:**
|
||||
|
||||
- Open modal → `loadSuggestions(imageId)` called alongside `loadTags` / `renderProvenance`.
|
||||
- Close modal → clear the suggestions list (the existing `closeModal` hook).
|
||||
- Arrow-key navigation between images re-runs `loadSuggestions` per image.
|
||||
|
||||
---
|
||||
|
||||
## 5. Tag vocabulary policy (hybrid)
|
||||
|
||||
Confirmed from brainstorming:
|
||||
|
||||
- **Character / copyright / rating** suggestions → auto-create the tag on accept if it doesn't exist in the DB. The `tag.kind` is set as follows:
|
||||
- WD14 `character` → `kind='character'`
|
||||
- WD14 `copyright` → `kind='fandom'` (ImageRepo's IP/franchise kind — `series` is reserved for sequential readers, not franchises)
|
||||
- WD14 `rating` → `kind='rating'`
|
||||
- **General** suggestions → filter to tags that already exist in the DB. Unknown general tags are rendered dimmed (disabled) in the UI; the accept endpoint rejects them as a safety net.
|
||||
- **Embedding-similarity** suggestions are always characters; they suggest only characters that already have a reference set, so vocabulary never grows via that path.
|
||||
|
||||
---
|
||||
|
||||
## 6. Scope boundaries
|
||||
|
||||
### In scope this phase
|
||||
|
||||
- `ml-worker` Docker service + image + requirements
|
||||
- Alembic migration for pgvector extension, five tables, HNSW index, seed config rows
|
||||
- SQLAlchemy models for all five tables (in `app/models.py` or a dedicated `app/models_ml.py` — follow whichever the codebase prefers for locality)
|
||||
- `app/tasks/ml.py` with `tag_and_embed`, `backfill`, `recompute_centroid`
|
||||
- Hook into `import_media_file` to chain `tag_and_embed` after commit
|
||||
- `app/services/tag_suggestions.py` suggestion-generation module
|
||||
- Three new Flask endpoints under the existing image blueprint
|
||||
- Modal UI additions: template block, CSS, `view-modal.js` suggestion logic
|
||||
- Settings → Maintenance tab: "Run ML backfill" button
|
||||
- Threshold values read from `tag_suggestion_config` (DB-editable via psql for now)
|
||||
|
||||
### Deferred
|
||||
|
||||
- Bulk-edit gallery multi-select suggestions (explicit next phase)
|
||||
- Admin UI for editing `tag_suggestion_config` thresholds
|
||||
- Admin panel showing character reference-count stats
|
||||
- Reranker / feedback-driven threshold tuning
|
||||
- Automatic backfill scheduling (manual trigger only for now)
|
||||
- Multi-centroid-per-character for characters with high intra-class variance (see parent spec §"Why centroids instead of querying all reference images")
|
||||
|
||||
---
|
||||
|
||||
## 7. Verification
|
||||
|
||||
Per the ImageRepo project norm, there is no automated test harness. Verification at implementation time is a manual browser check:
|
||||
|
||||
- Upload a new image; confirm `image_tag_prediction` and `image_embedding` rows appear after a few seconds.
|
||||
- Open the modal on an image that has predictions; confirm suggestions appear grouped by category with plausible confidences.
|
||||
- Accept a character suggestion; confirm the tag is applied, a `suggestion_feedback` row is written, and (if count delta is met) a `recompute_centroid` task runs and `character_reference_embedding` is updated.
|
||||
- Reject a suggestion; confirm a feedback row is written and the suggestion reappears on modal reopen.
|
||||
- Kick off a backfill from Settings → Maintenance; confirm it processes images in batches, is resumable after a restart, and does not stall the ml-worker.
|
||||
- Accept a general tag that doesn't exist in the DB (forcing the disabled-chip path); confirm the UI prevents submission and the endpoint returns `unknown_general_tag` if attempted.
|
||||
@@ -0,0 +1,177 @@
|
||||
# Character-Tag Integrity Design
|
||||
|
||||
**Date:** 2026-04-19
|
||||
**Branch:** `feat/tag-suggestions` (same branch as the tag-suggestion feature; this is a separate concern but bundles naturally).
|
||||
|
||||
## Problem
|
||||
|
||||
Two related defects erode the integrity of the `Tag` table's character/fandom data:
|
||||
|
||||
1. **Stale image↔fandom associations.** When a character tag gains a `fandom_id` via `POST /api/tag/<id>/set-fandom` (or via a rename that introduces `(Fandom)`), existing images already tagged with that character do not receive the fandom tag. Only *future* attachments through `add_tag` / `add_bulk_tag` auto-apply the fandom, because those paths explicitly append it. The result: images tagged before the association is set miss the fandom and won't show up under the fandom in filters or suggestions.
|
||||
|
||||
2. **Inconsistent casing on character/fandom display names.** The user (solo dev) introduces typos like `character:misty` alongside `character:Misty`. There is no normalization on create or rename, so the DB accumulates duplicates that diverge by case.
|
||||
|
||||
## Goals
|
||||
|
||||
- Every image tagged with a character whose tag has a `fandom_id` also has that fandom tag attached.
|
||||
- Character and fandom tag display names always start each whitespace-separated word with an upper-case letter.
|
||||
- Existing lowercase names are renamed one-time on migration, merging into any already-correctly-cased tag of the same kind.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Subtractive reconciliation (removing fandom tags from images when the character's fandom changes or is cleared). The DB does not record *why* a fandom was attached, so removal risks deleting user-intended tags.
|
||||
- Smart handling of apostrophes, hyphens, or Roman numerals in names. Users who need `"O'Brien"` or `"Mary-Kate"` can still type the exact casing via `set-fandom`; the normalizer only guarantees first-letter-of-each-word.
|
||||
- Normalization for other kinds (`artist`, `series`, `user`, `post`, `rating`). Scope stays on the two kinds the user asked about.
|
||||
- Retroactive removal of image↔fandom links. Only additive.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Normalization helper
|
||||
|
||||
New module `app/utils/tag_names.py`:
|
||||
|
||||
```python
|
||||
def normalize_display_name(s: str) -> str:
|
||||
"""Title-case each whitespace-separated word. Preserves internal punctuation.
|
||||
|
||||
Examples:
|
||||
"misty" -> "Misty"
|
||||
"misty ketchum" -> "Misty Ketchum"
|
||||
"o'brien" -> "O'brien"
|
||||
"mary-kate" -> "Mary-kate"
|
||||
"""
|
||||
return ' '.join(w[:1].upper() + w[1:] for w in s.split(' ') if w != '') \
|
||||
if s else s
|
||||
```
|
||||
|
||||
Called from every character/fandom write path:
|
||||
|
||||
- `add_tag` (`/image/<id>/tags/add`) — after parsing `(Fandom)` suffix, normalize both `char_name` and `fandom_name` before storing.
|
||||
- `add_bulk_tag` (in `app/main.py` around line 2240) — same.
|
||||
- `set_tag_fandom` (`/api/tag/<id>/set-fandom`) — normalize `fandom_name` when the `fandom_name` path is taken, normalize the character display base before reassembling the tag name.
|
||||
- `accept_image_suggestion` (WD14 accept path in `app/main.py`) — when auto-creating a new character or fandom tag from a WD14 suggestion, normalize first.
|
||||
- `_ensure_fandom_tag` (helper in `app/main.py`) — normalize internally so every caller benefits.
|
||||
- `_parse_character_fandom` (helper in `app/main.py`) — unchanged; parsing is orthogonal to normalization.
|
||||
|
||||
### Retroactive fandom backfill on `set_tag_fandom`
|
||||
|
||||
After the existing commit that renames the character tag and sets `fandom_id`, the endpoint runs a second transaction that attaches the fandom tag to every image already tagged with the character. The operation is a single SQL statement on the `image_tags` association table:
|
||||
|
||||
```sql
|
||||
INSERT INTO image_tags (image_id, tag_id)
|
||||
SELECT image_id, :fandom_tag_id
|
||||
FROM image_tags
|
||||
WHERE tag_id = :character_tag_id
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM image_tags it2
|
||||
WHERE it2.image_id = image_tags.image_id
|
||||
AND it2.tag_id = :fandom_tag_id
|
||||
);
|
||||
```
|
||||
|
||||
Running it in a second transaction after the rename commits means a failed backfill never rolls back the rename. A failed backfill leaves stale state that the sweep button will correct.
|
||||
|
||||
### "Sync character fandoms to images" sweep
|
||||
|
||||
New Celery task `sync_character_fandoms` in `app/tasks/maintenance.py` (new module) on the default queue:
|
||||
|
||||
```python
|
||||
@celery.task(name='app.tasks.maintenance.sync_character_fandoms',
|
||||
soft_time_limit=120, time_limit=180)
|
||||
def sync_character_fandoms():
|
||||
"""For every character tag with fandom_id, attach fandom to images
|
||||
that have the character but not the fandom."""
|
||||
with app.app_context():
|
||||
total_added = 0
|
||||
chars = Tag.query.filter_by(kind='character').filter(Tag.fandom_id.isnot(None)).all()
|
||||
for char in chars:
|
||||
inserted = _attach_fandom_to_images(char.id, char.fandom_id)
|
||||
total_added += inserted
|
||||
db.session.commit()
|
||||
log.info("sync_character_fandoms: added %d image↔fandom links across %d characters",
|
||||
total_added, len(chars))
|
||||
return {'characters_scanned': len(chars), 'links_added': total_added}
|
||||
```
|
||||
|
||||
New route `POST /settings/maintenance/sync-character-fandoms` that calls `sync_character_fandoms.apply_async()` (default queue, no `queue='ml'`).
|
||||
|
||||
Settings → Maintenance gets a third button "Sync character fandoms to images".
|
||||
|
||||
### One-time migration: rename + merge
|
||||
|
||||
New Alembic migration `h26041901_normalize_character_fandom_tag_names.py` (revision id follows the existing date-coded pattern; adjust `down_revision` to whatever `alembic current` is at deploy time).
|
||||
|
||||
Upgrade algorithm, entirely in raw SQL within a single transaction:
|
||||
|
||||
1. Build a working set: `SELECT id, name, kind, fandom_id FROM tag WHERE kind IN ('character', 'fandom')`.
|
||||
2. For each row, compute `normalized_name` by:
|
||||
- Splitting the name at the first `:` into `prefix` + `rest`.
|
||||
- For character tags: further split `rest` at ` (` to separate `char_base` from `fandom_part)`.
|
||||
- Normalize `char_base`.
|
||||
- If a fandom part exists, normalize it too and reassemble `{norm_char} ({norm_fandom})`.
|
||||
- For fandom tags: normalize `rest` directly.
|
||||
- Reassemble `{prefix}:{normalized_rest}`.
|
||||
3. For each `(id, old_name, new_name)` where `old_name != new_name`:
|
||||
- Check for a target row with `name = new_name AND kind = current_row.kind AND id != current_row.id`.
|
||||
- **No collision:** `UPDATE tag SET name = :new_name WHERE id = :id`.
|
||||
- **Collision (merge):**
|
||||
- Winner: the existing correctly-cased target (the one the current row would collide with).
|
||||
- Loser: the row being renamed.
|
||||
- `INSERT INTO image_tags(image_id, tag_id) SELECT DISTINCT image_id, :winner_id FROM image_tags WHERE tag_id = :loser_id ON CONFLICT DO NOTHING`.
|
||||
- `DELETE FROM image_tags WHERE tag_id = :loser_id`.
|
||||
- `UPDATE tag SET fandom_id = :winner_id WHERE fandom_id = :loser_id`.
|
||||
- `UPDATE tag_reference_embedding SET tag_name = :winner_name WHERE tag_name = :loser_name AND NOT EXISTS (SELECT 1 FROM tag_reference_embedding WHERE tag_name = :winner_name AND model_version = tag_reference_embedding.model_version)` — keeps winner's centroid if both have one.
|
||||
- `DELETE FROM tag_reference_embedding WHERE tag_name = :loser_name`.
|
||||
- `DELETE FROM tag WHERE id = :loser_id`.
|
||||
4. Log counts of renames vs merges to migration output.
|
||||
|
||||
Downgrade: no-op for merges (destructive), plain name-revert for pure renames is possible but not worth the bookkeeping. The downgrade function raises `NotImplementedError` with an explanatory message.
|
||||
|
||||
## Data flow — summary
|
||||
|
||||
```
|
||||
Write paths (add_tag, add_bulk_tag, accept_suggestion, set_tag_fandom)
|
||||
→ normalize_display_name(…)
|
||||
→ store {prefix}:{Normalized Name}
|
||||
→ attach to image (and auto-attach fandom if character has one)
|
||||
|
||||
set_tag_fandom (specifically)
|
||||
→ normalize + rename + set fandom_id (commit #1)
|
||||
→ SELECT image_ids tagged with this character
|
||||
→ INSERT (image_id, fandom_tag_id) for rows missing the fandom (commit #2)
|
||||
→ return JSON
|
||||
|
||||
Sweep button
|
||||
→ enqueue sync_character_fandoms on default queue
|
||||
→ iterate character tags with fandom_id
|
||||
→ per-tag: batch-insert missing links, commit per tag
|
||||
→ log summary
|
||||
|
||||
Migration (one-time)
|
||||
→ enumerate character:* and fandom:* tags
|
||||
→ compute normalized names
|
||||
→ rename survivors; merge collisions (reassign image_tags, fandom_id, tag_reference_embedding; delete loser)
|
||||
→ one transaction; no downgrade for merges
|
||||
```
|
||||
|
||||
## Error handling
|
||||
|
||||
- **Normalization of empty or whitespace-only string:** returns the input unchanged; callers already strip and reject empty via `abort(400)`.
|
||||
- **`set_tag_fandom` rename collision with differently-cased target:** post-normalization, both names converge, so the 409 check at `app/main.py:940` fires correctly. No behavior change.
|
||||
- **`set_tag_fandom` backfill DB error:** rename commits first (commit #1); backfill fails (commit #2 rolled back). Response still returns 200 with the renamed tag. Sweep button later recovers.
|
||||
- **Sweep task per-tag error:** try/except around each tag; log `sync_character_fandoms: tag %s failed: %s`, continue, return partial summary.
|
||||
- **Migration mid-flight error:** full rollback. User investigates via migration log.
|
||||
|
||||
## Testing plan
|
||||
|
||||
Manual verification on dev DB:
|
||||
|
||||
1. **Migration:** dev DB has `character:emelie`. Run migration → verify tag is renamed to `character:Emelie`, row count in `tag` unchanged, all `image_tags` links preserved.
|
||||
2. **Create with normalization:** via add-tag, submit `character:misty (pokemon)`. Verify tag lands as `character:Misty (Pokemon)`; fandom tag `fandom:Pokemon` is auto-created; both attached to the image.
|
||||
3. **Retroactive backfill on `set-fandom`:** pick an existing character with no fandom, attach it to 3 images. Call `set-fandom` with a new fandom_name. Verify all 3 images now have the fandom tag (even though `set-fandom` was the trigger).
|
||||
4. **Merge on migration:** manually seed `character:foo` and `character:Foo` with overlapping and distinct image links. Run migration. Verify one `character:Foo` survives with the union of image links; `character:foo` row is gone; `tag_reference_embedding` has one row (if either did); `fandom_id` references pointing at either row now point at the survivor.
|
||||
5. **Sweep button:** manually remove a fandom link from an image whose character tag has a fandom (simulating drift). Click "Sync character fandoms to images". Verify the fandom is re-attached and the task log shows `links_added: 1`.
|
||||
|
||||
## Open items
|
||||
|
||||
None. Design confirmed via brainstorming: additive-only backfill, title-case normalization via simple first-letter-per-word rule, covers character + fandom kinds, one-time migration with merge, sweep available as manual maintenance action.
|
||||
@@ -0,0 +1,269 @@
|
||||
## Tag Underscores + Modal Polish Design
|
||||
|
||||
**Date:** 2026-04-19
|
||||
**Branch:** `feat/tag-suggestions`
|
||||
**Builds on:** `2026-04-19-character-tag-integrity-design.md` (same branch, same normalizer module)
|
||||
|
||||
## Problem
|
||||
|
||||
Three related pain points around WD14-sourced suggestions and the suggestion modal:
|
||||
|
||||
1. **WD14 uses Danbooru-style underscored names.** Raw WD14 tags surface as `big_hair`, `long_hair`, `blue_sky`. The user's library uses spaces (`big hair`). Accepting a WD14 suggestion today writes the underscored form into the `Tag` table and attaches it to the image, so the library diverges from the user's naming convention.
|
||||
2. **Accepting a suggestion refreshes the attached-tags list later and more loosely than the manual "Add" button.** Manual add awaits `loadTags(imageId)` before showing feedback; accept fires `loadTags` un-awaited inside a 200 ms animation callback. The visible result: the modal's attached-tags list looks briefly stale after accept, and reject/accept ordering can race.
|
||||
3. **Accept (✓) and reject (✕) buttons have no persistent color cue.** Green/red only appear on hover, so the buttons read as decorative glyphs until the user hovers.
|
||||
|
||||
## Goals
|
||||
|
||||
- WD14 suggestion chips display and persist as space-separated names (`big hair`, not `big_hair`), regardless of kind.
|
||||
- Character and fandom names continue to be title-cased on write (`character:Misty`); general (null) names preserve the user's casing but gain underscore→space.
|
||||
- A one-time migration scrubs any already-persisted underscored names in the `Tag` table, merging into any existing space-separated duplicate.
|
||||
- Accepting a suggestion refreshes the modal's attached-tags list with the same timing semantics as manual add — the data refresh is awaited and not gated by animation.
|
||||
- Accept and reject buttons are persistently green and red, not only on hover.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Changing the WD14 model, thresholds, or suggestion logic. The transformation is cosmetic/naming.
|
||||
- Touching kinds beyond character, fandom, and null (general). Artist, post, series, rating, user, source, archive keep their current behavior.
|
||||
- Stripping other Danbooru-style punctuation (parentheses, backslashes). Underscore→space only.
|
||||
- Changing the reject endpoint behavior. It already records feedback without creating a tag; nothing to adjust on the backend.
|
||||
- Redesigning the chip layout. Only the button colors change.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Normalization helpers
|
||||
|
||||
`app/utils/tag_names.py` grows two additions:
|
||||
|
||||
```python
|
||||
def underscores_to_spaces(s: str) -> str:
|
||||
"""Replace underscores with spaces. Used for general/null-kind names and
|
||||
as a pre-step inside normalize_display_name for character/fandom."""
|
||||
return s.replace('_', ' ') if s else s
|
||||
|
||||
|
||||
def normalize_display_name(s: str) -> str:
|
||||
"""Title-case each whitespace-separated word. Underscores become spaces
|
||||
first so 'misty_ketchum' and 'Misty Ketchum' collapse onto the same name."""
|
||||
if not s:
|
||||
return s
|
||||
s = underscores_to_spaces(s)
|
||||
parts = s.split(' ')
|
||||
out: list[str] = []
|
||||
for p in parts:
|
||||
if p == '':
|
||||
out.append(p)
|
||||
else:
|
||||
out.append(p[:1].upper() + p[1:])
|
||||
return ' '.join(out)
|
||||
```
|
||||
|
||||
No new helper for "general" — write paths call `underscores_to_spaces` directly when handling null-kind. That keeps the module small (one concept per helper) and avoids a second name that does nothing more than call `.replace`.
|
||||
|
||||
### Write-path integration
|
||||
|
||||
#### `accept_image_suggestion` (`app/main.py:723`)
|
||||
|
||||
Accept is called only for suggestion categories `character`, `copyright`, and `general`. Kind mapping: `character → character`, `copyright → fandom`, `general → None`. Change:
|
||||
|
||||
- Before the `Tag.query.filter_by(name=tag_name)` lookup, canonicalize based on category:
|
||||
- `character` / `copyright`: split `prefix:rest`, apply `normalize_display_name` to `rest`, reassemble.
|
||||
- `general`: apply `underscores_to_spaces` to the whole name (no prefix expected).
|
||||
- Use the canonicalized `tag_name` for both the existence check and the new `Tag(...)` row. The existing character/fandom normalization block inside the "if tag is None" branch becomes redundant and is removed.
|
||||
|
||||
The net effect: `fandom:trigun_stampede` → `fandom:Trigun Stampede`; `big_hair` → `big hair`; `character:misty_ketchum` → `character:Misty Ketchum`. Accepting a WD14 chip produces the same Tag row the user would get by typing the clean name manually.
|
||||
|
||||
#### `add_tag` (`app/main.py:823`) and `bulk_add_tag` (`app/main.py:2279`)
|
||||
|
||||
These already call `normalize_display_name` on character/fandom. With `normalize_display_name` now stripping underscores internally, character and fandom paths are already covered. Add:
|
||||
|
||||
- For the no-prefix / `kind == "user"` path: `tag_name = underscores_to_spaces(tag_name)` before the Tag lookup.
|
||||
|
||||
Kinds that are **not** transformed: `artist`, `post`, `source`, `archive`, `series`, `rating`. Artists sometimes have `_` in their canonical identifier (e.g., `redjuice_redjuice`); post tags use `:` as the separator and may contain underscored platform IDs; source/archive/series are path- or identifier-like where underscores may be meaningful. Those paths pass their raw name through to the DB unchanged, matching current behavior.
|
||||
|
||||
Rule of thumb: the transform applies to exactly three kinds — `character`, `fandom`, and null (general) — the same set the migration touches.
|
||||
|
||||
### Suggestion-service transform
|
||||
|
||||
`app/services/tag_suggestions.py` — WD14 names are emitted to the modal still carrying underscores. Two choices:
|
||||
|
||||
- **Transform at emit time** (chosen): after `_wd14_suggestions` builds a row, rewrite `p.tag_name` using the same rule as the accept path. The chip displays `big hair` and the accept endpoint receives `big hair` in the POST body.
|
||||
- ~~Transform at paint time (in `view-modal.js`)~~ rejected — it would mean the display name and the POST body diverge, forcing the accept endpoint to re-derive the canonical form.
|
||||
|
||||
Implementation:
|
||||
|
||||
```python
|
||||
from app.utils.tag_names import normalize_display_name, underscores_to_spaces
|
||||
|
||||
_WD14_CATEGORY_TO_KIND = {
|
||||
'character': 'character',
|
||||
'copyright': 'fandom',
|
||||
}
|
||||
|
||||
def _canonicalize_wd14_name(raw: str, category: str) -> str:
|
||||
kind = _WD14_CATEGORY_TO_KIND.get(category)
|
||||
if ':' in raw:
|
||||
prefix, rest = raw.split(':', 1)
|
||||
transform = normalize_display_name if kind in ('character', 'fandom') else underscores_to_spaces
|
||||
return f"{prefix}:{transform(rest)}"
|
||||
return normalize_display_name(raw) if kind in ('character', 'fandom') else underscores_to_spaces(raw)
|
||||
```
|
||||
|
||||
The existing `already` set (tags already attached to the image) is also canonicalized — `{underscores_to_spaces(n) for n in already}` — so that an already-attached `big hair` correctly shadows a WD14 `big_hair` suggestion.
|
||||
|
||||
`_WD14_CATEGORY_TO_KIND` moves from `app/main.py` to the service module to avoid circular imports; `main.py` imports from the service.
|
||||
|
||||
### Migration `i26041901_normalize_underscores_and_general_tags.py`
|
||||
|
||||
One-time rename-or-merge covering every existing row in `tag` where the name contains `_` in the portion after the first `:` (or the whole name if there is no `:`). The down_revision is `h26041901` (the previous character/fandom normalization migration).
|
||||
|
||||
Algorithm (raw SQL, single transaction):
|
||||
|
||||
1. `SELECT id, name, kind, fandom_id FROM tag WHERE kind IN ('character', 'fandom') OR kind IS NULL` — widen the net to any kind that the write-paths now transform.
|
||||
2. For each row, compute `new_name`:
|
||||
- Split on first `:` into `prefix`, `rest` (or `rest = name` if no prefix).
|
||||
- For character: split `rest` at ` (` to isolate `char_base` and `fandom_part)`; run `normalize_display_name` on each; reassemble.
|
||||
- For fandom: `normalize_display_name(rest)`.
|
||||
- For null (general): `underscores_to_spaces(rest)`.
|
||||
- Reassemble with prefix (character/fandom always have prefixes; general usually has none).
|
||||
3. For each `(id, old_name, new_name)` where they differ:
|
||||
- Look for a `tag` row with `name = new_name AND kind = current.kind AND id != current.id`.
|
||||
- **No collision:** `UPDATE tag SET name = :new_name WHERE id = :id`.
|
||||
- **Collision (merge):** winner = existing correctly-named row, loser = current row.
|
||||
- `INSERT INTO image_tags(image_id, tag_id) SELECT DISTINCT image_id, :winner_id FROM image_tags WHERE tag_id = :loser_id ON CONFLICT DO NOTHING`
|
||||
- `DELETE FROM image_tags WHERE tag_id = :loser_id`
|
||||
- `UPDATE tag SET fandom_id = :winner_id WHERE fandom_id = :loser_id`
|
||||
- `UPDATE tag_reference_embedding SET tag_name = :winner_name WHERE tag_name = :loser_name AND NOT EXISTS (SELECT 1 FROM tag_reference_embedding t2 WHERE t2.tag_name = :winner_name AND t2.model_version = tag_reference_embedding.model_version)`
|
||||
- `DELETE FROM tag_reference_embedding WHERE tag_name = :loser_name`
|
||||
- `DELETE FROM tag WHERE id = :loser_id`
|
||||
4. Log rename count and merge count.
|
||||
|
||||
Downgrade: raises `NotImplementedError` (merges are destructive, same rationale as `h26041901`).
|
||||
|
||||
Code duplication note: this migration and `h26041901` share the "compute new name / rename-or-merge" shape but not the exact transform. Migrations are historical artifacts — they run once and are never edited — so repeating the scaffolding is preferred over extracting a shared helper module that the migrations would need to import.
|
||||
|
||||
### Modal JS: accept-path refresh parity
|
||||
|
||||
`app/static/js/view-modal.js:203-233` (`acceptSuggestion`):
|
||||
|
||||
```javascript
|
||||
async function acceptSuggestion(chip, imageId) {
|
||||
if (chip.dataset.disabled === 'true') return;
|
||||
chip.dataset.disabled = 'true';
|
||||
chip.style.pointerEvents = 'none';
|
||||
try {
|
||||
const r = await fetch(`/image/${imageId}/suggestions/accept`, { /* ...body as before... */ });
|
||||
const j = await r.json();
|
||||
if (!j.ok) {
|
||||
chip.dataset.disabled = 'false';
|
||||
chip.style.pointerEvents = '';
|
||||
if (tagActionFeedback) {
|
||||
tagActionFeedback.textContent = `Couldn't accept: ${j.error || 'error'}`;
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (typeof loadTags === 'function') await loadTags(imageId);
|
||||
if (tagActionFeedback) tagActionFeedback.textContent = `Added ${j.tag.name}`;
|
||||
if (typeof window.refreshItemTags === 'function') window.refreshItemTags(imageId);
|
||||
animateChipOut(chip); // fire-and-forget; data is already refreshed
|
||||
} catch {
|
||||
chip.dataset.disabled = 'false';
|
||||
chip.style.pointerEvents = '';
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Key changes:
|
||||
- `await loadTags(imageId)` before setting feedback and before the chip animation runs. Matches the manual `addTag` ordering at `view-modal.js:600-602`.
|
||||
- `animateChipOut` no longer takes a completion callback (it's become a pure visual-only helper); its `setTimeout(...)` just removes the node. The signature becomes `animateChipOut(chip)` — callers that still pass a callback lose the callback argument cleanly.
|
||||
- The chip is disabled via `dataset.disabled` for duplicate-click protection.
|
||||
|
||||
### Modal CSS: persistent green/red chip buttons
|
||||
|
||||
`app/static/style.css:980-1006`:
|
||||
|
||||
```css
|
||||
.suggestion-chip .sugg-btn.sugg-accept {
|
||||
color: #8be78b;
|
||||
border-color: rgba(139, 231, 139, 0.35);
|
||||
}
|
||||
.suggestion-chip .sugg-btn.sugg-accept:hover {
|
||||
background: rgba(139, 231, 139, 0.12);
|
||||
border-color: rgba(139, 231, 139, 0.6);
|
||||
}
|
||||
.suggestion-chip .sugg-btn.sugg-reject {
|
||||
color: #e78b8b;
|
||||
border-color: rgba(231, 139, 139, 0.35);
|
||||
}
|
||||
.suggestion-chip .sugg-btn.sugg-reject:hover {
|
||||
background: rgba(231, 139, 139, 0.12);
|
||||
border-color: rgba(231, 139, 139, 0.6);
|
||||
}
|
||||
```
|
||||
|
||||
The base `.sugg-btn` rule keeps `color: inherit` for anything that isn't accept/reject — safe-by-default if a future button gets added.
|
||||
|
||||
## Data flow — summary
|
||||
|
||||
```
|
||||
Suggestion emit:
|
||||
WD14 prediction ("big_hair", "character:misty_ketchum")
|
||||
→ _wd14_suggestions builds row
|
||||
→ _canonicalize_wd14_name rewrites name ("big hair", "character:Misty Ketchum")
|
||||
→ merged with embedding rows (already canonical)
|
||||
→ returned to modal
|
||||
|
||||
Accept:
|
||||
POST /image/<id>/suggestions/accept {tag_name: "big hair", category: "general"}
|
||||
→ normalize tag_name (underscores_to_spaces on general, normalize_display_name on character/fandom)
|
||||
→ Tag.query.filter_by(name=...) — single lookup
|
||||
→ if missing, create with kind = _WD14_CATEGORY_TO_KIND[category] or None
|
||||
→ attach to image, record feedback, commit
|
||||
→ return {ok: true, tag}
|
||||
Modal:
|
||||
→ await loadTags(imageId) — tag list refreshes (same timing as manual add)
|
||||
→ set feedback text
|
||||
→ refresh gallery thumbnail overlay
|
||||
→ animate chip out (visual only)
|
||||
|
||||
Migration:
|
||||
enumerate kind in (character, fandom, NULL) tag rows
|
||||
compute new name per kind
|
||||
rename survivors; merge collisions (image_tags union, fandom_id reassign, embedding keep-winner, delete loser)
|
||||
one transaction, no downgrade
|
||||
```
|
||||
|
||||
## Error handling
|
||||
|
||||
- **Canonicalization of empty string:** returns input unchanged; caller's existing `if not tag_name` check short-circuits.
|
||||
- **Accept with name already canonical:** transforms are idempotent; second call is a no-op on the name.
|
||||
- **Migration mid-flight error:** full transaction rollback. User investigates via migration log.
|
||||
- **WD14 emits a name with no underscores but mixed case (e.g., `Copyright:OnePiece`):** `normalize_display_name` applies to character/fandom regardless of underscores; result is `copyright:Onepiece`. The user can still rename via `set-fandom` later if needed. General with no underscore: no change.
|
||||
- **Accept fetch rejects or throws:** chip is re-enabled via `chip.dataset.disabled = 'false'`; user can retry.
|
||||
|
||||
## Testing plan
|
||||
|
||||
Manual verification on dev DB:
|
||||
|
||||
1. **Migration:** seed `tag` rows `big_hair` (kind=NULL), `fandom:trigun_stampede`, and a pre-existing `big hair` (kind=NULL) to force a merge. Run migration. Verify:
|
||||
- `big_hair` is gone, `big hair` remains with the union of image links.
|
||||
- `fandom:trigun_stampede` is renamed to `fandom:Trigun Stampede` (rename, not merge — no collision).
|
||||
- `tag_reference_embedding` rows for merged losers are deleted.
|
||||
2. **WD14 chip display:** open an image modal whose WD14 predictions include `long_hair`. Verify the chip shows "long hair" (not "long_hair").
|
||||
3. **Accept refresh parity:** click the ✓ on a suggestion. Observe: attached-tags list updates *before* the chip animation completes (no stale frame). Feedback text matches manual-add timing.
|
||||
4. **Accept canonicalization roundtrip:** click ✓ on a `big_hair` suggestion. Verify:
|
||||
- `Tag.query.filter_by(name='big hair').first()` returns a row.
|
||||
- No `big_hair` row exists in `tag`.
|
||||
- The image's tags include `big hair`.
|
||||
5. **Button color:** without hovering, verify ✓ is green-tinted and ✕ is red-tinted on all chips.
|
||||
6. **Duplicate-click protection:** rapid double-click on ✓. Verify only one accept POST fires.
|
||||
7. **Accept error path:** force the backend to return `{ok: false}`; verify chip re-enables, feedback shows the error, chip stays on screen.
|
||||
|
||||
## Open items
|
||||
|
||||
None. Design confirmed via brainstorming:
|
||||
- WD14 is the only source introducing underscores.
|
||||
- Canonicalize at one layer (service for display, accept for persistence — matching transforms).
|
||||
- General keeps case; only character/fandom title-case.
|
||||
- Migration covers character + fandom + null kinds with merge-on-collision.
|
||||
- Modal accept mirrors manual-add refresh ordering.
|
||||
- Chip buttons persist green/red tint; hover intensifies rather than introduces.
|
||||
@@ -0,0 +1,157 @@
|
||||
# User-Tag Embedding-Similarity Suggestions Design
|
||||
|
||||
**Date:** 2026-04-19
|
||||
**Branch:** `feat/tag-suggestions` (same branch as the base feature; this is an incremental extension)
|
||||
**Supersedes:** Partially — extends the character-only centroid mechanism documented in `docs/superpowers/tag-suggestion-system-spec.md`.
|
||||
|
||||
## Problem
|
||||
|
||||
WD14's tag vocabulary is Danbooru-style (`1girl`, `long_hair`, `blue_sky`). The user's library uses their own naming conventions. Today, WD14 is the only source of general/copyright suggestions, so users rarely see their own tags surfaced even when an image visually matches others they've already tagged.
|
||||
|
||||
Character tags already bypass this problem because SigLIP centroid similarity suggests the user's character tags based on visual resemblance to previously-tagged images. This design extends the same mechanism to the other tag kinds the user actually uses: `fandom` (copyright/IP) and null (general/topic).
|
||||
|
||||
## Goals
|
||||
|
||||
- Extend embedding-similarity suggestions from `character` only to `character` + `fandom` + null.
|
||||
- Keep WD14 alongside — it still runs, still produces suggestions, merges with embedding suggestions via name dedup.
|
||||
- Drop the `rating` kind from the UI and accept path entirely (the user doesn't need content gating).
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Replacing WD14. WD14 suggestions continue to surface for the three kinds in scope.
|
||||
- Per-kind tuning of thresholds or min-refs. Start with shared defaults; add knobs only if real usage shows a need.
|
||||
- Extending to `artist`, `series`, `post`, or `rating`. Scope stays minimal.
|
||||
- Changing the accept/reject UX.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Storage
|
||||
|
||||
The existing `character_reference_embedding` table becomes `tag_reference_embedding`:
|
||||
|
||||
- Rename table: `character_reference_embedding` → `tag_reference_embedding`.
|
||||
- Rename column: `character_tag` → `tag_name`.
|
||||
- Add column: `tag_kind TEXT NULL` (populated by recompute, nullable for general/topic).
|
||||
- Unique index migrates with the table: `(tag_name, model_version)`.
|
||||
- pgvector ivfflat index on `centroid` migrates with the table.
|
||||
|
||||
The SQLAlchemy model class renames from `CharacterReferenceEmbedding` to `TagReferenceEmbedding`.
|
||||
|
||||
### Eligible kinds
|
||||
|
||||
Embedding centroids are computed only for tags whose `kind` is in `{'character', 'fandom', None}`. A single module-level constant `ELIGIBLE_CENTROID_KINDS` lives in `app/services/tag_suggestions.py` and is imported where needed (recompute task, accept endpoint). Adding `artist` or `series` later is a one-tuple change.
|
||||
|
||||
`rating`, `post`, and any other kinds are skipped.
|
||||
|
||||
### Recompute task
|
||||
|
||||
`recompute_centroid(tag_name)` in `app/tasks/ml.py`:
|
||||
|
||||
1. Look up the Tag row to get its `kind`.
|
||||
2. If `kind` is not in `ELIGIBLE_CENTROID_KINDS`, log and return `{'status': 'ineligible_kind'}`.
|
||||
3. Query all embeddings for images tagged with `tag_name`. If empty, return `{'status': 'no_embeddings'}`.
|
||||
4. Compute mean vector.
|
||||
5. Upsert `TagReferenceEmbedding(tag_name=..., tag_kind=..., model_version=..., centroid=..., reference_count=...)` on the `(tag_name, model_version)` conflict.
|
||||
|
||||
### Batch recompute task
|
||||
|
||||
New task `recompute_all_centroids()`:
|
||||
|
||||
1. For each Tag where `kind IN (character, fandom, NULL)`, count attached images.
|
||||
2. For each tag with `count >= min_reference_images`, enqueue `recompute_centroid(tag_name)` on the `ml` queue.
|
||||
3. Logs progress per batch.
|
||||
|
||||
Exposed as a Settings → Maintenance button next to the existing "Run ML backfill" button.
|
||||
|
||||
### Suggestion service
|
||||
|
||||
In `app/services/tag_suggestions.py`:
|
||||
|
||||
- `_embedding_character_suggestions` renames to `_embedding_tag_suggestions`.
|
||||
- Query drops the character-only filter. Returns suggestions for all centroids above the similarity threshold with `reference_count >= min_reference_images`.
|
||||
- For each returned row, map `tag_kind` to a display category:
|
||||
- `character` → `character`
|
||||
- `fandom` → `copyright`
|
||||
- `None` → `general`
|
||||
|
||||
The existing `_merge` step dedups by name. If WD14 and embedding similarity both suggest the same tag, the higher-confidence one wins.
|
||||
|
||||
`get_suggestions` returns three groups: `character`, `copyright`, `general`. The `rating` group is removed.
|
||||
|
||||
### Accept endpoint
|
||||
|
||||
In `app/main.py`:
|
||||
|
||||
- `_WD14_CATEGORY_TO_KIND` drops the `'rating': 'rating'` entry. It becomes `{'character': 'character', 'copyright': 'fandom'}`.
|
||||
- The centroid-recompute trigger block currently guarded by `if tag.kind == 'character'` changes to `if tag.kind in ELIGIBLE_CENTROID_KINDS`. Because `ELIGIBLE_CENTROID_KINDS` includes `None`, null-kind general tags are covered by the same check.
|
||||
- Delta-triggered recompute stays at `centroid_recompute_delta = 3` globally.
|
||||
|
||||
### Config cleanup
|
||||
|
||||
The existing `tag_suggestion_config` rows are renamed/removed:
|
||||
|
||||
- `threshold_embedding_character` → `threshold_embedding` (same default: 0.85)
|
||||
- `min_reference_images_per_character` → `min_reference_images` (same default: 5)
|
||||
- `threshold_rating` → deleted
|
||||
- Other thresholds (`threshold_general`, `threshold_character_wd14`, `threshold_copyright`) stay unchanged since they still gate WD14 suggestions.
|
||||
|
||||
### UI
|
||||
|
||||
Modal template (`app/templates/_gallery_modal.html`) and `view-modal.js`:
|
||||
|
||||
- Suggestion category order becomes `['character', 'copyright', 'general']` — `rating` is removed.
|
||||
- Category label map drops the `rating` entry.
|
||||
- No visual changes to the chip layout (already updated in a prior change).
|
||||
|
||||
## Data flow on accept
|
||||
|
||||
```
|
||||
user clicks ✓
|
||||
→ POST /image/<id>/suggestions/accept
|
||||
→ tag created if missing (kind = WD14 category map, or None)
|
||||
→ tag attached to image
|
||||
→ if tag.kind in ELIGIBLE_CENTROID_KINDS or None, check delta:
|
||||
current_image_count - last_reference_count >= 3
|
||||
→ enqueue recompute_centroid(tag_name) on ml queue
|
||||
→ ml-worker runs recompute_centroid
|
||||
→ mean-pools embeddings, upserts tag_reference_embedding row
|
||||
→ next image's suggestion request benefits from the updated centroid
|
||||
```
|
||||
|
||||
## Migration plan for existing data
|
||||
|
||||
The database already has one row in `character_reference_embedding` per character tag touched so far. The migration:
|
||||
|
||||
1. `ALTER TABLE character_reference_embedding RENAME TO tag_reference_embedding`.
|
||||
2. `ALTER TABLE tag_reference_embedding RENAME COLUMN character_tag TO tag_name`.
|
||||
3. `ALTER TABLE tag_reference_embedding ADD COLUMN tag_kind TEXT NULL`.
|
||||
4. Backfill existing rows: `UPDATE tag_reference_embedding SET tag_kind = 'character'` (safe because every pre-existing row was written by the old character-only path).
|
||||
5. Drop `threshold_rating` row from `tag_suggestion_config`.
|
||||
6. Rename `threshold_embedding_character` → `threshold_embedding`.
|
||||
7. Rename `min_reference_images_per_character` → `min_reference_images`.
|
||||
|
||||
After migration, the user clicks "Recompute all centroids" once from Settings → Maintenance to populate centroids for any fandom/general tags they've already applied to 5+ images.
|
||||
|
||||
### Existing rating data is preserved
|
||||
|
||||
The migration does not delete any `Tag` rows with `kind='rating'` or their image-tag associations. WD14 continues to store rating predictions in `image_tag_prediction` (category='rating') on every tag_and_embed run. This data is simply no longer surfaced in the suggestions UI or accepted via the new endpoint paths. If the user ever wants rating gating back, re-adding it is a UI-only change.
|
||||
|
||||
## Error handling
|
||||
|
||||
- `recompute_centroid` for an ineligible kind: logs and returns `{'status': 'ineligible_kind'}`. No retry.
|
||||
- `recompute_centroid` for a tag with 0 images (e.g., all images that used this tag were deleted): logs and returns `{'status': 'no_embeddings'}`. No retry.
|
||||
- `recompute_all_centroids` failures in individual enqueues: logged, loop continues.
|
||||
- Accept endpoint centroid enqueue failure: swallowed (existing behavior — don't fail the accept).
|
||||
|
||||
## Testing plan
|
||||
|
||||
Manual verification after the implementation lands:
|
||||
|
||||
1. Migration runs cleanly on the dev DB. Existing character centroid survives with `tag_kind='character'`.
|
||||
2. Fresh suggestion request for an image returns three groups (no `rating`).
|
||||
3. Tag a fandom across 5 images, trigger recompute, verify new image shows that fandom as a suggestion.
|
||||
4. Tag a null-kind general topic across 5 images, trigger recompute, verify the general group now includes it.
|
||||
5. Accept a general-kind suggestion — verify `Tag(kind=None)` is created, image gets it, centroid recompute is enqueued after the delta threshold is met.
|
||||
6. Settings → Maintenance → "Recompute all centroids" button enqueues recomputes only for eligible-kind tags with ≥ 5 images.
|
||||
|
||||
No automated tests in this spec — the base feature didn't have any either, and the manual test plan in the parent spec still covers the integration surface.
|
||||
Reference in New Issue
Block a user