Adds a Celery task + Settings button that walks every image and applies
general-category WD14 predictions at or above the auto-accept threshold,
without needing the user to open each modal. Same side effects as the
modal's per-image auto-accept flow (creates kind='user' tag if needed,
attaches to image, writes SuggestionFeedback decision='accepted').
Lazy-imports app.services.tag_suggestions so the maintenance worker
doesn't pay its overhead unless this task fires. Amortizes _config()
and _existing_tag_names() across the loop. Commits per-batch (100
images) to keep transactions short and let later batches see freshly
created Tag rows.
Skips integrity-flagged images (get_suggestions already returns empty
for those). Idempotent — re-running just no-ops on already-applied
images via the existing 'tag not in img.tags' guard.
Trigger: Settings -> Maintenance -> "Apply auto-accepts library-wide"
(confirm-gated since it walks the entire library).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds per-image integrity tracking so corrupt files are detected, excluded
from random/showcase/ML/suggestion paths, and recoverable by dropping a
fresh copy in /import — closing the gap that surfaced as the WD14
'6 bytes not processed' OSError.
Schema (migration l26042501)
- image_record.integrity_status: unknown | ok | truncated | unreadable | missing
- image_record.integrity_checked_at: timestamptz
- partial index on status <> 'ok' for cheap report/filter queries
Verifier
- app/services/integrity.py: verify_path() dispatches by extension
- PIL two-stage (verify + load with LOAD_TRUNCATED_IMAGES disabled)
- ffprobe for video, zipfile.testzip for archives
- Truncation-vs-unreadable distinction via PIL message hints
Pipeline
- verify_media_integrity Celery task: per-image, idempotent
- verify_unverified_images sweep: only_unknown by default, skips
paths in active import tasks
- Hooked into the end of import_media_file (new + archive paths) and
the supersede branch
- supersede_image() resets status to 'unknown' so the post-supersede
verify writes a fresh truth
- Supersede-on-replace: a fresh /import/<artist>/<filename> matching
a flagged-corrupt record routes through _supersede_existing,
preserving tags/series/embeddings
Exclusions
- /, /api/random-images, tag_and_embed, ml.backfill enqueue, and
get_suggestions all filter integrity_status IN ('ok', 'unknown') so
flagged rows don't poison the gallery, ML, or suggestion math.
'unknown' is treated as healthy so post-migration data stays visible
until the sweep runs.
UI / report
- Settings -> Maintenance: 'Verify unknown' + 'Force re-verify all'
- GET /api/integrity/failed (paginated list of flagged rows)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New Celery task sync_character_fandoms_to_images on the maintenance
queue. For every character tag with a non-null fandom_id, runs the
same INSERT ... SELECT ... ON CONFLICT DO NOTHING that set_tag_fandom
already runs inline, attaching the fandom tag to every image that
has the character attached.
Why this exists: migration j26042101 extracted the '(Fandom)' suffix
from pre-refactor character names into tag.fandom_id, but it did NOT
walk image_tags to attach the fandom rows to every image that had
the character. Post-refactor auto-apply (in add_tag /
accept_image_suggestion / set_tag_fandom) only fires on NEW
add/accept events — pre-existing character attachments from before
the migration still show the character pill in the modal but no
fandom pill, because the fandom row was never added to image_tags.
Surfaced by Settings → Maintenance → 'Sync fandoms to images'.
Safe, additive, idempotent. Run once after the migration to close
the gap; subsequent set-fandom operations continue to maintain the
invariant inline.
Verified locally: seeded a character+fandom pair with only the
character attached to image 1; task added the fandom to image_tags;
final state has both rows.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When the user clicks the × on a tag pill, the tag is removed from the
image and a small '/Removed 'X' — [⊘ Blocklist] [×]/' control surfaces
below the tag input. One click on ⊘ Blocklist escalates the removal
to a global blocklist add (which also enqueues the retroactive sweep
Celery task, removing the tag from every other image that has it).
Fades after 8s, hides on image switch or modal close.
Targets the composition-descriptor case: WD14 tags like '1girl',
'breasts', 'simple background' that show up on many images. Without
this, after removing the pill the user has to navigate to Settings →
Maintenance → Suggestion blocklist to add the name, or wait for the
next suggestion chip to appear and use its ⊘. Now it's one click at
the point of removal.
Misclick recovery unchanged: the pill × alone only rejects per-image.
The inline offer is opt-in; ignore it or click × to dismiss and the
removal stays scoped to the current image only.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two capabilities that work together to turn the suggestion system from
a 'review every noisy chip' UX into a 'curated tags only, configurable
auto-apply' UX for a solo-user library.
Auto-accept threshold
Service (tag_suggestions.py):
- New default auto_accept_general_threshold = 0.95 in _DEFAULTS
- get_suggestions splits general-category hits at/above the threshold
into a separate auto_accept_candidates list; the core 'character'/
'copyright'/'general' keys stay the same shape for existing callers
- get_bulk_suggestions ignores the new key (no side effects in bulk)
Route (main.py GET /image/<id>/suggestions):
- For each candidate: find-or-create Tag(kind='user', name=name), attach
to image, log SuggestionFeedback(source='wd14'|etc, decision='accepted')
- Response adds 'auto_accepted: [{id, name, display_name, kind,
confidence, source}, ...]' so the modal can review + undo
Config endpoints (main.py):
GET /api/suggestions/config/auto-accept-threshold
POST /api/suggestions/config/auto-accept-threshold {threshold}
Values > 1.0 effectively disable the feature
UI (view-modal.js):
- New renderAutoAccepted block at top of suggestions section with
green-tinted chips carrying ✕ (undo for this image, logs rejection)
and ⊘ (blocklist + remove from all images)
- loadSuggestions refreshes the tag pill list when auto_accepted has
items so the user sees them in the Tags section too
Settings:
- Maintenance tab gains a number input + save button backed by
/api/suggestions/config/auto-accept-threshold
Retroactive blocklist sweep
New Celery task app.tasks.maintenance.sweep_blocklisted_tag_from_images
on the maintenance queue. Enqueued automatically when a name is added
via POST /api/suggestions/blocklist or appears new in the bulk replace.
Task body: finds kind='user' Tag matching the name, deletes its
image_tags rows, deletes the Tag itself. Scope limited to kind='user'
so deliberate character/fandom/artist tags sharing a blocklisted name
aren't silently destroyed.
Verification (local dev):
- Threshold GET/POST round-trip correct (default 0.95, persisted in
tag_suggestion_config)
- Image 633 at threshold=0.9: 4 general tags auto-applied, each with
a SuggestionFeedback row, returned in auto_accepted
- Blocklist add of 'sweeptestonly' with an attached test tag: Redis
maintenance queue depth = 1, task body removes 1 image_tags row +
the Tag row
- get_bulk_suggestions still works (auto_accept_candidates key skipped)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a tag_suggestion_blocklist table + service-layer filter so the
user can permanently suppress specific canonical tag names from the
suggestion stream (WD14 anatomy/composition tags like '1girl',
'breasts', 'nipples' that aren't useful for the discoverability use
case this app targets).
Data model
- migration k26042201: tag_suggestion_blocklist(name TEXT PK, created_at)
- model TagSuggestionBlocklistEntry
Service
- tag_suggestions._blocklisted_names() snapshots the current set
- get_suggestions filters merged results before grouping, so both
WD14- and embedding-sourced suggestions respect the blocklist
- get_bulk_suggestions inherits the filter via its per-image call
to get_suggestions
API
- GET /api/suggestions/blocklist -> {ok, names}
- POST /api/suggestions/blocklist -> add one
- POST /api/suggestions/blocklist/delete -> remove one
- POST /api/suggestions/blocklist/bulk -> replace the whole list
(backs the settings textarea save button)
UI
- modal suggestion chip gets a third action button (⊘) alongside
accept (✓) / reject (✕). Clicking it adds the name to the
blocklist, logs a rejection for ML feedback on this image, and
sweeps every chip on the page carrying that same name.
- Settings -> Maintenance -> Suggestion blocklist section with a
monospaced textarea (one name per line) + Save. Loads current
entries on mount.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1. Hide 'archive' kind from /tags default view. Archive tags only
appear when the user explicitly requests them via ?kind=archive.
_get_tags_with_previews gains a default_excluded_kinds tuple that
applies when no explicit kind filter is set.
2. Hide the 'Add' submit button on desktop (>=768px). Enter in the
tag input (and in the fandom picker when no row is highlighted)
already submits; the button is only needed for mobile virtual
keyboards where Enter-to-submit is unreliable.
3. Fandom picker keyboard parity: ArrowUp/Down navigate rows, Enter
picks a highlighted row or (if no row is highlighted) submits the
whole add-tag form. Escape clears the highlight. Mirrors the main
tagInput's keyboard block.
4. Tag editor merge bug: the edit modal populated nameInput.value
with data.tag.display_name — for a character with a fandom, that
value is 'Name (Fandom)'. Saving without stripping the suffix
routed to a rename that never collided with the canonical bare
name, so the merge offer never fired. Now uses data.tag.name (the
bare form). The (Fandom) display is still shown in the fandom
input which is rendered separately.
5. Fandom picker styling: match the main tag-form input — same
padding, border, background, focus state. Drop the separate
'Fandom' label; the placeholder 'Fandom (optional)' now carries
the label role.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
HTML/CSS for the inline fandom picker that slides in below the add-tag
input. JS wiring in the next commit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a header counter ('⚠ N characters need a fandom') that filters to
?null_fandom=1, plus an inline chip on each qualifying character card.
Backend accepts null_fandom_only in the list query.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Collapses the six-branch prefix-strip expression into a KIND_EMOJI
jinja dict. Anchor hrefs use ?tag_id=.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
tag.fandom_id is now authoritative. set_tag_fandom auto-applies the
fandom tag inline; add_tag already did. There's no drift to sync on
a schedule anymore.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the floating autocomplete popup with a reserved ~5-row inline
area below the add-tag input. It never covers provenance or suggestion
chips — they stay visible side-by-side with the autocomplete list while
the user types. The area scrolls when more matches are loaded, and is
prefilled with top tags on every image load so it's never empty.
The base .tag-autocomplete class (still used by bulk-select and the
fandom picker) keeps its floating behavior; the modal opts in to the
new inline layout via an added .tag-autocomplete-inline modifier.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bulk editor now loads consensus ML suggestions across the current
selection via a new POST /api/suggestions/bulk endpoint, powered by
get_bulk_suggestions() in tag_suggestions.py. A tag is surfaced only if
it was suggested for or already applied to >= 80% of the selection;
coverage counts include images that already have the tag so a near-
universal tag isn't penalized for dropping out of suggestion lists.
Accept-only chips (no reject) match the rest of the suggestions UX.
Frontend polish in the same commit:
- Showcase session dedup (exclude seen ids, reset on second lap) and
aspect-ratio-aware placeholder heights for better column balancing
- Modal touch-swipe navigation on the image wrapper, auto-focus of the
tag input on desktop (>=768px), and autocomplete flip-up when the
dropdown would cover the Suggestions section
- Settings template inline styles replaced with utility classes
- Inline series-editor script extracted to gallery-series-editor.js
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Consolidated merge of feat/tag-suggestions branch. Original 64-commit history
was lost to git-object corruption in a Nextcloud-synced checkout; this single
commit captures the equivalent diff.
Includes:
- pgvector-backed tag suggestion infra (WD14 + SigLIP centroids, ml-worker
container, Celery tasks, suggestion service, accept/reject endpoints + modal
UI with green/red chip buttons)
- Character/fandom integrity: title-case normalization on every write path,
fandom-id backfill, maintenance task + settings button, migrations g26041901
+ h26041901 to canonicalize legacy rows with case-only duplicate merging
- Tag-underscores + modal polish: WD14 name canonicalization at emit + accept
+ add/bulk-add paths, migration i26041901 for legacy-row rename-or-merge
across character/fandom/NULL kinds, suggestion-accept refresh parity via
awaited loadTags, persistent chip tint
Each settings tab is now a separate page load via ?tab= query param.
Flask renders only the active panel - no CSS or JS visibility logic needed.
Tab bar uses <a> links instead of buttons. Eliminates the display:none
cascade issue that was preventing Import and Maintenance tabs from showing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The class-based is-active/display approach was unreliable due to CSS cascade
issues. Switching to direct style.display manipulation bypasses specificity
entirely. Also added cache-busting ?v=2 to style.css link.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Link character tags to fandom tags via fandom_id FK with auto-apply
behavior. Adding character:Saber (Fate) auto-creates fandom:Fate and
applies it to the image. Renaming a fandom cascades to all linked
characters. Includes set-fandom endpoint for retroactive association
and fandom selector dropdown in the tag editor UI.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Quick scan (default):
- Pre-load all ImportTask records for O(1) duplicate checks
- Batch task creation commits (50 at a time)
- Cache import settings (5-min TTL)
- Skip pHash comparison for speed
Deep scan (on-demand):
- Full reprocessing of all files
- pHash similarity detection
- Optional thumbnail regeneration
- Optional sidecar re-application
Archive tag merging:
- Add archive tags to existing images when duplicates found
- Works for both hash and pHash duplicate detection
Also fixes:
- Add missing source/post tag icons to gallery templates
bvandeusen
Bryan Van Deusen