Adds per-image integrity tracking so corrupt files are detected, excluded
from random/showcase/ML/suggestion paths, and recoverable by dropping a
fresh copy in /import — closing the gap that surfaced as the WD14
'6 bytes not processed' OSError.
Schema (migration l26042501)
- image_record.integrity_status: unknown | ok | truncated | unreadable | missing
- image_record.integrity_checked_at: timestamptz
- partial index on status <> 'ok' for cheap report/filter queries
Verifier
- app/services/integrity.py: verify_path() dispatches by extension
- PIL two-stage (verify + load with LOAD_TRUNCATED_IMAGES disabled)
- ffprobe for video, zipfile.testzip for archives
- Truncation-vs-unreadable distinction via PIL message hints
Pipeline
- verify_media_integrity Celery task: per-image, idempotent
- verify_unverified_images sweep: only_unknown by default, skips
paths in active import tasks
- Hooked into the end of import_media_file (new + archive paths) and
the supersede branch
- supersede_image() resets status to 'unknown' so the post-supersede
verify writes a fresh truth
- Supersede-on-replace: a fresh /import/<artist>/<filename> matching
a flagged-corrupt record routes through _supersede_existing,
preserving tags/series/embeddings
Exclusions
- /, /api/random-images, tag_and_embed, ml.backfill enqueue, and
get_suggestions all filter integrity_status IN ('ok', 'unknown') so
flagged rows don't poison the gallery, ML, or suggestion math.
'unknown' is treated as healthy so post-migration data stays visible
until the sweep runs.
UI / report
- Settings -> Maintenance: 'Verify unknown' + 'Force re-verify all'
- GET /api/integrity/failed (paginated list of flagged rows)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a tag_suggestion_blocklist table + service-layer filter so the
user can permanently suppress specific canonical tag names from the
suggestion stream (WD14 anatomy/composition tags like '1girl',
'breasts', 'nipples' that aren't useful for the discoverability use
case this app targets).
Data model
- migration k26042201: tag_suggestion_blocklist(name TEXT PK, created_at)
- model TagSuggestionBlocklistEntry
Service
- tag_suggestions._blocklisted_names() snapshots the current set
- get_suggestions filters merged results before grouping, so both
WD14- and embedding-sourced suggestions respect the blocklist
- get_bulk_suggestions inherits the filter via its per-image call
to get_suggestions
API
- GET /api/suggestions/blocklist -> {ok, names}
- POST /api/suggestions/blocklist -> add one
- POST /api/suggestions/blocklist/delete -> remove one
- POST /api/suggestions/blocklist/bulk -> replace the whole list
(backs the settings textarea save button)
UI
- modal suggestion chip gets a third action button (⊘) alongside
accept (✓) / reject (✕). Clicking it adds the name to the
blocklist, logs a rejection for ML feedback on this image, and
sweeps every chip on the page carrying that same name.
- Settings -> Maintenance -> Suggestion blocklist section with a
monospaced textarea (one name per line) + Save. Loads current
entries on mount.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Move __table_args__ below column declarations to match the convention
used by SeriesPage, ImportTask, ImageTagPrediction, SuggestionFeedback.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Drop unique=True on Tag.name (the migration replaced UNIQUE(name) with
four partial indices). Declare the partial indices in __table_args__ so
Alembic autogenerate stops seeing drift on future schema changes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Single source of truth for human-readable tag rendering. Falls back to
tag.name when no fandom is attached, preserving pre-migration data shapes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Consolidated merge of feat/tag-suggestions branch. Original 64-commit history
was lost to git-object corruption in a Nextcloud-synced checkout; this single
commit captures the equivalent diff.
Includes:
- pgvector-backed tag suggestion infra (WD14 + SigLIP centroids, ml-worker
container, Celery tasks, suggestion service, accept/reject endpoints + modal
UI with green/red chip buttons)
- Character/fandom integrity: title-case normalization on every write path,
fandom-id backfill, maintenance task + settings button, migrations g26041901
+ h26041901 to canonicalize legacy rows with case-only duplicate merging
- Tag-underscores + modal polish: WD14 name canonicalization at emit + accept
+ add/bulk-add paths, migration i26041901 for legacy-row rename-or-merge
across character/fandom/NULL kinds, suggestion-accept refresh parity via
awaited loadTags, persistent chip tint
Link character tags to fandom tags via fandom_id FK with auto-apply
behavior. Adding character:Saber (Fate) auto-creates fandom:Fate and
applies it to the image. Renaming a fandom cascades to all linked
characters. Includes set-fandom endpoint for retroactive association
and fandom selector dropdown in the tag editor UI.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
bvandeusen
Bryan Van Deusen