docs: README matches current stack and covers ML suggestions

Quick Start had drifted: postgres:16-alpine instead of pgvector/pg16,
a single celery-worker + celery-beat layout instead of the actual
worker / scheduler / ml-worker split, and no mention of the ML
tag-suggestion surface. Updates the compose example to mirror the
real docker-compose.yml, adds ml-worker env vars and /models volume
docs, and documents the ML maintenance tools in Settings.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-04-19 21:18:31 -04:00
parent fb925f5e6a
commit 70957a307c
+52 -26
View File
@@ -10,6 +10,7 @@ A self-hosted image and video gallery application designed for organizing and vi
- **Video Support** - Playback for video files with automatic transcoding to MP4
- **Tagging System** - Organize images with tags (artist, character, series, rating, archive, user-defined)
- **Tag Autocomplete** - Quick tag entry with search suggestions
- **ML Tag Suggestions** - WD14 tagger + SigLIP embedding centroids propose tags you can accept/reject per image
- **Automatic Importing** - Celery-based task queue scans `/import` directory on schedule
- **Archive Extraction** - Supports ZIP, RAR, 7z and other archive formats
- **Duplicate Detection** - Perceptual hash (pHash) comparison to skip similar images
@@ -35,7 +36,7 @@ services:
retries: 5
postgres:
image: postgres:16-alpine
image: pgvector/pgvector:pg16
environment:
POSTGRES_USER: imagerepo
POSTGRES_PASSWORD: your_secure_password
@@ -52,7 +53,7 @@ services:
image: git.fabledsword.com/bvandeusen/imagerepo:latest
ports:
- "5000:5000"
environment:
environment: &app_env
- DB_USER=imagerepo
- DB_PASS=your_secure_password
- DB_HOST=postgres
@@ -68,16 +69,11 @@ services:
redis:
condition: service_healthy
celery-worker:
# Heavy processing: import, thumbnail, sidecar, default queues.
worker:
image: git.fabledsword.com/bvandeusen/imagerepo:latest
command: celery -A app.celery_app:celery worker --loglevel=info -Q scan,import,thumbnail,sidecar,default --concurrency=2
environment:
- DB_USER=imagerepo
- DB_PASS=your_secure_password
- DB_HOST=postgres
- DB_NAME=imagerepo
- CELERY_BROKER_URL=redis://redis:6379/0
- CELERY_RESULT_BACKEND=redis://redis:6379/0
command: celery -A app.celery_app:celery worker --loglevel=info -Q import,thumbnail,sidecar,default --concurrency=2
environment: *app_env
volumes:
- ./imagerepo/images:/images
- ./your-media:/import
@@ -87,20 +83,33 @@ services:
redis:
condition: service_healthy
celery-beat:
# Beat scheduler + maintenance/scan worker, split off so long imports don't starve periodic tasks.
scheduler:
image: git.fabledsword.com/bvandeusen/imagerepo:latest
command: celery -A app.celery_app:celery beat --loglevel=info
environment:
- DB_USER=imagerepo
- DB_PASS=your_secure_password
- DB_HOST=postgres
- DB_NAME=imagerepo
- CELERY_BROKER_URL=redis://redis:6379/0
- CELERY_RESULT_BACKEND=redis://redis:6379/0
- IMPORT_EVERY_SECONDS=28800
command: celery -A app.celery_app:celery worker --beat --loglevel=info -Q maintenance,scan --concurrency=1
environment: *app_env
volumes:
- ./imagerepo/images:/images
- ./your-media:/import
depends_on:
- redis
- celery-worker
postgres:
condition: service_healthy
redis:
condition: service_healthy
# CPU-only ML inference: WD14 tags + SigLIP embeddings. Models self-heal into ${MODELS_DIR} on start.
ml-worker:
image: git.fabledsword.com/bvandeusen/imagerepo-ml:latest
command: celery -A app.celery_app:celery worker --loglevel=info -Q ml --concurrency=1
environment: *app_env
volumes:
- ./imagerepo/images:/images:ro
- ./imagerepo/models:/models
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
volumes:
redis_data:
@@ -114,9 +123,10 @@ Then visit `http://localhost:5000` in your browser.
ImageRepo uses a task queue architecture for background processing:
- **Web** - Flask application serving the UI and API
- **Celery Worker** - Processes import, thumbnail, and metadata tasks
- **Celery Beat** - Schedules periodic tasks (directory scans, recovery)
- **PostgreSQL** - Primary database for all data
- **Worker** - Heavy processing: `import`, `thumbnail`, `sidecar`, `default` queues
- **Scheduler** - Celery Beat + a `maintenance`/`scan` worker (kept separate so long imports don't starve periodic tasks)
- **ML Worker** - CPU-only WD14 + SigLIP inference on the `ml` queue (separate image, models self-heal on start)
- **PostgreSQL** - Primary database, uses `pgvector` extension for SigLIP embedding similarity
- **Redis** - Message broker for Celery task queue
## Volumes
@@ -125,6 +135,7 @@ ImageRepo uses a task queue architecture for background processing:
|------|-------------|
| `/images` | Where imported images and thumbnails are stored |
| `/import` | Source directory the importer scans for new media |
| `/models` | ml-worker only — WD14 + SigLIP weights (~4 GB, fetched on first start) |
## Environment Variables
@@ -176,6 +187,16 @@ These can also be configured via the Settings page in the UI.
| `ARCHIVE_MIN_FREE_GB` | `0` | Minimum free disk space (GB) required to start extraction (0 = disabled) |
| `ARCHIVE_NUM_WIDTH` | `4` | Zero-padding width for archive sequence numbers in tags |
### ML Tag Suggestions (ml-worker only)
| Variable | Default | Description |
|----------|---------|-------------|
| `ML_MODEL_DIR` | `/models` | Where WD14 + SigLIP weights are written/read |
| `WD14_REPO` | `SmilingWolf/wd-eva02-large-tagger-v3` | HuggingFace repo for the WD14 tagger |
| `SIGLIP_REPO` | `google/siglip-so400m-patch14-384` | HuggingFace repo for the SigLIP encoder |
| `WD14_REVISION` | (latest) | Pin WD14 to a specific commit SHA |
| `SIGLIP_REVISION` | (latest) | Pin SigLIP to a specific commit SHA |
### Other
| Variable | Default | Description |
@@ -227,6 +248,11 @@ The Settings page (`/settings`) provides:
- **Find Duplicates** - Scan for visually similar images using pHash
- **Reset Image Database** - Clear all image records (files remain on disk)
### ML Maintenance Tools
- **Run ML backfill** - Enqueue `tag_and_embed` for every image missing predictions or embeddings for the current model versions. Safe to re-run; paginates forward and drops already-processed images. Expected runtime on a fresh DB: hours to days on a single ml-worker.
- **Recompute all centroids** - Rebuild per-tag SigLIP centroids from current image tags. Run after the initial backfill drains, or whenever bulk manual tagging has drifted suggestions.
- **Sync character fandoms** - Additively re-apply each character's fandom tag to every image already tagged with that character. Never removes existing fandom tags.
### Import Filters
Configure filtering rules that apply during import:
- Minimum dimensions