bvandeusen/imagerepo
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added dedup tools to settings. first pass of mass tag editting to gallery view.

Browse Source
This commit is contained in:
Bryan Van Deusen
2026-01-20 23:45:54 -05:00
parent 87bcb633f6
commit ede1457abc
10 changed files with 1411 additions and 182 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+136 -51
View File
@@ -7,10 +7,10 @@ A self-hosted image and video gallery application designed for organizing and vi
- **Gallery View** - Browse images in a paginated grid with thumbnail previews
- **Showcase View** - Randomized masonry layout for discovery
- **Image Modal** - Full-size viewing with zoom, pan, and keyboard navigation
- **Video Support** - Playback for video files with thumbnail generation
- **Video Support** - Playback for video files with automatic transcoding to MP4
- **Tagging System** - Organize images with tags (artist, character, series, rating, archive, user-defined)
- **Tag Autocomplete** - Quick tag entry with search suggestions
- **Automatic Importing** - Scans `/import` directory on schedule or manual trigger
- **Automatic Importing** - Celery-based task queue scans `/import` directory on schedule
- **Archive Extraction** - Supports ZIP, RAR, 7z and other archive formats
- **Duplicate Detection** - Perceptual hash (pHash) comparison to skip similar images
- **Import Filters** - Skip images by dimensions or transparency percentage
@@ -24,44 +24,129 @@ A self-hosted image and video gallery application designed for organizing and vi
```yaml
services:
imagerepo:
redis:
image: redis:7-alpine
volumes:
- redis_data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 5s
retries: 5
postgres:
image: postgres:16-alpine
environment:
POSTGRES_USER: imagerepo
POSTGRES_PASSWORD: your_secure_password
POSTGRES_DB: imagerepo
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U imagerepo"]
interval: 10s
timeout: 5s
retries: 5
web:
image: git.fabledsword.com/bvandeusen/imagerepo:latest
ports:
- "5000:5000"
volumes:
- ./imagerepo/db:/db # SQLite database storage
- ./imagerepo/images:/images # Imported images and thumbnails
- ./your-media:/import # Source directory to import from
environment:
- THUMBS_VERBOSE=1
- THUMBS_LOG_EVERY=50
- DB_USER=imagerepo
- DB_PASS=your_secure_password
- DB_HOST=postgres
- DB_NAME=imagerepo
- CELERY_BROKER_URL=redis://redis:6379/0
- CELERY_RESULT_BACKEND=redis://redis:6379/0
volumes:
- ./imagerepo/images:/images
- ./your-media:/import
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
celery-worker:
image: git.fabledsword.com/bvandeusen/imagerepo:latest
command: celery -A app.celery_app:celery worker --loglevel=info -Q scan,import,thumbnail,sidecar,default --concurrency=2
environment:
- DB_USER=imagerepo
- DB_PASS=your_secure_password
- DB_HOST=postgres
- DB_NAME=imagerepo
- CELERY_BROKER_URL=redis://redis:6379/0
- CELERY_RESULT_BACKEND=redis://redis:6379/0
volumes:
- ./imagerepo/images:/images
- ./your-media:/import
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
celery-beat:
image: git.fabledsword.com/bvandeusen/imagerepo:latest
command: celery -A app.celery_app:celery beat --loglevel=info
environment:
- DB_USER=imagerepo
- DB_PASS=your_secure_password
- DB_HOST=postgres
- DB_NAME=imagerepo
- CELERY_BROKER_URL=redis://redis:6379/0
- CELERY_RESULT_BACKEND=redis://redis:6379/0
- IMPORT_EVERY_SECONDS=28800
depends_on:
- redis
- celery-worker
volumes:
redis_data:
postgres_data:
```
Then visit `http://localhost:5000` in your browser.
## Architecture
ImageRepo uses a task queue architecture for background processing:
- **Web** - Flask application serving the UI and API
- **Celery Worker** - Processes import, thumbnail, and metadata tasks
- **Celery Beat** - Schedules periodic tasks (directory scans, recovery)
- **PostgreSQL** - Primary database for all data
- **Redis** - Message broker for Celery task queue
## Volumes
| Path | Description |
|------|-------------|
| `/db` | Database storage (SQLite by default) |
| `/images` | Where imported images and thumbnails are stored |
| `/import` | Source directory the importer scans for new media |
## Environment Variables
### Database Configuration
By default, ImageRepo uses SQLite. Set `DB_TYPE=postgresql` to use PostgreSQL instead.
### Database Configuration (PostgreSQL)
| Variable | Default | Description |
|----------|---------|-------------|
| `DB_TYPE` | `sqlite` | Database type: `sqlite` or `postgresql` |
| `DB_NAME` | `app.db` | Database name (SQLite filename or PostgreSQL database) |
| `DB_NAME` | `imagerepo` | PostgreSQL database name |
| `DB_USER` | `imagerepo` | PostgreSQL username |
| `DB_PASS` | `postgres` | PostgreSQL password |
| `DB_HOST` | `postgres` | PostgreSQL host |
| `DB_PORT` | `5432` | PostgreSQL port |
### Celery Configuration
| Variable | Default | Description |
|----------|---------|-------------|
| `CELERY_BROKER_URL` | `redis://redis:6379/0` | Redis broker URL |
| `CELERY_RESULT_BACKEND` | `redis://redis:6379/0` | Redis result backend URL |
| `CELERY_WORKER_CONCURRENCY` | `2` | Number of worker processes |
| `IMPORT_EVERY_SECONDS` | `28800` | Directory scan interval (8 hours) |
### Import Settings
These can also be configured via the Settings page in the UI.
@@ -72,16 +157,16 @@ These can also be configured via the Settings page in the UI.
| `IMPORT_MIN_HEIGHT` | `0` | Minimum image height in pixels (0 = no minimum) |
| `IMPORT_SKIP_TRANSPARENT` | `false` | Skip mostly-transparent images (PNG/GIF/WebP) |
| `IMPORT_TRANSPARENCY_THRESHOLD` | `0.9` | Transparency % threshold (0.0-1.0) for skipping |
| `IMPORT_PHASH_THRESHOLD` | `2` | Duplicate detection sensitivity (0=disabled, 1=strict, 2=normal, 5=loose, 10=very loose) |
| `IMPORT_PHASH_THRESHOLD` | `10` | Duplicate detection sensitivity (0=disabled, lower=stricter) |
### Thumbnail Generation
### Thumbnail & Video Processing
| Variable | Default | Description |
|----------|---------|-------------|
| `THUMBS_VERBOSE` | `0` | Enable verbose logging (`1`, `true`, or `yes`) |
| `THUMBS_LOG_EVERY` | `50` | Log progress every N thumbnails |
| `THUMBS_BATCH_SIZE` | `500` | Process thumbnails in batches of N |
| `THUMBS_FFMPEG_TIMEOUT` | `60` | Timeout in seconds for video thumbnail extraction |
| `FFMPEG_TRANSCODE_TIMEOUT` | `600` | Timeout in seconds for video transcoding |
### Archive Extraction
@@ -100,8 +185,12 @@ These can also be configured via the Settings page in the UI.
## Import Behavior
The importer runs:
- Automatically every 8 hours
- Manually via the "Trigger Image Import" button in Settings
- Automatically every 8 hours (configurable via `IMPORT_EVERY_SECONDS`)
- Manually via the "Trigger Import Scan" button in Settings
### Video Transcoding
Non-MP4 video formats (.mov, .avi, .mkv, .webm, .m4v, .wmv, .flv) are automatically transcoded to H.264 MP4 during import for universal browser playback.
### Folder Structure
@@ -125,7 +214,7 @@ The importer uses the folder structure to auto-tag images:
### Supported Formats
**Images:** PNG, JPG, JPEG, GIF, WebP, BMP, TIFF
**Videos:** MP4, WebM, MKV, AVI, MOV
**Videos:** MP4, MOV, AVI, MKV, WebM, M4V, WMV, FLV (transcoded to MP4)
**Archives:** ZIP, RAR, 7z, and others supported by `unar`/`7z`
## Settings Page
@@ -133,8 +222,9 @@ The importer uses the folder structure to auto-tag images:
The Settings page (`/settings`) provides:
### Admin Actions
- **Trigger Image Import** - Manually start an import scan
- **Trigger Import Scan** - Manually start an import scan
- **Regenerate Thumbnails** - Rebuild all thumbnails
- **Find Duplicates** - Scan for visually similar images using pHash
- **Reset Image Database** - Clear all image records (files remain on disk)
### Import Filters
@@ -143,6 +233,11 @@ Configure filtering rules that apply during import:
- Transparency filtering
- Duplicate detection sensitivity
### Duplicate Management
- Scan for potential duplicates based on perceptual hash distance
- Review and confirm duplicates for deletion
- Keep the higher resolution version automatically
### Delete Images by Tag
Bulk delete all images associated with a specific tag (e.g., remove an entire artist's collection).
@@ -156,7 +251,10 @@ Scan and selectively delete images that match filter criteria (transparency or d
| `/api/random-images` | GET | Get random images for showcase |
| `/api/tags/search` | GET | Search tags for autocomplete |
| `/api/import-settings` | GET/POST | Read/write import filter settings |
| `/api/import-stats` | GET | Get last import statistics |
| `/api/import/trigger` | POST | Trigger directory scan |
| `/api/import/status` | GET | Get import queue status |
| `/api/duplicates/scan` | POST | Scan for duplicate images |
| `/api/duplicates/delete` | POST | Delete confirmed duplicates |
| `/api/filter-scan` | GET | Scan images matching filter criteria |
| `/api/filter-delete` | POST | Delete selected images |
| `/api/delete-by-tag` | POST | Delete all images with a tag |
@@ -172,38 +270,20 @@ In the image modal:
- **Click image** - Toggle zoom
- **Drag** (when zoomed) - Pan around image
## PostgreSQL Setup
## Scaling Workers
To use PostgreSQL instead of SQLite:
To increase import throughput:
```yaml
services:
imagerepo:
image: git.fabledsword.com/bvandeusen/imagerepo:latest
ports:
- "5000:5000"
environment:
- DB_TYPE=postgresql
- DB_HOST=postgres
- DB_USER=imagerepo
- DB_PASS=your_secure_password
- DB_NAME=imagerepo
volumes:
- ./imagerepo/images:/images
- ./your-media:/import
depends_on:
- postgres
```bash
# Scale worker containers
docker-compose up --scale celery-worker=4
postgres:
image: postgres:15
environment:
- POSTGRES_USER=imagerepo
- POSTGRES_PASSWORD=your_secure_password
- POSTGRES_DB=imagerepo
volumes:
- ./postgres-data:/var/lib/postgresql/data
# Or increase concurrency per worker
CELERY_WORKER_CONCURRENCY=4 docker-compose up
```
Total parallelism = containers × concurrency
## Development
### Local Setup
@@ -213,6 +293,9 @@ services:
git clone https://git.fabledsword.com/bvandeusen/imagerepo.git
cd imagerepo
# Start PostgreSQL and Redis
docker-compose up -d postgres redis
# Create virtual environment
python -m venv venv
source venv/bin/activate # or `venv\Scripts\activate` on Windows
@@ -220,6 +303,9 @@ source venv/bin/activate # or `venv\Scripts\activate` on Windows
# Install dependencies
pip install -r requirements.txt
# Run migrations
flask db upgrade
# Run with Flask dev server
flask run
```
@@ -228,7 +314,6 @@ flask run
```bash
docker build -t imagerepo .
docker run -p 5000:5000 -v ./images:/images -v ./import:/import imagerepo
```
## License