bvandeusen
afef95a87d
The agent workload is download-bound (download 400–5462ms vs GPU ~300–600ms), so the old N-slot serial chain (each slot: lease→download→decode→GPU→submit) left the fast GPU idle during every download. Rearchitect worker.py into a producer/consumer pipeline: downloader pool (autoscaled by BUFFER OCCUPANCY) → bounded queue → 1–2 GPU consumers (detect+embed→submit) - Downloaders are I/O-bound → many overlap; the autoscaler now tunes DOWNLOADER count by buffer fill (empty = GPU starving → add; full = outpacing GPU → add a 2nd consumer if it has util/VRAM headroom and lifts throughput, else trim). - Bounded buffer (12) = backpressure: a full buffer blocks downloaders, capping RAM + lease look-ahead. VRAM pressure sheds a consumer immediately. - Heartbeat thread keeps every held lease alive (buffered jobs wait on the GPU; curator's 180s TTL would otherwise reclaim them mid-buffer). - Preserves all resilience: lease exp-backoff, submit-path retry (#169), release-on-stop, region caps + video early-exit (#171). Stop drains BOTH pools and releases every held lease at once (single held-set as source of truth). - Consumers SHARE one embedder + proposers instance (a 2nd consumer adds concurrent inference, not N× VRAM — bounds the VRAM creep seen with N slots). - UI reworked for the pipeline: tiles show downloaders · buffer · on-GPU · processed · errors, a buffer-occupancy meter, and a consumers/waited-out line; the dial now tunes downloaders. Build marker 2026-07-01.1. Also fix the operator-flagged detector warning: yolo11n + the comic-panel model threw "'Conv' object has no attribute 'bn'" on every image (ultralytics' load- time Conv+BN fusion on a version-mismatched graph), silently disabling 2 of 3 crop proposers and spamming the log per image. Disable that fusion (unfused inference is correct, marginally slower) and permanently self-disable a proposer on the first inference failure instead of re-throwing forever. Refs milestone 122. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
FabledCurator
Self-hosted media curation — gallery, ML tagging, and subscription-driven downloading in one app. Part of the FabledSword family.
Combines what was ImageRepo (gallery, ML, importer) and GallerySubscriber (gallery-dl wrapper, subscriptions, credential capture) into a single product.
Status
Pre-v1. Not yet functional.
Quick start
For local development and testing, just:
docker compose up -d
# UI: http://localhost:8080
That uses sane dev defaults baked into docker-compose.yml and the dev
override (docker-compose.override.yml, auto-merged) — local builds, DEBUG
logging, exposed Postgres + Redis ports on the host. No .env required.
For a production-like deployment, override the dev defaults via shell env
or a .env file (see .env.example for the variable names) and use:
docker compose -f docker-compose.yml up -d
# (skips the override so containers pull registry images)
Deployment posture
FabledCurator is designed to run inside a self-hosted homelab environment over plain HTTP. If you want TLS, terminate it at your reverse proxy. The app does not generate certificates, redirect to HTTPS, or set HSTS.
CI / Forgejo setup
The repo's workflows expect:
-
Runner label
python-ci— a Forgejo runner with Python 3.14, ruff, and Node 22 pre-installed. Bothci.ymlandbuild.ymluse this label. The runner image (runner-base:python-ci) is built fromCI-Runner/CI-python/in the operator's workspace;make pushfrom that directory builds and pushes a new image when toolchain pins change. -
Repo secret
RELEASE_TOKEN— a Forgejo PAT with the following scopes:write:package+read:package— fordocker pushtogit.fabledsword.comwrite:release— for future release-cutting workflowswrite:issue— for future issue-management automation
Generate at https://git.fabledsword.com/user/settings/applications. The injected
GITHUB_TOKENcannot be used because it lackswrite:package.
License
Personal project; use at your own discretion.