bvandeusen/FabledCurator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 33 Wiki Activity

33 Releases 33 Tags

RSS Feed
  • v26.05.23.1 27f7f3fd01
    Compare

    v26.05.23.1 — migration durability + dogfood UX Stable

    bvandeusen released this 2026-05-23 11:21:49 -04:00 | 758 commits to dev since this release

    Patch release rolling up live-migration findings and dogfood polish.

    Fixes

    • Engine pool leak: Celery task modules were creating a fresh engine per task call; high-fire-rate import_media_file exhausted Postgres max_connections. Centralized into one engine per worker process.
    • pg_dump URL: strip SQLAlchemy +psycopg/+asyncpg driver suffix before handing the URL to libpq backup/restore tools.
    • scan_directory idempotency: skip paths with non-failed ImportTask so re-triggered scans don't 3× duplicate work.
    • Modal redirect bug: <ImageViewer> now mounted globally in App.vue; tile clicks in Showcase/Artist views open as overlay instead of navigating to /gallery.

    UX

    • Thumbnail tiles 1.5× larger (gallery + masonry).
    • Settings view fluid width; new "Subscriptions" stat card on the system dashboard.
    • /subscriptions redesigned to a <v-data-table> of artists with expandable nested source rows (GS-style).

    Operator notes

    • Pull the new image; force-update app, worker, ml-worker, scheduler, beat services to pick up the engine-pool fix and the ruff cleanup. web (frontend) image is in the same build.
    • The Postgres max_connections=300 workaround in compose can stay; the new engine pool keeps active connections low single-digits per worker.
    • Backup tooling now works against postgresql+psycopg://... URLs end-to-end — backup → ingest cycle no longer needs the manual driver-strip workaround.
    Downloads
  • v26.05.23.0 c5bf564f53
    Compare

    v26.05.23.0 — FC-5 migration follow-ups Stable

    bvandeusen released this 2026-05-22 22:37:27 -04:00 | 766 commits to dev since this release

    First patch release of FabledCurator. Two follow-ups caught during the v26.05.22.0 production migration cut. Both unblock realistic-size migrations.

    Fixes

    pg_dump + zstd now present in the runtime image

    The FC-5 backup + rollback migrators shell out to pg_dump (DB snapshot) and tar --zstd (image-tree archive). The v26.05.22.0 image only had libpq5 (psycopg's client library) — neither pg_dump nor zstd was installed. Result: backup ran for ~10 minutes then surfaced [Errno 2] No such file or directory: 'pg_dump'. Patched the Dockerfile to install postgresql-client and zstd alongside the existing image deps.

    Quart request body cap lifted to 1 GiB

    Werkzeug's default form-memory cap (MAX_FORM_MEMORY_SIZE = 500 KB) was rejecting realistic IR exports — 281k image_tag associations from a real-world library produces a ~61 MB JSON export — with 413 Request Entity Too Large before the request reached the ir_ingest handler. MAX_CONTENT_LENGTH + MAX_FORM_MEMORY_SIZE now both set to 1 GiB in the app factory.

    Operator notes (post-pull)

    After your stack pulls :latest (force-update each service or redeploy the stack):

    1. Backup endpoint works. You can drop --skip-backup from ./FC-migrate.sh and the pre-migration snapshot will land in /images/_backups/.
    2. IR ingest accepts large exports. Re-run the migration — GS rows from your earlier ingest are preserved (idempotent on slug), so the GS step will skip and IR will go through.

    If you already started the migration on v26.05.22.0 and skipped the backup: just update services + re-run. The migrators are idempotent on natural keys.

    Downloads
  • v26.05.22.0 602c7d275d
    Compare

    v26.05.22.0 — FabledCurator v1 (FC-1 → FC-5) Stable

    bvandeusen released this 2026-05-22 14:16:23 -04:00 | 769 commits to dev since this release

    First FabledCurator release. Merges FC-1 (Foundation) → FC-5 (Migration tooling) onto main from dev. CalVer per FabledRulebook/forgejo.md.

    What's in this build

    The full FabledCurator merge of ImageRepo + GallerySubscriber:

    • FC-1 Foundation — Quart + Vue scaffold, unified Postgres schema (migrations 0001–0015), Celery + Redis, Docker setup.
    • FC-2 Image backbone (a–e) — gallery / showcase / modal viewer, importer with sha+phash dedup, ML pipeline (WD14 + SigLIP + centroids), discovery surfaces (tag directory, merge, bulk select, series), provenance system (artist as row, post + image_provenance + post_attachment), integrity verification.
    • FC-3 Subscription backbone (a–f) — Sources management, encrypted Credentials store, gallery-dl Downloader, Scheduling + source health with exponential backoff, Posts stream, Artists directory.
    • FC-5 Migration tooling — file-based ingest UI for ImageRepo + GallerySubscriber exports (/api/migrate/*, LegacyMigrationCard), backup + rollback, ML re-queue, sha256-sample verify.

    Coordinated export scripts on main of the legacy repos:

    • bvandeusen/imagerepo @ 2fb1a2a — scripts/export_for_fabledcurator.py.
    • bvandeusen/GallerySubscriber @ d47b4c8 — scripts/export_for_fabledcurator.py.

    Operator notes (migration workflow)

    1. Pull this image: git.fabledsword.com/bvandeusen/fabledcurator:main (or :latest).
    2. docker compose up -d with the standard FC compose file. On first boot alembic upgrade head brings the schema from empty to 0015.
    3. Run the export scripts on the host:
      • python ~/Nextcloud/Projects/ImageRepo/scripts/export_for_fabledcurator.py > ~/ir-export.json
      • python ~/Nextcloud/Projects/GallerySubscriber/scripts/export_for_fabledcurator.py --secret-key "$GS_SECRET_KEY" > ~/gs-export.json
    4. Bind-mount IR's images dir at /import/imagerepo:ro per the docker-compose.yml example.
    5. In FC UI → Settings → Maintenance → Legacy migration:
      • Click Backup FC (creates pre-migration snapshot).
      • Upload gs-export.json, click Ingest GS.
      • Upload ir-export.json, click Ingest IR.
      • Switch to Settings → Import tab → trigger the filesystem scan.
      • Back to Maintenance → Apply IR tags → Queue ML → Verify.
    6. Once verified, delete the export JSON files (the GS one contains plaintext credentials).

    Constraints

    Homelab HTTP. No TLS in the app itself. Self-hosted single-operator. ImageRepo + GallerySubscriber become archived once this build's migration completes on the homelab.

    Downloads