bvandeusen/FabledCurator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 33 Wiki Activity

FC-1 → FC-5: first dev→main merge for the FabledCurator v1 build #1

Merged
bvandeusen merged 0 commits from dev into main 2026-05-22 14:15:46 -04:00
Conversation 0 Commits - Files Changed -
bvandeusen commented 2026-05-22 14:15:37 -04:00 (Migrated from git.fabledsword.com)
Copy Link
Copy Source

First merge of dev into main for FabledCurator. Rolls up every stage from FC-1 (Foundation) through FC-5 (Migration tooling). CI-green on dev at 599697c.

What this PR contains

FabledCurator is the merge of ImageRepo + GallerySubscriber into a single Vue+Quart app. Five sub-projects, all landed:

FC-1 — Foundation

Repo scaffold, Quart app skeleton, Vue 3 + Vuetify + Pinia + Vite frontend, full unified schema in migration 0001 (Artist, Source, Credential, Post, ImageRecord, ImageProvenance, Tag, DownloadEvent + the rest), Celery + Redis with queue lanes, Dockerfile (web/worker/scheduler/ml roles) + Dockerfile.ml, docker-compose, Forgejo CI workflow.

FC-2 — Image backbone (a-e)

  • FC-2a Viewing core — gallery, showcase, modal viewer, settings, importer with sha+phash dedup, thumbnail generation, tag system + autocomplete.
  • FC-2b ML pipeline — WD14 tagging, SigLIP embeddings, centroids, allowlist/aliases/blocklist, suggestion engine.
  • FC-2c Discovery/browsing — tag directory, tag merge, bulk select, series management + reader.
  • FC-2d Image dedup + provenance — phash supersession, post metadata, archive extraction + PostAttachment, sidecar parsing, provenance read system, provenance UI, artist deconfliction (image_record.artist_id becomes single source of truth).
  • FC-2e Integrity verification — verify_integrity task with weekly beat schedule, integrity_status surfaced in UI.

FC-3 — Subscription backbone (a-f)

  • FC-3a Sources — CRUD service + API + UI for Source per Artist, /subscriptions.
  • FC-3b Credentials — encrypted Credential store (Fernet + file key), platforms registry, extension API key.
  • FC-3c Downloader — gallery-dl Celery task on download queue, attach_in_place Importer seam, on-demand POST /api/sources/<id>/check, /downloads timeline, rich DownloadEvent metadata, Patreon vanity → campaign-ID auto-heal.
  • FC-3d Scheduling + health — tick_due_sources beat task (60s) with exponential backoff, SourceHealthDot UI, cleanup_old_download_events daily retention, Source.consecutive_failures column.
  • FC-3e Posts stream — /posts cursor-paginated unified feed mirroring GalleryService.scroll, artist + platform filters, HTML-stripped descriptions, inline thumbs + attachment chips.
  • FC-3f Artists directory — /artists browse grid placed between Gallery and Tags, ArtistDirectoryService mirroring TagDirectoryService (cursor + window-fn previews + correlated EXISTS platform filter).

Operational

  • FC-CI-rebuild — pinned container.image: ci-python:3.14 across all CI/build jobs, shipped ci-requirements.md.

FC-5 — Migration tooling

  • MigrationRun model + migration 0015.
  • backup / rollback services (pg_dump + tar.zst).
  • gs_ingest — reads gallerysubscriber-export-v1.json, creates Artist/Source/Credential (re-encrypted with FC's key).
  • ir_ingest — reads imagerepo-export-v1.json, creates Tag rows + writes sha256-keyed manifest.
  • tag_apply — joins the manifest to ImageRecord rows after the operator's filesystem scan.
  • ml_queue / verify services.
  • /api/migrate/<kind> blueprint with multipart upload for ingest kinds.
  • LegacyMigrationCard UI slotted into Settings → Maintenance.

Coordinated with two small export scripts that landed on main of the respective repos:

  • bvandeusen/imagerepo @ 2fb1a2a: scripts/export_for_fabledcurator.py.
  • bvandeusen/GallerySubscriber @ d47b4c8: scripts/export_for_fabledcurator.py.

Post-merge operator workflow

  1. Pull fabledcurator:main (or the v26.05.22.0 release).
  2. alembic upgrade head (runs 0001 → 0015 on a fresh DB).
  3. Boot FC.
  4. Operator-side: run the IR + GS export scripts to produce ir-export.json + gs-export.json.
  5. Bind-mount IR's images dir at /import/imagerepo:ro per the docker-compose.yml example.
  6. Settings → Maintenance → Legacy migration: Backup → Ingest GS → Ingest IR → trigger filesystem scan → Apply IR tags → Queue ML → Verify.

Constraints honored throughout

Homelab HTTP. dev kept-as-is per slice. Single-line commits with the standard trailer. CI-verified before this merge.

First merge of `dev` into `main` for FabledCurator. Rolls up every stage from FC-1 (Foundation) through FC-5 (Migration tooling). CI-green on `dev` at `599697c`. ## What this PR contains FabledCurator is the merge of ImageRepo + GallerySubscriber into a single Vue+Quart app. Five sub-projects, all landed: ### FC-1 — Foundation Repo scaffold, Quart app skeleton, Vue 3 + Vuetify + Pinia + Vite frontend, full unified schema in migration `0001` (Artist, Source, Credential, Post, ImageRecord, ImageProvenance, Tag, DownloadEvent + the rest), Celery + Redis with queue lanes, Dockerfile (web/worker/scheduler/ml roles) + Dockerfile.ml, docker-compose, Forgejo CI workflow. ### FC-2 — Image backbone (a-e) - **FC-2a Viewing core** — gallery, showcase, modal viewer, settings, importer with sha+phash dedup, thumbnail generation, tag system + autocomplete. - **FC-2b ML pipeline** — WD14 tagging, SigLIP embeddings, centroids, allowlist/aliases/blocklist, suggestion engine. - **FC-2c Discovery/browsing** — tag directory, tag merge, bulk select, series management + reader. - **FC-2d Image dedup + provenance** — phash supersession, post metadata, archive extraction + PostAttachment, sidecar parsing, provenance read system, provenance UI, artist deconfliction (`image_record.artist_id` becomes single source of truth). - **FC-2e Integrity verification** — `verify_integrity` task with weekly beat schedule, integrity_status surfaced in UI. ### FC-3 — Subscription backbone (a-f) - **FC-3a Sources** — CRUD service + API + UI for `Source` per Artist, `/subscriptions`. - **FC-3b Credentials** — encrypted `Credential` store (Fernet + file key), platforms registry, extension API key. - **FC-3c Downloader** — gallery-dl Celery task on `download` queue, `attach_in_place` Importer seam, on-demand `POST /api/sources/<id>/check`, `/downloads` timeline, rich `DownloadEvent` metadata, Patreon vanity → campaign-ID auto-heal. - **FC-3d Scheduling + health** — `tick_due_sources` beat task (60s) with exponential backoff, `SourceHealthDot` UI, `cleanup_old_download_events` daily retention, `Source.consecutive_failures` column. - **FC-3e Posts stream** — `/posts` cursor-paginated unified feed mirroring `GalleryService.scroll`, artist + platform filters, HTML-stripped descriptions, inline thumbs + attachment chips. - **FC-3f Artists directory** — `/artists` browse grid placed between Gallery and Tags, `ArtistDirectoryService` mirroring `TagDirectoryService` (cursor + window-fn previews + correlated EXISTS platform filter). ### Operational - **FC-CI-rebuild** — pinned `container.image: ci-python:3.14` across all CI/build jobs, shipped `ci-requirements.md`. ### FC-5 — Migration tooling - `MigrationRun` model + migration `0015`. - `backup` / `rollback` services (pg_dump + tar.zst). - `gs_ingest` — reads `gallerysubscriber-export-v1.json`, creates Artist/Source/Credential (re-encrypted with FC's key). - `ir_ingest` — reads `imagerepo-export-v1.json`, creates Tag rows + writes sha256-keyed manifest. - `tag_apply` — joins the manifest to ImageRecord rows after the operator's filesystem scan. - `ml_queue` / `verify` services. - `/api/migrate/<kind>` blueprint with multipart upload for ingest kinds. - `LegacyMigrationCard` UI slotted into Settings → Maintenance. Coordinated with two small export scripts that landed on `main` of the respective repos: - `bvandeusen/imagerepo` @ `2fb1a2a`: `scripts/export_for_fabledcurator.py`. - `bvandeusen/GallerySubscriber` @ `d47b4c8`: `scripts/export_for_fabledcurator.py`. ## Post-merge operator workflow 1. Pull `fabledcurator:main` (or the `v26.05.22.0` release). 2. `alembic upgrade head` (runs 0001 → 0015 on a fresh DB). 3. Boot FC. 4. Operator-side: run the IR + GS export scripts to produce `ir-export.json` + `gs-export.json`. 5. Bind-mount IR's images dir at `/import/imagerepo:ro` per the `docker-compose.yml` example. 6. Settings → Maintenance → Legacy migration: Backup → Ingest GS → Ingest IR → trigger filesystem scan → Apply IR tags → Queue ML → Verify. ## Constraints honored throughout Homelab HTTP. `dev` kept-as-is per slice. Single-line commits with the standard trailer. CI-verified before this merge.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
bvandeusen
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#1
Delete Branch "dev"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?