bvandeusen
609bc82acc
Before building, each job asks the registry whether this artifact's content is already published. On a hit it skips the build entirely and repoints the channel and date tags at the existing manifest with `imagetools create` — registry-side, no layer transfer, seconds. This is the step that stops a push touching only `agent/` from rebuilding web and ml, and stops a merge to main rebuilding what dev already built. The question is asked with a new `artifacts.sh identity`, not with the date tag: the date tag is day-precise and last-one-wins, so two different builds share it and it cannot answer "is this content published?". The commit sha would move on every push and never hit, which is the redundant rebuild being removed. The revision does both jobs — content-unique, and stable across pushes that did not touch the artifact. Identity is channel-qualified for web and only for web, because web is the only image that takes a build-arg: FC_CHANNEL is baked in and reported by /api/extension/manifest, so its dev and main builds of one revision are genuinely different images. ml and agent take none, which is what lets a merge reuse dev's build rather than rebuilding the agent's CUDA image to produce bytes that already exist. tests/test_artifact_identity.py reads the Dockerfiles and fails if that list drifts from the ARG declarations, in either direction — collapsing the channels ships an instance that reports the wrong one, and splitting them needlessly rebuilds every merge. Failure direction is deliberate: an inspect that errors for any reason reads as a miss and the build runs. Only a real 200 skips one. A tag-push never claims the identity. It rebuilds a revision main already published, and image configs are not bit-reproducible, so re-pushing r-<rev> would point an immutable tag at fresh bytes — rule 145's exact prohibition. It publishes only its own :v... label and otherwise reuses. Base-image freshness, decided rather than left implicit: an artifact whose source stops moving stops picking up base updates under its pinned tag. That is what a pin means, and rule 145 already says the refresh belongs on the moving tag instead. Filed as #3154 rather than folded in here, because the naive version regresses :latest on the next unrelated push. ci.yml's backend lane gains fetch-depth: 0 — the new tests derive real revisions, and on a depth-1 clone that derivation returns the tip sha or fails, so the lane would go green while asserting nothing. The three build jobs' shadow steps are renamed and re-commented: those values stopped being informational at step 3, and a step captioned "nothing reads this" beside steps that do is worse than no caption.
FabledCurator
Self-hosted media curation — gallery, ML tagging, and subscription-driven downloading in one app. Part of the FabledSword family.
Combines what was ImageRepo (gallery, ML, importer) and GallerySubscriber (gallery-dl wrapper, subscriptions, credential capture) into a single product.
Status
In production. main is continuously deployed — every merge to main builds
and publishes :latest images, so whatever is on main is what is running.
Day-to-day work happens on dev, which publishes :dev images.
What's in here
Five deployable pieces, built by .forgejo/workflows/build.yml:
| Piece | Built from | Image | Role |
|---|---|---|---|
| Web / workers | Dockerfile |
fabledcurator |
Quart API + the built Vue SPA in one image. entrypoint.sh picks the role: web, worker, scheduler. The maintenance-long service is a second worker pinned to the long-running maintenance queue. |
| ML worker | Dockerfile.ml |
fabledcurator-ml |
Same app, plus requirements-ml.txt — tagging and embedding models that run in-container. |
| GPU agent | agent/Dockerfile |
fabledcurator-agent |
Optional desktop-GPU worker (agent/). Leases jobs over HTTP only — never touches the database or Redis. Run it for a burst, stop it to reclaim the card. See agent/README.md. |
| Firefox extension | extension/ |
signed XPI | MV3 extension: pushes platform session cookies into FC and adds a creator as a Source in one click. AMO-signed on both dev and main (one signature per extension change, shared by the two channels), bundled into that channel's web image and served from Settings → Maintenance. See extension/README.md. |
| Data | — | pgvector/pgvector:pg16, redis:7-alpine |
Postgres with pgvector for embeddings; Redis as the Celery broker. |
Quick start
For local development and testing, just:
docker compose up -d
# UI: http://localhost:8080
That uses sane dev defaults baked into docker-compose.yml and the dev
override (docker-compose.override.yml, auto-merged) — local builds, DEBUG
logging, exposed Postgres + Redis ports on the host. No .env required.
For a production-like deployment, override the dev defaults via shell env
or a .env file (see .env.example for the variable names) and use:
docker compose -f docker-compose.yml up -d
# (skips the override so containers pull registry images)
The GPU agent is deployed separately, on the machine with the card —
agent/docker-compose.yml, not this stack.
Deployment posture
FabledCurator is designed to run inside a self-hosted homelab environment over plain HTTP. If you want TLS, terminate it at your reverse proxy. The app does not generate certificates, redirect to HTTPS, or set HSTS.
CI / Forgejo setup
Three workflows: ci.yml (lint, extension-version check, backend unit tests,
frontend build, integration), extension.yml (extension lint, vitest, XPI
content verification), and build.yml (sign + publish).
The toolchain each job runs in is its container.image, not its runs-on
label. runs-on: python-ci only schedules the job onto a runner; every job
then names the image it actually wants. ci-requirements.md is the current,
authoritative list of images and per-job installs — read that rather than a
copy here, so the two can't drift.
The repo expects one secret:
-
RELEASE_TOKEN— a Forgejo PAT with:write:package+read:package— fordocker pushtogit.fabledsword.comwrite:release— for theext-<version>releases that cache the signed XPIwrite:issue— for issue-management automation
Generate at https://git.fabledsword.com/user/settings/applications. The injected
GITHUB_TOKENcannot be used because it lackswrite:package.
AMO signing additionally needs MOZILLA_AMO_JWT_KEY / MOZILLA_AMO_JWT_SECRET; it runs on
main only and is cached per version, since AMO rejects a re-signed version.
License
Personal project; use at your own discretion.