bvandeusen
d38585ed94
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
Build images / build-ml (push) Successful in 4s
CI / extension-version (push) Successful in 3s
Build images / build-agent (push) Successful in 4s
Build images / build-web (push) Successful in 5s
CI / frontend-build (push) Successful in 22s
extension / lint (push) Successful in 30s
CI / backend-lint-and-test (push) Successful in 48s
CI / integration (push) Successful in 3m57s
extension / lint (pull_request) Successful in 28s
A doc describing a tag scheme the pipeline stopped using is worse than no doc — it is a confident wrong answer, and #3159 named it as the drift that had just bitten rule 148. build.yml (each edit ×3, once per build job): * The derived-values block claimed three values including a `tag` subcommand that no longer exists, and said "on dev the date tag is computed nowhere else". There are two values and no date tag. * The BASE-IMAGE FRESHNESS paragraph argued from a pin that no longer exists — "a date tag has to keep serving the bytes it served (fabledcurator:2026.7.17 still resolves to July's image)". Milestone 318 removed the argument's premise rather than answering it: with no version tags there is no immutable name a refresh could contradict, and rule 145 already allows a moving tag to be republished. Deleted the argument, kept a line saying why there is nothing to argue about (#3159's phrasing: the workflow becomes obviously correct instead of carefully correct). * The repoint comment said ":c-<sha> and the date pin". There is no date pin. * "its set is too narrow — the direction that serves stale bytes on a pin" now names the failure that actually exists: the reuse check hits and the channel serves a web image bundling the previous XPI (#3156). README gains a "Versions and tags" section — the three tags, why there is no fourth, and the fact that with no registry name carrying it, the instance's own report is the only answer to "which build is this?". Also fixes two stale claims: three workflows (four since release.yml) and "AMO signing runs on main only", which has been false since milestone 271 step 6 signed on both channels. ci-requirements.md was missing milestones 313 and 318 entirely: no artifacts.sh, no label-keyed reuse, no FC_VERSION. Added those, plus the #3156/#3202 membership test (can changing this file change the published bytes — not is it copied in), and corrected packaging.sh's "two consumers" to three now that artifacts.sh reads it. docker-compose.yml needed nothing — it names `:dev` directly and never offered a pin target. Note #3127 and rule 145's relation notes are updated in Scribe.
114 lines
5.7 KiB
Markdown
114 lines
5.7 KiB
Markdown
# FabledCurator
|
|
|
|
Self-hosted media curation — gallery, ML tagging, and subscription-driven downloading in one app. Part of the FabledSword family.
|
|
|
|
Combines what was [ImageRepo](https://git.fabledsword.com/bvandeusen/ImageRepo) (gallery, ML, importer) and [GallerySubscriber](https://git.fabledsword.com/bvandeusen/GallerySubscriber) (gallery-dl wrapper, subscriptions, credential capture) into a single product.
|
|
|
|
## Status
|
|
|
|
In production. `main` is continuously deployed — every merge to `main` builds
|
|
and publishes `:latest` images, so whatever is on `main` is what is running.
|
|
Day-to-day work happens on `dev`, which publishes `:dev` images.
|
|
|
|
## Versions and tags
|
|
|
|
Three image tags exist, and no others:
|
|
|
|
| Tag | Branch | Meaning |
|
|
| --- | --- | --- |
|
|
| `:latest` | `main` | Production. Moves on every merge. |
|
|
| `:c-<sha>` | `main` | Immutable — the rollback unit, all three images together. |
|
|
| `:dev` | `dev` | The rolling test channel. Moves on every push. |
|
|
|
|
There are deliberately **no version tags**. Nothing pins one, and a per-build
|
|
name nobody reads is upkeep for a model FC does not run (family rule 145; the
|
|
reasoning is note #3127 §5). Rolling back is `docker pull …:c-<sha>`.
|
|
|
|
Each artifact still has a version, derived rather than chosen: the commit time
|
|
of the newest change to that artifact's *own* shipped files, as
|
|
`YYYY.MM.DD.HHMM` UTC (rule 148). Four artifacts, four independent versions —
|
|
a push touching only `agent/` re-versions the agent and leaves web and ml
|
|
alone, and CI skips the builds whose content did not move.
|
|
|
|
Because no registry name carries it, the running instance's own report is the
|
|
only answer to "which build is this?". The foot of Settings shows
|
|
`FabledCurator 2026.08.29.0201 · dev`, and `/api/health` returns the same two
|
|
fields.
|
|
|
|
Release tags are optional bookmarks — FC went twelve weeks without one and
|
|
nothing was wrong. Pushing `v<version>` publishes a Forgejo release listing the
|
|
commits since the previous tag; it builds no image.
|
|
|
|
## What's in here
|
|
|
|
Five deployable pieces, built by `.forgejo/workflows/build.yml`:
|
|
|
|
| Piece | Built from | Image | Role |
|
|
| --- | --- | --- | --- |
|
|
| **Web / workers** | `Dockerfile` | `fabledcurator` | Quart API + the built Vue SPA in one image. `entrypoint.sh` picks the role: `web`, `worker`, `scheduler`. The `maintenance-long` service is a second `worker` pinned to the long-running maintenance queue. |
|
|
| **ML worker** | `Dockerfile.ml` | `fabledcurator-ml` | Same app, plus `requirements-ml.txt` — tagging and embedding models that run in-container. |
|
|
| **GPU agent** | `agent/Dockerfile` | `fabledcurator-agent` | Optional desktop-GPU worker (`agent/`). Leases jobs over **HTTP only** — never touches the database or Redis. Run it for a burst, stop it to reclaim the card. See `agent/README.md`. |
|
|
| **Firefox extension** | `extension/` | signed XPI | MV3 extension: pushes platform session cookies into FC and adds a creator as a Source in one click. AMO-signed on both `dev` and `main` (one signature per extension change, shared by the two channels), bundled into that channel's web image and served from Settings → Maintenance. See `extension/README.md`. |
|
|
| **Data** | — | `pgvector/pgvector:pg16`, `redis:7-alpine` | Postgres with pgvector for embeddings; Redis as the Celery broker. |
|
|
|
|
## Quick start
|
|
|
|
For local development and testing, just:
|
|
|
|
```bash
|
|
docker compose up -d
|
|
# UI: http://localhost:8080
|
|
```
|
|
|
|
That uses sane dev defaults baked into `docker-compose.yml` and the dev
|
|
override (`docker-compose.override.yml`, auto-merged) — local builds, DEBUG
|
|
logging, exposed Postgres + Redis ports on the host. No `.env` required.
|
|
|
|
For a production-like deployment, override the dev defaults via shell env
|
|
or a `.env` file (see `.env.example` for the variable names) and use:
|
|
|
|
```bash
|
|
docker compose -f docker-compose.yml up -d
|
|
# (skips the override so containers pull registry images)
|
|
```
|
|
|
|
The GPU agent is deployed separately, on the machine with the card —
|
|
`agent/docker-compose.yml`, not this stack.
|
|
|
|
## Deployment posture
|
|
|
|
FabledCurator is designed to run inside a self-hosted homelab environment over plain HTTP. If you want TLS, terminate it at your reverse proxy. The app does not generate certificates, redirect to HTTPS, or set HSTS.
|
|
|
|
## CI / Forgejo setup
|
|
|
|
Four workflows: `ci.yml` (lint, extension-version check, backend unit tests,
|
|
frontend build, integration), `extension.yml` (extension lint, vitest, XPI
|
|
content verification), `build.yml` (sign + publish), and `release.yml`, which
|
|
runs only on a `v*` tag and publishes a changelog without building anything.
|
|
|
|
**The toolchain each job runs in is its `container.image`, not its `runs-on`
|
|
label.** `runs-on: python-ci` only schedules the job onto a runner; every job
|
|
then names the image it actually wants. `ci-requirements.md` is the current,
|
|
authoritative list of images and per-job installs — read that rather than a
|
|
copy here, so the two can't drift.
|
|
|
|
The repo expects one secret:
|
|
|
|
- **`RELEASE_TOKEN`** — a Forgejo PAT with:
|
|
- `write:package` + `read:package` — for `docker push` to `git.fabledsword.com`
|
|
- `write:release` — for the `ext-<version>` releases that cache the signed XPI
|
|
- `write:issue` — for issue-management automation
|
|
|
|
Generate at https://git.fabledsword.com/user/settings/applications. The injected `GITHUB_TOKEN` cannot be used because it lacks `write:package`.
|
|
|
|
AMO signing additionally needs `MOZILLA_AMO_JWT_KEY` / `MOZILLA_AMO_JWT_SECRET`.
|
|
It runs on **both** channels and is cached per version: because the version is
|
|
derived from commit time, `dev` and `main` derive the same number for the same
|
|
source, so `main` finds `dev`'s signature already cached and makes no second AMO
|
|
call. That cache is why signing must be one-shot — AMO rejects a re-signed
|
|
version.
|
|
|
|
## License
|
|
|
|
Personal project; use at your own discretion.
|