docs: write the install path for someone who is not the operator (#3271)
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 9s
Build images / build-ml (push) Successful in 29s
Build images / build-web (push) Successful in 23s
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 4s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m42s

FC has no login — no User model, no session auth, nothing. That was a
deliberate call for a single-operator tool and it stays (operator, this
session), but it was nowhere in the docs, and the app stores live Patreon /
SubscribeStar / Pixiv session cookies on accounts that carry a payment
method. Anyone standing this up from the README could reasonably have put it
behind a TLS-terminating proxy and considered it handled.

So the no-auth posture is now stated three times, in the three places someone
decides where to bind the port: README has a "Before you expose it" section
above the install instructions, .env.example explains why there is no auth
variable in it, and the compose header says it before the first service.

SECURITY.md claimed the opposite. It listed "a multi-user sharing ACL —
instances can be shared" among the things worth protecting; there are no
accounts to share between. That was rule 47 applied to a codebase that does
not implement it, and it would have told a researcher FC holds a boundary it
does not. Replaced with the real posture, including that TLS without an
authenticating layer in front changes nothing.

Also corrected, all of it stale rather than wrong-at-the-time:

- EXTENSION_API_KEY was dead config. config.py read it into a field nothing
  consumed; the real key is generated into app_setting on first use and
  managed in the UI. Removed from config.py, compose and .env.example.
- .env.example pointed at docs/superpowers/specs/… — there is no docs/ dir —
  and described the extension key as "lands in FC-3", closed 2026-05-21.
- The /import mount comment described an FC-5 ImageRepo migration run from
  "Settings → Maintenance → Legacy migration", a surface with no frontend.
- README said the extension installs from Settings → Maintenance. It is on
  Subscriptions → Settings.

README is now split: running FC above the line, developing FC below it, with
requirements, first run, the extension, upgrading and troubleshooting on the
running side. First run documents the one real gap it found — a new installer
with a library on disk has no button to import it, only POST
/api/import/trigger, because the manual-scan UI was retired 2026-07-02 when
that stopped mattering for an established install. Filed as #3367.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw
This commit is contained in:
2026-09-02 00:16:23 -04:00
co-authored by Claude Opus 5
parent aa71cbbdbf
commit 8a4af589f1
5 changed files with 334 additions and 94 deletions
+203 -45
View File
@@ -1,14 +1,201 @@
# FabledCurator
Self-hosted media curation — gallery, ML tagging, and subscription-driven downloading in one app. Part of the FabledSword family.
Self-hosted media curation — a gallery, ML auto-tagging, and subscription-driven
downloading in one application. Part of the FabledSword family.
Combines what was [ImageRepo](https://git.fabledsword.com/bvandeusen/ImageRepo) (gallery, ML, importer) and [GallerySubscriber](https://git.fabledsword.com/bvandeusen/GallerySubscriber) (gallery-dl wrapper, subscriptions, credential capture) into a single product.
## What it does
## Status
You point it at creators you follow. It downloads what they post, files it,
tags it, and gives you something better than a folder full of images to look
through afterwards.
In production. `main` is continuously deployed — every merge to `main` builds
and publishes `:latest` images, so whatever is on `main` is what is running.
Day-to-day work happens on `dev`, which publishes `:dev` images.
- **Gallery and browsing.** Images, videos and multi-page works, organised by
artist, tag, post and series. A Showcase front page, a filterable gallery, a
similarity-driven Explore view, and a page-turning reader for series.
- **Subscriptions.** Follows creators on Patreon, SubscribeStar, Pixiv and
anything `gallery-dl` supports, on a schedule. Handles paywalled posts using
your own logged-in session.
- **ML tagging.** Runs image models in-container to suggest tags, group
characters, find near-duplicates and power similarity search. Suggestions are
reviewable — it proposes, you confirm, and it learns which proposals you keep
rejecting.
- **Importing.** Ingests an existing library from disk, deduplicates it by
content hash, and reads metadata sidecars.
- **Maintenance.** Backups, library audits, thumbnail and embedding backfills,
orphan cleanup — all from the UI, all as background jobs you can watch.
Everything is configured from the Settings UI and stored in the database. There
is no config file to edit beyond a handful of bootstrap environment variables.
## Before you expose it
**FabledCurator has no login.** There are no user accounts, no passwords and no
permission model. Anything that can reach the port is an administrator.
That matters more here than it would in most self-hosted apps, because of what
this one stores: **live platform session cookies for Patreon, SubscribeStar and
Pixiv** — accounts that usually have a payment method attached. Whoever reaches
the port can read them, alongside your entire library.
So:
- Bind it to a LAN, a VPN, or a tunnel you control.
- Do not port-forward it. Do not put it on a public hostname.
- A reverse proxy that adds TLS but no authentication **does not help**. If you
want it reachable from outside, put an authenticating proxy in front of it —
a forward-auth provider, HTTP basic auth, an identity-aware tunnel — and treat
that layer as the only thing standing between the internet and your accounts.
This is a deliberate design decision for a single-operator tool on a trusted
network, not a bug and not an oversight. It is stated here because it decides
how you are allowed to deploy it. [SECURITY.md](SECURITY.md) covers the rest of
the threat model.
## Requirements
- **Docker** with Compose v2.
- **~4 GB RAM** for the app, plus whatever Postgres needs for your library size.
- **Disk** for your media, plus several GB for ML model weights.
- **No GPU required.** The ML worker runs on CPU — tagging and embedding are
slower, and that is the whole difference. A GPU is only involved if you
separately run the optional agent (below), which is a different machine's job.
## Install
```bash
git clone https://git.fabledsword.com/bvandeusen/FabledCurator.git
cd FabledCurator
cp .env.example .env
$EDITOR .env # set DB_PASSWORD and SECRET_KEY
docker compose -f docker-compose.yml up -d
```
Then open <http://localhost:8080>.
**The `-f docker-compose.yml` is required, not decoration.** Compose
auto-merges `docker-compose.override.yml` when you leave it off, and that
override builds the images locally from source — the contributor path, not
yours. Naming the file explicitly skips the override and pulls the published
`:latest` images, which is the stable channel built from `main`.
If you forget it, the symptom is a long build instead of a quick pull.
## First run
The database schema is created automatically on first start — the web container
runs its migrations before serving. Nothing to initialise by hand.
A few things are worth knowing about the first few minutes:
- **The ML worker downloads its model weights on first boot**, several GB from
HuggingFace into `./models`. Until that finishes, tagging is queued rather
than broken. It is idempotent — a restart resumes rather than refetches.
- **The gallery starts empty**, and that is the expected state. Add a creator
under **Subscriptions** and it fills as posts come down.
- **If you already have a library on disk**, note that importing it is
currently an API call rather than a button — the manual-scan UI was retired
in July 2026, when imports started arriving entirely via downloads and the
extension. Mount the folder at `./import` and kick it off with:
```bash
curl -X POST http://localhost:8080/api/import/trigger
```
Progress shows up under **Settings → Activity**. This is a known rough
edge, not the intended shape (#3367).
- **To download from a paywalled account**, FabledCurator needs that account's
session — see the browser extension below. Without one it can still fetch
public posts.
- **Check Settings → Overview** to confirm the workers are alive. Every long
operation in FabledCurator is a background job, so if the queues are not
running, the UI will look like it is ignoring you rather than like it is
broken.
## The browser extension
A Firefox extension does two jobs: it hands your logged-in platform sessions to
FabledCurator so it can download on your behalf, and it adds a creator as a
subscription in one click from their page.
It ships **inside the web image** — there is no add-on store listing to find.
Go to **Subscriptions → Settings**, find the *Browser extension* card, and click
**Install Firefox extension**. The XPI is Mozilla-signed, so Firefox installs it
like any other add-on; the button serves it directly rather than making you
download and side-load a file.
It pairs with your instance using an API key generated automatically on first
use. The bar directly under that card shows the key and can rotate it.
See [extension/README.md](extension/README.md) for what it does in detail.
## The GPU agent
Optional, and separate. If you have a desktop with a graphics card, you can run
an agent on it that leases ML jobs from FabledCurator over HTTP, does them on
the GPU, and hands the results back. It never touches the database or Redis, so
it is safe to run somewhere the rest of the stack is not.
Run it for a burst of tagging, stop it to get your card back. It deploys from
`agent/docker-compose.yml`, not the main stack — see
[agent/README.md](agent/README.md).
## Upgrading
```bash
docker compose -f docker-compose.yml pull
docker compose -f docker-compose.yml up -d
```
Migrations run automatically on start. Take a database backup first — Settings →
Maintenance has one — because the schema moves forward and does not move back.
## Deployment posture
FabledCurator is built to run inside a homelab over plain HTTP. It does not
generate certificates, redirect to HTTPS, or set HSTS. If you want TLS,
terminate it at your reverse proxy. See [Before you expose it](#before-you-expose-it)
for why TLS alone is not enough.
## Troubleshooting
**The UI loads but nothing ever finishes.** The web container is up and the
workers are not. `docker compose -f docker-compose.yml ps` — check `worker`,
`scheduler` and `ml-worker` are healthy, not restarting.
**`docker compose up` started building instead of pulling.** You left off
`-f docker-compose.yml`, so the dev override took over. See [Install](#install).
**Downloads fail with an auth error.** The stored session for that platform has
expired. Re-capture it with the extension; sessions do not last forever.
**Which build am I running?** The foot of Settings shows a version and a
channel, and `/api/health` returns the same two fields. There are no version
tags on the images, so this is the authoritative answer.
---
# Developing FabledCurator
Everything below is about working on FabledCurator rather than running it. If
you are installing it, you are done — see [CONTRIBUTING.md](CONTRIBUTING.md) if
you want to send a patch.
## Status and channels
In production. `main` is continuously deployed — every merge builds and
publishes `:latest`, so whatever is on `main` is what is running. Day-to-day
work happens on `dev`, which publishes `:dev`.
For local development, the dev override handles everything:
```bash
docker compose up -d # note: no -f, so the override applies
```
That builds the images from source, turns on DEBUG logging, and exposes
Postgres and Redis on the host. No `.env` required.
## Versions and tags
@@ -47,48 +234,10 @@ Five deployable pieces, built by `.forgejo/workflows/build.yml`:
| --- | --- | --- | --- |
| **Web / workers** | `Dockerfile` | `fabledcurator` | Quart API + the built Vue SPA in one image. `entrypoint.sh` picks the role: `web`, `worker`, `scheduler`. The `maintenance-long` service is a second `worker` pinned to the long-running maintenance queue. |
| **ML worker** | `Dockerfile.ml` | `fabledcurator-ml` | Same app, plus `requirements-ml.txt` — tagging and embedding models that run in-container. |
| **GPU agent** | `agent/Dockerfile` | `fabledcurator-agent` | Optional desktop-GPU worker (`agent/`). Leases jobs over **HTTP only** — never touches the database or Redis. Run it for a burst, stop it to reclaim the card. See `agent/README.md`. |
| **GPU agent** | `agent/Dockerfile` | `fabledcurator-agent` | Optional desktop-GPU worker (`agent/`). Leases jobs over **HTTP only** — never touches the database or Redis. See `agent/README.md`. |
| **Firefox extension** | `extension/` | signed XPI | MV3 extension: pushes platform session cookies into FC and adds a creator as a Source in one click. AMO-signed on both `dev` and `main` (one signature per extension change, shared by the two channels), bundled into that channel's web image and served from Settings → Maintenance. See `extension/README.md`. |
| **Data** | — | `pgvector/pgvector:pg16`, `redis:7-alpine` | Postgres with pgvector for embeddings; Redis as the Celery broker. |
## Quick start
For local development and testing, just:
```bash
docker compose up -d
# UI: http://localhost:8080
```
That uses sane dev defaults baked into `docker-compose.yml` and the dev
override (`docker-compose.override.yml`, auto-merged) — local builds, DEBUG
logging, exposed Postgres + Redis ports on the host. No `.env` required.
For a production-like deployment, override the dev defaults via shell env
or a `.env` file (see `.env.example` for the variable names) and use:
```bash
docker compose -f docker-compose.yml up -d
# (skips the dev override, so containers pull published :latest images)
```
`-f` is doing real work there: it tells Compose to use *only* that file, which
skips `docker-compose.override.yml` and its local builds. What you get is the
`:latest` images — the stable channel, built from `main`. This is the install
path, and it is the one to use if you are running FabledCurator rather than
working on it.
`:dev` is the other channel: rebuilt from the `dev` branch several times a day,
bleeding edge, no stability promise. Nothing in this repo points an installer at
it, and nothing should.
The GPU agent is deployed separately, on the machine with the card —
`agent/docker-compose.yml`, not this stack.
## Deployment posture
FabledCurator is designed to run inside a self-hosted homelab environment over plain HTTP. If you want TLS, terminate it at your reverse proxy. The app does not generate certificates, redirect to HTTPS, or set HSTS.
## CI / Forgejo setup
Four workflows: `ci.yml` (lint, extension-version check, backend unit tests,
@@ -118,6 +267,15 @@ source, so `main` finds `dev`'s signature already cached and makes no second AMO
call. That cache is why signing must be one-shot — AMO rejects a re-signed
version.
## History
FabledCurator combines what was
[ImageRepo](https://git.fabledsword.com/bvandeusen/ImageRepo) (gallery, ML,
importer) and
[GallerySubscriber](https://git.fabledsword.com/bvandeusen/GallerySubscriber)
(gallery-dl wrapper, subscriptions, credential capture) into a single product.
Both are superseded; neither is maintained.
## License
**GNU Affero General Public License v3.0** — see [LICENSE](LICENSE).