fix: an idle GPU agent could not check in, so the roster called it stopped

Operator, 2026-09-23: *"I'm running the gpu agent on my device and it
currently reads as 'offline' but it's running and has checked in recently."*

It had checked in — twelve minutes ago. Two cadences that never agreed:

    idle lease poll ceiling   900s   agent/fc_agent/worker.py (sleep mode)
    heartbeat while idle      never  gated on holding leases
    roster "stopped" after    300s   api/system_health.py

The roster records an agent check-in on `lease` and `heartbeat`. The heartbeat
loop was gated on `if ids:`, so an agent holding no leases sent nothing at
all — leaving the lease poll as the only check-in, and sleep mode backs that
off exponentially to a 900s ceiling. 900 against 300: an IDLE agent was
structurally guaranteed to read as stopped. Nothing was broken; nothing was
misconfigured; the two halves simply disagreed.

Not a recent regression. Sleep mode landed 2026-07-02; the roster adopted the
lease as its check-in on 2026-09-02 — *"A lease IS the check-in … Recorded on
the call that was already happening"* — without noticing that the call it was
piggybacking on had been deliberately slowed ten weeks earlier.

The heartbeat now sends whether or not it holds leases. An empty one extends
nothing (`id.in_([])` matches no rows) and costs one small POST every 45s —
against the 6/min lease poll sleep mode exists to avoid, that is not a cadence
worth protecting, and it is what makes "is the agent alive" answerable at all.

Still gated on `self._running`: a worker that has been stopped is not checking
in for work, and reporting it as present would be a different lie.

Two things I could NOT determine from the code, both needing the live table:
whether a stale `agent:agent` row exists from an older build that omitted
`agent_id` (the server defaults it), and whether changing `AGENT_ID` has ever
stranded an abandoned row — nothing prunes `service_seen`, so either would sit
there reading "stopped" forever.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR

.forgejo/workflows/baseline.yml

@@ -79,7 +79,7 @@ jobs:
       - name: Resolve the Postgres service and install deps
         run: |
           set -eux
-          # Same service-IP dance as ci.yml's integration job; see the long
+          # Same service-IP dance as build.yml's integration job; see the long
           # comment there for why the job name must stay separator-free.
           PG=$(docker ps --filter "name=compare" --filter "ancestor=pgvector/pgvector:pg16" -q | head -n1)
           test -n "$PG"
@@ -89,7 +89,7 @@ jobs:
           echo "DB_HOST=$PG_IP" >> "$GITHUB_ENV"
           # Socket probe in python, not bash's /dev/tcp — these steps run under
           # `sh -e`, where that path does not exist. Same fix and same reasoning
-          # as ci.yml's integration job; see the comment there.
+          # as build.yml's integration job; see the comment there.
           pg_ready=""
           for i in $(seq 1 60); do
             if python -c "import socket,sys; s=socket.socket(); s.settimeout(2); sys.exit(0 if s.connect_ex(('$PG_IP', 5432)) == 0 else 1)"; then

.forgejo/workflows/ci.yml

@@ -1,294 +0,0 @@
-name: CI
-
-# CI lanes per FabledRulebook/forgejo.md "CI philosophy":
-#   - lint: ruff only, no dep install — fast-fail for the common lint bounce.
-#   - extension-version: the derived version resolves and is a shape AMO takes.
-#   - backend-lint-and-test: `pytest -m "not integration"`, no service containers.
-#   - frontend-build: vitest unit + vite build.
-#   - integration: pgvector + redis service containers; alembic + `pytest -m integration`.
-
-on:
-  push:
-    branches: [dev, main]
-  # Renovate opens PRs from `renovate/*` branches into `dev`. Those branches
-  # never push to dev/main, so the push trigger above gives them NO pre-merge
-  # CI — a bump could only be validated after it was already merged. This
-  # pull_request trigger (base `dev` only) validates Renovate PRs before merge.
-  # It deliberately does NOT fire on dev→main PRs (base `main`), which still
-  # rely on the dev push run — so no duplicate runs. FC has no fork PRs
-  # (single-operator Forgejo repo), so secrets-on-PR is not a concern.
-  pull_request:
-    branches: [dev]
-
-jobs:
-  # Fast-fail lint lane. ruff is pre-installed in the ci-python image, so
-  # this runs with NO dependency install and surfaces the most common bounce
-  # class (lint: I001 / UP037 / ASYNC109 / W293 …) in seconds — instead of
-  # after the backend job's ~30-60s wheel install. ruff is static analysis,
-  # so no DB/secret env is needed.
-  lint:
-    runs-on: python-ci
-    container:
-      image: git.fabledsword.com/bvandeusen/ci-python:3.14
-    steps:
-      - uses: actions/checkout@v4
-      - name: Ruff lint
-        # agent/ included so the GPU-agent is linted before its image is built
-        # (build.yml only `docker build`s it — this is where it gets checked).
-        # scripts/ likewise: release_notes.py runs only on a tag push, so a
-        # syntax or import error there would otherwise surface at the one
-        # moment nobody wants to debug a workflow.
-        run: ruff check backend/ tests/ alembic/ agent/ scripts/
-      - name: Agent syntax check
-        # The agent's runtime deps (torch/transformers/ultralytics) aren't in the
-        # CI image, so we can't import it — but compileall parses every module,
-        # catching syntax errors before the image build.
-        run: python -m compileall -q agent/fc_agent
-
-  # The extension version is DERIVED, not hand-maintained (milestone 271 step
-  # 4): build.yml computes it from the commit TIME of the newest packaged
-  # extension change and stamps it into manifest.json / package.json at build
-  # time. The guard that used to live here — "packaged files changed but nobody
-  # bumped the version" — was therefore checking a fact that had stopped
-  # existing. Worse than useless: it would have failed this lane on every real
-  # extension change, demanding a bump that decides nothing. Retired 2026-08-27
-  # rather than left running beside the new mechanism (rule 22).
-  #
-  # Two things are still worth asserting, and this is the only lane that can:
-  # the extension.yml suite runs on node:24-slim, which is exactly why
-  # version.spec.js sticks to packaging.sh's git-free subcommands.
-  #   1. the derivation actually resolves on this commit
-  #   2. the derived string is one AMO will accept, checked against Mozilla's
-  #      own published grammar rather than a loose "digits and dots"
-  #
-  # The MAJOR.MINOR-agreement check that used to be (2) is gone with milestone
-  # 318 step 8: the committed version no longer seeds anything, so there is no
-  # hand-set part left for the two files to disagree about.
-  #
-  # Deliberately NOT checked here: that the derived value beats what has already
-  # been signed. That guard belongs in build.yml, where it compares against the
-  # real ext-* releases. Comparing against origin/main here would be wrong —
-  # dev legitimately derives a LOWER value whenever main is ahead on the
-  # extension, and a lane that fails for being behind is a lane people learn to
-  # ignore.
-  extension-version:
-    runs-on: python-ci
-    container:
-      image: git.fabledsword.com/bvandeusen/ci-python:3.14
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          # The derivation needs real history: a depth-1 clone sees one commit
-          # and produces a wrong, too-low value RATHER THAN FAILING. Checking
-          # that here is half the point of the lane.
-          fetch-depth: 0
-      - name: Extension version derives cleanly
-        run: |
-          set -eu
-          # busybox sh on the act_runner — no bashisms (family rule).
-          VERSION=$(sh extension/scripts/packaging.sh version)
-          echo "derived: $VERSION"
-
-          # Mozilla's published grammar for AMO, transcribed verbatim from
-          # MDN's manifest.json/version page:
-          #
-          #     ^(0|[1-9][0-9]{0,8})([.](0|[1-9][0-9]{0,8})){0,3}$
-          #
-          # Not the looser `^[0-9]+(\.[0-9]+)*$` this lane used to carry. That
-          # one passes `2026.08.29.0201`, which AMO REJECTS — a segment must be
-          # the single digit 0 or start 1-9 — and it also passes five segments,
-          # where AMO allows four. Both would surface as a failed sign with the
-          # version already burned: AMO 409s on re-signing, so a rejected value
-          # cannot be reclaimed and cannot be reused. This lane is the cheap
-          # place to find out. (#3138, milestone 318 step 8.)
-          if ! echo "$VERSION" | grep -qE '^(0|[1-9][0-9]{0,8})(\.(0|[1-9][0-9]{0,8})){0,3}$'; then
-            echo "ERROR: derived version '$VERSION' is not a version AMO accepts."
-            echo "AMO's grammar: ^(0|[1-9][0-9]{0,8})([.](0|[1-9][0-9]{0,8})){0,3}$"
-            echo "Most likely cause: a zero-padded segment (08, 0201). The rest"
-            echo "of the family pads; the extension must not — see packaging.sh."
-            exit 1
-          fi
-
-          # ...and the shape this project actually derives. AMO would happily
-          # take `1.0.3500147` too, so the grammar check alone would not notice
-          # a regression to the pre-318 shape — which orders BELOW everything
-          # signed since, and is unrecoverable once Firefox has the higher one.
-          if ! echo "$VERSION" | grep -qE '^20[0-9][0-9]\.[0-9]{1,2}\.[0-9]{1,2}\.[0-9]{1,4}$'; then
-            echo "ERROR: derived version '$VERSION' is not YYYY.M.D.HHMM."
-            echo "Rule 148's CalVer is what build.yml signs; the old"
-            echo "1.0.<minutes> shape would order below every ext-2026.* release."
-            exit 1
-          fi
-          echo "OK: derived version $VERSION"
-
-  backend-lint-and-test:
-    runs-on: python-ci
-    container:
-      image: git.fabledsword.com/bvandeusen/ci-python:3.14
-    env:
-      # DB_PASSWORD and SECRET_KEY are required by config.py at import time
-      # even though unit tests don't actually touch the DB or use the secret.
-      DB_PASSWORD: ci_unit_test_placeholder
-      SECRET_KEY: ci_unit_test_placeholder
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          # Full history for tests/test_artifact_identity.py, which derives
-          # each artifact's revision to check the identity scheme. On a
-          # depth-1 clone that derivation either fails or returns the tip sha
-          # — so the lane would go green while asserting nothing, which is
-          # the one outcome worse than a red one.
-          fetch-depth: 0
-
-      # Cache step removed 2026-05-26: act_runner's cache backend has been
-      # broken on this homelab runner since 2026-05-15 (first as request-
-      # timeout warnings, then as hard "Cannot find module .../dist/restore/
-      # index.js" failures that tank the whole job). The cache step targeted
-      # ~/.cache/pip but the install below uses `uv pip install` primarily,
-      # whose own cache lives at ~/.cache/uv — so the cache step's real
-      # benefit was marginal even when working. Cost of removal: ~30s of
-      # wheel downloads per job. Future re-enable: mount ~/.cache/uv as a
-      # docker volume at the runner level (skips actions/cache entirely),
-      # or fix the runner-side cache backend (clear /var/run/act/actions/*,
-      # pin act_runner version, etc.).
-
-      - name: Install Python deps
-        # ruff is pre-installed in the ci-python image (see CI-Runner/CI-python/
-        # Dockerfile's RUFF_VERSION). Per FabledRulebook ci-runners.md, toolchain
-        # versions live on the runner image, not here.
-        # uv: 5-10x faster wheel resolve than pip for cold caches.
-        # Falls back to pip install on uv-missing runners (older images).
-        run: |
-          if command -v uv >/dev/null 2>&1; then
-            uv pip install --system -r requirements.txt pytest pytest-asyncio
-          else
-            pip install -r requirements.txt pytest pytest-asyncio
-          fi
-
-      # Ruff moved to the dedicated fast `lint` job above (fails in seconds,
-      # no dep install). This job is now unit tests only.
-      - name: Pytest (unit only — integration runs in the integration job)
-        run: pytest tests/ -v -m "not integration"
-
-  frontend-build:
-    runs-on: python-ci
-    container:
-      image: git.fabledsword.com/bvandeusen/ci-python:3.14
-    defaults:
-      run:
-        working-directory: frontend
-    steps:
-      - uses: actions/checkout@v4
-      # No package-lock.json is tracked yet (we don't run npm locally per
-      # feedback-no-local-runs). Using `npm install` instead of `npm ci`.
-      # If we want strict lockfile-based reproducibility later, commit a
-      # package-lock.json and flip this back to `npm ci`.
-      - run: npm install --no-audit --no-fund
-      # No type-check step: the frontend is pure JS (no .ts files, no JSDoc),
-      # so a type-checker has nothing to do. The vue-tsc devDep + its `check`
-      # script were dropped 2026-07-11 rather than bumped to v3. If we add
-      # TS/JSDoc later, re-add a tsconfig.json + vue-tsc + a type-check step.
-      - run: npm run test:unit
-      - run: npm run build
-
-  # Single integration job — collapsed from a 3-way shard split on 2026-06-04.
-  # The shards existed to parallelize ~8.5min of integration tests; once the
-  # throwaway Postgres runs with fsync OFF (the durability step below) the whole
-  # suite runs in ~45s, so the split only triplicated the ~2min fixed overhead
-  # (container + `uv pip install` + `alembic upgrade head`) and burned 3 of 6
-  # runner slots for no wall-clock gain. One job now: spin up once, install
-  # once, migrate once, run every integration test.
-  #
-  # The docker-ps filter scopes to THIS job's own Postgres/Redis service
-  # containers by job name. act_runner strips underscores from job names when
-  # labelling containers (`int_api` matched nothing on 2026-05-25), so the name
-  # stays separator-free (`integration`). The step prints `docker ps -a` first
-  # so a future naming-convention shift surfaces in the log without a
-  # guess-and-push cycle.
-  #
-  # Pre-baking requirements.txt into ci-python:3.14 is intentionally NOT done —
-  # per ci-requirements.md, FC is the only Python consumer of that image and the
-  # CI-Runner "add deps to image when used by >1 project" rule keeps it per-job.
-  integration:
-    runs-on: python-ci
-    container:
-      image: git.fabledsword.com/bvandeusen/ci-python:3.14
-    env:
-      DB_USER: fabledcurator
-      DB_PASSWORD: ci_integration
-      DB_PORT: "5432"
-      DB_NAME: fabledcurator_test
-      SECRET_KEY: ci_integration_placeholder
-    services:
-      postgres:
-        image: pgvector/pgvector:pg16
-        env:
-          POSTGRES_USER: fabledcurator
-          POSTGRES_PASSWORD: ci_integration
-          POSTGRES_DB: fabledcurator_test
-        options: >-
-          --health-cmd "pg_isready -U fabledcurator"
-          --health-interval 10s
-          --health-timeout 5s
-          --health-retries 10
-      redis:
-        image: redis:7-alpine
-        options: >-
-          --health-cmd "redis-cli ping"
-          --health-interval 10s
-          --health-timeout 5s
-          --health-retries 10
-    steps:
-      - uses: actions/checkout@v4
-      - name: Integration suite (resolve service IPs, migrate, test)
-        run: |
-          set -eux
-          echo "=== container landscape (diagnostic for filter scoping) ==="
-          docker ps -a --format '{{.ID}} {{.Image}} -> {{.Names}}'
-          echo "=== end landscape ==="
-          PG=$(docker ps --filter "name=integration" --filter "ancestor=pgvector/pgvector:pg16" -q | head -n1)
-          RD=$(docker ps --filter "name=integration" --filter "ancestor=redis:7-alpine" -q | head -n1)
-          test -n "$PG" && test -n "$RD"
-          PG_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$PG")
-          RD_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$RD")
-          test -n "$PG_IP" && test -n "$RD_IP"
-          export DB_HOST="$PG_IP"
-          export CELERY_BROKER_URL="redis://$RD_IP:6379/0"
-          export CELERY_RESULT_BACKEND="redis://$RD_IP:6379/0"
-          # These steps run under `sh -e`, not bash, so bash's /dev/tcp magic
-          # path does not exist here — the probe this loop used to run could
-          # never succeed and simply burned the full 120s on every run, green
-          # or red, then continued without having established anything. Python
-          # is in the image and needs no installed package for a socket
-          # connect, so it is the probe. Exhausting the budget is now a named
-          # failure rather than a silent fall-through (rule 156): if Postgres
-          # is genuinely not up, that is what the log should say, instead of
-          # whatever the first query happens to raise two minutes later.
-          pg_ready=""
-          for i in $(seq 1 60); do
-            if python -c "import socket,sys; s=socket.socket(); s.settimeout(2); sys.exit(0 if s.connect_ex(('$PG_IP', 5432)) == 0 else 1)"; then
-              pg_ready=1
-              break
-            fi
-            sleep 2
-          done
-          if [ -z "$pg_ready" ]; then
-            echo "postgres at $PG_IP:5432 did not accept a connection within 120s"
-            exit 1
-          fi
-          if command -v uv >/dev/null 2>&1; then
-            uv pip install --system -r requirements.txt pytest pytest-asyncio
-          else
-            pip install -r requirements.txt pytest pytest-asyncio
-          fi
-          # Relax durability on the throwaway CI Postgres so the per-test
-          # TRUNCATE's commit-fsync — the integration teardown's dominant cost
-          # (~1.5-2s/test, which collapsed the suite from ~13min to ~45s) — is
-          # skipped. fsync/full_page_writes are sighup GUCs and synchronous_commit
-          # is user-context, so ALTER SYSTEM + pg_reload_conf() applies them with
-          # NO restart. Ephemeral DB ⇒ fsync-off is safe. Non-fatal so a perms
-          # surprise can't red the job; fabledcurator is the postgres image's
-          # bootstrap superuser.
-          python -c "import os,psycopg; c=psycopg.connect(host=os.environ['DB_HOST'],port=5432,user=os.environ['DB_USER'],password=os.environ['DB_PASSWORD'],dbname=os.environ['DB_NAME'],autocommit=True); [c.execute(q) for q in ('ALTER SYSTEM SET fsync=off','ALTER SYSTEM SET synchronous_commit=off','ALTER SYSTEM SET full_page_writes=off','SELECT pg_reload_conf()')]; c.close()" || echo 'WARN: durability GUC relax failed (continuing)'
-          alembic upgrade head
-          pytest tests/ -v -m integration --durations=15

.forgejo/workflows/extension.yml

@@ -1,5 +1,6 @@
 name: extension
-# Lint + unit tests. The sign-and-publish dance moved into build.yml's
+# Lint + unit tests. Deliberately NOT a publishing lane, which is why it is
+# not part of build.yml's gate. The sign-and-publish dance moved into build.yml's
 # `sign-extension` job (2026-05-25) — `:latest` now always bundles the XPI
 # because sign-extension runs as a build-web dependency in the SAME workflow,
 # eliminating the prior race between build.yml and a separate extension.yml.
@@ -10,19 +11,17 @@ on:
     paths:
       - 'extension/**'
       - '.forgejo/workflows/extension.yml'
-      # test/version.spec.js asserts things ABOUT the other two workflows —
-      # that neither inlines the packaged-file set, and that build.yml derives
+      # test/version.spec.js asserts things ABOUT build.yml — that it does not
+      # inline the packaged-file set, and that build.yml derives
       # the shipped version rather than reading it out of the repo. A
       # workflow-only edit can therefore break this suite, so it has to trigger
       # it. build.yml joined the list at milestone 271 step 5, when the spec
       # started asserting against it.
-      - '.forgejo/workflows/ci.yml'
       - '.forgejo/workflows/build.yml'
   pull_request:
     branches: [main]
     paths:
       - 'extension/**'
-      - '.forgejo/workflows/ci.yml'
       - '.forgejo/workflows/build.yml'
   workflow_dispatch:
 

Dockerfile

@@ -28,17 +28,75 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
         postgresql-client \
         zstd \
         megatools \
+    # PID 1 for every role. See the ENTRYPOINT note at the foot of this file:
+    # without it the image needs `init: true` in whatever runs it, which is a
+    # deployment remembering a flag for the image to behave correctly.
+        tini \
         libjpeg62-turbo \
         libwebp7 \
         libpng16-16 \
         ca-certificates \
+    # opencv-python-headless (via requirements-ml.txt) links these even in its
+    # headless build. Came from Dockerfile.ml when the images merged
+    # (milestone 422 step 6).
+        libgl1 \
+        libglib2.0-0 \
     && rm -rf /var/lib/apt/lists/*
 
 WORKDIR /app
 
-COPY requirements.txt ./
+COPY requirements.txt requirements-ml.txt ./
 RUN pip install -r requirements.txt
 
+# --- ML, merged from Dockerfile.ml (milestone 422 step 6) --------------------
+#
+# ONE image now serves every lane. It was two because the ML lane ran in its
+# own container; with the single-container layout (step 5) running every lane
+# in one process tree, a second image would mean the `ml` lane could never be
+# enabled from the UI — there would be no worker in this container to enable.
+#
+# THE COST, MEASURED from run 7273 rather than guessed — and it is far
+# smaller than the estimate this comment first carried, which said "everyone
+# pulls ~4GB":
+#
+#   torch 2.12.1+cpu wheel     192.3 MB
+#   torchvision 0.27.1+cpu       1.8 MB
+#   transformers / onnxruntime / opencv / sklearn and friends
+#                               62.0, 35.3, 23.6, 16.7, 12.3, 9.2, 6.9 MB
+#   largest newly-pushed layer 222.07 MB
+#
+# So the ML code adds a few hundred MB to the pull, not gigabytes. The CPU
+# index is what makes that true: the default PyPI torch wheel bundles the
+# NVIDIA CUDA runtime and is ~2GB on its own.
+#
+# The GIGABYTES are in the MODEL — ~3.5GB of SigLIP weights — and those are
+# NOT in this image. They arrive only when the operator enables the lane,
+# which is what lets rule 164 permit a runtime fetch at all ("optional and
+# clearly off"). That also settles the trade this step was asked to weigh:
+# baking the weights in would add ~3.5GB to every pull for a feature many
+# adopters never enable, against ~350MB for the code that makes the switch
+# available. Off-by-default wins by an order of magnitude, which was NOT
+# obvious before measuring — the estimate had the two costs within 15% of
+# each other.
+#
+# `--index-url`, not `--extra-index-url`: the latter would let pip resolve a
+# +cu wheel anyway, and the whole saving above depends on it not doing that.
+#
+# CPU-only torch from the PyTorch CPU index. Nothing here uses a GPU — the
+# GPU agent is a separate service with its own image.
+RUN pip install --index-url https://download.pytorch.org/whl/cpu \
+        "torch>=2.12,<3.0" "torchvision>=0.27,<0.28"
+RUN pip install -r requirements-ml.txt
+
+# Where the model lands. Deliberately NOT a VOLUME instruction: that mints an
+# anonymous volume when nobody mounts one, which survives `docker rm` and
+# accumulates 3.5GB copies nobody can find. The compose files mount it
+# explicitly instead, so an unmounted run simply re-downloads — visible, and
+# recoverable.
+ENV HF_HOME=/models/.huggingface \
+    TRANSFORMERS_CACHE=/models/.huggingface \
+    ML_MODEL_DIR=/models
+
 COPY backend/ ./backend/
 COPY alembic/ ./alembic/
 COPY alembic.ini ./
@@ -72,5 +130,51 @@ ENV FC_VERSION=${FC_VERSION}
 
 EXPOSE 8080
 
-ENTRYPOINT ["./entrypoint.sh"]
-CMD ["web"]
+# ONE healthcheck for every role, because the image knows which role it is
+# running and a deployment should not have to repeat it. `healthcheck` reads
+# the role entrypoint.sh recorded and asks the right question: HTTP for web,
+# a self-addressed celery ping for a worker lane, both-for-every-lane for the
+# consolidated `all`.
+#
+# start-period covers the SLOWEST role, which is `all`: alembic, then
+# hypercorn, then four celery workers registering with the broker. A web-only
+# container is ready long before this; the cost of the shared number is that
+# a broken one takes a little longer to be called broken.
+#
+# A service may still declare its own healthcheck and docker will prefer it —
+# the escape hatch for a deployment that wants something different.
+HEALTHCHECK --interval=30s --timeout=15s --start-period=90s --retries=3 \
+  CMD ["python", "-m", "backend.app.scripts.healthcheck"]
+
+# tini is PID 1, and the image brings its own rather than asking the
+# deployment for one.
+#
+# PID 1 carries a duty no other process has: every orphaned process in the
+# container reparents to it and must be reaped, or it stays a zombie holding
+# a PID slot. This app makes orphans in normal operation — six service
+# modules shell out (gallery-dl, ffmpeg, pg_dump, the external fetchers) and
+# celery's prefork pool forks children that spawn them.
+#
+# Whatever the role, something that is not an init ends up as PID 1:
+# supervisord for `all`, hypercorn for `web`, celery for a worker. The fix
+# was `init: true` in the compose/stack file, which is out of the norm and
+# put correct process handling in the hands of whoever deploys the image —
+# the same mistake as declaring the healthcheck per service. A flag that is
+# silently dropped (an older Swarm, a `docker run` without it) costs reaping
+# with no signal at all.
+#
+# So the image owns it. `docker run <image>` is correct on its own, and
+# nothing downstream has to know. The smoke asserts /proc/1/comm is tini.
+ENTRYPOINT ["/usr/bin/tini", "--", "./entrypoint.sh"]
+# The DEFAULT is the whole application, not one lane of it.
+#
+# `docker run fabledcurator` with no command starts hypercorn plus every
+# worker lane under supervisord — the shape an adopter wants and the shape the
+# consolidated stack runs. It was `web`, which meant the single-container
+# layout only worked if you knew to ask for it by name, and a compose file
+# that forgot `command:` got a web server with nothing processing its queues:
+# a gallery that loads, accepts an import, and never finishes one.
+#
+# The multi-service stack is unaffected — every service there names its role
+# explicitly, which is exactly what makes it the multi-service stack.
+CMD ["all"]

Dockerfile.ml

@@ -1,43 +0,0 @@
-# syntax=docker/dockerfile:1.25
-
-FROM python:3.14-slim
-ENV PYTHONUNBUFFERED=1 \
-    PYTHONDONTWRITEBYTECODE=1 \
-    PIP_NO_CACHE_DIR=1 \
-    PIP_DISABLE_PIP_VERSION_CHECK=1 \
-    HF_HOME=/models/.huggingface \
-    TRANSFORMERS_CACHE=/models/.huggingface \
-    ML_MODEL_DIR=/models
-
-RUN apt-get update && apt-get install -y --no-install-recommends \
-        ffmpeg \
-        libpq5 \
-        libjpeg62-turbo \
-        libwebp7 \
-        libpng16-16 \
-        libgl1 \
-        libglib2.0-0 \
-        ca-certificates \
-    && rm -rf /var/lib/apt/lists/*
-
-WORKDIR /app
-
-COPY requirements-ml.txt requirements.txt ./
-# CPU-only torch: the default PyPI wheel bundles the CUDA runtime (~5.6GB
-# layer); this pipeline never uses a GPU. --index-url (not --extra-index-url)
-# guarantees only +cpu wheels are considered, so no nvidia-*-cu12 deps.
-RUN pip install --index-url https://download.pytorch.org/whl/cpu \
-        "torch>=2.12,<3.0" "torchvision>=0.27,<0.28"
-RUN pip install -r requirements-ml.txt
-
-COPY backend/ ./backend/
-COPY alembic/ ./alembic/
-COPY alembic.ini ./
-COPY entrypoint.sh ./
-RUN chmod +x entrypoint.sh
-
-# Models self-heal into /models on first start (FC-2 implements this)
-VOLUME ["/models"]
-
-ENTRYPOINT ["./entrypoint.sh"]
-CMD ["ml-worker"]

README.md

@@ -245,9 +245,9 @@ reasoning is note #3127 §5). Rolling back is `docker pull …:c-<sha>`.
 
 Each artifact still has a version, derived rather than chosen: the commit time
 of the newest change to that artifact's *own* shipped files, as
-`YYYY.MM.DD.HHMM` UTC (rule 148). Four artifacts, four independent versions —
-a push touching only `agent/` re-versions the agent and leaves web and ml
-alone, and CI skips the builds whose content did not move.
+`YYYY.MM.DD.HHMM` UTC (rule 148). Three artifacts, three independent versions
+— a push touching only `agent/` re-versions the agent and leaves web and the
+extension alone, and CI skips the builds whose content did not move.
 
 Because no registry name carries it, the running instance's own report is the
 only answer to "which build is this?". The foot of Settings shows
@@ -260,22 +260,31 @@ commits since the previous tag; it builds no image.
 
 ## What's in here
 
-Five deployable pieces, built by `.forgejo/workflows/build.yml`:
+Four deployable pieces, built by `.forgejo/workflows/build.yml`:
 
 | Piece | Built from | Image | Role |
 | --- | --- | --- | --- |
-| **Web / workers** | `Dockerfile` | `fabledcurator` | Quart API + the built Vue SPA in one image. `entrypoint.sh` picks the role: `web`, `worker`, `scheduler`. The `maintenance-long` service is a second `worker` pinned to the long-running maintenance queue. |
-| **ML worker** | `Dockerfile.ml` | `fabledcurator-ml` | Same app, plus `requirements-ml.txt` — tagging and embedding models that run in-container. |
+| **Web / workers** | `Dockerfile` | `fabledcurator` | Quart API + the built Vue SPA in one image. `entrypoint.sh` picks the role: `web`, `worker`, `scheduler`, `ml-worker`, or `all` (every lane under supervisord, the single-container layout). The `maintenance-long` service is a second `worker` pinned to the long-running maintenance queue. |
 | **GPU agent** | `agent/Dockerfile` | `fabledcurator-agent` | Optional desktop-GPU worker (`agent/`). Leases jobs over **HTTP only** — never touches the database or Redis. See `agent/README.md`. |
 | **Firefox extension** | `extension/` | signed XPI | MV3 extension: pushes platform session cookies into FC and adds a creator as a Source in one click. AMO-signed on both `dev` and `main` (one signature per extension change, shared by the two channels), bundled into that channel's web image and served from Settings → Maintenance. See `extension/README.md`. |
 | **Data** | — | `pgvector/pgvector:pg16`, `redis:7-alpine` | Postgres with pgvector for embeddings; Redis as the Celery broker. |
 
 ## CI / Forgejo setup
 
-Four workflows: `ci.yml` (lint, extension-version check, backend unit tests,
-frontend build, integration), `extension.yml` (extension lint, vitest, XPI
-content verification), `build.yml` (sign + publish), and `release.yml`, which
-runs only on a `v*` tag and publishes a changelog without building anything.
+Three workflows: `build.yml` (the five verification lanes — lint,
+extension-version check, backend unit tests, frontend build, integration — and
+then sign + publish), `extension.yml` (extension lint, vitest, XPI content
+verification), and `release.yml`, which runs only on a `v*` tag and publishes a
+changelog without building anything.
+
+**The lanes and the publish are one workflow on purpose.** They were two
+(`ci.yml` and `build.yml`) until 2026-09-23, on the same push trigger, which
+meant the build could not see the tests' verdict and published whatever it
+built — a red unit lane and a fresh `:dev` image, in the same minute. A
+`needs:` edge only exists inside one workflow graph, so the two are one graph
+and the gate is that edge: a lane that fails, **or that merely skips**, leaves
+the publishing jobs unrun. Pull-request runs (Renovate bumps into `dev`) are
+the lanes and nothing else.
 
 **The toolchain each job runs in is its `container.image`, not its `runs-on`
 label.** `runs-on: python-ci` only schedules the job onto a runner; every job

agent/README.md

@@ -21,7 +21,7 @@ docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi
 ## 1. Get a token
 In FC: **Settings → Tagging → GPU agent → Generate token** (or Rotate). Copy it.
 
-## 2. Pull (CI publishes it alongside the web/ml images)
+## 2. Pull (CI publishes it alongside the web image)
 ```sh
 docker pull git.fabledsword.com/bvandeusen/fabledcurator-agent:latest
 ```

agent/fc_agent/worker.py

@@ -342,15 +342,44 @@ class Worker:
 
     # --- background loops ---------------------------------------------------
     def _heartbeat_loop(self) -> None:
-        """Keep every held lease alive so buffered jobs waiting on the GPU aren't
-        reclaimed by curator's 180s TTL. Errors are swallowed by client.heartbeat;
-        a reclaimed lease just re-leases elsewhere — never fatal."""
+        """Keep every held lease alive, and say we are here even when holding none.
+
+        Leases: buffered jobs waiting on the GPU would otherwise be reclaimed by
+        curator's 180s TTL. Errors are swallowed by client.heartbeat; a reclaimed
+        lease just re-leases elsewhere — never fatal.
+
+        ## Why this sends with an EMPTY list rather than skipping
+
+        Curator's roster records a check-in on this call (and on `lease`), and
+        calls an agent stopped after 300s of silence. This loop used to be
+        gated on `if ids:` — so an agent holding no leases sent nothing at all,
+        and the only check-in left was the lease poll, which sleep mode backs
+        off exponentially to a 900s ceiling (see IDLE_POLL_MAX_SECONDS).
+
+        900 against 300: an IDLE agent was structurally guaranteed to read as
+        stopped. Operator, 2026-09-23: *"I'm running the gpu agent on my device
+        and it currently reads as 'offline' but it's running and has checked in
+        recently."* It had — twelve minutes ago, partway up the backoff ladder.
+
+        The two halves were written ten weeks apart and never reconciled: sleep
+        mode landed 2026-07-02, and the roster adopted the lease as its
+        check-in on 2026-09-02 without noticing the call it was piggybacking on
+        had been deliberately slowed.
+
+        An empty heartbeat extends nothing (`id.in_([])` matches no rows) and
+        costs one small POST every 45s — against the 6/min lease poll sleep
+        mode exists to avoid, that is not a cadence worth protecting, and it is
+        what makes "is the agent alive" answerable at all.
+
+        Still gated on `self._running`: a worker that has been stopped is not
+        checking in for work, and reporting it as present would be a different
+        lie.
+        """
         while True:
             if self._running:
                 with self._held_lock:
                     ids = list(self._held)
-                if ids:
-                    self.client.heartbeat(ids)
+                self.client.heartbeat(ids)
             time.sleep(HEARTBEAT_INTERVAL)
 
     def _queue_poll_loop(self):

alembic/versions/0098_phash_256_bit_rehash.py

@@ -0,0 +1,90 @@
+"""Widen image_record.phash to 256-bit and re-hash the library (issue #4223).
+
+The operator reported a 15-image variant pack landing as 3 records, and then
+that variants were STILL being dropped with `phash_threshold` at 0. Zero was
+already the floor of the dial, so no setting could have fixed it: at
+`hash_size=8` a pHash is 64 bits of coarse light/dark layout, and variant
+artwork sharing a composition produces the SAME 64 bits. Distance 0 meant
+"identical hash", never "identical image".
+
+`utils/phash.py` moves to `hash_size=16` (256 bits, what ImageRepo always
+used) and adds an aspect-ratio gate plus a pixel-level confirm, so a merge is
+accepted on the files rather than on the hash.
+
+## Why this NULLs every phash
+
+Widening the column does not correct the values already in it. Every stored
+hash is a 64-bit hash of an image the app will now hash at 256 bits, and the
+two cannot be compared — `find_similar` skips a mismatched-length candidate
+rather than guessing, so leaving them would silently mean "no dedup, forever,
+for everything imported before today". NULL is the state `backfill_phash`
+already knows how to repair: it is NULL-only, keyset-paginated and
+restart-safe, and the beat schedule runs it daily.
+
+Until that backfill finishes, image dedup degrades to sha256 only —
+duplicates may be kept. That is the safe direction, and the only one
+available: the alternative is comparing hashes of different widths, which
+would drop artwork. NOTHING here deletes or supersedes a file.
+
+## Why the threshold is reset rather than carried over
+
+`phash_threshold` counts bits, and the denominator went from 64 to 256. The
+stored number would keep its value while meaning something four times
+tighter. There is no honest carry-over, so every row goes to the new default
+of 24 — including the operator's 0, which was a workaround for the bug this
+revision fixes.
+
+Revision ID: 0098
+Revises: 0097
+Create Date: 2026-09-21
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0098"
+down_revision: Union[str, None] = "0097"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # varchar(32) -> varchar(64): widening a length limit is a catalog-only
+    # change in Postgres, so this does not rewrite the table or its index.
+    op.alter_column(
+        "image_record", "phash",
+        existing_type=sa.String(32),
+        type_=sa.String(64),
+        existing_nullable=True,
+    )
+    op.execute("UPDATE image_record SET phash = NULL WHERE phash IS NOT NULL")
+    op.alter_column(
+        "import_settings", "phash_threshold",
+        existing_type=sa.Integer(),
+        server_default="24",
+        existing_nullable=False,
+    )
+    op.execute("UPDATE import_settings SET phash_threshold = 24")
+
+
+def downgrade() -> None:
+    # The 64-bit hashes this replaced are gone, and a 64-char value does not
+    # fit back into varchar(32) — so the column is cleared again on the way
+    # down and left for backfill_phash to refill at whatever HASH_SIZE the
+    # code is running. Rule #22: no legacy to preserve.
+    op.execute("UPDATE image_record SET phash = NULL WHERE phash IS NOT NULL")
+    op.alter_column(
+        "image_record", "phash",
+        existing_type=sa.String(64),
+        type_=sa.String(32),
+        existing_nullable=True,
+    )
+    op.alter_column(
+        "import_settings", "phash_threshold",
+        existing_type=sa.Integer(),
+        server_default="10",
+        existing_nullable=False,
+    )
+    op.execute("UPDATE import_settings SET phash_threshold = 10")

alembic/versions/0099_library_placement_run.py

@@ -0,0 +1,99 @@
+"""library_placement_run — the placement reconciler's plan/apply/undo ledger.
+
+Milestone #421 step 3. The survey (#4245) measured 33,789 ImageRecord rows
+sitting outside their artist's canonical directory, across 56 artists. This
+table holds one run of the sweep that trues them up: the plan, what it did,
+and where every file came from.
+
+## Why the moves live in a table rather than a log line
+
+`ImageRecord.path` is the only pointer at the bytes, so a move rewrites the
+row. Once that write lands, the previous location exists nowhere — unless it
+was recorded first. `moves` is that record, which is what makes a 33,789-file
+operation something the operator can undo per artist after looking at the
+result, rather than a one-way door.
+
+An `applied` row is therefore HISTORY, not state (lesson #4226). Any future
+retention on this table may prune `ready`, `cancelled` and `error` runs; an
+`applied` one is only disposable once someone decides undo is no longer
+wanted. That is deliberately not a timer's decision, and no pruning is added
+here.
+
+Revision ID: 0099
+Revises: 0098
+Create Date: 2026-09-21
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+from sqlalchemy.dialects import postgresql
+
+revision: str = "0099"
+down_revision: Union[str, None] = "0098"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "library_placement_run",
+        sa.Column("id", sa.Integer(), nullable=False),
+        sa.Column(
+            "status", sa.String(length=16), server_default="running",
+            nullable=False,
+        ),
+        # SET NULL, not CASCADE: deleting an artist must not destroy the
+        # record of where their files were moved.
+        sa.Column("artist_id", sa.Integer(), nullable=True),
+        sa.Column(
+            "started_at", sa.DateTime(timezone=True),
+            server_default=sa.text("now()"), nullable=False,
+        ),
+        sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
+        sa.Column(
+            "planned_count", sa.Integer(), server_default="0", nullable=False,
+        ),
+        sa.Column(
+            "moved_count", sa.Integer(), server_default="0", nullable=False,
+        ),
+        sa.Column(
+            "refused_count", sa.Integer(), server_default="0", nullable=False,
+        ),
+        sa.Column(
+            "moves", postgresql.JSONB(astext_type=sa.Text()),
+            server_default=sa.text("'[]'::jsonb"), nullable=False,
+        ),
+        sa.Column(
+            "refusals", postgresql.JSONB(astext_type=sa.Text()),
+            server_default=sa.text("'[]'::jsonb"), nullable=False,
+        ),
+        sa.Column("error", sa.Text(), nullable=True),
+        sa.ForeignKeyConstraint(
+            ["artist_id"], ["artist.id"],
+            name="fk_library_placement_run_artist_id", ondelete="SET NULL",
+        ),
+        sa.PrimaryKeyConstraint("id"),
+    )
+    op.create_index(
+        "ix_library_placement_run_status", "library_placement_run", ["status"],
+    )
+    op.create_index(
+        "ix_library_placement_run_artist_id", "library_placement_run",
+        ["artist_id"],
+    )
+
+
+def downgrade() -> None:
+    # Dropping this table destroys the only record of where moved files came
+    # from. That is correct for a downgrade — the code that reads it is going
+    # away too — but it is worth saying out loud rather than discovering.
+    op.drop_index(
+        "ix_library_placement_run_artist_id",
+        table_name="library_placement_run",
+    )
+    op.drop_index(
+        "ix_library_placement_run_status", table_name="library_placement_run",
+    )
+    op.drop_table("library_placement_run")

alembic/versions/0100_drop_library_placement_run.py

@@ -0,0 +1,109 @@
+"""Drop library_placement_run — the placement reconciler is removed.
+
+Milestone #421 built a sweep that compared each image's `artist_id` to the
+name of the directory its file sat in, and called every mismatch a misplaced
+file. On the operator's library that reported 33,789 of 63,605 images as
+wrongly filed.
+
+That number was an artefact of the comparison, not a fact about the library:
+
+- **97.1%** of it was one artist's own folder spelled differently —
+  `Telepurte/` versus `telepurte/`. Same artist, same art, nothing wrong.
+- Of the 1% that sat in a differently-named folder, querying `ImageProvenance`
+  — which records the post and source each file was actually downloaded from —
+  showed 87 where provenance agreed with the FOLDER and not the record, and 40
+  genuinely posted by several creators. The sweep would have misfiled or
+  arbitrarily picked for roughly 41% of that set.
+
+The system already knows where every file came from. The reconciler inferred
+it from a column and a directory name instead, and manufactured work out of a
+naming convention. Operator's call, 2026-09-21: *"the current system
+consistently records where items are and where they came from this is just
+complicating something works and doesn't need fixing."*
+
+Rule #22 — no legacy to preserve. The table goes with the code.
+
+## What is deliberately kept
+
+`utils.paths.canonical_subdir` stays: new filesystem imports derive their
+directory from the artist's slug, matching what the downloader has always
+done. It is not part of this tool and removing it would be churn for no fix.
+Run 1's 327 moved files (`InsoUwu/` -> `insouwu/`) also stay where they are —
+same artist either way, and the gallery renders them correctly.
+
+Revision ID: 0100
+Revises: 0099
+Create Date: 2026-09-21
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+from sqlalchemy.dialects import postgresql
+
+revision: str = "0100"
+down_revision: Union[str, None] = "0099"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.drop_index(
+        "ix_library_placement_run_artist_id",
+        table_name="library_placement_run",
+    )
+    op.drop_index(
+        "ix_library_placement_run_status", table_name="library_placement_run",
+    )
+    op.drop_table("library_placement_run")
+
+
+def downgrade() -> None:
+    # Recreates the table only. The three runs it held (one applied, two
+    # planned-and-never-run) are not restored and are not worth restoring —
+    # the code that reads them is gone.
+    op.create_table(
+        "library_placement_run",
+        sa.Column("id", sa.Integer(), nullable=False),
+        sa.Column(
+            "status", sa.String(length=16), server_default="running",
+            nullable=False,
+        ),
+        sa.Column("artist_id", sa.Integer(), nullable=True),
+        sa.Column(
+            "started_at", sa.DateTime(timezone=True),
+            server_default=sa.text("now()"), nullable=False,
+        ),
+        sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
+        sa.Column(
+            "planned_count", sa.Integer(), server_default="0", nullable=False,
+        ),
+        sa.Column(
+            "moved_count", sa.Integer(), server_default="0", nullable=False,
+        ),
+        sa.Column(
+            "refused_count", sa.Integer(), server_default="0", nullable=False,
+        ),
+        sa.Column(
+            "moves", postgresql.JSONB(astext_type=sa.Text()),
+            server_default=sa.text("'[]'::jsonb"), nullable=False,
+        ),
+        sa.Column(
+            "refusals", postgresql.JSONB(astext_type=sa.Text()),
+            server_default=sa.text("'[]'::jsonb"), nullable=False,
+        ),
+        sa.Column("error", sa.Text(), nullable=True),
+        sa.ForeignKeyConstraint(
+            ["artist_id"], ["artist.id"],
+            name="fk_library_placement_run_artist_id", ondelete="SET NULL",
+        ),
+        sa.PrimaryKeyConstraint("id"),
+    )
+    op.create_index(
+        "ix_library_placement_run_status", "library_placement_run", ["status"],
+    )
+    op.create_index(
+        "ix_library_placement_run_artist_id", "library_placement_run",
+        ["artist_id"],
+    )

alembic/versions/0101_clear_failure_state_on_disabled_sources.py

@@ -0,0 +1,66 @@
+"""Clear failure state on sources that are disabled (#4279).
+
+`failing_sources_clause()` now means "enabled AND erroring", so a disabled
+source no longer counts as failing. That fixes what the surfaces REPORT; it
+does not touch what the rows already CARRY, and the rows are the reason the
+operator saw a banner for six days with no way to act on it (lesson #4202 —
+a guard does not undo the value already stored).
+
+## The row this exists for
+
+Ebi77 (source 19): the membership sweep stopped it as `former_patron` at
+02:50 on 2026-09-15 and correctly cleared its failure state. A deep scan was
+armed twenty minutes later — `/backfill` had no `enabled` guard, which this
+release also fixes — and could not complete without access, so the recovery
+sweep stranded it:
+
+    consecutive_failures = 1
+    last_error = "stranded by recovery sweep (no terminal status after time_limit)"
+
+Nothing could clear that. A disabled source is never scheduled, so no
+successful run resets the counter; `SourceService.update` clears failure
+state only on an explicit disable, and the source was already disabled; and
+the card's Retry routes to `/check`, which refuses a disabled source.
+
+## Why every disabled source, not just that one
+
+The clear matches what `SourceService.update` already does when a source is
+disabled through the app — "disable the subs you're not paying for without
+them lingering as failing" — so this brings rows disabled by any OTHER path
+(the membership sweep, a retired platform in 0097) into line with the rows
+disabled by hand. Same shape as 0097: a repair migration reaches the live
+instance on deploy rather than waiting for someone to find the row.
+
+Enabled sources are untouched — a real failure on a live source must keep
+showing.
+
+Revision ID: 0101
+Revises: 0100
+Create Date: 2026-09-21
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+
+revision: str = "0101"
+down_revision: Union[str, None] = "0100"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.execute(
+        "UPDATE source SET last_error = NULL, error_type = NULL, "
+        "consecutive_failures = 0 "
+        "WHERE NOT enabled "
+        "AND (last_error IS NOT NULL OR error_type IS NOT NULL "
+        "     OR consecutive_failures <> 0)"
+    )
+
+
+def downgrade() -> None:
+    # Irreversible by design: the cleared strings and counts are not recorded
+    # anywhere, and restoring a failure state nobody can act on would only
+    # re-create the banner this removes. Rule #22 owes no story backwards.
+    pass

alembic/versions/0102_drop_pixiv_data_and_orphan_credentials.py

@@ -0,0 +1,138 @@
+"""Drop pixiv's ledgers, and delete credentials for platforms FC no longer has.
+
+Milestone #406 step 6 (with issue #3980 folded in). Phase 1 unregistered pixiv
+and the commit alongside this one deleted its client, downloader, ingester and
+models. This removes the data those models described, and the stored secrets of
+every platform that has been retired.
+
+## The two ledger tables
+
+`pixiv_seen_media` and `pixiv_failed_media` are the per-source seen / dead-letter
+ledgers for a downloader that no longer exists. They were created in
+`0089_baseline.py`, so dropping them needs a new revision rather than an edit
+there.
+
+## The credentials
+
+Written as *delete every credential whose platform is not registered* rather
+than as `platform = 'pixiv'`, at the explicit ask in this step's plan. That is
+what makes one migration cover two retirements:
+
+- **pixiv** — a live OAuth refresh token for a service FC no longer talks to.
+- **deviantart** — issue #3980. #3069 retired DeviantArt in code on 2026-08-27
+  and left its stored session behind; seven weeks later it was still there.
+
+And it is the only way either row can go. The credentials UI
+(`subscriptions/SettingsTab.vue`) renders one card per platform returned by
+`/api/platforms`, then looks the credential up by key — so a row whose platform
+is unregistered has no card, no Remove button, and no way for the operator to
+reach it. `CredentialService.list()` would return it; nothing asks.
+
+The registered set is written out literally instead of importing
+`known_platform_keys()`. A migration is a statement about one moment in the
+schema's history: if it imported the live registry, retiring a fifth platform
+in 2027 would silently change what this 2026 revision did on a fresh database.
+The list below is the registry as of 2026-09-21.
+
+## What is deliberately kept
+
+**Every pixiv `Source` row.** The original plan deleted them; the operator's
+call on 2026-09-21 was to keep them, and the reason is that `platform` is
+stored ONLY on `Source` — neither `Post` nor `ImageRecord` carries it. Both
+FKs are `ON DELETE SET NULL`, so a delete would not lose the art, but it would
+drop every pixiv image into the gallery's `__unsourced__` bucket and strip the
+platform chip off every pixiv post. The rows stay disabled (0097) and their
+platform is unregistered, so nothing schedules them, nothing downloads through
+them, and `POST /api/sources` will not make another. Keeping them costs
+nothing and keeps the attribution the milestone's goal — *"the art already
+downloaded from pixiv stays"* — is actually about.
+
+Every pixiv `Post` and `ImageRecord` is likewise untouched.
+
+Revision ID: 0102
+Revises: 0101
+Create Date: 2026-09-21
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0102"
+down_revision: Union[str, None] = "0101"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+# services/platforms/__init__.py's PLATFORMS as of this revision. See the
+# docstring for why this is a literal and not an import.
+_REGISTERED_PLATFORMS = ("patreon", "subscribestar", "hentaifoundry", "discord")
+
+
+def upgrade() -> None:
+    op.execute(
+        sa.text(
+            "DELETE FROM credential WHERE platform NOT IN :registered"
+        ).bindparams(
+            sa.bindparam("registered", value=_REGISTERED_PLATFORMS, expanding=True)
+        )
+    )
+    op.drop_index("ix_pixiv_failed_media_source_id", table_name="pixiv_failed_media")
+    op.drop_table("pixiv_failed_media")
+    op.drop_index("ix_pixiv_seen_media_source_id", table_name="pixiv_seen_media")
+    op.drop_table("pixiv_seen_media")
+
+
+def downgrade() -> None:
+    # The tables come back empty, and the credentials do not come back at all:
+    # they were encrypted blobs, this migration does not copy them anywhere,
+    # and restoring a live token for a platform FC cannot talk to would only
+    # re-create the liability. Rule #22 owes no story backwards.
+    op.create_table(
+        "pixiv_seen_media",
+        sa.Column("id", sa.Integer(), nullable=False),
+        sa.Column("source_id", sa.Integer(), nullable=False),
+        sa.Column("filehash", sa.String(length=128), nullable=False),
+        sa.Column("url", sa.Text(), nullable=True),
+        sa.Column(
+            "created_at", sa.DateTime(timezone=True),
+            server_default=sa.text("now()"), nullable=False,
+        ),
+        sa.ForeignKeyConstraint(
+            ["source_id"], ["source.id"],
+            name=op.f("fk_pixiv_seen_media_source_id_source"), ondelete="CASCADE",
+        ),
+        sa.PrimaryKeyConstraint("id", name=op.f("pk_pixiv_seen_media")),
+        sa.UniqueConstraint(
+            "source_id", "filehash", name="uq_pixiv_seen_media_source_id",
+        ),
+    )
+    op.create_index(
+        op.f("ix_pixiv_seen_media_source_id"), "pixiv_seen_media", ["source_id"],
+        unique=False,
+    )
+    op.create_table(
+        "pixiv_failed_media",
+        sa.Column("id", sa.Integer(), nullable=False),
+        sa.Column("source_id", sa.Integer(), nullable=False),
+        sa.Column("filehash", sa.String(length=128), nullable=False),
+        sa.Column("url", sa.Text(), nullable=True),
+        sa.Column("error", sa.Text(), nullable=True),
+        sa.Column("attempts", sa.Integer(), server_default="1", nullable=False),
+        sa.Column(
+            "created_at", sa.DateTime(timezone=True),
+            server_default=sa.text("now()"), nullable=False,
+        ),
+        sa.ForeignKeyConstraint(
+            ["source_id"], ["source.id"],
+            name=op.f("fk_pixiv_failed_media_source_id_source"), ondelete="CASCADE",
+        ),
+        sa.PrimaryKeyConstraint("id", name=op.f("pk_pixiv_failed_media")),
+        sa.UniqueConstraint(
+            "source_id", "filehash", name="uq_pixiv_failed_media_source_id",
+        ),
+    )
+    op.create_index(
+        op.f("ix_pixiv_failed_media_source_id"), "pixiv_failed_media", ["source_id"],
+        unique=False,
+    )

alembic/versions/0103_worker_lane_settings.py

@@ -0,0 +1,121 @@
+"""worker_lane — settings-backed slots for each celery lane.
+
+Milestone 422 step 1. One row per lane, holding only what an operator can
+change: how many slots it runs, the ceiling they have set for themselves, and
+whether it consumes its queues at all.
+
+## What is deliberately not a column
+
+**The queues.** They are decided by `celery_app.py`'s `task_routes`, not by
+preference, so a stored copy could contradict the routing table with nothing
+to notice until a queue had no consumer. They live in
+`services/worker_lanes.py`.
+
+**The derived ceiling.** Computed from the container's cgroup limits on every
+read. A row written on a 32GB host and later run in a 4GB container must be
+bounded by the 4GB; a stored ceiling would quietly authorise what the box can
+no longer hold.
+
+## The seeded values
+
+Written out literally rather than imported from `worker_lanes.LANES`. A
+migration is a statement about one moment in the schema's history — if it
+imported the live defaults, changing them in 2027 would silently change what
+this 2026 revision does on a fresh database. The two are allowed to diverge
+afterwards, and that is correct: `LANES` supplies defaults for a lane added
+later, this file records what was seeded today.
+
+    lane              slots  cap  enabled
+    worker                1    4  yes
+    scheduler             1    2  yes
+    maintenance_long      1    2  yes
+    ml                    0    1  NO
+
+One of each, per the operator (2026-09-22: *"that starting value should be one
+of each"*), and far below their own production numbers — worker 8 and ml 2 are
+tuned for their hardware and are not a sane first boot for a stranger.
+
+**ml ships at zero and disabled**, which is milestone 422 step 6's requirement
+arriving early: enabling the lane is what triggers the SigLIP download, and
+rule 164 permits a runtime fetch only for a feature that is "optional and
+clearly off". Seeding it on would make every fresh install reach HuggingFace.
+
+The caps start low on purpose. A cap that begins at the ceiling is a rubber
+stamp; starting at 4/2/2/1 means raising slots within the cap is ordinary and
+raising the cap is a deliberate act.
+
+## Existing installs
+
+Nothing is migrated FROM. The `CELERY_QUEUES` / `CELERY_CONCURRENCY` env vars
+stay exactly as they are and remain the baseline each lane boots at; these
+rows are the adjustment applied on top (step 3). So this migration changes no
+behaviour on a running stack — it only makes the numbers storable.
+
+Revision ID: 0103
+Revises: 0102
+Create Date: 2026-09-22
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0103"
+down_revision: Union[str, None] = "0102"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+# (name, slots, slots_cap, enabled) — see the docstring for why these are
+# literals and not an import.
+_SEED = (
+    ("worker", 1, 4, True),
+    ("scheduler", 1, 2, True),
+    ("maintenance_long", 1, 2, True),
+    ("ml", 0, 1, False),
+)
+
+
+def upgrade() -> None:
+    worker_lane = op.create_table(
+        "worker_lane",
+        sa.Column("name", sa.String(length=32), nullable=False),
+        sa.Column("slots", sa.Integer(), nullable=False),
+        sa.Column("slots_cap", sa.Integer(), nullable=False),
+        sa.Column("enabled", sa.Boolean(), nullable=False),
+        sa.Column(
+            "updated_at", sa.DateTime(timezone=True),
+            server_default=sa.text("now()"), nullable=False,
+        ),
+        sa.PrimaryKeyConstraint("name", name=op.f("pk_worker_lane")),
+        # Bare constraint names: Base.metadata's naming convention prepends
+        # ck_worker_lane_, and pre-prefixing doubles it — the defect alembic
+        # 0088 had to rename four constraints for (#3275). op.f() marks these
+        # as already-final so autogenerate does not propose renaming them.
+        sa.CheckConstraint("slots >= 0", name=op.f("ck_worker_lane_slots_non_negative")),
+        sa.CheckConstraint("slots_cap >= 0", name=op.f("ck_worker_lane_cap_non_negative")),
+        # The invariant that makes the cap mean anything, in the database
+        # rather than only in the service: a row violating it is not a
+        # rejected request, it is a lane that step 3's reconcile will drive UP
+        # to a number the operator capped.
+        sa.CheckConstraint("slots <= slots_cap", name=op.f("ck_worker_lane_slots_within_cap")),
+    )
+    # No index beyond the primary key, deliberately — four rows, forever. Same
+    # reasoning as service_seen, and the lesson of #3301, which removed seven
+    # indexes that were write cost buying nothing.
+    op.bulk_insert(
+        worker_lane,
+        [
+            {"name": name, "slots": slots, "slots_cap": cap, "enabled": enabled}
+            for name, slots, cap, enabled in _SEED
+        ],
+    )
+
+
+def downgrade() -> None:
+    # The rows go with the table. They are settings with shipped defaults, not
+    # operator data that predates this revision — a downgrade returns the stack
+    # to reading its concurrency from env, which is where it reads it from
+    # today anyway.
+    op.drop_table("worker_lane")

alembic/versions/0104_worker_lane_autoscale.py

@@ -0,0 +1,51 @@
+"""worker_lane.autoscale — may this lane grow itself?
+
+Milestone 422 step 7. One boolean, defaulting FALSE on every existing row and
+on every new one.
+
+## Why the default is false and not "sensible"
+
+This is the only part of the milestone that acts without anyone watching. The
+manual dial (step 4) and the reconcile (step 3) both do exactly what someone
+asked for; this one decides. Shipping it on would mean every install starts
+with a process that changes its own resource usage based on a heuristic tuned
+against nobody's workload.
+
+Off also makes the failure mode benign: if the signal is wrong, nothing
+happens until an operator opts a lane in, and they opted in while watching.
+
+## Why per lane and not one global switch
+
+The lanes are not alike in what a slot costs. A `worker` slot is a process;
+an `ml` slot is another copy of a ~3.5GB model. A global switch would enable
+growth on a lane whose behaviour under load nobody has observed, and the one
+it would hurt most is the one whose cost is least visible.
+
+Revision ID: 0104
+Revises: 0103
+Create Date: 2026-09-22
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0104"
+down_revision: Union[str, None] = "0103"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "worker_lane",
+        sa.Column(
+            "autoscale", sa.Boolean(),
+            server_default=sa.text("false"), nullable=False,
+        ),
+    )
+
+
+def downgrade() -> None:
+    op.drop_column("worker_lane", "autoscale")

alembic/versions/0105_worker_lane_one_cap.py

@@ -0,0 +1,133 @@
+"""worker_lane — one number: the cap. `slots`, `enabled` and `autoscale` go.
+
+Milestone 422, reshaped by the operator 2026-09-23:
+
+    "auto should be always on, not a setting, so that idle instances quiet
+    down when not running. the number that is visible and something the user
+    can tweak and manage should be the cap itself the number of running
+    workers is handled by the autoscaling function which is always on."
+
+## What each dropped column was, and why it is not needed
+
+**`slots`** — how many workers the lane should run. That is a MEASUREMENT,
+not a preference: the autoscaler moves the live pool between one and the cap
+according to the backlog, and reads it back from the worker every minute.
+Storing it made it look like something to keep in agreement with the cap,
+which is exactly what the operator had to do.
+
+**`autoscale`** — whether the lane was allowed to size itself. It gated the
+mechanism behind a per-lane opt-in, so a lane nobody enabled simply never
+gave its slots back. Always on now, which is the only way "idle instances
+quiet down" can be true of an install nobody has configured.
+
+**`enabled`** — whether the lane consumes its queues. Derived from `cap > 0`.
+It and `slots = 0` were two spellings of one fact and were free to disagree;
+this migration picks the one an operator can see.
+
+## Why the caps are rewritten rather than preserved
+
+The old defaults were 4 / 2 / 2 / 1, chosen when the number meant "the most
+you may raise SLOTS to" — a bound on a manual control, deliberately loose
+because moving within it was the ordinary act. The number now means "the most
+workers this lane may actually use", which is a different promise, and
+carrying the old figure over would silently quadruple the worker lane on
+every existing install at the moment this deploys.
+
+So every row is reset to the new defaults: **one for each required lane, zero
+for ML.** That loses whatever an operator had set — which is the honest
+trade, because what they set was an answer to a different question. The UI
+now tells a busy lane's operator to raise its cap, which is how the number
+gets back up on an install that needs it.
+
+ML at zero also keeps rule 164's carve-out intact: no consumers, so no model
+download until someone raises the cap.
+
+Revision ID: 0105
+Revises: 0104
+Create Date: 2026-09-23
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0105"
+down_revision: Union[str, None] = "0104"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+# (name, cap) — the same values `services/worker_lanes.LANES` declares. Seeded
+# here as literals rather than imported: a migration must describe the schema
+# at ITS point in history, and importing the live table would make this file
+# change meaning every time that table does.
+_CAPS = (
+    ("worker", 1),
+    ("scheduler", 1),
+    ("maintenance_long", 1),
+    ("ml", 0),
+)
+
+
+def upgrade() -> None:
+    # The constraints go first: they name `slots`, so dropping the column out
+    # from under them fails on Postgres.
+    #
+    # `op.f()` around each name, and it is load-bearing. Without it alembic
+    # runs the name through Base.metadata's naming convention, which prepends
+    # `ck_worker_lane_` to a string that already carries it — and the DROP
+    # goes looking for `ck_worker_lane_ck_worker_lane_slots_within_cap`, which
+    # no database has. That is #3275 exactly, from the other direction:
+    # alembic 0088 had to RENAME four constraints created with the same
+    # doubling. Caught here by the integration lane, run 7365.
+    op.drop_constraint(
+        op.f("ck_worker_lane_slots_within_cap"), "worker_lane", type_="check",
+    )
+    op.drop_constraint(
+        op.f("ck_worker_lane_slots_non_negative"), "worker_lane", type_="check",
+    )
+    op.drop_column("worker_lane", "slots")
+    op.drop_column("worker_lane", "enabled")
+    op.drop_column("worker_lane", "autoscale")
+
+    # Reset to the new meaning. See the docstring: the old value answered a
+    # different question, and carrying it over would raise every lane.
+    for name, cap in _CAPS:
+        op.execute(
+            sa.text("UPDATE worker_lane SET slots_cap = :cap WHERE name = :name")
+            .bindparams(cap=cap, name=name)
+        )
+
+    # A lane the old seed never wrote — or one an operator added by hand — is
+    # left alone rather than guessed at. `_rows_by_name` creates any missing
+    # row at the lane's default on first read.
+
+
+def downgrade() -> None:
+    op.add_column(
+        "worker_lane",
+        sa.Column("slots", sa.Integer(), nullable=False, server_default="1"),
+    )
+    op.add_column(
+        "worker_lane",
+        sa.Column(
+            "enabled", sa.Boolean(), nullable=False, server_default=sa.text("true"),
+        ),
+    )
+    op.add_column(
+        "worker_lane",
+        sa.Column(
+            "autoscale", sa.Boolean(), nullable=False, server_default=sa.text("false"),
+        ),
+    )
+    # Restore the pre-0105 invariants. `slots` comes back as 1 everywhere and
+    # the caps are 1/1/1/0, so a lane at cap 0 would violate `slots <= cap` —
+    # hence the clamp before the constraint is added.
+    op.execute(sa.text("UPDATE worker_lane SET slots = 0 WHERE slots_cap = 0"))
+    op.execute(sa.text("UPDATE worker_lane SET enabled = (slots_cap > 0)"))
+    op.create_check_constraint(
+        op.f("ck_worker_lane_slots_non_negative"), "worker_lane", "slots >= 0",
+    )
+    op.create_check_constraint(
+        op.f("ck_worker_lane_slots_within_cap"), "worker_lane", "slots <= slots_cap",
+    )

backend/app/api/__init__.py

@@ -41,6 +41,7 @@ def all_blueprints() -> list[Blueprint]:
     from .system_health import system_health_bp
     from .tags import tags_bp
     from .thumbnails import thumbnails_bp
+    from .workers import workers_bp
     return [
         api_bp,
         attachments_bp,
@@ -52,6 +53,7 @@ def all_blueprints() -> list[Blueprint]:
         showcase_bp,
         settings_bp,
         system_activity_bp,
+        workers_bp,
         system_health_bp,
         system_backup_bp,
         admin_bp,

backend/app/api/admin.py

@@ -475,6 +475,20 @@ async def trigger_reclaim_attachments():
     return _queued(async_result)
 
 
+@admin_bp.route("/maintenance/repair-discord-downloads", methods=["POST"])
+async def trigger_repair_discord_downloads():
+    """Clean re-download of the Discord files broken by the `None` naming
+    (#3999). Body {"dry_run": bool}; dry_run is the DEFAULT, because the apply
+    deletes files and makes gallery-dl forget every Discord download. Returns the
+    Celery task id — poll /maintenance/task-result/<id> for the summary."""
+    from ..tasks.admin import repair_discord_downloads_task
+
+    body = await request.get_json(silent=True) or {}
+    dry_run = bool(body.get("dry_run", True))
+    async_result = repair_discord_downloads_task.delay(dry_run=dry_run)
+    return _queued(async_result)
+
+
 @admin_bp.route("/maintenance/dedup-videos", methods=["POST"])
 async def trigger_dedup_videos():
     """Tier-1 video dedup (#871). Body {"dry_run": bool}: dry_run=true previews

backend/app/api/artists.py

@@ -65,6 +65,16 @@ async def autocomplete():
     ])
 
 
+@artists_bp.route("/names", methods=["GET"])
+async def names():
+    """Every artist, id + name + slug, alphabetical. For filter pickers that
+    list artists before anything is typed; `autocomplete` deliberately returns
+    nothing for an empty query."""
+    async with get_session() as session:
+        rows = await ArtistService(session).all_names()
+    return jsonify([{"id": i, "name": n, "slug": s} for i, n, s in rows])
+
+
 @artists_bp.route("/directory", methods=["GET"])
 async def directory():
     """FC-3f: cursor-paginated artists directory.

backend/app/api/extension.py

@@ -109,8 +109,8 @@ async def quick_add_source():
         if not await _ext_key_required(session):
             return _bad("unauthorized", status=401)
         try:
-            # crypto lets a pixiv add resolve the artist's display name via the
-            # stored OAuth token (else it falls back to the numeric id). #130.
+            # crypto lets an add resolve the artist's display name via the
+            # stored credential (else it falls back to the URL handle). #130.
             result = await ExtensionService(session, _get_crypto()).quick_add_source(url)
         except UnknownPlatformError as exc:
             return _bad(

backend/app/api/sources.py

@@ -201,6 +201,16 @@ async def set_backfill(source_id: int):
             rec = await SourceService(session).get(source_id)
             if rec is None:
                 return _bad("not_found", status=404)
+            # A disabled source must not be armable for a deep walk — the same
+            # rule /check has carried all along (see `source_disabled` below).
+            # Arming one anyway is how #4279 happened: the membership sweep had
+            # stopped Ebi77 as `former_patron`, a deep scan was armed twenty
+            # minutes later, the walk could not complete without access, and
+            # the recovery sweep stranded it with a failure count no surface
+            # could clear — a disabled source is never scheduled again, and
+            # Retry routes to /check, which refuses it.
+            if not rec.enabled:
+                return _bad("source_disabled", detail="enable the source first")
             native = uses_native_ingester(rec.platform)
             if native:
                 cred = CredentialService(session, _get_crypto())

backend/app/api/system_activity.py

@@ -21,18 +21,22 @@ from ..config import get_config
 from ..extensions import get_session
 from ..models import TaskRun
 from ..services.scheduler_service import scheduler_status
+from ..services.worker_lanes import LANES
 
 system_activity_bp = Blueprint(
     "system_activity", __name__, url_prefix="/api/system/activity",
 )
 
-# Canonical queue order — must match celery_app.task_routes. UI renders
-# in this order; queues with no LLEN response show as null rather than
-# absent.
-_QUEUE_NAMES = (
-    "default", "import", "thumbnail", "ml",
-    "download", "scan", "maintenance", "maintenance_long",
-)
+# Every queue, grouped by the lane that consumes it. DERIVED from
+# `worker_lanes.LANES` (milestone 422 step 1) rather than written out:
+# this was a hand-kept third copy of "which queues exist", alongside
+# celery_app.task_routes and service_roster.ROLE_NAMES, and its own comment
+# admitted the coupling — "must match celery_app.task_routes".
+#
+# The rendered ORDER changes with this: lane order rather than the previous
+# hand-chosen one. That is the better grouping for a lane-oriented UI, and
+# queues with no LLEN response still show as null rather than absent.
+_QUEUE_NAMES = tuple(q for lane in LANES for q in lane.queues)
 
 # Cache module-level so all requests share the cache between polls.
 # Tests can reset via direct dict mutation if needed.

backend/app/api/workers.py

@@ -0,0 +1,109 @@
+"""Worker lanes: what each is doing, and the dial that changes it.
+
+Milestone 422 step 2. The write half of a surface `api/system_activity.py`
+only reads.
+
+## Why this is a separate blueprint
+
+`system_activity` says in its own first line that it is read-only, and it
+answers a different question: its `/workers` is keyed on celery HOSTNAME and
+reports which nodes answered. That stays as it is — the existing
+SystemActivityTab consumes it.
+
+This is keyed on LANE, joins the stored cap to the live pool, and accepts
+writes. Two endpoints answering "which celery processes exist" and "how much
+work is each lane allowed to do" are not the same endpoint, and folding the
+second into the first would make a read-only module a write one.
+"""
+
+from __future__ import annotations
+
+from datetime import UTC, datetime
+
+from quart import Blueprint, jsonify, request
+
+from ..extensions import get_session
+from ..services.worker_control import (
+    LaneUpdateRefused,
+    lane_settings,
+    lane_view,
+    push_lane_cap,
+    store_lane_cap,
+)
+from ..services.worker_lanes import LANES_BY_NAME
+from ._responses import error_response as _bad
+
+workers_bp = Blueprint("workers", __name__, url_prefix="/api/system/workers")
+
+
+@workers_bp.route("", methods=["GET"])
+async def list_lanes():
+    """Every lane: its cap, the ceiling above it, and what is live.
+
+    Response: {lanes: [...], fetched_at: iso8601}
+
+    Deliberately NOT cached, unlike system_activity's 2s/5s caches. This is
+    the surface an operator watches while dragging a stepper, and a cached
+    reply would show them the value from before their own change and read as
+    the control having failed.
+    """
+    # The session closes BEFORE the broker work. Holding a Postgres connection
+    # across a celery inspect is what made this page block the whole site —
+    # see `worker_control.LaneSettings`. This endpoint polls every 15s and the
+    # inspect budget is 11s, so each poll was pinning a connection for most of
+    # the interval.
+    async with get_session() as session:
+        settings = await lane_settings(session)
+    lanes = await lane_view(settings)
+    return jsonify({
+        "lanes": lanes,
+        "fetched_at": datetime.now(UTC).isoformat(),
+    })
+
+
+@workers_bp.route("/<name>", methods=["POST"])
+async def update_lane(name: str):
+    """Set a lane's cap. Stores it, then makes the live lane obey it.
+
+    ONE field, since 2026-09-23. It used to take `slots`, `slots_cap`,
+    `enabled` and `autoscale`; how many workers are running is now a
+    measurement the sizing pass owns, and `enabled` is `cap > 0`.
+
+    Two failure kinds, deliberately different statuses:
+
+    * **400** — the value is not allowed (negative, or above what this
+      container can hold). Nothing was stored. The body carries `detail`,
+      which is the sentence the UI shows; a refused control with no reason
+      reads as a bug.
+    * **200 with `applied: false`** — the value WAS stored but could not be
+      pushed, because the lane is not currently answering. Not an error: the
+      sizing pass carries it within a minute, and the UI should say "saved,
+      not yet live" rather than "that didn't work".
+    """
+    lane = LANES_BY_NAME.get(name)
+    if lane is None:
+        return _bad("unknown_lane", detail=name, known=sorted(LANES_BY_NAME))
+
+    body = await request.get_json()
+    if not isinstance(body, dict):
+        return _bad("invalid_body", detail="body must be a JSON object")
+
+    if "slots_cap" not in body:
+        return _bad("invalid_body", detail="give slots_cap")
+    value = body["slots_cap"]
+    # Rejected rather than coerced: `True` is an int in Python, and silently
+    # reading it as a cap of 1 would be a control that appears to work and
+    # sets something nobody asked for.
+    if not isinstance(value, int) or isinstance(value, bool):
+        return _bad("invalid_body", detail="slots_cap must be an integer")
+
+    # Store, close the session, THEN push. Same reason as the GET above, and
+    # more sharply here: a cap change could do three broker round trips, all
+    # of them previously with a connection held.
+    async with get_session() as session:
+        try:
+            was_cap = await store_lane_cap(session, lane, value)
+        except LaneUpdateRefused as exc:
+            return _bad("refused", detail=str(exc))
+    result = await push_lane_cap(lane, value, was_cap=was_cap)
+    return jsonify(result)

backend/app/celery_app.py

@@ -111,6 +111,25 @@ def make_celery() -> Celery:
                 "task": "backend.app.tasks.maintenance.recover_interrupted_tasks",
                 "schedule": 300.0,  # every 5 minutes
             },
+            "size-worker-lanes": {
+                "task": "backend.app.tasks.maintenance.size_worker_lanes",
+                "schedule": 60.0,  # every minute.
+                #
+                # ONE entry, replacing `autoscale-worker-lanes` (60s) and
+                # `reconcile-worker-lanes` (300s) on 2026-09-23. They were two
+                # sweeps over one number and most of the autoscaler's design
+                # existed to stop the reconcile undoing its work; with the
+                # stored `slots` gone there is nothing to disagree about.
+                #
+                # A minute because it reacts to a BACKLOG, and a five-minute
+                # reaction to a queue filling up is no reaction. It also now
+                # carries what the reconcile was for — a worker restarted at
+                # its ENV concurrency is corrected on the next tick rather
+                # than after five.
+                #
+                # Cheap when settled: one inspect plus one LLEN sweep, and no
+                # control messages at all once every lane matches.
+            },
             "cleanup-old-tasks": {
                 "task": "backend.app.tasks.maintenance.cleanup_old_tasks",
                 "schedule": 86400.0,  # daily
@@ -120,6 +139,14 @@ def make_celery() -> Celery:
                 "schedule": 86400.0,  # daily — sweep .part/.partial left by a
                 # download/import killed mid-write (graceful-shutdown fallout)
             },
+            "backfill-phash-daily": {
+                "task": "backend.app.tasks.maintenance.backfill_phash",
+                "schedule": 86400.0,  # daily — NULL-only, so a no-op once the
+                # library is hashed. This is what makes migration 0098's
+                # re-hash happen on its own: 0098 NULLs every phash, and
+                # without a scheduled refill the library would sit
+                # dedup-disabled until someone ran a deep scan (#4223).
+            },
             "train-heads-nightly": {
                 "task": "backend.app.tasks.ml.scheduled_train_heads",
                 "schedule": 86400.0,  # passive cadence; manual retrain stays available

backend/app/models/__init__.py

@@ -26,8 +26,6 @@ from .membership_sync import MembershipSync
 from .ml_settings import MLSettings
 from .patreon_failed_media import PatreonFailedMedia
 from .patreon_seen_media import PatreonSeenMedia
-from .pixiv_failed_media import PixivFailedMedia
-from .pixiv_seen_media import PixivSeenMedia
 from .platform_membership import PlatformMembership
 from .post import Post
 from .post_association import PostAssociation
@@ -46,6 +44,7 @@ from .tag_head import TagHead
 from .tag_positive_confirmation import TagPositiveConfirmation
 from .tag_suggestion_rejection import TagSuggestionRejection
 from .task_run import TaskRun
+from .worker_lane import WorkerLane
 
 __all__ = [
     "Base",
@@ -58,8 +57,6 @@ __all__ = [
     "Credential",
     "PatreonFailedMedia",
     "PatreonSeenMedia",
-    "PixivFailedMedia",
-    "PixivSeenMedia",
     "SubscribeStarFailedMedia",
     "SubscribeStarSeenMedia",
     "Post",
@@ -98,4 +95,5 @@ __all__ = [
     "TagPositiveConfirmation",
     "TagSuggestionRejection",
     "TaskRun",
+    "WorkerLane",
 ]

backend/app/models/image_record.py

@@ -64,7 +64,11 @@ class ImageRecord(Base):
     # that 0001 also built was an exact duplicate of it — dropped in 0089
     # (#3301). Lookups by sha256 use the constraint's index.
     sha256: Mapped[str] = mapped_column(String(64), nullable=False)
-    phash: Mapped[str | None] = mapped_column(String(32), nullable=True, index=True)
+    # 64 hex chars = the 256-bit hash utils.phash emits at hash_size=16. Was
+    # String(32) (64-bit) until migration 0098; the narrow column was the
+    # reason for the undersized hash, and the undersized hash was collapsing
+    # variant artwork into one record (#4223).
+    phash: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
     size_bytes: Mapped[int] = mapped_column(BigInteger, nullable=False)
     mime: Mapped[str] = mapped_column(String(64), nullable=False)
     width: Mapped[int | None] = mapped_column(Integer, nullable=True)

backend/app/models/import_settings.py

@@ -42,7 +42,12 @@ class ImportSettings(Base):
     single_color_threshold: Mapped[float] = mapped_column(Float, nullable=False, default=0.95, server_default="0.95")
     single_color_tolerance: Mapped[int] = mapped_column(Integer, nullable=False, default=30, server_default="30")
 
-    phash_threshold: Mapped[int] = mapped_column(Integer, nullable=False, default=10, server_default="10")
+    # Hamming distance over a 256-bit pHash (utils.phash, hash_size=16). The
+    # unit CHANGED in migration 0098 — it used to be bits out of 64 — so the
+    # old default of 10 is not this scale's 10, and 0098 resets every row.
+    # This is now the cheap PRE-FILTER: the aspect + pixel gates decide, which
+    # is what lets it be generous enough to catch a re-encoded rescale.
+    phash_threshold: Mapped[int] = mapped_column(Integer, nullable=False, default=24, server_default="24")
 
     # FC-3c downloader knobs
     download_rate_limit_seconds: Mapped[float] = mapped_column(

backend/app/models/pixiv_failed_media.py

@@ -1,45 +0,0 @@
-"""PixivFailedMedia — per-source dead-letter ledger of Pixiv media that keeps
-failing to download/validate.
-
-Mirror of PatreonFailedMedia/SubscribeStarFailedMedia. Media that fails every
-walk (404'd pximg URL, deleted work, persistently-corrupt bytes) would
-otherwise re-error forever and re-burn backfill chunks. After ``attempts``
-reaches the dead-letter threshold the ingester skips it on routine
-tick/backfill walks (recovery still re-attempts). A later clean download
-clears the row.
-
-`filehash` is the same synthesized ``<illust_id>:p<num>`` /
-``<illust_id>:ugoira`` key the seen-ledger uses. UNIQUE (source_id, filehash)
-is the upsert key.
-"""
-
-from datetime import datetime
-
-from sqlalchemy import ForeignKey, Integer, String, Text, UniqueConstraint, func
-from sqlalchemy.orm import Mapped, mapped_column
-from sqlalchemy.types import DateTime
-
-from .base import Base
-
-
-class PixivFailedMedia(Base):
-    __tablename__ = "pixiv_failed_media"
-    __table_args__ = (
-        UniqueConstraint(
-            "source_id", "filehash", name="uq_pixiv_failed_media_source_id"
-        ),
-    )
-
-    id: Mapped[int] = mapped_column(Integer, primary_key=True)
-    source_id: Mapped[int] = mapped_column(
-        ForeignKey("source.id", ondelete="CASCADE"), nullable=False, index=True
-    )
-    filehash: Mapped[str] = mapped_column(String(128), nullable=False)
-    attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=1, server_default="1")
-    last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
-    first_failed_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    last_failed_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )

backend/app/models/pixiv_seen_media.py

@@ -1,42 +0,0 @@
-"""PixivSeenMedia — per-source ledger of Pixiv media already
-downloaded+processed.
-
-Mirror of PatreonSeenMedia/SubscribeStarSeenMedia for the Pixiv native
-ingester (replacing gallery-dl). One queryable row per (source, media) so
-routine walks skip media we've already ingested; recovery mode bypasses the
-ledger to re-walk.
-
-Pixiv original URLs carry no content hash, so `filehash` is always the
-synthesized ``<illust_id>:p<num>`` (page) / ``<illust_id>:ugoira`` (frame
-zip) key — stable across any URL-shape drift. String(128) matches the sibling
-ledgers.
-"""
-
-from datetime import datetime
-
-from sqlalchemy import ForeignKey, Integer, String, UniqueConstraint, func
-from sqlalchemy.orm import Mapped, mapped_column
-from sqlalchemy.types import DateTime
-
-from .base import Base
-
-
-class PixivSeenMedia(Base):
-    __tablename__ = "pixiv_seen_media"
-    __table_args__ = (
-        # Dedup key the downloader upserts against: one ledger row per
-        # (source, media). A second sighting of the same media is a no-op.
-        UniqueConstraint(
-            "source_id", "filehash", name="uq_pixiv_seen_media_source_id"
-        ),
-    )
-
-    id: Mapped[int] = mapped_column(Integer, primary_key=True)
-    source_id: Mapped[int] = mapped_column(
-        ForeignKey("source.id", ondelete="CASCADE"), nullable=False, index=True
-    )
-    filehash: Mapped[str] = mapped_column(String(128), nullable=False)
-    post_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
-    seen_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )

backend/app/models/worker_lane.py

@@ -0,0 +1,85 @@
+"""worker_lane — the most workers the operator will let each lane use.
+
+Milestone 422 step 1, reshaped 2026-09-23. One row per lane in
+`services/worker_lanes.LANES`, and ONE COLUMN an operator sets.
+
+## Why there is only one number now
+
+There were three — `slots`, `slots_cap` and `autoscale` — because the manual
+dial was built first and the autoscaler arrived last, beside a control that
+already existed rather than in place of it.
+
+Operator: *"auto should be always on, not a setting, so that idle instances
+quiet down when not running. the number that is visible and something the
+user can tweak and manage should be the cap itself the number of running
+workers is handled by the autoscaling function which is always on."*
+
+So `slots` is gone. How many workers a lane is running right now is a
+MEASUREMENT — read live from the worker, moved by the autoscaler, never
+stored. Storing it made it look like a preference, which meant the operator
+had to keep two numbers in agreement and the autoscaler had to be told it was
+allowed to touch one of them.
+
+`autoscale` is gone for the same reason: it gated the mechanism behind a
+choice, and a lane nobody opted in simply never gave its slots back.
+
+`enabled` is gone too, and is now DERIVED: a cap of zero means no consumers.
+"Off" and "may use no workers" were two spellings of one fact, free to
+disagree.
+
+## What remains
+
+    1  <=  live pool  <=  slots_cap  <=  derived_ceiling
+           (autoscaler)   (this row)     (computed)
+
+The floor is one PROCESS, not zero: billiard will not run an empty pool, and
+the parked process is what `add_consumer` lands on when the cap goes back up.
+
+The DERIVED CEILING is deliberately absent from this table. It is computed
+from the container's cgroup limits on every read, so a row written on a 32GB
+host and later run in a 4GB container is bounded by the 4GB — a stored
+ceiling would quietly authorise what the box can no longer hold.
+"""
+
+from datetime import datetime
+
+from sqlalchemy import (
+    CheckConstraint,
+    DateTime,
+    Integer,
+    String,
+    func,
+)
+from sqlalchemy.orm import Mapped, mapped_column
+
+from .base import Base
+
+
+class WorkerLane(Base):
+    __tablename__ = "worker_lane"
+    __table_args__ = (
+        # Bare name — Base.metadata's naming convention prepends
+        # ck_worker_lane_. Pre-prefixing here doubles it, which is what
+        # alembic 0088 had to rename four constraints for (#3275).
+        CheckConstraint("slots_cap >= 0", name="cap_non_negative"),
+    )
+
+    # The lane name from services/worker_lanes.LANES — never a container
+    # hostname. See models/service_seen.py for why: celery's worker names here
+    # are `celery@<container id>`, minted fresh on every deploy.
+    name: Mapped[str] = mapped_column(String(32), primary_key=True)
+
+    # The most workers this lane may use. Zero means off — no consumers, so
+    # the lane takes no work and (for ML) downloads no model.
+    #
+    # There is no upper CHECK here, because the bound it would need is the
+    # derived ceiling, and no column holds that: it depends on the cgroup the
+    # container is running in right now. Enforced at write instead.
+    slots_cap: Mapped[int] = mapped_column(Integer, nullable=False)
+
+    updated_at: Mapped[datetime] = mapped_column(
+        DateTime(timezone=True),
+        nullable=False,
+        server_default=func.now(),
+        onupdate=func.now(),
+    )

backend/app/scripts/gen_supervisord.py

@@ -0,0 +1,227 @@
+"""Emit a supervisord config for the single-container layout.
+
+Milestone 422 step 5. Writes to stdout; `entrypoint.sh all` redirects it to a
+file and execs supervisord against it.
+
+## Why this is generated and not a checked-in .conf
+
+A static config would spell out each lane's `-Q` list, and that would be a
+FIFTH hand-kept copy of the queue names — after `celery_app.task_routes`, and
+the three collapsed in steps 1, 2 and 4 (`service_roster.ROLE_NAMES`,
+`system_activity._QUEUE_NAMES`, and the Activity filter). Every one of those
+had already drifted by the time it was found.
+
+Generating from `worker_lanes.LANES` makes a stronger guarantee than "they
+match today": the processes this container runs and the lanes the application
+believes in are the same list, so a lane added to `LANES` gets a process
+without anyone remembering to add one, and a queue can never end up with no
+consumer because a config file was missed.
+
+## Why supervisord
+
+It is one pip dependency on an image that is already Python, and it does the
+four things this needs without being clever: restart a program that exits,
+give each one its OWN stop timeout, signal the process GROUP rather than the
+leader, and put every program's output on one stdout.
+
+The process-group part is not a detail. Celery's prefork pool forks children,
+and a TERM delivered only to the parent leaves them running — which is how a
+"graceful" shutdown turns into orphaned workers holding tasks. `stopasgroup`
+and `killasgroup` are both set for every program.
+
+s6-overlay is the other standard answer and would work; it needs a build-time
+download and a second mental model, and its advantage (correct PID-1 signal
+and zombie handling) is available here from `init: true` in compose, which
+puts tini in front of supervisord. Neither choice reaches the application —
+nothing in FC talks to the supervisor — so this is reversible without touching
+a line of product code.
+
+## Every lane, including ml
+
+Step 6 merged the images, so this one carries torch and the ML requirements
+and the `ml` lane gets a program like any other. It starts at one slot with
+its consumers CANCELLED — `enabled=false` in the seeded settings — so it
+holds a process and no model. That matters: `add_consumer` needs a running
+worker to reach, and without one the UI switch would have nothing to switch.
+
+Nothing is downloaded by starting it. The model fetch is enqueued when the
+lane is enabled, which is what lets rule 164 permit a runtime fetch at all.
+"""
+
+from __future__ import annotations
+
+import argparse
+import shlex
+import sys
+
+from ..services.worker_lanes import LANES, MIN_POOL_SLOTS, Lane
+
+# One number for the whole container, and it must cover the SLOWEST lane —
+# docker gives the container a single stop timeout, where compose today gives
+# each service its own (90/60/180/120s). `maintenance_long` is the 180s one:
+# DB backups, library audits and translation backfill. Anything less turns a
+# routine restart into a SIGKILL mid-backup.
+#
+# Per-program values below are the old per-service ones, preserved: supervisord
+# waits `stopwaitsecs` for each, and they stop in parallel, so the container's
+# own timeout needs to cover the max rather than the sum.
+STOP_WAIT_SECONDS: dict[str, int] = {
+    "worker": 90,
+    "scheduler": 60,
+    "maintenance_long": 180,
+    "ml": 120,
+}
+DEFAULT_STOP_WAIT = 60
+
+def _program(lane: Lane, *, slots: int) -> str:
+    """One [program:x] block.
+
+    `stdout_logfile=/dev/fd/1` with maxbytes 0 puts the lane's output straight
+    on the container's stdout unbuffered, so `docker logs` shows every lane
+    interleaved rather than supervisord swallowing them into rotated files.
+
+    The output is prefixed through `sed` so a line can be attributed to a lane
+    — four celery workers and hypercorn on one stream are otherwise
+    indistinguishable. The shell that the pipe requires is exactly why
+    `stopasgroup` matters: the signal has to reach the celery process, not the
+    `sh` holding the pipeline.
+    """
+    inner = f"./entrypoint.sh {lane.entrypoint_role}"
+    prefixed = f"{inner} 2>&1 | sed -u 's/^/[{lane.name}] /'"
+    stop_wait = STOP_WAIT_SECONDS.get(lane.name, DEFAULT_STOP_WAIT)
+    return "\n".join([
+        f"[program:{lane.name}]",
+        f"command=sh -c {shlex.quote(prefixed)}",
+        # QUOTED, and that is load-bearing. supervisord parses `environment`
+        # as a COMMA-separated KEY=VALUE list, so an unquoted queue list reads
+        # as CELERY_QUEUES=default followed by three malformed entries — and
+        # the lane would consume only its first queue. Silent: the worker
+        # starts, reports healthy, and simply never picks up `import`.
+        f'environment=CELERY_QUEUES="{",".join(lane.queues)}",'
+        f"CELERY_CONCURRENCY={slots},"
+        # A UNIQUE celery node name per lane, and the reason is not cosmetic.
+        # These processes share one hostname, so celery's default
+        # `celery@<hostname>` made all four the SAME node: inspect collapsed
+        # their replies, three lanes read as absent, and which three varied
+        # per call (run 7319). The healthcheck could never pass, and
+        # pool_grow's `destination` would have addressed an arbitrary lane.
+        f"CELERY_NODENAME={lane.name}",
+        "autostart=true",
+        "autorestart=true",
+        # A lane that dies instantly and repeatedly is a broken image, not a
+        # transient fault. Backing off stops it burning a core in a restart
+        # loop while still recovering from a one-off crash.
+        "startretries=3",
+        "startsecs=5",
+        f"stopwaitsecs={stop_wait}",
+        "stopasgroup=true",
+        "killasgroup=true",
+        "stdout_logfile=/dev/fd/1",
+        "stdout_logfile_maxbytes=0",
+        "redirect_stderr=true",
+        "",
+    ])
+
+
+# supervisord's control socket. /tmp for the same reason the generated config
+# lives there — writable by every role, and per-container by nature.
+SOCKET_PATH = "/tmp/supervisor.sock"
+
+
+def _web_program() -> str:
+    """hypercorn. Started FIRST (priority) because its role runs
+    `alembic upgrade head`, and a worker that boots against an un-migrated
+    schema fails in a way that looks like application breakage."""
+    prefixed = "./entrypoint.sh web 2>&1 | sed -u 's/^/[web] /'"
+    return "\n".join([
+        "[program:web]",
+        f"command=sh -c {shlex.quote(prefixed)}",
+        "priority=1",
+        "autostart=true",
+        "autorestart=true",
+        "startretries=3",
+        "startsecs=5",
+        # Short: HTTP requests and the occasional file download. Matches the
+        # 30s the operator's production stack gives the web service.
+        "stopwaitsecs=30",
+        "stopasgroup=true",
+        "killasgroup=true",
+        "stdout_logfile=/dev/fd/1",
+        "stdout_logfile_maxbytes=0",
+        "redirect_stderr=true",
+        "",
+    ])
+
+
+def render() -> str:
+    parts = [
+        "\n".join([
+            "[supervisord]",
+            # PID 1 in the container, so it must not daemonise.
+            "nodaemon=true",
+            # supervisord's OWN log. /dev/fd/1 keeps it on the container's
+            # stdout beside the programs rather than in a file nobody reads.
+            "logfile=/dev/fd/1",
+            "logfile_maxbytes=0",
+            "loglevel=info",
+            "",
+        ]),
+        # THE CONTROL SOCKET, and it is not optional furniture.
+        #
+        # Without these three sections supervisord runs perfectly and
+        # `supervisorctl` cannot talk to it at all:
+        #
+        #     Error: .ini file does not include supervisorctl section
+        #
+        # Which is the first thing anyone reaches for when a lane misbehaves
+        # in the consolidated container — `docker exec <c> supervisorctl
+        # status` to see which processes are up, or `restart ml` to bounce one
+        # without taking the whole application down with it. Consolidation
+        # took away `docker ps` as the way to see the lanes; this is what
+        # replaces it, and shipping without it would have left an operator
+        # with one container, five processes inside it, and no way to ask
+        # about any of them.
+        #
+        # Found by the smoke's own diagnostic line on run 7322, which printed
+        # this error instead of a process list. It was behind `|| true`, so it
+        # cost nothing and said so anyway — the argument for printing evidence
+        # even where nothing depends on it.
+        #
+        # /tmp, like the generated config itself: writable by every role
+        # without assuming a volume, and per-container state that must not
+        # outlive the container.
+        "\n".join([
+            "[unix_http_server]",
+            f"file={SOCKET_PATH}",
+            "chmod=0700",
+            "",
+            "[rpcinterface:supervisor]",
+            "supervisor.rpcinterface_factory = "
+            "supervisor.rpcinterface:make_main_rpcinterface",
+            "",
+            "[supervisorctl]",
+            f"serverurl=unix://{SOCKET_PATH}",
+            "",
+        ]),
+        _web_program(),
+    ]
+    # Lanes after web, in LANES order, so the log reads in a stable sequence.
+    for lane in LANES:
+        # A lane configured at zero slots still gets a PROCESS, at one slot
+        # with its consumers cancelled by the reconcile. Without a running
+        # worker there is nothing for `add_consumer` to reach, so enabling the
+        # lane from the UI could not work at all — the process has to exist for
+        # the switch to have something to switch.
+        parts.append(_program(lane, slots=MIN_POOL_SLOTS))
+    return "\n".join(parts)
+
+
+def main(argv: list[str] | None = None) -> int:
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.parse_args(argv)
+    sys.stdout.write(render())
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

backend/app/scripts/healthcheck.py

@@ -0,0 +1,173 @@
+"""The container's healthcheck. Picks the right check from the role it runs.
+
+Exit 0 healthy, non-zero unhealthy.
+
+## Why this is in the IMAGE and not in every compose file
+
+Because the image is the only thing that knows what it is running. A
+deployment had to declare a healthcheck per service, which meant every
+compose file, stack file and README repeated the same knowledge:
+
+    web         -> curl /api/health
+    worker      -> celery inspect ping -d celery@$HOSTNAME
+    all         -> both, for every lane
+
+Three checks, written out by hand, once per service, in every file anyone
+ever wrote — and none of them wrong until a role changed. Operator, 2026-09-23:
+*"why isn't the healthcheck built into the image or base on what command runs
+if one is passed in."* There was no reason. The role is already a fact the
+container holds; asking the deployment to restate it is the same duplication
+the lane table exists to remove one level down.
+
+So `entrypoint.sh` records the role it started, the Dockerfile declares ONE
+`HEALTHCHECK` that runs this, and a stack file says nothing at all. Declaring
+one anyway still works — docker lets a service override the image's — which
+is the escape hatch for a deployment that genuinely wants something else.
+
+## What each role is asked
+
+* **web** — hypercorn answers `/api/health`. No database: the endpoint is a
+  no-DB 200 that proves the app booted and is serving after `alembic upgrade
+  head`, which is what a rolling deploy needs to know.
+* **worker / scheduler / ml-worker** — THIS container's celery node answers a
+  ping over the broker. Not "some worker answered": the node name is pinned
+  to this container, or a healthy sibling would keep a dead one looking alive.
+* **all** — both halves, for every lane in the table. The failure mode
+  consolidation creates is that docker can no longer see the lanes as
+  separate services, so a web-only check reports a healthy container with
+  every worker dead.
+* **shell / alembic / anything else** — nothing to check. These are one-shot
+  or interactive; a liveness probe on them has no meaning, so it passes
+  rather than inventing a verdict.
+
+## An unrecorded role passes rather than failing
+
+If the role file is missing, the entrypoint did not run — someone used
+`--entrypoint` or ran a bare command. That is a debugging shape, and a
+healthcheck that cannot tell what it is looking at must not assert that the
+thing is broken (snippet #3969: an unswept read is not a verdict). It says so
+on stdout and exits 0.
+"""
+
+from __future__ import annotations
+
+import os
+import socket
+import sys
+import urllib.error
+import urllib.request
+
+# Written by entrypoint.sh at boot. /tmp because it is the one path writable
+# by every role without assuming a volume, and the value is per-container
+# state that must NOT survive into a new container.
+ROLE_FILE = os.environ.get("FC_ROLE_FILE", "/tmp/fc-role")
+
+WEB_URL = "http://localhost:8080/api/health"
+WEB_TIMEOUT = 5.0
+
+# A broker round trip, so it gets a deadline (rule 156). Generous relative to
+# `inspect`'s 2s elsewhere: this runs every 30s with retries, and a transient
+# blip flagging a worker unhealthy would roll back a deployment that is fine.
+PING_TIMEOUT = 10.0
+
+CELERY_ROLES = {"worker", "scheduler", "ml-worker"}
+# Roles with nothing to probe. Listed rather than treated as the default, so
+# an unknown role takes the "I cannot tell" path and says so.
+NO_CHECK_ROLES = {"shell", "bash", "alembic"}
+
+
+def current_role() -> str | None:
+    """The role this container was started with, or None if nothing recorded."""
+    env = os.environ.get("FC_ROLE")
+    if env:
+        return env.strip()
+    try:
+        with open(ROLE_FILE) as fh:
+            return fh.read().strip() or None
+    except OSError:
+        return None
+
+
+def _web_ok() -> tuple[bool, str]:
+    try:
+        with urllib.request.urlopen(WEB_URL, timeout=WEB_TIMEOUT) as resp:
+            if resp.status == 200:
+                return True, ""
+            return False, f"web returned {resp.status}"
+    except (urllib.error.URLError, OSError) as exc:
+        return False, f"web unreachable: {exc}"
+
+
+def _this_node_ok() -> tuple[bool, str]:
+    """Ping THIS container's celery node, by name.
+
+    Pinned to this node deliberately. A bare `ping()` is answered by any
+    worker on the broker, so in a stack with several replicas a dead one
+    would go on reporting healthy for as long as a sibling was alive — the
+    healthcheck would be measuring the cluster, not the container it is in.
+    """
+    from ..celery_app import celery as celery_app
+
+    node = f"{os.environ.get('CELERY_NODENAME', 'celery')}@{socket.gethostname()}"
+    try:
+        replies = celery_app.control.ping(destination=[node], timeout=PING_TIMEOUT)
+    except Exception as exc:  # noqa: BLE001 — a probe reports, never raises
+        return False, f"could not reach the broker: {exc}"
+    if not replies:
+        return False, f"{node} did not answer a ping"
+    return True, ""
+
+
+def _lanes_ok() -> tuple[bool, str]:
+    """Every lane in the table is answering.
+
+    Deliberately ignores whether a lane is ENABLED: a disabled lane still runs
+    its process with its consumers cancelled, so it answers `inspect` and is
+    healthy. Health is "is the process alive"; whether it should be consuming
+    is a settings question the reconcile owns, and conflating them would make
+    turning a lane off in the UI mark the container unhealthy.
+    """
+    from ..services.worker_control import inspect_lanes_sync
+    from ..services.worker_lanes import LANES
+
+    live = inspect_lanes_sync()
+    missing = sorted(lane.name for lane in LANES if not live[lane.name].present)
+    if missing:
+        return False, "lanes not answering: " + ", ".join(missing)
+    return True, ""
+
+
+def main(argv: list[str] | None = None) -> int:
+    role = current_role()
+
+    if role is None:
+        # Not a failure. See the module docstring: the entrypoint did not run,
+        # so there is no role to check against and no basis for a verdict.
+        print("no role recorded; nothing to check")
+        return 0
+
+    if role in NO_CHECK_ROLES:
+        print(f"{role}: nothing to check")
+        return 0
+
+    checks = []
+    if role == "all":
+        checks = [_web_ok, _lanes_ok]
+    elif role == "web":
+        checks = [_web_ok]
+    elif role in CELERY_ROLES:
+        checks = [_this_node_ok]
+    else:
+        print(f"unknown role {role!r}; nothing to check")
+        return 0
+
+    for check in checks:
+        ok, detail = check()
+        if not ok:
+            print(detail, file=sys.stderr)
+            return 1
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

backend/app/scripts/wait_for_deps.py

@@ -0,0 +1,146 @@
+"""Block until Postgres and Redis accept connections. Exit 0 ready, 1 timed out.
+
+## Why the container has to do this itself
+
+Compose has `depends_on: {condition: service_healthy}`, and **Swarm ignores
+it**. `docker stack deploy` has no ordering primitive at all: every service in
+the stack starts at once, so FabledCurator races Postgres on every cold
+deploy and always has.
+
+The multi-service stack hid how sharp that is. `web` ran `alembic upgrade
+head`, failed against a Postgres that was still doing `initdb`, and the task
+died — but Swarm restarts a failed task forever, so the service came up a few
+seconds later and nobody saw a problem worth naming.
+
+Consolidation removes that safety net. supervisord gives each program
+`startretries=3`, so a web program that fails three times in the first
+seconds goes FATAL and **stays** FATAL: supervisord keeps running, the
+container keeps running, and the application never starts. The healthcheck
+catches it — but as a container that is permanently unhealthy for a reason
+that has nothing to do with the image, on a stack whose database simply took
+twenty seconds to initialise.
+
+Operator, 2026-09-23: *"it's a single container that need to connect
+successfully to redis and postgres before starting work shouldn't that simply
+be a check (with retries) at the start of the container."* Yes.
+
+## A TCP connect, not a query
+
+The same probe `build.yml`'s integration lane and the build smoke already use.
+It answers the question that is actually being asked — is something listening
+— and it cannot fail for a reason that retrying will never fix.
+
+A real query would be a stronger readiness signal and a worse gate: a wrong
+password or a missing database is not a transient condition, and a loop that
+waits for one to heal turns a five-second misconfiguration into a two-minute
+timeout with a misleading message. Those belong to alembic, which runs
+seconds later and says exactly what is wrong.
+
+The Postgres image is well behaved here: during `initdb` it serves on a unix
+socket only and opens TCP when it is ready for clients, so the connect is a
+good proxy for "ready" rather than merely "process exists".
+"""
+
+from __future__ import annotations
+
+import argparse
+import os
+import socket
+import sys
+import time
+from urllib.parse import urlparse
+
+# Long enough for a first-ever `initdb` on a slow disk, which is the worst
+# case this exists for and is measured in tens of seconds, not minutes. A
+# deploy that is genuinely misconfigured should fail while someone is still
+# watching it rather than hold the container open for a quarter of an hour.
+DEFAULT_TIMEOUT = 120.0
+CONNECT_TIMEOUT = 2.0
+RETRY_DELAY = 1.0
+# Progress every N attempts. `docker logs` on a container that is waiting must
+# say what it is waiting for — silence is indistinguishable from a hang.
+REPORT_EVERY = 5
+
+
+def _target(url: str | None, default_port: int) -> tuple[str, int] | None:
+    """(host, port) from a connection URL, or None if there is nothing to wait for."""
+    if not url:
+        return None
+    parsed = urlparse(url)
+    if not parsed.hostname:
+        return None
+    return parsed.hostname, parsed.port or default_port
+
+
+def targets() -> list[tuple[str, tuple[str, int]]]:
+    """What this container must reach, read from the same env the app reads.
+
+    Derived rather than passed in, so the wait cannot drift from what the
+    application will actually connect to — a gate that checks a different
+    host than the app uses is worse than no gate.
+    """
+    out: list[tuple[str, tuple[str, int]]] = []
+
+    host = os.environ.get("DB_HOST")
+    if host:
+        out.append(("postgres", (host, int(os.environ.get("DB_PORT") or 5432))))
+
+    broker = _target(os.environ.get("CELERY_BROKER_URL"), 6379)
+    if broker:
+        out.append(("redis", broker))
+
+    return out
+
+
+def _accepts(host: str, port: int) -> bool:
+    try:
+        with socket.create_connection((host, port), timeout=CONNECT_TIMEOUT):
+            return True
+    except OSError:
+        return False
+
+
+def wait(
+    name: str, host: str, port: int, deadline: float, now=time.monotonic,
+) -> bool:
+    attempt = 0
+    while True:
+        if _accepts(host, port):
+            print(f"[wait] {name} at {host}:{port} is accepting connections")
+            return True
+        attempt += 1
+        if now() >= deadline:
+            print(
+                f"[wait] TIMEOUT: {name} at {host}:{port} never accepted a "
+                f"connection ({attempt} attempts)",
+                file=sys.stderr,
+            )
+            return False
+        if attempt % REPORT_EVERY == 0:
+            left = int(deadline - now())
+            print(f"[wait] {name} at {host}:{port} not ready yet, {left}s left")
+        time.sleep(RETRY_DELAY)
+
+
+def main(argv: list[str] | None = None) -> int:
+    ap = argparse.ArgumentParser(description="Wait for Postgres and Redis.")
+    ap.add_argument("--timeout", type=float, default=DEFAULT_TIMEOUT)
+    args = ap.parse_args(argv)
+
+    wanted = targets()
+    if not wanted:
+        # Nothing configured to wait for. Not an error: `shell` and one-off
+        # runs are legitimate, and refusing to start would make this gate the
+        # reason a debugging container will not boot.
+        print("[wait] no database or broker configured; nothing to wait for")
+        return 0
+
+    deadline = time.monotonic() + args.timeout
+    for name, (host, port) in wanted:
+        if not wait(name, host, port, deadline):
+            return 1
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

backend/app/services/artist_service.py

@@ -300,6 +300,18 @@ class ArtistService:
         await self.session.commit()
         return artist
 
+    async def all_names(self) -> list[tuple[int, str, str]]:
+        """Every artist as (id, name, slug), alphabetical.
+
+        For pickers that should show a full list before anything is typed (the
+        Latest feed's artist filter). Three columns and no joins, so it stays
+        cheap on a library of thousands of artists.
+        """
+        rows = (await self.session.execute(
+            select(Artist.id, Artist.name, Artist.slug).order_by(func.lower(Artist.name))
+        )).all()
+        return [(r.id, r.name, r.slug) for r in rows]
+
     async def autocomplete(self, prefix: str, limit: int = 20) -> list[Artist]:
         cleaned = (prefix or "").strip()
         if not cleaned:

backend/app/services/backup_service.py

@@ -24,6 +24,25 @@ from pathlib import Path
 
 _BACKUPS_DIRNAME = "_backups"
 
+# Excluded from the images tarball, and each for its own reason (#4233, #4234):
+#
+#   _backups    — the archive would otherwise contain every previous archive.
+#                 This is not hypothetical: the 2026-05-23/24 runs, taken before
+#                 this exclude existed, grew 43G -> 107G -> ... -> 2123G as each
+#                 swallowed its predecessors, and cost 4.3T of the images
+#                 filesystem until they were reclaimed on 2026-09-21.
+#   _quarantine — holds files deliberately pulled OUT of the library.
+#   secrets     — `credential_key.b64`, the key that decrypts the stored
+#                 Patreon/SubscribeStar session credentials.
+#   cookies     — those session cookies themselves.
+#
+# The last two are the ones worth stating plainly: an images tarball is a media
+# archive, and a media archive that carries the key to the operator's accounts
+# is a credential leak wearing a backup's name. Encryption at rest buys nothing
+# when the key rides along in the same file. A restore therefore does NOT
+# re-establish credentials — you sign in again, which is the correct outcome.
+_IMAGES_EXCLUDED_DIRNAMES = ("_backups", "_quarantine", "secrets", "cookies")
+
 # Subprocess-level guardrails BEYOND the Celery soft_time_limit. The Celery
 # soft limit signals the Python process; subprocess.Popen in a blocking syscall
 # ignores that signal, so these bound the worst case directly. Each sits just
@@ -173,8 +192,10 @@ def backup_images(
         [
             "tar", "--zstd", "-cf", str(tar_path),
             "-C", str(images_root.parent), images_root.name,
-            f"--exclude={images_root.name}/_backups",
-            f"--exclude={images_root.name}/_quarantine",
+            *(
+                f"--exclude={images_root.name}/{name}"
+                for name in _IMAGES_EXCLUDED_DIRNAMES
+            ),
         ],
         _IMAGES_SUBPROCESS_TIMEOUT_S,
     )

backend/app/services/credential_crypto.py

@@ -24,6 +24,7 @@ rows undecryptable (recovery = delete the rows and re-upload).
 
 import logging
 import os
+import tempfile
 from pathlib import Path
 
 from cryptography.fernet import Fernet, InvalidToken
@@ -80,10 +81,59 @@ class CredentialCrypto:
         parent = self._key_path.parent
         parent.mkdir(parents=True, exist_ok=True)
         os.chmod(parent, 0o700)
+
+        # Written to a temp file and LINKED into place, not written directly.
+        #
+        # hypercorn starts several worker processes and each one builds the
+        # app, so on a first boot they all reach this at once. A plain
+        # `write_bytes` creates the file at size zero and fills it a moment
+        # later, which gives a second process an `exists()` of True and a
+        # `read_bytes()` of b"" — and the app dies with
+        #
+        #     ValueError: Fernet key must be 32 url-safe base64-encoded bytes.
+        #
+        # Seen on run 7368's smoke, and it is a race rather than a certainty:
+        # the same image had booted cleanly on the three runs before it. A
+        # first boot that fails one time in five is worse than one that fails
+        # every time, because it looks like the deployment rather than the code.
+        #
+        # `os.link` is the atomic part: it either creates the name or raises
+        # FileExistsError, and it cannot expose a half-written file. NOT
+        # `os.replace`, which would succeed — so two processes that both
+        # generated a key would each think they had won, and the loser's key
+        # would overwrite the one the winner had already handed to Fernet.
+        # `mkstemp`, not a pid-derived name. The first cut spelled the temp
+        # file `.credential_key.b64.<pid>.tmp`, which assumes one bootstrap per
+        # process — and the test that exercises this with eight THREADS shares
+        # one pid, so all eight raced the same filename and six died with
+        # FileNotFoundError when another had already unlinked it. The
+        # assumption held for hypercorn's workers and would have held in
+        # production; it was still an assumption the code did not need to make.
         key = Fernet.generate_key()
-        self._key_path.write_bytes(key)
-        os.chmod(self._key_path, 0o600)
-        return key
+        fd, tmp_name = tempfile.mkstemp(
+            dir=parent, prefix=f".{self._key_path.name}.", suffix=".tmp",
+        )
+        tmp = Path(tmp_name)
+        try:
+            with os.fdopen(fd, "wb") as fh:
+                fh.write(key)
+            os.chmod(tmp, 0o600)
+            try:
+                os.link(tmp, self._key_path)
+            except FileExistsError:
+                # Another process created it between our `exists()` check and
+                # here. Theirs is as good as ours, and using it is what keeps
+                # every worker on ONE key.
+                log.info(
+                    "another process created %s first; using that key",
+                    self._key_path,
+                )
+        finally:
+            tmp.unlink(missing_ok=True)
+        # Read back rather than returning `key`: on the losing branch the file
+        # holds somebody else's, and returning ours would leave this worker
+        # encrypting with a key no other worker can read.
+        return self._key_path.read_bytes()
 
     def encrypt(self, plaintext: str) -> bytes:
         return self._fernet.encrypt(plaintext.encode("utf-8"))

backend/app/services/db_helpers.py

@@ -16,7 +16,7 @@ from __future__ import annotations
 
 from collections.abc import Awaitable, Callable
 
-from sqlalchemy import Select
+from sqlalchemy import Select, and_
 from sqlalchemy.exc import IntegrityError
 from sqlalchemy.ext.asyncio import AsyncSession
 
@@ -67,12 +67,26 @@ async def get_or_create[T](
 
 
 def failing_sources_clause():
-    """A source is FAILING when its runs are actually erroring.
+    """A source is FAILING when it is ENABLED and its runs are erroring.
 
     Deliberately not `last_error IS NOT NULL` — a tier-limited source clears
     last_error and keeps a chip, and must never be counted as broken.
+
+    The `enabled` half was folded in 2026-09-21 (#4279). A disabled source is
+    one FC deliberately stopped — most often because the membership sweep saw
+    `former_patron` — and "stopped because you no longer subscribe" is not
+    "failing". Worse, it is a failure nobody can clear: a disabled source is
+    never scheduled, so no successful run ever resets the counter, and the
+    card's Retry button routes to `/check`, which refuses a disabled source
+    outright. Ebi77 sat in the banner for six days with no action available.
+
+    This also settles a disagreement the two callers already had. The
+    scheduler's status count paired this clause with `enabled.is_(True)`;
+    `SourceService.list(failing=True)` did not. One counted Ebi77, the other
+    did not — the exact drift the note above this function warns about, which
+    is why the `enabled` test belongs IN the predicate rather than beside it.
     """
-    return Source.consecutive_failures > 0
+    return and_(Source.enabled.is_(True), Source.consecutive_failures > 0)
 
 
 def no_access_sources_clause():

backend/app/services/discord_repair.py

@@ -0,0 +1,183 @@
+"""Repair the Discord downloads made before the naming fix (issue #3999).
+
+Until dc840fe, gallery-dl's Discord patterns asked for keys the extractor never
+emits, so every Discord download landed as
+`<artist>/discord/None/<date>_None_<original name>`, next to a sidecar named
+after the attachment's ORIGINAL name. That broke two things:
+
+* **No Post, no date.** `find_sidecar` can never pair those names, so these
+  files were imported as loose images with no Post, and a card shows the
+  download time.
+* **No trustworthy metadata to relink from.** Every `image.png` in a channel
+  wrote the same `image.json`, so the surviving sidecar describes whichever
+  message was written last. The message id is gone from the filename too.
+
+The operator chose a clean re-download (2026-09-13) over relinking in place:
+delete the broken files and their records, make gallery-dl forget it fetched
+them, and backfill every Discord source again under the fixed naming.
+
+## Why the archive is cleared for ALL of Discord
+
+gallery-dl records a download as `discord{message_id}_{num}` (upstream
+`DiscordExtractor.archive_fmt`, prefixed with the category). The broken files
+lost their message ids, so there is no way to forget one source's entries and
+not another's. Every Discord download made before the fix is broken, so
+forgetting all of them is exactly right. Anything downloaded AFTER the fix still
+exists on disk under its correct name, and gallery-dl's `skip` sees the file and
+does not fetch it again.
+
+## What it does not touch
+
+Discord posts FC grouped itself (#388 E2) are built from Posts, and these files
+never had one, so there is nothing grouped to unwind. Images outside a
+`discord/None/` directory are never selected: the path filter requires both the
+`None` directory and the `_None_` filename, the pair only the bug produced.
+
+Operator-triggered only (Settings, preview first). Never on a beat.
+"""
+
+from __future__ import annotations
+
+import logging
+import sqlite3
+from pathlib import Path
+
+from sqlalchemy import func, select
+from sqlalchemy.orm import Session
+
+from ..models import ImageRecord, Source
+from .cleanup_service import delete_images
+from .gallery_dl import archive_path
+from .source_service import arm_backfill
+
+log = logging.getLogger(__name__)
+
+# `%` and `_` are LIKE wildcards, so the literal underscores around None are
+# escaped. `________` is the eight-digit date prefix the old pattern wrote.
+_BROKEN_PATH_LIKE = r"%/discord/None/________\_None\_%"
+
+# Upstream keys asset downloads as `asset_{server_id}_{id}`. FC never fetches
+# server assets, but excluding them keeps this to exactly the message
+# attachments the bug mangled.
+_ARCHIVE_SQL_MATCH = r"entry LIKE 'discord%' AND entry NOT LIKE 'discordasset\_%' ESCAPE '\'"
+_COUNT_SQL = "SELECT COUNT(*) FROM archive WHERE " + _ARCHIVE_SQL_MATCH
+_DELETE_SQL = "DELETE FROM archive WHERE " + _ARCHIVE_SQL_MATCH
+
+
+def broken_directories(images_root: Path) -> list[Path]:
+    """Every `<artist>/discord/None` directory. Artist folders that differ only by
+    case (`Conto` and `conto`) are separate directories and both are found."""
+    return sorted(d for d in Path(images_root).glob("*/discord/None") if d.is_dir())
+
+
+def count_archive_entries(archive: Path) -> int:
+    return _archive(archive, delete=False)
+
+
+def forget_archive_entries(archive: Path) -> int:
+    return _archive(archive, delete=True)
+
+
+def _archive(archive: Path, *, delete: bool) -> int:
+    if not archive.is_file():
+        return 0
+    # A download running at the same moment holds this file briefly. Waiting
+    # 30s for its lock beats failing the repair over a transient contention.
+    conn = sqlite3.connect(str(archive), timeout=30)
+    try:
+        has_table = conn.execute(
+            "SELECT 1 FROM sqlite_master WHERE type='table' AND name='archive'"
+        ).fetchone()
+        if not has_table:
+            return 0
+        if not delete:
+            return conn.execute(_COUNT_SQL).fetchone()[0]
+        cur = conn.execute(_DELETE_SQL)
+        conn.commit()
+        return cur.rowcount
+    finally:
+        conn.close()
+
+
+def _sweep_directory(directory: Path) -> tuple[int, bool]:
+    """Remove what the record deletes left behind: the collided sidecars, plus any
+    file that never became a record (a quarantined or rejected download). Then
+    the directory itself, if it is empty. Returns (files removed, dir removed)."""
+    removed = 0
+    for f in directory.iterdir():
+        if f.is_file():
+            try:
+                f.unlink()
+                removed += 1
+            except OSError as exc:
+                log.warning("discord repair: could not remove %s: %s", f, exc)
+    try:
+        directory.rmdir()
+        return removed, True
+    except OSError:
+        return removed, False
+
+
+def repair_discord_downloads(
+    session: Session, *, images_root: Path, dry_run: bool,
+) -> dict:
+    images_root = Path(images_root)
+    archive = archive_path(images_root)
+
+    broken = select(ImageRecord.id, ImageRecord.size_bytes).where(
+        ImageRecord.path.like(_BROKEN_PATH_LIKE, escape="\\")
+    )
+    rows = session.execute(broken).all()
+    image_ids = [r.id for r in rows]
+    directories = broken_directories(images_root)
+    sources = session.execute(
+        select(Source).where(Source.platform == "discord")
+    ).scalars().all()
+
+    summary = {
+        "images": len(image_ids),
+        "bytes": sum(r.size_bytes or 0 for r in rows),
+        "directories": len(directories),
+        "sources": len(sources),
+        "enabled_sources": sum(1 for s in sources if s.enabled),
+    }
+
+    if dry_run:
+        summary["archive_entries"] = count_archive_entries(archive)
+        return summary
+
+    deleted = delete_images(session, image_ids=image_ids, images_root=images_root)
+
+    swept = 0
+    directories_removed = 0
+    for d in directories:
+        n, gone = _sweep_directory(d)
+        swept += n
+        directories_removed += int(gone)
+
+    # Only after the files are gone. Forgetting first and failing half way would
+    # leave gallery-dl free to re-fetch into a directory still full of the old
+    # copies.
+    forgotten = forget_archive_entries(archive)
+
+    for source in sources:
+        arm_backfill(source)
+    session.commit()
+
+    remaining = session.execute(
+        select(func.count(ImageRecord.id)).where(
+            ImageRecord.path.like(_BROKEN_PATH_LIKE, escape="\\")
+        )
+    ).scalar_one()
+
+    summary.update(
+        images_deleted=deleted["images_deleted"],
+        files_failed=deleted["files_failed"],
+        leftover_files_removed=swept,
+        directories_removed=directories_removed,
+        archive_entries=forgotten,
+        backfills_started=len(sources),
+        remaining=remaining,
+    )
+    log.info("discord repair applied: %s", summary)
+    return summary

backend/app/services/download_backends.py

@@ -26,14 +26,12 @@ from pathlib import Path
 from .gallery_dl import DownloadResult, ErrorType
 from .patreon_ingester import PatreonIngester
 from .patreon_resolver import extract_vanity, resolve_campaign_id_for_source
-from .pixiv_client import user_id_from_url
-from .pixiv_ingester import PixivIngester
 from .platforms import known_platform_keys
 from .subscribestar_ingester import SubscribeStarIngester
 
 # Platforms whose download + verify go through the native ingester rather than
 # gallery-dl. gallery-dl still serves the rest (hentaifoundry, discord) until
-# they migrate too. pixiv left this set when it was retired (milestone #406).
+# they migrate too.
 NATIVE_INGESTER_PLATFORMS = frozenset({"patreon", "subscribestar"})
 
 
@@ -67,7 +65,6 @@ def _native_ingester_cls(platform: str):
     dispatch pick up the replacement."""
     return {
         "patreon": PatreonIngester,
-        "pixiv": PixivIngester,
         "subscribestar": SubscribeStarIngester,
     }[platform]
 
@@ -127,25 +124,17 @@ async def _resolve_native_campaign_id(
     platform: str, url: str, cookies_path: str | None, overrides: dict,
 ) -> tuple[str | None, str | None]:
     """`(campaign_id, resolved_campaign_id)` for a native source. SubscribeStar's
-    feed id IS the creator URL; Pixiv's is the numeric user id parsed straight
-    from it (no lookup → resolved None either way). Patreon resolves the
+    feed id IS the creator URL (no lookup → resolved None). Patreon resolves the
     campaign id from the vanity URL (resolved non-None when a lookup actually ran,
     so phase 3 caches it)."""
     if platform == "subscribestar":
         return url, None
-    if platform == "pixiv":
-        return user_id_from_url(url), None
     return await resolve_campaign_id_for_source(url, cookies_path, overrides)
 
 
 def _campaign_resolution_error(platform: str, url: str) -> str:
     """Operator-facing message for a native source whose campaign id could not
     be resolved — names the platform's own lookup mechanism."""
-    if platform == "pixiv":
-        return (
-            f"Could not extract a pixiv user id. source_url={url!r} — expected "
-            "a URL like https://www.pixiv.net/users/<id>."
-        )
     vanity = extract_vanity(url)
     return (
         f"Could not resolve Patreon campaign id. source_url={url!r}; "
@@ -172,8 +161,8 @@ async def _run_native_ingester(
         platform, ctx["url"], ctx["cookies_path"], overrides
     )
     if not campaign_id:
-        # Patreon: vanity lookup failed. Pixiv: no numeric user id in the URL.
-        # (SubscribeStar's campaign id is the URL itself — never lands here.)
+        # Patreon: vanity lookup failed. (SubscribeStar's campaign id is the
+        # URL itself — never lands here.)
         url = ctx["url"]
         return (
             DownloadResult(
@@ -205,7 +194,7 @@ async def _run_native_ingester(
         validate=gdl._validate_files,
         rate_limit=rate_limit,
         request_sleep=request_sleep,
-        # Uniform across adapters: token platforms (pixiv) authenticate with
+        # Uniform across adapters: a token platform would authenticate with
         # it, cookie platforms accept-and-ignore — so this construction stays
         # platform-agnostic.
         auth_token=ctx["auth_token"],
@@ -252,16 +241,11 @@ async def verify_source_credential(
     if uses_native_ingester(platform):
         # Native ingester platforms verify via their own lightweight auth probe.
         # SubscribeStar's probe takes the creator URL directly; Patreon's
-        # resolves the campaign id first; Pixiv's is one OAuth refresh (the
-        # exact call that fails when the token is bad — no feed walk).
+        # resolves the campaign id first.
         if platform == "subscribestar":
             from .subscribestar_ingester import verify_subscribestar_credential
 
             return await verify_subscribestar_credential(url, cookies_path, config_overrides)
-        if platform == "pixiv":
-            from .pixiv_ingester import verify_pixiv_credential
-
-            return await verify_pixiv_credential(auth_token)
         from .patreon_ingester import verify_patreon_credential
 
         return await verify_patreon_credential(url, cookies_path, config_overrides)

backend/app/services/extension_service.py

@@ -61,8 +61,8 @@ _PLATFORM_PATTERNS: list[tuple[str, re.Pattern[str]]] = [
 class ExtensionService:
     def __init__(self, session: AsyncSession, crypto=None) -> None:
         self.session = session
-        # Optional decryptor for resolving a token-auth platform's display name
-        # (pixiv) at add-time. None → skip resolution, fall back to the handle.
+        # Optional decryptor for resolving a platform's display name at
+        # add-time. None → skip resolution, fall back to the handle.
         self._crypto = crypto
 
     async def quick_add_source(self, url: str) -> dict:
@@ -112,12 +112,12 @@ class ExtensionService:
         self, platform: str, raw_slug: str, url: str
     ) -> str:
         """The real display name for a new artist, resolved from the platform at
-        add-time (#130). Our native platforms each have a name source — pixiv the
-        app API (token), patreon the campaigns API, subscribestar the profile
-        page (both cookies). Other platforms (and any failure — no credential,
-        network error) fall back to the URL handle, which is already readable.
+        add-time (#130). Our native platforms each have a name source — patreon
+        the campaigns API, subscribestar the profile page (both cookies). Other
+        platforms (and any failure — no credential, network error) fall back to
+        the URL handle, which is already readable.
         The resolvers are sync, so they run in an executor."""
-        if self._crypto is None or platform not in ("pixiv", "patreon", "subscribestar"):
+        if self._crypto is None or platform not in ("patreon", "subscribestar"):
             return raw_slug
         import asyncio
 
@@ -125,15 +125,7 @@ class ExtensionService:
         cred = CredentialService(self.session, self._crypto)
         loop = asyncio.get_running_loop()
         try:
-            if platform == "pixiv":
-                token = await cred.get_token("pixiv")
-                if not token:
-                    return raw_slug
-                from .pixiv_client import PixivClient
-                name = await loop.run_in_executor(
-                    None, PixivClient(token).resolve_display_name, raw_slug
-                )
-            elif platform == "patreon":
+            if platform == "patreon":
                 cookies = await cred.get_cookies_path("patreon")
                 from .patreon_resolver import resolve_display_name
                 name = await loop.run_in_executor(

backend/app/services/gallery_dl.py

@@ -94,6 +94,58 @@ BACKFILL_CHUNK_SECONDS = 600
 _DEFAULT_GDL_TIMEOUT_SECONDS = 870
 
 
+# --- Discord naming ---------------------------------------------------------
+#
+# Derived from a REAL sidecar (operator's instance, 2026-09-13), not from memory
+# of gallery-dl's extractor. What gallery-dl's discord extractor actually emits
+# for an attachment: `channel` is a plain STRING (the channel's name), the
+# message is `message_id`, the attachment's position in it is `num`, and there
+# is NO `id` key at all.
+#
+# The previous patterns asked for `{channel[name]}` and `{id}`. Both render as
+# "None", so every Discord download since the platform was added landed in a
+# directory called `None` as `<date>_None_<original name>`. Worse, the sidecar was
+# named `{filename}.json` — the attachment's ORIGINAL name — which (a) `find_
+# sidecar` can never pair with `<date>_None_<name>.png`, so no Discord file ever
+# got a Post or a post date, and (b) collides: every `image.png` in a channel
+# overwrote the same `image.json`, so the one sidecar that survived described
+# whichever message happened to be written last.
+#
+# The fix names the sidecar EXACTLY like the media minus its extension, so
+# `find_sidecar`'s first candidate (`media.with_suffix(".json")`) is the match
+# and the name is unique per attachment. tests/test_gallery_dl_naming.py renders
+# these patterns against a sanitized copy of the real sidecar, so a key that
+# does not exist fails CI instead of silently becoming "None".
+DISCORD_FILENAME = "{date:%Y%m%d}_{message_id}_{num:>02}_{filename}.{extension}"
+DISCORD_DIRECTORY = ["{channel}"]
+
+
+def sidecar_name_for(media_pattern: str) -> str | None:
+    """The metadata filename pattern that names a sidecar exactly like its media.
+
+    Returns None for a pattern that does not end in `.{extension}`, since then
+    there is no media stem to mirror and the caller must fall back.
+    """
+    suffix = ".{extension}"
+    if not media_pattern.endswith(suffix):
+        return None
+    return media_pattern[: -len(suffix)] + ".json"
+
+
+def metadata_postprocessor(filename: str) -> dict:
+    return {"name": "metadata", "mode": "json", "directory": ".", "filename": filename}
+
+
+def archive_path(images_root: Path) -> Path:
+    """gallery-dl's download archive: the record of what it has already fetched.
+
+    One definition, because the Discord repair (services/discord_repair.py) has
+    to find the same file the downloader writes, without constructing a service
+    whose __init__ creates directories.
+    """
+    return Path(images_root) / ".gallery-dl" / "archive.sqlite3"
+
+
 @dataclass
 class SourceConfig:
     """Per-source overrides loaded from Source.config_overrides JSON.
@@ -376,8 +428,11 @@ class GalleryDLService:
         },
         "discord": {
             "content_types": ["all"],
-            "directory": ["{channel[name]}"],
-            "filename": "{date:%Y%m%d}_{id}_{filename}.{extension}",
+            "directory": DISCORD_DIRECTORY,
+            "filename": DISCORD_FILENAME,
+            # Overrides the global `{filename}.json` sidecar for this extractor
+            # only — see the Discord naming note above.
+            "postprocessors": [metadata_postprocessor(sidecar_name_for(DISCORD_FILENAME))],
             "embeds": "all",
             "stickers": True,
             "reactions": False,
@@ -402,7 +457,7 @@ class GalleryDLService:
         config = {
             "extractor": {
                 "base-directory": str(self.images_root),
-                "archive": str(self._config_dir / "archive.sqlite3"),
+                "archive": str(archive_path(self.images_root)),
                 "skip": True,
                 "sleep": self._rate_limit,
                 "sleep-request": max(0.5, self._rate_limit / 4),
@@ -497,6 +552,17 @@ class GalleryDLService:
         if source_config.filename_pattern:
             platform_section["filename"] = source_config.filename_pattern
 
+        # A platform that names its sidecar after its media must keep doing so
+        # under a per-source filename override, or the pairing breaks exactly the
+        # way Discord's did. No metadata wanted means no platform postprocessor
+        # either — the global list was already dropped above.
+        if "postprocessors" in platform_section:
+            mirrored = sidecar_name_for(platform_section.get("filename") or "")
+            if not source_config.save_metadata or mirrored is None:
+                platform_section.pop("postprocessors")
+            else:
+                platform_section["postprocessors"] = [metadata_postprocessor(mirrored)]
+
         platform_section["metadata"] = source_config.save_metadata
 
         return config

backend/app/services/gallery_service.py

@@ -45,7 +45,7 @@ from .tag_query import (
 # provenance (filesystem imports). Returned by facets() as a null-valued
 # bucket; the frontend maps that null back to this sentinel in the URL so the
 # bucket is selectable. Underscore-wrapped so it can't collide with a real
-# gallery-dl platform name (patreon/pixiv/...).
+# gallery-dl platform name (patreon/hentaifoundry/...).
 UNSOURCED_PLATFORM = "__unsourced__"
 
 
@@ -322,7 +322,7 @@ def _gallery_images(rows, artists: dict[int, dict]) -> list[GalleryImage]:
     ]
 
 
-def _diversify_similar(src, rows, limit, *, dup_threshold=8, lam=0.40):
+def _diversify_similar(src, rows, limit, *, dup_threshold=32, lam=0.40):
     """Trim a nearest-cosine candidate pool down to `limit` diverse picks.
 
     1. pHash collapse: drop any candidate whose perceptual hash is within
@@ -338,6 +338,11 @@ def _diversify_similar(src, rows, limit, *, dup_threshold=8, lam=0.40):
     2026-07-01 — dropped 0.55→0.40, dup 6→8, paired with a wider pool in
     `similar()`).
 
+    `dup_threshold` counts Hamming bits, so it moved 8→32 when the pHash went
+    from 64 to 256 bits (#4223, migration 0098) — the same fraction of the
+    hash, i.e. the tuning the operator chose, unchanged. This collapse is
+    DISPLAY-only: it hides a near-dup from one rail, it never drops a record.
+
     Falls back to nearest-order (`rows[:limit]`) on any failure or a small pool.
     """
     if len(rows) <= 1:

backend/app/services/importer.py

@@ -33,13 +33,14 @@ from ..models import (
 )
 from ..utils import safe_probe
 from ..utils.paths import (
+    canonical_subdir,
     derive_subdir,
     derive_top_level_artist,
     filehash_from_url,
     hash_suffixed_name,
     safe_ext,
 )
-from ..utils.phash import compute_phash, find_similar
+from ..utils.phash import compute_phash, find_similar, fingerprint_path, fingerprints_match
 from ..utils.sidecar import find_sidecar, parse_sidecar
 from ..utils.slug import slugify
 from .archive_extractor import extract_archive, is_archive
@@ -234,6 +235,38 @@ class Importer:
                 (phash, width or 0, height or 0, image_id)
             )
 
+    def _pixel_confirmer(self, source: Path):
+        """Build `find_similar`'s gate-3 callback for an incoming file.
+
+        pHash proposes; this accepts. A candidate is a duplicate only if its
+        file really is the same picture as `source` at a different size —
+        which is the only merge the operator asked for (#4223). Everything
+        else (a missing file, an unreadable one, a deleted row) returns
+        False: the destructive outcomes here are dropping a download and
+        overwriting a kept file, so an unanswerable question must not read
+        as "yes".
+
+        Both sides' fingerprints are computed lazily and cached, so an
+        import that matches nothing costs no I/O at all and an archive
+        member that keeps hitting the same candidate pays for it once.
+        """
+        new_fp: list = []
+        cand_fps: dict[int, object] = {}
+
+        def confirm(candidate_id: int) -> bool:
+            if not new_fp:
+                new_fp.append(fingerprint_path(source))
+            if new_fp[0] is None:
+                return False
+            if candidate_id not in cand_fps:
+                rec = self.session.get(ImageRecord, candidate_id)
+                cand_fps[candidate_id] = (
+                    fingerprint_path(Path(rec.path)) if rec and rec.path else None
+                )
+            return fingerprints_match(new_fp[0], cand_fps[candidate_id])
+
+        return confirm
+
     def _get_or_create(self, stmt, factory):
         """Race-safe find-or-create. Run `stmt` (scalar_one_or_none); if a
         row exists, return it. Otherwise open a savepoint and INSERT
@@ -862,6 +895,7 @@ class Importer:
                 rel, match_id = find_similar(
                     phash, width or 0, height or 0,
                     candidates, self.settings.phash_threshold,
+                    confirm=self._pixel_confirmer(source),
                 )
                 if rel == "larger_exists":
                     # Enrich-on-duplicate (parity with attach_in_place).
@@ -911,7 +945,7 @@ class Importer:
                 )
                 return ImportResult(status="superseded", image_id=match_id)
 
-        dest = self._copy_to_library(source, sha, attribution_path)
+        dest = self._copy_to_library(source, sha, attribution_path, path_artist)
 
         record = ImageRecord(
             path=str(dest),
@@ -1241,6 +1275,7 @@ class Importer:
                 rel, match_id = find_similar(
                     phash, width or 0, height or 0,
                     candidates, self.settings.phash_threshold,
+                    confirm=self._pixel_confirmer(path),
                 )
                 if rel == "larger_exists":
                     # Enrich-on-duplicate: link the near-dup's post to the
@@ -1566,7 +1601,8 @@ class Importer:
         )
 
     def _copy_to_library(
-        self, source: Path, sha: str, attribution_path: Path
+        self, source: Path, sha: str, attribution_path: Path,
+        artist: Artist | None = None,
     ) -> Path:
         """Copy `source` to its final library path. Returns the destination.
 
@@ -1574,8 +1610,18 @@ class Importer:
         _import_media (filesystem scan) and _supersede (when new_path is
         not passed). FC-3c's attach_in_place skips this helper entirely
         — the file is already at its final home.
+
+        `artist`, when resolved, decides the top-level directory: the
+        library is keyed on the Artist row's slug, NOT on however the
+        import folder happened to be capitalised. Without that, an import
+        from `/import/Conto/` and a download for the same artist write to
+        `Conto/` and `conto/` respectively and the library grows a second
+        home for one artist (milestone #421).
         """
-        subdir = derive_subdir(attribution_path, self.import_root)
+        subdir = canonical_subdir(
+            derive_subdir(attribution_path, self.import_root),
+            artist.slug if artist else None,
+        )
         dest_dir = self.images_root / subdir if subdir else self.images_root
         dest_dir.mkdir(parents=True, exist_ok=True)
         dest_name = hash_suffixed_name(source.stem, sha, source.suffix)
@@ -1610,7 +1656,16 @@ class Importer:
         that path (FC-3c attach_in_place case) — skip the copy step.
         Otherwise the file is copied via _copy_to_library."""
         if new_path is None:
-            dest = self._copy_to_library(source, sha, source)
+            # The KEPT row's artist decides the destination, not the incoming
+            # file's folder — a supersede rewrites `existing.path`, so writing
+            # it anywhere but that artist's canonical directory would move a
+            # row OUT of the tree milestone #421 is consolidating. ImageRecord
+            # carries `artist_id` with no relationship attribute, so this is a
+            # lookup rather than `existing.artist`.
+            kept_artist = artist
+            if kept_artist is None and existing.artist_id is not None:
+                kept_artist = self.session.get(Artist, existing.artist_id)
+            dest = self._copy_to_library(source, sha, source, kept_artist)
         else:
             dest = new_path
 

backend/app/services/ingest_core.py

@@ -113,9 +113,9 @@ class Ingester:
         # (e.g. "Patreon API", "SubscribeStar markup").
         self._drift_label = drift_label or platform
         # #862 canary opt-out: platforms whose posts legitimately have empty
-        # bodies across large samples (pixiv — caption-less artists are common)
-        # would false-positive the zero-bodies-means-drift alarm; their clients
-        # catch drift structurally (response-shape checks) instead. The
+        # bodies across large samples would false-positive the
+        # zero-bodies-means-drift alarm; their clients catch drift structurally
+        # (response-shape checks) instead. The
         # "bodies X/N" summary line still surfaces the ratio either way.
         self._body_canary = body_canary
 

backend/app/services/membership_reconcile.py

@@ -8,9 +8,10 @@ trustworthy enough to act on.
 1. `subscribed_not_tracked` — you pay for this and FC does not follow it. The
    adoption win, and the only bucket carrying an action.
 2. `tracked_not_subscribed` — FC follows this and the roster does not show you
-   paying for it. REPORT ONLY, by the operator's decision (2026-09-11): it says
-   what it sees and links to the existing Subscriptions row, and offers no
-   one-click disable.
+   paying for it. No longer shown on the card: the operator reversed the
+   2026-09-11 "report only" call on 2026-09-13. The lapsed half of it now ACTS,
+   in `apply_membership_lapses` below (#3995). The absent half still only
+   reports, because absence proves nothing.
 3. `matched` — the healthy set. Counted, not listed loudly.
 4. `unidentified` — sources this join cannot speak to at all. Reported as
    exactly that, because the alternative is filing them under a verdict.
@@ -39,7 +40,7 @@ rendered as lapsed. That is the whole reason it returns a tri-state.
 
 from __future__ import annotations
 
-from datetime import datetime
+from datetime import UTC, datetime
 
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
@@ -47,12 +48,12 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from ..models import Artist, MembershipSync, PlatformMembership, Source
 from .membership_roster import (
     get_sync_state,
-    has_paid_access,
     identity_keys_for_source,
     pair_sources_with_memberships,
     roster_is_fresh,
     url_tail,
 )
+from .native_ingest_common import has_paid_access
 
 # Why a source appears in `tracked_not_subscribed`. Ordered strongest first —
 # the UI renders a different sentence per basis, because collapsing them into
@@ -233,3 +234,126 @@ async def reconcile_all(session: AsyncSession, now: datetime | None = None) -> d
             await reconcile(session, platform=p, now=now) for p in sorted(platforms)
         ]
     }
+
+
+# ---------------------------------------------------------------------------
+# Stop pulling what the account no longer pays for (#3995)
+# ---------------------------------------------------------------------------
+#
+# Operator decision, 2026-09-13, reversing the 2026-09-11 "report only" call
+# for this direction: "if I kill a subscription on patreon I would like the
+# pulling to stop on curator as well", with automatic resume on resubscribing.
+#
+# This is a SOURCE-level action taken by the daily sweep, visible on the source
+# row and reversible there. It is not a fetch-path decision. The line C5 draws,
+# that the roster never decides a POST is inaccessible, still holds: nothing
+# here reads per-post access, and no download path reads the roster
+# (`test_no_fetch_path_can_read_the_roster`). The scheduler keeps selecting on
+# `enabled` alone.
+#
+# Acts ONLY on positive evidence. A source whose matched membership says access
+# has ended is stopped. A source with NO matched membership is left alone,
+# because absence has innocent causes: a creator rename, a source never walked
+# so no id is cached, a membership the platform stopped listing. Stopping on
+# absence would switch off things the operator still pays for.
+#
+# Two app-managed config_overrides keys carry the state. The `_` prefix is
+# already the "FC writes this, an operator edit preserves it" family.
+#   _membership_stopped  set when the sweep stops a source; the sweep resumes
+#                        ONLY sources carrying it, so a source the operator
+#                        switched off by hand is never switched back on
+#   _membership_kept     set by SourceService.update when the operator turns a
+#                        stopped source back ON: a deliberate choice to keep
+#                        pulling a lapsed creator, which the next sweep must
+#                        not undo. Cleared when the membership is paid again.
+STOPPED_KEY = "_membership_stopped"
+KEPT_KEY = "_membership_kept"
+
+
+def _access_expires_at(m: PlatformMembership) -> datetime | None:
+    """When paid access actually ends, if the platform says.
+
+    Patreon keeps a cancelled membership's access until the end of the billing
+    period and reports that date (`member.access_expires_at`, note #3992).
+    SubscribeStar's page gives no such date, so a cancelled SubscribeStar
+    membership stops at once. Returns None when there is no usable date.
+    """
+    details = m.details or {}
+    raw = details.get("access_expires_at") or (details.get("member") or {}).get("access_expires_at")
+    if not isinstance(raw, str) or not raw:
+        return None
+    try:
+        parsed = datetime.fromisoformat(raw.replace("Z", "+00:00"))
+    except ValueError:
+        return None
+    return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
+
+
+async def apply_membership_lapses(
+    session: AsyncSession, *, platform: str, now: datetime | None = None,
+) -> dict:
+    """Stop sources whose paid access has ended; resume the ones this stopped.
+
+    Refuses to act on a roster that isn't fresh, for the same reason C4 refuses
+    to draw conclusions from one.
+    """
+    now = now or datetime.now(UTC)
+    state = await get_sync_state(session, platform)
+    if not roster_is_fresh(state, now=now):
+        return {"platform": platform, "skipped": "roster not fresh", "stopped": 0, "resumed": 0}
+
+    memberships = (await session.execute(
+        select(PlatformMembership).where(PlatformMembership.platform == platform)
+    )).scalars().all()
+    sources = (await session.execute(
+        select(Source).where(Source.platform == platform)
+    )).scalars().all()
+    pairs = pair_sources_with_memberships(list(sources), list(memberships))
+
+    stopped: list[int] = []
+    resumed: list[int] = []
+    for source in sources:
+        pair = pairs.get(source.id)
+        if pair is None:
+            continue  # absence is never acted on, see above
+        m, _kind = pair
+        paid = has_paid_access(
+            m.platform, m.status,
+            is_free_member=bool((m.details or {}).get("is_free_member")),
+        )
+        co = dict(source.config_overrides or {})
+
+        if paid is True:
+            changed = co.pop(KEPT_KEY, None) is not None
+            if STOPPED_KEY in co:
+                co.pop(STOPPED_KEY)
+                source.enabled = True
+                resumed.append(source.id)
+                changed = True
+            if changed:
+                source.config_overrides = co
+            continue
+
+        # Unknown status: never a reason to stop something (has_paid_access's
+        # tri-state exists for exactly this).
+        if paid is None:
+            continue
+        if not source.enabled or co.get(KEPT_KEY):
+            continue
+        expires = _access_expires_at(m)
+        if expires is not None and expires > now:
+            continue  # still inside the paid-through period
+
+        co[STOPPED_KEY] = {"at": now.isoformat(), "status": m.status}
+        source.config_overrides = co
+        source.enabled = False
+        # The same clean slate a manual disable gives (SourceService.update,
+        # #1285), so a stopped source doesn't linger as failing or gated.
+        source.last_error = None
+        source.error_type = None
+        source.consecutive_failures = 0
+        stopped.append(source.id)
+
+    await session.commit()
+    return {"platform": platform, "stopped": len(stopped), "resumed": len(resumed)}
+

backend/app/services/membership_roster.py

@@ -20,8 +20,10 @@ own word — `active_patron`, not some normalised FC value. The mapping from
 those words to FC's meaning is a read-site concern and belongs in code that can
 be corrected without a migration, because the vocabulary comes from whatever
 each platform says and will be discovered per platform rather than designed up
-front. `MEMBERSHIP_STATUS` below is a place for that knowledge to accumulate as
-platforms are characterised; it is deliberately empty of guesses today.
+front. `native_ingest_common.MEMBERSHIP_STATUS` is where that knowledge
+accumulates as platforms are characterised, and it holds no guesses. It lives
+there rather than here because platform clients need it, and a client may not
+import this module (test_gated_reason.py).
 """
 
 from __future__ import annotations
@@ -35,78 +37,10 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from ..models import MembershipSync, PlatformMembership, Source
+from .native_ingest_common import has_paid_access
 
 log = logging.getLogger(__name__)
 
-# Platform word -> whether the account currently has paid access.
-#
-# Every entry here must come from a CHARACTERISED response, never from API docs
-# or a plausible guess — project rule 130, and inventing a status before seeing
-# it in a real payload is exactly the failure it names.
-#
-# patreon: from a live capture of the operator's own session, 2026-09-10
-# (Scribe note #3886). Only two values were OBSERVED in `patron_status` and
-# only those two are here.
-#
-# `declined_patron` is deliberately ABSENT even though it looks obviously
-# right. It appears in the request's `filter[membership_type]`, and the capture
-# proved that filter is NOT the same vocabulary as the attribute — a row
-# selected by the filter as `free_member` came back with
-# `patron_status: former_patron`, a word the filter does not contain. Reading
-# the filter as an enum is the specific mistake the capture caught; adding
-# `declined_patron` on the strength of it would be repeating that mistake one
-# step later.
-#
-# Unknown words are NOT an error: an unrecognised status means the roster
-# records evidence it cannot yet interpret, which is a better state than
-# dropping the row or asserting a meaning for it.
-#
-# subscribestar: from a live capture of the account's /subscriptions page,
-# 2026-09-13 (Scribe note #3989). SubscribeStar gives NO per-row status word —
-# a membership's state is which of two tables it sits in — so the "word" stored
-# is the table card's own `data-identifier`, verbatim. Those two identifiers are
-# the whole vocabulary; there is nothing further to characterise later.
-MEMBERSHIP_STATUS: dict[str, dict[str, bool]] = {
-    "patreon": {
-        "active_patron": True,
-        "former_patron": False,
-    },
-    "subscribestar": {
-        "active_subscriptions": True,
-        "cancelled_subscriptions": False,
-    },
-}
-
-
-def has_paid_access(
-    platform: str, status: str | None, *, is_free_member: bool = False,
-) -> bool | None:
-    """Does this membership mean the account currently PAYS for access?
-
-    Returns None for a status this code has not been taught, which callers must
-    treat as "unknown" rather than as False. The difference matters: False says
-    the operator has lost access, and asserting that from an unrecognised word
-    would tell them to cancel a source they are still paying for.
-
-    `is_free_member` is a second axis, not a status, and that is Patreon's
-    design rather than ours: the capture shows a free follow expressed as a
-    boolean alongside `patron_status`, so a "current" membership can still be
-    one nobody is paying for. Taking status alone would report a free follower
-    as a paying patron, and C4 would then never offer to clean it up.
-
-    (Honest limit: the capture contains no ACTIVE free member, so it cannot
-    demonstrate the two axes coming apart. The separation is what the payload's
-    shape says; the sample only shows it is possible, not that it happens.)
-    """
-    if status is None:
-        return None
-    known = MEMBERSHIP_STATUS.get(platform, {}).get(status)
-    if known is None:
-        return None
-    if not known:
-        return False
-    return not is_free_member
-
 
 async def touch_membership(
     session: AsyncSession,

backend/app/services/native_ingest_common.py

@@ -231,7 +231,7 @@ class Membership:
 
     `status` carries the PLATFORM's own word, verbatim and unmapped
     (`active_patron`, `former_patron`, ...). Deciding what it means is the read
-    site's job — `membership_roster.has_paid_access` — precisely so an
+    site's job — `has_paid_access`, below — precisely so an
     unrecognised word records as evidence rather than as a decision.
 
     `is_free_member` is SEPARATE from status and must stay that way. Patreon
@@ -396,3 +396,81 @@ class BaseNativeDownloader:
         sidecar_path = media_path.with_suffix(".json")
         sidecar_path.write_text(json.dumps(data, indent=2))
         return sidecar_path
+
+# --- membership status vocabulary (#387) ------------------------------------
+#
+# Lives here, beside `Membership`, rather than in `membership_roster`. It is
+# pure platform knowledge with no database behind it, and the platform clients
+# need it too. Patreon's must tell a lapsed membership to a deleted creator
+# (skippable) from a paid one it cannot attribute (drift), and a client may not
+# import `membership_roster`: test_gated_reason.py forbids any fetch path from
+# reaching the roster, so the roster can explain a skip but never cause one.
+#
+# Platform word -> whether the account currently has paid access.
+#
+# Every entry here must come from a CHARACTERISED response, never from API docs
+# or a plausible guess — project rule 130, and inventing a status before seeing
+# it in a real payload is exactly the failure it names.
+#
+# patreon: from a live capture of the operator's own session, 2026-09-10
+# (Scribe note #3886). Only two values were OBSERVED in `patron_status` and
+# only those two are here.
+#
+# `declined_patron` is deliberately ABSENT even though it looks obviously
+# right. It appears in the request's `filter[membership_type]`, and the capture
+# proved that filter is NOT the same vocabulary as the attribute — a row
+# selected by the filter as `free_member` came back with
+# `patron_status: former_patron`, a word the filter does not contain. Reading
+# the filter as an enum is the specific mistake the capture caught; adding
+# `declined_patron` on the strength of it would be repeating that mistake one
+# step later.
+#
+# Unknown words are NOT an error: an unrecognised status means the roster
+# records evidence it cannot yet interpret, which is a better state than
+# dropping the row or asserting a meaning for it.
+#
+# subscribestar: from a live capture of the account's /subscriptions page,
+# 2026-09-13 (Scribe note #3989). SubscribeStar gives NO per-row status word —
+# a membership's state is which of two tables it sits in — so the "word" stored
+# is the table card's own `data-identifier`, verbatim. Those two identifiers are
+# the whole vocabulary; there is nothing further to characterise later.
+MEMBERSHIP_STATUS: dict[str, dict[str, bool]] = {
+    "patreon": {
+        "active_patron": True,
+        "former_patron": False,
+    },
+    "subscribestar": {
+        "active_subscriptions": True,
+        "cancelled_subscriptions": False,
+    },
+}
+
+
+def has_paid_access(
+    platform: str, status: str | None, *, is_free_member: bool = False,
+) -> bool | None:
+    """Does this membership mean the account currently PAYS for access?
+
+    Returns None for a status this code has not been taught, which callers must
+    treat as "unknown" rather than as False. The difference matters: False says
+    the operator has lost access, and asserting that from an unrecognised word
+    would tell them to cancel a source they are still paying for.
+
+    `is_free_member` is a second axis, not a status, and that is Patreon's
+    design rather than ours: the capture shows a free follow expressed as a
+    boolean alongside `patron_status`, so a "current" membership can still be
+    one nobody is paying for. Taking status alone would report a free follower
+    as a paying patron, and C4 would then never offer to clean it up.
+
+    (Honest limit: the capture contains no ACTIVE free member, so it cannot
+    demonstrate the two axes coming apart. The separation is what the payload's
+    shape says; the sample only shows it is possible, not that it happens.)
+    """
+    if status is None:
+        return None
+    known = MEMBERSHIP_STATUS.get(platform, {}).get(status)
+    if known is None:
+        return None
+    if not known:
+        return False
+    return not is_free_member

backend/app/services/patreon_client.py

@@ -58,6 +58,7 @@ from .native_ingest_common import (
     NativeDriftError,
     NativeIngestError,
     basename_from_url,
+    has_paid_access,
     make_session,
     retry_after_seconds,
 )
@@ -631,7 +632,25 @@ class PatreonClient:
                 "cannot tell a complete roster from a truncated one"
             )
 
-    def _membership(self, member: dict, index: dict) -> Membership:
+    def _membership(self, member: dict, index: dict) -> Membership | None:
+        """One member row as a Membership, or None for a row the roster can skip.
+
+        The one skippable row is a LAPSED membership whose creator no longer
+        exists. The live roster (note #3886, CORRECTION 3) returned 104 rows,
+        because FC sends no membership-type filter and so gets lapses going back
+        years. One of them, a membership that ended in 2017, carried no
+        `campaign` relationship at all: the key is absent, not null, and its
+        reward names no campaign either. The creator's page is gone.
+
+        Raising on that row made the whole roster unusable over one membership
+        nobody can act on. Skipping it changes no conclusion. A lapsed
+        membership already means "not paying", absence means the same, and no
+        Source can be matched to a campaign that no longer has an id.
+
+        The refusal stays for every other row. An active or unrecognised
+        membership without a creator is something FC cannot vouch for, and
+        dropping it would read downstream as a cancellation.
+        """
         attrs = member.get("attributes") or {}
         if "patron_status" not in attrs:
             raise PatreonDriftError(
@@ -640,6 +659,17 @@ class PatreonClient:
 
         campaign_ids = self._related_ids(member, "campaign")
         if not campaign_ids:
+            paid = has_paid_access(
+                "patreon", attrs.get("patron_status"),
+                is_free_member=bool(attrs.get("is_free_member")),
+            )
+            if paid is False:
+                log.info(
+                    "Patreon roster: skipping a lapsed membership with no campaign "
+                    "(creator deleted); status=%s access_expires_at=%s",
+                    attrs.get("patron_status"), attrs.get("access_expires_at"),
+                )
+                return None
             raise PatreonDriftError(
                 "Patreon member resource has no campaign relationship — a "
                 "membership we cannot attribute to a creator is not usable"
@@ -700,7 +730,9 @@ class PatreonClient:
             index = self._transform(response)
             rows = [m for m in (response.get("data") or []) if isinstance(m, dict)]
             for member in rows:
-                yield self._membership(member, index)
+                membership = self._membership(member, index)
+                if membership is not None:
+                    yield membership
 
             seen += len(rows)
             total = int(response["meta"]["pagination"]["total"] or 0)

backend/app/services/pixiv_client.py

@@ -1,579 +0,0 @@
-"""Native Pixiv client — the Pixiv adapter's read path.
-
-Pixiv has a real (if unofficial) API: the mobile app API gallery-dl drives
-(`PixivAppAPI`). Per the downloader ground rule — gallery-dl is the
-known-working base — this client mirrors gallery-dl 1.32.5's request profile
-EXACTLY: the same iOS app headers on every request, the same OAuth
-refresh-token dance against oauth.secure.pixiv.net (X-Client-Time +
-X-Client-Hash), and the same `/v1/user/illusts` walk paginated by `next_url`.
-Deviating from that profile is how the SubscribeStar/Patreon spikes broke, so
-any change here should be diffed against gallery-dl's extractor first.
-
-Feed shape (characterized from gallery-dl 1.32.5, extractor/pixiv.py):
-  - `GET /v1/user/illusts?user_id=<id>` returns `{"illusts": [work...],
-    "next_url": "https://app-api...?user_id=..&offset=30" | null}`.
-  - Pagination: re-issue the SAME endpoint with `next_url`'s query params. The
-    query string doubles as our resumable page cursor (re-fetching it re-serves
-    the same page — the ingest-core resume contract).
-  - A work carries id/title/type(illust|manga|ugoira)/caption(HTML)/
-    create_date(ISO+09:00)/tags[{name,translated_name}]/user/page_count/
-    x_restrict/series/total_view/total_bookmarks/meta_single_page/meta_pages.
-  - Files: multi-page → meta_pages[].image_urls.original; single page →
-    meta_single_page.original_image_url; ugoira → `/v1/ugoira/metadata` zip
-    (600x600 → 1920x1080 URL swap, gallery-dl's default non-original mode).
-
-`campaign_id` for Pixiv is the numeric user id (extracted from the source URL
-by `user_id_from_url` — no network resolver needed).
-
-Gated works: pixiv serves a `https://s.pximg.net/common/images/limit_*.png`
-placeholder as the "original" when a work is blocked for this account
-(sanity-level filter, my-pixiv lock, deleted). gallery-dl's fallback for those
-is a web-AJAX scrape that needs PHPSESSID browser cookies — FC stores only the
-OAuth refresh token, so (exactly like our previous gallery-dl configuration,
-which warned "No PHPSESSID cookie set") those works are skipped, via the
-post_is_gated seam. Auth failures are loud (rotate the refresh token); a
-response missing the fields we depend on is DRIFT (update this client).
-
-FC runs on a plain-HTTP homelab; nothing here uses a secure-context Web API.
-"""
-
-from __future__ import annotations
-
-import hashlib
-import logging
-import time
-from collections.abc import Iterator
-from dataclasses import dataclass
-from datetime import UTC, datetime
-from urllib.parse import parse_qsl, urlsplit
-
-import requests
-
-from ..utils.paths import safe_ext
-from .native_ingest_common import (
-    _MAX_429_RETRIES,
-    NativeAuthError,
-    NativeDriftError,
-    NativeIngestError,
-    make_session,
-    retry_after_seconds,
-)
-
-log = logging.getLogger(__name__)
-
-_TIMEOUT_SECONDS = 30.0
-_API_ROOT = "https://app-api.pixiv.net"
-_OAUTH_URL = "https://oauth.secure.pixiv.net/auth/token"
-
-# gallery-dl's public Pixiv-app credentials (PixivAppAPI, also pixivpy's) —
-# these identify the official iOS app to the API, NOT the operator; the
-# operator's identity is the OAuth refresh token.
-_CLIENT_ID = "MOBrBDS8blbauoSck0ZfDbtuzpyT"
-_CLIENT_SECRET = "lsACyCD94FhDUtGTXi3QzcFE2uU1hqtDaKeqrdwj"
-_HASH_SECRET = (
-    "28c1fdd170a5204386cb1313c7077b34"
-    "f83e4aaf4aa829ce78c231e05b0bae2c"
-)
-
-# The exact header set gallery-dl 1.32.5 installs on its session — the proven
-# app-API request profile. The Referer also unlocks i.pximg.net media GETs
-# (403 without it), so the downloader reuses this constant.
-PIXIV_APP_HEADERS = {
-    "App-OS": "ios",
-    "App-OS-Version": "16.7.2",
-    "App-Version": "7.19.1",
-    "User-Agent": "PixivIOSApp/7.19.1 (iOS 16.7.2; iPhone12,8)",
-    "Referer": "https://app-api.pixiv.net/",
-}
-
-# Placeholder image prefix pixiv serves instead of a blocked work's original
-# (limit_sanity_level / limit_mypixiv / limit_unknown variants).
-_LIMIT_URL = "https://s.pximg.net/common/images/limit_"
-
-# The app API reports rate-limiting as an error MESSAGE (often on HTTP 403),
-# not only as HTTP 429. gallery-dl sleeps 300s in-walk; sleeping that long
-# inside our time-boxed chunk would eat the whole budget, so we surface it as
-# a typed 429 and let download_service's cooldown machinery honor the wait.
-_RATE_LIMIT_RETRY_AFTER = 300.0
-
-_TITLE_MAX = 50  # gallery-dl pixiv filename template: {title[:50]}
-
-_RATINGS = {0: "General", 1: "R-18", 2: "R-18G"}
-
-
-class PixivAPIError(NativeIngestError):
-    """Base for native Pixiv client failures. status_code / retry_after are
-    inherited from NativeIngestError."""
-
-
-class PixivAuthError(PixivAPIError, NativeAuthError):
-    """Auth failure — missing/expired/revoked OAuth refresh token. Fix =
-    rotate the credential (Settings → Credentials → Pixiv), not update the
-    client. Maps to error_type 'auth_error'."""
-
-
-class PixivDriftError(PixivAPIError, NativeDriftError):
-    """A response did not match the shape this client depends on (missing
-    `illusts`, un-parseable JSON where JSON was promised). Fail loud so the
-    run flags 'the Pixiv app API changed' instead of silently importing
-    nothing. Maps to API_DRIFT."""
-
-
-@dataclass
-class MediaItem:
-    """One resolved downloadable file belonging to a Pixiv work.
-
-    Fields mirror the other native clients' MediaItem so the downloader and
-    ledger are structurally the same. Pixiv original URLs carry no content
-    hash, so `filehash` is always None and the ledger keys on
-    `<post_id>:<media_id>` where media_id is `p<num>` (page) or `ugoira`
-    (the frame zip) — stable across URL-shape drift.
-    """
-
-    url: str
-    filename: str
-    kind: str
-    filehash: str | None
-    post_id: str
-    media_id: str
-
-
-def user_id_from_url(url: str) -> str | None:
-    """The numeric pixiv user id from a source URL, or None.
-
-    Handles the modern forms FC accepts as sources
-    (https://www.pixiv.net/users/<id>, /en/users/<id>) plus the legacy
-    member.php?id=<id>. This IS the campaign id — no network resolver.
-    """
-    parts = urlsplit(url or "")
-    if "pixiv.net" not in parts.netloc:
-        return None
-    segs = [s for s in parts.path.split("/") if s]
-    if segs and segs[0] == "en":
-        segs = segs[1:]
-    if len(segs) >= 2 and segs[0] == "users" and segs[1].isdigit():
-        return segs[1]
-    if segs and segs[0] == "member.php":
-        qid = dict(parse_qsl(parts.query)).get("id", "")
-        if qid.isdigit():
-            return qid
-    return None
-
-
-def _work_filename(work: dict, num: int, url: str) -> str:
-    """gallery-dl layout parity: `{id}_{title[:50]}_{num:>02}.{extension}`
-    (the downloader sanitizes the final segment)."""
-    title = work.get("title")
-    title50 = (title if isinstance(title, str) else "")[:_TITLE_MAX]
-    ext = safe_ext(urlsplit(url).path.rsplit("/", 1)[-1])
-    return f"{work.get('id')}_{title50}_{num:02d}{ext}"
-
-
-class PixivClient:
-    """Synchronous Pixiv app-API read client. Construct with the operator's
-    OAuth refresh token (the same token-type Credential the gallery-dl path
-    consumed as `extractor.pixiv.refresh-token`)."""
-
-    def __init__(
-        self,
-        refresh_token: str | None,
-        *,
-        request_sleep: float = 0.0,
-        max_retries: int = _MAX_429_RETRIES,
-        session: requests.Session | None = None,
-    ):
-        self.refresh_token = refresh_token
-        self._request_sleep = request_sleep or 0.0
-        self._max_retries = max_retries
-        # No cookies — the app API authenticates via the Bearer token _login
-        # installs. make_session still supplies the retry/UA plumbing; the
-        # extra_headers overwrite its browser UA with the app profile.
-        self._session = (
-            session if session is not None
-            else make_session(None, extra_headers=PIXIV_APP_HEADERS)
-        )
-        self._authed_user: dict = {}
-        # Monotonic deadline after which the access token must be refreshed;
-        # 0 forces a refresh on first use.
-        self._token_deadline = 0.0
-
-    # -- auth ----------------------------------------------------------------
-
-    def _login(self) -> None:
-        """Exchange the refresh token for a Bearer access token (gallery-dl's
-        `_login_impl`, including the X-Client-Time/X-Client-Hash pair the
-        endpoint validates). No-op while the current token is still fresh."""
-        if time.monotonic() < self._token_deadline:
-            return
-        if not self.refresh_token:
-            raise PixivAuthError(
-                "No Pixiv refresh token configured — add the OAuth refresh "
-                "token as the Pixiv credential (token type)."
-            )
-        # gallery-dl stamps naive-UTC with a literal +00:00 suffix.
-        now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S+00:00")
-        headers = {
-            "X-Client-Time": now,
-            "X-Client-Hash": hashlib.md5(
-                (now + _HASH_SECRET).encode()
-            ).hexdigest(),
-        }
-        data = {
-            "client_id": _CLIENT_ID,
-            "client_secret": _CLIENT_SECRET,
-            "grant_type": "refresh_token",
-            "refresh_token": self.refresh_token,
-            "get_secure_url": "1",
-        }
-        try:
-            resp = self._session.post(
-                _OAUTH_URL, data=data, headers=headers,
-                timeout=_TIMEOUT_SECONDS,
-            )
-        except requests.RequestException as exc:
-            raise PixivAPIError(f"Pixiv OAuth request failed: {exc}") from exc
-        if resp.status_code >= 400:
-            raise PixivAuthError(
-                "Pixiv rejected the refresh token (HTTP "
-                f"{resp.status_code}) — rotate the Pixiv credential.",
-                status_code=resp.status_code,
-            )
-        try:
-            payload = resp.json()["response"]
-            access = payload["access_token"]
-        except (ValueError, KeyError, TypeError) as exc:
-            raise PixivDriftError(
-                f"Pixiv OAuth response shape changed: {exc}"
-            ) from exc
-        self._authed_user = payload.get("user") or {}
-        self._session.headers["Authorization"] = f"Bearer {access}"
-        # expires_in is 3600 today; refresh 60s early so a long walk never
-        # rides an expiring token into a spurious 400.
-        expires_in = payload.get("expires_in")
-        lifetime = float(expires_in) if isinstance(expires_in, (int, float)) else 3600.0
-        self._token_deadline = time.monotonic() + max(60.0, lifetime - 60.0)
-
-    # -- request -------------------------------------------------------------
-
-    def _call(self, endpoint: str, params: dict) -> dict:
-        """Authenticated app-API GET → parsed JSON body, with the shared 429
-        backoff and the loud auth/drift/rate-limit mapping."""
-        self._login()
-        if self._request_sleep > 0:
-            time.sleep(self._request_sleep)
-        url = _API_ROOT + endpoint
-        attempt = 0
-        while True:
-            try:
-                resp = self._session.get(
-                    url, params=params, timeout=_TIMEOUT_SECONDS
-                )
-            except requests.RequestException as exc:
-                raise PixivAPIError(
-                    f"Pixiv request failed ({endpoint}): {exc}"
-                ) from exc
-            if resp.status_code == 429 and attempt < self._max_retries:
-                attempt += 1
-                delay = retry_after_seconds(resp, attempt)
-                log.warning(
-                    "Pixiv 429 (%s) — backing off %.1fs (retry %d/%d)",
-                    endpoint, delay, attempt, self._max_retries,
-                )
-                time.sleep(delay)
-                continue
-            break
-
-        try:
-            body = resp.json()
-        except ValueError as exc:
-            raise PixivDriftError(
-                f"Pixiv returned non-JSON for {endpoint} "
-                f"(HTTP {resp.status_code})"
-            ) from exc
-
-        error = body.get("error") if isinstance(body, dict) else None
-        message = ""
-        if isinstance(error, dict):
-            message = str(
-                error.get("user_message") or error.get("message") or ""
-            )
-        # Rate limiting first: the app API reports it as an error MESSAGE
-        # (often on HTTP 403), which must not be mistaken for an auth failure.
-        if resp.status_code == 429 or "rate limit" in message.lower():
-            raise PixivAPIError(
-                f"Pixiv rate limit hit ({endpoint}): {message or 'HTTP 429'}",
-                status_code=429,
-                retry_after=_RATE_LIMIT_RETRY_AFTER,
-            )
-        if resp.status_code in (400, 401, 403):
-            # Invalid/expired access token surfaces as 400 invalid_grant-style
-            # errors on the app API; 401/403 are straight auth rejections.
-            raise PixivAuthError(
-                f"Pixiv rejected the request ({endpoint}, HTTP "
-                f"{resp.status_code}): {message or 'auth rejected'} — "
-                "rotate the Pixiv refresh token.",
-                status_code=resp.status_code,
-            )
-        if resp.status_code >= 400:
-            raise PixivAPIError(
-                f"Pixiv API error ({endpoint}, HTTP {resp.status_code}): "
-                f"{message or 'unknown error'}",
-                status_code=resp.status_code,
-            )
-        if error:
-            # HTTP 200 carrying an error object — unexpected, but never
-            # silently treat it as data.
-            raise PixivAPIError(
-                f"Pixiv API error ({endpoint}): {message or error}",
-                status_code=resp.status_code,
-            )
-        return body
-
-    # -- normalization -------------------------------------------------------
-
-    @staticmethod
-    def _normalize(work: dict) -> dict:
-        """Wrap an app-API work in the `{"id", "attributes", ...}` post shape
-        the platform-agnostic core and shared helpers read. The raw work rides
-        along under `_work` for extract_media / the post record."""
-        title = work.get("title")
-        caption = work.get("caption")
-        wtype = work.get("type")
-        return {
-            "id": work.get("id"),
-            "attributes": {
-                "title": title if isinstance(title, str) else "",
-                "content": caption if isinstance(caption, str) else "",
-                "published_at": work.get("create_date"),
-                "post_type": wtype if isinstance(wtype, str) else "illust",
-            },
-            "_work": work,
-        }
-
-    # -- post-first seams ----------------------------------------------------
-
-    @staticmethod
-    def post_record_key(post: dict) -> tuple[str, str] | None:
-        """`(ledger_key, post_id)` gating post-record capture through the seen
-        ledger (`post:<id>` synthetic key), or None when the work has no id."""
-        pid = post.get("id")
-        pid = str(pid) if pid is not None else ""
-        if not pid:
-            return None
-        return (f"post:{pid}", pid)
-
-    @staticmethod
-    def post_meta(post: dict) -> dict:
-        attrs = post.get("attributes") or {}
-        return {"title": attrs.get("title") or None, "date": attrs.get("published_at")}
-
-    @staticmethod
-    def post_is_gated(post: dict) -> bool:
-        """True when this account cannot fetch the work's real files: pixiv
-        substitutes a `limit_*` placeholder for the original (sanity-level
-        filter / my-pixiv lock / deleted), or zeroes the author (deleted
-        account). Mirrors #874 semantics: gated content leaves NO trace — a
-        placeholder thumbnail and an empty stub would only pollute the
-        archive. (gallery-dl's PHPSESSID web-scrape fallback for these is out
-        of scope: FC holds no pixiv browser cookies — module docstring.)"""
-        work = post.get("_work") or {}
-        user = work.get("user") or {}
-        if not user.get("id"):
-            return True
-        if work.get("meta_pages"):
-            return False
-        single = work.get("meta_single_page") or {}
-        original = single.get("original_image_url")
-        return isinstance(original, str) and original.startswith(_LIMIT_URL)
-
-    # -- media ---------------------------------------------------------------
-
-    def extract_media(self, post: dict, included_index: dict) -> list[MediaItem]:
-        """Resolve a work's downloadable files (gallery-dl's `_extract_files`):
-        multi-page originals, the single-page original, or the ugoira frame
-        zip. `included_index` is unused (pixiv works are self-contained)."""
-        work = post.get("_work") or {}
-        pid = str(post.get("id") or "")
-        if not pid or self.post_is_gated(post):
-            return []
-
-        if work.get("type") == "ugoira":
-            return self._ugoira_media(work, pid)
-
-        meta_pages = work.get("meta_pages") or []
-        if meta_pages:
-            items = []
-            for num, page in enumerate(meta_pages):
-                urls = page.get("image_urls") or {}
-                url = urls.get("original")
-                if not isinstance(url, str) or not url:
-                    continue
-                items.append(
-                    MediaItem(
-                        url=url,
-                        filename=_work_filename(work, num, url),
-                        kind="image",
-                        filehash=None,
-                        post_id=pid,
-                        media_id=f"p{num}",
-                    )
-                )
-            return items
-
-        single = work.get("meta_single_page") or {}
-        url = single.get("original_image_url")
-        if not isinstance(url, str) or not url or url.startswith(_LIMIT_URL):
-            return []
-        return [
-            MediaItem(
-                url=url,
-                filename=_work_filename(work, 0, url),
-                kind="image",
-                filehash=None,
-                post_id=pid,
-                media_id="p0",
-            )
-        ]
-
-    def _ugoira_meta(self, work: dict, pid: str) -> dict | None:
-        """Fetch + memoize the ugoira metadata (frames + zip urls) for a work.
-
-        Idempotent and cached on the work dict, so the post record and the
-        media extraction share ONE `/v1/ugoira/metadata` call regardless of
-        which runs first (the core writes the post record BEFORE it extracts
-        media). Returns None — and caches the miss — on a non-auth failure
-        (matching gallery-dl's downgrade); auth failures stay loud."""
-        if "_ugoira_meta" in work:
-            return work["_ugoira_meta"]
-        try:
-            body = self._call("/v1/ugoira/metadata", {"illust_id": pid})
-            meta = body["ugoira_metadata"]
-        except PixivAuthError:
-            raise
-        except (PixivAPIError, KeyError, TypeError) as exc:
-            log.warning("Pixiv ugoira metadata failed for %s: %s", pid, exc)
-            work["_ugoira_meta"] = None
-            return None
-        work["_ugoira_meta"] = meta
-        # Frame delays: a future ugoira→video conversion needs the timings (the
-        # zip alone has none), so the post record captures them.
-        work["_ugoira_frames"] = meta.get("frames") or []
-        return meta
-
-    def fetch_ugoira_frames(self, post: dict) -> None:
-        """Populate `post['_work']['_ugoira_frames']` for an ugoira post (no-op
-        otherwise). The core writes the post record BEFORE extract_media, so
-        without this the frame timings would never reach the record; this
-        fetches (and memoizes, so extract_media reuses it) the metadata. Injected
-        into the downloader by the ingester, mirroring Patreon's content_fetcher.
-        Auth errors propagate; other failures leave frames unset."""
-        work = post.get("_work") or {}
-        if work.get("type") != "ugoira":
-            return
-        pid = str(post.get("id") or "")
-        if pid:
-            self._ugoira_meta(work, pid)
-
-    def _ugoira_media(self, work: dict, pid: str) -> list[MediaItem]:
-        """The ugoira frame zip (gallery-dl's default non-original mode):
-        `/v1/ugoira/metadata` → zip_urls.medium with the 600x600→1920x1080
-        swap. A metadata failure downgrades to 'no media' with a warning
-        (matching gallery-dl) instead of failing the walk — except auth
-        failures, which stay loud."""
-        meta = self._ugoira_meta(work, pid)
-        if meta is None:
-            return []
-        try:
-            zip_url = meta["zip_urls"]["medium"]
-        except (KeyError, TypeError) as exc:
-            log.warning("Pixiv ugoira zip url missing for %s: %s", pid, exc)
-            return []
-        url = zip_url.replace("_ugoira600x600", "_ugoira1920x1080", 1)
-        return [
-            MediaItem(
-                url=url,
-                filename=_work_filename(work, 0, url),
-                kind="ugoira",
-                filehash=None,
-                post_id=pid,
-                media_id="ugoira",
-            )
-        ]
-
-    # -- iteration -----------------------------------------------------------
-
-    def iter_posts(
-        self, campaign_id: str, cursor: str | None = None
-    ) -> Iterator[tuple[dict, dict, str | None]]:
-        """Yield (post, {}, page_cursor) for every work in the user's feed.
-
-        `campaign_id` is the numeric pixiv user id. `cursor` is the query
-        string of the app API's `next_url` (offset pagination); None fetches
-        page 1. The yielded `page_cursor` is the cursor that FETCHED this
-        work's page, so the core checkpoints a value that re-serves the same
-        page on resume (the shared cursor contract)."""
-        if not str(campaign_id or "").isdigit():
-            raise PixivDriftError(
-                f"Pixiv campaign id must be a numeric user id, got "
-                f"{campaign_id!r}"
-            )
-        current = cursor
-        while True:
-            page_cursor = current
-            if current is None:
-                params: dict = {"user_id": campaign_id}
-            else:
-                params = dict(parse_qsl(current))
-            data = self._call("/v1/user/illusts", params)
-            works = data.get("illusts")
-            if not isinstance(works, list):
-                raise PixivDriftError(
-                    "Pixiv user-illusts response had no 'illusts' list "
-                    f"(keys: {sorted(data)[:8]})"
-                )
-            for work in works:
-                if not isinstance(work, dict):
-                    continue
-                yield self._normalize(work), {}, page_cursor
-            next_url = data.get("next_url")
-            if not next_url:
-                return
-            current = str(next_url).rpartition("?")[2]
-
-    # -- user detail ---------------------------------------------------------
-
-    def resolve_display_name(self, user_id: str) -> str | None:
-        """The pixiv user's display name via `/v1/user/detail` (gallery-dl's
-        user_detail) — used to name the Artist when a source is added by numeric
-        id. None on any failure (the caller falls back to the id)."""
-        try:
-            body = self._call("/v1/user/detail", {"user_id": str(user_id)})
-        except PixivAPIError:
-            return None
-        name = (body.get("user") or {}).get("name") if isinstance(body, dict) else None
-        return name if isinstance(name, str) and name.strip() else None
-
-    # -- verify --------------------------------------------------------------
-
-    def verify_auth(self) -> tuple[bool | None, str]:
-        """Cheap credential probe: run the OAuth refresh (the thing that fails
-        when the token is bad) without walking any feed."""
-        try:
-            self._token_deadline = 0.0  # force a real refresh
-            self._login()
-        except PixivAuthError as exc:
-            return False, f"Pixiv rejected the credential — {exc}"
-        except PixivAPIError as exc:
-            return None, f"Couldn't verify (network/HTTP issue): {exc}"
-        account = self._authed_user.get("account") or self._authed_user.get("name")
-        suffix = f" as {account}" if account else ""
-        return True, f"Credentials valid — Pixiv OAuth refresh succeeded{suffix}."
-
-
-def rating_label(x_restrict) -> str | None:
-    """Human rating from pixiv's x_restrict (0/1/2) — written into the post
-    record so the archive keeps the R-18 flag without the reader needing to
-    know pixiv's numeric scheme."""
-    if isinstance(x_restrict, bool) or not isinstance(x_restrict, int):
-        return None
-    return _RATINGS.get(x_restrict)

backend/app/services/pixiv_downloader.py

@@ -1,276 +0,0 @@
-"""Native Pixiv media downloader — the Pixiv counterpart to
-patreon_downloader / subscribestar_downloader.
-
-Given a normalized Pixiv work and its resolved `MediaItem`s
-(pixiv_client.extract_media), download the originals to gallery-dl's on-disk
-layout (so pre-cutover gallery-dl downloads are recognized on disk and not
-re-fetched), write the post-first sidecars the importer consumes, and report
-per-media outcomes.
-
-On-disk layout (matches FC's gallery-dl pixiv config, PLATFORM_DEFAULTS:
-base-directory `<images_root>/<artist_slug>/pixiv` + `directory:
-["{category}"]` + filename `{id}_{title[:50]}_{num:>02}.{extension}`):
-
-    <images_root>/<artist_slug>/pixiv/pixiv/<id>_<title50>_<NN>.<ext>
-
-— note the intentional DOUBLE `pixiv` segment: gallery-dl appended
-`{category}` under a base-directory that already ended in the platform name,
-and tier-2 disk-skip parity requires reproducing that exactly. The layout is
-FLAT (no per-post directory), so the post-first record is `_post_<id>.json`
-in the same directory (the id suffix prevents the collisions a bare
-`_post.json` would have here; phase 3 receives explicit post_record_paths, so
-the name is a convention, not a discovery key).
-
-Simpler than Patreon (no Mux/yt-dlp video branch) — the one special file is
-the ugoira frame zip, downloaded as-is; FC's archive-containment import
-extracts the frames, and the frame DELAYS ride the post record (the zip
-carries none — a future ugoira→video conversion needs them).
-
-PURE: no DB; the seen-skip is an injected predicate. FC runs on a plain-HTTP
-homelab; nothing here uses a secure-context Web API.
-"""
-
-from __future__ import annotations
-
-import json
-import logging
-import re
-import time
-from collections.abc import Callable
-from pathlib import Path
-
-import requests
-
-from .native_ingest_common import (
-    BaseNativeDownloader,
-    MediaOutcome,
-    PostRecordOutcome,
-)
-from .pixiv_client import PIXIV_APP_HEADERS, rating_label
-
-log = logging.getLogger(__name__)
-
-# Control chars (0x00–0x1f + 0x7f DEL) — gallery-dl's default `path-remove`.
-_GDL_PATH_REMOVE_RE = re.compile(r"[\x00-\x1f\x7f]")
-
-
-def gdl_clean_filename(name: str) -> str:
-    """Reproduce gallery-dl's on-disk filename EXACTLY as it wrote it on this
-    Linux host, so the tier-2 disk-skip recognizes pre-cutover files instead of
-    re-downloading them.
-
-    gallery-dl's PathFormat.build_filename is `clean_path(clean_segment(name))`.
-    On Linux (verified against gallery-dl 1.32.5 path.py) the defaults resolve to:
-      - path-restrict "auto" → "/"          → clean_segment replaces ONLY "/" → "_"
-      - path-remove   "\\x00-\\x1f\\x7f"       → clean_path DELETES control chars
-      - path-strip    "auto" → ""           → NO trailing dot/space stripping
-    Crucially it does NOT touch the Windows-forbidden set (<>:"|?*) — those stay
-    raw in titles on disk. A stricter sanitizer here would rename any such title,
-    miss the on-disk match, and re-pull the whole work. Order mirrors gallery-dl
-    (segment inner, path outer); for these disjoint char sets it's commutative.
-    """
-    return _GDL_PATH_REMOVE_RE.sub("", name.replace("/", "_"))
-
-# Enrichment keys copied verbatim from the app-API work dict into the post
-# record (they're already JSON scalars/objects). Everything lands in
-# Post.raw_metadata via the importer, so the archive keeps pixiv's stats and
-# structure without a schema change.
-_WORK_PASSTHROUGH_KEYS = (
-    "type",
-    "page_count",
-    "width",
-    "height",
-    "total_view",
-    "total_bookmarks",
-    "total_comments",
-    "is_bookmarked",
-    "illust_ai_type",
-    "series",
-)
-
-
-class PixivDownloader(BaseNativeDownloader):
-    """Download resolved Pixiv media to gallery-dl's on-disk layout.
-    Subclasses BaseNativeDownloader for the shared streaming GET
-    (transient-retry + Range-resume) and validation/quarantine. PURE: no DB."""
-
-    def __init__(
-        self,
-        images_root: Path,
-        cookies_path: str | None = None,
-        *,
-        validate: bool = True,
-        rate_limit: float = 0.0,
-        session: requests.Session | None = None,
-        ugoira_frames_fetcher: Callable[[dict], None] | None = None,
-    ):
-        super().__init__(
-            images_root, cookies_path, platform="pixiv",
-            validate=validate, rate_limit=rate_limit, session=session,
-        )
-        # Injected by the ingester (client.fetch_ugoira_frames) so write_post_record
-        # can populate frame timings — which extract_media memoizes, but the core
-        # writes the post record FIRST. Mirrors Patreon's content_fetcher.
-        self._ugoira_frames_fetcher = ugoira_frames_fetcher
-        if session is None:
-            # i.pximg.net 403s any GET without the app Referer; mirror the
-            # client's full app-header profile (gallery-dl serves media off
-            # the same session it drives the API with). An injected session
-            # (tests) owns its own headers.
-            self.session.headers.update(PIXIV_APP_HEADERS)
-
-    # -- public ------------------------------------------------------------
-
-    def download_post(
-        self,
-        post: dict,
-        media_items: list,
-        artist_slug: str,
-        *,
-        is_seen: Callable[[object], bool] = lambda m: False,
-        should_stop: Callable[[], bool] = lambda: False,
-        recapture: bool = False,
-    ) -> list[MediaOutcome]:
-        """Download every media item of one work; return per-item outcomes.
-        Mirrors SubscribeStarDownloader.download_post (two-tier skip, mid-post
-        time-box, recapture surfacing)."""
-        flat_dir = self._flat_dir(artist_slug)
-        outcomes: list[MediaOutcome] = []
-        for media in media_items:
-            if should_stop():
-                break
-            try:
-                outcomes.append(
-                    self._download_one(
-                        post, media, flat_dir, artist_slug, is_seen,
-                        recapture=recapture,
-                    )
-                )
-            except Exception as exc:  # resilient: isolate one item's failure
-                log.warning(
-                    "Pixiv media failed (work %s, %s): %s",
-                    post.get("id"), getattr(media, "media_id", "?"), exc,
-                )
-                outcomes.append(
-                    MediaOutcome(media=media, status="error", path=None, error=str(exc))
-                )
-        return outcomes
-
-    def _flat_dir(self, artist_slug: str) -> Path:
-        # Double platform segment — gallery-dl layout parity (module docstring).
-        return self.images_root / artist_slug / "pixiv" / "pixiv"
-
-    # -- per-item ----------------------------------------------------------
-
-    def _download_one(
-        self,
-        post: dict,
-        media,
-        flat_dir: Path,
-        artist_slug: str,
-        is_seen: Callable[[object], bool],
-        *,
-        recapture: bool = False,
-    ) -> MediaOutcome:
-        seen = is_seen(media)
-        if seen and not recapture:
-            return MediaOutcome(media=media, status="skipped_seen", path=None, error=None)
-
-        # The client's filename already carries the {id}_{title50}_{NN} shape
-        # (raw title, gallery-dl-template order); clean it to the byte-exact
-        # name gallery-dl wrote on disk so tier-2 disk-skip matches (else a
-        # re-download of the whole work). See gdl_clean_filename.
-        media_path = flat_dir / gdl_clean_filename(media.filename)
-
-        if media_path.exists():  # tier-2: already on disk
-            return MediaOutcome(
-                media=media, status="skipped_disk", path=media_path, error=None
-            )
-        # recapture: a seen item not on disk is NOT re-downloaded (recovery's job).
-        if seen:
-            return MediaOutcome(media=media, status="skipped_seen", path=None, error=None)
-
-        flat_dir.mkdir(parents=True, exist_ok=True)
-        if self._rate_limit > 0:
-            time.sleep(self._rate_limit)
-
-        out_path = self._fetch_get(media.url, media_path)
-        reason, quarantine_dest = self._validate_path(out_path, artist_slug, media.url)
-        if reason is not None:
-            return MediaOutcome(
-                media=media, status="quarantined", path=quarantine_dest, error=reason,
-            )
-        self._write_minimal_sidecar(post, out_path, source_url=media.url)
-        return MediaOutcome(media=media, status="downloaded", path=out_path, error=None)
-
-    # -- post record ---------------------------------------------------------
-
-    def write_post_record(self, post: dict, artist_slug: str) -> PostRecordOutcome:
-        """Write the post-first `_post_<id>.json` — the sole writer of the post
-        body/metadata on the native path. Beyond the standard body fields, the
-        record carries pixiv's own structure (tags + EN translations, rating,
-        series, view/bookmark counts, AI flag, dimensions, author, ugoira frame
-        delays) so the archive keeps what the platform knows about the work."""
-        attrs = post.get("attributes") or {}
-        work = post.get("_work") or {}
-        title = attrs.get("title") if isinstance(attrs.get("title"), str) else None
-        post_type = attrs.get("post_type") if isinstance(attrs.get("post_type"), str) else None
-        pid = str(post.get("id") or "")
-        if not pid:
-            return PostRecordOutcome(
-                path=None, post_type=post_type, title=title, body_chars=0,
-            )
-
-        content = attrs.get("content")
-        content = content if isinstance(content, str) else ""
-        data: dict = {
-            "category": "pixiv",
-            "id": pid,
-            "title": title or "",
-            "content": content,
-            "published_at": attrs.get("published_at"),
-            # The post permalink is synthesized by platforms/pixiv.py
-            # derive_post_url from `id` at parse time — no url key here.
-            "rating": rating_label(work.get("x_restrict")),
-        }
-        for key in _WORK_PASSTHROUGH_KEYS:
-            if key in work:
-                data[key] = work[key]
-        tags = work.get("tags")
-        if isinstance(tags, list):
-            data["tags"] = [
-                {
-                    "name": t.get("name"),
-                    "translated_name": t.get("translated_name"),
-                }
-                for t in tags
-                if isinstance(t, dict)
-            ]
-        user = work.get("user")
-        if isinstance(user, dict):
-            data["user"] = {
-                "id": user.get("id"),
-                "account": user.get("account"),
-                "name": user.get("name"),
-            }
-        # Ugoira frame timings. extract_media memoizes these, but the core writes
-        # the post record BEFORE extracting media, so fetch them here (shared +
-        # idempotent via the client's memoization) so the record actually keeps
-        # them — the zip carries no timings.
-        if (
-            work.get("type") == "ugoira"
-            and not work.get("_ugoira_frames")
-            and self._ugoira_frames_fetcher is not None
-        ):
-            self._ugoira_frames_fetcher(post)
-        frames = work.get("_ugoira_frames")
-        if frames:
-            data["ugoira_frames"] = frames
-
-        flat_dir = self._flat_dir(artist_slug)
-        flat_dir.mkdir(parents=True, exist_ok=True)
-        path = flat_dir / f"_post_{pid}.json"
-        path.write_text(json.dumps(data, indent=2, ensure_ascii=False))
-        return PostRecordOutcome(
-            path=path, post_type=post_type, title=title, body_chars=len(content),
-        )

backend/app/services/pixiv_ingester.py

@@ -1,121 +0,0 @@
-"""Native Pixiv ingester — the Pixiv ADAPTER over the platform-agnostic core
-(`ingest_core.Ingester`).
-
-Thin counterpart to patreon_ingester / subscribestar_ingester: wires the Pixiv
-client/downloader/ledger models/constraints/key into the core and supplies the
-Pixiv failure mapping. The modes (tick / backfill / recovery / recapture), the
-seen + dead-letter ledgers, cursor checkpointing, and the post-first capture
-all live in the core. `download_service.download_source` drives
-`PixivIngester.run` exactly as it drives the other two.
-
-`campaign_id` is the numeric pixiv user id (download_backends extracts it from
-the source URL — no network resolver). Auth is the operator's OAuth refresh
-token (the token-type Credential), passed as `auth_token` — pixiv is the first
-native platform authenticating by token rather than cookies, so the uniform
-constructor accepts both and ignores what it doesn't need.
-
-FC runs on a plain-HTTP homelab; nothing here uses a secure-context Web API.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import logging
-from collections.abc import Callable
-from pathlib import Path
-
-from ..models import PixivFailedMedia, PixivSeenMedia
-from .ingest_core import DEAD_LETTER_THRESHOLD, Ingester
-from .pixiv_client import MediaItem, PixivAPIError, PixivClient
-from .pixiv_downloader import PixivDownloader
-
-__all__ = [
-    "DEAD_LETTER_THRESHOLD",
-    "PixivIngester",
-    "_ledger_key",
-    "verify_pixiv_credential",
-]
-
-log = logging.getLogger(__name__)
-
-_LEDGER_KEY_MAX = 128
-
-
-def _ledger_key(media: MediaItem) -> str:
-    """Stable per-media identity for the cross-run seen-ledger. Pixiv original
-    URLs carry no content hash, so the key is the page/zip identity scoped to
-    its work: `<illust_id>:p<num>` / `<illust_id>:ugoira`. Bounded to the
-    column width."""
-    if media.filehash:
-        return media.filehash
-    return f"{media.post_id}:{media.media_id}"[:_LEDGER_KEY_MAX]
-
-
-class PixivIngester(Ingester):
-    """Walk a pixiv user's works, download unseen originals, return a
-    `DownloadResult`. A thin adapter over `ingest_core.Ingester`; `client` /
-    `downloader` are injectable seams so unit tests run without network."""
-
-    def __init__(
-        self,
-        images_root: Path,
-        cookies_path: str | None,
-        session_factory: Callable[[], object],
-        *,
-        validate: bool = True,
-        rate_limit: float = 0.0,
-        request_sleep: float = 0.0,
-        auth_token: str | None = None,
-        client: PixivClient | None = None,
-        downloader: PixivDownloader | None = None,
-    ):
-        self.images_root = Path(images_root)
-        self.cookies_path = str(cookies_path) if cookies_path else None
-        resolved_client = (
-            client
-            if client is not None
-            else PixivClient(auth_token, request_sleep=request_sleep)
-        )
-        resolved_downloader = (
-            downloader
-            if downloader is not None
-            else PixivDownloader(
-                self.images_root, cookies_path, validate=validate, rate_limit=rate_limit,
-                # write_post_record runs before extract_media in the core, so it
-                # fetches ugoira frame timings via the SAME client (shared,
-                # memoized) — else the record's ugoira_frames stays empty.
-                ugoira_frames_fetcher=resolved_client.fetch_ugoira_frames,
-            )
-        )
-        super().__init__(
-            client=resolved_client,
-            downloader=resolved_downloader,
-            session_factory=session_factory,
-            seen_model=PixivSeenMedia,
-            failed_model=PixivFailedMedia,
-            seen_constraint="uq_pixiv_seen_media_source_id",
-            failed_constraint="uq_pixiv_failed_media_source_id",
-            ledger_key=_ledger_key,
-            platform="pixiv",
-            error_base=PixivAPIError,
-            # API_DRIFT message phrasing; the base Ingester._failure_result owns
-            # the auth/drift/HTTP→error_type mapping (shared across platforms).
-            drift_label="Pixiv app API",
-            # Captions are legitimately empty for many pixiv artists, so the
-            # zero-bodies #862 canary would false-positive here; the client's
-            # response-shape checks (missing `illusts` → drift) cover the same
-            # failure class structurally.
-            body_canary=False,
-        )
-
-
-async def verify_pixiv_credential(
-    auth_token: str | None,
-) -> tuple[bool | None, str]:
-    """Native Pixiv credential probe — one OAuth refresh via
-    PixivClient.verify_auth (the exact call that fails when the token is
-    bad; no feed walk). Returns the uniform `(ok, message)` contract so
-    download_backends.verify_source_credential treats it like the others."""
-    client = PixivClient(auth_token)
-    loop = asyncio.get_running_loop()
-    return await loop.run_in_executor(None, client.verify_auth)

backend/app/services/platforms/__init__.py

@@ -13,9 +13,8 @@ URL patterns match GS exactly so the existing browser extension
 hits FC unmodified. deviantart was dropped at #3069 (2026-08-27) —
 FC downloaders are art-dedicated services only. pixiv was retired at
 milestone #406 (2026-09-13, rule #171): unregistered here first, which
-switches it off everywhere this registry is consulted; `pixiv.py` and the
-pixiv client/downloader/ingester stay in the tree, uncalled, until the
-milestone's phase 2 deletes them.
+switched it off everywhere this registry is consulted, then removed from
+the tree entirely in the milestone's phase 2 (2026-09-21).
 """
 
 from .base import (

backend/app/services/platforms/base.py

@@ -24,8 +24,8 @@ from typing import Literal
 # external_post_id chain: `post_id` MUST come before `id` because
 # SubscribeStar gallery-dl puts the per-attachment id in `id` and the
 # actual post id in `post_id`; picking `id` first fragments
-# multi-image SubscribeStar posts into N Post rows. Patreon/Pixiv have
-# no `post_id` so `id` still wins for them; HF uses `index`, Discord
+# multi-image SubscribeStar posts into N Post rows. Patreon has
+# no `post_id` so `id` still wins for it; HF uses `index`, Discord
 # uses `message_id` — all reached via the remaining chain entries.
 # (Banked 2026-05-27 during the sidecar audit.)
 DEFAULT_EXTERNAL_POST_ID_KEYS: tuple[str, ...] = (
@@ -62,7 +62,7 @@ class PlatformInfo:
     # --- Behavioral hooks ---
     # Synthesize a post permalink from sidecar data. Required when
     # gallery-dl's `url` field is the file/CDN URL rather than the post
-    # permalink (subscribestar/pixiv/hf/discord). None = trust the bare
+    # permalink (subscribestar/hf/discord). None = trust the bare
     # `url` field (patreon).
     derive_post_url: Callable[[dict], str | None] | None = None
 

backend/app/services/platforms/pixiv.py

@@ -1,38 +0,0 @@
-"""Pixiv — one quirk.
-
-post_url: the sidecar's `url` (legacy gallery-dl era) is the image URL
-on `i.pximg.net`, and the native post record (#129) writes no url key
-at all — the permalink is synthesized from `id` here either way:
-/artworks/<id>. external_post_id (= `id`) was already correct, so no
-override there.
-
-Downloads run through the native ingester (pixiv_ingester.py), not
-gallery-dl; this registry entry still owns URL validation, sidecar
-parsing, and the credential surface (the OAuth refresh token).
-"""
-
-from .base import GD_DEFAULTS, PlatformInfo, str_id_value
-
-
-def derive_post_url(data: dict) -> str | None:
-    pid = str_id_value(data.get("id"))
-    if pid:
-        return f"https://www.pixiv.net/artworks/{pid}"
-    return None
-
-
-INFO = PlatformInfo(
-    key="pixiv",
-    name="Pixiv",
-    description="Download artwork from Pixiv artists",
-    auth_type="token",
-    requires_auth=True,
-    url_pattern=r"^https?://(www\.)?pixiv\.net/",
-    url_examples=[
-        "https://www.pixiv.net/users/12345678",
-        "https://www.pixiv.net/en/users/12345678",
-    ],
-    default_config={**GD_DEFAULTS, "content_types": ["all"]},
-    notes="Requires OAuth refresh token. Run `gallery-dl oauth:pixiv` to obtain one.",
-    derive_post_url=derive_post_url,
-)

backend/app/services/scheduler_service.py

@@ -229,7 +229,7 @@ async def scheduler_status(session: AsyncSession) -> dict:
     # links to cannot disagree about what they are counting.
     failing_sources = (await session.execute(
         select(func.count()).select_from(Source)
-        .where(Source.enabled.is_(True), failing_sources_clause())
+        .where(failing_sources_clause())
     )).scalar_one()
     no_access_sources = (await session.execute(
         select(func.count()).select_from(Source)

backend/app/services/service_roster.py

@@ -36,6 +36,7 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from ..models import ServiceSeen
+from .worker_lanes import LANES
 
 log = logging.getLogger(__name__)
 
@@ -50,16 +51,44 @@ REFRESH_TTL_SECONDS = 20.0
 # page that exists to explain it.
 INSPECT_TIMEOUT_SECONDS = 2.0
 
+# How many broadcast round trips `_inspect_celery_sync` makes. Named, because
+# the wrapper's budget is derived from it and the two must not drift.
+#
+# `active_queues()` and `active()` are separate broadcasts, and a broadcast
+# with no `destination` cannot know how many replies to expect — so each one
+# waits out its full timeout rather than returning on the last reply. The sync
+# call therefore costs ~2 x INSPECT_TIMEOUT_SECONDS in the ordinary case, not
+# once.
+INSPECT_ROUND_TRIPS = 2
+
+# Slack for the thread handoff. `asyncio.to_thread` hands work to the default
+# executor, and on a loaded web process — the operator's showcase page pulling
+# ninety thumbnails a second — the thread may not even be scheduled inside the
+# budget, let alone finish.
+#
+# This exists because the wrapper used to allow `INSPECT_TIMEOUT_SECONDS * 2`,
+# which LOOKS like a safety factor and is exactly the worst case with nothing
+# left over. Observed on the operator's first consolidated deploy, 2026-09-23:
+# a TimeoutError traceback per refresh while the two inspect calls were
+# working perfectly. A budget equal to the work is a budget that fails under
+# any load at all.
+INSPECT_SLACK_SECONDS = 3.0
+
 # Queue set -> the name an operator recognises. Sorted-tuple keys, because the
 # order celery reports them in is not guaranteed.
 #
-# A deployment that slices CELERY_QUEUES differently falls through to the raw
-# queue list rather than being given a name this table invented for it: a
+# DERIVED from `worker_lanes.LANES` (milestone 422 step 1) rather than written
+# out here. It was a hand-kept second copy of the same fact, and it had already
+# drifted: `maintenance_long` is a live lane with four task routes pointing at
+# it and a dedicated worker in the operator's stack, and this map did not know
+# it — so the System tab labelled it `Worker (maintenance_long)`. One list of
+# lanes now names them everywhere.
+#
+# A deployment that slices CELERY_QUEUES differently still falls through to the
+# raw queue list rather than being given a name this code invented for it: a
 # wrong-but-confident label on a status page is worse than an ugly true one.
 ROLE_NAMES: dict[tuple[str, ...], str] = {
-    ("default", "download", "import", "thumbnail"): "Worker",
-    ("maintenance", "scan"): "Scheduler",
-    ("ml",): "ML worker",
+    lane.queue_key: lane.display_name for lane in LANES
 }
 
 
@@ -80,6 +109,10 @@ def _inspect_celery_sync() -> dict[tuple[str, ...], dict]:
     from ..celery_app import celery as celery_app
 
     insp = celery_app.control.inspect(timeout=INSPECT_TIMEOUT_SECONDS)
+    # TWO broadcasts, each waiting out its own timeout — see
+    # INSPECT_ROUND_TRIPS, which the caller's budget is derived from. Adding a
+    # third call here without updating that constant puts the wrapper back
+    # under the work it is waiting for.
     active_queues = insp.active_queues() or {}
     active_tasks = insp.active() or {}
 
@@ -131,7 +164,10 @@ async def refresh_celery_roster(session: AsyncSession) -> None:
     try:
         grouped = await asyncio.wait_for(
             asyncio.to_thread(_inspect_celery_sync),
-            timeout=INSPECT_TIMEOUT_SECONDS * 2,
+            timeout=(
+                INSPECT_TIMEOUT_SECONDS * INSPECT_ROUND_TRIPS
+                + INSPECT_SLACK_SECONDS
+            ),
         )
     except Exception:
         log.warning("service roster: celery inspect failed; roster not refreshed", exc_info=True)

backend/app/services/source_service.py

@@ -19,6 +19,7 @@ from ..models import (
 )
 from .db_helpers import failing_sources_clause
 from .gallery_dl import ErrorType
+from .membership_reconcile import KEPT_KEY, STOPPED_KEY
 from .membership_roster import gated_reasons_for_sources
 from .platforms import known_platform_keys
 from .scheduler_service import compute_next_check_at
@@ -158,6 +159,38 @@ def _is_app_managed(key: str) -> bool:
 BACKFILL_MAX_CHUNKS = 200
 
 
+def arm_backfill(source: Source) -> None:
+    """Arm a fresh run-until-done backfill on `source` (plan #693). Mutation
+    only — the caller commits. Shared by `SourceService.start_backfill` and the
+    sync Discord repair, so both clear exactly the same resume state."""
+    co = dict(source.config_overrides or {})
+    co["_backfill_state"] = "running"
+    for k in ("_backfill_cursor", "_backfill_cursor_stalls", "_backfill_chunks",
+              "_backfill_posts"):
+        co.pop(k, None)
+    source.config_overrides = co
+    source.backfill_runs_remaining = BACKFILL_MAX_CHUNKS
+
+
+def _record_manual_enable_choice(source: Source, *, enabled: bool) -> None:
+    """Keep the membership sweep (#3995) from overriding the operator.
+
+    Turning a source the sweep STOPPED back on is a deliberate choice to keep
+    pulling a lapsed creator, so it is marked kept and the next sweep leaves it
+    alone. Turning a source off by hand drops any sweep marker, so the sweep
+    never switches back on something the operator switched off themselves.
+    """
+    co = dict(source.config_overrides or {})
+    if enabled and STOPPED_KEY in co:
+        co.pop(STOPPED_KEY)
+        co[KEPT_KEY] = True
+    elif not enabled:
+        co.pop(STOPPED_KEY, None)
+    else:
+        return
+    source.config_overrides = co
+
+
 class SourceService:
     def __init__(self, session: AsyncSession):
         self.session = session
@@ -415,6 +448,9 @@ class SourceService:
         for key, value in fields.items():
             setattr(source, key, value)
 
+        if "enabled" in fields:
+            _record_manual_enable_choice(source, enabled=bool(fields["enabled"]))
+
         if url_changed:
             # Repointing a source at a different creator makes a cached campaign
             # id WRONG, not merely stale, and `patreon_resolver` consults that
@@ -473,13 +509,7 @@ class SourceService:
         )).scalar_one_or_none()
         if source is None:
             raise LookupError(f"source id={source_id} not found")
-        co = dict(source.config_overrides or {})
-        co["_backfill_state"] = "running"
-        for k in ("_backfill_cursor", "_backfill_cursor_stalls", "_backfill_chunks",
-                  "_backfill_posts"):
-            co.pop(k, None)
-        source.config_overrides = co
-        source.backfill_runs_remaining = BACKFILL_MAX_CHUNKS
+        arm_backfill(source)
         await self.session.commit()
         return await self._row_to_record(source)
 

backend/app/services/subscribestar_client.py

@@ -316,7 +316,7 @@ _ROSTER_URL = f"{_ROSTER_BASE}/subscriptions"
 # `data-identifier`, the one vocabulary that names a state: the table class
 # inside the cancelled card says `for-unsubscribed_users`, a different word for
 # the same list (note #3989, CORRECTION 1). The identifier is stored verbatim as
-# Membership.status and mapped in membership_roster.MEMBERSHIP_STATUS.
+# Membership.status and mapped in native_ingest_common.MEMBERSHIP_STATUS.
 _ROSTER_ACTIVE = "active_subscriptions"
 _ROSTER_CANCELLED = "cancelled_subscriptions"
 

backend/app/services/worker_control.py

@@ -0,0 +1,812 @@
+"""Read and change a lane's live pool, over the broker.
+
+Milestone 422 step 2. The half of the milestone that does something.
+
+## No docker socket is involved, and that is the point
+
+Milestone 365 put "acting on the state" out of scope because restarting a
+dead worker needs a docker socket the web container deliberately does not
+have. That is true of RESTARTING a container. It is not true of changing how
+much work a RUNNING worker does: celery's remote control sends a message over
+the broker and the worker resizes its own pool. Same Redis the app already
+uses, no new privilege, no new surface.
+
+    pool_grow / pool_shrink   how many slots a lane runs
+    add_consumer / cancel_consumer   whether it consumes its queues at all
+
+The operator ruled the socket out independently (2026-09-22: *"this feature
+is a very invasive idea in my mind and I'd like to avoid it"*), and nothing
+here raises the question.
+
+## The setting is PER PROCESS, not per lane total
+
+`pool_grow(n, destination=[...])` adds n slots to EACH destination it names.
+While the stack still runs several containers per lane — the operator's
+production `worker` is `replicas: 2` — a single delta applied to a lane's
+total would be wrong for every replica.
+
+So `slots` means what `CELERY_CONCURRENCY` means: the pool size of one
+process. The reconcile below drives EACH replica to that number
+independently, computing its own delta from that replica's current pool, so
+replicas that have drifted apart (one restarted, one was grown) converge
+rather than being moved in lockstep from a shared baseline.
+
+After step 5 there is one process per lane and the distinction disappears.
+It matters now, and getting it wrong now would be invisible — the totals
+would simply be double what the UI claimed.
+
+## Why reserved() is read alongside the queue depth
+
+Celery PREFETCHES: a worker pulls more messages than it can run and holds
+them in memory. Those have already left the Redis list, so `LLEN` — which is
+what `/api/system/activity/queues` reports — can read 0 while thirty tasks
+are waiting inside a worker. Any judgement about backlog that uses only LLEN
+under-reports, which matters for the UI and is disqualifying for step 7's
+autoscaler.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+from dataclasses import dataclass, field
+from datetime import UTC, datetime
+
+from sqlalchemy import func, select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from ..models import TaskRun, WorkerLane
+from .worker_lanes import (
+    LANES,
+    LANES_BY_QUEUE_KEY,
+    MIN_POOL_SLOTS,
+    Lane,
+    derived_ceiling,
+)
+
+log = logging.getLogger(__name__)
+
+# celery control is a broker round trip on a request path, so it gets a
+# deadline (rule 156) — the same reasoning and the same budget as
+# service_roster's inspect. A broker that stopped answering must make this
+# report "not present", which is true, rather than hang the page.
+CONTROL_TIMEOUT_SECONDS = 2.0
+
+
+# The WORST case of `inspect_lanes_sync`, for callers that need a deadline.
+#
+# One broadcast plus three targeted reads. The targeted three normally return
+# as soon as the named nodes answer; each can still cost a full timeout if a
+# node disappears mid-read, so the bound stays four.
+CONTROL_ROUND_TRIPS = 4
+
+# Slack for the `asyncio.to_thread` handoff. A budget equal to the work is a
+# budget that fails under load — the roster carried exactly that bug into the
+# operator's first consolidated deploy and logged a TimeoutError per refresh
+# while the inspect calls underneath were working fine.
+CONTROL_SLACK_SECONDS = 3.0
+
+INSPECT_BUDGET_SECONDS = (
+    CONTROL_TIMEOUT_SECONDS * CONTROL_ROUND_TRIPS + CONTROL_SLACK_SECONDS
+)
+
+
+@dataclass
+class LaneLiveState:
+    """What `celery inspect` says about one lane right now.
+
+    `present=False` is NOT "zero slots" — it is "nothing answered". A lane
+    whose worker is restarting, or whose broker is unreachable, must read as
+    unknown rather than as stopped: an unswept absence is not a verdict
+    (snippet #3969). The reconcile in step 3 skips an absent lane rather than
+    correcting it, which is only safe because this distinction is kept.
+    """
+
+    present: bool = False
+    replicas: int = 0
+    active: int = 0
+    reserved: int = 0
+    hostnames: list[str] = field(default_factory=list)
+    # The queues this lane is actually consuming right now, across replicas.
+    # Distinct from the lane's CONFIGURED queues: `cancel_consumer` stops a
+    # worker consuming one without changing what it was started with, which
+    # is how `enabled=false` is implemented. The reconcile needs this to tell
+    # "already disabled" from "needs disabling" — without it, it would re-send
+    # add_consumer for every queue on every tick forever (lesson #4183).
+    consuming: set[str] = field(default_factory=set)
+    # Pool size PER HOSTNAME, not aggregated. The resize below computes each
+    # replica's own delta from its own current pool, so replicas that have
+    # drifted apart converge instead of being moved in lockstep from a shared
+    # baseline — which is what an aggregate here would silently reintroduce.
+    pools: dict[str, int] = field(default_factory=dict)
+
+    @property
+    def pool(self) -> int | None:
+        """One number for the UI. `max` rather than a sum: `slots` means the
+        pool size of ONE process (see the module docstring), so the largest
+        replica is the honest answer to "what is this lane set to". None when
+        no replica reported — unknown, never zero."""
+        return max(self.pools.values()) if self.pools else None
+
+    @property
+    def capacity(self) -> int:
+        """Total slots across replicas — how many tasks this lane can run at
+        once. Distinct from `pool`, and the two must not be confused: `pool`
+        is the DIAL (one process's size, what grow/shrink move), `capacity` is
+        the CAPABILITY. Asking "is this lane saturated" compares `active`,
+        which is summed across replicas, against this — against `pool` it
+        would call two half-busy replicas of 4 saturated at 4 active."""
+        return sum(self.pools.values())
+
+
+def _lane_for_queues(queues: tuple[str, ...]) -> Lane | None:
+    return LANES_BY_QUEUE_KEY.get(tuple(sorted(queues)))
+
+
+def inspect_lanes_sync() -> dict[str, LaneLiveState]:
+    """Live state per lane name. Sync — callers wrap in asyncio.to_thread.
+
+    Never raises. Every lane is present in the result; ones nothing answered
+    for carry `present=False`, so a caller cannot accidentally read a missing
+    lane as an empty one by iterating only what came back.
+    """
+    out = {lane.name: LaneLiveState() for lane in LANES}
+    try:
+        from ..celery_app import celery as celery_app
+
+        # ONE broadcast, then three TARGETED reads.
+        #
+        # A broadcast with no `destination` cannot know how many replies to
+        # expect, so it waits out its whole timeout rather than returning on
+        # the last one. Four of those is four full timeouts — about eight
+        # seconds — and `lane_view` sits on the Settings card, so that was the
+        # load time of the Worker lanes page every time it was opened.
+        #
+        # Naming the destinations lets celery stop as soon as those nodes have
+        # answered, which for workers in this same container is milliseconds.
+        # The worst case is unchanged: a node that vanishes between the
+        # broadcast and the targeted reads costs a full timeout waiting for a
+        # reply that is not coming.
+        insp = celery_app.control.inspect(timeout=CONTROL_TIMEOUT_SECONDS)
+        active_queues = insp.active_queues() or {}
+
+        # Nothing answered — and the three reads below exist only to describe
+        # what did. Returning here also makes the broker-down case FAST
+        # (one timeout, not four), which is exactly when the healthcheck and
+        # the card need an answer rather than a long wait.
+        if not active_queues:
+            return out
+
+        targeted = celery_app.control.inspect(
+            destination=sorted(active_queues),
+            timeout=CONTROL_TIMEOUT_SECONDS,
+        )
+        stats = targeted.stats() or {}
+        active = targeted.active() or {}
+        reserved = targeted.reserved() or {}
+    except Exception:
+        log.warning("worker_control: celery inspect failed", exc_info=True)
+        return out
+
+    for hostname, queues in active_queues.items():
+        lane = _lane_for_queues(tuple(q["name"] for q in queues))
+        if lane is None:
+            # A deployment slicing CELERY_QUEUES differently. Reported by the
+            # roster under its raw queue list; it simply has no lane row to
+            # control, which is honest rather than an error.
+            continue
+        state = out[lane.name]
+        state.present = True
+        state.replicas += 1
+        state.hostnames.append(hostname)
+        state.active += len(active.get(hostname, []))
+        state.reserved += len(reserved.get(hostname, []))
+        state.consuming.update(q["name"] for q in queues)
+
+        # `pool.max-concurrency` is the number pool_grow/pool_shrink move and
+        # the number the UI shows. Absent on a worker whose stats did not
+        # answer, which leaves pool=None — unknown, not zero.
+        pool = (stats.get(hostname) or {}).get("pool", {}).get("max-concurrency")
+        if isinstance(pool, int):
+            state.pools[hostname] = pool
+
+    for state in out.values():
+        state.hostnames.sort()
+    return out
+
+
+def effective_slots(target: int) -> int:
+    """What a pool can actually be set to. Never below one process.
+
+    Used wherever a target is COMPARED as well as wherever one is sent: a
+    reconcile that compares against the unclamped number sees a difference
+    that no control message can ever close, and re-sends it every tick.
+    """
+    return max(MIN_POOL_SLOTS, target)
+
+
+def set_lane_slots_sync(
+    lane: Lane, target: int, live: LaneLiveState | None = None,
+) -> tuple[bool, str | None]:
+    """Drive every replica of `lane` to `target` slots. Returns (applied, err).
+
+    Per-replica deltas rather than one shared delta: see the module docstring.
+    A replica already at the target is issued nothing at all, which is what
+    makes step 3's periodic reconcile converge instead of re-sending a grow of
+    zero forever (lesson #4183 — an enforcer without a reachable fixed point
+    re-does its own work every tick).
+
+    `applied=False` is not a failure of the SETTING. The caller has already
+    stored the value; this says only that the live push did not land, and the
+    reconcile will carry it when the lane answers again.
+    """
+    target = effective_slots(target)
+    try:
+        from ..celery_app import celery as celery_app
+
+        if live is None:
+            live = inspect_lanes_sync()[lane.name]
+        if not live.present:
+            return False, "lane is not running"
+        if not live.pools:
+            return False, "worker did not report its pool size"
+
+        control = celery_app.control
+        unreported = [h for h in live.hostnames if h not in live.pools]
+        for hostname, current in live.pools.items():
+            delta = target - current
+            if delta > 0:
+                control.pool_grow(delta, destination=[hostname])
+            elif delta < 0:
+                control.pool_shrink(-delta, destination=[hostname])
+        if unreported:
+            # Resized what could be resized, and said which could not. Silence
+            # here would leave a replica running at a size the UI claims it is
+            # not, with nothing anywhere recording the gap.
+            return False, f"no pool size reported by {', '.join(sorted(unreported))}"
+        return True, None
+    except Exception as exc:  # noqa: BLE001 — reported, never raised at a caller
+        log.warning("worker_control: could not resize %s", lane.name, exc_info=True)
+        return False, str(exc)
+
+
+def set_lane_enabled_sync(
+    lane: Lane, enabled: bool, live: LaneLiveState | None = None,
+) -> tuple[bool, str | None]:
+    """Start or stop `lane` consuming its queues, without killing the process.
+
+    `cancel_consumer` rather than a shutdown: a stopped consumer keeps its
+    worker alive and answering `inspect`, so a disabled lane stays visible and
+    can be turned back on. A killed worker would read as absent, which is the
+    same signal as a crash — and the whole point of the roster (#365) is that
+    those two must not look alike.
+    """
+    try:
+        from ..celery_app import celery as celery_app
+
+        if live is None:
+            live = inspect_lanes_sync()[lane.name]
+        if not live.present:
+            return False, "lane is not running"
+        control = celery_app.control
+        for queue in lane.queues:
+            if enabled:
+                control.add_consumer(queue, destination=live.hostnames)
+            else:
+                control.cancel_consumer(queue, destination=live.hostnames)
+        return True, None
+    except Exception as exc:  # noqa: BLE001
+        log.warning(
+            "worker_control: could not %s %s",
+            "enable" if enabled else "disable", lane.name, exc_info=True,
+        )
+        return False, str(exc)
+
+
+# --- the settings half, which is async ----------------------------------------
+#
+# Sync celery control above, async DB below, in one module. Same split
+# `service_roster` already runs (`_inspect_celery_sync` beside `touch_service`)
+# — the boundary is the transport, not the concern, and "control the workers"
+# is one concern.
+
+
+async def _rows_by_name(session: AsyncSession) -> dict[str, WorkerLane]:
+    """Every lane's row, creating any that are missing from its LANES defaults.
+
+    Self-heals rather than depending on a migration having run for a lane
+    added later: alembic 0103 seeded the four that existed on 2026-09-22, and
+    a fifth added to LANES afterwards gets its row the first time anything
+    asks. Without this, a new lane would read as absent and the UI would
+    simply not show it.
+    """
+    rows = {
+        row.name: row
+        for row in (await session.execute(select(WorkerLane))).scalars()
+    }
+    missing = [lane for lane in LANES if lane.name not in rows]
+    for lane in missing:
+        row = WorkerLane(name=lane.name, slots_cap=lane.default_slots_cap)
+        session.add(row)
+        rows[lane.name] = row
+    if missing:
+        await session.commit()
+    return rows
+
+
+@dataclass
+class LaneSettings:
+    """What the DATABASE knows about the lanes — read and finished with before
+    anything touches the broker.
+
+    This exists because holding a Postgres connection across a celery round
+    trip is what made the System tab block the whole site (operator,
+    2026-09-23: *"something about changing the cap number is blocking to the
+    website"*).
+
+    `lane_view` used to take the session and keep it open through an inspect
+    whose budget is eleven seconds — and that page polls every fifteen. With a
+    lane not answering, every inspect ran to nearly its full budget, so each
+    poll pinned a connection for ten seconds. SQLAlchemy's default pool is
+    five connections plus ten overflow; a couple of browser tabs, the health
+    endpoint doing the same thing, and a cap change adding two more inspects
+    exhausts that, and every OTHER request then waits on a connection.
+
+    Nothing was slow in itself. The slowness was a scarce resource held across
+    it, which is why it surfaced as the whole site stalling rather than as one
+    slow page.
+    """
+
+    caps: dict[str, int]
+    oldest_by_queue: dict[str, datetime]
+
+
+async def lane_settings(session: AsyncSession) -> LaneSettings:
+    """Every DB read the lane view needs, in one short-lived session."""
+    rows = await _rows_by_name(session)
+    return LaneSettings(
+        caps={name: row.slots_cap for name, row in rows.items()},
+        oldest_by_queue=await _oldest_running_by_queue(session),
+    )
+
+
+async def lane_view(settings: LaneSettings) -> list[dict]:
+    """Every lane: what is configured, what is live, what it may grow to.
+
+    Takes the settings rather than a session ON PURPOSE — see `LaneSettings`.
+    Everything below this line is broker work, and no database connection is
+    held while it happens.
+
+    `pending` is the honest backlog — Redis depth PLUS reserved — because
+    celery prefetches and LLEN alone reads 0 while a worker holds tasks in
+    memory.
+    """
+    # A deadline, because this is a request path and `to_thread` on its own is
+    # an await with no bound (rule 156). `inspect_lanes_sync` never raises and
+    # every inner call has its own timeout, so the only way past the budget is
+    # the thread not being scheduled — and a page that renders "not answering"
+    # is a better answer than one that does not render.
+    try:
+        live = await asyncio.wait_for(
+            asyncio.to_thread(inspect_lanes_sync),
+            timeout=INSPECT_BUDGET_SECONDS,
+        )
+    except TimeoutError:
+        log.warning(
+            "worker_control: inspect exceeded %ss; reporting every lane as "
+            "not answering", INSPECT_BUDGET_SECONDS,
+        )
+        live = {lane.name: LaneLiveState() for lane in LANES}
+    depths = await asyncio.to_thread(_queue_depths_sync)
+    oldest = settings.oldest_by_queue
+
+    now = datetime.now(UTC)
+    out = []
+    for lane in LANES:
+        cap = settings.caps[lane.name]
+        state = live[lane.name]
+        # None for a queue the broker did not answer for, which must not be
+        # silently summed as zero — an unknown depth is not an empty one.
+        known = [depths.get(q) for q in lane.queues]
+        depth = sum(d for d in known if d is not None) if any(
+            d is not None for d in known
+        ) else None
+        out.append({
+            "name": lane.name,
+            "display_name": lane.display_name,
+            "queues": list(lane.queues),
+            "slots_cap": cap,
+            "ceiling": derived_ceiling(lane),
+            # DERIVED, never stored. A cap of zero means no consumers, so
+            # "off" and "may use no workers" cannot disagree.
+            "enabled": cap > 0,
+            "memory_bound": lane.memory_bound,
+            "optional": lane.optional,
+            # What raising this lane's cap will download, so the UI can say
+            # WHICH model and how big BEFORE the first slot is asked for
+            # rather than after a multi-GB fetch has started. `measured`
+            # travels with the numbers: the UI must not present an estimate
+            # as a fact.
+            "models": [
+                {
+                    "repo": m.repo,
+                    "download_bytes": m.approx_download_bytes,
+                    "resident_bytes": m.approx_resident_bytes,
+                    "measured": m.measured,
+                }
+                for m in lane.models
+            ],
+            "live": {
+                "present": state.present,
+                "replicas": state.replicas,
+                "pool": state.pool,
+                "active": state.active,
+                "reserved": state.reserved,
+            },
+            "queue_depth": depth,
+            "pending": None if depth is None else depth + state.reserved,
+            # How long the oldest still-running task on this lane has been
+            # going, in minutes. The operator asked for a trigger here — grow
+            # a lane whose tasks run past some duration — and it stayed a
+            # REPORT: a long task does not finish sooner because the lane
+            # gained a slot, so scaling on it would spend memory to change
+            # nothing. Shown so they can see a lane wedged on one slow job,
+            # which is the genuinely useful half of the idea.
+            "oldest_running_minutes": _minutes_since(
+                min(
+                    (oldest[q] for q in lane.queues if q in oldest),
+                    default=None,
+                ),
+                now,
+            ),
+        })
+    return out
+
+
+async def _oldest_running_by_queue(session: AsyncSession) -> dict[str, datetime]:
+    """When the longest-running unfinished task on each queue started.
+
+    Read from `task_run`, which is OUR OWN table on OUR OWN wall clock, and
+    deliberately not from celery's `inspect active()`. Those entries carry a
+    `time_start` taken from the WORKER's `time.monotonic()` — a clock with an
+    arbitrary origin per process. Subtracting it from this process's wall
+    clock produces a number that looks like a duration and is meaningless, and
+    it would be meaningless in the direction that matters: plausible.
+
+    `task_run` also already carries the per-queue staleness thresholds the
+    recovery sweep uses, so a row still `running` here is one the system
+    itself considers legitimately in flight rather than abandoned.
+    """
+    result = await session.execute(
+        select(TaskRun.queue, func.min(TaskRun.started_at))
+        .where(TaskRun.status == "running", TaskRun.finished_at.is_(None))
+        .group_by(TaskRun.queue)
+    )
+    return {queue: started for queue, started in result if started is not None}
+
+
+def _minutes_since(started: datetime | None, now: datetime) -> int | None:
+    """Whole minutes, or None when nothing is running. Never negative: a row
+    written by a container whose clock is a few seconds ahead must read as 0
+    rather than as a task that starts in the future."""
+    if started is None:
+        return None
+    return max(0, int((now - started).total_seconds() // 60))
+
+
+def _queue_depths_sync() -> dict[str, int | None]:
+    """Redis LLEN per queue. None for one that did not answer — see lane_view.
+
+    Sync; the caller threads it. A per-queue try/except so one bad queue does
+    not cost the whole report, matching `api/system_activity._read_queues_sync`.
+    """
+    import redis
+
+    from ..config import get_config
+
+    out: dict[str, int | None] = {}
+    try:
+        client = redis.Redis.from_url(get_config().celery_broker_url)
+    except Exception:
+        log.warning("worker_control: no broker for queue depths", exc_info=True)
+        return {q: None for lane in LANES for q in lane.queues}
+    for lane in LANES:
+        for queue in lane.queues:
+            try:
+                out[queue] = int(client.llen(queue))
+            except Exception:  # noqa: BLE001 — a hiccup must not break the UI
+                out[queue] = None
+    return out
+
+
+class LaneUpdateRefused(ValueError):
+    """A requested value is outside what the lane may hold. Carries the reason
+    the UI shows — a greyed control with no explanation reads as a bug."""
+
+
+async def store_lane_cap(
+    session: AsyncSession, lane: Lane, slots_cap: int,
+) -> int:
+    """Validate and store the cap. Returns the PREVIOUS cap. DB only.
+
+    Split from the live push for the reason `LaneSettings` gives at length: a
+    Postgres connection must not be held across a celery round trip. Everything
+    here is fast and finished with before `push_lane_cap` starts.
+    """
+    rows = await _rows_by_name(session)
+    row = rows[lane.name]
+
+    ceiling = derived_ceiling(lane)
+    if slots_cap < 0:
+        raise LaneUpdateRefused("a cap cannot be negative")
+    if slots_cap > ceiling:
+        raise LaneUpdateRefused(
+            f"a cap of {slots_cap} is above what this container can hold "
+            f"({ceiling} for {lane.display_name})"
+        )
+
+    was_cap = row.slots_cap
+    row.slots_cap = slots_cap
+    await session.commit()
+    # The previous value, because the push needs the DIRECTION: lowering a cap
+    # has to reach the running lane now, and raising one has nothing to say.
+    return was_cap
+
+
+async def push_lane_cap(lane: Lane, slots_cap: int, *, was_cap: int) -> dict:
+    """Make the running lane obey a cap that is already stored. NO database.
+
+    ## What is pushed, and what is not
+
+    Consumers follow the cap immediately in BOTH directions: zero means off,
+    and off must take effect when it is asked for rather than up to a minute
+    later.
+
+    The pool is only ever pushed DOWNWARD. Raising a cap is permission, not a
+    request — growing on permission would put workers on a lane with nothing
+    to do — so the sizing pass spends it on its next tick if there is work.
+    That also makes the common case (raising a cap) free: no broker round trip
+    AT ALL, which is the difference between a control that answers instantly
+    and one that takes ten seconds. Keyed on the previous cap rather than on
+    "is it on" — the first cut only knew on/off, so it inspected on every
+    raise to find out whether the pool needed lowering, and the control it was
+    meant to make instant still waited out an inspect.
+
+    A failed push is not a failed setting. The value is already stored and the
+    sizing pass carries it within a minute; `applied: false` with a reason
+    lets the UI say "saved, not yet live" rather than "that didn't work"
+    (lesson #4202 — a live change that does not survive, with nothing saying
+    so).
+    """
+    was_on, now_on = was_cap > 0, slots_cap > 0
+    applied, error = True, None
+    if now_on != was_on:
+        applied, error = await asyncio.to_thread(set_lane_enabled_sync, lane, now_on)
+    if applied and not now_on:
+        # Down to the floor at once. The pool cannot be emptied, so "off" is
+        # one parked process with its consumers cancelled.
+        applied, error = await asyncio.to_thread(
+            set_lane_slots_sync, lane, MIN_POOL_SLOTS,
+        )
+    elif applied and now_on and slots_cap < was_cap:
+        # LOWERED on a running lane. Only this direction needs a message, and
+        # only when the pool is actually above the new cap — so it reads the
+        # live pool rather than resizing blind. A raise never reaches here.
+        live = await asyncio.to_thread(inspect_lanes_sync)
+        current = live[lane.name].pool
+        if current is not None and current > slots_cap:
+            applied, error = await asyncio.to_thread(
+                set_lane_slots_sync, lane, slots_cap, live=live[lane.name],
+            )
+
+    # Raising the cap off zero is what triggers the model download (milestone
+    # 422 step 6). Never at boot: that made every start of the ML role reach
+    # HuggingFace for ~3.5GB, and rule 164 permits a runtime fetch only for a
+    # feature that is optional and clearly OFF.
+    #
+    # On the TRANSITION, so re-saving a cap on a lane already running does not
+    # re-enqueue. And only when the consumer change landed: enqueueing onto a
+    # queue nothing is consuming would leave the task pending with no
+    # explanation until the lane returns.
+    fetching = False
+    if now_on and not was_on and lane.models and applied:
+        fetching = _enqueue_model_fetch()
+
+    return {
+        "name": lane.name,
+        "slots_cap": slots_cap,
+        "ceiling": derived_ceiling(lane),
+        "enabled": now_on,
+        "applied": applied,
+        "apply_error": error,
+        # Tells the UI to say a download has started rather than leaving the
+        # operator to wonder why a lane they just turned on is busy.
+        "fetching_models": fetching,
+    }
+
+
+def _enqueue_model_fetch() -> bool:
+    """Queue the model download. Returns whether it was accepted.
+
+    Import inside the function: `backend.app.tasks.ml` pulls in torch, and web
+    must not pay that import cost on a module that every settings request
+    touches.
+
+    Never raises. A broker that will not take the task is worth reporting, but
+    the SETTING has already been stored and the lane is already enabled — so
+    failing the whole request here would roll back nothing and tell the
+    operator their change did not happen when it did.
+    """
+    try:
+        from ..tasks.ml import ensure_models
+
+        ensure_models.delay()
+        return True
+    except Exception:  # noqa: BLE001 — reported, never raised at a caller
+        log.warning("worker_control: could not enqueue the model fetch", exc_info=True)
+        return False
+
+
+
+# --- the sizing pass: one sweep, always on ------------------------------------
+#
+# This replaced BOTH `reconcile_lanes_sync` (step 3) and `autoscale_lanes_sync`
+# (step 7) on 2026-09-23. They were two enforcers over one number, and the
+# whole of step 7's hardest reasoning — a stored value that is a FLOOR, a
+# target of `max(stored, current)` so the reconcile does not undo what the
+# autoscaler added — existed only to stop them fighting. Delete one of them and
+# the problem is not solved, it is absent.
+#
+# Operator: *"auto should be always on, not a setting, so that idle instances
+# quiet down when not running. the number that is visible and something the
+# user can tweak and manage should be the cap itself the number of running
+# workers is handled by the autoscaling function which is always on."*
+#
+# So there is one pass, it runs every minute, it reads the live pool rather
+# than any stored number, and the only thing it obeys is the cap.
+#
+# It also subsumes what the reconcile existed for. `pool_grow` is not durable:
+# a worker restarted by its supervisor comes back at its ENV concurrency,
+# silently below what the lane should be running. This pass reads the live
+# pool every minute and sizes from the backlog, so that worker is corrected on
+# the next tick — sooner than the five-minute reconcile managed, and without a
+# second sweep that could disagree with this one.
+
+# How much work justifies a slot. `pending` is depth + reserved, so it already
+# counts what celery has prefetched into worker memory — one task, one slot.
+#
+# Growth is IMMEDIATE and shrink is one slot per tick, deliberately asymmetric.
+# A backlog of four thousand should not take an hour to reach the cap, and a
+# lane that idles for one minute should not drop every process it has: the
+# cost of being one slot too large for a minute is a sleeping process, and the
+# cost of being too small is work not happening. For ML the asymmetry matters
+# most — every new slot reloads a multi-GB model, so the slow shrink is what
+# stops a quiet patch from paying that cost again a minute later.
+SHRINK_STEP = 1
+
+
+@dataclass
+class LaneSizing:
+    """What the pass did to one lane, and why — in the operator's terms.
+
+    A reason on every outcome including "held", because a sizing pass that
+    only speaks when it acts is one nobody can debug when it does not.
+    """
+
+    lane: str
+    action: str  # "grew" | "shrank" | "held" | "skipped"
+    slots: int
+    reason: str
+
+
+def wanted_slots(cap: int, active: int, pending: int | None) -> int:
+    """How many workers this lane has work for right now, within its cap.
+
+    One slot per task in flight or waiting, floored at one process and
+    ceilinged by the cap. `pending` of None means the broker did not answer
+    for this lane's queues — an unknown backlog is not an empty one (snippet
+    #3969), so it contributes nothing rather than being read as zero.
+
+    A cap of zero still returns one: billiard cannot run an empty pool, and
+    the parked process is what `add_consumer` lands on when the cap goes back
+    up. "Off" is expressed by cancelling consumers, not by emptying the pool.
+    """
+    if cap <= 0:
+        return MIN_POOL_SLOTS
+    return max(MIN_POOL_SLOTS, min(cap, active + (pending or 0)))
+
+
+def size_lanes_sync(caps: dict[str, int]) -> list[LaneSizing]:
+    """Size every lane to its backlog, within the cap. The whole control loop.
+
+    `caps` is lane name -> slots_cap, read from the database by the caller.
+    This function touches no database: the celery task that schedules it owns
+    the session, and keeping the DB out of here is what lets it be called from
+    anywhere that already knows the caps.
+
+    ## It must converge and then go quiet
+
+    One `inspect` for all lanes, and `set_lane_slots_sync` issues nothing to a
+    replica already at its target. A settled system therefore performs one
+    broker round trip plus one LLEN sweep per tick and sends no control
+    messages at all — the reachable fixed point lesson #4183 is about. An
+    enforcer that re-sent a grow of zero every tick would churn forever and
+    bury a real correction in its own noise.
+
+    ## An absent lane is SKIPPED, not corrected
+
+    `present=False` means nothing answered — a worker restarting, or an
+    unreachable broker. It does NOT mean zero slots. Deciding from that would
+    be a verdict drawn from an unswept read, and here it is worse than
+    useless: there is nothing to send the message to.
+    """
+    live = inspect_lanes_sync()
+    depths = _queue_depths_sync()
+    out: list[LaneSizing] = []
+
+    for lane in LANES:
+        cap = caps.get(lane.name)
+        if cap is None:
+            continue
+        state = live[lane.name]
+        if not state.present:
+            out.append(LaneSizing(lane.name, "skipped", 0, "lane is not answering"))
+            continue
+
+        # Consumers first, and only when they DISAGREE. Sending add_consumer
+        # for every queue on every tick of a settled system is the exact churn
+        # above, and invisible: add_consumer on a queue already consumed is
+        # harmless and reports success.
+        should_consume = cap > 0
+        if should_consume != state.consuming.issuperset(lane.queues):
+            ok, err = set_lane_enabled_sync(lane, should_consume, live=state)
+            if not ok:
+                out.append(LaneSizing(
+                    lane.name, "held", state.pool or 0,
+                    f"could not {'start' if should_consume else 'stop'} "
+                    f"consuming: {err}",
+                ))
+                continue
+
+        current = state.pool
+        if current is None:
+            out.append(LaneSizing(
+                lane.name, "held", 0, "worker did not report its pool size",
+            ))
+            continue
+
+        known = [depths.get(q) for q in lane.queues]
+        depth = (
+            sum(d for d in known if d is not None)
+            if any(d is not None for d in known) else None
+        )
+        pending = None if depth is None else depth + state.reserved
+        want = wanted_slots(cap, state.active, pending)
+
+        if want > current:
+            new = want
+            verb = "grew"
+        elif want < current:
+            # One at a time on the way down. See SHRINK_STEP.
+            new = max(want, current - SHRINK_STEP)
+            verb = "shrank"
+        else:
+            out.append(LaneSizing(
+                lane.name, "held", current,
+                f"{pending if pending is not None else '?'} waiting, "
+                f"{state.active} busy, cap {cap}",
+            ))
+            continue
+
+        ok, err = set_lane_slots_sync(lane, new, live=state)
+        if not ok:
+            out.append(LaneSizing(
+                lane.name, "held", current, f"could not resize: {err}",
+            ))
+            continue
+        out.append(LaneSizing(
+            lane.name, verb, new,
+            f"{pending if pending is not None else '?'} waiting, "
+            f"{state.active} busy, cap {cap}",
+        ))
+    return out

backend/app/services/worker_lanes.py

@@ -0,0 +1,366 @@
+"""The worker lanes: what they are, and how many slots each may be given.
+
+Milestone 422 step 1. This module is the ONE place that knows the lane set;
+`models/worker_lane.py` holds only what the operator can change about them.
+
+## Why the queues are here and not in the table
+
+A lane's queue set is not a preference — it is decided by `celery_app.py`'s
+`task_routes`, which is what puts a backup on `maintenance_long` and a
+thumbnail on `thumbnail`. An operator cannot move a task to another lane, so
+storing the queues as settings would create a row that can disagree with the
+routing table, and nothing would notice until a queue had no consumer.
+
+So: queues and display names are code, slots and caps are data. The table
+stores three numbers and a flag, and nothing that could contradict celery.
+
+This also collapses a duplicate rather than adding one.
+`service_roster.ROLE_NAMES` was a second copy of "queue set -> the name an
+operator recognises", and it had already drifted: `maintenance_long` is a
+live lane with four task routes pointing at it, and the roster did not know
+its name, so the System tab rendered it as `Worker (maintenance_long)`. That
+map is now derived from `LANES` below, so a lane added here is named
+everywhere at once.
+
+## Why the ceiling is derived rather than configured
+
+Consolidating the stack into one container (step 5) widens the OOM blast
+radius: today an ml-worker that exhausts memory is killed by Docker on its
+own, and web keeps serving. In one container the kernel picks a victim from
+the whole cgroup, and it may pick hypercorn — so a tagging task can take the
+UI down with it, on exactly the modest hardware least able to spare the
+memory.
+
+Operator, 2026-09-22: *"ram isn't an issue for me but some users might run
+this on weaker hardware and I don't want it to kill their servers."*
+
+So the maximum is computed from what the container actually has, and the
+operator's own `slots_cap` must fit under it. Three numbers, not two, and the
+ordering is the point:
+
+    slots  <=  slots_cap  <=  derived_ceiling
+    (live)     (operator)     (this module)
+
+The operator can always lower their cap. They cannot raise it past what the
+box can hold. The derived ceiling is never stored — a row that outlived a
+change in container limits must not carry a stale one.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+from dataclasses import dataclass
+from pathlib import Path
+
+log = logging.getLogger(__name__)
+
+# --- the lanes ---------------------------------------------------------------
+
+
+GIB = 1024 ** 3
+
+
+@dataclass(frozen=True)
+class ModelRequirement:
+    """A model a lane must download before it can do anything.
+
+    Surfaced to the UI so the operator is told WHICH model, how big, and what
+    it costs to hold — before they turn the lane on, not after a multi-GB
+    download has already started. The lane is optional and its cost is not
+    obvious from its name, which is the whole reason this is structured data
+    rather than a sentence in a component.
+
+    `measured=False` means the numbers are ESTIMATES and the UI must say so.
+    They come from the checkpoint's parameter count and dtype, not from a
+    build — and a number presented as fact decides whether someone's server
+    survives, so it is labelled rather than rounded confidently.
+    """
+
+    # The Hugging Face repo id, which is the honest answer to "which model".
+    repo: str
+    # Roughly what the download costs, for the operator's bandwidth and disk.
+    approx_download_bytes: int
+    # Roughly what ONE slot holds while running. Prefork forks a child per
+    # slot and each loads its own copy, so this multiplies.
+    approx_resident_bytes: int
+    measured: bool = False
+
+
+# SigLIP so400m — the only model FabledCurator itself downloads.
+#
+# What it is for, which is NOT obvious from the lane's name: it produces the
+# image embeddings that back similarity search, duplicate grouping and the
+# tag heads. WD14 tagging is the GPU AGENT's job, not this lane's — the
+# comment in celery_app.py naming both is stale since B3 (#1238), when the
+# agent took over and this lane was left as the CPU embed fallback for stacks
+# running no agent at all (see MLSettings.cpu_embed_enabled).
+#
+# Both numbers are ESTIMATES, derived from the checkpoint rather than from a
+# build: ~877M parameters at fp32 is ~3.5GB of weights, and holding them plus
+# activations and the torch runtime is what the resident figure covers. They
+# err high. Replace them with measurements — download the repo and read its
+# size; run one embed and read the worker child's VmHWM — and set
+# `measured=True` when you do.
+SIGLIP_MODEL = ModelRequirement(
+    repo="google/siglip-so400m-patch14-384",
+    approx_download_bytes=3_500_000_000,
+    approx_resident_bytes=4 * GIB,
+    measured=False,
+)
+
+
+@dataclass(frozen=True)
+class Lane:
+    """A worker lane. `name` is the stable key the settings row is keyed on.
+
+    Keyed on a lane NAME rather than a container hostname for the reason
+    `models/service_seen.py` gives at length: celery's worker names here are
+    `celery@<container id>` and are minted fresh on every deploy, so anything
+    keyed on them records a death and a birth every time the stack updates.
+    """
+
+    name: str
+    display_name: str
+    queues: tuple[str, ...]
+    # Which `entrypoint.sh` role starts this lane. NOT always the lane name:
+    # `maintenance_long` is the plain `worker` role pointed at a different
+    # queue, exactly as docker-compose starts it today (`command: ["worker"]`
+    # with CELERY_QUEUES=maintenance_long). Recorded here so the generated
+    # supervisord config and the compose file cannot disagree about it.
+    entrypoint_role: str
+    # THE cap a lane starts with — and, since 2026-09-23, the only number an
+    # operator sets for it. How many workers actually run is the autoscaler's
+    # job; this is the most it may use. Zero means the lane is off.
+    #
+    # One, and zero for ML. Deliberately far below the operator's own
+    # production numbers, which are tuned for their hardware and are not a
+    # sane first boot for a stranger — and low enough that a busy instance
+    # tells them to raise it rather than quietly consuming the machine.
+    default_slots_cap: int
+    # True when a slot costs a copy of the ML model rather than just a process.
+    # The only lane whose ceiling is decided by memory instead of by cores.
+    memory_bound: bool = False
+    # Models this lane downloads the first time it is enabled. Empty for every
+    # lane that needs none, which is how the UI knows whether to warn at all.
+    models: tuple[ModelRequirement, ...] = ()
+    # An optional lane is one the product works without. Shown as such, so
+    # nobody turns on a multi-GB download believing it is required.
+    optional: bool = False
+
+    @property
+    def queue_key(self) -> tuple[str, ...]:
+        """The sorted queue set, which is how `service_seen` identifies a
+        running worker. The join between what is configured here and what
+        `celery inspect` reports."""
+        return tuple(sorted(self.queues))
+
+
+# ONE CAP PER LANE, and that is the whole of what an operator sets.
+#
+# Operator, 2026-09-23: *"auto should be always on, not a setting, so that
+# idle instances quiet down when not running. the number that is visible and
+# something the user can tweak and manage should be the cap itself the number
+# of running workers is handled by the autoscaling function which is always
+# on."*
+#
+# Until then a lane had THREE operator values — `slots`, `slots_cap` and
+# `autoscale` — because the manual dial was built first (steps 2-4) and the
+# autoscaler arrived last (step 7) as an opt-in beside a control that already
+# existed. Nothing ever asked whether the dial should still exist once
+# something could move it automatically. It should not: "how many are running
+# right now" is a measurement, not a preference.
+#
+# One of each, and ML at zero. ML at zero is also rule 164's carve-out: a cap
+# of zero means no consumers, so a fresh install never loads a model or
+# reaches HuggingFace, and raising the cap is what triggers the fetch.
+#
+# These are far below the operator's own production numbers, and deliberately
+# so — they are what a stranger's first boot should do, not what a tuned
+# machine can. The UI is what closes that gap: a lane sitting at its cap with
+# a backlog says so, and says raising the cap is the fix. Without that a
+# conservative default is just a slow instance nobody knows how to speed up.
+LANES: tuple[Lane, ...] = (
+    Lane(
+        name="worker",
+        display_name="Worker",
+        queues=("default", "import", "thumbnail", "download"),
+        entrypoint_role="worker",
+        default_slots_cap=1,
+    ),
+    Lane(
+        name="scheduler",
+        display_name="Scheduler",
+        queues=("maintenance", "scan"),
+        entrypoint_role="scheduler",
+        default_slots_cap=1,
+    ),
+    Lane(
+        name="maintenance_long",
+        display_name="Long maintenance",
+        queues=("maintenance_long",),
+        entrypoint_role="worker",
+        default_slots_cap=1,
+    ),
+    Lane(
+        name="ml",
+        display_name="ML tagging",
+        queues=("ml",),
+        entrypoint_role="ml-worker",
+        default_slots_cap=0,
+        memory_bound=True,
+        models=(SIGLIP_MODEL,),
+        optional=True,
+    ),
+)
+
+LANES_BY_NAME: dict[str, Lane] = {lane.name: lane for lane in LANES}
+LANES_BY_QUEUE_KEY: dict[tuple[str, ...], Lane] = {
+    lane.queue_key: lane for lane in LANES
+}
+
+
+# --- what the container actually has -----------------------------------------
+
+# cgroup v2 first, then v1. A container started without an explicit memory
+# limit reports "max" on v2 and a sentinel near 2**63 on v1; both mean "no
+# limit", and the answer then is the host's RAM.
+_CGROUP_V2_MEMORY = Path("/sys/fs/cgroup/memory.max")
+_CGROUP_V1_MEMORY = Path("/sys/fs/cgroup/memory/memory.limit_in_bytes")
+_CGROUP_V2_CPU = Path("/sys/fs/cgroup/cpu.max")
+_CGROUP_V1_CPU_QUOTA = Path("/sys/fs/cgroup/cpu/cpu.cfs_quota_us")
+_CGROUP_V1_CPU_PERIOD = Path("/sys/fs/cgroup/cpu/cpu.cfs_period_us")
+
+# A v1 "unlimited" is PAGE_SIZE-aligned LONG_MAX, not a round number, so it is
+# recognised by magnitude rather than by equality. Anything claiming more than
+# a petabyte is a sentinel, not a machine.
+_UNLIMITED_ABOVE = 1 << 50
+
+# DERIVED from the model requirement above, never restated. The ceiling and
+# the number shown to the operator before they enable the lane have to be the
+# same figure, or the UI promises something the cap will then refuse.
+ML_BYTES_PER_SLOT = SIGLIP_MODEL.approx_resident_bytes
+
+# Held back for hypercorn and the non-ML lanes before any ML slot is offered.
+# In the consolidated container these share one cgroup with ML, and they are
+# the processes an OOM kill must not take (see the module docstring).
+RESERVED_BYTES = 2 * GIB
+
+# The smallest pool a lane can actually run: ONE process, never zero.
+#
+# billiard refuses to remove the last worker in a pool, so a lane asked to
+# shrink to nothing gets `ValueError("Can't shrink pool. All processes
+# busy!")` and the sizing pass re-sends the doomed message forever. Found on
+# the operator's live deploy, 2026-09-23.
+#
+# It is also what makes "off" expressible: a lane at cap 0 keeps this one
+# parked process with its consumers cancelled, so it still answers `inspect`
+# (and so reads as present rather than crashed), and `add_consumer` has
+# something to reach when the cap goes back up.
+#
+# Lives HERE rather than in `worker_control` because `gen_supervisord` needs
+# it at container boot and must not import the models package to get it.
+MIN_POOL_SLOTS = 1
+
+# The floor a cores-derived ceiling never goes below. A single-core box still
+# needs to be able to run its lanes; the ceiling exists to stop absurd values,
+# not to make a small machine unusable.
+MIN_CEILING = 1
+
+# What an unreadable limit yields. Low rather than unlimited, on purpose: not
+# knowing how much memory there is must never read as "plenty". An unswept
+# absence is not a verdict.
+UNKNOWN_CEILING = 1
+
+
+def _read_int(path: Path) -> int | None:
+    try:
+        raw = path.read_text().strip()
+    except OSError:
+        return None
+    if raw == "max":
+        return None
+    try:
+        return int(raw)
+    except ValueError:
+        return None
+
+
+def container_memory_bytes() -> int | None:
+    """The memory this container may use, or None when it cannot be read.
+
+    None means UNKNOWN, never UNLIMITED. Every caller must treat it as the
+    conservative case — the whole point of the ceiling is to protect a machine
+    whose size we are unsure of.
+    """
+    for path in (_CGROUP_V2_MEMORY, _CGROUP_V1_MEMORY):
+        value = _read_int(path)
+        if value is not None and value < _UNLIMITED_ABOVE:
+            return value
+        if value is not None:
+            # A sentinel: the cgroup exists but sets no limit, so the real
+            # bound is the host's.
+            break
+    try:
+        return os.sysconf("SC_PHYS_PAGES") * os.sysconf("SC_PAGE_SIZE")
+    except (ValueError, OSError, AttributeError):
+        return None
+
+
+def container_cpu_count() -> int | None:
+    """Effective cores, honouring a cgroup CPU quota.
+
+    `os.cpu_count()` reports the HOST's cores from inside a container, so a
+    quota of 2.0 on a 32-core host would otherwise offer 32 slots. The
+    operator's own stack sets `cpus: '4.0'` on ml-worker, so this is a real
+    configuration here and not a hypothetical.
+    """
+    quota: float | None = None
+    try:
+        raw = _CGROUP_V2_CPU.read_text().strip().split()
+        if raw and raw[0] != "max":
+            quota = int(raw[0]) / int(raw[1])
+    except (OSError, ValueError, IndexError, ZeroDivisionError):
+        pass
+    if quota is None:
+        q = _read_int(_CGROUP_V1_CPU_QUOTA)
+        p = _read_int(_CGROUP_V1_CPU_PERIOD)
+        if q is not None and p and q > 0:
+            quota = q / p
+    if quota is not None and quota > 0:
+        return max(1, int(quota))
+    return os.cpu_count()
+
+
+def derived_ceiling(lane: Lane) -> int:
+    """The most slots `lane` may be given on this container.
+
+    Never stored. Recomputed on every read so a container whose limits changed
+    is bounded by what it has NOW rather than by what it had when its row was
+    written.
+    """
+    if lane.memory_bound:
+        total = container_memory_bytes()
+        if total is None:
+            log.warning(
+                "worker_lanes: cannot read a memory limit; capping %s at %d",
+                lane.name, UNKNOWN_CEILING,
+            )
+            return UNKNOWN_CEILING
+        usable = total - RESERVED_BYTES
+        if usable < ML_BYTES_PER_SLOT:
+            # Honestly zero. A box that cannot hold one model alongside the web
+            # process must be told it cannot run tagging, not sold a slot that
+            # will OOM the container the first time it is used.
+            return 0
+        return int(usable // ML_BYTES_PER_SLOT)
+
+    cores = container_cpu_count()
+    if cores is None:
+        return UNKNOWN_CEILING
+    return max(MIN_CEILING, cores)
+
+
+def ceilings() -> dict[str, int]:
+    """Every lane's ceiling, for the settings API and the UI."""
+    return {lane.name: derived_ceiling(lane) for lane in LANES}

backend/app/tasks/admin.py

@@ -122,6 +122,28 @@ def prune_missing_file_records_task(self) -> dict:
     return {"checked": checked, "missing": len(missing_ids), "deleted": deleted}
 
 
+@celery.task(
+    name="backend.app.tasks.admin.repair_discord_downloads_task",
+    bind=True,
+    autoretry_for=(OperationalError, DBAPIError),
+    retry_backoff=15, retry_backoff_max=180, max_retries=1,
+    soft_time_limit=1800, time_limit=2400,  # 30 min / 40 min
+)
+def repair_discord_downloads_task(self, dry_run: bool = True) -> dict:
+    """Clean re-download of the Discord files broken by the `None` naming
+    (#3999). dry_run (the default) returns the projection; apply deletes the
+    broken images and their files, makes gallery-dl forget every Discord
+    download, and restarts every Discord source's backfill. Defaults to the SAFE
+    preview because the apply deletes files. Operator-triggered only."""
+    from ..services.discord_repair import repair_discord_downloads
+
+    SessionLocal = _sync_session_factory()
+    with SessionLocal() as session:
+        return repair_discord_downloads(
+            session, images_root=IMAGES_ROOT, dry_run=dry_run,
+        )
+
+
 @celery.task(
     name="backend.app.tasks.admin.dedup_videos_task",
     bind=True,

backend/app/tasks/maintenance.py

@@ -502,10 +502,16 @@ def prune_task_runs() -> dict:
     soft_time_limit=1800, time_limit=2100,
 )
 def backfill_phash() -> int:
-    """Recompute phash for stored images that have none (imported before
-    FC-2d-i+ii). Keyset-paginated by id (restart-safe), NULL-only fill,
-    idempotent. Videos legitimately keep phash NULL. A missing/unreadable
-    file is logged and left NULL — never fails the task."""
+    """Recompute phash for stored images that have none. Keyset-paginated by
+    id (restart-safe), NULL-only fill, idempotent. Videos legitimately keep
+    phash NULL. A missing/unreadable file is logged and left NULL — never
+    fails the task.
+
+    Two sources of NULLs: images imported before FC-2d-i+ii, and migration
+    0098, which cleared every phash so the library could be re-hashed at
+    hash_size=16 (#4223). The daily beat entry exists for the second — until
+    a row is refilled it takes no part in dedup, which is why this runs on a
+    schedule rather than waiting for a deep scan."""
     SessionLocal = _sync_session_factory()
     updated = 0
     last_id = 0
@@ -1247,6 +1253,7 @@ def sync_memberships() -> str:
     from ..services.artist_membership_service import rescan as membership_rescan
     from ..services.credential_crypto import CredentialCrypto
     from ..services.credential_service import CredentialService
+    from ..services.membership_reconcile import apply_membership_lapses
     from ..services.membership_roster import roster_user_id, sync_platform
     from ..services.patreon_client import PatreonClient
     from ..services.subscribestar_client import SubscribeStarClient
@@ -1298,9 +1305,18 @@ def sync_memberships() -> str:
                     )
 
                 async with async_factory() as session:
-                    results.append(
-                        await sync_platform(session, platform=platform, fetch=fetch)
-                    )
+                    result = await sync_platform(session, platform=platform, fetch=fetch)
+                results.append(result)
+
+                # #3995: stop pulling sources whose paid access has ended, and
+                # resume the ones this stopped once they are paid again. Only
+                # right after a successful sync, so it always acts on the roster
+                # just written, never on a stale one.
+                if result.get("ok"):
+                    async with async_factory() as session:
+                        result["lapses"] = await apply_membership_lapses(
+                            session, platform=platform,
+                        )
 
             # #388 E4: offer the freshly-synced roster to the artists FC already
             # tracks. Chained here rather than given its own beat entry because
@@ -1321,9 +1337,67 @@ def sync_memberships() -> str:
         if "skipped" in r:
             parts.append(f"{r['platform']}=skipped({r['skipped']})")
         elif r.get("ok"):
-            parts.append(f"{r['platform']}={r['count']}")
+            lapses = r.get("lapses") or {}
+            detail = (
+                f"(stopped={lapses['stopped']},resumed={lapses['resumed']})"
+                if lapses.get("stopped") or lapses.get("resumed") else ""
+            )
+            parts.append(f"{r['platform']}={r['count']}{detail}")
         else:
             parts.append(f"{r['platform']}=FAILED({r['error']})")
     if res.get("suggested") is not None:
         parts.append(f"suggested={res['suggested']}")
     return " ".join(parts) or "no platforms"
+
+@celery.task(name="backend.app.tasks.maintenance.size_worker_lanes")
+def size_worker_lanes() -> dict:
+    """Size every lane to its backlog, within the cap the operator set.
+
+    ONE sweep, replacing `reconcile_worker_lanes` and `autoscale_worker_lanes`
+    (2026-09-23). They were two enforcers over one number: the reconcile drove
+    the pool to a stored `slots`, the autoscaler moved it away from that same
+    value, and most of the autoscaler's design existed to keep the reconcile
+    from undoing its work. Deleting the stored number deletes the conflict.
+
+    It still does what the reconcile existed for. `pool_grow` is not durable —
+    a worker restarted by its supervisor comes back at its ENV concurrency,
+    silently below what the lane should run — and this reads the LIVE pool
+    every minute, so that worker is corrected on the next tick rather than
+    after five.
+
+    Returns every lane's outcome INCLUDING the ones it held, each with a
+    reason. A pass that only speaks when it acts cannot be debugged on the day
+    it does not.
+    """
+    from ..models import WorkerLane
+    from ..services.worker_control import size_lanes_sync
+
+    # Read INSIDE the session. Reading a column off a detached instance
+    # happens to work while the attribute is still loaded and stops working
+    # the moment anything expires it — a failure that would appear long after
+    # this line, in a sweep nobody is watching.
+    with _sync_session_factory()() as session:
+        caps = {
+            row.name: row.slots_cap
+            for row in session.execute(select(WorkerLane)).scalars()
+        }
+    if not caps:
+        # Migration 0103/0105 seed these, so an empty table means they have
+        # not run yet. Nothing to assert — and inventing defaults here would
+        # let this task disagree with the seed it is meant to be enforcing.
+        return {"sized": []}
+
+    sized = size_lanes_sync(caps)
+    for d in sized:
+        if d.action not in ("held", "skipped"):
+            log.info(
+                "worker lanes: %s %s to %s slots — %s",
+                d.lane, d.action, d.slots, d.reason,
+            )
+    return {
+        "sized": [
+            {"lane": d.lane, "action": d.action, "slots": d.slots,
+             "reason": d.reason}
+            for d in sized
+        ],
+    }

backend/app/tasks/ml.py

@@ -668,3 +668,31 @@ def scheduled_retract_auto_tags() -> str:
     with SessionLocal() as session:
         n_ccip = retract_auto_applied_ccip(session)
     return f"head={n_head} ccip={n_ccip}"
+
+
+@celery.task(name="backend.app.tasks.ml.ensure_models", bind=True)
+def ensure_models(self) -> dict:
+    """Fetch the models this lane needs, if they are not already present.
+
+    Milestone 422 step 6. This used to run in `entrypoint.sh` before celery
+    started, which made every boot of the ML role reach HuggingFace for
+    ~3.5GB — a startup dependency on a third party, for a feature the operator
+    may never use. Rule 164 permits a runtime fetch only for something
+    "optional and clearly off", so it moved here: enqueued the moment the lane
+    is ENABLED, never at boot.
+
+    Being a task rather than a startup step is what makes it visible: it gets
+    a TaskRun row like any other, so the download shows in Activity with a
+    duration and a status, and a failure is something the operator can see and
+    retry rather than a container that quietly never became useful.
+
+    Idempotent — `download_models` fetches only what is missing — so enabling
+    an already-provisioned lane costs one no-op task rather than a re-download.
+    That matters because the reconcile may enqueue it again.
+    """
+    from ..scripts.download_models import main as download
+
+    rc = download()
+    if rc != 0:
+        raise RuntimeError(f"model download failed with exit code {rc}")
+    return {"ok": True}

backend/app/utils/paths.py

@@ -60,6 +60,37 @@ def derive_subdir(source_path: Path, import_root: Path) -> str:
     return str(rel) if str(rel) != "." else ""
 
 
+def canonical_subdir(subdir: str, artist_slug: str | None) -> str:
+    """`subdir` with its TOP-LEVEL segment replaced by the artist's slug.
+
+        canonical_subdir("Conto/patreon", "conto")  -> "conto/patreon"
+        canonical_subdir("Conto", "conto")          -> "conto"
+        canonical_subdir("Conto/patreon", None)     -> "Conto/patreon"
+        canonical_subdir("", "conto")               -> ""
+
+    `derive_subdir` mirrors the IMPORT tree's folder names verbatim, so a
+    filesystem import out of `/import/Conto/...` used to write
+    `<images_root>/Conto/...` while the download path wrote `<root>/conto/...`
+    for the very same Artist row. One artist, two directories, forever — 57
+    such families had accumulated by 2026-09-21, and the database never had
+    duplicate artists at all (milestone #421).
+
+    The slug is the canonical name because it is the Artist row's own
+    identifier: it is what `/api/artists` reports, what the ingesters already
+    write, and the one spelling that cannot vary with how a folder happened to
+    be capitalised on the way in.
+
+    Two deliberate pass-throughs. NO artist resolved means there is nothing
+    authoritative to canonicalise against, and an EMPTY subdir is a file
+    landing at the images root — those have no artist folder to correct, and
+    what becomes of them is its own decision (task #4247), not a side effect
+    of this helper.
+    """
+    if not artist_slug or not subdir:
+        return subdir
+    return str(Path(artist_slug, *Path(subdir).parts[1:]))
+
+
 def hash_suffixed_name(stem: str, sha256_hex: str, ext: str) -> str:
     """Builds 'stem__<first10ofhash><ext>'.
 

backend/app/utils/phash.py

@@ -1,57 +1,222 @@
 """Perceptual-hash dedup helpers (ported from ImageRepo).
 
-hash_size=8 -> 64-bit hash -> 16-hex-char string, which fits the existing
-ImageRecord.phash String(32) column (no image_record migration). IR uses
-hash_size=16; the deliberate FC deviation keeps the schema unchanged. The
-Hamming threshold is the operator-exposed dial (ImportSettings).
+hash_size=16 -> 256-bit hash -> 64-hex-char string, which is why
+`ImageRecord.phash` is String(64) (widened in migration 0098).
+
+## Why 16, after running at 8 from FC-2d until 2026-09-21
+
+`hash_size=8` keeps only the top-left 8x8 block of the DCT — 64 bits
+describing an image's coarse light/dark layout and nothing else. Variant
+artwork that shares a composition (same pose and framing, a different
+outfit / expression / overlay) collided OUTRIGHT at that size, so the
+operator's near-duplicate dial could not separate variants from rescales
+even at its floor: `phash_threshold=0` means "the same 64 bits", not "the
+same image", and packs of 15 variants were landing as 3 records
+(issue #4223). IR always used 16; FC's deviation to 8 was made to fit the
+old String(32) column without a migration — a schema convenience that cost
+the operator artwork.
+
+## The hash no longer decides a merge on its own
+
+Dropping a file or superseding one is destructive, so `find_similar` now
+runs three gates, cheapest first:
+
+  1. Hamming distance within the operator's `phash_threshold` — the cheap
+     indexed pre-filter that PROPOSES candidates.
+  2. Aspect ratio within ASPECT_TOL. A crop or a re-canvas is not a
+     rescale, and this is the same identity test the tier-1 video near-dup
+     path already uses (`_VIDEO_DUP_ASPECT_TOL`).
+  3. A pixel-level confirm on the candidate's actual file, supplied by the
+     caller (the importer opens the files; this module stays I/O-free apart
+     from `fingerprint_path`).
+
+Because the confirm is what ACCEPTS a match, the threshold can stay
+generous enough to tolerate a re-encode without putting variants at risk.
+
+Every gate fails CLOSED: unknown dimensions, an unreadable candidate, a
+hash that won't parse — all mean "not a duplicate". The two failure
+directions are not symmetrical. Too strict keeps a redundant lower-res copy,
+which the operator can see and delete; too loose deletes artwork that only
+a re-walk of the source can bring back.
 """
 
 import imagehash
+from PIL import Image, ImageChops, ImageStat
 
-HASH_SIZE = 8
+HASH_SIZE = 16
+
+# Gate 2. Matching `importer._VIDEO_DUP_ASPECT_TOL` — a rescale preserves
+# aspect ratio to within rounding, so this only has to absorb off-by-one
+# pixel dimensions.
+ASPECT_TOL = 0.02
+
+# Gate 3. Both images are reduced to one FINGERPRINT_SIZE-square grayscale
+# thumbnail and compared directly, which is the question actually being
+# asked: "are these the same picture at different resolutions?"
+#
+# Two criteria, because they catch different things. MEAN absolute
+# difference catches a global change (a recolour, a filter, a different
+# shading pass) that leaves the composition intact. The CHANGED-PIXEL
+# fraction catches a LOCAL one — an added overlay, a different expression,
+# an alternate outfit on part of the figure — which a mean over 4096 pixels
+# would otherwise dilute into noise.
+#
+# Tuned to fail closed (see the module docstring). A true rescale lands near
+# zero on both; these ceilings sit well above that and well below a variant.
+FINGERPRINT_SIZE = 64
+FINGERPRINT_MAX_MEAN_DIFF = 6.0
+FINGERPRINT_CHANGED_LEVEL = 64
+FINGERPRINT_MAX_CHANGED_FRACTION = 0.02
+
+
+def _seek_first_frame(pil_image) -> None:
+    """Animated images (multi-frame WebP/GIF/APNG) are hashed and
+    fingerprinted on frame 0 — the conventional choice for animated
+    content, and the one that keeps PIL from iterating every frame."""
+    if getattr(pil_image, "is_animated", False):
+        try:
+            pil_image.seek(0)
+        except Exception:
+            pass
 
 
 def compute_phash(pil_image) -> str | None:
     """Perceptual hash of an opened PIL image, as a hex string. None on any
     failure (videos/unreadable/non-image).
 
-    For animated images (multi-frame WebP/GIF/APNG), explicitly seek to
-    frame 0 first. Without this, some PIL operations downstream of
-    imagehash.phash (convert("L"), resize) can iterate all frames and
-    blow past Celery's hard time limit on large animations
-    (operator-flagged 2026-05-26 against animated WebPs). The pHash of
-    frame 0 is the conventional choice for animated content.
+    Frame 0 for animated images: without the seek, PIL operations
+    downstream of imagehash.phash (convert("L"), resize) can iterate all
+    frames and blow past Celery's hard time limit on large animations
+    (operator-flagged 2026-05-26 against animated WebPs).
     """
     try:
-        if getattr(pil_image, "is_animated", False):
-            try:
-                pil_image.seek(0)
-            except Exception:
-                pass
+        _seek_first_frame(pil_image)
         return str(imagehash.phash(pil_image, hash_size=HASH_SIZE))
     except Exception:
         return None
 
 
+def fingerprint(pil_image):
+    """A small grayscale thumbnail of an opened PIL image, for gate 3.
+
+    Returns a detached PIL image (so the caller may close the original) or
+    None on any failure. PIL-only on purpose: numpy is an imagehash
+    transitive dependency, not a declared one for this path.
+    """
+    try:
+        _seek_first_frame(pil_image)
+        return pil_image.convert("L").resize(
+            (FINGERPRINT_SIZE, FINGERPRINT_SIZE), Image.LANCZOS
+        )
+    except Exception:
+        return None
+
+
+def fingerprint_path(path) -> Image.Image | None:
+    """`fingerprint` for a file on disk. None if it cannot be read — which
+    the gate treats as "not a duplicate"."""
+    try:
+        with Image.open(path) as im:
+            return fingerprint(im)
+    except Exception:
+        return None
+
+
+def fingerprint_diff(
+    a, b, *, changed_level: int = FINGERPRINT_CHANGED_LEVEL,
+) -> tuple[float, float] | None:
+    """(mean absolute difference, fraction of pixels past `changed_level`) for
+    two fingerprints. None if either is missing or the comparison fails.
+
+    This is the MEASUREMENT behind `fingerprints_match`, split out so the
+    calibration report (scripts/phash_gate_report.py) can show how far a pair
+    sat from the limits instead of only which side of them it fell on. The
+    constants were chosen without a real-library sample; the numbers this
+    returns are what moves them.
+    """
+    if a is None or b is None:
+        return None
+    try:
+        diff = ImageChops.difference(a, b)
+        hist = diff.histogram()
+        total = sum(hist)
+        if not total:
+            return None
+        return (ImageStat.Stat(diff).mean[0], sum(hist[changed_level:]) / total)
+    except Exception:
+        return None
+
+
+def fingerprints_match(
+    a, b,
+    *,
+    max_mean_diff: float = FINGERPRINT_MAX_MEAN_DIFF,
+    changed_level: int = FINGERPRINT_CHANGED_LEVEL,
+    max_changed_fraction: float = FINGERPRINT_MAX_CHANGED_FRACTION,
+) -> bool:
+    """True when two fingerprints are the same picture: no large global
+    drift AND no meaningful local region that differs. False on any
+    failure."""
+    measured = fingerprint_diff(a, b, changed_level=changed_level)
+    if measured is None:
+        return False
+    mean, changed_fraction = measured
+    return mean <= max_mean_diff and changed_fraction <= max_changed_fraction
+
+
+def aspect_matches(
+    width: int | None, height: int | None,
+    cand_width: int | None, cand_height: int | None,
+    *, tol: float = ASPECT_TOL,
+) -> bool:
+    """Gate 2. False when either side's dimensions are unknown — an
+    unmeasurable candidate is not a proven duplicate."""
+    if not width or not height or not cand_width or not cand_height:
+        return False
+    a, b = width / height, cand_width / cand_height
+    if a <= 0 or b <= 0:
+        return False
+    return abs(a - b) / max(a, b) <= tol
+
+
 def find_similar(
     phash_hex: str,
     width: int,
     height: int,
     candidates: list[tuple[str, int, int, int]],
     threshold: int,
+    *,
+    confirm=None,
 ) -> tuple[str, int | None]:
     """candidates: (phash_hex, width, height, image_id). Returns one of
     ("none", None) / ("larger_exists", id) / ("smaller_exists", id).
-    First qualifying candidate wins (IR loop order)."""
+    First candidate to pass EVERY gate wins (IR loop order).
+
+    `confirm` is gate 3: an optional callable(image_id) -> bool, called only
+    for a candidate that already passed the hash and aspect gates, and
+    expected to compare the two files' pixels. A rejected candidate does not
+    end the search — the loop moves on, so a false pre-filter hit cannot
+    mask a real duplicate further down the list. Omitting it leaves the
+    hash+aspect behaviour, which is what the unit tests exercise.
+    """
     new_h = imagehash.hex_to_hash(phash_hex)
     for cand_hex, cw, ch, cid in candidates:
         try:
             dist = new_h - imagehash.hex_to_hash(cand_hex)
         except Exception:
+            # Includes the mismatched-length case while a library re-hash
+            # (migration 0098) is still in flight: an old 64-bit hash cannot
+            # be compared to a new 256-bit one, and skipping it degrades to
+            # "no dedup yet" rather than to a wrong merge.
             continue
-        if dist <= threshold:
-            if cw >= width and ch >= height:
-                return ("larger_exists", cid)
-            if width > cw or height > ch:
-                return ("smaller_exists", cid)
+        if dist > threshold:
+            continue
+        if not aspect_matches(width, height, cw, ch):
+            continue
+        if confirm is not None and not confirm(cid):
+            continue
+        if cw >= width and ch >= height:
+            return ("larger_exists", cid)
+        if width > cw or height > ch:
+            return ("smaller_exists", cid)
     return ("none", None)

ci-requirements.md

@@ -46,8 +46,8 @@ per `docs/process.md`'s "add deps to the image when used by >1 project".
 - Integration uses Fabled-Git Actions `services:` + socket-discovered bridge IPs
   because `act_runner` (swarm-runner v0.6+) puts services on the default
   bridge with no embedded DNS. The pattern is documented in the rulebook's
-  `fabled-git.md` "CI philosophy" section and FC's `ci.yml` is the canonical
-  example.
+  `fabled-git.md` "CI philosophy" section and FC's `build.yml` integration lane
+  is the canonical example.
 - No `package-lock.json` is tracked yet (FC's `feedback_no_local_runs`
   memory bans `npm install` locally). Using `npm install` rather than
   `npm ci` until a lockfile lands.
@@ -83,7 +83,7 @@ per `docs/process.md`'s "add deps to the image when used by >1 project".
   digit `0` or starts 1-9, and there are at most four. `2026.08.29.0201` is
   rejected; `2026.8.29.201` is the same value one character narrower per
   segment, and rule 148 defines comparison as numeric per segment, so nothing is
-  reordered. `ci.yml`'s `extension-version` lane asserts the derived string
+  reordered. `build.yml`'s `extension-version` lane asserts the derived string
   against that exact regex, plus a `YYYY.M.D.HHMM` shape check that would catch
   a regression to the pre-318 `1.0.<minutes>` — which AMO would accept and which
   orders below everything already signed. Checking here is the whole point: AMO
@@ -91,7 +91,8 @@ per `docs/process.md`'s "add deps to the image when used by >1 project".
   `scripts/artifacts.sh version extension` **delegates** to `packaging.sh` so
   the two cannot answer differently.
 - Every job that derives anything checks out with `fetch-depth: 0` — all four
-  `build.yml` jobs, `ci.yml`'s `extension-version` and `backend-lint-and-test`
+  publishing `build.yml` jobs, its `extension-version` and `backend-lint-and-test`
+  lanes
   (for `tests/test_artifact_paths.py` and `test_artifact_identity.py`), and
   `release.yml`, which additionally walks the tag graph. A depth-1 clone sees
   one commit and derives a wrong, too-low value **rather than failing**, so the
@@ -99,8 +100,8 @@ per `docs/process.md`'s "add deps to the image when used by >1 project".
 - **`scripts/artifacts.sh` is the same shape one level up: one definition per
   artifact of what it is built from, and the two values derived from it.**
   `revision` (12 hex of the newest commit touching that set) and `version`
-  (`YYYY.MM.DD.HHMM` UTC, rule 148). Four artifacts, four independent answers,
-  so a push touching only `agent/` leaves web and ml alone.
+  (`YYYY.MM.DD.HHMM` UTC, rule 148). Three artifacts, three independent
+  answers, so a push touching only `agent/` leaves web and the extension alone.
   `tests/test_artifact_paths.py` reads each Dockerfile and asserts every COPY
   source is covered, so adding a COPY without updating the script fails CI.
 - **A file that DECIDES an artifact's identity belongs in its set even though it
@@ -153,7 +154,8 @@ per `docs/process.md`'s "add deps to the image when used by >1 project".
   layer store at all**, where the old `docker` driver at least reused whatever
   the runner's dockerd happened to hold. Measured on run 4896, the first builds
   after the driver moved: web 3m44s (was 2m23s), ml 3m49s (was 3m20s), agent
-  11m12s (was 9m26s) — every one slower. A `:buildcache` tag is read by every
+  11m12s (was 9m26s) — every one slower. (`ml` was its own build then; #4311
+  retired it once it became the same bytes as web under a second name.) A `:buildcache` tag is read by every
   build that runs, is one moving ref per image, holds cache blobs rather than a
   shippable artifact, and is overwritten in place, so it is not a return of the
   per-version tags milestone 318 withdrew (#3114).

docker-compose.override.yml

@@ -47,7 +47,7 @@ services:
   ml-worker:
     build:
       context: .
-      dockerfile: Dockerfile.ml
+      dockerfile: Dockerfile
     environment:
       LOG_LEVEL: DEBUG
     volumes:

docker-compose.single.yml

@@ -0,0 +1,104 @@
+# FabledCurator in three containers — the install path.
+#
+#     docker compose -f docker-compose.single.yml up -d
+#
+# Milestone 422 step 5. FabledCurator runs web and every worker lane inside
+# ONE container, with Postgres and Redis beside it. How much work each lane
+# does is then a dial in the web UI (Settings -> Activity -> Worker lanes),
+# live, with no compose edit and no restart.
+#
+# THE MULTI-SERVICE STACK IS NOT REPLACED. `docker-compose.yml` still runs the
+# five app services separately and is the right shape for a Swarm deployment
+# spread across hosts, where per-service rolling rollback and placement
+# constraints matter. This file is the adopter path: one box, one command.
+#
+# What consolidating costs, stated here rather than discovered later:
+#   - Everything shares one host, so there is no spreading work across nodes.
+#   - Rollback is all-or-nothing; there is no rolling back `web` alone.
+#   - One stop timeout for the whole container, sized to the slowest lane.
+#
+# NOT a cost, recorded so it is not rediscovered and raised again: the
+# multi-service stack mounts /images:ro on ml-worker and one container cannot
+# mount one path two ways. Operator ruled that a non-issue (2026-09-22) — it
+# is the same codebase either way.
+#
+# FabledCurator has no authentication. Whatever can reach ${PORT} is an
+# administrator, including over the stored platform session cookies. Do not
+# publish this port beyond a network you trust — see "Before you expose it"
+# in README.md.
+
+services:
+  redis:
+    image: redis:7-alpine
+    volumes:
+      - redis_data:/data
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+    restart: unless-stopped
+
+  postgres:
+    image: pgvector/pgvector:pg16
+    environment:
+      POSTGRES_USER: ${DB_USER:-curator}
+      POSTGRES_PASSWORD: ${DB_PASSWORD:-postgres}
+      POSTGRES_DB: ${DB_NAME:-curator}
+    volumes:
+      - postgres_data:/var/lib/postgresql/data
+    # pgvector index builds and the gallery's TABLESAMPLE reads both want more
+    # shared memory than docker's 64MB default.
+    shm_size: 512m
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U ${DB_USER:-curator} -d ${DB_NAME:-curator}"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+    restart: unless-stopped
+
+  fabledcurator:
+    image: git.fabledsword.com/bvandeusen/fabledcurator:latest
+    # No `command:`. Everything — hypercorn plus one celery process per lane
+    # under supervisord — is what the image does by default, and supervisord's
+    # config is generated from the application's own lane table so the two
+    # cannot disagree. `command: ["all"]` still works and means the same thing.
+    # Sized to the SLOWEST lane, not the average. maintenance_long runs DB
+    # backups, library audits and translation backfill, and gets 180s to
+    # finish a chunk; the lanes stop in parallel, so this covers the max
+    # rather than their sum. Below this, a routine restart becomes a SIGKILL
+    # mid-backup — which is recoverable (the work is chunked and idempotent)
+    # but wastes however long it had run.
+    stop_grace_period: 200s
+    # No healthcheck here either. The image declares one that reads the role
+    # it is running, and for this one that means BOTH halves: hypercorn
+    # answers AND every lane is answering the broker. A web-only check would
+    # report a healthy container while every lane inside it had crashed —
+    # the failure mode consolidation creates, since docker can no longer see
+    # the lanes as separate services.
+    environment:
+      DB_USER: ${DB_USER:-curator}
+      DB_PASSWORD: ${DB_PASSWORD:-postgres}
+      DB_HOST: postgres
+      DB_PORT: "5432"
+      DB_NAME: ${DB_NAME:-curator}
+      CELERY_BROKER_URL: redis://redis:6379/0
+      CELERY_RESULT_BACKEND: redis://redis:6379/0
+      SECRET_KEY: ${SECRET_KEY:-change-me-before-you-expose-this}
+      EXTENSION_API_KEY: ${EXTENSION_API_KEY:-}
+      LOG_LEVEL: ${LOG_LEVEL:-INFO}
+    ports:
+      - "${PORT:-8080}:8080"
+    volumes:
+      - ${IMAGES_DIR:-./images}:/images
+      # Read-only. The filesystem scan copies out of here and never writes to
+      # it, so a mistake cannot reach the source library.
+      - ${IMPORT_DIR:-./import}:/import:ro
+    depends_on:
+      postgres: { condition: service_healthy }
+      redis: { condition: service_healthy }
+    restart: unless-stopped
+
+volumes:
+  redis_data:
+  postgres_data:

docker-compose.yml

@@ -233,7 +233,7 @@ services:
       redis:    { condition: service_healthy }
 
   ml-worker:
-    image: git.fabledsword.com/bvandeusen/fabledcurator-ml:latest
+    image: git.fabledsword.com/bvandeusen/fabledcurator:latest
     command: ["ml-worker"]
     # A single GPU inference pass can run tens of seconds — let it finish.
     stop_grace_period: 120s

entrypoint.sh

@@ -1,7 +1,78 @@
 #!/usr/bin/env bash
 set -euo pipefail
 
-ROLE="${1:-web}"
+# Defaults to the whole application (see the Dockerfile's CMD). Kept in step
+# with that CMD deliberately: they are two doors to the same decision, and a
+# disagreement between them would only show up as `docker run --entrypoint`
+# behaving differently from `docker run`.
+ROLE="${1:-all}"
+
+# CELERY NODE NAME. Every celery role below starts with `-n $CELERY_NODENAME@%h`.
+#
+# Celery's default node name is `celery@<hostname>`, and in the single-
+# container layout all four lanes share one hostname — so all four registered
+# as the SAME node. celery's own words for it, observed on run 7319:
+#
+#   DuplicateNodenameWarning: Received multiple replies from node name:
+#   celery@72adc5b706a7
+#
+# `inspect` then collapses four replies into one dict key and the last one
+# wins, so three lanes read as absent and WHICH three varies per call:
+#
+#   lanes not answering: maintenance_long, ml, worker
+#   lanes not answering: maintenance_long, scheduler, worker
+#
+# That is fatal twice over. The composite healthcheck can never pass, so the
+# container is permanently unhealthy; and `pool_grow(destination=[hostname])`
+# addresses a lane BY that name, so the UI dial and the autoscaler would have
+# resized whichever lane happened to answer rather than the one asked for.
+#
+# The generated supervisord config sets this per lane. Unset — which is every
+# service in the multi-service stack — it falls back to `celery`, exactly
+# celery's own default, so `celery@$HOSTNAME` healthchecks there still work.
+: "${CELERY_NODENAME:=celery}"
+
+# RECORD THE ROLE, for the image's own HEALTHCHECK to read.
+#
+# The container is the only thing that knows what it was asked to run, and
+# before this every compose and stack file had to restate it as a healthcheck
+# of its own. The Dockerfile now declares one check that dispatches on this.
+#
+# Written ONCE, by the outermost invocation. The `all` role starts the other
+# roles through this same script under supervisord, and those children must
+# not overwrite the container's role with their own — a lane starting would
+# turn the composite check into a web-only one, silently. FC_ROLE is exported,
+# so a child sees it set and skips.
+#
+# Best effort: a read-only /tmp is not a reason to refuse to boot. The
+# healthcheck treats a missing file as "nothing to check" rather than as a
+# failure, for the same reason.
+if [ -z "${FC_ROLE:-}" ]; then
+  export FC_ROLE="$ROLE"
+  printf '%s\n' "$ROLE" > "${FC_ROLE_FILE:-/tmp/fc-role}" 2>/dev/null || true
+fi
+
+# WAIT FOR POSTGRES AND REDIS before doing anything that needs them.
+#
+# Swarm has no ordering primitive — it ignores `depends_on` entirely — so
+# every service in a stack starts at once and this container races its own
+# database on every cold deploy.
+#
+# The multi-service stack hid how sharp that is: a `web` task that failed
+# `alembic upgrade head` against a still-initialising Postgres simply died,
+# and Swarm restarted it until it worked. Consolidation removes that. Each
+# supervisord program gets `startretries=3`, so three quick failures put the
+# program in FATAL and leave it there — supervisord keeps running, the
+# container keeps running, and the application never starts. It would present
+# as a permanently unhealthy container whose image was fine and whose
+# database merely took twenty seconds to come up.
+#
+# Skipped for `shell`, which exists precisely for the case where something
+# else is broken and you want a prompt rather than a gate.
+case "$ROLE" in
+  shell|bash) ;;
+  *) python -m backend.app.scripts.wait_for_deps ;;
+esac
 shift || true
 
 case "$ROLE" in
@@ -28,6 +99,7 @@ case "$ROLE" in
     CONCURRENCY="${CELERY_CONCURRENCY:-2}"
     echo "[entrypoint] Starting Celery worker queues=$QUEUES concurrency=$CONCURRENCY"
     exec celery -A backend.app.celery_app:celery worker \
+      -n "${CELERY_NODENAME:-celery}@%h" \
       --loglevel=info \
       -Q "$QUEUES" \
       --concurrency="$CONCURRENCY"
@@ -35,22 +107,61 @@ case "$ROLE" in
 
   scheduler)
     QUEUES="${CELERY_QUEUES:-maintenance,scan}"
-    echo "[entrypoint] Starting Celery beat+worker queues=$QUEUES"
+    # Honours CELERY_CONCURRENCY like the `worker` role does. It was hardcoded
+    # to 1, which was harmless while only compose started this lane and set no
+    # concurrency for it — but the generated supervisord config (milestone 422
+    # step 5) passes one, and a value silently ignored at boot would leave the
+    # lane at 1 until the reconcile sweep noticed, with nothing saying why.
+    CONCURRENCY="${CELERY_CONCURRENCY:-1}"
+    echo "[entrypoint] Starting Celery beat+worker queues=$QUEUES concurrency=$CONCURRENCY"
     exec celery -A backend.app.celery_app:celery worker \
+      -n "${CELERY_NODENAME:-celery}@%h" \
       --beat \
       --loglevel=info \
       -Q "$QUEUES" \
-      --concurrency=1
+      --concurrency="$CONCURRENCY"
     ;;
 
   ml-worker)
-    echo "[entrypoint] Ensuring ML models present in /models..."
-    python -m backend.app.scripts.download_models
-    echo "[entrypoint] Starting ML Celery worker (ml queue)"
+    # NO MODEL DOWNLOAD HERE (milestone 422 step 6). This used to run
+    # download_models before celery started, which made every boot of this
+    # role reach HuggingFace for ~3.5GB. Rule 164 permits a runtime fetch only
+    # for a feature that is "optional and clearly off" — so the fetch moved to
+    # the moment the operator ENABLES the lane, where it is visible, retryable
+    # and attributable, instead of being a silent precondition of starting.
+    #
+    # The worker therefore starts with no model present, which is correct: it
+    # is not consuming the ml queue until the lane is enabled, and enabling it
+    # is what enqueues ensure_models.
+    QUEUES="${CELERY_QUEUES:-ml}"
+    CONCURRENCY="${CELERY_CONCURRENCY:-1}"
+    echo "[entrypoint] Starting ML Celery worker queues=$QUEUES concurrency=$CONCURRENCY"
     exec celery -A backend.app.celery_app:celery worker \
+      -n "${CELERY_NODENAME:-celery}@%h" \
       --loglevel=info \
-      -Q ml \
-      --concurrency=1
+      -Q "$QUEUES" \
+      --concurrency="$CONCURRENCY"
+    ;;
+
+  all)
+    # The single-container layout (milestone 422 step 5): hypercorn plus one
+    # celery process per lane, under supervisord, in one container beside
+    # Postgres and Redis.
+    #
+    # The config is GENERATED from services/worker_lanes.LANES rather than
+    # checked in, so the processes this container runs and the lanes the
+    # application believes in cannot disagree — see the generator's docstring
+    # for why a static .conf would have been a fifth copy of the queue names.
+    #
+    # supervisord is PID 1 here and never reads the database. Every lane boots
+    # at its LANES default; the reconcile sweep raises it to whatever the
+    # operator stored, within one tick. That ordering is deliberate: settings
+    # adjust a baseline that already works, and can never prevent a boot.
+    CONF="${SUPERVISOR_CONF:-/tmp/supervisord.conf}"
+    echo "[entrypoint] Generating $CONF from the lane table"
+    python -m backend.app.scripts.gen_supervisord > "$CONF"
+    echo "[entrypoint] Starting supervisord (web + worker lanes)"
+    exec supervisord -c "$CONF"
     ;;
 
   shell|bash)
@@ -63,7 +174,7 @@ case "$ROLE" in
 
   *)
     echo "[entrypoint] Unknown role: $ROLE" >&2
-    echo "[entrypoint] Valid roles: web | worker | scheduler | ml-worker | shell | alembic" >&2
+    echo "[entrypoint] Valid roles: all | web | worker | scheduler | maintenance_long | ml | ml-worker | shell | alembic" >&2
     exit 1
     ;;
 esac

extension/README.md

@@ -1,7 +1,7 @@
 # FabledCurator Firefox Extension
 
 Self-hosted Firefox extension that pushes session cookies from supported
-platforms (Patreon, SubscribeStar, Hentai-Foundry, Discord, Pixiv)
+platforms (Patreon, SubscribeStar, Hentai-Foundry, Discord)
 into FabledCurator, and lets you add a creator as a Source from their
 page in one click.
 
@@ -33,7 +33,6 @@ npm run build                # unsigned XPI in web-ext-artifacts/
 - [ ] Options page accepts FC URL + key, indicator turns green
 - [ ] Cookie export: log into patreon.com, click Patreon card → "X cookies exported"
 - [ ] Discord token: open discord.com, click Discord card → "Token captured"
-- [ ] Pixiv OAuth: click Pixiv card → login redirects, token stored
 - [ ] Add as source: visit patreon.com/<creator>, click floating button → toast
 - [ ] Subscriptions list: popup → "Sources" tab → list renders
 - [ ] Check now: click play icon on source row → no error toast
@@ -69,7 +68,7 @@ rejected, and at most four segments are allowed. The extension therefore emits
 **the same numbers unpadded**: `2026.8.29.201` where the rest of the family
 says `2026.08.29.0201`. Rule 148 already defines comparison as numeric per
 segment, under which the two are equal, so nothing is reordered by the choice
-and left-padding each segment recovers the family string exactly. `ci.yml`'s
+and left-padding each segment recovers the family string exactly. `build.yml`'s
 `extension-version` lane checks the derived string against that regex on every
 push — the cheap place to find out, because AMO 409s on re-signing and a
 rejected version is burned for good.

extension/package.json

@@ -3,7 +3,7 @@
   "version": "1.0.11",
   "private": true,
   "description": "Firefox extension for FabledCurator",
-  "comment_ignore_files": "The --ignore-files list comes from scripts/packaging.sh, the single source of truth shared with ci.yml's guard and the derived-version patch count. `set -f` is REQUIRED before the substitution: without it the shell globs `test/**` against the working tree and silently narrows the pattern to whatever files happen to exist.",
+  "comment_ignore_files": "The --ignore-files list comes from scripts/packaging.sh, the single source of truth shared with build.yml's guard and the derived-version patch count. `set -f` is REQUIRED before the substitution: without it the shell globs `test/**` against the working tree and silently narrows the pattern to whatever files happen to exist.",
   "scripts": {
     "lint":  "set -f; web-ext lint --source-dir=. --no-config-discovery --ignore-files $(sh scripts/packaging.sh ignore)",
     "start": "set -f; web-ext run --source-dir=. --no-config-discovery --ignore-files $(sh scripts/packaging.sh ignore) --firefox=firefox",

extension/test/artist-url-samples.json

@@ -0,0 +1,138 @@
+{
+  "$comment": [
+    "THE SHARED ARTIFACT for the JS<->Py artist-pattern mirror (issue #3093).",
+    "",
+    "extension/lib/platforms.js PLATFORM_ARTIST_PATTERNS and",
+    "backend/app/services/extension_service.py _PLATFORM_PATTERNS are two hand-kept",
+    "copies of one table, and they gate OPPOSITE HALVES of a single interaction:",
+    "the JS copy decides whether the 'Add to FC' button appears, the Python copy",
+    "decides whether the resulting POST is accepted. Drift is therefore never",
+    "cosmetic -- JS looser than Py shows a button that 400s, Py looser than JS",
+    "silently never offers a button for a URL the backend would take. Issue #1485",
+    "was the second of those, and its fix had to be applied to both files by hand.",
+    "",
+    "Neither runtime imports the other. This file is the shared artifact instead:",
+    "both suites read it and assert it against their OWN copy of the patterns, so",
+    "a change to one copy alone fails the other runtime's suite.",
+    "  - extension/test/platforms.spec.js  (vitest)",
+    "  - tests/test_extension_artist_patterns.py  (pytest)",
+    "",
+    "Lives under extension/test/ because that path is excluded from BOTH the XPI",
+    "file set and the extension version derivation (see scripts/packaging.sh:",
+    "NOT_PACKAGED_TRACKED and NOT_VERSION_RELEVANT both carry 'test/**'), so",
+    "adding samples here never ships bytes and never forces a re-sign.",
+    "",
+    "Adding a case: put the URL in the platform's match/no_match list with a",
+    "'why'. Run both suites. If only one goes red, you have found drift, which is",
+    "the entire point of the file.",
+    "",
+    "'slug' is read by the Python side only -- its _derive returns (platform,",
+    "slug) where the JS isArtistPage returns a boolean. It is not optional on a",
+    "match entry; the backend deriving the WRONG slug from a URL both copies",
+    "agree on is its own defect class, and this pins it."
+  ],
+
+  "patreon": {
+    "match": [
+      {
+        "url": "https://www.patreon.com/maewix",
+        "slug": "maewix",
+        "why": "bare creator root"
+      },
+      {
+        "url": "https://patreon.com/maewix",
+        "slug": "maewix",
+        "why": "www is optional"
+      },
+      {
+        "url": "http://patreon.com/maewix",
+        "slug": "maewix",
+        "why": "http as well as https"
+      },
+      {
+        "url": "https://www.patreon.com/c/Atole",
+        "slug": "Atole",
+        "why": "#1485: the /c/ creator shape"
+      },
+      {
+        "url": "https://www.patreon.com/cw/Atole",
+        "slug": "Atole",
+        "why": "#1485: /cw/ is the 'creator workspace' URL Patreon serves once you are SUBSCRIBED -- exactly when the button matters most, and exactly what the pre-#1485 pattern missed"
+      },
+      {
+        "url": "https://www.patreon.com/cw/Atole/posts",
+        "slug": "Atole",
+        "why": "#1485: a creator inner page still derives the creator"
+      },
+      {
+        "url": "https://www.patreon.com/Atole/membership",
+        "slug": "Atole",
+        "why": "#1485: inner page on the bare shape"
+      }
+    ],
+    "no_match": [
+      { "url": "https://www.patreon.com/home", "why": "nav page, not a creator" },
+      { "url": "https://www.patreon.com/search", "why": "nav page" },
+      { "url": "https://www.patreon.com/messages", "why": "nav page" },
+      { "url": "https://www.patreon.com/notifications", "why": "nav page" },
+      { "url": "https://www.patreon.com/library", "why": "nav page" },
+      { "url": "https://www.patreon.com/settings", "why": "nav page" },
+      { "url": "https://www.patreon.com/posts", "why": "nav page; also the post-permalink prefix" },
+      { "url": "https://www.patreon.com/home/anything", "why": "a nav page's inner path is still not a creator" },
+      { "url": "https://www.patreon.com/settings/profile", "why": "as above" }
+    ]
+  },
+
+  "subscribestar": {
+    "match": [
+      {
+        "url": "https://www.subscribestar.com/foobar",
+        "slug": "foobar",
+        "why": "creator root on the .com TLD"
+      },
+      {
+        "url": "https://subscribestar.adult/foobar",
+        "slug": "foobar",
+        "why": "creator root on the .adult TLD"
+      },
+      {
+        "url": "https://subscribestar.adult/foobar/",
+        "slug": "foobar",
+        "why": "trailing slash is tolerated"
+      }
+    ],
+    "no_match": [
+      { "url": "https://subscribestar.adult/feed", "why": "nav page" },
+      { "url": "https://subscribestar.adult/messages", "why": "nav page" },
+      { "url": "https://subscribestar.adult/library", "why": "nav page" },
+      {
+        "url": "https://subscribestar.adult/foobar/posts",
+        "why": "SubscribeStar's pattern end-anchors on the creator root, unlike Patreon's -- an inner page does NOT derive. Pinned so the asymmetry between the two platforms stays deliberate rather than becoming a silently-fixed bug in one copy only."
+      }
+    ]
+  },
+
+  "hentaifoundry": {
+    "match": [
+      {
+        "url": "https://www.hentai-foundry.com/user/Foo",
+        "slug": "Foo",
+        "why": "user page"
+      },
+      {
+        "url": "https://www.hentai-foundry.com/user/Foo/profile",
+        "slug": "Foo",
+        "why": "inner page still derives the user"
+      },
+      {
+        "url": "https://hentai-foundry.com/user/Foo",
+        "slug": "Foo",
+        "why": "www is optional"
+      }
+    ],
+    "no_match": [
+      { "url": "https://www.hentai-foundry.com/pictures/popular", "why": "gallery listing, not a user" },
+      { "url": "https://www.hentai-foundry.com/", "why": "site root" }
+    ]
+  }
+}

extension/test/platforms.spec.js

@@ -190,3 +190,60 @@ describe('manifest.json agrees with the platform table', () => {
     }
   })
 })
+
+describe('the JS<->Py artist-pattern mirror (#3093)', () => {
+  // PLATFORM_ARTIST_PATTERNS here and extension_service._PLATFORM_PATTERNS in
+  // the backend are two hand-kept copies of one table, and they gate OPPOSITE
+  // halves of a single interaction: this copy decides whether the "Add to FC"
+  // button appears, the Python copy decides whether the resulting POST is
+  // accepted. So JS-looser-than-Py shows a button that 400s, and
+  // Py-looser-than-JS never offers a button for a URL the backend would take.
+  // #1485 was the second of those, and its fix had to be applied to both
+  // files by hand.
+  //
+  // "Keep in sync by hand; reviewers catch drift" is the same guarantee
+  // manifest.json had before #3069, where deviantart survived seven weeks.
+  //
+  // The two-runtimes objection to a shared SOURCE file is fair, so the shared
+  // artifact is the SAMPLES instead: both suites read this JSON and assert it
+  // against their own copy of the patterns, and neither imports the other.
+  // The sibling half is tests/test_extension_artist_patterns.py; adding a
+  // sample there covers it here for free, and vice versa.
+  const samples = Object.fromEntries(
+    Object.entries(
+      JSON.parse(readFileSync(path.join(EXT_DIR, 'test', 'artist-url-samples.json'), 'utf8'))
+    ).filter(([key]) => !key.startsWith('$'))
+  )
+
+  for (const [platform, spec] of Object.entries(samples)) {
+    // `slug` on a match entry is read by the Python half only — isArtistPage
+    // answers a boolean, while the backend's _derive returns (platform, slug).
+    for (const { url, why } of spec.match) {
+      it(`shows the button on ${url} — ${why}`, () => {
+        expect(isArtistPage(url, platform)).toBe(true)
+      })
+    }
+    for (const { url, why } of spec.no_match) {
+      it(`hides the button on ${url} — ${why}`, () => {
+        expect(isArtistPage(url, platform)).toBe(false)
+      })
+    }
+  }
+
+  it('has samples for every platform that has an artist pattern', () => {
+    // The guard's own coverage check: without it, deleting a platform's
+    // samples would make this block pass by testing less. Discord is
+    // deliberately in neither — it is channel-based, with no creator page to
+    // put a button on, so it has no artist pattern on either side.
+    expect(Object.keys(samples).sort()).toEqual(Object.keys(PLATFORM_ARTIST_PATTERNS).sort())
+  })
+
+  it('has samples in both directions for every platform', () => {
+    // A platform with only positive samples pins half the invariant. The
+    // no_match half is the one that catches a pattern quietly widening.
+    for (const [platform, spec] of Object.entries(samples)) {
+      expect(spec.match.length, `${platform} match samples`).toBeGreaterThan(0)
+      expect(spec.no_match.length, `${platform} no_match samples`).toBeGreaterThan(0)
+    }
+  })
+})

extension/test/version.spec.js

@@ -10,7 +10,7 @@ const readText = (...seg) => readFileSync(path.join(EXT_DIR, ...seg), 'utf8')
 
 // Only the git-free subcommands are exercised here: `version` shells out to
 // git, and the extension lane runs on node:24-bookworm-slim which may not ship
-// it. That one is covered where git is guaranteed — ci.yml's extension-version
+// it. That one is covered where git is guaranteed — build.yml's extension-version
 // lane and build.yml both run on ci-python.
 const packaging = (cmd) =>
   execFileSync('sh', [path.join(EXT_DIR, 'scripts', 'packaging.sh'), cmd], {
@@ -39,7 +39,7 @@ describe('packaging.sh — the single definition of what ships', () => {
   it('emits glob patterns literally, never expanded against the working tree', () => {
     // The script iterates its lists with deliberate word-splitting, so it must
     // run with pathname expansion off. Without that, invoking it from a cwd
-    // where test/ exists (exactly how ci.yml and vitest call it) expands
+    // where test/ exists (exactly how build.yml and vitest call it) expands
     // `test/**` into the individual spec files, and the pathspec silently stops
     // covering anything added later.
     const pathspec = packaging('pathspec')
@@ -116,7 +116,7 @@ describe('consumers delegate rather than keeping their own copy', () => {
   )
 
   it('no workflow hardcodes the packaged-file set', () => {
-    // ci.yml used to substitute `packaging.sh pathspec` directly, for the
+    // build.yml used to substitute `packaging.sh pathspec` directly, for the
     // manual-bump guard that milestone 271 step 5 retired. Nothing inlines the
     // set today, and nothing should start to: a literal :(exclude)extension/...
     // in a workflow means someone bypassed the shared definition, which is
@@ -157,7 +157,7 @@ describe('extension version', () => {
     //
     // It is still asserted, for one reason: `npm run build` locally packages
     // whatever is committed, so a value AMO would reject turns a local build
-    // into a confusing failure with no CI signal ahead of it. ci.yml checks
+    // into a confusing failure with no CI signal ahead of it. build.yml checks
     // the same grammar against the DERIVED value, which is the one AMO sees.
     for (const file of ['manifest.json', 'package.json']) {
       expect(read(file).version, `${file} version is not AMO-shaped`).toMatch(AMO)

frontend/public/favicon.svg

@@ -15,7 +15,10 @@
        Colours are theme tokens (frontend/src/theme/fabled-tokens.js): obsidian
        plate, accent gold. The plate is kept here (unlike logo.svg) so the tab
        icon is self-contained against any browser chrome; on the nav it is
-       invisible because it matches --fc-chrome-rgb exactly. -->
+       invisible because it matches the fc-chrome-rgb custom property exactly.
+       No double hyphen may appear inside this comment: XML forbids it, and a
+       browser refuses to render an SVG that does not parse (it happened once —
+       tests/test_public_svgs.py). -->
   <rect width="32" height="32" rx="6" fill="#14171A"/>
   <rect x="6.2" y="4.2" width="19.6" height="23.6" rx="1.4"
         fill="none" stroke="#A87338" stroke-width="2.4"/>

frontend/src/components/AppShell.vue

@@ -17,6 +17,22 @@ const route = useRoute()
 
 <style scoped>
 .fc-content {
+  /* The full brand mark as one large, faint backdrop behind every page,
+     pinned to the viewport so content scrolls over it. Opaque surfaces
+     (cards, the nav) cover it; it shows in the gutters and on bare page
+     ground. The series reader is immersive and skips the shell, so reading
+     is never drawn over it.
+
+     Faded by laying the page colour over it at 94%, NOT with `opacity` on an
+     overlay element: an overlay needs this element to be z-indexed above it,
+     which makes all page content one stacking context under the nav and can
+     trap an in-page overlay beneath it. A background changes no stacking.
+     The mark is gold and parchment, close to the text colours, so it has to
+     stay this faint to keep text over it readable. */
+  background:
+    linear-gradient(rgba(var(--v-theme-background), 0.94), rgba(var(--v-theme-background), 0.94)),
+    url('/logo.svg') center / min(88vmin, 1100px) no-repeat;
+  background-attachment: fixed;
   min-height: 100vh;
   /* NO padding-top: the TopNav is position:sticky, so it already reserves its
      own space in the v-app flex column — content flows directly below it. The

frontend/src/components/posts/PostCard.vue

@@ -1,5 +1,8 @@
 <template>
-  <v-card class="fc-post-card" variant="outlined">
+  <v-card
+    ref="cardEl" class="fc-post-card" variant="outlined"
+    :class="{ 'fc-post-card--wide': wide }"
+  >
     <div class="fc-post-card__head">
       <!-- Posts with no live subscription have source=null (alembic 0030);
            show a "filesystem import" affordance instead of a platform chip. -->
@@ -58,7 +61,7 @@
         </button>
         <div
           v-if="rail.length || moreCount" class="fc-post-card__rail"
-          :style="{ '--fc-rail-cols': railCols }"
+          :style="{ '--fc-rail-cols': railCols, '--fc-grid-cols': Math.min(2, railCols) }"
         >
           <button
             v-for="t in rail" :key="t.image_id" type="button"
@@ -221,20 +224,33 @@ const synthesisTitle = computed(() => {
 })
 
 const hero = computed(() => images.value[0])
-// The thumbnail strip spans the hero's full width (CSS grid, equal columns),
-// rather than a fixed 3-cell cap. Show up to RAIL_MAX cells; when there are
-// more images than fit, the last cell becomes a "+N" overflow tile so the
-// count stays accurate.
-const RAIL_MAX = 5
+
+// Filmstrip layout (milestone #407, option A). On a very wide window a card
+// that just grew would give one post the whole screen, so a WIDE card instead
+// pins the hero to a fixed height and moves the extra images into a 2-column
+// grid BESIDE it. Measured on the card rather than the viewport because the
+// same card renders in the Latest feed, Browse, and the in-context view, each
+// at a different width.
+const WIDE_CARD_PX = 1100
+const cardEl = ref(null)
+const wide = ref(false)
+
+// The narrow layout's strip spans the hero's full width (CSS grid, equal
+// columns); the wide layout's grid is 2×2. Show up to that many cells; when
+// there are more images than fit, the last cell becomes a "+N" overflow tile so
+// the count stays accurate.
+const RAIL_MAX_NARROW = 5
+const RAIL_MAX_WIDE = 4
 const serverMore = computed(() => props.post.thumbnails_more || 0)
 const afterHero = computed(() => images.value.slice(1))
+const railMax = computed(() => (wide.value ? RAIL_MAX_WIDE : RAIL_MAX_NARROW))
 const hasOverflow = computed(
-  () => serverMore.value > 0 || afterHero.value.length > RAIL_MAX,
+  () => serverMore.value > 0 || afterHero.value.length > railMax.value,
 )
 const rail = computed(() =>
   hasOverflow.value
-    ? afterHero.value.slice(0, RAIL_MAX - 1)
-    : afterHero.value.slice(0, RAIL_MAX),
+    ? afterHero.value.slice(0, railMax.value - 1)
+    : afterHero.value.slice(0, railMax.value),
 )
 const visibleCount = computed(() => (images.value.length ? 1 + rail.value.length : 0))
 const moreCount = computed(() => {
@@ -349,8 +365,17 @@ function measureOverflow () {
 }
 
 let ro = null
+let cardRo = null
 onMounted(() => {
   nextTick(measureOverflow)
+  const root = cardEl.value?.$el
+  if (typeof ResizeObserver !== 'undefined' && root) {
+    cardRo = new ResizeObserver((entries) => {
+      const w = entries[0]?.contentRect?.width ?? 0
+      wide.value = w >= WIDE_CARD_PX
+    })
+    cardRo.observe(root)
+  }
   // Re-measure when the card resizes (the container-query clamp differs by
   // width). Guarded for happy-dom / older runtimes without ResizeObserver.
   if (typeof ResizeObserver !== 'undefined' && descEl.value) {
@@ -358,7 +383,10 @@ onMounted(() => {
     ro.observe(descEl.value)
   }
 })
-onBeforeUnmount(() => { if (ro) { ro.disconnect(); ro = null } })
+onBeforeUnmount(() => {
+  if (ro) { ro.disconnect(); ro = null }
+  if (cardRo) { cardRo.disconnect(); cardRo = null }
+})
 
 async function toggleDesc () {
   if (!descExpanded.value) {
@@ -495,6 +523,40 @@ function formatBytes (n) {
   color: rgb(var(--v-theme-accent));
 }
 
+/* --- Filmstrip layout (wide cards, #407 A) ---------------------------------
+   The hero has a HEIGHT, not a width: a wide card must not turn into a
+   full-screen post, so its height stays roughly a third of the viewport
+   whatever the window's width. The extra images sit beside it as square cells
+   whose size derives from that same height, so the grid always ends flush with
+   the hero's bottom edge. */
+.fc-post-card--wide {
+  --fc-hero-h: clamp(260px, 34vh, 460px);
+  --fc-grid-gap: 8px;
+  --fc-cell: calc((var(--fc-hero-h) - var(--fc-grid-gap)) / 2);
+}
+.fc-post-card--wide .fc-post-card__body { flex-direction: row; gap: 24px; }
+.fc-post-card--wide .fc-post-card__media {
+  flex: 0 0 auto;
+  display: flex;
+  gap: var(--fc-grid-gap);
+}
+.fc-post-card--wide .fc-post-card__hero {
+  width: auto;
+  height: var(--fc-hero-h);
+}
+.fc-post-card--wide .fc-post-card__rail {
+  margin-top: 0;
+  gap: var(--fc-grid-gap);
+  grid-template-columns: repeat(var(--fc-grid-cols, 2), var(--fc-cell));
+  grid-auto-rows: var(--fc-cell);
+}
+.fc-post-card--wide .fc-post-card__rail-cell,
+.fc-post-card--wide .fc-post-card__rail-more { height: 100%; }
+.fc-post-card--wide .fc-post-card__text { flex: 1 1 0; min-width: 0; }
+/* Text is secondary here — long reads happen in the expanded view — so the
+   clamp keeps the text column no taller than the images beside it. */
+.fc-post-card--wide .fc-post-card__desc--clamped { -webkit-line-clamp: 4; }
+
 .fc-post-card__title {
   font-family: 'Fraunces', Georgia, serif;
   font-size: 18px; font-weight: 700;

frontend/src/components/posts/PostsFilterBar.vue

@@ -1,21 +1,22 @@
 <template>
   <div class="fc-posts-filters">
+    <!-- The full artist list, filtered client-side as you type. This used to
+         search the server only once something was typed, so opening the
+         dropdown showed an empty menu. -->
     <v-autocomplete
       v-model="artistModel"
       :items="artistOptions"
       :loading="artistLoading"
-      :search="artistQuery"
       item-title="name"
       item-value="id"
       label="Artist"
       density="compact"
       hide-details
       clearable
-      no-filter
       return-object
+      no-data-text="No matching artists"
       class="fc-posts-filters__artist"
-      @update:search="onArtistSearch"
-      @update:model-value="onArtistPicked"
+      @update:model-value="emitFilters"
     />
 
     <v-select
@@ -39,7 +40,8 @@
 
 <script setup>
 import { computed, onMounted, ref, watch } from 'vue'
-import { useSourcesStore } from '../../stores/sources.js'
+
+import { useApi } from '../../composables/useApi.js'
 import { usePlatformsStore } from '../../stores/platforms.js'
 
 const props = defineProps({
@@ -48,18 +50,19 @@ const props = defineProps({
 })
 const emit = defineEmits(['update:filters'])
 
-const sourcesStore = useSourcesStore()
+const api = useApi()
 const platformsStore = usePlatformsStore()
 
-// Artist autocomplete state
-const artistQuery = ref('')
 const artistOptions = ref([])
 const artistLoading = ref(false)
 const artistModel = ref(null)
 
-// Platform v-select state
+// `list`, not `platforms`: the store has never had a `platforms` property, so
+// this read undefined and the dropdown opened empty.
+// test/storeUsage.spec.js now fails on any read of a property the store does
+// not define.
 const platformItems = computed(() =>
-  (platformsStore.platforms || []).map(p => ({ title: p.key, value: p.key }))
+  platformsStore.list.map(p => ({ title: p.name || p.key, value: p.key }))
 )
 const platformModel = ref(props.platform)
 
@@ -67,29 +70,6 @@ const hasFilters = computed(
   () => artistModel.value != null || platformModel.value != null
 )
 
-let _searchTimer = null
-function onArtistSearch(q) {
-  artistQuery.value = q
-  clearTimeout(_searchTimer)
-  if (!q || !q.trim()) {
-    artistOptions.value = artistModel.value ? [artistModel.value] : []
-    return
-  }
-  _searchTimer = setTimeout(async () => {
-    artistLoading.value = true
-    try {
-      const results = await sourcesStore.autocompleteArtist(q, 20)
-      artistOptions.value = results
-    } finally {
-      artistLoading.value = false
-    }
-  }, 200)
-}
-
-function onArtistPicked() {
-  emitFilters()
-}
-
 function emitFilters() {
   emit('update:filters', {
     artist_id: artistModel.value?.id ?? null,
@@ -100,30 +80,38 @@ function emitFilters() {
 function clearAll() {
   artistModel.value = null
   platformModel.value = null
-  artistOptions.value = []
-  artistQuery.value = ''
   emitFilters()
 }
 
-onMounted(async () => {
-  await platformsStore.loadAll()
-  // If a deep-link arrives with an artist_id, hydrate the selected display.
-  if (props.artistId != null) {
-    const seed = { id: props.artistId, name: `Artist #${props.artistId}` }
-    artistModel.value = seed
-    artistOptions.value = [seed]
+// A deep link carries only the id. Once the list is loaded, show the real name
+// rather than a placeholder.
+function selectArtistById(id) {
+  if (id == null) {
+    artistModel.value = null
+    return
   }
+  if (artistModel.value?.id === id) return
+  artistModel.value = artistOptions.value.find(a => a.id === id)
+    || { id, name: `Artist #${id}` }
+}
+
+async function loadArtists() {
+  artistLoading.value = true
+  try {
+    artistOptions.value = await api.get('/api/artists/names')
+  } catch {
+    artistOptions.value = []
+  } finally {
+    artistLoading.value = false
+  }
+}
+
+onMounted(async () => {
+  await Promise.all([platformsStore.loadAll(), loadArtists()])
+  selectArtistById(props.artistId)
 })
 
-watch(() => props.artistId, (val) => {
-  if (val == null) {
-    artistModel.value = null
-  } else if (artistModel.value?.id !== val) {
-    const seed = { id: val, name: `Artist #${val}` }
-    artistModel.value = seed
-    artistOptions.value = [seed]
-  }
-})
+watch(() => props.artistId, (val) => selectArtistById(val))
 
 watch(() => props.platform, (val) => {
   if (platformModel.value !== val) platformModel.value = val || null

frontend/src/components/settings/DiscordRepairCard.vue

@@ -0,0 +1,118 @@
+<template>
+  <!-- #3999: Discord downloads made before the naming fix landed in a `None`
+       folder with no post and a download-time date. This re-downloads them
+       cleanly. Preview first; the apply deletes files. -->
+  <MaintenanceTile
+    icon="mdi-download-off-outline"
+    title="Repair Discord downloads"
+    blurb="Re-download Discord images that were saved with no post and the wrong date."
+    destructive
+    :open="applying || previewing"
+  >
+    <p class="text-body-2 mb-3">
+      Before the naming fix, every Discord download was saved into a folder
+      called <code>None</code>. Those images never got a post, so they show the
+      time Curator downloaded them rather than when they were posted.
+      <strong>Apply</strong> deletes those images, clears Discord from the
+      download history, and starts a fresh backfill of every Discord source so
+      they come back with their posts and dates. Tags you added by hand to those
+      images are lost. Nothing outside a Discord <code>None</code> folder is
+      touched.
+    </p>
+
+    <div class="d-flex align-center flex-wrap" style="gap: 12px;">
+      <v-btn
+        color="primary" variant="tonal" rounded="pill"
+        :loading="previewing" :disabled="applying" @click="preview"
+      >
+        <v-icon start>mdi-magnify</v-icon> Preview
+      </v-btn>
+      <v-btn
+        color="error" rounded="pill"
+        :loading="applying"
+        :disabled="previewing || !canApply"
+        @click="confirmOpen = true"
+      >
+        <v-icon start>mdi-download-off-outline</v-icon> Apply
+      </v-btn>
+    </div>
+
+    <v-alert
+      v-if="summary" :type="summaryType" variant="tonal" class="mt-4"
+      density="comfortable"
+    >
+      <span v-if="applied">
+        Deleted {{ summary.images_deleted }} image(s) and cleared
+        {{ summary.archive_entries }} Discord download record(s). Backfills
+        started on {{ summary.backfills_started }} Discord source(s).
+      </span>
+      <span v-else-if="hasWork">
+        {{ summary.images }} broken image(s) ({{ humanBytes(summary.bytes) }})
+        across {{ summary.directories }} folder(s), and
+        {{ summary.archive_entries }} Discord download record(s) to clear.
+        {{ summary.sources }} Discord source(s) will backfill again.
+      </span>
+      <span v-else>Nothing to repair — no broken Discord downloads found.</span>
+
+      <div v-if="applied && summary.files_failed" class="mt-1 text-caption">
+        {{ summary.files_failed }} file(s) could not be removed — see the worker log.
+      </div>
+      <div v-if="applied && summary.remaining" class="mt-1 text-caption">
+        {{ summary.remaining }} broken image(s) are still present. Run it again.
+      </div>
+    </v-alert>
+
+    <QueueStatusBar queue="maintenance_long" queue-label="Maintenance" />
+
+    <v-dialog v-model="confirmOpen" max-width="440">
+      <v-card>
+        <v-card-title>Repair Discord downloads?</v-card-title>
+        <v-card-text class="text-body-2">
+          This permanently deletes <strong>{{ summary?.images ?? 0 }}</strong>
+          Discord image(s) ({{ humanBytes(summary?.bytes) }}) and re-downloads
+          them from Discord with their posts and dates. Any tags you added to
+          those images by hand will not come back.
+        </v-card-text>
+        <v-card-actions>
+          <v-spacer />
+          <v-btn variant="text" @click="confirmOpen = false">Cancel</v-btn>
+          <v-btn color="error" @click="apply">Repair</v-btn>
+        </v-card-actions>
+      </v-card>
+    </v-dialog>
+  </MaintenanceTile>
+</template>
+
+<script setup>
+import { computed, ref } from 'vue'
+
+import { useMaintenanceTask } from '../../composables/useMaintenanceTask.js'
+import { humanBytes } from '../../utils/bytes.js'
+import MaintenanceTile from '../common/MaintenanceTile.vue'
+import QueueStatusBar from './QueueStatusBar.vue'
+
+const confirmOpen = ref(false)
+
+const { previewing, applying, summary, applied, preview, apply: applyTask } = useMaintenanceTask({
+  endpoint: '/api/admin/maintenance/repair-discord-downloads',
+  storageKey: 'fc.maint.repairDiscordDownloads',
+  appliedToast: 'Discord downloads cleared — backfills started',
+})
+
+// The archive count matters too: files could already be gone while gallery-dl
+// still believes it has them, which would stop the backfill re-fetching.
+const hasWork = computed(
+  () => !!summary.value && (summary.value.images > 0 || summary.value.archive_entries > 0),
+)
+const canApply = computed(() => hasWork.value && !applied.value)
+const summaryType = computed(() => {
+  if (applied.value) return 'success'
+  return hasWork.value ? 'info' : 'success'
+})
+
+// The confirm dialog gates the destructive apply; close it, then run.
+function apply () {
+  confirmOpen.value = false
+  applyTask()
+}
+</script>

frontend/src/components/settings/ImportFiltersForm.vue

@@ -10,16 +10,20 @@
       <div class="fc-phash">
         <div class="fc-phash__title">Near-duplicate sensitivity</div>
         <div class="fc-help mb-1">
-          How aggressively imports merge look-alike images (perceptual-hash
-          distance). <strong>Lower it if edits/variants of the same image are
-          being dropped as duplicates;</strong> raise it to collapse more
-          look-alikes. Applies to new imports.
+          How wide a net imports cast when looking for the same image at a
+          different resolution (perceptual-hash distance, out of 256 bits).
+          A match is only merged if the two files also share an aspect ratio
+          <em>and</em> their pixels agree, so variant artwork — a different
+          outfit, expression or overlay on the same pose — is kept even at a
+          generous setting. <strong>Raise it if higher-resolution re-uploads
+          are landing as separate copies;</strong> lower it to merge less.
+          Applies to new imports.
         </div>
         <v-row no-gutters class="align-center">
           <v-col cols="12" sm="9">
             <v-slider
               v-model="local.phash_threshold"
-              :min="0" :max="16" :step="1"
+              :min="0" :max="64" :step="1"
               :ticks="PHASH_TICKS" show-ticks="always" tick-size="4"
               thumb-label color="accent" hide-details
               class="fc-phash__slider"
@@ -139,8 +143,10 @@ const store = useImportStore()
 // instead of relying on the old tab's mount hook.
 onMounted(() => { if (!store.settings) store.loadSettings() })
 // Labelled stops so the less-initiated get the gist without knowing what a
-// Hamming distance is. 0 = byte-for-byte only; 10 = the shipped default.
-const PHASH_TICKS = { 0: 'Exact', 4: 'Strict', 10: 'Default', 16: 'Loose' }
+// Hamming distance is. 0 = an identical hash only; 24 = the shipped default.
+// Bits out of 256 (utils/phash hash_size=16). The scale changed with
+// migration 0098 — these are NOT the old 0-16 stops renamed (#4223).
+const PHASH_TICKS = { 0: 'Exact', 12: 'Strict', 24: 'Default', 48: 'Loose' }
 // Downloader + schedule-defaults fields moved to
 // /subscriptions?tab=settings (operator decision 2026-05-27). This form
 // now only owns image-import filters.
@@ -148,7 +154,7 @@ const local = reactive({
   min_width: 0, min_height: 0,
   skip_transparent: false, transparency_threshold: 0.9,
   skip_single_color: false, single_color_threshold: 0.95,
-  phash_threshold: 10,
+  phash_threshold: 24,
   wip_title_tagging_enabled: true,
   wip_soft_title_tagging_enabled: false,
 })

frontend/src/components/settings/MaintenancePanel.vue

@@ -15,6 +15,7 @@
         <ImportFiltersForm />
         <TranslationCard />
         <DiscordGroupingCard />
+        <DiscordRepairCard />
         <PostAssociationsCard />
         <MembershipRosterCard />
         <MembershipSuggestionsCard />
@@ -85,6 +86,7 @@ import VideoEmbeddingCard from './VideoEmbeddingCard.vue'
 import CropProposersCard from './CropProposersCard.vue'
 import HeadsCard from './HeadsCard.vue'
 import DiscordGroupingCard from './DiscordGroupingCard.vue'
+import DiscordRepairCard from './DiscordRepairCard.vue'
 import MembershipRosterCard from './MembershipRosterCard.vue'
 import MembershipSuggestionsCard from './MembershipSuggestionsCard.vue'
 import PostAssociationsCard from './PostAssociationsCard.vue'

frontend/src/components/settings/SystemActivityTab.vue

@@ -198,6 +198,16 @@ const filterErrorType = ref(null)
 const filterTask = ref(null)        // server-side task-name search (All activity)
 const failureSearch = ref('')       // client-side search over loaded failures
 
+// This filters HISTORY — task_run.queue — which is why it stays a written-out
+// list rather than being derived from the lanes endpoint like the other queue
+// lists were in milestone 422. Two reasons, and the second is the real one:
+// a derived list is empty whenever that endpoint is down, and more importantly
+// it would HIDE the filter for any queue that has rows but no longer has a
+// lane serving it, which is exactly when someone is looking.
+//
+// It had drifted regardless — `maintenance_long` was missing, so activity on
+// the long-maintenance lane could not be filtered for at all despite four task
+// routes pointing there. Added.
 const queueOptions = [
   { title: 'All queues', value: null },
   { title: 'import', value: 'import' },
@@ -206,6 +216,7 @@ const queueOptions = [
   { title: 'download', value: 'download' },
   { title: 'scan', value: 'scan' },
   { title: 'maintenance', value: 'maintenance' },
+  { title: 'maintenance_long', value: 'maintenance_long' },
   { title: 'default', value: 'default' },
 ]
 const statusOptions = [

frontend/src/components/settings/SystemHealthTab.vue

@@ -1,8 +1,22 @@
 <template>
   <!-- A Settings tab, not a page of its own (operator 2026-09-02): the first
        cut hung this off the health dot alone, which is a target you have to
-       already suspect something to look for. Settings is where someone goes
-       to ask the instance about itself, so it lives beside Activity. -->
+       already suspect something to look for. Settings is where someone goes to
+       ask the instance about itself.
+
+       ONE table, not two panes (operator 2026-09-23: "I feel that we can
+       probably combine the two sections into a single table and to format it
+       in such a way that it appears more bounded and less free-form or open").
+
+       They were the roster (milestone 365, what is running) above the worker
+       lanes (milestone 422, how hard) — and the four celery lanes appeared in
+       BOTH, once as a status row and again as a control row. The page said
+       "ML tagging is running" and then, four hundred pixels later, "ML tagging
+       1 / 1 busy". Two answers to one question, from two endpoints, free to
+       disagree with each other on screen.
+
+       So it is one row per part now, and a lane's row simply has controls
+       where a datastore's has none. -->
   <div>
     <div class="d-flex align-center mb-1">
       <v-spacer />
@@ -11,100 +25,322 @@
       </span>
     </div>
 
-    <p class="fc-sys__lede text-body-2 mb-5">
-      Every moving part of FabledCurator and whether it is still checking in.
-      Parts are learned as they appear, so anything that has run at least once
-      stays listed — that is what lets a stopped one be noticed rather than
-      simply vanishing.
+    <p class="fc-sys__lede text-body-2 mb-4">
+      Every moving part of FabledCurator: whether it is still checking in, and —
+      for the worker lanes — how much work it is allowed to take at once. Parts
+      are learned as they appear, so anything that has run at least once stays
+      listed. That is what lets a stopped one be noticed rather than simply
+      vanishing.
     </p>
 
     <v-alert
-      v-if="store.lastError" type="error" variant="tonal" density="compact" class="mb-4"
+      v-if="store.lastError" type="error" variant="tonal" density="compact"
+      class="mb-4"
     >
       Could not reach FabledCurator: {{ store.lastError }}
     </v-alert>
 
-    <v-card v-else variant="flat" class="fc-sys__card">
-      <div v-if="!store.parts.length" class="pa-6 text-center fc-sys__muted">
-        Still gathering — this fills in on the first check.
-      </div>
+    <v-alert
+      v-if="notice"
+      :type="notice.type" variant="tonal" density="compact"
+      class="mb-4" closable
+      @click:close="notice = null"
+    >
+      {{ notice.text }}
+    </v-alert>
 
-      <div
-        v-for="part in store.parts" :key="part.key"
-        class="fc-sys__row" :class="`fc-sys__row--${part.state}`"
-      >
-        <span class="fc-sys__dot" :class="`fc-sys__dot--${part.state}`" />
+    <v-card v-if="!store.lastError" variant="flat" class="fc-parts">
+      <v-table density="compact" class="fc-parts__table">
+        <thead>
+          <tr>
+            <th>Part</th>
+            <th>Queues</th>
+            <th class="text-right">Waiting</th>
+            <th class="text-right">Workers</th>
+            <th class="text-center" style="width: 160px;">Max workers</th>
+          </tr>
+        </thead>
+        <tbody>
+          <tr v-if="!rows.length">
+            <td colspan="6" class="text-center py-6 fc-sys__muted">
+              Still gathering — this fills in on the first check.
+            </td>
+          </tr>
 
-        <div class="fc-sys__body">
-          <div class="fc-sys__name">
-            {{ part.name }}
-            <span class="fc-sys__kind">{{ kindLabel(part.kind) }}</span>
-          </div>
-          <!-- The sentence, not just a chip. At the moment someone is deciding
-               whether to go and open Portainer, "has not checked in for 6 min"
-               is the thing that answers them. -->
-          <div class="fc-sys__detail">{{ part.detail }}</div>
-        </div>
+          <tr v-for="row in rows" :key="row.key">
+            <td>
+              <div class="fc-parts__name">
+                <span class="fc-sys__dot" :class="`fc-sys__dot--${row.state}`" />
+                {{ row.name }}
+                <span class="fc-sys__kind">{{ row.kindLabel }}</span>
+              </div>
+              <!-- The sentence, not just a chip. At the moment someone is
+                   deciding whether to go and open Portainer, "has not checked
+                   in for 6 min" is the thing that answers them. -->
+              <div class="fc-parts__detail">{{ row.detail }}</div>
+              <!-- The operator asked for a lane held at full for a long time to
+                   TRIGGER growth. It reports instead: extra slots do not make a
+                   long task finish sooner, so scaling on this would spend
+                   memory to change nothing. Seeing a lane wedged on one slow
+                   job is the useful half. -->
+              <div v-if="row.stuckFor" class="fc-parts__detail text-warning">
+                all workers busy for {{ row.stuckFor }}
+              </div>
+              <!-- The nudge. Operator: "there needs to be something that tells
+                   the user to bump those numbers to improve processing rate or
+                   they'd never know the controls exist." It fires only when
+                   raising the cap would actually help — see `laneAdvice`. -->
+              <div v-if="row.advice" class="fc-parts__advice">
+                <v-icon size="14" class="mr-1">mdi-arrow-up-bold-outline</v-icon>
+                {{ row.advice }}
+              </div>
+            </td>
 
-        <div class="fc-sys__meta">
-          <div v-if="part.last_seen_at" :title="part.last_seen_at">
-            seen {{ formatRelative(part.last_seen_at) }}
-          </div>
-          <div v-if="part.latency_ms != null">{{ part.latency_ms }} ms</div>
-          <div v-if="part.queues?.length" class="fc-sys__queues">{{ part.queues.join(', ') }}</div>
-        </div>
-      </div>
+            <td class="fc-parts__queues">{{ row.queues || '—' }}</td>
+
+            <!-- depth + reserved. LLEN alone reads 0 while a worker holds
+                 prefetched tasks in memory, which is the number that would make
+                 someone think a buried lane was idle. -->
+            <td class="text-right fc-parts__num">
+              {{ row.lane ? (row.lane.pending ?? '—') : '—' }}
+            </td>
+
+            <!-- Running, and how many of them are busy. Both are
+                 MEASUREMENTS — the sizing pass owns this number, nobody sets
+                 it — which is why neither is a control. -->
+            <td class="text-right fc-parts__num">
+              <template v-if="row.lane && row.lane.live.present">
+                {{ row.lane.live.active }} / {{ row.lane.live.pool ?? '?' }}
+              </template>
+              <template v-else>—</template>
+            </td>
+
+            <td class="text-center">
+              <template v-if="row.lane">
+                <div class="d-flex align-center justify-center">
+                  <v-btn
+                    icon="mdi-minus" size="x-small" variant="text"
+                    :disabled="busy === row.lane.name || row.lane.slots_cap <= 0"
+                    :aria-label="`Allow ${row.name} fewer workers`"
+                    @click="step(row.lane, -1)"
+                  />
+                  <span class="fc-parts__slots">{{ row.lane.slots_cap }}</span>
+                  <v-btn
+                    icon="mdi-plus" size="x-small" variant="text"
+                    :disabled="busy === row.lane.name
+                      || row.lane.slots_cap >= row.lane.ceiling"
+                    :aria-label="`Allow ${row.name} more workers`"
+                    @click="step(row.lane, 1)"
+                  />
+                </div>
+                <!-- The REASON a higher number is unavailable, always on
+                     screen — and, just as importantly, the headroom that says
+                     raising it is even possible. A greyed control with no
+                     explanation reads as a bug. -->
+                <div class="fc-parts__sub">
+                  {{ row.lane.ceiling === 0
+                    ? 'no room on this machine'
+                    : `up to ${row.lane.ceiling}` }}
+                </div>
+              </template>
+              <span v-else class="fc-sys__muted">—</span>
+            </td>
+          </tr>
+        </tbody>
+      </v-table>
     </v-card>
 
-    <p v-if="store.thresholds" class="fc-sys__foot text-caption mt-4">
-      A part is called stale after
-      {{ Math.round(store.thresholds.stale_after_seconds / 60) }} min without a
-      check-in and treated as stopped after
-      {{ Math.round(store.thresholds.down_after_seconds / 60) }} min. The window
-      is deliberately wide: a rolling deploy briefly runs two of a service and
-      then neither, and an indicator that reddened on every update would stop
-      being read.
-    </p>
+    <!-- What the one control actually means. Operator, 2026-09-23: *"there's
+         nothing to describe what 'auto' means or why their needs to be or
+         should be on/off toggles."* There were three controls and no sentence
+         explaining any of them; there is one now, and it is explained. -->
+    <div class="fc-parts__legend mt-4">
+      <p class="mb-2">
+        <strong>Max workers</strong> is the only thing you set. How many
+        workers a lane is actually running at any moment is decided for you —
+        it grows to meet a backlog and gives the workers back when the queue
+        empties, so an idle instance settles down to one of each without being
+        told to. The cap is the ceiling on that, never a target, so raising it
+        costs nothing until there is work to spend it on.
+      </p>
+      <p class="mb-2">
+        <strong>A cap of zero turns the lane off.</strong> It keeps one parked
+        process and stops taking work, so it stays listed here rather than
+        looking like a crash. Three of the four lanes need to be running for
+        FabledCurator to work at all; ML tagging is the one that is genuinely
+        optional, and it ships at zero because switching it on downloads a
+        model. <em>Up to N</em> beneath each dial is what this machine can
+        hold — memory for ML tagging, processor cores for the rest — and it is
+        recalculated from the container's real limits every time this page
+        loads.
+      </p>
+      <p class="mb-2">
+        The shipped caps are one of each, which is right for a first boot and
+        wrong for a busy library. A lane that is running everything its cap
+        allows while work piles up <strong>will say so in its row</strong>, and
+        raising that cap is the answer when it does.
+      </p>
+      <p v-if="store.thresholds" class="mb-0">
+        A part is called <strong>stale</strong> after
+        {{ Math.round(store.thresholds.stale_after_seconds / 60) }} min without a
+        check-in and treated as <strong>stopped</strong> after
+        {{ Math.round(store.thresholds.down_after_seconds / 60) }} min. The window
+        is deliberately wide: a rolling deploy briefly runs two of a service and
+        then neither, and an indicator that reddened on every update would stop
+        being read.
+      </p>
+    </div>
+
+    <!-- What an optional, off lane will cost BEFORE it is given a slot. Every
+         number comes from the lane payload, and `measured` travels with them —
+         an estimate is labelled rather than rounded into something that reads
+         like a fact. -->
+    <v-alert
+      v-for="lane in offOptionalLanes" :key="`advisory-${lane.name}`"
+      type="info" variant="tonal" density="compact" class="mt-4"
+    >
+      <div class="font-weight-medium mb-1">
+        {{ lane.display_name }} is optional and currently off
+      </div>
+      <p class="mb-2">
+        It computes image embeddings on the CPU — what similarity search,
+        duplicate grouping and tag suggestions are built on. You only need it if
+        you are <em>not</em> running the GPU agent, which does the same work
+        faster.
+      </p>
+      <p class="mb-1">Raising its cap above zero downloads, once:</p>
+      <ul class="mb-2">
+        <li v-for="m in lane.models" :key="m.repo">
+          <code>{{ m.repo }}</code> —
+          {{ approx(m.measured) }}{{ gb(m.download_bytes) }} to download, and
+          about {{ gb(m.resident_bytes) }} of RAM for
+          <strong>each</strong> slot while it runs.
+        </li>
+      </ul>
+      <p class="mb-0 text-caption">
+        Each worker loads its own copy, which is why this machine allows it
+        at most {{ lane.ceiling }}.
+        <template v-if="lane.ceiling === 0">
+          It has too little memory to run this at all.
+        </template>
+      </p>
+    </v-alert>
   </div>
 </template>
 
 <script setup>
-import { onMounted } from 'vue'
+import { computed, onMounted, onUnmounted, ref } from 'vue'
 
+import { laneStuckFor, useSystemActivityStore } from '../../stores/systemActivity.js'
 import { useSystemHealthStore } from '../../stores/systemHealth.js'
 import { formatRelative } from '../../utils/date.js'
+import { laneAdvice, mergeParts } from '../../utils/systemParts.js'
 
 const store = useSystemHealthStore()
+const lanesStore = useSystemActivityStore()
 
-function kindLabel(kind) {
-  if (kind === 'celery') return 'background worker'
-  if (kind === 'agent') return 'GPU agent'
-  if (kind === 'datastore') return 'datastore'
-  return kind
+// Which lane is mid-write. Per-lane rather than a global flag so adjusting one
+// lane does not freeze the others.
+const busy = ref(null)
+const notice = ref(null)
+
+// The roster half needs no timer: TopNav already polls that same store every
+// 15s for the health dot, on every route this tab is reachable from. The LANES
+// have no such poller, so this tab owns them — 15s and `document.hidden`
+// guarded, because `/api/system/workers` costs a broker round trip and
+// v-window keeps a visited tab MOUNTED, so a naive interval would keep firing
+// behind Maintenance and behind a backgrounded browser tab.
+const LANE_POLL_MS = 15000
+let pollId = null
+
+onMounted(() => {
+  store.refresh()
+  lanesStore.loadLanes()
+  pollId = setInterval(() => {
+    if (document.hidden) return
+    lanesStore.loadLanes()
+  }, LANE_POLL_MS)
+})
+onUnmounted(() => { clearInterval(pollId) })
+
+// One row per moving part, lanes joined to their roster entry. The join lives
+// in `utils/systemParts.js` because its failure is silent: a lane that stops
+// matching its part does not throw, it grows a SECOND row for the same worker
+// — the duplication this table was built to remove, returning through the code
+// that removed it.
+const rows = computed(() => mergeParts(
+  store.parts, lanesStore.lanes?.lanes ?? [], laneStuckFor,
+).map((row) => ({ ...row, advice: laneAdvice(row.lane) })))
+
+const offOptionalLanes = computed(() =>
+  (lanesStore.lanes?.lanes ?? []).filter(
+    (l) => l.optional && l.slots_cap === 0 && l.models?.length,
+  ),
+)
+
+function gb(bytes) {
+  return `${(bytes / 1024 ** 3).toFixed(1)} GB`
 }
 
-// No timer of its own. TopNav already polls this same pinia store every 15s
-// for the health dot, and it is mounted on every route this tab is reachable
-// from — a second interval here would just double the request rate for a 5s
-// freshness gain. v-window keeps a visited item MOUNTED (hidden, not
-// destroyed), so a local timer would also have kept firing behind Maintenance.
-// One refresh on open, so arriving at the tab doesn't wait out the nav's tick.
-onMounted(() => { store.refresh() })
+// An unmeasured figure says so. It decides whether someone's server survives,
+// and presenting an estimate as a measurement is the failure this guards.
+function approx(measured) {
+  return measured ? '' : 'about '
+}
+
+async function apply(lane, fields) {
+  busy.value = lane.name
+  notice.value = null
+  try {
+    const reply = await lanesStore.setLane(lane.name, fields)
+    if (reply && reply.fetching_models) {
+      notice.value = {
+        type: 'info',
+        text: `${lane.display_name} is on. Downloading its model now — watch `
+          + 'progress under Activity. It only happens once.',
+      }
+    } else if (reply && reply.applied === false) {
+      // Saved but not pushed — the lane is restarting, or the broker blipped.
+      // NOT an error: the reconcile carries it when the lane answers again,
+      // and saying "failed" would invite setting it a second time.
+      notice.value = {
+        type: 'info',
+        text: `Saved. ${lane.display_name} is not answering right now — `
+          + 'it will pick this up within a few minutes.',
+      }
+    }
+  } catch (e) {
+    // The endpoint's `detail` is written to be read by a person ("cap 10000 is
+    // above what this container can hold"). Surface it rather than a status
+    // code, and never swallow it — a control that silently does nothing is
+    // worse than one that refuses out loud.
+    //
+    // `e.body`, not `e.detail`: ApiError puts the parsed response on `.body`
+    // and sets `.message` to the short `error` key.
+    notice.value = { type: 'error', text: e.body?.detail || e.message }
+  } finally {
+    busy.value = null
+  }
+}
+
+// The cap is the only thing an operator sets, and it doubles as the switch:
+// zero means no consumers. How many workers actually run is the sizing pass's
+// business — always on, reading the live pool every minute — so nothing here
+// sends a worker count.
+function step(lane, delta) {
+  return apply(lane, { slots_cap: lane.slots_cap + delta })
+}
 </script>
 
 <style scoped>
-.fc-sys__lede, .fc-sys__muted, .fc-sys__checked, .fc-sys__foot {
+/* These lived in app.css for one commit, globalised so the roster and the
+   worker lanes could share them. Merging the two into one table left a single
+   consumer, so they come back here — a global utility with one caller is a
+   rule nobody can find from the markup that uses it. */
+.fc-sys__lede, .fc-sys__muted, .fc-sys__checked {
   color: rgb(var(--v-theme-on-surface) / 0.66);
 }
 .fc-sys__checked { font-size: 0.78rem; }
-.fc-sys__card { background: rgb(var(--v-theme-on-surface) / 0.04); }
-
-.fc-sys__row {
-  display: flex; align-items: center; gap: 12px;
-  padding: 12px 16px;
-  border-bottom: 1px solid rgb(var(--v-theme-on-surface) / 0.08);
-}
-.fc-sys__row:last-child { border-bottom: 0; }
 
 .fc-sys__dot { width: 9px; height: 9px; border-radius: 50%; flex: 0 0 auto; }
 .fc-sys__dot--ok      { background: rgb(var(--v-theme-success)); }
@@ -112,17 +348,63 @@ onMounted(() => { store.refresh() })
 .fc-sys__dot--down    { background: rgb(var(--v-theme-error)); }
 .fc-sys__dot--unknown { background: rgb(var(--v-theme-on-surface) / 0.35); }
 
-.fc-sys__body { min-width: 0; flex: 1 1 auto; }
-.fc-sys__name { font-weight: 600; }
 .fc-sys__kind {
-  margin-left: 8px; font-weight: 400; font-size: 0.72rem; text-transform: uppercase;
+  font-weight: 400; font-size: 0.72rem; text-transform: uppercase;
   letter-spacing: 0.04em; color: rgb(var(--v-theme-on-surface) / 0.5);
 }
-.fc-sys__detail { font-size: 0.82rem; color: rgb(var(--v-theme-on-surface) / 0.72); }
 
-.fc-sys__meta {
-  text-align: right; font-size: 0.75rem; flex: 0 0 auto;
-  font-variant-numeric: tabular-nums; color: rgb(var(--v-theme-on-surface) / 0.6);
+/* Bounded rather than free-form (operator 2026-09-23). What was here before was
+   dotted rows floating on the page background with no header and no edge, so
+   nothing said where the list began or what any column meant. */
+.fc-parts {
+  background: rgb(var(--v-theme-on-surface) / 0.04);
+  border: 1px solid rgb(var(--v-theme-on-surface) / 0.12);
+  border-radius: 6px;
+  overflow: hidden;
+}
+.fc-parts__table :deep(thead th) {
+  font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.05em;
+  color: rgb(var(--v-theme-on-surface) / 0.55);
+  border-bottom: 1px solid rgb(var(--v-theme-on-surface) / 0.18) !important;
+  white-space: nowrap;
+}
+.fc-parts__table :deep(tbody td) {
+  border-bottom: 1px solid rgb(var(--v-theme-on-surface) / 0.08) !important;
+  padding-top: 8px; padding-bottom: 8px;
+}
+.fc-parts__table :deep(tbody tr:last-child td) { border-bottom: 0 !important; }
+
+.fc-parts__name {
+  display: flex; align-items: center; gap: 8px; font-weight: 600;
+}
+/* Clears the dot, so the sentence lines up under the name rather than under
+   the indicator. */
+.fc-parts__detail {
+  font-size: 0.8rem; color: rgb(var(--v-theme-on-surface) / 0.72);
+  padding-left: 17px;
+}
+.fc-parts__advice {
+  font-size: 0.8rem; padding-left: 17px; margin-top: 2px;
+  color: rgb(var(--v-theme-accent));
+}
+.fc-parts__queues {
+  font-size: 0.78rem; color: rgb(var(--v-theme-on-surface) / 0.6);
+}
+.fc-parts__num { font-variant-numeric: tabular-nums; }
+.fc-parts__sub {
+  font-size: 0.68rem; text-transform: uppercase; letter-spacing: 0.04em;
+  color: rgb(var(--v-theme-on-surface) / 0.5); white-space: nowrap;
+}
+
+/* Fixed width so the number does not shift the +/- buttons as it changes
+   between one and two digits. */
+.fc-parts__slots {
+  display: inline-block; min-width: 1.75em; text-align: center;
+  font-variant-numeric: tabular-nums;
+}
+
+.fc-parts__legend {
+  font-size: 0.82rem; line-height: 1.55;
+  color: rgb(var(--v-theme-on-surface) / 0.66);
 }
-.fc-sys__queues { opacity: 0.75; }
 </style>

frontend/src/components/subscriptions/MembershipReconcileCard.vue

@@ -1,12 +1,20 @@
 <template>
   <!-- Renders nothing when there is nothing to say — same posture as
        NeedsAttentionCard. A reconciliation card that always shows would train
-       the operator to scroll past it. -->
-  <v-card v-if="anythingToSay" variant="tonal" class="mb-4 fc-recon">
+       the operator to scroll past it. It can also be DISMISSED, and stays
+       dismissed until what it would say changes. -->
+  <v-card v-if="visible" variant="tonal" class="mb-4 fc-recon">
     <v-card-text>
+      <v-btn
+        icon="mdi-close" size="small" variant="text" class="fc-recon__dismiss"
+        title="Hide until something changes" aria-label="Hide until something changes"
+        @click="dismiss"
+      />
       <div v-for="p in interesting" :key="p.platform" class="fc-recon__platform">
-        <!-- Bucket 1: the adoption win. The only direction with an action,
-             because adding a source is the reversible half. -->
+        <!-- The adoption win: subscriptions FC doesn't follow yet. The only
+             bucket shown. Sources you follow but no longer pay for are not
+             listed here (operator, 2026-09-13) — the membership sweep stops
+             pulling those instead (#3995). -->
         <template v-if="p.subscribed_not_tracked.length">
           <div class="fc-recon__head">
             <v-icon icon="mdi-account-plus-outline" size="small" class="me-2" />
@@ -36,37 +44,13 @@
           </div>
         </template>
 
-        <!-- Bucket 2: REPORT ONLY. No disable control here by design — the
-             source list on this same page is where that decision belongs. -->
-        <template v-if="p.tracked_not_subscribed.length">
-          <div class="fc-recon__head">
-            <v-icon icon="mdi-help-circle-outline" size="small" class="me-2" />
-            <strong>
-              {{ p.tracked_not_subscribed.length }}
-              {{ p.platform }}
-              {{ p.tracked_not_subscribed.length === 1 ? 'source' : 'sources' }}
-              your roster doesn't account for
-            </strong>
-          </div>
-          <div v-for="s in p.tracked_not_subscribed" :key="s.id" class="fc-recon__row">
-            <div class="fc-recon__body">
-              <strong>{{ s.artist.name }}</strong>
-              <div class="fc-recon__dim">{{ reasonFor(s) }}</div>
-            </div>
-          </div>
-          <div class="fc-recon__dim fc-recon__note">
-            Nothing has been changed. If you want one of these to stop checking,
-            disable it in the list below.
-          </div>
-        </template>
-
-        <!-- The roster could not be trusted, so bucket 2 was not computed. Said
-             in words: an empty list here must never read as "all clear". -->
+        <!-- The roster could not be trusted. Said in words: an empty list here
+             must never read as "all clear". -->
         <div v-if="!p.fresh" class="fc-recon__dim fc-recon__note">
           <v-icon icon="mdi-clock-alert-outline" size="small" class="me-1" />
           <template v-if="!p.last_success_at">
             Your {{ p.platform }} roster has never synced, so FC can't tell which
-            sources you still subscribe to.
+            creators you subscribe to.
           </template>
           <template v-else>
             Your {{ p.platform }} roster last synced
@@ -81,44 +65,62 @@
 </template>
 
 <script setup>
-import { computed, onMounted } from 'vue'
+import { computed, onMounted, ref } from 'vue'
 
 import { formatRelative } from '../../utils/date.js'
 import { useMembershipReconcileStore } from '../../stores/membershipReconcile.js'
 
-// #387 C4. Two directions, deliberately unequal: subscriptions FC doesn't
-// follow get a one-click add, while sources the roster doesn't account for are
-// REPORTED ONLY (operator decision, 2026-09-11). The asymmetry is the point —
-// adding a source is trivially undone, and "you no longer subscribe to this" is
-// computed from an absence that has three possible causes.
+// #387 C4. Offers the subscriptions FC doesn't follow yet. The other direction
+// (sources FC follows that the roster says you no longer pay for) used to be
+// listed here as a report. The operator didn't want it listed (2026-09-13); the
+// membership sweep stops pulling those sources instead (#3995).
 const store = useMembershipReconcileStore()
 
 // An untrustworthy roster is only worth mentioning when there is something it
 // would have reconciled — otherwise a failed sweep on a platform with no
 // sources yet would put a warning on a page with nothing to warn about.
 const interesting = computed(() => store.platforms.filter(
-  p => p.subscribed_not_tracked.length
-    || p.tracked_not_subscribed.length
-    || (!p.fresh && p.tracked_total)
+  p => p.subscribed_not_tracked.length || (!p.fresh && p.tracked_total)
 ))
-const anythingToSay = computed(() => interesting.value.length > 0)
 
-function reasonFor (s) {
-  if (s.basis === 'lapsed') {
-    return `Your membership reads "${s.membership?.status}" — you're not a paying supporter of this creator right now.`
-  }
-  if (s.basis === 'absent_exact') {
-    return "FC knows this creator's id on the platform, and it isn't in your roster."
-  }
-  // absent_handle — the weakest claim, and it says so. A renamed creator looks
-  // exactly like this, which is why it is not phrased as a conclusion.
-  return "No membership matched this source's address. It may simply have been renamed."
+// Dismissal is keyed to WHAT the card says, not to the card: a fingerprint of
+// the offered memberships and any stale-roster warnings. Dismissing hides this
+// exact set; a new subscription (or a roster going stale) changes the
+// fingerprint and brings the card back. Per-browser, which is enough for a
+// single-operator instance.
+const DISMISS_KEY = 'fc.recon.dismissed'
+const fingerprint = computed(() => interesting.value
+  .flatMap(p => [
+    ...p.subscribed_not_tracked.map(m => `${p.platform}:${m.id}`),
+    ...(p.fresh ? [] : [`${p.platform}:stale`]),
+  ])
+  .sort()
+  .join('|'))
+
+function readDismissed () {
+  try { return localStorage.getItem(DISMISS_KEY) } catch { return null }
 }
+const dismissed = ref(readDismissed())
+
+function dismiss () {
+  dismissed.value = fingerprint.value
+  try { localStorage.setItem(DISMISS_KEY, fingerprint.value) } catch { /* private mode */ }
+}
+
+const visible = computed(
+  () => interesting.value.length > 0 && fingerprint.value !== dismissed.value,
+)
 
 onMounted(() => { store.load() })
 </script>
 
 <style scoped>
+.fc-recon { position: relative; }
+.fc-recon__dismiss {
+  position: absolute;
+  top: 6px;
+  right: 6px;
+}
 .fc-recon__platform + .fc-recon__platform {
   margin-top: 16px;
 }

frontend/src/components/subscriptions/SourceRow.vue

@@ -39,8 +39,25 @@
       {{ formatRelative(source.next_check_at, { future: true }) }}
     </td>
     <td>
+      <!-- #3995: the membership sweep stopped this source because paid access
+           ended. First in the chain — it explains why an otherwise healthy
+           source is off. Neutral, never error: nothing is broken. -->
       <v-chip
-        v-if="(source.consecutive_failures || 0) > 0"
+        v-if="membershipStopped"
+        size="x-small" variant="tonal" label
+        prepend-icon="mdi-account-cancel-outline"
+      >Membership ended
+        <v-tooltip activator="parent" location="top" max-width="420">
+          <span>
+            Curator stopped checking this source because your membership
+            {{ membershipStopped.status ? `reads "${membershipStopped.status}"` : 'ended' }}.
+            It starts again on its own if you resubscribe. Turn it back on to
+            keep checking anyway.
+          </span>
+        </v-tooltip>
+      </v-chip>
+      <v-chip
+        v-else-if="(source.consecutive_failures || 0) > 0"
         size="x-small" color="error" variant="tonal" label
       >{{ source.consecutive_failures }}
         <!-- #1: show the actual failure reason on hover instead of a bare count. -->
@@ -109,6 +126,12 @@ const props = defineProps({
 // operator's subscription, not anything FC can retry. The count is only joined
 // in by the list endpoint, so phrase it without one when it's absent rather
 // than rendering a fabricated zero.
+// #3995: set by the membership sweep when it stops a source whose paid access
+// ended; cleared when it resumes, or when the operator toggles the source.
+const membershipStopped = computed(
+  () => props.source.config_overrides?._membership_stopped || null,
+)
+
 const noAccessTip = computed(() => {
   const n = props.source.tier_gated_count
   const what = n

frontend/src/stores/systemActivity.js

@@ -3,6 +3,29 @@ import { ref } from 'vue'
 
 import { useApi } from '../composables/useApi.js'
 
+// Both halves, never one (milestone 422 step 7). Saturation alone is a lane
+// doing its job; a long-running task alone may be the only thing on a lane
+// with slots to spare. Only together do they mean "nothing else on this lane
+// can start", which is the thing worth interrupting someone about.
+//
+// The operator asked for this to TRIGGER growth. It reports instead: a long
+// task does not finish sooner because the lane gained a slot, so autoscaling
+// on it would spend memory to change nothing. Exported rather than left in
+// the card so the AND is pinned by a test — a warning that fires on half the
+// condition is a warning people learn to ignore.
+export const LANE_STUCK_MINUTES = 15
+
+export function laneStuckFor(lane) {
+  const mins = lane?.oldest_running_minutes
+  if (mins === null || mins === undefined || mins < LANE_STUCK_MINUTES) return null
+  // Unknown is not busy: a lane nothing answered for has no live reading to
+  // call saturated, and `pool` of 0 or null is not a full pool.
+  if (!lane.live?.present || !lane.live.pool) return null
+  if (lane.live.active < lane.live.pool) return null
+  if (mins < 120) return `${mins} minutes`
+  return `${Math.floor(mins / 60)} hours`
+}
+
 export const useSystemActivityStore = defineStore('systemActivity', () => {
   const api = useApi()
 
@@ -12,13 +35,21 @@ export const useSystemActivityStore = defineStore('systemActivity', () => {
   const recentRuns = ref([])   // last-60s rows (for Overview summary)
   const failures = ref(null)     // { recent, count_by_type, since }
 
+  // Worker lanes (milestone 422): the configured slots joined to the live
+  // pool. Lives here rather than in its own store because it is the same
+  // domain the queues and workers above describe — a second store polling
+  // /api/system/* would be two things to keep in step.
+  const lanes = ref(null)        // { lanes: [...], fetched_at }
+
   // Paginated runs (Activity tab "All recent activity" pane).
   const runs = ref([])
   const runsCursor = ref(null)
   const runsHasMore = ref(false)
   const runsFilter = ref({ queue: null, status: null, task: null, limit: 50 })
 
-  const loading = ref({ queues: false, workers: false, runs: false, failures: false })
+  const loading = ref({
+    queues: false, workers: false, runs: false, failures: false, lanes: false,
+  })
   const lastError = ref(null)
 
   async function loadQueues() {
@@ -45,6 +76,32 @@ export const useSystemActivityStore = defineStore('systemActivity', () => {
     }
   }
 
+  async function loadLanes() {
+    loading.value.lanes = true
+    lastError.value = null
+    try {
+      lanes.value = await api.get('/api/system/workers')
+    } catch (e) {
+      lastError.value = e.message
+    } finally {
+      loading.value.lanes = false
+    }
+  }
+
+  // Change one lane. Returns the endpoint's reply so the caller can tell a
+  // stored-but-not-yet-live change (`applied: false`) from a live one — the
+  // difference between "saved, the lane is restarting" and "that failed",
+  // which the UI must not collapse into one message.
+  //
+  // Deliberately NOT swallowing the error: a refused value (400) carries the
+  // sentence explaining why, and the card shows it. Returning null on failure
+  // would leave the operator with a control that silently did nothing.
+  async function setLane(name, fields) {
+    const reply = await api.post(`/api/system/workers/${name}`, { body: fields })
+    await loadLanes()
+    return reply
+  }
+
   async function loadRecentRuns() {
     // Used by the Overview summary card: pull last 60s of runs to compute
     // per-queue ok/err counts. One call covers all queues; UI groups.
@@ -107,10 +164,10 @@ export const useSystemActivityStore = defineStore('systemActivity', () => {
   }
 
   return {
-    queues, workers, recentRuns, failures, summary,
+    queues, workers, recentRuns, failures, summary, lanes,
     runs, runsCursor, runsHasMore, runsFilter,
     loading, lastError,
-    loadQueues, loadWorkers, loadRecentRuns,
+    loadQueues, loadWorkers, loadRecentRuns, loadLanes, setLane,
     loadRuns, loadFailures, loadSummary, setFilter,
   }
 })

frontend/src/utils/platformColor.js

@@ -1,9 +1,13 @@
 // Single source of truth for platform → color + icon mapping. Used by
-// PlatformChip and any other GS-style platform-tagged surface. The five
+// PlatformChip and any other GS-style platform-tagged surface. The four
 // platforms FC supports map 1:1 to the GS palette; unknown platforms fall
 // back to grey + mdi-web — which is deliberately what a retired platform
-// hits: a pre-#3069 deviantart source row still renders, as its raw key on
-// a grey chip. Operator-confirmed scope 2026-05-27. The ICONS key set is
+// hits: a pre-#3069 deviantart source row, or a post from pixiv (retired at
+// milestone #406), still renders, as its raw key on a grey chip. So a
+// retired platform is REMOVED from these maps, never kept "so old rows
+// look right" — the fallback is what makes old rows look right, and keeping
+// the entry would break the contract pin below. Operator-confirmed scope
+// 2026-05-27. The ICONS key set is
 // pinned against backend known_platform_keys() by
 // tests/test_fe_be_contract.py.
 
@@ -12,7 +16,6 @@ const ICONS = {
   subscribestar: 'mdi-star',
   hentaifoundry: 'mdi-palette',
   discord:       'mdi-discord',
-  pixiv:         'mdi-alpha-p-box',
 }
 
 const COLORS = {
@@ -20,7 +23,6 @@ const COLORS = {
   subscribestar: 'amber',
   hentaifoundry: 'purple',
   discord:       'indigo',
-  pixiv:         'blue',
 }
 
 const LABELS = {
@@ -28,7 +30,6 @@ const LABELS = {
   subscribestar: 'SubscribeStar',
   hentaifoundry: 'HentaiFoundry',
   discord:       'Discord',
-  pixiv:         'Pixiv',
 }
 
 export function platformIcon(platform) {

frontend/src/utils/systemParts.js

@@ -0,0 +1,144 @@
+// Joining the roster to the worker lanes, for the System tab's one table.
+//
+// Extracted from the component rather than left inline because the failure
+// this can have is SILENT and is exactly the thing the merge exists to fix: if
+// a lane stops matching its roster part, nothing throws — the table simply
+// grows a second row for the same worker, one with controls and one without,
+// which is the duplication the operator asked to be rid of, returned by the
+// code that removed it.
+//
+//   Operator, 2026-09-23: "I feel that we can probably combine the two
+//   sections into a single table."
+
+// A learned roster part and a lane are the same thing seen from two sides, and
+// the QUEUES are what identify it — `service_roster.refresh_celery_roster`
+// keys a celery part on exactly `"celery:" + ",".join(sorted(queues))`.
+//
+// Matched on the sorted set rather than on that string so the join survives a
+// change to how the key is spelled, and so neither side has to agree about
+// ORDER: the lane table lists a lane's queues in the order the role reads them
+// (`default, import, thumbnail, download`) while the roster sorts them
+// (`default, download, import, thumbnail`).
+export function queueKey(queues) {
+  return [...(queues || [])].sort().join(',')
+}
+
+// Worst first. A stopped datastore is why someone opened this tab.
+export const SEVERITY = { down: 3, stale: 2, unknown: 1, ok: 0 }
+
+export function kindLabel(kind) {
+  if (kind === 'celery') return 'worker lane'
+  if (kind === 'agent') return 'GPU agent'
+  if (kind === 'datastore') return 'datastore'
+  return kind
+}
+
+/**
+ * One row per moving part, with a lane attached where there is one.
+ *
+ * @param parts   the roster's parts, as /api/system/health returns them
+ * @param lanes   the lane rows, as /api/system/workers returns them
+ * @param stuckFor a lane -> "40 minutes" | null reporter (laneStuckFor)
+ */
+export function mergeParts(parts, lanes, stuckFor = () => null) {
+  const unmatched = {}
+  for (const lane of lanes || []) unmatched[queueKey(lane.queues)] = lane
+
+  const out = []
+  for (const part of parts || []) {
+    const key = queueKey(part.queues)
+    const lane = part.kind === 'celery' ? unmatched[key] : undefined
+    if (lane) delete unmatched[key]
+    out.push({
+      key: part.key,
+      name: part.name,
+      kindLabel: lane?.optional ? 'optional lane' : kindLabel(part.kind),
+      state: part.state,
+      // A lane capped at zero is OFF, not broken. Say so, rather than let the
+      // roster's heartbeat sentence report the operator's own choice as a
+      // fault — the roster cannot know the difference, and the lane can.
+      detail: lane && lane.slots_cap === 0 ? 'off — cap is zero' : part.detail,
+      queues: (part.queues || []).join(', '),
+      lane,
+      stuckFor: lane ? stuckFor(lane) : null,
+      severity: SEVERITY[part.state] ?? SEVERITY.unknown,
+    })
+  }
+
+  // A lane the roster has not learned yet. Parts appear only once they have
+  // checked in, while the lane table is known up front — so without this, the
+  // lane an operator most needs to find (an optional one, never yet started)
+  // would be the only one missing from the table.
+  for (const lane of Object.values(unmatched)) out.push(laneRow(lane, stuckFor))
+
+  // Severity leads; then lanes ahead of everything else, because they are the
+  // rows you can actually do something about; then by name.
+  return out.sort((a, b) =>
+    b.severity - a.severity
+    || Number(Boolean(b.lane)) - Number(Boolean(a.lane))
+    || a.name.localeCompare(b.name))
+}
+
+function laneRow(lane, stuckFor) {
+  const on = lane.slots_cap > 0
+  let state = 'unknown'
+  if (lane.live?.present) state = on ? (stuckFor(lane) ? 'stale' : 'ok') : 'unknown'
+  else if (on) state = 'down'
+  return {
+    key: `lane:${lane.name}`,
+    name: lane.display_name,
+    kindLabel: lane.optional ? 'optional lane' : 'worker lane',
+    state,
+    detail: lane.live?.present
+      ? (on ? 'running' : 'off — cap is zero')
+      : 'has not checked in yet',
+    queues: (lane.queues || []).join(', '),
+    lane,
+    stuckFor: stuckFor(lane),
+    severity: SEVERITY[state],
+  }
+}
+
+
+// How much has to be waiting before we tell someone to raise a cap.
+//
+// Not "anything at all". A lane at its cap with three items queued is working
+// normally and will be empty in a moment; a notice there is one people learn
+// to scroll past, and at that point it is worse than not having it.
+export const ADVISE_BACKLOG = 10
+
+/**
+ * The sentence that tells an operator the cap is now the limiting factor.
+ *
+ * Operator, 2026-09-23: *"there needs to be something that tells you user to
+ * bump those numbers to improve processing rate or they'd never know the
+ * controls exist."* The defaults are deliberately one-of-each, so on a busy
+ * instance the shipped configuration IS the bottleneck — and a conservative
+ * default nobody knows how to raise is just a slow product.
+ *
+ * Only fires when raising the cap would actually help: there is real work
+ * waiting, every worker the cap allows is already running, and the cap is
+ * below what this machine can hold. A lane already at its ceiling gets
+ * nothing, because there is nothing it could be told to do.
+ */
+export function laneAdvice(lane) {
+  if (!lane) return null
+  const pending = lane.pending
+  if (pending == null || pending < ADVISE_BACKLOG) return null
+
+  if (lane.slots_cap === 0) {
+    return `${pending.toLocaleString()} waiting, and this lane is off. `
+      + 'Raise its cap to start working through them.'
+  }
+  if (lane.ceiling <= lane.slots_cap) {
+    // At the machine's limit, not the operator's. Saying "raise the cap"
+    // here would be advice they cannot take.
+    return null
+  }
+  if (!lane.live?.present || (lane.live.pool ?? 0) < lane.slots_cap) return null
+
+  return `${pending.toLocaleString()} waiting and all ${lane.slots_cap} `
+    + `worker${lane.slots_cap === 1 ? '' : 's'} busy. `
+    + `Raise the cap to run more at once — this machine allows up to `
+    + `${lane.ceiling}.`
+}

frontend/src/views/ArtistsView.vue

@@ -58,8 +58,9 @@ useInfiniteScroll(sentinelEl, () => {
 
 onMounted(async () => {
   await platformsStore.loadAll()
-  platformItems.value = (platformsStore.platforms || []).map(p => ({
-    title: p.key, value: p.key,
+  // `list` — the store has no `platforms` property (test/storeUsage.spec.js).
+  platformItems.value = platformsStore.list.map(p => ({
+    title: p.name || p.key, value: p.key,
   }))
   // Apply any deep-linked q (and load). setQuery resets + fetches.
   store.setQuery(searchTerm.value || '')

frontend/src/views/PostsView.vue

@@ -1,5 +1,8 @@
 <template>
-  <v-container class="pt-2 pb-6" max-width="900">
+  <!-- Width is set in CSS, not with `max-width` here: below 1600px it is
+       today's 900px column, and above it the feed widens and gains the day
+       gutter (milestone #407). -->
+  <v-container fluid class="pt-2 pb-6 fc-posts">
     <!-- In-context view: deep-linked to one post, with bidirectional infinite
          scroll — newer posts load above, older posts below. -->
     <template v-if="postIdFilter != null">
@@ -48,14 +51,19 @@
     </template>
 
     <!-- Normal feed -->
-    <template v-else>
-      <FeedStatusRibbon v-if="statusRibbon" />
-
-      <PostsFilterBar
-        :artist-id="artistFilter"
-        :platform="platformFilter"
-        @update:filters="onFilters"
-      />
+    <div v-else class="fc-posts__main">
+      <!-- Filters and status sit in one row above the feed. #407 option E put
+           them in a sticky left rail; the operator found it wasted space
+           (2026-09-13), so on a wide window this row lines up with the feed
+           column instead, clear of the day gutter. -->
+      <div class="fc-posts__toolbar">
+        <FeedStatusRibbon v-if="statusRibbon" class="fc-posts__status" />
+        <PostsFilterBar
+          :artist-id="artistFilter"
+          :platform="platformFilter"
+          @update:filters="onFilters"
+        />
+      </div>
 
       <v-alert v-if="store.error" type="error" variant="tonal" closable class="mb-3">
         {{ String(store.error) }}
@@ -74,14 +82,27 @@
       </div>
 
       <div v-else>
-        <PostCard v-for="p in store.items" :key="p.id" :post="p" />
+        <!-- Day groups (#407 D). The heading sits above its posts on a narrow
+             window and in a sticky left gutter on a wide one. -->
+        <section v-for="d in days" :key="d.key" class="fc-posts__day">
+          <header class="fc-posts__day-head">
+            <span class="fc-posts__day-label">{{ d.label }}</span>
+            <span class="fc-posts__day-count">
+              {{ d.posts.length }} post{{ d.posts.length === 1 ? '' : 's' }}
+              · {{ d.artistCount }} artist{{ d.artistCount === 1 ? '' : 's' }}
+            </span>
+          </header>
+          <div class="fc-posts__day-posts">
+            <PostCard v-for="p in d.posts" :key="p.id" :post="p" />
+          </div>
+        </section>
 
         <div ref="sentinel" class="fc-posts__sentinel">
           <v-progress-circular v-if="store.loading" indeterminate color="accent" size="24" />
           <span v-else-if="store.done" class="fc-posts__end">End of stream</span>
         </div>
       </div>
-    </template>
+    </div>
   </v-container>
 </template>
 
@@ -121,6 +142,48 @@ const hasActiveFilter = computed(() =>
   artistFilter.value != null || platformFilter.value != null || searchFilter.value != null
 )
 
+// --- day groups (#407 D) ---
+// CONSECUTIVE runs, not a bucket per date. The feed's sort key includes
+// `resurfaced_at` (a Discord grouping that grew moves back to the top), which
+// the payload does not carry, so a resurfaced post can sit above newer ones.
+// Bucketing by date would pull it out of order; a run gives it its own heading
+// where it actually appears. Counts cover what has LOADED, and grow as the
+// infinite scroll fetches more of the same day.
+function dayKey (iso) {
+  const d = new Date(iso)
+  return `${d.getFullYear()}-${d.getMonth()}-${d.getDate()}`
+}
+function dayLabel (iso) {
+  const d = new Date(iso)
+  const today = new Date()
+  const startOf = (x) => new Date(x.getFullYear(), x.getMonth(), x.getDate()).getTime()
+  const days = Math.round((startOf(today) - startOf(d)) / 86400000)
+  if (days === 0) return 'Today'
+  if (days === 1) return 'Yesterday'
+  if (days > 1 && days < 7) return d.toLocaleDateString(undefined, { weekday: 'long' })
+  const sameYear = d.getFullYear() === today.getFullYear()
+  return d.toLocaleDateString(undefined, {
+    month: 'short', day: 'numeric', ...(sameYear ? {} : { year: 'numeric' }),
+  })
+}
+const days = computed(() => {
+  const groups = []
+  for (const p of store.items) {
+    const iso = p.post_date || p.downloaded_at
+    const key = dayKey(iso)
+    let g = groups[groups.length - 1]
+    if (!g || g.dayKey !== key) {
+      // Suffix with the run index so a day that appears twice (see above)
+      // still has a unique v-for key.
+      g = { key: `${key}#${groups.length}`, dayKey: key, label: dayLabel(iso), posts: [], artists: new Set() }
+      groups.push(g)
+    }
+    g.posts.push(p)
+    if (p.artist?.id != null) g.artists.add(p.artist.id)
+  }
+  return groups.map((g) => ({ ...g, artistCount: g.artists.size }))
+})
+
 // Drop only `post_id` and stay where we are — keeps Browse's `tab=posts` (and
 // any active artist/platform scope) intact instead of resetting the surface.
 const allPostsTarget = computed(() => {
@@ -232,6 +295,61 @@ onUnmounted(() => { teardownFeed(); teardownAround() })
 </script>
 
 <style scoped>
+/* Below the breakpoint: today's layout exactly — a 900px column with the
+   filters and ribbon inline above the feed. */
+.fc-posts { max-width: 900px; }
+.fc-posts__day-head {
+  display: flex;
+  align-items: baseline;
+  gap: 10px;
+  padding: 4px 0 8px;
+  color: rgb(var(--v-theme-on-surface-variant));
+}
+.fc-posts__day-label {
+  font-family: 'Fraunces', Georgia, serif;
+  font-weight: 600;
+  font-size: 1rem;
+  color: rgb(var(--v-theme-accent));
+}
+.fc-posts__day-count { font-size: 0.78rem; }
+
+/* Wide window (#407 A + D). Each day's heading moves into a sticky gutter
+   beside its posts; the column widens and the cards switch to their filmstrip
+   layout on their own (PostCard measures itself). The toolbar lines up with the
+   feed column, clear of the gutter. */
+@media (min-width: 1600px) {
+  .fc-posts { max-width: 2080px; }
+  .fc-posts__main { max-width: 2080px; margin: 0 auto; }
+  .fc-posts__toolbar {
+    display: flex;
+    align-items: center;
+    flex-wrap: wrap;
+    gap: 8px 24px;
+    margin-left: calc(150px + 24px);
+    margin-bottom: 8px;
+  }
+  /* Status reads after the filters on a wide row, but stays first when the
+     narrow layout stacks them (it is the front door's headline). */
+  .fc-posts__status { order: 2; padding: 0; }
+  .fc-posts__toolbar :deep(.fc-posts-filters) { padding-bottom: 0; }
+  .fc-posts__day {
+    display: grid;
+    grid-template-columns: 150px minmax(0, 1fr);
+    gap: 0 24px;
+  }
+  .fc-posts__day-head {
+    position: sticky;
+    top: calc(var(--fc-nav-h, 64px) + 16px);
+    align-self: start;
+    flex-direction: column;
+    align-items: flex-end;
+    gap: 2px;
+    padding-top: 12px;
+    text-align: right;
+  }
+  .fc-posts__day-label { font-size: 1.1rem; }
+}
+
 .fc-posts__loading,
 .fc-posts__empty {
   display: flex;

frontend/test/storeUsage.spec.js

@@ -0,0 +1,63 @@
+// Reads of a store property that does not exist.
+//
+// JavaScript returns `undefined` for a missing property instead of failing, and
+// the frontend CI has no type-checker to catch it. `platformsStore.platforms`
+// was read for months by the Latest feed's filter bar and the Artists view. The
+// store exposes `list` and `byKey`, so both platform dropdowns opened empty and
+// only someone clicking them noticed.
+//
+// This scans the source for `platformsStore.<name>` and checks each name
+// against what the store really defines.
+
+import { readdirSync, readFileSync, statSync } from 'node:fs'
+import { join } from 'node:path'
+import { fileURLToPath } from 'node:url'
+
+import { createPinia, setActivePinia } from 'pinia'
+import { describe, expect, it } from 'vitest'
+
+import { usePlatformsStore } from '../src/stores/platforms.js'
+
+const SRC = fileURLToPath(new URL('../src', import.meta.url))
+const READ = /\bplatformsStore\.([A-Za-z_$][\w$]*)/g
+
+function sourceFiles (dir) {
+  return readdirSync(dir).flatMap((name) => {
+    const path = join(dir, name)
+    if (statSync(path).isDirectory()) return sourceFiles(path)
+    return /\.(vue|js)$/.test(name) ? [path] : []
+  })
+}
+
+function unknownReads (text, known) {
+  return [...text.matchAll(READ)].map(m => m[1]).filter(name => !known.has(name))
+}
+
+function storeKeys () {
+  setActivePinia(createPinia())
+  return new Set(Object.keys(usePlatformsStore()))
+}
+
+describe('platforms store usage', () => {
+  it('every platformsStore.<name> read in src names something the store defines', () => {
+    const known = storeKeys()
+    const offenders = sourceFiles(SRC).flatMap((file) =>
+      unknownReads(readFileSync(file, 'utf8'), known)
+        .map(name => `${file.slice(SRC.length + 1)}: platformsStore.${name}`),
+    )
+    expect(offenders).toEqual([])
+  })
+
+  it('the scan finds the read that shipped broken (positive control)', () => {
+    const known = storeKeys()
+    expect(known.has('list')).toBe(true)
+    expect(unknownReads('(platformsStore.platforms || []).map(p => p)', known))
+      .toEqual(['platforms'])
+  })
+
+  it('the scan actually walks the source tree', () => {
+    const hasRead = /\bplatformsStore\./
+    const withReads = sourceFiles(SRC).filter(f => hasRead.test(readFileSync(f, 'utf8')))
+    expect(withReads.length).toBeGreaterThan(2)
+  })
+})

frontend/test/systemParts.spec.js

@@ -0,0 +1,192 @@
+import { describe, expect, it } from 'vitest'
+
+import { laneAdvice, mergeParts, queueKey } from '../src/utils/systemParts.js'
+
+// The System tab's one table (operator 2026-09-23: "combine the two sections
+// into a single table"). What is pinned here is the JOIN, because its failure
+// is silent and is precisely the thing the merge exists to fix: a lane that
+// stops matching its roster part does not throw — the table grows a SECOND row
+// for the same worker, one with controls and one without.
+
+const PART = {
+  key: 'celery:default,download,import,thumbnail',
+  kind: 'celery',
+  name: 'Worker',
+  state: 'ok',
+  detail: 'Worker is running',
+  // The roster SORTS a celery part's queues into its key.
+  queues: ['default', 'download', 'import', 'thumbnail'],
+}
+
+const LANE = {
+  name: 'worker',
+  display_name: 'Worker',
+  // The lane table lists them in the order the role reads them, which is NOT
+  // sorted. If the join ever compares these two lists directly rather than as
+  // sets, this fixture is what catches it.
+  queues: ['default', 'import', 'thumbnail', 'download'],
+  slots_cap: 4,
+  ceiling: 8,
+  live: { present: true, replicas: 1, pool: 2, active: 0 },
+  pending: 0,
+}
+
+const POSTGRES = {
+  key: 'postgres', kind: 'datastore', name: 'PostgreSQL',
+  state: 'ok', detail: 'answering', latency_ms: 2.5,
+}
+
+describe('queueKey', () => {
+  it('does not care what order either side lists its queues in', () => {
+    expect(queueKey(LANE.queues)).toBe(queueKey(PART.queues))
+  })
+
+  it('survives a part that has no queues at all', () => {
+    // A datastore, and also the stale `Worker ()` row a previous deployment
+    // left in the roster with an empty queue set. Neither must match a lane.
+    expect(queueKey(undefined)).toBe('')
+    expect(queueKey([])).toBe('')
+  })
+})
+
+describe('mergeParts', () => {
+  it('gives a worker ONE row, carrying its controls', () => {
+    const rows = mergeParts([PART, POSTGRES], [LANE])
+
+    expect(rows).toHaveLength(2)
+    const worker = rows.find((r) => r.name === 'Worker')
+    expect(worker.lane).toBe(LANE)
+    expect(rows.filter((r) => r.name === 'Worker')).toHaveLength(1)
+  })
+
+  it('leaves a datastore without a lane rather than guessing one', () => {
+    const pg = mergeParts([PART, POSTGRES], [LANE]).find((r) => r.key === 'postgres')
+    expect(pg.lane).toBeUndefined()
+  })
+
+  it('still lists a lane the roster has never seen', () => {
+    // Parts are learned as they appear; the lane table is known up front. The
+    // lane an operator most needs to find — an optional one, never started —
+    // is exactly the one with no roster entry.
+    const ml = {
+      ...LANE, name: 'ml', display_name: 'ML tagging', queues: ['ml'],
+      slots_cap: 0, optional: true,
+      live: { present: false, replicas: 0, pool: null, active: 0 },
+    }
+    const rows = mergeParts([POSTGRES], [ml])
+
+    const row = rows.find((r) => r.name === 'ML tagging')
+    expect(row.lane).toBe(ml)
+    expect(row.kindLabel).toBe('optional lane')
+  })
+
+  it('does not call a lane capped at zero broken', () => {
+    // The roster only knows a heartbeat age, so it goes on saying "is running"
+    // for a lane the operator deliberately dialled to nothing. The lane knows
+    // the difference; reporting the operator's own choice as a fault is how an
+    // indicator stops being read.
+    const off = { ...LANE, slots_cap: 0 }
+    const row = mergeParts([PART], [off])[0]
+
+    expect(row.detail).toBe('off — cap is zero')
+  })
+
+  it('puts the broken thing first, whatever it is', () => {
+    const down = { ...POSTGRES, state: 'down', detail: 'not answering' }
+    const rows = mergeParts([PART, down], [LANE])
+
+    expect(rows[0].name).toBe('PostgreSQL')
+  })
+
+  it('otherwise puts the rows you can act on first', () => {
+    const rows = mergeParts([POSTGRES, PART], [LANE])
+
+    expect(rows.map((r) => r.name)).toEqual(['Worker', 'PostgreSQL'])
+  })
+
+  it('reports a wedged lane, and only through the reporter it was given', () => {
+    // `laneStuckFor` is passed in rather than imported, so this file does not
+    // re-test the store's rule — it tests that the merge asks.
+    const asked = []
+    const rows = mergeParts([PART], [LANE], (lane) => {
+      asked.push(lane.name)
+      return '40 minutes'
+    })
+
+    expect(asked).toEqual(['worker'])
+    expect(rows[0].stuckFor).toBe('40 minutes')
+    // The roster still owns a matched row's state — `stuckFor` is a note
+    // beside it, not a verdict that overrides the heartbeat.
+    expect(rows[0].state).toBe('ok')
+  })
+
+  it('handles an empty everything without inventing rows', () => {
+    expect(mergeParts([], [])).toEqual([])
+    expect(mergeParts(undefined, undefined)).toEqual([])
+  })
+})
+
+
+// The nudge. Operator, 2026-09-23: *"there needs to be something that tells
+// you user to bump those numbers to improve processing rate or they'd never
+// know the controls exist."*
+//
+// The caps ship at one of each, so on a busy instance the SHIPPED
+// CONFIGURATION is the bottleneck. What is pinned here is that it fires only
+// when raising the cap would actually help — a notice on a lane that is
+// keeping up, or one already at the machine's limit, is a notice people learn
+// to scroll past, and at that point it is worse than not having it.
+
+describe('laneAdvice', () => {
+  const busyAtCap = {
+    slots_cap: 1, ceiling: 7, pending: 4060,
+    live: { present: true, pool: 1, active: 1 },
+  }
+
+  it('speaks when the cap is the limiting factor', () => {
+    const advice = laneAdvice(busyAtCap)
+    expect(advice).toContain('4,060 waiting')
+    expect(advice).toContain('Raise the cap')
+    // The headroom, so it is an instruction rather than a complaint.
+    expect(advice).toContain('7')
+  })
+
+  it('says something different about a lane that is switched off', () => {
+    const advice = laneAdvice({ ...busyAtCap, slots_cap: 0 })
+    expect(advice).toContain('this lane is off')
+  })
+
+  it('stays quiet when the lane is keeping up', () => {
+    expect(laneAdvice({ ...busyAtCap, pending: 2 })).toBeNull()
+  })
+
+  it('stays quiet when the cap is not the thing holding it back', () => {
+    // Four allowed, two running — the sizing pass has room it has not taken,
+    // so the queue is not the cap's fault.
+    expect(laneAdvice({
+      ...busyAtCap, slots_cap: 4, live: { present: true, pool: 2, active: 2 },
+    })).toBeNull()
+  })
+
+  it('stays quiet at the machine ceiling, where the advice is untakeable', () => {
+    // The one case where saying "raise the cap" would send someone to a
+    // control that will refuse them.
+    expect(laneAdvice({ ...busyAtCap, ceiling: 1 })).toBeNull()
+  })
+
+  it('stays quiet about a lane that is not answering', () => {
+    // An unswept read is not a verdict: nothing replied, so "all workers
+    // busy" is a claim nobody made.
+    expect(laneAdvice({
+      ...busyAtCap, live: { present: false, pool: null, active: 0 },
+    })).toBeNull()
+  })
+
+  it('stays quiet when the backlog is unknown rather than reading it as huge', () => {
+    expect(laneAdvice({ ...busyAtCap, pending: null })).toBeNull()
+  })
+
+  it('says nothing about a row that is not a lane at all', () => {
+    expect(laneAdvice(undefined)).toBeNull()
+  })
+})

frontend/test/workerLanes.spec.js

@@ -0,0 +1,192 @@
+import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'
+import { setActivePinia, createPinia } from 'pinia'
+import { laneStuckFor, useSystemActivityStore } from '../src/stores/systemActivity.js'
+
+// Milestone 422 step 4. Covers the store half of the worker-lane dial — the
+// part that decides what the card can tell the operator.
+//
+// The distinction being protected: a change that was STORED but not pushed
+// (`applied: false`, because the lane is restarting) is not a failure, and a
+// REFUSED value (400) is. Collapsing those two into one message is how a
+// control stops being trustworthy — one invites waiting, the other invites
+// changing what you asked for.
+
+function stubFetch(handler) {
+  globalThis.fetch = vi.fn(async (url, init) => {
+    const { status, body } = handler(url, init)
+    return {
+      ok: status >= 200 && status < 300,
+      status,
+      statusText: String(status),
+      text: async () => (body == null ? '' : JSON.stringify(body)),
+    }
+  })
+}
+
+const LANES_BODY = {
+  lanes: [
+    {
+      name: 'worker', display_name: 'Worker',
+      queues: ['default', 'import', 'thumbnail', 'download'],
+      slots_cap: 1, ceiling: 8, enabled: true, memory_bound: false,
+      live: { present: true, replicas: 1, pool: 1, active: 0, reserved: 3 },
+      queue_depth: 5, pending: 8,
+    },
+    {
+      name: 'ml', display_name: 'ML tagging', queues: ['ml'],
+      slots_cap: 0, ceiling: 2, enabled: false, memory_bound: true,
+      live: { present: false, replicas: 0, pool: null, active: 0, reserved: 0 },
+      queue_depth: null, pending: null,
+    },
+  ],
+  fetched_at: '2026-09-22T12:00:00Z',
+}
+
+describe('worker lanes store', () => {
+  beforeEach(() => setActivePinia(createPinia()))
+  afterEach(() => vi.restoreAllMocks())
+
+  it('loads the lanes', async () => {
+    stubFetch(() => ({ status: 200, body: LANES_BODY }))
+    const s = useSystemActivityStore()
+    await s.loadLanes()
+    expect(s.lanes.lanes.map((l) => l.name)).toEqual(['worker', 'ml'])
+  })
+
+  it('a load failure records the error rather than throwing at the caller', async () => {
+    // The System tab polls this every 15s. An unhandled rejection per tick
+    // would drown the console and stop the other pollers in the same
+    // function.
+    stubFetch(() => ({ status: 500, body: { error: 'boom' } }))
+    const s = useSystemActivityStore()
+    await expect(s.loadLanes()).resolves.toBeUndefined()
+    expect(s.lastError).toBeTruthy()
+  })
+
+  it('setLane posts only the fields it was given', async () => {
+    // One field: the cap. How many workers are running is a measurement the
+    // sizing pass owns, so there is nothing else for the UI to send.
+    const calls = []
+    stubFetch((url, init) => {
+      calls.push({ url, init })
+      if (init?.method === 'POST') {
+        return { status: 200, body: { name: 'worker', slots_cap: 2, applied: true } }
+      }
+      return { status: 200, body: LANES_BODY }
+    })
+    const s = useSystemActivityStore()
+    await s.setLane('worker', { slots_cap: 2 })
+
+    const post = calls.find((c) => c.init?.method === 'POST')
+    expect(post.url).toContain('/api/system/workers/worker')
+    expect(JSON.parse(post.init.body)).toEqual({ slots_cap: 2 })
+  })
+
+  it('setLane refetches so the card shows the server truth, not the guess', async () => {
+    // The reply is one lane; the table renders all of them plus live pool
+    // and pending. Patching the local row from the reply would leave every
+    // other column stale and eventually wrong.
+    let gets = 0
+    stubFetch((url, init) => {
+      if (init?.method === 'POST') return { status: 200, body: { applied: true } }
+      gets += 1
+      return { status: 200, body: LANES_BODY }
+    })
+    const s = useSystemActivityStore()
+    await s.setLane('worker', { slots_cap: 2 })
+    expect(gets).toBe(1)
+  })
+
+  it('a stored-but-unapplied change comes back as applied:false, not an error', async () => {
+    // The lane is restarting. The value IS saved and the reconcile will carry
+    // it — so this must reach the card as information, not as a failure that
+    // invites the operator to set it again.
+    stubFetch((url, init) => {
+      if (init?.method === 'POST') {
+        return {
+          status: 200,
+          body: { applied: false, apply_error: 'lane is not running', slots_cap: 2 },
+        }
+      }
+      return { status: 200, body: LANES_BODY }
+    })
+    const s = useSystemActivityStore()
+    const reply = await s.setLane('worker', { slots_cap: 2 })
+    expect(reply.applied).toBe(false)
+    expect(reply.apply_error).toContain('not running')
+  })
+
+  it('a refused value throws so the card can show the reason', async () => {
+    // Deliberately NOT swallowed. The detail is written to be read by a person
+    // ("above what this container can hold"), and a control that silently does
+    // nothing is worse than one that refuses out loud.
+    stubFetch((url, init) => {
+      if (init?.method === 'POST') {
+        return {
+          status: 400,
+          body: { error: 'refused', detail: 'cap 10000 is above what this container can hold (2 for ML tagging)' },
+        }
+      }
+      return { status: 200, body: LANES_BODY }
+    })
+    const s = useSystemActivityStore()
+    // Assert the REASON is reachable, not merely that it threw. `toThrow()`
+    // alone passes whether the card can read the sentence or not — which is
+    // how the first version of the card shipped reading `e.detail` (always
+    // undefined) and would have shown the operator the bare word "refused".
+    const err = await s.setLane('ml', { slots_cap: 10000 }).catch((e) => e)
+    expect(err.status).toBe(400)
+    expect(err.body.detail).toContain('container can hold')
+  })
+})
+
+// --- the long-running-task warning (step 7) ----------------------------------
+//
+// The operator asked for "a task runs for x concurrent time" to TRIGGER
+// growth. It became a warning: extra slots do not make a running task finish
+// sooner. What is pinned here is the AND — a warning that fires on half its
+// condition is one people learn to ignore, and at that point it is worse than
+// not having it.
+
+describe('laneStuckFor', () => {
+  const lane = (over = {}) => ({
+    oldest_running_minutes: 40,
+    live: { present: true, pool: 4, active: 4 },
+    ...over,
+  })
+
+  it('warns when every slot is busy and the oldest task is old', () => {
+    expect(laneStuckFor(lane())).toBe('40 minutes')
+  })
+
+  it('says nothing when the lane has a free slot', () => {
+    expect(laneStuckFor(lane({ live: { present: true, pool: 4, active: 3 } })))
+      .toBeNull()
+  })
+
+  it('says nothing for a saturated lane whose tasks are young', () => {
+    expect(laneStuckFor(lane({ oldest_running_minutes: 2 }))).toBeNull()
+  })
+
+  it('says nothing when nothing is running', () => {
+    expect(laneStuckFor(lane({ oldest_running_minutes: null }))).toBeNull()
+  })
+
+  it('says nothing about a lane that is not answering', () => {
+    // Unknown is not busy. A lane nothing answered for has no live reading to
+    // call saturated, and asserting one would be a verdict from an unswept
+    // read — the same distinction `present` exists for everywhere else here.
+    expect(laneStuckFor(lane({ live: { present: false, pool: null, active: 0 } })))
+      .toBeNull()
+  })
+
+  it('says nothing when the pool is zero', () => {
+    // A lane with no workers is not "fully busy at zero" — it is off.
+    expect(laneStuckFor(lane({ live: { present: true, pool: 0, active: 0 } })))
+      .toBeNull()
+  })
+
+  it('switches to hours once minutes stop being readable', () => {
+    expect(laneStuckFor(lane({ oldest_running_minutes: 195 }))).toBe('3 hours')
+  })
+})

requirements-ml.txt

@@ -3,9 +3,9 @@
 # ML stack — versions current as of 2026-05-14 with Python 3.14 wheel coverage.
 
 # torch + torchvision are NOT listed here: they are installed CPU-only from
-# the PyTorch CPU index in Dockerfile.ml. The default PyPI torch wheel bundles
+# the PyTorch CPU index in Dockerfile. The default PyPI torch wheel bundles
 # the NVIDIA CUDA runtime (a ~5.6GB image layer); this pipeline is CPU-only,
-# so Dockerfile.ml uses the +cpu wheels from
+# so Dockerfile uses the +cpu wheels from
 # https://download.pytorch.org/whl/cpu instead.
 #
 # IMPORTANT: torchvision 0.27 declares requires_python "!=3.14.1,>=3.10" —

requirements.txt

@@ -43,3 +43,14 @@ py7zr>=1,<2
 # #830). Handles Drive's confirm-token + virus-scan interstitial. mega.nz uses
 # the `megatools` binary instead (Debian apt pkg in the runtime image, not pip).
 gdown>=6,<7
+
+# Process supervisor for the single-container layout (milestone 422 step 5).
+# `entrypoint.sh all` generates its config from the lane table and execs it as
+# PID 1, running hypercorn plus one celery process per lane in one container.
+# Unused by the multi-service compose path, where docker supervises instead.
+#
+# Chosen over s6-overlay because it is a pip install on an image that is
+# already Python, and gives per-program stop timeouts plus stopasgroup —
+# celery's prefork pool forks children, and a TERM that reaches only the
+# parent leaves them orphaned holding tasks.
+supervisor>=4.2,<5

scripts/artifacts.sh

@@ -8,7 +8,7 @@
 # never reaches the image — see DERIVER below, and #3156 for the same finding
 # about packaging.sh.
 #
-# Four artifacts, four independent versions. An artifact whose shipped files
+# Three artifacts, three independent versions. An artifact whose shipped files
 # did not change keeps its version and does not rebuild — that is the whole
 # point, and it is why each path set must match its Dockerfile rather than
 # being a plausible guess. Getting a set wrong is quiet in BOTH directions:
@@ -49,11 +49,7 @@ ROOT=$(git rev-parse --show-toplevel)
 # frontend/public/extension/ before the docker build), so an extension change
 # changes the web image. The extension's packaged set is appended in cmd_paths
 # rather than restated — one definition, per #2397.
-WEB_PATHS='Dockerfile requirements.txt backend alembic alembic.ini entrypoint.sh frontend :(exclude)frontend/test :(exclude)frontend/test/**'
-
-# ml (Dockerfile.ml, context `.`) — no frontend, no extension. Note it copies
-# BOTH requirements-ml.txt and requirements.txt.
-ML_PATHS='Dockerfile.ml requirements-ml.txt requirements.txt backend alembic alembic.ini entrypoint.sh'
+WEB_PATHS='Dockerfile requirements.txt requirements-ml.txt backend alembic alembic.ini entrypoint.sh frontend :(exclude)frontend/test :(exclude)frontend/test/**'
 
 # agent (agent/Dockerfile, context `agent`) — copies requirements.txt and
 # fc_agent only. agent/README.md, agent/docker-compose.yml and agent/ruff.toml
@@ -67,7 +63,7 @@ AGENT_PATHS='agent/Dockerfile agent/requirements.txt agent/fc_agent'
 #
 # Why web and nothing else. Every artifact stamps `fc.revision`, but only web
 # also stamps a version (build.yml line ~488 feeds `version web` to the
-# FC_VERSION build arg; ml and agent ask for `revision` alone, and the
+# FC_VERSION build arg; the agent asks for `revision` alone, and the
 # extension takes its version from packaging.sh). For a revision-only artifact
 # this file needs no entry: any change to how the revision is COMPUTED changes
 # the derived value, which then disagrees with the label on the published image
@@ -90,7 +86,7 @@ DERIVER='scripts/artifacts.sh'
 
 
 usage() {
-  echo "usage: artifacts.sh {paths|revision|version|epoch} {web|ml|agent|extension}" >&2
+  echo "usage: artifacts.sh {paths|revision|version|epoch} {web|agent|extension}" >&2
   exit 2
 }
 
@@ -104,7 +100,6 @@ ext_paths() {
 cmd_paths() {
   case "$1" in
     web)       echo "$WEB_PATHS $DERIVER $(ext_paths)" ;;
-    ml)        echo "$ML_PATHS" ;;
     agent)     echo "$AGENT_PATHS" ;;
     extension) ext_paths ;;
     *)         usage ;;
@@ -115,8 +110,21 @@ cmd_paths() {
 # Unquoted on purpose: the pathspec must word-split into separate args.
 # Globbing is already off script-wide.
 newest() {
-  # shellcheck disable=SC2046
-  set -- "$(cd "$ROOT" && git log --format='%ct %H' HEAD -- $(cmd_paths "$1") \
+  # Resolve the path set into a variable rather than inlining it as
+  # `git log ... -- $(cmd_paths "$1")`. An empty substitution there leaves
+  # `git log HEAD --` with NO pathspec, which answers with the newest commit
+  # in the whole repository instead of failing — see the dispatch guard at the
+  # foot of this file for what that cost and why the real check lives there.
+  #
+  # This `|| exit 2` cannot be that check (it exits a subshell, since every
+  # caller wraps `newest` in one) — it is belt and braces for a future caller
+  # that reaches `newest` without passing the guard.
+  paths=$(cmd_paths "$1") || exit 2
+
+  # Unquoted on purpose: the pathspec must word-split into separate args.
+  # Globbing is off script-wide (`set -f`).
+  # shellcheck disable=SC2086
+  set -- "$(cd "$ROOT" && git log --format='%ct %H' HEAD -- $paths \
             | sort -n | tail -1)"
   if [ -z "$1" ]; then
     echo "artifacts.sh: no commit touches this artifact's shipped files" >&2
@@ -213,6 +221,28 @@ cmd_version() {
 }
 
 [ $# -ge 2 ] || usage
+
+# Validate the ARTIFACT here, in the main shell, before anything dispatches.
+#
+# It cannot be done deeper down. `cmd_revision`, `cmd_version` and `cmd_epoch`
+# all call `newest` inside a command substitution, and `newest` resolves the
+# path set inside another one — so an `exit` from either kills only that
+# subshell. The script carried on, printed nothing, and exited 0.
+#
+# Before this, `revision ml` was worse than nothing: the empty substitution
+# left `git log HEAD --` with no pathspec, so it answered with the newest
+# commit in the WHOLE REPOSITORY — a real-looking 12-char sha on stdout, exit
+# 0, and the usage line on stderr where no caller reads it. The reuse check
+# would have compared that against a published label, missed, and rebuilt
+# every push forever without a red run anywhere.
+#
+# Latent while every name callers passed was valid. #4311 removed `ml` from
+# the set, which made a name that used to work start taking that path.
+case "$2" in
+  web|agent|extension) ;;
+  *) usage ;;
+esac
+
 case "$1" in
   paths)    cmd_paths "$2" ;;
   revision) cmd_revision "$2" ;;

scripts/phash_gate_report.py

@@ -0,0 +1,197 @@
+"""What the near-duplicate gates would decide about a folder of real artwork.
+
+Read-only. Opens image files, touches no database, imports nothing from the
+app but `utils/phash.py` itself — so what it reports is what the importer
+would actually do, not a re-implementation that could drift from it.
+
+## Why this exists
+
+Issue #4223 replaced a single 64-bit pHash comparison with three gates
+(threshold -> aspect ratio -> pixel confirm). The threshold is the operator's
+dial and the aspect tolerance mirrors the video path, but the three pixel
+constants — FINGERPRINT_MAX_MEAN_DIFF, FINGERPRINT_CHANGED_LEVEL,
+FINGERPRINT_MAX_CHANGED_FRACTION — were chosen without ever measuring real
+artwork, because FC verifies in CI and CI has only synthetic fixtures. This
+prints the measurements those constants should have been chosen from.
+
+Point it at a folder whose right answer you already know — a variant set that
+should stay whole, or an image you have at two resolutions that should
+collapse — and read the MARGIN column. A pair that lands just inside a limit
+is the one that will flip on the next slightly-different file.
+
+## Usage
+
+    python scripts/phash_gate_report.py <dir> [--threshold N] [--recursive]
+    python scripts/phash_gate_report.py <file_a> <file_b> [...]
+
+No local Python environment needed — run it inside the published image, which
+already has PIL and imagehash, with the folder mounted read-only:
+
+    docker run --rm --entrypoint python -e PYTHONPATH=/app \
+      -v /path/to/art:/data:ro -v "$PWD/scripts":/scripts:ro \
+      git.fabledsword.com/bvandeusen/fabledcurator:dev \
+      /scripts/phash_gate_report.py /data
+
+(The image ships `backend/` at /app but not `scripts/`, hence the second
+mount and PYTHONPATH. The `:ro` on the art folder is the point — this reads.)
+
+`:dev` is the rolling channel this branch publishes to (rule 147) — the same
+bytes that would run in the app, without merging anything to main.
+"""
+
+import argparse
+import itertools
+import sys
+from pathlib import Path
+
+import imagehash
+from PIL import Image
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+from backend.app.utils.phash import (  # noqa: E402
+    ASPECT_TOL,
+    FINGERPRINT_MAX_CHANGED_FRACTION,
+    FINGERPRINT_MAX_MEAN_DIFF,
+    HASH_SIZE,
+    aspect_matches,
+    compute_phash,
+    find_similar,
+    fingerprint,
+    fingerprint_diff,
+    fingerprints_match,
+)
+
+IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".gif", ".webp", ".bmp", ".tif", ".tiff"}
+DEFAULT_THRESHOLD = 24
+
+
+class Shot:
+    """One image, measured once."""
+
+    def __init__(self, path: Path):
+        self.path = path
+        self.phash = None
+        self.width = 0
+        self.height = 0
+        self.fingerprint = None
+        self.error = None
+        try:
+            with Image.open(path) as im:
+                self.width, self.height = im.size
+                self.phash = compute_phash(im)
+                self.fingerprint = fingerprint(im)
+        except Exception as exc:
+            self.error = str(exc)
+
+    @property
+    def ok(self) -> bool:
+        return self.phash is not None and self.fingerprint is not None
+
+    @property
+    def label(self) -> str:
+        return f"{self.path.name} ({self.width}x{self.height})"
+
+
+def collect(paths, recursive: bool) -> list[Path]:
+    found: list[Path] = []
+    for p in paths:
+        p = Path(p)
+        if p.is_dir():
+            walk = p.rglob("*") if recursive else p.glob("*")
+            found += sorted(f for f in walk if f.suffix.lower() in IMAGE_EXTS)
+        elif p.is_file():
+            found.append(p)
+    return found
+
+
+def verdict(a: Shot, b: Shot, threshold: int) -> tuple[str, str]:
+    """Run the REAL find_similar for "a is being imported, b is in the
+    library". Returns (verdict, the gate that decided it)."""
+    rel, _ = find_similar(
+        a.phash, a.width, a.height,
+        [(b.phash, b.width, b.height, 1)],
+        threshold,
+        confirm=lambda _: fingerprints_match(a.fingerprint, b.fingerprint),
+    )
+    if rel == "larger_exists":
+        return ("DROP", f"{a.path.name} dropped; {b.path.name} kept (>= in both)")
+    if rel == "smaller_exists":
+        return ("SUPERSEDE", f"{a.path.name} replaces {b.path.name}'s file")
+    # Not a match — say which gate refused, cheapest first, since that is the
+    # constant to move if the answer is wrong.
+    dist = imagehash.hex_to_hash(a.phash) - imagehash.hex_to_hash(b.phash)
+    if dist > threshold:
+        return ("KEEP BOTH", f"hash distance {dist} > threshold {threshold}")
+    if not aspect_matches(a.width, a.height, b.width, b.height):
+        return ("KEEP BOTH", "aspect ratios differ")
+    return ("KEEP BOTH", "pixels differ")
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
+    ap.add_argument("paths", nargs="+", help="a directory, or two or more files")
+    ap.add_argument("--threshold", type=int, default=DEFAULT_THRESHOLD,
+                    help=f"phash_threshold to simulate (default {DEFAULT_THRESHOLD})")
+    ap.add_argument("--recursive", action="store_true", help="walk subdirectories")
+    ap.add_argument("--limit", type=int, default=60,
+                    help="refuse more than this many images (pairs grow as N^2)")
+    args = ap.parse_args()
+
+    files = collect(args.paths, args.recursive)
+    if len(files) < 2:
+        print(f"Need at least 2 images; found {len(files)}.", file=sys.stderr)
+        return 2
+    if len(files) > args.limit:
+        print(
+            f"{len(files)} images would be {len(files) * (len(files) - 1) // 2} "
+            f"pairs. Narrow the folder or raise --limit.", file=sys.stderr
+        )
+        return 2
+
+    print(f"hash_size={HASH_SIZE} ({HASH_SIZE * HASH_SIZE} bits)  "
+          f"threshold={args.threshold}  aspect_tol={ASPECT_TOL}")
+    print(f"pixel limits: mean <= {FINGERPRINT_MAX_MEAN_DIFF}, "
+          f"changed <= {FINGERPRINT_MAX_CHANGED_FRACTION:.1%}\n")
+
+    shots = [Shot(f) for f in files]
+    for s in shots:
+        if not s.ok:
+            print(f"  ! unreadable, excluded: {s.path.name} — {s.error}")
+    shots = [s for s in shots if s.ok]
+    if len(shots) < 2:
+        print("Not enough readable images.", file=sys.stderr)
+        return 2
+
+    print(f"{'verdict':<10} {'dist':>5} {'mean':>7} {'changed':>8}  pair")
+    print("-" * 78)
+    counts: dict[str, int] = {}
+    rows = []
+    for a, b in itertools.combinations(shots, 2):
+        v, why = verdict(a, b, args.threshold)
+        counts[v] = counts.get(v, 0) + 1
+        dist = imagehash.hex_to_hash(a.phash) - imagehash.hex_to_hash(b.phash)
+        measured = fingerprint_diff(a.fingerprint, b.fingerprint)
+        mean, changed = measured if measured else (float("nan"), float("nan"))
+        rows.append((v, dist, mean, changed, a, b, why))
+
+    # Closest pairs first: the interesting decisions are the near-misses at
+    # both limits, not the obvious strangers at the bottom of the list.
+    for v, dist, mean, changed, a, b, why in sorted(rows, key=lambda r: r[1]):
+        print(f"{v:<10} {dist:>5} {mean:>7.2f} {changed:>7.2%}  "
+              f"{a.label} vs {b.label}")
+        print(f"{'':<10} {'':>5} {'':>7} {'':>8}  -> {why}")
+
+    print("\n" + "  ".join(f"{k}: {n}" for k, n in sorted(counts.items())))
+    print(
+        "\nRead the margin, not just the verdict. A pair you consider the SAME "
+        f"image should sit far under mean {FINGERPRINT_MAX_MEAN_DIFF} / changed "
+        f"{FINGERPRINT_MAX_CHANGED_FRACTION:.0%}; a pair you consider DIFFERENT "
+        "artwork\nshould sit far over. Anything that only just cleared a limit "
+        "is what will flip on the next file, and is the reason to move a constant."
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/release_notes.py

@@ -83,7 +83,6 @@ API = "https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator"
 
 IMAGES = (
     "git.fabledsword.com/bvandeusen/fabledcurator",
-    "git.fabledsword.com/bvandeusen/fabledcurator-ml",
     "git.fabledsword.com/bvandeusen/fabledcurator-agent",
 )