Merge pull request 'Sidecar synthetic anchor cleanup + tier-gated classifier fix' (#39) from dev into main

Three coupled operator-reported pains from the 2026-05-31 download
event audit:

1. `[patreon][warning] Not allowed to view post N` was bubbling up as
   an error event, bumping consecutive_failures and parking the source
   in "needs attention." The classifier's tier-gated branch was gated
   on `return_code in (1, 4)`. Gallery-dl returns a different exit
   code for mixed-failure runs (e.g. paywall warnings + a missing
   yt-dlp dep flipping the exit bits), so the branch never fired and
   the path fell through to UNKNOWN_ERROR. Widen the gate: when no
   source-level error fired AND tier-gated warnings are present,
   classify as TIER_LIMITED regardless of return code.

2. Knuxy event #38275 (2026-05-31) ran 30 min and finalized with
   "stranded by recovery sweep (no terminal status after time_limit)"
   + empty stdout/stderr. Root cause: subprocess.run timeout (900s)
   and Celery soft_time_limit (900s) raced; when Celery won, SIGKILL
   wiped the in-memory captured output and the DownloadEvent ended up
   empty-logged 18 minutes later when the sweep finalized it. Drop
   gallery-dl's default subprocess timeout to 870s — a 30s margin
   shy of Celery's soft limit — so subprocess.TimeoutExpired always
   wins the race and captures the partial stdout/stderr via the
   existing handler.

3. `[downloader.ytdl][error] Cannot import yt-dlp or youtube-dl` was
   firing on every video attachment, causing per-item download
   failures that masked legitimate tier-gated classification.
   Add yt-dlp>=2025.1 to requirements.txt. Once it's in the image,
   video posts download normally and the per-item failure noise
   disappears.

Tests added:
- pure tier-gated stderr with exit code 128 → TIER_LIMITED + success
- mixed tier-gated + yt-dlp + per-item failures → still TIER_LIMITED

.forgejo/workflows/build.yml

@@ -2,26 +2,259 @@ name: Build images
 
 on:
   push:
-    branches: [dev, main]
+    # `:dev` builds dropped 2026-05-26 — operator tests from `:latest` after
+    # merge-to-main, not from the dev branch image. Saves one full docker
+    # build per dev push.
+    branches: [main]
+    # Tag-push triggers an immutable per-version image build (e.g.
+    # `:v26.05.26.5`) — gives a real rollback story alongside the floating
+    # `:main` / `:latest`. Layer reuse keeps the registry-storage cost
+    # negligible per tag. Doesn't overlap with the push-to-main build (that
+    # one publishes `:main` + `:latest`; the tag-push build publishes only
+    # `:<tag>`).
+    tags: ['v*']
 
 # Requires repo secret RELEASE_TOKEN — a Forgejo PAT with scopes:
 #   - write:package, read:package  (for docker push to git.fabledsword.com)
-#   - write:release                (for future release-cutting workflows)
+#   - write:release                (for ext-<version> release asset cache)
 #   - write:issue                  (for future issue-management automation)
 # The injected GITHUB_TOKEN cannot be used — it lacks write:package.
 
 jobs:
-  build-web:
+  # Sign-or-fetch-from-cache: signs the extension via AMO if no ext-<version>
+  # Forgejo release exists yet, otherwise downloads the cached signed XPI.
+  # Result is uploaded as an Actions artifact for build-web to consume.
+  #
+  # Why this lives in build.yml (not a separate workflow): the merge-commit's
+  # docker image tagged `:latest` MUST carry the XPI. A separate sign workflow
+  # racing build.yml leaves `:latest` without the XPI for ~5min (until the
+  # commit-back triggers another build). Inline ordering eliminates the race.
+  # Cache strategy: Forgejo Release Assets — picked 2026-05-25 over Generic
+  # Packages (cleaner API surface) and commit-back-to-side-branch (no extra
+  # branch to manage). AMO blocks re-signing the same version (returns 409),
+  # so signing is intentionally one-shot per version bump.
+  sign-extension:
+    if: github.ref == 'refs/heads/main'
     runs-on: python-ci
     container:
       image: git.fabledsword.com/bvandeusen/ci-python:3.14
     steps:
       - uses: actions/checkout@v4
 
+      - name: Resolve extension version
+        id: extver
+        run: |
+          VERSION=$(grep -E '"version"' extension/package.json | head -1 | sed -E 's/.*"version"[[:space:]]*:[[:space:]]*"([^"]+)".*/\1/')
+          echo "version=$VERSION" >> "$GITHUB_OUTPUT"
+          echo "Resolved extension version: $VERSION"
+
+      - name: Check Forgejo release-asset cache
+        id: cache
+        env:
+          TOKEN: ${{ secrets.RELEASE_TOKEN }}
+        run: |
+          set -eu
+          VERSION=${{ steps.extver.outputs.version }}
+          STATUS=$(curl -s -o release.json -w "%{http_code}" \
+            -H "Authorization: token $TOKEN" \
+            "https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator/releases/tags/ext-$VERSION" || echo 000)
+          echo "Tag lookup HTTP status: $STATUS"
+          # JSON parsing via python (ci-python:3.14 has stdlib json; jq is
+          # not in the image and adding it per ci-requirements.md is not
+          # warranted for a single consumer — operator-flagged 2026-05-26
+          # after a sign job failed with `jq: not found`).
+          if [ "$STATUS" = "200" ]; then
+            ASSET_ID=$(python3 -c "import json; r=json.load(open('release.json')); xpis=[a for a in r.get('assets', []) if a.get('name','').endswith('.xpi')]; print(xpis[0]['id'] if xpis else '')")
+            if [ -n "$ASSET_ID" ]; then
+              echo "cached=true" >> "$GITHUB_OUTPUT"
+              echo "asset_id=$ASSET_ID" >> "$GITHUB_OUTPUT"
+              echo "Cached XPI exists at ext-$VERSION (asset id $ASSET_ID); skipping AMO sign"
+            else
+              echo "cached=false" >> "$GITHUB_OUTPUT"
+              echo "Release ext-$VERSION exists but has no .xpi asset; will re-sign + re-upload"
+            fi
+          else
+            echo "cached=false" >> "$GITHUB_OUTPUT"
+            echo "No release named ext-$VERSION; will sign via AMO and upload"
+          fi
+
+      # No "download cached XPI in sign-extension" step: build-web
+      # fetches directly from the Forgejo ext-<version> release asset
+      # (removed 2026-05-26 alongside the actions/upload-artifact
+      # removal — sign-extension's job is just to ensure the cache
+      # exists on Forgejo; the build-web side reads it independently).
+
+      - name: Sign via AMO (cache miss)
+        if: steps.cache.outputs.cached != 'true'
+        run: |
+          cd extension && npm install --no-save --no-audit --no-fund && npm run sign
+        env:
+          WEB_EXT_API_KEY: ${{ secrets.MOZILLA_AMO_JWT_KEY }}
+          WEB_EXT_API_SECRET: ${{ secrets.MOZILLA_AMO_JWT_SECRET }}
+
+      - name: Upload signed XPI to ext-<version> release (cache miss)
+        if: steps.cache.outputs.cached != 'true'
+        env:
+          TOKEN: ${{ secrets.RELEASE_TOKEN }}
+        run: |
+          set -eux
+          VERSION=${{ steps.extver.outputs.version }}
+          # AMO renames signed XPIs with its internal addon-id-safe-string;
+          # canonicalize to fabledcurator-<version>.xpi so the FC server's
+          # whitelist (backend/app/frontend.py expects 'fabledcurator-*.xpi')
+          # keeps working.
+          SIGNED=$(ls extension/web-ext-artifacts/*.xpi | head -1)
+          XPI="extension/web-ext-artifacts/fabledcurator-$VERSION.xpi"
+          cp "$SIGNED" "$XPI"
+          # Find-or-create the ext-<version> release. Track whether WE
+          # created it so an upload failure below can roll back (don't
+          # leave an empty release tombstone that the next run's
+          # cache-check mistakes for a partial-failure state).
+          STATUS=$(curl -s -o release.json -w "%{http_code}" \
+            -H "Authorization: token $TOKEN" \
+            "https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator/releases/tags/ext-$VERSION" || echo 000)
+          if [ "$STATUS" = "200" ]; then
+            CREATED_BY_US=false
+          else
+            curl -s -X POST -H "Authorization: token $TOKEN" -H "Content-Type: application/json" \
+              -d "{\"tag_name\":\"ext-$VERSION\",\"name\":\"Extension $VERSION (signed XPI cache)\",\"body\":\"Internal cache for the signed XPI consumed by build.yml's build-web job. Not a user-facing FC release.\",\"target_commitish\":\"main\"}" \
+              -o release.json \
+              "https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator/releases"
+            CREATED_BY_US=true
+          fi
+          RELEASE_ID=$(python3 -c "import json; print(json.load(open('release.json'))['id'])")
+          test -n "$RELEASE_ID"
+          # Rollback-on-failure: if the asset upload fails AND we just
+          # created the release in this run, delete it. Prevents an empty
+          # ext-<version> release from poisoning the next workflow run
+          # (operator-flagged 2026-05-26 — without rollback the next run
+          # saw 'release exists, no asset → cache miss → sign' which AMO
+          # then rejected with 409 'Version already exists').
+          rollback_if_we_created() {
+            if [ "$CREATED_BY_US" = "true" ]; then
+              echo "Rolling back: deleting just-created release $RELEASE_ID"
+              curl -s -X DELETE -H "Authorization: token $TOKEN" \
+                "https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator/releases/$RELEASE_ID" || true
+              curl -s -X DELETE -H "Authorization: token $TOKEN" \
+                "https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator/tags/ext-$VERSION" || true
+            fi
+          }
+          trap 'rollback_if_we_created' EXIT
+          HTTP_CODE=$(curl -s -X POST -H "Authorization: token $TOKEN" \
+            -F "attachment=@$XPI" \
+            -o /dev/null -w "%{http_code}" \
+            "https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator/releases/$RELEASE_ID/assets?name=fabledcurator-$VERSION.xpi")
+          if [ "$HTTP_CODE" != "201" ] && [ "$HTTP_CODE" != "200" ]; then
+            echo "Asset upload failed with HTTP $HTTP_CODE"
+            exit 1
+          fi
+          # Upload succeeded — clear the rollback trap.
+          trap - EXIT
+          echo "Uploaded fabledcurator-$VERSION.xpi to ext-$VERSION release"
+
+      # No actions/upload-artifact step: Forgejo Actions (and our
+       # act_runner) doesn't support upload-artifact@v4+ (GHES limitation
+       # surfaced 2026-05-26). Instead build-web reads the signed XPI
+       # straight from the ext-<version> Forgejo release we just uploaded
+       # to. Same source of truth; no double-store.
+
+  build-web:
+    needs: [sign-extension]
+    # sign-extension is main-only; on dev it's skipped, build-web still runs.
+    if: always() && (needs.sign-extension.result == 'success' || needs.sign-extension.result == 'skipped')
+    runs-on: python-ci
+    container:
+      image: git.fabledsword.com/bvandeusen/ci-python:3.14
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Download signed XPI from Forgejo release asset (main + tags)
+        # Fires on main-push AND on tag-push. Tag-push builds re-package the
+        # same source code as the preceding main-push build but with an
+        # immutable version tag — they need the XPI too, otherwise the
+        # versioned image ships without the signed extension.
+        #
+        # Tag-push vs main-push race (operator-flagged 2026-05-27 after
+        # v26.05.27.0 hit it): a release cut fires BOTH workflows almost
+        # simultaneously. Main-push runs sign-extension (1-5min AMO round
+        # trip) before publishing the ext-<version> release; tag-push
+        # skips sign-extension (gated to main) and races straight to
+        # this download step. Tag-push lost every time. Fix: poll the
+        # ext-<version> release endpoint with a sleep+retry loop (30s
+        # for up to 10min total) before giving up. Main-push's signing
+        # eventually wins and tag-push picks the release up on a later
+        # iteration.
+        if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/')
+        env:
+          TOKEN: ${{ secrets.RELEASE_TOKEN }}
+        run: |
+          set -eux
+          VERSION=$(grep -E '"version"' extension/package.json | head -1 | sed -E 's/.*"version"[[:space:]]*:[[:space:]]*"([^"]+)".*/\1/')
+          # Poll for the ext-<version> release. main-push's sign-extension
+          # step (AMO round-trip, 1-5min) needs to finish + upload before
+          # tag-push can fetch. 30s * 20 = up to 10min wait, then hard-fail.
+          for attempt in $(seq 1 20); do
+            STATUS=$(curl -s -o release.json -w "%{http_code}" \
+              -H "Authorization: token $TOKEN" \
+              "https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator/releases/tags/ext-$VERSION" || echo 000)
+            if [ "$STATUS" = "200" ]; then
+              echo "Found ext-$VERSION release on attempt $attempt"
+              break
+            fi
+            if [ "$attempt" = "20" ]; then
+              echo "ERROR: ext-$VERSION release not available after 10min of polling"
+              echo "Last HTTP status: $STATUS"
+              exit 1
+            fi
+            echo "Attempt $attempt: ext-$VERSION not yet published (HTTP $STATUS); sleeping 30s"
+            sleep 30
+          done
+          # Extract the .xpi asset's browser_download_url (Forgejo's
+          # /releases/assets/<id> endpoint returns ASSET METADATA, not
+          # the binary blob — operator-flagged 2026-05-26: my prior
+          # code curl'd the metadata endpoint without -f and wrote the
+          # resulting 404-page-not-found text into fabledcurator-*.xpi,
+          # which Firefox then rejected as "corrupt").
+          # browser_download_url is the canonical binary endpoint and
+          # is also publicly accessible (no token needed) but we pass
+          # the token anyway for symmetry with private-repo support.
+          DOWNLOAD_URL=$(python3 -c "import json; r=json.load(open('release.json')); xpis=[a for a in r.get('assets', []) if a.get('name','').endswith('.xpi')]; print(xpis[0]['browser_download_url'])")
+          test -n "$DOWNLOAD_URL"
+          echo "Downloading XPI from: $DOWNLOAD_URL"
+          mkdir -p frontend/public/extension
+          DEST="frontend/public/extension/fabledcurator-$VERSION.xpi"
+          # -f = fail on HTTP error (prevents silent corruption like the
+          # 2026-05-26 incident); -L = follow redirects.
+          curl -sfL -H "Authorization: token $TOKEN" -o "$DEST" "$DOWNLOAD_URL"
+          # Sanity check: the binary should start with the ZIP magic (PK\x03\x04).
+          # If it's anything else, the next docker build will ship a corrupt XPI.
+          MAGIC=$(head -c 2 "$DEST" | od -An -c | tr -d ' \n')
+          if [ "$MAGIC" != "PK" ]; then
+            echo "ERROR: downloaded XPI does not start with ZIP magic 'PK' (got '$MAGIC')"
+            echo "File contents preview:"
+            head -c 200 "$DEST"
+            exit 1
+          fi
+          cp "$DEST" "frontend/public/extension/fabledcurator-latest.xpi"
+          ls -la frontend/public/extension/
+
       - name: Determine tag
         id: tag
         run: |
-          if [ "${GITHUB_REF##*/}" = "main" ]; then
+          # Three trigger shapes:
+          #   refs/tags/v…   → tag-push: publish ONLY the immutable version
+          #                    tag (e.g. :v26.05.26.5). Don't touch :latest;
+          #                    that already got published by the main-push
+          #                    build for the merge commit.
+          #   refs/heads/main → push to main (incl. PR merge commits):
+          #                    publish :main + :latest (floating).
+          #   anything else  → safety net; shouldn't fire given the `on:`
+          #                    config above (dev was dropped). Tag :dev to
+          #                    surface the unexpected run in the registry.
+          if [ "${GITHUB_REF#refs/tags/}" != "${GITHUB_REF}" ]; then
+            TAG_NAME="${GITHUB_REF#refs/tags/}"
+            echo "tags=git.fabledsword.com/bvandeusen/fabledcurator:${TAG_NAME}" >> "$GITHUB_OUTPUT"
+          elif [ "${GITHUB_REF##*/}" = "main" ]; then
             echo "tags=git.fabledsword.com/bvandeusen/fabledcurator:main,git.fabledsword.com/bvandeusen/fabledcurator:latest" >> "$GITHUB_OUTPUT"
           else
             echo "tags=git.fabledsword.com/bvandeusen/fabledcurator:dev" >> "$GITHUB_OUTPUT"
@@ -52,7 +285,13 @@ jobs:
       - name: Determine tag
         id: tag
         run: |
-          if [ "${GITHUB_REF##*/}" = "main" ]; then
+          # Mirrors build-web's three-shape logic (tag-push / main-push /
+          # safety-net dev). The -ml image follows the same release cadence
+          # as the web image.
+          if [ "${GITHUB_REF#refs/tags/}" != "${GITHUB_REF}" ]; then
+            TAG_NAME="${GITHUB_REF#refs/tags/}"
+            echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:${TAG_NAME}" >> "$GITHUB_OUTPUT"
+          elif [ "${GITHUB_REF##*/}" = "main" ]; then
             echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:main,git.fabledsword.com/bvandeusen/fabledcurator-ml:latest" >> "$GITHUB_OUTPUT"
           else
             echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:dev" >> "$GITHUB_OUTPUT"

.forgejo/workflows/ci.yml

@@ -1,17 +1,34 @@
 name: CI
 
 # CI lanes per FabledRulebook/forgejo.md "CI philosophy":
-#   - backend-lint-and-test: ruff + `pytest -m "not integration"`, no service containers.
+#   - lint: ruff only, no dep install — fast-fail for the common lint bounce.
+#   - backend-lint-and-test: `pytest -m "not integration"`, no service containers.
 #   - frontend-build: vitest unit + vite build.
 #   - integration: pgvector + redis service containers; alembic + `pytest -m integration`.
 
 on:
   push:
     branches: [dev, main]
-  pull_request:
-    branches: [main]
+  # pull_request trigger intentionally absent — with branches: [dev, main]
+  # above, every PR commit already fires CI via the push event on dev. Adding
+  # pull_request would duplicate runs on dev→main PRs. FC has no fork PRs
+  # (single-operator Forgejo repo) so push coverage is complete.
 
 jobs:
+  # Fast-fail lint lane. ruff is pre-installed in the ci-python image, so
+  # this runs with NO dependency install and surfaces the most common bounce
+  # class (lint: I001 / UP037 / ASYNC109 / W293 …) in seconds — instead of
+  # after the backend job's ~30-60s wheel install. ruff is static analysis,
+  # so no DB/secret env is needed.
+  lint:
+    runs-on: python-ci
+    container:
+      image: git.fabledsword.com/bvandeusen/ci-python:3.14
+    steps:
+      - uses: actions/checkout@v4
+      - name: Ruff lint
+        run: ruff check backend/ tests/ alembic/
+
   backend-lint-and-test:
     runs-on: python-ci
     container:
@@ -24,15 +41,33 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
+      # Cache step removed 2026-05-26: act_runner's cache backend has been
+      # broken on this homelab runner since 2026-05-15 (first as request-
+      # timeout warnings, then as hard "Cannot find module .../dist/restore/
+      # index.js" failures that tank the whole job). The cache step targeted
+      # ~/.cache/pip but the install below uses `uv pip install` primarily,
+      # whose own cache lives at ~/.cache/uv — so the cache step's real
+      # benefit was marginal even when working. Cost of removal: ~30s of
+      # wheel downloads per job. Future re-enable: mount ~/.cache/uv as a
+      # docker volume at the runner level (skips actions/cache entirely),
+      # or fix the runner-side cache backend (clear /var/run/act/actions/*,
+      # pin act_runner version, etc.).
+
       - name: Install Python deps
         # ruff is pre-installed in the ci-python image (see CI-Runner/CI-python/
         # Dockerfile's RUFF_VERSION). Per FabledRulebook ci-runners.md, toolchain
         # versions live on the runner image, not here.
-        run: pip install -r requirements.txt pytest pytest-asyncio
-
-      - name: Ruff lint
-        run: ruff check backend/ tests/ alembic/
+        # uv: 5-10x faster wheel resolve than pip for cold caches.
+        # Falls back to pip install on uv-missing runners (older images).
+        run: |
+          if command -v uv >/dev/null 2>&1; then
+            uv pip install --system -r requirements.txt pytest pytest-asyncio
+          else
+            pip install -r requirements.txt pytest pytest-asyncio
+          fi
 
+      # Ruff moved to the dedicated fast `lint` job above (fails in seconds,
+      # no dep install). This job is now unit tests only.
       - name: Pytest (unit only — integration runs in the integration job)
         run: pytest tests/ -v -m "not integration"
 
@@ -57,17 +92,28 @@ jobs:
       - run: npm run test:unit
       - run: npm run build
 
-  integration:
-    # This act_runner (swarm-runner v0.6.1) puts service containers on the
-    # default bridge with NO service-name DNS, and publishing fixed host
-    # ports collides with the operator's running docker-compose dev stack on
-    # the same shared daemon. Workaround: publish NO host ports, and reach
-    # each service by its bridge IP — discovered at runtime via the mounted
-    # docker socket (the ci-python image ships /usr/bin/docker). Default-bridge
-    # containers can talk by IP (only embedded DNS is missing), so IP
-    # addressing is reliable here. Everything runs in ONE step so resolved
-    # values don't depend on cross-step env passing. Pattern documented in
-    # FabledRulebook/forgejo.md "CI philosophy".
+  # Integration suite split into THREE parallel shards (2026-05-25, runner
+  # capacity bumped 2→6). Each shard gets its own Postgres + Redis service
+  # set and runs alembic + a disjoint subset of integration tests. Shards
+  # share no DB state, so the autouse TRUNCATE fixture in tests/conftest.py
+  # stays single-threaded per shard but multiple shards run in parallel
+  # wall-clock. Approximate split — rebalance once --durations=15 output
+  # reveals which shard is the long pole.
+  #
+  # Each shard's docker-ps filter uses its own unique job name to scope
+  # service-container resolution. act_runner appears to strip underscores
+  # from job names when building container labels — `int_api` yielded
+  # zero matches on 2026-05-25 — so shards use no-separator names
+  # (`intapi`, `intimp`, `intcore`) instead. Each step prints
+  # `docker ps -a` first so a future naming-convention shift surfaces in
+  # the log without another guess-and-push cycle.
+  #
+  # Pre-baking requirements.txt into ci-python:3.14 is intentionally NOT
+  # done — per ci-requirements.md, FC is the only Python consumer of that
+  # image and the CI-Runner project's "add deps to image when used by >1
+  # project" rule keeps the install per-job.
+
+  intapi:
     runs-on: python-ci
     container:
       image: git.fabledsword.com/bvandeusen/ci-python:3.14
@@ -98,15 +144,14 @@ jobs:
           --health-retries 10
     steps:
       - uses: actions/checkout@v4
-
-      - name: Integration suite (resolve service IPs, migrate, test)
+      - name: API integration shard (resolve service IPs, migrate, test)
         run: |
           set -eux
-          # Scope to THIS job's service containers (act_runner names them
-          # ...JOB-integration...); the operator's compose stack uses the
-          # same images but different names, so it won't match.
-          PG=$(docker ps --filter "name=JOB-integration" --filter "ancestor=pgvector/pgvector:pg16" -q | head -n1)
-          RD=$(docker ps --filter "name=JOB-integration" --filter "ancestor=redis:7-alpine" -q | head -n1)
+          echo "=== container landscape (diagnostic for filter scoping) ==="
+          docker ps -a --format '{{.ID}} {{.Image}} -> {{.Names}}'
+          echo "=== end landscape ==="
+          PG=$(docker ps --filter "name=intapi" --filter "ancestor=pgvector/pgvector:pg16" -q | head -n1)
+          RD=$(docker ps --filter "name=intapi" --filter "ancestor=redis:7-alpine" -q | head -n1)
           test -n "$PG" && test -n "$RD"
           PG_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$PG")
           RD_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$RD")
@@ -114,11 +159,139 @@ jobs:
           export DB_HOST="$PG_IP"
           export CELERY_BROKER_URL="redis://$RD_IP:6379/0"
           export CELERY_RESULT_BACKEND="redis://$RD_IP:6379/0"
-          # Wait for Postgres to accept TCP (bash /dev/tcp; no extra tools).
           for i in $(seq 1 60); do
             (echo > "/dev/tcp/$PG_IP/5432") >/dev/null 2>&1 && break
             sleep 2
           done
-          pip install -r requirements.txt pytest pytest-asyncio
+          if command -v uv >/dev/null 2>&1; then
+            uv pip install --system -r requirements.txt pytest pytest-asyncio
+          else
+            pip install -r requirements.txt pytest pytest-asyncio
+          fi
           alembic upgrade head
-          pytest tests/ -v -m integration
+          pytest tests/test_api_*.py -v -m integration --durations=15
+
+  intimp:
+    runs-on: python-ci
+    container:
+      image: git.fabledsword.com/bvandeusen/ci-python:3.14
+    env:
+      DB_USER: fabledcurator
+      DB_PASSWORD: ci_integration
+      DB_PORT: "5432"
+      DB_NAME: fabledcurator_test
+      SECRET_KEY: ci_integration_placeholder
+    services:
+      postgres:
+        image: pgvector/pgvector:pg16
+        env:
+          POSTGRES_USER: fabledcurator
+          POSTGRES_PASSWORD: ci_integration
+          POSTGRES_DB: fabledcurator_test
+        options: >-
+          --health-cmd "pg_isready -U fabledcurator"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 10
+      redis:
+        image: redis:7-alpine
+        options: >-
+          --health-cmd "redis-cli ping"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 10
+    steps:
+      - uses: actions/checkout@v4
+      - name: Importer integration shard (resolve service IPs, migrate, test)
+        run: |
+          set -eux
+          echo "=== container landscape (diagnostic for filter scoping) ==="
+          docker ps -a --format '{{.ID}} {{.Image}} -> {{.Names}}'
+          echo "=== end landscape ==="
+          PG=$(docker ps --filter "name=intimp" --filter "ancestor=pgvector/pgvector:pg16" -q | head -n1)
+          RD=$(docker ps --filter "name=intimp" --filter "ancestor=redis:7-alpine" -q | head -n1)
+          test -n "$PG" && test -n "$RD"
+          PG_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$PG")
+          RD_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$RD")
+          test -n "$PG_IP" && test -n "$RD_IP"
+          export DB_HOST="$PG_IP"
+          export CELERY_BROKER_URL="redis://$RD_IP:6379/0"
+          export CELERY_RESULT_BACKEND="redis://$RD_IP:6379/0"
+          for i in $(seq 1 60); do
+            (echo > "/dev/tcp/$PG_IP/5432") >/dev/null 2>&1 && break
+            sleep 2
+          done
+          if command -v uv >/dev/null 2>&1; then
+            uv pip install --system -r requirements.txt pytest pytest-asyncio
+          else
+            pip install -r requirements.txt pytest pytest-asyncio
+          fi
+          alembic upgrade head
+          pytest tests/test_importer*.py tests/test_import_*.py tests/test_migration_*.py tests/test_phash_*.py tests/test_sidecar_*.py tests/test_scan_*.py tests/test_archive_extractor.py tests/test_backfill_phash.py -v -m integration --durations=15
+
+  intcore:
+    runs-on: python-ci
+    container:
+      image: git.fabledsword.com/bvandeusen/ci-python:3.14
+    env:
+      DB_USER: fabledcurator
+      DB_PASSWORD: ci_integration
+      DB_PORT: "5432"
+      DB_NAME: fabledcurator_test
+      SECRET_KEY: ci_integration_placeholder
+    services:
+      postgres:
+        image: pgvector/pgvector:pg16
+        env:
+          POSTGRES_USER: fabledcurator
+          POSTGRES_PASSWORD: ci_integration
+          POSTGRES_DB: fabledcurator_test
+        options: >-
+          --health-cmd "pg_isready -U fabledcurator"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 10
+      redis:
+        image: redis:7-alpine
+        options: >-
+          --health-cmd "redis-cli ping"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 10
+    steps:
+      - uses: actions/checkout@v4
+      - name: Core integration shard (everything not api / importer / migration / phash / sidecar / scan / archive / backfill)
+        run: |
+          set -eux
+          echo "=== container landscape (diagnostic for filter scoping) ==="
+          docker ps -a --format '{{.ID}} {{.Image}} -> {{.Names}}'
+          echo "=== end landscape ==="
+          PG=$(docker ps --filter "name=intcore" --filter "ancestor=pgvector/pgvector:pg16" -q | head -n1)
+          RD=$(docker ps --filter "name=intcore" --filter "ancestor=redis:7-alpine" -q | head -n1)
+          test -n "$PG" && test -n "$RD"
+          PG_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$PG")
+          RD_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$RD")
+          test -n "$PG_IP" && test -n "$RD_IP"
+          export DB_HOST="$PG_IP"
+          export CELERY_BROKER_URL="redis://$RD_IP:6379/0"
+          export CELERY_RESULT_BACKEND="redis://$RD_IP:6379/0"
+          for i in $(seq 1 60); do
+            (echo > "/dev/tcp/$PG_IP/5432") >/dev/null 2>&1 && break
+            sleep 2
+          done
+          if command -v uv >/dev/null 2>&1; then
+            uv pip install --system -r requirements.txt pytest pytest-asyncio
+          else
+            pip install -r requirements.txt pytest pytest-asyncio
+          fi
+          alembic upgrade head
+          pytest tests/ -v -m integration --durations=15 \
+            --ignore-glob='tests/test_api_*.py' \
+            --ignore-glob='tests/test_importer*.py' \
+            --ignore-glob='tests/test_import_*.py' \
+            --ignore-glob='tests/test_migration_*.py' \
+            --ignore-glob='tests/test_phash_*.py' \
+            --ignore-glob='tests/test_sidecar_*.py' \
+            --ignore-glob='tests/test_scan_*.py' \
+            --ignore-glob='tests/test_archive_extractor.py' \
+            --ignore-glob='tests/test_backfill_phash.py'

.forgejo/workflows/extension.yml

@@ -0,0 +1,29 @@
+name: extension
+# Lint-only workflow. The sign-and-publish dance moved into build.yml's
+# `sign-extension` job (2026-05-25) — `:latest` now always bundles the XPI
+# because sign-extension runs as a build-web dependency in the SAME workflow,
+# eliminating the prior race between build.yml and a separate extension.yml.
+# Signed XPIs are cached in Forgejo Release Assets named `ext-<version>`.
+on:
+  push:
+    branches: [dev, main]
+    paths:
+      - 'extension/**'
+      - '.forgejo/workflows/extension.yml'
+  pull_request:
+    branches: [main]
+    paths:
+      - 'extension/**'
+  workflow_dispatch:
+
+jobs:
+  lint:
+    runs-on: python-ci
+    container:
+      image: node:22-bookworm-slim
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install web-ext
+        run: cd extension && npm install --no-save --no-audit --no-fund
+      - name: Lint
+        run: cd extension && npm run lint

alembic/versions/0016_fc3i_task_run.py

@@ -0,0 +1,86 @@
+"""fc3i: task_run table
+
+Revision ID: 0016
+Revises: 0015
+Create Date: 2026-05-24
+
+Additive only. New table records every Celery task attempt via signal
+handlers (backend.app.celery_signals). Status is plain String(16) not
+Postgres ENUM (per feedback_check_existing_enums: ENUM columns hard-
+fail at INSERT, String columns extend cleanly).
+
+Composite indexes anticipate the three dashboard panes:
+- (queue, started_at desc)   — per-lane recent activity
+- (status, started_at desc)  — recent failures pane
+- (task_name, started_at desc) — drill-down by task
+
+Indexed columns get individual indexes via `index=True` on the model;
+the composites below cover the multi-column lookups.
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0016"
+down_revision: Union[str, None] = "0015"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "task_run",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column("celery_task_id", sa.String(length=64), nullable=False),
+        sa.Column("queue", sa.String(length=32), nullable=False),
+        sa.Column("task_name", sa.String(length=128), nullable=False),
+        sa.Column("target_id", sa.Integer(), nullable=True),
+        sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
+        sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
+        sa.Column("duration_ms", sa.Integer(), nullable=True),
+        sa.Column(
+            "status", sa.String(length=16), nullable=False,
+            server_default="running",
+        ),
+        sa.Column("error_type", sa.String(length=128), nullable=True),
+        sa.Column("error_message", sa.Text(), nullable=True),
+        sa.Column("retry_count", sa.Integer(), nullable=True),
+        sa.Column("worker_hostname", sa.String(length=128), nullable=True),
+        sa.Column("args_summary", sa.String(length=255), nullable=True),
+    )
+
+    # Single-column indexes (matches Mapped[...].index=True on model).
+    op.create_index("ix_task_run_celery_task_id", "task_run", ["celery_task_id"])
+    op.create_index("ix_task_run_queue", "task_run", ["queue"])
+    op.create_index("ix_task_run_task_name", "task_run", ["task_name"])
+    op.create_index("ix_task_run_started_at", "task_run", ["started_at"])
+    op.create_index("ix_task_run_finished_at", "task_run", ["finished_at"])
+    op.create_index("ix_task_run_status", "task_run", ["status"])
+
+    # Composite indexes for dashboard query patterns.
+    op.create_index(
+        "ix_task_run_queue_started",
+        "task_run", ["queue", sa.text("started_at DESC")],
+    )
+    op.create_index(
+        "ix_task_run_status_started",
+        "task_run", ["status", sa.text("started_at DESC")],
+    )
+    op.create_index(
+        "ix_task_run_name_started",
+        "task_run", ["task_name", sa.text("started_at DESC")],
+    )
+
+
+def downgrade() -> None:
+    op.drop_index("ix_task_run_name_started", table_name="task_run")
+    op.drop_index("ix_task_run_status_started", table_name="task_run")
+    op.drop_index("ix_task_run_queue_started", table_name="task_run")
+    op.drop_index("ix_task_run_status", table_name="task_run")
+    op.drop_index("ix_task_run_finished_at", table_name="task_run")
+    op.drop_index("ix_task_run_started_at", table_name="task_run")
+    op.drop_index("ix_task_run_task_name", table_name="task_run")
+    op.drop_index("ix_task_run_queue", table_name="task_run")
+    op.drop_index("ix_task_run_celery_task_id", table_name="task_run")
+    op.drop_table("task_run")

alembic/versions/0017_fc3h_backup_run.py

@@ -0,0 +1,82 @@
+"""fc3h: backup_run table
+
+Revision ID: 0017
+Revises: 0016
+Create Date: 2026-05-24
+
+Additive. New table records every backup/restore attempt with artifact
+metadata. Lifecycle tracking lives in task_run from FC-3i; this is
+artifact-only (paths, sizes, tag, restore lineage).
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0017"
+down_revision: Union[str, None] = "0016"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "backup_run",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column("kind", sa.String(length=16), nullable=False),
+        sa.Column(
+            "status", sa.String(length=16), nullable=False,
+            server_default="pending",
+        ),
+        sa.Column("tag", sa.String(length=64), nullable=True),
+        sa.Column("triggered_by", sa.String(length=32), nullable=False),
+        sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
+        sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
+        sa.Column("sql_path", sa.Text(), nullable=True),
+        sa.Column("tar_path", sa.Text(), nullable=True),
+        sa.Column("size_bytes", sa.BigInteger(), nullable=True),
+        sa.Column("error", sa.Text(), nullable=True),
+        sa.Column(
+            "manifest", sa.JSON(), nullable=False, server_default="{}",
+        ),
+        sa.Column(
+            "restored_from_id", sa.Integer(),
+            sa.ForeignKey("backup_run.id", ondelete="SET NULL"),
+            nullable=True,
+        ),
+    )
+
+    # Single-column indexes (matches Mapped[...].index=True).
+    op.create_index("ix_backup_run_kind", "backup_run", ["kind"])
+    op.create_index("ix_backup_run_status", "backup_run", ["status"])
+    op.create_index("ix_backup_run_tag", "backup_run", ["tag"])
+    op.create_index("ix_backup_run_started_at", "backup_run", ["started_at"])
+    op.create_index("ix_backup_run_finished_at", "backup_run", ["finished_at"])
+
+    # Composite indexes for dashboard query patterns.
+    op.create_index(
+        "ix_backup_run_kind_started",
+        "backup_run", ["kind", sa.text("started_at DESC")],
+    )
+    op.create_index(
+        "ix_backup_run_status_finished",
+        "backup_run", ["status", sa.text("finished_at DESC")],
+    )
+    # Partial index: only tagged rows participate in retention-exempt query.
+    op.create_index(
+        "ix_backup_run_tag_partial",
+        "backup_run", ["tag"],
+        postgresql_where=sa.text("tag IS NOT NULL"),
+    )
+
+
+def downgrade() -> None:
+    op.drop_index("ix_backup_run_tag_partial", table_name="backup_run")
+    op.drop_index("ix_backup_run_status_finished", table_name="backup_run")
+    op.drop_index("ix_backup_run_kind_started", table_name="backup_run")
+    op.drop_index("ix_backup_run_finished_at", table_name="backup_run")
+    op.drop_index("ix_backup_run_started_at", table_name="backup_run")
+    op.drop_index("ix_backup_run_tag", table_name="backup_run")
+    op.drop_index("ix_backup_run_status", table_name="backup_run")
+    op.drop_index("ix_backup_run_kind", table_name="backup_run")
+    op.drop_table("backup_run")

alembic/versions/0018_fc3h_backup_settings.py

@@ -0,0 +1,62 @@
+"""fc3h: backup_* knobs on import_settings
+
+Revision ID: 0018
+Revises: 0017
+Create Date: 2026-05-24
+
+Adds four columns to the singleton import_settings row:
+  - backup_db_nightly_enabled (default False — opt-in)
+  - backup_db_nightly_hour_utc (default 3)
+  - backup_db_keep_last_n (default 14)
+  - backup_images_keep_last_n (default 3)
+
+server_default ensures the singleton row is backfilled in place
+without an UPDATE statement.
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0018"
+down_revision: Union[str, None] = "0017"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "import_settings",
+        sa.Column(
+            "backup_db_nightly_enabled", sa.Boolean(),
+            nullable=False, server_default=sa.false(),
+        ),
+    )
+    op.add_column(
+        "import_settings",
+        sa.Column(
+            "backup_db_nightly_hour_utc", sa.Integer(),
+            nullable=False, server_default="3",
+        ),
+    )
+    op.add_column(
+        "import_settings",
+        sa.Column(
+            "backup_db_keep_last_n", sa.Integer(),
+            nullable=False, server_default="14",
+        ),
+    )
+    op.add_column(
+        "import_settings",
+        sa.Column(
+            "backup_images_keep_last_n", sa.Integer(),
+            nullable=False, server_default="3",
+        ),
+    )
+
+
+def downgrade() -> None:
+    op.drop_column("import_settings", "backup_images_keep_last_n")
+    op.drop_column("import_settings", "backup_db_keep_last_n")
+    op.drop_column("import_settings", "backup_db_nightly_hour_utc")
+    op.drop_column("import_settings", "backup_db_nightly_enabled")

alembic/versions/0019_import_batch_refreshed.py

@@ -0,0 +1,38 @@
+"""import_batch.refreshed counter for deep-scan sidecar re-application
+
+Revision ID: 0019
+Revises: 0018
+Create Date: 2026-05-25
+
+Adds a `refreshed` counter to `import_batch`, mirroring the existing
+`imported`/`skipped`/`failed`/`attachments` columns. Deep scan now
+re-applies sidecar metadata to already-imported files (the IR feature
+that didn't make the FC port the first time); a "refreshed" outcome
+increments this counter so the UI can surface "X new, Y refreshed"
+instead of the misleading "Scan complete — no new files" message.
+
+server_default=0 backfills existing rows in place — no UPDATE needed.
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0019"
+down_revision: Union[str, None] = "0018"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "import_batch",
+        sa.Column(
+            "refreshed", sa.Integer(),
+            nullable=False, server_default=sa.text("0"),
+        ),
+    )
+
+
+def downgrade() -> None:
+    op.drop_column("import_batch", "refreshed")

alembic/versions/0020_library_audit_run.py

@@ -0,0 +1,65 @@
+"""fc-cleanup: library_audit_run table for async transparency/single_color audits
+
+Revision ID: 0020
+Revises: 0019
+Create Date: 2026-05-26
+
+The table backs the async audit lifecycle: rule + params snapshot, status
+state machine ('running' → 'ready' → 'applied'/'cancelled'/'error'), and
+the matched_ids JSONB array that the apply step deletes. Capped at 50k IDs
+per row by the scan task (oversize = rule too aggressive, operator narrows
+before re-running).
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+from sqlalchemy.dialects import postgresql
+
+revision: str = "0020"
+down_revision: Union[str, None] = "0019"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "library_audit_run",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column("rule", sa.String(32), nullable=False),
+        sa.Column("params", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
+        sa.Column(
+            "status", sa.String(16),
+            nullable=False, server_default="running",
+        ),
+        sa.Column(
+            "started_at", sa.DateTime(timezone=True),
+            nullable=False, server_default=sa.func.now(),
+        ),
+        sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
+        sa.Column(
+            "scanned_count", sa.Integer(),
+            nullable=False, server_default="0",
+        ),
+        sa.Column(
+            "matched_count", sa.Integer(),
+            nullable=False, server_default="0",
+        ),
+        sa.Column(
+            "matched_ids", postgresql.JSONB(astext_type=sa.Text()),
+            nullable=False, server_default=sa.text("'[]'::jsonb"),
+        ),
+        sa.Column("error", sa.Text(), nullable=True),
+    )
+    op.create_index(
+        "ix_library_audit_run_rule", "library_audit_run", ["rule"],
+    )
+    op.create_index(
+        "ix_library_audit_run_status", "library_audit_run", ["status"],
+    )
+
+
+def downgrade() -> None:
+    op.drop_index("ix_library_audit_run_status", table_name="library_audit_run")
+    op.drop_index("ix_library_audit_run_rule", table_name="library_audit_run")
+    op.drop_table("library_audit_run")

alembic/versions/0021_image_provenance_unique.py

@@ -0,0 +1,54 @@
+"""provenance-race: dedupe + UNIQUE(image_record_id, post_id) on image_provenance
+
+Revision ID: 0021
+Revises: 0020
+Create Date: 2026-05-26
+
+Closes the race in Importer._apply_sidecar's existence-check + INSERT pattern.
+Two workers writing for the same (image, post) pair both saw no existing row
+and both inserted, leaving duplicates that then broke .scalar_one_or_none()
+on every subsequent deep-scan rederive against those images
+(MultipleResultsFound). Most plausibly seeded when the 5-min recovery sweep
+re-enqueued a still-running long-import task and the second worker collided
+with the first inside _apply_sidecar.
+
+Migration steps:
+  1. DELETE all but min(id) per (image_record_id, post_id) pair. Operator's
+     DB had 2 affected pairs at write-time; harmless no-op if zero.
+  2. Add UNIQUE constraint so the importer's new savepoint+IntegrityError
+     recovery path can trip on collision and re-select, mirroring
+     uq_source_artist_platform_url and uq_post_source_external_id.
+"""
+from typing import Sequence, Union
+
+from alembic import op
+
+revision: str = "0021"
+down_revision: Union[str, None] = "0020"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.execute(
+        """
+        DELETE FROM image_provenance ip1
+        USING image_provenance ip2
+        WHERE ip1.image_record_id = ip2.image_record_id
+          AND ip1.post_id = ip2.post_id
+          AND ip1.id > ip2.id
+        """
+    )
+    op.create_unique_constraint(
+        "uq_image_provenance_image_post",
+        "image_provenance",
+        ["image_record_id", "post_id"],
+    )
+
+
+def downgrade() -> None:
+    op.drop_constraint(
+        "uq_image_provenance_image_post",
+        "image_provenance",
+        type_="unique",
+    )

alembic/versions/0022_source_per_artist_platform.py

@@ -0,0 +1,223 @@
+"""source-collapse: one Source per (artist, platform) — consolidate junk per-post Sources
+
+Revision ID: 0022
+Revises: 0021
+Create Date: 2026-05-26
+
+Closes the operator-flagged 2026-05-26 issue where the filesystem importer
+called _find_or_create_source(url=sd.post_url), creating one Source row per
+imported post URL. Operator's Atole artist had 406 Source rows where there
+should have been 1 (the /cw/Atole subscription Source).
+
+Source represents a subscription feed (one per artist+platform — the
+gallery-dl URL polled by the FC-3 downloader). Posts hang off it. The
+filesystem importer was misusing Source as a per-post key.
+
+Migration steps per (artist_id, platform) group with >1 Source:
+  1. Pick canonical — prefer a URL NOT matching '/posts/<id>$' (real
+     campaign URL like /cw/Atole); else min(id).
+  2. PRE-merge any Posts under non-canonical sources whose
+     external_post_id ALREADY exists under the canonical source. (Same
+     gallery-dl post imported via two different sidecar paths can plant
+     two Post rows with identical external_post_id under different
+     Sources for the same artist.) Repoint ImageProvenance +
+     ImageRecord.primary_post_id to the canonical-side Post, dedupe
+     ImageProvenance against alembic 0021's uq, then delete the
+     non-canonical-side Post. This MUST happen before step 3 — Postgres
+     fires uq_post_source_external_id row-by-row during the bulk UPDATE
+     and the merge-after-reparent ordering 500s on first collision
+     (operator-hit during v26.05.26.1 deploy, 2026-05-26).
+  3. Reparent remaining Posts onto canonical (no collisions possible now).
+  4. Reparent ImageProvenance.source_id off the non-canonical sources.
+  5. Delete the orphan Source rows.
+  6. If the canonical Source's URL still looks like a per-post URL (no
+     campaign URL existed among candidates), rewrite it to
+     'sidecar:<platform>:<artist_slug>' so the artist detail page shows
+     something readable.
+"""
+from typing import Sequence, Union
+
+from alembic import op
+from sqlalchemy import text
+
+revision: str = "0022"
+down_revision: Union[str, None] = "0021"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+_POST_URL_RE = r"/posts/[^/]+$"
+
+
+def upgrade() -> None:
+    conn = op.get_bind()
+
+    # Find (artist_id, platform) groups with > 1 Source row.
+    groups = conn.execute(text("""
+        SELECT artist_id, platform
+        FROM source
+        GROUP BY artist_id, platform
+        HAVING COUNT(*) > 1
+    """)).fetchall()
+
+    for artist_id, platform in groups:
+        rows = conn.execute(
+            text("""
+                SELECT id, url FROM source
+                WHERE artist_id = :a AND platform = :p
+                ORDER BY id ASC
+            """),
+            {"a": artist_id, "p": platform},
+        ).fetchall()
+
+        # Canonical: first row whose URL doesn't look like a per-post URL;
+        # else min(id).
+        canonical_id = None
+        for sid, url in rows:
+            if not _matches_post_url(url):
+                canonical_id = sid
+                break
+        if canonical_id is None:
+            canonical_id = rows[0][0]
+
+        other_ids = [sid for sid, _ in rows if sid != canonical_id]
+        if not other_ids:
+            continue
+
+        # STEP 2: PRE-merge ALL Posts with duplicate external_post_id
+        # across the entire (canonical + others) group, BEFORE the bulk
+        # reparent. Two cases must both be handled:
+        #   (A) canonical has Post X with epid=N; an "other" source has
+        #       Post Y with epid=N → after bulk UPDATE, (canonical, N)
+        #       collides with itself.
+        #   (B) two different "other" sources each have a Post with
+        #       epid=N; canonical has none → after bulk UPDATE, both
+        #       are repointed to (canonical, N) and the second collides.
+        # The earlier version of this migration only handled (A); the
+        # operator's deploy 2026-05-26 tripped (B) at line 139.
+        # Fix: group ALL Posts in the (artist, platform) by epid; for
+        # any group with count>1, pick the keep (prefer one already
+        # under canonical; else lowest id) and merge the rest into it.
+        all_posts = conn.execute(
+            text("""
+                SELECT external_post_id, id, source_id
+                FROM post
+                WHERE source_id = :canonical OR source_id = ANY(:others)
+                ORDER BY external_post_id, id
+            """),
+            {"canonical": canonical_id, "others": other_ids},
+        ).fetchall()
+        by_epid: dict = {}
+        for epid, post_id, src_id in all_posts:
+            by_epid.setdefault(epid, []).append((post_id, src_id))
+        for _epid, posts in by_epid.items():
+            if len(posts) <= 1:
+                continue
+            # Prefer a Post already under canonical as the keep.
+            canonical_posts = [p for p in posts if p[1] == canonical_id]
+            if canonical_posts:
+                keep_id = canonical_posts[0][0]
+            else:
+                keep_id = posts[0][0]  # already sorted by id ASC
+            drop_ids = [p[0] for p in posts if p[0] != keep_id]
+            for drop_id in drop_ids:
+                # Pre-delete image_provenance rows under drop_ whose
+                # image_record_id ALREADY has a provenance under keep —
+                # the UPDATE below would otherwise repoint them and
+                # trip uq_image_provenance_image_post (alembic 0021)
+                # row-by-row before any after-the-fact dedupe could
+                # run. Operator's v26.05.26.3 deploy 2026-05-26 tripped
+                # this at line 123.
+                conn.execute(
+                    text("""
+                        DELETE FROM image_provenance
+                        WHERE post_id = :drop_
+                          AND image_record_id IN (
+                              SELECT image_record_id FROM image_provenance
+                              WHERE post_id = :keep
+                          )
+                    """),
+                    {"keep": keep_id, "drop_": drop_id},
+                )
+                # Now safe to repoint the survivors.
+                conn.execute(
+                    text("""
+                        UPDATE image_provenance SET post_id = :keep
+                        WHERE post_id = :drop_
+                    """),
+                    {"keep": keep_id, "drop_": drop_id},
+                )
+                conn.execute(
+                    text("""
+                        UPDATE image_record SET primary_post_id = :keep
+                        WHERE primary_post_id = :drop_
+                    """),
+                    {"keep": keep_id, "drop_": drop_id},
+                )
+                conn.execute(
+                    text("DELETE FROM post WHERE id = :drop_"),
+                    {"drop_": drop_id},
+                )
+
+        # STEP 3: Bulk reparent the remaining Posts off the other
+        # Sources. After step 2, no collisions on
+        # (canonical, external_post_id) are possible.
+        conn.execute(
+            text("""
+                UPDATE post SET source_id = :canonical
+                WHERE source_id = ANY(:others)
+            """),
+            {"canonical": canonical_id, "others": other_ids},
+        )
+
+        # STEP 4: Reparent ImageProvenance.source_id (denormalized FK).
+        # No UNIQUE on source_id; safe bulk update.
+        conn.execute(
+            text("""
+                UPDATE image_provenance SET source_id = :canonical
+                WHERE source_id = ANY(:others)
+            """),
+            {"canonical": canonical_id, "others": other_ids},
+        )
+
+        # STEP 5: Drop the orphan Sources.
+        conn.execute(
+            text("DELETE FROM source WHERE id = ANY(:others)"),
+            {"others": other_ids},
+        )
+
+        # If the canonical's URL still looks per-post (no campaign URL
+        # existed among the candidates), rewrite to a synthetic anchor so
+        # the artist detail page renders something readable.
+        canonical_url = conn.execute(
+            text("SELECT url FROM source WHERE id = :id"),
+            {"id": canonical_id},
+        ).scalar_one()
+        if _matches_post_url(canonical_url):
+            slug = conn.execute(
+                text("SELECT slug FROM artist WHERE id = :id"),
+                {"id": artist_id},
+            ).scalar_one()
+            conn.execute(
+                text("""
+                    UPDATE source
+                    SET url = :new_url, enabled = false
+                    WHERE id = :id
+                """),
+                {
+                    "id": canonical_id,
+                    "new_url": f"sidecar:{platform}:{slug}",
+                },
+            )
+
+
+def downgrade() -> None:
+    # Lossy migration — orphan Sources deleted, Posts reparented, Posts
+    # merged. No safe downgrade. If you need to roll back the schema
+    # invariant, fork from 0021 and re-run filesystem imports.
+    pass
+
+
+def _matches_post_url(url: str) -> bool:
+    """True if url ends with /posts/<token> (gallery-dl-style per-post URL)."""
+    import re
+    return bool(re.search(_POST_URL_RE, url or ""))

alembic/versions/0023_drop_meta_rating_tag_kinds.py

@@ -0,0 +1,99 @@
+"""drop meta + rating tag kinds — operator-retired 2026-05-26
+
+Revision ID: 0023
+Revises: 0022
+Create Date: 2026-05-26
+
+Operator decided meta + rating aren't valid tag kinds for FC. Per-row
+behavior: DELETE existing rows (operator chose "clean break" over
+"convert to general"). All cascading FKs (image_tag, tag_alias,
+tag_allowlist, tag_reference_embedding, tag_suggestion_rejection,
+series_page) use ondelete="CASCADE" so a single DELETE on tag cleans
+the related rows in one go.
+
+After the data cleanup, recreate the tag_kind ENUM without 'meta' /
+'rating' (Postgres has no `ALTER TYPE ... DROP VALUE`; standard
+rename-create-cast-drop dance). The server default 'general' is
+dropped before the type swap and restored after.
+"""
+from typing import Sequence, Union
+
+from alembic import op
+
+revision: str = "0023"
+down_revision: Union[str, None] = "0022"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # 1. Delete tags of the retired kinds. CASCADE handles related tables.
+    op.execute("DELETE FROM tag WHERE kind IN ('meta', 'rating')")
+
+    # 2. Drop the CHECK constraint that references the enum's literal
+    #    values. Postgres can't resolve `kind = 'character'` across the
+    #    type swap below — the literal would bind to the new tag_kind
+    #    but the column is on tag_kind_old, producing
+    #    "operator does not exist: tag_kind = tag_kind_old".
+    #    (Operator-hit during the v26.05.26.5 deploy attempt; ck was
+    #    originally added by alembic 0002.) Recreated post-swap.
+    op.drop_constraint(
+        "ck_tag_fandom_requires_character", "tag", type_="check"
+    )
+
+    # 3. Drop the server default — ALTER COLUMN TYPE can't carry it
+    #    across the type swap below.
+    op.execute("ALTER TABLE tag ALTER COLUMN kind DROP DEFAULT")
+
+    # 4. Recreate the tag_kind enum without meta/rating.
+    op.execute("ALTER TYPE tag_kind RENAME TO tag_kind_old")
+    op.execute(
+        "CREATE TYPE tag_kind AS ENUM ("
+        "'artist', 'character', 'fandom', 'general', "
+        "'series', 'archive', 'post'"
+        ")"
+    )
+    op.execute(
+        "ALTER TABLE tag "
+        "ALTER COLUMN kind TYPE tag_kind "
+        "USING kind::text::tag_kind"
+    )
+    op.execute("DROP TYPE tag_kind_old")
+
+    # 5. Restore the server default.
+    op.execute("ALTER TABLE tag ALTER COLUMN kind SET DEFAULT 'general'")
+
+    # 6. Restore the CHECK constraint (now bound to the new tag_kind).
+    op.create_check_constraint(
+        "ck_tag_fandom_requires_character",
+        "tag",
+        "(fandom_id IS NULL) OR (kind = 'character')",
+    )
+
+
+def downgrade() -> None:
+    # Add the values back to the enum so old code can boot. The deleted
+    # tag rows are gone permanently — no safe restore.
+    op.drop_constraint(
+        "ck_tag_fandom_requires_character", "tag", type_="check"
+    )
+    op.execute("ALTER TABLE tag ALTER COLUMN kind DROP DEFAULT")
+    op.execute("ALTER TYPE tag_kind RENAME TO tag_kind_old")
+    op.execute(
+        "CREATE TYPE tag_kind AS ENUM ("
+        "'artist', 'character', 'fandom', 'general', "
+        "'series', 'archive', 'post', 'meta', 'rating'"
+        ")"
+    )
+    op.execute(
+        "ALTER TABLE tag "
+        "ALTER COLUMN kind TYPE tag_kind "
+        "USING kind::text::tag_kind"
+    )
+    op.execute("DROP TYPE tag_kind_old")
+    op.execute("ALTER TABLE tag ALTER COLUMN kind SET DEFAULT 'general'")
+    op.create_check_constraint(
+        "ck_tag_fandom_requires_character",
+        "tag",
+        "(fandom_id IS NULL) OR (kind = 'character')",
+    )

alembic/versions/0024_backfill_post_title_from_description.py

@@ -0,0 +1,80 @@
+"""backfill post.post_title from description first-line — 2026-05-27
+
+Revision ID: 0024
+Revises: 0023
+Create Date: 2026-05-27
+
+SubscribeStar gallery-dl always writes `title: ""` and embeds the leading
+sentence inside `content` HTML. FC's sidecar parser was leaving
+post_title NULL for every SubscribeStar post since FC-3 shipped. The
+parser fix (sidecar._first_line_text fallback) now synthesizes a title
+at parse time; this migration applies the same logic retroactively to
+existing rows.
+
+Operator-flagged 2026-05-27 after inspecting
+/mnt/Data/Patreon/Cheunart/subscribestar/ sidecars.
+
+Idempotent: only touches rows where post_title IS NULL or empty AND
+description IS NOT NULL. Re-running the migration is a no-op.
+"""
+from __future__ import annotations
+
+import re
+from typing import Sequence, Union
+
+from alembic import op
+from sqlalchemy import text
+
+revision: str = "0024"
+down_revision: Union[str, None] = "0023"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+_TAG_RE = re.compile(r"<[^>]+>")
+_WS_RE = re.compile(r"\s+")
+
+
+def _first_line_text(body: str, limit: int = 120) -> str | None:
+    """Mirror of sidecar._first_line_text. Kept inline so the migration
+    doesn't carry a runtime import dependency from app code that may
+    have moved by the time the migration is replayed years from now."""
+    if not body:
+        return None
+    text_ = _TAG_RE.sub(" ", body)
+    text_ = text_.replace("\xa0", " ")
+    for line in text_.splitlines():
+        line = _WS_RE.sub(" ", line).strip()
+        if line:
+            if len(line) > limit:
+                return line[: limit - 1].rstrip() + "…"
+            return line
+    return None
+
+
+def upgrade() -> None:
+    bind = op.get_bind()
+    rows = bind.execute(
+        text(
+            "SELECT id, description FROM post "
+            "WHERE (post_title IS NULL OR post_title = '') "
+            "AND description IS NOT NULL AND description <> ''"
+        )
+    ).fetchall()
+    updated = 0
+    for row in rows:
+        derived = _first_line_text(row.description)
+        if not derived:
+            continue
+        bind.execute(
+            text("UPDATE post SET post_title = :t WHERE id = :id"),
+            {"t": derived, "id": row.id},
+        )
+        updated += 1
+    print(f"0024: backfilled post_title on {updated} row(s)")
+
+
+def downgrade() -> None:
+    # No safe restore — we can't tell which post_titles were derived vs
+    # genuinely present. Leave the column alone on rollback.
+    pass

alembic/versions/0025_fix_subscribestar_post_ids.py

@@ -0,0 +1,288 @@
+"""sidecar-audit followup: correct external_post_id + post_url across all platforms
+
+Revision ID: 0025
+Revises: 0024
+Create Date: 2026-05-27
+
+Closes the operator-flagged 2026-05-27 sidecar audit findings. Three
+data-correctness bugs across non-Patreon platforms had been silently
+corrupting Posts since FC-3 shipped; the parser fix (sidecar.py, same
+commit) addresses new imports. This migration cleans up existing rows.
+
+Per-platform actions:
+
+  subscribestar — gallery-dl wrote the per-attachment id in `id` and
+    the actual post id in `post_id`. FC's parser picked `id`, so every
+    multi-image SubscribeStar post was fragmented into N Post rows.
+    1. For each SubscribeStar Post, read its sidecar (via the related
+       ImageRecord's on-disk path), pull `post_id`, overwrite
+       external_post_id and post_url.
+    2. Merge groups of Posts under one source that now share an
+       external_post_id (fragments of the same actual post). Same
+       ImageProvenance pre-delete + repoint dance as alembic 0022.
+
+  hentaifoundry — sidecars have NO `url` field; `src` is the image
+    URL. FC's parser stored post_url=NULL. Read each HF Post's sidecar
+    for `user` + `index`, derive the canonical /pictures/user/<u>/<i>
+    permalink. external_post_id (= `index`) was already correct.
+
+  discord — gallery-dl wrote the CDN attachment URL in `url`. FC's
+    parser stored that as post_url. Read each Discord Post's sidecar
+    for the server/channel/message triple, derive the proper
+    discord.com/channels/.../<message> permalink. external_post_id (=
+    `message_id`) was already correct.
+
+  pixiv — pure-SQL backfill: replace any `i.pximg.net`-style URL on
+    Post.post_url with the derived `/artworks/<id>` permalink. Pixiv
+    external_post_id (= `id`) was already correct; no sidecar IO
+    needed.
+
+Idempotent: re-running on already-corrected data is a no-op (skips
+rows whose derived value matches what's already stored).
+
+Posts whose related ImageRecord paths don't resolve on disk (orphaned
+filesystem state) are skipped with a count in the migration output —
+those will be picked up by a future deep-scan.
+"""
+from __future__ import annotations
+
+import json
+import re
+from pathlib import Path
+from typing import Sequence, Union
+
+from alembic import op
+from sqlalchemy import text
+
+revision: str = "0025"
+down_revision: Union[str, None] = "0024"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+# Mirror of sidecar._NUMBERING_PREFIX. Kept inline so the migration is
+# self-contained (the operator's banked rule:
+# reference_postgres_enum_swap_drop_checks.md says migrations shouldn't
+# import from runtime app code).
+_NUMBERING_PREFIX = re.compile(r"^\d+_(.+)$")
+
+
+def _find_sidecar(media_path: Path) -> Path | None:
+    """gallery-dl writes the sidecar under the unprefixed stem
+    (`HOLLOW-ICHIGO.json`) while the media file gets a NN_ ordering
+    prefix (`01_HOLLOW-ICHIGO.png`). Try in order:
+      1. <stem>.json next to the media
+      2. <media>.json next to the media (full-name variant)
+      3. strip the NN_ prefix from the stem, then <stripped>.json
+    """
+    if not media_path:
+        return None
+    cand = media_path.with_suffix(".json")
+    if cand.is_file():
+        return cand
+    cand = media_path.parent / f"{media_path.name}.json"
+    if cand.is_file():
+        return cand
+    m = _NUMBERING_PREFIX.match(media_path.stem)
+    if m:
+        cand = media_path.parent / f"{m.group(1)}.json"
+        if cand.is_file():
+            return cand
+    return None
+
+
+def _str_id(v) -> str | None:
+    """str() a JSON scalar id; reject bool (JSON booleans are ints in
+    Python's eyes but they aren't valid sidecar ids)."""
+    if isinstance(v, bool):
+        return None
+    if isinstance(v, (str, int)) and str(v).strip():
+        return str(v).strip()
+    return None
+
+
+def _str_field(v) -> str | None:
+    if isinstance(v, str) and v.strip():
+        return v.strip()
+    return None
+
+
+def upgrade() -> None:
+    conn = op.get_bind()
+
+    # ── PART 1: Per-platform corrections requiring filesystem IO ─────
+    # SubscribeStar, HentaiFoundry, Discord all need fields from the
+    # sidecar to construct the right post_url. We walk each Post's
+    # related ImageRecord.path to find the sidecar, read it, derive,
+    # and update.
+    targets = conn.execute(text("""
+        SELECT p.id, p.external_post_id, p.post_url, s.platform
+        FROM post p
+        JOIN source s ON s.id = p.source_id
+        WHERE s.platform IN ('subscribestar', 'hentaifoundry', 'discord')
+    """)).fetchall()
+
+    stats: dict[str, dict[str, int]] = {
+        plat: {"read": 0, "updated": 0, "no_sidecar": 0}
+        for plat in ("subscribestar", "hentaifoundry", "discord")
+    }
+    for post_row in targets:
+        plat = post_row.platform
+        path = _first_attachment_path(conn, post_row.id)
+        if not path:
+            stats[plat]["no_sidecar"] += 1
+            continue
+        sidecar = _find_sidecar(Path(path))
+        if sidecar is None:
+            stats[plat]["no_sidecar"] += 1
+            continue
+        try:
+            data = json.loads(sidecar.read_text(encoding="utf-8"))
+        except (OSError, json.JSONDecodeError):
+            stats[plat]["no_sidecar"] += 1
+            continue
+        stats[plat]["read"] += 1
+
+        new_epid = post_row.external_post_id
+        new_url = None
+        if plat == "subscribestar":
+            pid = _str_id(data.get("post_id"))
+            if pid:
+                new_epid = pid
+                new_url = f"https://www.subscribestar.com/posts/{pid}"
+        elif plat == "hentaifoundry":
+            user = _str_field(data.get("user")) or _str_field(data.get("artist"))
+            idx = _str_id(data.get("index"))
+            if user and idx:
+                new_url = f"https://www.hentai-foundry.com/pictures/user/{user}/{idx}"
+        elif plat == "discord":
+            sid = _str_id(data.get("server_id"))
+            cid = _str_id(data.get("channel_id"))
+            mid = _str_id(data.get("message_id"))
+            if sid and cid and mid:
+                new_url = f"https://discord.com/channels/{sid}/{cid}/{mid}"
+
+        # Idempotent: skip if nothing changed.
+        if new_epid == post_row.external_post_id and new_url == post_row.post_url:
+            continue
+        conn.execute(
+            text("""
+                UPDATE post
+                SET external_post_id = :epid, post_url = :url
+                WHERE id = :id
+            """),
+            {"epid": new_epid, "url": new_url, "id": post_row.id},
+        )
+        stats[plat]["updated"] += 1
+
+    for plat, s in stats.items():
+        print(
+            f"0025: {plat} — read {s['read']} sidecars, "
+            f"updated {s['updated']} Posts, "
+            f"{s['no_sidecar']} Posts had no resolvable sidecar"
+        )
+
+    # ── PART 2: Merge SubscribeStar fragments now sharing epid ───────
+    # After Part 1, each group of Posts under one source with the SAME
+    # new external_post_id is a fragment-set of the same actual post.
+    # Merge to one canonical row. Pre-handle the same ImageProvenance
+    # collision pattern as alembic 0022 (uq_image_provenance_image_post).
+    fragment_groups = conn.execute(text("""
+        SELECT p.source_id, p.external_post_id,
+               ARRAY_AGG(p.id ORDER BY p.id ASC) AS post_ids
+        FROM post p
+        JOIN source s ON s.id = p.source_id
+        WHERE s.platform = 'subscribestar'
+          AND p.external_post_id IS NOT NULL
+        GROUP BY p.source_id, p.external_post_id
+        HAVING COUNT(*) > 1
+    """)).fetchall()
+
+    merged = 0
+    for grp in fragment_groups:
+        post_ids = list(grp.post_ids)
+        keep_id, *drop_ids = post_ids
+        for drop_id in drop_ids:
+            # Pre-DELETE colliding ImageProvenance under drop_ that
+            # already exist under keep (alembic 0022 banked the pattern).
+            conn.execute(
+                text("""
+                    DELETE FROM image_provenance
+                    WHERE post_id = :drop_
+                      AND image_record_id IN (
+                          SELECT image_record_id FROM image_provenance
+                          WHERE post_id = :keep
+                      )
+                """),
+                {"keep": keep_id, "drop_": drop_id},
+            )
+            conn.execute(
+                text("""
+                    UPDATE image_provenance SET post_id = :keep
+                    WHERE post_id = :drop_
+                """),
+                {"keep": keep_id, "drop_": drop_id},
+            )
+            conn.execute(
+                text("""
+                    UPDATE image_record SET primary_post_id = :keep
+                    WHERE primary_post_id = :drop_
+                """),
+                {"keep": keep_id, "drop_": drop_id},
+            )
+            conn.execute(
+                text("""
+                    UPDATE post_attachment SET post_id = :keep
+                    WHERE post_id = :drop_
+                """),
+                {"keep": keep_id, "drop_": drop_id},
+            )
+            conn.execute(
+                text("DELETE FROM post WHERE id = :drop_"),
+                {"drop_": drop_id},
+            )
+            merged += 1
+    print(f"0025: subscribestar — merged {merged} duplicate Post fragments")
+
+    # ── PART 3: Pixiv post_url backfill (pure SQL) ───────────────────
+    # Pixiv's external_post_id is already correct (gallery-dl's `id` is
+    # the post id). Only post_url needs derivation: replace anything
+    # under i.pximg.net (the file URL) with the /artworks/<id> permalink.
+    pixiv_updated = conn.execute(text("""
+        UPDATE post p
+        SET post_url = 'https://www.pixiv.net/artworks/' || p.external_post_id
+        FROM source s
+        WHERE p.source_id = s.id
+          AND s.platform = 'pixiv'
+          AND p.external_post_id IS NOT NULL
+          AND (p.post_url IS NULL
+               OR p.post_url LIKE 'https://i.pximg.net/%'
+               OR p.post_url LIKE 'http://i.pximg.net/%')
+    """)).rowcount
+    print(f"0025: pixiv — backfilled post_url on {pixiv_updated} Posts")
+
+
+def _first_attachment_path(conn, post_id: int) -> str | None:
+    """Return any ImageRecord.path attached to this post (via
+    ImageProvenance). Lowest-id row keeps the migration deterministic
+    so re-running on the same DB picks the same sidecar."""
+    row = conn.execute(
+        text("""
+            SELECT ir.path
+            FROM image_provenance ip
+            JOIN image_record ir ON ir.id = ip.image_record_id
+            WHERE ip.post_id = :pid
+            ORDER BY ip.id ASC
+            LIMIT 1
+        """),
+        {"pid": post_id},
+    ).first()
+    return row[0] if row else None
+
+
+def downgrade() -> None:
+    # Lossy: external_post_id values were overwritten with the correct
+    # post_id; original per-attachment ids weren't preserved. Post-merge
+    # also deleted drop rows. No safe restore. To roll back the schema
+    # invariant, fork from 0024 and re-run sidecar imports.
+    pass

alembic/versions/0026_import_task_recovery_count_refetched.py

@@ -0,0 +1,53 @@
+"""import_task.recovery_count + refetched — poison-pill circuit breaker
+
+Revision ID: 0026
+Revises: 0025
+Create Date: 2026-05-28
+
+Backs the import-task resilience work (operator-flagged 2026-05-28):
+
+- recovery_count: how many times recover_interrupted_tasks has
+  re-queued this row from a stuck 'processing' state. A row that
+  hard-crashes the worker (OOM / segfault on a corrupt or oversized
+  input) leaves no terminal flip, so the sweep re-queues it — and
+  without a cap it would loop forever, re-crashing the worker each
+  time. After MAX_RECOVERY_ATTEMPTS the sweep marks it 'failed' with a
+  diagnostic instead.
+
+- refetched: whether a one-shot re-download has already been attempted
+  for this task's file. Bounds the Layer-2 re-fetch remediation to a
+  single attempt so source-side corruption doesn't loop.
+
+Both default to 0 / false; additive, no backfill needed.
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0026"
+down_revision: Union[str, None] = "0025"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "import_task",
+        sa.Column(
+            "recovery_count", sa.Integer(), nullable=False,
+            server_default="0",
+        ),
+    )
+    op.add_column(
+        "import_task",
+        sa.Column(
+            "refetched", sa.Boolean(), nullable=False,
+            server_default=sa.false(),
+        ),
+    )
+
+
+def downgrade() -> None:
+    op.drop_column("import_task", "refetched")
+    op.drop_column("import_task", "recovery_count")

alembic/versions/0027_drop_migration_run.py

@@ -0,0 +1,50 @@
+"""drop migration_run — one-and-done GS/IR migration tooling removed
+
+Revision ID: 0027
+Revises: 0026
+Create Date: 2026-05-29
+
+The GS/IR migration tooling (services/migrators, /api/migrate, the
+run_migration task, LegacyMigrationCard, and the MigrationRun model) was
+removed after the migration cutover completed. This drops its now-orphaned
+run-log table. Downgrade recreates the table (mirrors the old model) so the
+migration is reversible.
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+from sqlalchemy.dialects.postgresql import JSONB
+
+revision: str = "0027"
+down_revision: Union[str, None] = "0026"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.drop_table("migration_run")
+
+
+def downgrade() -> None:
+    op.create_table(
+        "migration_run",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column("kind", sa.String(length=32), nullable=False),
+        sa.Column("status", sa.String(length=32), nullable=False),
+        sa.Column("dry_run", sa.Boolean(), nullable=False, server_default=sa.false()),
+        sa.Column(
+            "started_at", sa.DateTime(timezone=True), nullable=False,
+            server_default=sa.func.now(),
+        ),
+        sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
+        sa.Column(
+            "counts", JSONB(), nullable=False, server_default=sa.text("'{}'::jsonb"),
+        ),
+        sa.Column("error", sa.Text(), nullable=True),
+        sa.Column(
+            "metadata", JSONB(), nullable=False, server_default=sa.text("'{}'::jsonb"),
+        ),
+    )
+    op.create_index("ix_migration_run_kind", "migration_run", ["kind"])
+    op.create_index("ix_migration_run_status", "migration_run", ["status"])

alembic/versions/0028_collapse_sidecar_synthetics_into_real_sources.py

@@ -0,0 +1,190 @@
+"""collapse-sidecar-synthetic: repoint Posts/ImageProvenance/DownloadEvents
+from `sidecar:<platform>:<slug>` synthetic Source anchors onto the real
+Source for the same (artist, platform) when one exists, then delete the
+synthetic.
+
+Revision ID: 0028
+Revises: 0027
+Create Date: 2026-05-31
+
+Background: alembic 0022 (2026-05-26) consolidated the old per-post-URL
+Source rows into one canonical Source per (artist, platform). When NO
+real campaign URL was salvageable among the candidates, it rewrote the
+canonical row to url='sidecar:<platform>:<slug>' enabled=false as a
+disabled anchor for any Posts already attached.
+
+That was fine while it was the only Source for that artist+platform.
+But: the unique constraint on Source is (artist_id, platform, url), not
+(artist_id, platform). When the operator later added the real
+subscription via the UI / extension / etc., a SECOND row landed —
+the real one — with id > the synthetic. Both coexisted.
+
+Two follow-on problems surfaced 2026-05-31:
+
+  1. The Subscriptions UI listed both rows. The synthetic was disabled
+     so the scheduler never polled it, but it looked like a phantom
+     subscription. (Fixed in same commit by SourceService.list filter.)
+  2. importer._source_for_sidecar picked Source by `ORDER BY id ASC
+     LIMIT 1`, so EVERY gallery-dl download since the real Source was
+     added attached its Post to the SYNTHETIC anchor, not the real
+     Source. (Fixed in same commit by preferring non-sidecar URLs.)
+
+This migration is the data half of the cleanup: for every (artist,
+platform) with both a synthetic AND a real Source, repoint the
+synthetic's children (Posts, ImageProvenance, DownloadEvents) onto the
+real Source and delete the synthetic. Reuses the same epid/provenance
+collision dance from alembic 0022 because the same uniqueness
+constraints fire row-by-row during bulk UPDATEs.
+
+Lone synthetic anchors — those where no real Source for the same
+(artist, platform) exists (e.g., filesystem-imported artist with no
+subscription added) — are LEFT INTACT. They anchor real imported
+content; deleting them would CASCADE-delete the Posts the operator
+imported. The SourceService.list filter hides them from the UI; the
+operator can delete them by hand if they want the underlying imports
+gone.
+"""
+from typing import Sequence, Union
+
+from alembic import op
+from sqlalchemy import text
+
+revision: str = "0028"
+down_revision: Union[str, None] = "0027"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    conn = op.get_bind()
+
+    # Find (artist_id, platform) groups where BOTH a sidecar synthetic
+    # and at least one real Source exist.
+    groups = conn.execute(text("""
+        SELECT artist_id, platform
+        FROM source
+        GROUP BY artist_id, platform
+        HAVING bool_or(url LIKE 'sidecar:%')
+           AND bool_or(url NOT LIKE 'sidecar:%')
+    """)).fetchall()
+
+    for artist_id, platform in groups:
+        rows = conn.execute(
+            text("""
+                SELECT id, url FROM source
+                WHERE artist_id = :a AND platform = :p
+                ORDER BY id ASC
+            """),
+            {"a": artist_id, "p": platform},
+        ).fetchall()
+
+        synthetic_ids = [sid for sid, url in rows if url.startswith("sidecar:")]
+        real_rows = [(sid, url) for sid, url in rows if not url.startswith("sidecar:")]
+        if not synthetic_ids or not real_rows:
+            continue  # belt+suspenders; the GROUP BY already filtered
+
+        # Canonical real: lowest-id non-sidecar Source.
+        canonical_id = real_rows[0][0]
+
+        # STEP A: PRE-merge Post collisions on (canonical, external_post_id).
+        # Mirror alembic 0022's pre-merge logic — when synth has Post X
+        # epid=N and real has Post Y epid=N, the bulk UPDATE below would
+        # trip uq_post_source_external_id row-by-row. Group all Posts
+        # under (canonical + synthetics) by epid; for any group >1,
+        # pick a keep (prefer one already under canonical, else lowest
+        # id) and merge the rest into it.
+        all_posts = conn.execute(
+            text("""
+                SELECT external_post_id, id, source_id
+                FROM post
+                WHERE source_id = :canonical OR source_id = ANY(:synths)
+                ORDER BY external_post_id, id
+            """),
+            {"canonical": canonical_id, "synths": synthetic_ids},
+        ).fetchall()
+        by_epid: dict = {}
+        for epid, post_id, src_id in all_posts:
+            by_epid.setdefault(epid, []).append((post_id, src_id))
+        for _epid, posts in by_epid.items():
+            if len(posts) <= 1:
+                continue
+            canonical_side = [p for p in posts if p[1] == canonical_id]
+            keep_id = canonical_side[0][0] if canonical_side else posts[0][0]
+            drop_ids = [p[0] for p in posts if p[0] != keep_id]
+            for drop_id in drop_ids:
+                # Pre-delete image_provenance rows under drop_ whose
+                # image_record_id already has provenance under keep —
+                # avoids tripping uq_image_provenance_image_post (0021)
+                # row-by-row during the repoint UPDATE.
+                conn.execute(
+                    text("""
+                        DELETE FROM image_provenance
+                        WHERE post_id = :drop_
+                          AND image_record_id IN (
+                              SELECT image_record_id FROM image_provenance
+                              WHERE post_id = :keep
+                          )
+                    """),
+                    {"keep": keep_id, "drop_": drop_id},
+                )
+                conn.execute(
+                    text("""
+                        UPDATE image_provenance SET post_id = :keep
+                        WHERE post_id = :drop_
+                    """),
+                    {"keep": keep_id, "drop_": drop_id},
+                )
+                conn.execute(
+                    text("""
+                        UPDATE image_record SET primary_post_id = :keep
+                        WHERE primary_post_id = :drop_
+                    """),
+                    {"keep": keep_id, "drop_": drop_id},
+                )
+                conn.execute(
+                    text("DELETE FROM post WHERE id = :drop_"),
+                    {"drop_": drop_id},
+                )
+
+        # STEP B: Bulk reparent the remaining Posts off the synthetics.
+        conn.execute(
+            text("""
+                UPDATE post SET source_id = :canonical
+                WHERE source_id = ANY(:synths)
+            """),
+            {"canonical": canonical_id, "synths": synthetic_ids},
+        )
+
+        # STEP C: Reparent ImageProvenance.source_id (denormalized FK;
+        # no UNIQUE on source_id, safe bulk).
+        conn.execute(
+            text("""
+                UPDATE image_provenance SET source_id = :canonical
+                WHERE source_id = ANY(:synths)
+            """),
+            {"canonical": canonical_id, "synths": synthetic_ids},
+        )
+
+        # STEP D: Reparent any DownloadEvent.source_id. Synthetics are
+        # enabled=false so the scheduler never created events for them;
+        # this is belt+suspenders for any rows planted by manual force
+        # or older code paths.
+        conn.execute(
+            text("""
+                UPDATE download_event SET source_id = :canonical
+                WHERE source_id = ANY(:synths)
+            """),
+            {"canonical": canonical_id, "synths": synthetic_ids},
+        )
+
+        # STEP E: Drop the now-empty synthetics.
+        conn.execute(
+            text("DELETE FROM source WHERE id = ANY(:synths)"),
+            {"synths": synthetic_ids},
+        )
+
+
+def downgrade() -> None:
+    # Lossy migration — synthetic Sources deleted, Posts repointed and
+    # potentially merged. No safe downgrade.
+    pass

backend/app/__init__.py

@@ -3,13 +3,23 @@
 import logging
 from pathlib import Path
 
-from quart import Quart
+from quart import Quart, request
 
 from .api import all_blueprints
 from .config import get_config
 from .frontend import frontend_bp
 from .services.credential_crypto import CredentialCrypto
 
+# Browser-extension origins. The FabledCurator extension fetches from
+# moz-extension://<uuid>/ on Firefox and chrome-extension://<uuid>/ on
+# Chromium-based browsers. Operator-flagged 2026-05-26: extension's
+# 'Test connection' returned `NetworkError` because the X-Extension-Key
+# header on /api/credentials triggers a CORS preflight that our routes
+# don't handle. Whitelisting only these two schemes (not opening CORS
+# up generally) lets the extension talk to a plain-HTTP self-hosted FC
+# without weakening the no-CORS posture for normal browser usage.
+_EXTENSION_ORIGIN_SCHEMES = ("moz-extension://", "chrome-extension://")
+
 _CREDENTIAL_KEY_PATH = Path("/images/secrets/credential_key.b64")
 
 
@@ -23,18 +33,39 @@ def create_app() -> Quart:
 
     app = Quart(__name__)
     app.secret_key = cfg.secret_key
-    # FC-5: legacy IR ingest JSON can run to tens of MB (hundreds of
-    # thousands of image_tag_associations). Werkzeug's default form
-    # memory cap is 500KB; raise both ceilings so the multipart upload
-    # for /api/migrate/ir_ingest doesn't 413.
-    app.config["MAX_CONTENT_LENGTH"] = 1024 * 1024 * 1024   # 1 GB
-    app.config["MAX_FORM_MEMORY_SIZE"] = 1024 * 1024 * 1024  # 1 GB
 
     for bp in all_blueprints():
         app.register_blueprint(bp)
     # Registered last so /api/* routes win over the SPA catch-all.
     app.register_blueprint(frontend_bp)
 
+    @app.before_request
+    async def _extension_cors_preflight():
+        # Short-circuit OPTIONS preflight from the browser extension with a
+        # 204 + CORS headers (the after_request hook below adds them).
+        # Without this, OPTIONS lands on routes that only declared POST/GET
+        # methods and 405s before the after_request gets a chance.
+        if request.method != "OPTIONS":
+            return None
+        origin = request.headers.get("Origin", "")
+        if any(origin.startswith(s) for s in _EXTENSION_ORIGIN_SCHEMES):
+            return "", 204
+        return None
+
+    @app.after_request
+    async def _extension_cors_headers(response):
+        origin = request.headers.get("Origin", "")
+        if any(origin.startswith(s) for s in _EXTENSION_ORIGIN_SCHEMES):
+            response.headers["Access-Control-Allow-Origin"] = origin
+            response.headers["Access-Control-Allow-Methods"] = (
+                "GET, POST, PATCH, DELETE, OPTIONS"
+            )
+            response.headers["Access-Control-Allow-Headers"] = (
+                "Content-Type, X-Extension-Key"
+            )
+            response.headers["Access-Control-Max-Age"] = "86400"
+        return response
+
     @app.after_serving
     async def _dispose_db_engine() -> None:
         from .extensions import dispose_engine

backend/app/api/__init__.py

@@ -14,16 +14,18 @@ api_bp.add_url_rule("/health", view_func=health.get_health, methods=["GET"])
 
 
 def all_blueprints() -> list[Blueprint]:
+    from .admin import admin_bp
     from .aliases import aliases_bp
     from .allowlist import allowlist_bp
     from .artist import artist_bp
     from .artists import artists_bp
     from .attachments import attachments_bp
+    from .cleanup import cleanup_bp
     from .credentials import credentials_bp
     from .downloads import downloads_bp
+    from .extension import extension_bp
     from .gallery import gallery_bp
     from .import_admin import import_admin_bp
-    from .migrate import migrate_bp
     from .ml_admin import ml_admin_bp
     from .platforms import platforms_bp
     from .posts import posts_bp
@@ -32,7 +34,10 @@ def all_blueprints() -> list[Blueprint]:
     from .showcase import showcase_bp
     from .sources import sources_bp
     from .suggestions import suggestions_bp
+    from .system_activity import system_activity_bp
+    from .system_backup import system_backup_bp
     from .tags import tags_bp
+    from .thumbnails import thumbnails_bp
     return [
         api_bp,
         attachments_bp,
@@ -43,15 +48,20 @@ def all_blueprints() -> list[Blueprint]:
         artists_bp,
         showcase_bp,
         settings_bp,
+        system_activity_bp,
+        system_backup_bp,
+        admin_bp,
+        cleanup_bp,
         import_admin_bp,
-        migrate_bp,
         suggestions_bp,
         allowlist_bp,
         aliases_bp,
         ml_admin_bp,
+        thumbnails_bp,
         sources_bp,
         platforms_bp,
         posts_bp,
         credentials_bp,
+        extension_bp,
         downloads_bp,
     ]

backend/app/api/_responses.py

@@ -0,0 +1,16 @@
+"""Shared API response helpers."""
+
+from quart import jsonify
+
+
+def error_response(
+    error: str, *, status: int = 400, detail: str | None = None, **extra,
+):
+    """JSON error body + HTTP status. `detail` is included only when given;
+    `extra` keys are merged into the body. Returns the (response, status)
+    tuple Quart expects. Imported as `_bad` by the blueprints."""
+    body = {"error": error}
+    if detail is not None:
+        body["detail"] = detail
+    body.update(extra)
+    return jsonify(body), status

backend/app/api/admin.py

@@ -0,0 +1,224 @@
+"""FC-3k: /api/admin — destructive admin actions.
+
+Five action surfaces:
+  POST   /api/admin/artists/<slug>/cascade-delete  (Tier C)
+  POST   /api/admin/images/bulk-delete             (Tier C)
+  DELETE /api/admin/tags/<int:tag_id>              (Tier B)
+  POST   /api/admin/tags/<int:dest_id>/merge       (Tier B)
+  POST   /api/admin/tags/prune-unused              (Tier A)
+  POST   /api/admin/tags/purge-legacy              (Tier A)
+  GET    /api/admin/tags/<int:tag_id>/usage-count  (helper)
+
+Tier-C ops take a dry_run body flag (returns projection inline,
+no dispatch) and a confirm body field (server-recomputed token).
+Long-running ops dispatch a maintenance-queue Celery task; the UI
+tails FC-3i's /api/system/activity/runs to surface progress.
+"""
+from __future__ import annotations
+
+import hashlib
+
+from quart import Blueprint, jsonify, request
+from sqlalchemy import select
+
+from ..extensions import get_session
+from ..models import Artist
+from ..services.cleanup_service import project_artist_cascade, project_bulk_image_delete
+from ._responses import error_response as _bad
+
+admin_bp = Blueprint("admin", __name__, url_prefix="/api/admin")
+
+
+def _bulk_image_confirm_token(image_ids: list[int]) -> str:
+    """Stable 8-hex token derived from the sorted id list. Mutates
+    when the selection changes; stays the same across modal opens of
+    the same selection so the operator can paste without confusion."""
+    canon = ",".join(str(i) for i in sorted(image_ids))
+    digest = hashlib.sha256(canon.encode("utf-8")).hexdigest()
+    return digest[:8]
+
+
+@admin_bp.route("/artists/<slug>/cascade-delete", methods=["POST"])
+async def artist_cascade_delete(slug: str):
+    body = await request.get_json(silent=True) or {}
+    dry_run = bool(body.get("dry_run", False))
+    supplied_confirm = body.get("confirm", "")
+
+    async with get_session() as session:
+        artist = (await session.execute(
+            select(Artist).where(Artist.slug == slug)
+        )).scalar_one_or_none()
+        if artist is None:
+            return _bad("not_found", status=404)
+        artist_id = artist.id
+
+        projected = await session.run_sync(
+            lambda sync_sess: project_artist_cascade(sync_sess, slug=slug)
+        )
+
+    if dry_run:
+        return jsonify(projected)
+
+    expected = f"delete-artist-{artist_id}"
+    if supplied_confirm != expected:
+        return _bad(
+            "confirm_mismatch",
+            detail=f"confirm must equal {expected!r}",
+            expected=expected,
+        )
+
+    from ..tasks.admin import delete_artist_cascade_task
+    async_result = delete_artist_cascade_task.delay(artist_id=artist_id)
+    return jsonify({"task_id": async_result.id}), 202
+
+
+@admin_bp.route("/images/bulk-delete", methods=["POST"])
+async def images_bulk_delete():
+    body = await request.get_json(silent=True) or {}
+    image_ids = body.get("image_ids")
+    if not isinstance(image_ids, list) or not image_ids:
+        return _bad("invalid_image_ids", detail="image_ids must be non-empty list of int")
+    try:
+        image_ids = [int(i) for i in image_ids]
+    except (TypeError, ValueError):
+        return _bad("invalid_image_ids", detail="image_ids must contain only ints")
+
+    dry_run = bool(body.get("dry_run", False))
+    supplied_confirm = body.get("confirm", "")
+
+    async with get_session() as session:
+        projected = await session.run_sync(
+            lambda sync_sess: project_bulk_image_delete(
+                sync_sess, image_ids=image_ids,
+            )
+        )
+
+    sha8 = _bulk_image_confirm_token(image_ids)
+    expected = f"delete-images-{sha8}"
+
+    if dry_run:
+        # Hand the canonical Tier-C confirm token back with the
+        # projection so the frontend doesn't have to recompute SHA-256
+        # client-side via crypto.subtle (Secure-Context-gated,
+        # undefined on plain-HTTP origins per the homelab posture).
+        # Operator-flagged 2026-05-27.
+        projected["confirm_token"] = expected
+        return jsonify(projected)
+
+
+    if supplied_confirm != expected:
+        return _bad(
+            "confirm_mismatch",
+            detail=f"confirm must equal {expected!r}",
+            expected=expected,
+        )
+
+    from ..tasks.admin import bulk_delete_images_task
+    async_result = bulk_delete_images_task.delay(image_ids=image_ids)
+    return jsonify({"task_id": async_result.id}), 202
+
+
+@admin_bp.route("/tags/<int:tag_id>", methods=["DELETE"])
+async def tag_delete(tag_id: int):
+    """Tier-B sync delete. UI yes/no modal is the only confirmation."""
+    from ..services.cleanup_service import delete_tag
+
+    async with get_session() as session:
+        try:
+            result = await session.run_sync(
+                lambda sync_sess: delete_tag(sync_sess, tag_id=tag_id)
+            )
+        except LookupError:
+            return _bad("not_found", status=404)
+    return jsonify(result)
+
+
+@admin_bp.route("/tags/<int:dest_id>/merge", methods=["POST"])
+async def tag_merge(dest_id: int):
+    """Wraps TagService.merge. Source repoints to dest, dest survives,
+    source row deleted, protective alias auto-created if source was
+    ML-applied or allowlisted."""
+    from ..services.tag_service import TagMergeConflict, TagService, TagValidationError
+
+    body = await request.get_json(silent=True) or {}
+    source_id = body.get("source_id")
+    if not isinstance(source_id, int) or source_id == dest_id:
+        return _bad("invalid_source_id", detail="source_id must be int and differ from dest")
+
+    async with get_session() as session:
+        try:
+            result = await TagService(session).merge(
+                source_id=source_id, target_id=dest_id,
+            )
+        except TagMergeConflict as exc:
+            return _bad("merge_conflict", status=409, detail=str(exc))
+        except TagValidationError as exc:
+            return _bad("tag_kind_mismatch", detail=str(exc))
+        except LookupError:
+            return _bad("not_found", status=404)
+
+    # MergeResult is a frozen dataclass — flatten to dict.
+    return jsonify({
+        "result": {
+            "target_id": result.target_id,
+            "target_name": result.target_name,
+            "target_kind": result.target_kind,
+            "merged_count": result.merged_count,
+            "alias_created": result.alias_created,
+            "source_deleted": result.source_deleted,
+        },
+    })
+
+
+@admin_bp.route("/tags/<int:tag_id>/usage-count", methods=["GET"])
+async def tag_usage_count(tag_id: int):
+    """Helper for the Tier-B yes/no prompt; surfaces "N associations"
+    in the dialog so the operator knows what they're nuking."""
+    from ..services.cleanup_service import count_tag_associations
+
+    async with get_session() as session:
+        count = await session.run_sync(
+            lambda sync_sess: count_tag_associations(
+                sync_sess, tag_id=tag_id,
+            )
+        )
+    return jsonify({"count": count})
+
+
+@admin_bp.route("/tags/prune-unused", methods=["POST"])
+async def tags_prune_unused():
+    """Tier-A: dry-run preview list IS the prompt. UI calls with
+    dry_run=true first, shows the list, operator clicks button to
+    re-call with dry_run=false."""
+    from ..services.cleanup_service import prune_unused_tags
+
+    body = await request.get_json(silent=True) or {}
+    dry_run = bool(body.get("dry_run", False))
+
+    async with get_session() as session:
+        result = await session.run_sync(
+            lambda sync_sess: prune_unused_tags(
+                sync_sess, dry_run=dry_run,
+            )
+        )
+    return jsonify(result)
+
+
+@admin_bp.route("/tags/purge-legacy", methods=["POST"])
+async def tags_purge_legacy():
+    """Tier-A: delete legacy IR-migration tags — archive/post/artist
+    kinds (e.g. `BlenderKnight:Hannah_BJ_Loops`) PLUS general tags with
+    a legacy name prefix (`source:*`, from IR's source kind that fell
+    back to general). dry-run preview returns per-kind + per-prefix
+    counts + a sample so the UI shows exactly what'll go before the
+    operator confirms with dry_run=false."""
+    from ..services.cleanup_service import purge_legacy_tags
+
+    body = await request.get_json(silent=True) or {}
+    dry_run = bool(body.get("dry_run", False))
+
+    async with get_session() as session:
+        result = await session.run_sync(
+            lambda sync_sess: purge_legacy_tags(sync_sess, dry_run=dry_run)
+        )
+    return jsonify(result)

backend/app/api/cleanup.py

@@ -0,0 +1,195 @@
+"""FC-Cleanup: /api/cleanup/* — retroactive enforcement of import filters.
+
+Endpoints:
+  POST /min-dimension/preview     synchronous SQL audit
+  POST /min-dimension/delete      synchronous SQL delete (Tier-C token)
+  POST /audit                      async transparency / single_color start
+  GET  /audit                      list recent audit_run rows
+  GET  /audit/<id>                 single audit_run row
+  POST /audit/<id>/apply           apply matched_ids deletes (Tier-C token)
+  POST /audit/<id>/cancel          flip running audit to cancelled
+
+Unused-tags retroactive prune intentionally NOT in this namespace —
+TagMaintenanceCard (Maintenance tab → moved to Cleanup tab in v26.05.25.7)
+uses the existing /api/admin/tags/prune-unused endpoint via the admin
+store. No duplicate route here.
+
+Confirm-token format matches modal/DestructiveConfirmModal.vue convention:
+  `delete-min-dim-<sha8(w,h)>` for min-dim delete
+  `delete-audit-<id>` for audit apply
+(Modal hardcodes action ∈ {'restore', 'delete'}; "apply audit" is semantically a delete of the matched images, so we use `delete-audit-<id>`.)
+"""
+
+from __future__ import annotations
+
+import hashlib
+from pathlib import Path
+
+from quart import Blueprint, jsonify, request
+from sqlalchemy import select
+
+from ..extensions import get_session
+from ..models import LibraryAuditRun
+from ..services import cleanup_service
+from ._responses import error_response as _bad
+
+cleanup_bp = Blueprint("cleanup", __name__, url_prefix="/api/cleanup")
+
+IMAGES_ROOT = Path("/images")
+
+
+def _min_dim_token(min_w: int, min_h: int) -> str:
+    # SHA-256 (not MD5) — Web Crypto's subtle.digest rejects MD5; both
+    # sides use SHA-256 truncated to 8 hex chars.
+    canon = f"{min_w}x{min_h}"
+    return f"delete-min-dim-{hashlib.sha256(canon.encode()).hexdigest()[:8]}"
+
+
+def _serialize_audit_run(audit: LibraryAuditRun) -> dict:
+    return {
+        "id": audit.id,
+        "rule": audit.rule,
+        "params": audit.params,
+        "status": audit.status,
+        "started_at": audit.started_at.isoformat() if audit.started_at else None,
+        "finished_at": audit.finished_at.isoformat() if audit.finished_at else None,
+        "scanned_count": audit.scanned_count,
+        "matched_count": audit.matched_count,
+        "matched_ids": audit.matched_ids,
+        "error": audit.error,
+    }
+
+
+@cleanup_bp.route("/min-dimension/preview", methods=["POST"])
+async def min_dim_preview():
+    body = await request.get_json(silent=True) or {}
+    try:
+        min_w = int(body.get("min_width", 0))
+        min_h = int(body.get("min_height", 0))
+    except (TypeError, ValueError):
+        return _bad("invalid_dimensions")
+    if min_w < 0 or min_h < 0:
+        return _bad("invalid_dimensions")
+    async with get_session() as session:
+        projection = await session.run_sync(
+            lambda s: cleanup_service.project_min_dimension_violations(
+                s, min_width=min_w, min_height=min_h,
+            )
+        )
+    # Hand the canonical Tier-C delete token back with the preview so
+    # the frontend doesn't have to recompute SHA-256 client-side.
+    # window.crypto.subtle is Secure-Context-gated and undefined on
+    # plain-HTTP origins (homelab posture); without this the Delete
+    # button silently swallowed the TypeError and never opened the
+    # confirm modal. Operator-flagged 2026-05-27.
+    projection["confirm_token"] = _min_dim_token(min_w, min_h)
+    return jsonify(projection)
+
+
+@cleanup_bp.route("/min-dimension/delete", methods=["POST"])
+async def min_dim_delete():
+    body = await request.get_json(silent=True) or {}
+    try:
+        min_w = int(body.get("min_width", 0))
+        min_h = int(body.get("min_height", 0))
+    except (TypeError, ValueError):
+        return _bad("invalid_dimensions")
+    if min_w < 0 or min_h < 0:
+        return _bad("invalid_dimensions")
+    supplied = body.get("confirm", "")
+    expected = _min_dim_token(min_w, min_h)
+    if supplied != expected:
+        return _bad("confirm_mismatch", expected=expected)
+    async with get_session() as session:
+        deleted = await session.run_sync(
+            lambda s: cleanup_service.delete_min_dimension_violations(
+                s, min_width=min_w, min_height=min_h, images_root=IMAGES_ROOT,
+            )
+        )
+        await session.commit()
+    return jsonify({"deleted": deleted})
+
+
+@cleanup_bp.route("/audit", methods=["POST"])
+async def audit_create():
+    body = await request.get_json(silent=True) or {}
+    rule = body.get("rule")
+    params = body.get("params") or {}
+    if rule not in ("transparency", "single_color"):
+        return _bad("invalid_rule")
+    if not isinstance(params, dict):
+        return _bad("invalid_params")
+    async with get_session() as session:
+        try:
+            audit_id = await session.run_sync(
+                lambda s: cleanup_service.start_audit_run(
+                    s, rule=rule, params=params,
+                )
+            )
+        except cleanup_service.AuditAlreadyRunning as running_id:
+            return _bad(
+                "audit_already_running", status=409,
+                running_id=int(str(running_id)),
+            )
+        except ValueError as exc:
+            return _bad(str(exc))
+        await session.commit()
+    return jsonify({"audit_id": audit_id, "status": "running"}), 202
+
+
+@cleanup_bp.route("/audit/<int:audit_id>", methods=["GET"])
+async def audit_get(audit_id: int):
+    async with get_session() as session:
+        audit = (await session.execute(
+            select(LibraryAuditRun).where(LibraryAuditRun.id == audit_id)
+        )).scalar_one_or_none()
+        if audit is None:
+            return _bad("not_found", status=404)
+        return jsonify(_serialize_audit_run(audit))
+
+
+@cleanup_bp.route("/audit", methods=["GET"])
+async def audit_history():
+    try:
+        limit = min(int(request.args.get("limit", "20")), 100)
+    except ValueError:
+        return _bad("invalid_limit")
+    async with get_session() as session:
+        rows = (await session.execute(
+            select(LibraryAuditRun)
+            .order_by(LibraryAuditRun.id.desc())
+            .limit(limit)
+        )).scalars().all()
+        return jsonify({"runs": [_serialize_audit_run(r) for r in rows]})
+
+
+@cleanup_bp.route("/audit/<int:audit_id>/apply", methods=["POST"])
+async def audit_apply(audit_id: int):
+    body = await request.get_json(silent=True) or {}
+    confirm = body.get("confirm", "")
+    async with get_session() as session:
+        try:
+            deleted = await session.run_sync(
+                lambda s: cleanup_service.apply_audit_run(
+                    s, audit_id=audit_id, confirm_token=confirm,
+                    images_root=IMAGES_ROOT,
+                )
+            )
+        except cleanup_service.AuditNotReady as exc:
+            return _bad("audit_not_ready", current_status=str(exc))
+        except cleanup_service.ConfirmTokenMismatch as exc:
+            return _bad("confirm_mismatch", expected=str(exc))
+        except ValueError as exc:
+            return _bad("not_found", status=404, detail=str(exc))
+        await session.commit()
+    return jsonify({"deleted": deleted})
+
+
+@cleanup_bp.route("/audit/<int:audit_id>/cancel", methods=["POST"])
+async def audit_cancel(audit_id: int):
+    async with get_session() as session:
+        await session.run_sync(
+            lambda s: cleanup_service.cancel_audit_run(s, audit_id=audit_id)
+        )
+        await session.commit()
+    return jsonify({"cancelled": True})

backend/app/api/credentials.py

@@ -20,6 +20,7 @@ from ..services.credential_service import (
     UnknownPlatformError,
     WrongAuthTypeError,
 )
+from ._responses import error_response as _bad
 
 credentials_bp = Blueprint("credentials", __name__, url_prefix="/api/credentials")
 
@@ -38,14 +39,6 @@ def _get_crypto() -> CredentialCrypto:
     return _crypto
 
 
-def _bad(error: str, *, status: int = 400, detail: str | None = None, **extra):
-    body = {"error": error}
-    if detail is not None:
-        body["detail"] = detail
-    body.update(extra)
-    return jsonify(body), status
-
-
 async def _ext_key_ok(session) -> bool:
     """If X-Extension-Key is supplied, it must match the stored value.
     Missing header → True (browser path; accepted per homelab posture).
@@ -124,3 +117,56 @@ async def delete_credential(platform: str):
         except LookupError:
             return _bad("not_found", status=404)
     return "", 204
+
+
+@credentials_bp.route("/<platform>/verify", methods=["POST"])
+async def verify_credential(platform: str):
+    """Test the stored credential by running gallery-dl --simulate
+    against one of the platform's enabled sources. On success stamps
+    last_verified. Returns {valid: bool|null, reason, last_verified?}.
+    valid=null means "couldn't test" (no credential, or no enabled
+    source to point at)."""
+    from ..models import Artist, Source
+    from ..services.gallery_dl import GalleryDLService, SourceConfig
+
+    async with get_session() as session:
+        if not await _ext_key_ok(session):
+            return _bad("unauthorized", status=401)
+        svc = CredentialService(session, _get_crypto())
+        record = await svc.get(platform)
+        if record is None:
+            return jsonify({"valid": None, "reason": "No credential stored for this platform."})
+
+        # Pick an enabled source for this platform to point the probe at.
+        row = (await session.execute(
+            select(Source, Artist)
+            .join(Artist, Artist.id == Source.artist_id)
+            .where(Source.platform == platform, Source.enabled.is_(True))
+            .order_by(Source.id.asc())
+        )).first()
+        if row is None:
+            return jsonify({
+                "valid": None,
+                "reason": "No enabled source for this platform to verify against — add a subscription first.",
+            })
+        source, artist = row
+
+        cookies_path = await svc.get_cookies_path(platform)
+        auth_token = await svc.get_token(platform)
+
+    gdl = GalleryDLService(images_root=Path("/images"))
+    ok, message = await gdl.verify(
+        url=source.url,
+        artist_slug=artist.slug,
+        platform=platform,
+        source_config=SourceConfig.from_dict(source.config_overrides or {}),
+        cookies_path=str(cookies_path) if cookies_path else None,
+        auth_token=auth_token,
+    )
+
+    last_verified = None
+    if ok:
+        async with get_session() as session:
+            ts = await CredentialService(session, _get_crypto()).mark_verified(platform)
+            last_verified = ts.isoformat() if ts else None
+    return jsonify({"valid": ok, "reason": message, "last_verified": last_verified})

backend/app/api/downloads.py

@@ -5,8 +5,10 @@ status/source/artist. Returns slim records.
 Detail view: full DownloadEvent including the metadata JSONB.
 """
 
+from datetime import UTC, datetime, timedelta
+
 from quart import Blueprint, jsonify, request
-from sqlalchemy import select
+from sqlalchemy import func, select
 
 from ..extensions import get_session
 from ..models import Artist, DownloadEvent, Source
@@ -95,6 +97,83 @@ async def list_downloads():
     return jsonify([_list_record(e, s, a) for e, s, a in rows])
 
 
+@downloads_bp.route("/stats", methods=["GET"])
+async def downloads_stats():
+    """Status-grouped count over download_event for the dashboard stat chips.
+
+    `?window_hours=` (default 24) bounds by `started_at`. The full set of
+    statuses is always present in the response (zero for missing) so the
+    UI doesn't have to fill in defaults.
+    """
+    try:
+        window_hours = int(request.args.get("window_hours", "24"))
+    except ValueError:
+        return jsonify({"error": "invalid_window_hours"}), 400
+    if window_hours < 1 or window_hours > 24 * 365:
+        return jsonify({"error": "invalid_window_hours"}), 400
+
+    since = datetime.now(UTC) - timedelta(hours=window_hours)
+    out = {"pending": 0, "running": 0, "ok": 0, "error": 0, "skipped": 0}
+    async with get_session() as session:
+        stmt = (
+            select(DownloadEvent.status, func.count())
+            .where(DownloadEvent.started_at >= since)
+            .group_by(DownloadEvent.status)
+        )
+        for status, n in (await session.execute(stmt)).all():
+            if status in out:
+                out[status] = int(n)
+    return jsonify(out)
+
+
+@downloads_bp.route("/activity", methods=["GET"])
+async def downloads_activity():
+    """Hourly download-event counts over the last `?hours=` (default 24).
+
+    Returns a fixed-length, oldest-first bucket array so the UI can render
+    a sparkline directly. Bucketing is done in Python against UTC to dodge
+    session-timezone ambiguity in SQL date_trunc.
+    """
+    try:
+        hours = int(request.args.get("hours", "24"))
+    except ValueError:
+        return jsonify({"error": "invalid_hours"}), 400
+    hours = max(1, min(168, hours))
+
+    now = datetime.now(UTC)
+    end = now.replace(minute=0, second=0, microsecond=0)
+    start = end - timedelta(hours=hours - 1)
+    buckets = [
+        {"hour": (start + timedelta(hours=i)).isoformat(),
+         "ok": 0, "error": 0, "other": 0, "total": 0}
+        for i in range(hours)
+    ]
+
+    async with get_session() as session:
+        rows = (await session.execute(
+            select(DownloadEvent.started_at, DownloadEvent.status)
+            .where(DownloadEvent.started_at >= start)
+        )).all()
+
+    for started_at, status in rows:
+        if started_at is None:
+            continue
+        sa = started_at if started_at.tzinfo else started_at.replace(tzinfo=UTC)
+        idx = int((sa - start).total_seconds() // 3600)
+        if not (0 <= idx < hours):
+            continue
+        b = buckets[idx]
+        if status == "ok":
+            b["ok"] += 1
+        elif status == "error":
+            b["error"] += 1
+        else:
+            b["other"] += 1
+        b["total"] += 1
+
+    return jsonify({"hours": hours, "buckets": buckets})
+
+
 @downloads_bp.route("/<int:event_id>", methods=["GET"])
 async def get_download(event_id: int):
     async with get_session() as session:
@@ -108,3 +187,20 @@ async def get_download(event_id: int):
             return jsonify({"error": "not_found"}), 404
         event, source, artist = row
     return jsonify(_detail_record(event, source, artist))
+
+
+@downloads_bp.route("/recover-stalled", methods=["POST"])
+async def recover_stalled():
+    """Trigger the recover_stalled_download_events sweep on demand.
+
+    The same sweep runs every 5 min via Beat (see celery_app.beat_schedule);
+    this endpoint exists so the operator can force-clear stuck pending/
+    running download_events from the Subscriptions → Downloads maintenance
+    menu without waiting for the next scheduled tick.
+    """
+    # Local import: avoids registering maintenance tasks during blueprint
+    # import (Celery task discovery races with the API import otherwise).
+    from ..tasks.maintenance import recover_stalled_download_events
+
+    recover_stalled_download_events.delay()
+    return jsonify({"queued": True}), 202

backend/app/api/extension.py

@@ -0,0 +1,119 @@
+"""FC-3g: /api/extension — quick-add-source for the Firefox extension
++ install-time manifest for the Settings card.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import hashlib
+import re
+from pathlib import Path
+
+from quart import Blueprint, jsonify, request
+from sqlalchemy import select
+
+from ..extensions import get_session
+from ..models import AppSetting
+from ..services.extension_service import (
+    ExtensionService,
+    InvalidUrlError,
+    UnknownPlatformError,
+)
+from ..services.source_service import KNOWN_PLATFORMS
+from ._responses import error_response as _bad
+
+extension_bp = Blueprint("extension", __name__, url_prefix="/api/extension")
+
+# Default XPI directory; tests override via monkeypatching this module-
+# level constant.
+XPI_DIR = Path("/app/frontend/dist/extension")
+
+_XPI_VERSION_RE = re.compile(r"fabledcurator-(?P<version>[\w.-]+)\.xpi$")
+
+
+async def _ext_key_required(session) -> bool:
+    """Unlike /api/credentials (which accepts the browser path with no
+    header), quick-add-source writes server state and must be explicitly
+    authenticated."""
+    supplied = request.headers.get("X-Extension-Key")
+    if supplied is None:
+        return False
+    stored = (await session.execute(
+        select(AppSetting.value).where(AppSetting.key == "extension_api_key")
+    )).scalar_one_or_none()
+    return stored is not None and supplied == stored
+
+
+def _extract_version(xpi_name: str) -> str:
+    m = _XPI_VERSION_RE.search(xpi_name)
+    return m.group("version") if m else "unknown"
+
+
+def _sha256(path: Path) -> str:
+    h = hashlib.sha256()
+    with path.open("rb") as fp:
+        for chunk in iter(lambda: fp.read(65536), b""):
+            h.update(chunk)
+    return h.hexdigest()
+
+
+@extension_bp.route("/quick-add-source", methods=["POST"])
+async def quick_add_source():
+    body = await request.get_json(silent=True)
+    if not isinstance(body, dict):
+        return _bad("invalid_body", detail="body must be a JSON object")
+    url = body.get("url")
+    if not isinstance(url, str) or not url.strip():
+        return _bad("invalid_body", detail="url is required")
+
+    async with get_session() as session:
+        if not await _ext_key_required(session):
+            return _bad("unauthorized", status=401)
+        try:
+            result = await ExtensionService(session).quick_add_source(url)
+        except UnknownPlatformError as exc:
+            return _bad(
+                "unknown_platform",
+                detail=str(exc),
+                known=sorted(KNOWN_PLATFORMS),
+            )
+        except InvalidUrlError as exc:
+            return _bad("invalid_url", detail=str(exc))
+    return jsonify(result), (201 if result["created_source"] else 200)
+
+
+def _read_manifest_sync() -> dict | None:
+    """All the filesystem-touching work for /api/extension/manifest,
+    in a sync helper so the async route can dispatch it via
+    asyncio.to_thread (ASYNC240: no pathlib I/O in async functions)."""
+    if not XPI_DIR.is_dir():
+        return None
+    # Exclude the `fabledcurator-latest.xpi` alias when picking the file to
+    # extract a version from — it's a copy of the latest versioned XPI,
+    # written at the same mtime by build.yml, and would otherwise tie or
+    # win the sort (operator-flagged 2026-05-26: UI displayed "v latest"
+    # because `_extract_version("fabledcurator-latest.xpi")` returns
+    # the literal "latest"). The alias still serves as `latest_url`.
+    versioned = [
+        p for p in XPI_DIR.glob("fabledcurator-*.xpi")
+        if p.name != "fabledcurator-latest.xpi"
+    ]
+    if not versioned:
+        return None
+    versioned.sort(key=lambda p: p.stat().st_mtime)
+    latest = versioned[-1]
+    return {
+        "installed": True,
+        "version": _extract_version(latest.name),
+        "xpi_url": f"/extension/{latest.name}",
+        "latest_url": "/extension/fabledcurator-latest.xpi",
+        "sha256": _sha256(latest),
+    }
+
+
+@extension_bp.route("/manifest", methods=["GET"])
+async def extension_manifest():
+    info = await asyncio.to_thread(_read_manifest_sync)
+    if info is None:
+        return jsonify({"installed": False}), 404
+    return jsonify(info)

backend/app/api/gallery.py

@@ -42,6 +42,8 @@ async def scroll():
                     "width": i.width,
                     "height": i.height,
                     "created_at": i.created_at.isoformat(),
+                    "posted_at": i.posted_at.isoformat() if i.posted_at else None,
+                    "effective_date": i.effective_date.isoformat(),
                     "thumbnail_url": i.thumbnail_url,
                     "artist": i.artist,
                 }

backend/app/api/import_admin.py

@@ -47,10 +47,13 @@ async def status():
         if active:
             payload["active_batch"] = {
                 "id": active.id,
+                "source_path": active.source_path,
+                "scan_mode": active.scan_mode,
                 "total_files": active.total_files,
                 "imported": active.imported,
                 "skipped": active.skipped,
                 "failed": active.failed,
+                "refreshed": active.refreshed,
                 "started_at": active.started_at.isoformat(),
             }
         return jsonify(payload)
@@ -100,24 +103,139 @@ async def list_tasks():
 
 @import_admin_bp.route("/retry-failed", methods=["POST"])
 async def retry_failed():
+    # Fold SELECT into UPDATE…WHERE…RETURNING — the prior SELECT-then-
+    # UPDATE-WHERE-id-IN pattern blew past psycopg's 65535-parameter
+    # ceiling once failed_ids exceeded ~65k rows.
     async with get_session() as session:
-        failed_ids = (
-            await session.execute(select(ImportTask.id).where(ImportTask.status == "failed"))
-        ).scalars().all()
-        if not failed_ids:
-            return jsonify({"retried": 0})
-        await session.execute(
+        result = await session.execute(
             update(ImportTask)
-            .where(ImportTask.id.in_(failed_ids))
-            .values(status="queued", error=None, started_at=None, finished_at=None)
+            .where(ImportTask.status == "failed")
+            .values(
+                status="queued", error=None,
+                started_at=None, finished_at=None,
+            )
+            .returning(ImportTask.id, ImportTask.task_type)
         )
+        failed = result.all()
+        if not failed:
+            return jsonify({"retried": 0})
         await session.commit()
 
-    from ..tasks.import_file import import_media_file
-    for tid in failed_ids:
-        import_media_file.delay(tid)
+    from ..tasks.import_file import enqueue_import
+    for tid, task_type in failed:
+        enqueue_import(tid, task_type)
 
-    return jsonify({"retried": len(failed_ids)})
+    return jsonify({"retried": len(failed)})
+
+
+@import_admin_bp.route("/tasks/<int:task_id>/refetch", methods=["POST"])
+async def refetch_task(task_id: int):
+    """Layer-2 one-shot re-download: delete the (corrupt) file behind a
+    failed import task and re-run its source's downloader to fetch a
+    fresh copy. Only works for files that resolve to an enabled,
+    real-URL subscription Source; filesystem-only imports return
+    no_source.
+
+    Returns one of: refetch_queued (+source_id) / no_source /
+    already_refetched / not_found / not_failed.
+    """
+    async with get_session() as session:
+        result = await session.run_sync(_refetch_task_sync, task_id)
+    if result["status"] == "not_found":
+        return jsonify(result), 404
+    if result["status"] == "not_failed":
+        return jsonify(result), 400
+    return jsonify(result)
+
+
+def _refetch_task_sync(session, task_id: int) -> dict:
+    from pathlib import Path
+
+    from ..models import ImportSettings
+    from ..services.refetch_service import attempt_refetch
+
+    task = session.get(ImportTask, task_id)
+    if task is None:
+        return {"status": "not_found"}
+    if task.status != "failed":
+        return {"status": "not_failed"}
+    settings = ImportSettings.load_sync(session)
+    return attempt_refetch(session, task, Path(settings.import_scan_path))
+
+
+@import_admin_bp.route("/clear-stuck", methods=["POST"])
+async def clear_stuck():
+    """Force any non-terminal ImportTask (status in pending/queued/
+    processing) to 'failed' AND finalize any ImportBatch that ends up
+    with no active children. Escape hatch for the operator when the
+    automatic recover_interrupted_tasks sweep keeps re-queueing the
+    same stuck row forever (e.g., underlying file is genuinely broken
+    and the import keeps OSError-looping at PIL load).
+
+    Idempotent + non-destructive: rows survive as 'failed' so the
+    Retry-Failed button can re-attempt them once whatever was broken
+    is fixed. Banked 2026-05-25 — operator hit 3 large PNGs that
+    autoretry-looped for 2 days after a corrupt-data PIL OSError.
+    """
+    async with get_session() as session:
+        # Fold SELECT into UPDATE…WHERE — see /retry-failed for the
+        # 65535-parameter ceiling rationale. rowcount is enough here
+        # because we don't need the ids afterward (no .delay()).
+        clear_result = await session.execute(
+            update(ImportTask)
+            .where(
+                ImportTask.status.in_(["pending", "queued", "processing"])
+            )
+            .values(
+                status="failed",
+                finished_at=datetime.now(UTC),
+                error=(
+                    "manually cleared via /api/import/clear-stuck "
+                    "— stuck in non-terminal state; retry once "
+                    "underlying cause (corrupt file, missing model, "
+                    "etc.) is resolved"
+                ),
+            )
+        )
+        tasks_failed = clear_result.rowcount or 0
+
+        # Finalize any 'running' ImportBatch that no longer has any
+        # active children. The "Scanning..." banner is driven by
+        # /api/import/status finding a running batch; left untouched,
+        # it would persist forever after the stuck-task clear.
+        running_batches = (
+            await session.execute(
+                select(ImportBatch.id).where(ImportBatch.status == "running")
+            )
+        ).scalars().all()
+        finalized_batches = 0
+        for batch_id in running_batches:
+            still_active = (
+                await session.execute(
+                    select(ImportTask.id)
+                    .where(ImportTask.batch_id == batch_id)
+                    .where(ImportTask.status.in_(
+                        ["pending", "queued", "processing"]
+                    ))
+                    .limit(1)
+                )
+            ).scalar_one_or_none()
+            if still_active is None:
+                await session.execute(
+                    update(ImportBatch)
+                    .where(ImportBatch.id == batch_id)
+                    .values(
+                        status="complete",
+                        finished_at=datetime.now(UTC),
+                    )
+                )
+                finalized_batches += 1
+        await session.commit()
+
+    return jsonify({
+        "tasks_failed": tasks_failed,
+        "batches_finalized": finalized_batches,
+    })
 
 
 @import_admin_bp.route("/clear-completed", methods=["POST"])

backend/app/api/migrate.py

@@ -1,141 +0,0 @@
-"""FC-5: /api/migrate — trigger and poll migration runs.
-
-Ingest kinds (gs_ingest, ir_ingest) accept multipart/form-data with an
-`export_file` field. All other kinds accept JSON. Apply-without-backup
-guard rejects non-dry-run ingests unless a pre_migration-tagged backup
-exists in the last 24h (override with body.force=true).
-"""
-
-import json
-from datetime import UTC, datetime, timedelta
-from pathlib import Path
-
-from quart import Blueprint, jsonify, request
-from sqlalchemy import select
-
-from ..extensions import get_session
-from ..models import MigrationRun
-from ..tasks.migration import run_migration
-
-migrate_bp = Blueprint("migrate", __name__, url_prefix="/api/migrate")
-
-_VALID_KINDS = frozenset({
-    "backup", "gs_ingest", "ir_ingest", "tag_apply",
-    "ml_queue", "verify", "rollback",
-})
-_INGEST_KINDS = frozenset({"gs_ingest", "ir_ingest"})
-_APPLY_KINDS = frozenset({"gs_ingest", "ir_ingest", "tag_apply", "rollback"})
-
-
-def _bad(error: str, *, status: int = 400, **extra):
-    body = {"error": error}
-    body.update(extra)
-    return jsonify(body), status
-
-
-def _has_recent_pre_migration_backup() -> bool:
-    from ..services.migrators import backup as backup_mod
-    images_root = Path("/images")
-    manifest = backup_mod.find_latest_backup(images_root, tag="pre_migration")
-    if manifest is None:
-        return False
-    created_at_str = manifest.get("created_at")
-    if not created_at_str:
-        return False
-    created_at = datetime.fromisoformat(created_at_str)
-    return (datetime.now(UTC) - created_at) < timedelta(hours=24)
-
-
-def _run_to_dict(run: MigrationRun) -> dict:
-    return {
-        "id": run.id,
-        "kind": run.kind,
-        "status": run.status,
-        "dry_run": run.dry_run,
-        "started_at": run.started_at.isoformat(),
-        "finished_at": run.finished_at.isoformat() if run.finished_at else None,
-        "counts": run.counts or {},
-        "error": run.error,
-        "metadata": run.metadata_ or {},
-    }
-
-
-@migrate_bp.route("/<kind>", methods=["POST"])
-async def create_run(kind: str):
-    if kind not in _VALID_KINDS:
-        return _bad("unknown_kind", detail=f"kind must be one of {sorted(_VALID_KINDS)}")
-
-    # Ingest kinds accept multipart/form-data; everything else takes JSON.
-    if kind in _INGEST_KINDS:
-        form = await request.form
-        files = await request.files
-        if "export_file" not in files:
-            return _bad("missing_export_file", detail="multipart export_file required")
-        export_file = files["export_file"]
-        try:
-            raw = export_file.read()
-            data = json.loads(raw.decode("utf-8"))
-        except (UnicodeDecodeError, json.JSONDecodeError) as exc:
-            return _bad("invalid_export_file", detail=str(exc))
-        dry_run = str(form.get("dry_run", "false")).lower() in ("true", "1", "yes")
-        force = str(form.get("force", "false")).lower() in ("true", "1", "yes")
-        params: dict = {"data": data, "dry_run": dry_run}
-    else:
-        body = await request.get_json()
-        if body is None:
-            body = {}
-        if not isinstance(body, dict):
-            return _bad("invalid_body")
-        dry_run = bool(body.get("dry_run", False))
-        force = bool(body.get("force", False))
-        params = dict(body)
-
-    is_apply = (kind in _APPLY_KINDS) and not dry_run
-    if is_apply and not force and not _has_recent_pre_migration_backup():
-        return _bad(
-            "no_backup",
-            detail="apply action requires a pre_migration-tagged backup "
-                   "in the last 24h (or force=true).",
-        )
-
-    if kind == "backup":
-        params.setdefault("tag", "pre_migration")
-
-    async with get_session() as session:
-        run = MigrationRun(kind=kind, status="pending", dry_run=dry_run)
-        session.add(run)
-        await session.commit()
-        await session.refresh(run)
-        run_id = run.id
-
-    run_migration.delay(run_id, kind, params)
-    return jsonify({"run_id": run_id, "status": "pending"}), 202
-
-
-@migrate_bp.route("/runs/<int:run_id>", methods=["GET"])
-async def get_run(run_id: int):
-    async with get_session() as session:
-        run = (await session.execute(
-            select(MigrationRun).where(MigrationRun.id == run_id)
-        )).scalar_one_or_none()
-    if run is None:
-        return _bad("not_found", status=404)
-    return jsonify(_run_to_dict(run))
-
-
-@migrate_bp.route("/runs", methods=["GET"])
-async def list_runs():
-    try:
-        limit = int(request.args.get("limit", "10"))
-    except ValueError:
-        return _bad("invalid_limit")
-    if limit < 1 or limit > 100:
-        return _bad("invalid_limit")
-
-    async with get_session() as session:
-        rows = (await session.execute(
-            select(MigrationRun)
-            .order_by(MigrationRun.id.desc())
-            .limit(limit)
-        )).scalars().all()
-    return jsonify([_run_to_dict(r) for r in rows])

backend/app/api/posts.py

@@ -5,18 +5,11 @@ from quart import Blueprint, jsonify, request
 from ..extensions import get_session
 from ..services.post_feed_service import PostFeedService
 from ..services.source_service import KNOWN_PLATFORMS
+from ._responses import error_response as _bad
 
 posts_bp = Blueprint("posts", __name__, url_prefix="/api/posts")
 
 
-def _bad(error: str, *, status: int = 400, detail: str | None = None, **extra):
-    body = {"error": error}
-    if detail is not None:
-        body["detail"] = detail
-    body.update(extra)
-    return jsonify(body), status
-
-
 @posts_bp.route("", methods=["GET"])
 async def list_posts():
     args = request.args
@@ -25,6 +18,8 @@ async def list_posts():
     artist_id_raw = args.get("artist_id")
     platform = args.get("platform") or None
     limit_raw = args.get("limit", "24")
+    direction = args.get("direction", "older")
+    around_raw = args.get("around")
 
     try:
         limit = int(limit_raw)
@@ -33,6 +28,16 @@ async def list_posts():
     if limit < 1 or limit > 100:
         return _bad("invalid_limit", detail="limit must be between 1 and 100")
 
+    if direction not in ("older", "newer"):
+        return _bad("invalid_direction", detail="direction must be 'older' or 'newer'")
+
+    around_id = None
+    if around_raw is not None:
+        try:
+            around_id = int(around_raw)
+        except ValueError:
+            return _bad("invalid_around", detail="around must be an integer post id")
+
     artist_id = None
     if artist_id_raw is not None:
         try:
@@ -47,11 +52,20 @@ async def list_posts():
         )
 
     async with get_session() as session:
-        try:
-            page = await PostFeedService(session).scroll(
-                cursor=cursor, artist_id=artist_id,
+        svc = PostFeedService(session)
+        if around_id is not None:
+            result = await svc.around(
+                post_id=around_id, artist_id=artist_id,
                 platform=platform, limit=limit,
             )
+            if result is None:
+                return _bad("not_found", status=404, detail=f"post id={around_id}")
+            return jsonify(result)
+        try:
+            page = await svc.scroll(
+                cursor=cursor, artist_id=artist_id,
+                platform=platform, limit=limit, direction=direction,
+            )
         except ValueError as exc:
             # Service raises ValueError for malformed cursors only;
             # limit bounds are validated above.

backend/app/api/settings.py

@@ -31,9 +31,7 @@ _EDITABLE_FIELDS = (
 @settings_bp.route("/settings/import", methods=["GET"])
 async def get_import_settings():
     async with get_session() as session:
-        row = (
-            await session.execute(select(ImportSettings).where(ImportSettings.id == 1))
-        ).scalar_one()
+        row = await ImportSettings.load(session)
     return jsonify({
         "min_width": row.min_width,
         "min_height": row.min_height,
@@ -99,9 +97,7 @@ async def update_import_settings():
             return _bad_int("download_failure_warning_threshold", 1, 100)
 
     async with get_session() as session:
-        row = (
-            await session.execute(select(ImportSettings).where(ImportSettings.id == 1))
-        ).scalar_one()
+        row = await ImportSettings.load(session)
         for field in _EDITABLE_FIELDS:
             if field in body:
                 setattr(row, field, body[field])
@@ -141,11 +137,23 @@ async def system_stats():
         ).all()
         integrity_counts = {row[0]: row[1] for row in integrity_rows}
 
-        # Active batch (most recent running)
+        # Active batch = running batch that still has outstanding work.
+        # Plain "most recent running" picks a freshly-created scan that
+        # enqueued zero new files and hides the older batch that's
+        # actually being processed; the EXISTS clause filters those
+        # empty batches out.
         active_batch_row = (
             await session.execute(
                 select(ImportBatch)
-                .where(ImportBatch.status == "running")
+                .where(
+                    ImportBatch.status == "running",
+                    select(ImportTask.id)
+                    .where(
+                        ImportTask.batch_id == ImportBatch.id,
+                        ImportTask.status.in_(["pending", "queued", "processing"]),
+                    )
+                    .exists(),
+                )
                 .order_by(ImportBatch.started_at.desc())
                 .limit(1)
             )

backend/app/api/sources.py

@@ -5,6 +5,7 @@ from sqlalchemy import select
 
 from ..extensions import get_session
 from ..models import DownloadEvent, Source
+from ..services.scheduler_service import active_platform_cooldowns, scheduler_status
 from ..services.source_service import (
     KNOWN_PLATFORMS,
     ArtistNotFoundError,
@@ -14,18 +15,11 @@ from ..services.source_service import (
     SourceService,
     UnknownPlatformError,
 )
+from ._responses import error_response as _bad
 
 sources_bp = Blueprint("sources", __name__, url_prefix="/api/sources")
 
 
-def _bad(error: str, *, status: int = 400, detail: str | None = None, **extra):
-    body = {"error": error}
-    if detail is not None:
-        body["detail"] = detail
-    body.update(extra)
-    return jsonify(body), status
-
-
 @sources_bp.route("", methods=["GET"])
 async def list_sources():
     artist_id_raw = request.args.get("artist_id")
@@ -35,11 +29,19 @@ async def list_sources():
             artist_id = int(artist_id_raw)
         except ValueError:
             return _bad("invalid_artist_id", detail="artist_id must be an integer")
+    failing = request.args.get("failing", "").lower() in ("1", "true", "yes")
     async with get_session() as session:
-        records = await SourceService(session).list(artist_id=artist_id)
+        records = await SourceService(session).list(artist_id=artist_id, failing=failing)
     return jsonify([r.to_dict() for r in records])
 
 
+@sources_bp.route("/schedule-status", methods=["GET"])
+async def schedule_status():
+    """FC-dashboards: scheduler health for the Subscriptions hub."""
+    async with get_session() as session:
+        return jsonify(await scheduler_status(session))
+
+
 @sources_bp.route("/<int:source_id>", methods=["GET"])
 async def get_source(source_id: int):
     async with get_session() as session:
@@ -123,7 +125,16 @@ async def check_source(source_id: int):
     """FC-3c: enqueue a download for this source.
 
     Returns 202 with the new DownloadEvent id. If a pending/running
-    event already exists for this source, returns 409 with that id."""
+    event already exists for this source, returns 409 with that id. If
+    the source's platform is currently in a rate-limit cooldown, returns
+    **202 with `{status: "deferred", cooldown_until, platform}`** and
+    does NOT create an event or dispatch — the bulk retry path uses this
+    to avoid bowling N sources right back into the rate limit the
+    cooldown is preventing. Single-click "retry this one source" passes
+    `?force=true` to override the cooldown (operator-explicit, useful
+    for rapid auth-fix testing). The in-flight guard always applies.
+    """
+    force = (request.args.get("force") or "").lower() in ("1", "true", "yes")
     async with get_session() as session:
         source = (await session.execute(
             select(Source).where(Source.id == source_id)
@@ -133,6 +144,19 @@ async def check_source(source_id: int):
         if not source.enabled:
             return _bad("source_disabled", detail="enable the source first")
 
+        # Cooldown gate (unless explicitly overridden). Checked before the
+        # in-flight guard because a deferred retry doesn't need to create
+        # or check for an event at all.
+        if not force:
+            cooldowns = await active_platform_cooldowns(session)
+            expires_at = cooldowns.get(source.platform)
+            if expires_at is not None:
+                return jsonify({
+                    "status": "deferred",
+                    "platform": source.platform,
+                    "cooldown_until": expires_at.isoformat(),
+                }), 202
+
         in_flight = (await session.execute(
             select(DownloadEvent.id).where(
                 DownloadEvent.source_id == source_id,

backend/app/api/system_activity.py

@@ -0,0 +1,244 @@
+"""FC-3i: system activity dashboard endpoints.
+
+Read-only. Combines Redis-broker queue depths (LLEN per queue),
+Celery worker introspection (celery inspect), and the task_run DB
+history into the surfaces the SystemActivityTab UI consumes.
+
+All filesystem/sync-client work goes through asyncio.to_thread per
+ASYNC230/240 (mirrors backend.app.api.extension's pattern).
+"""
+
+from __future__ import annotations
+
+import asyncio
+import time
+from datetime import UTC, datetime, timedelta
+
+from quart import Blueprint, jsonify, request
+from sqlalchemy import desc, func, select
+
+from ..config import get_config
+from ..extensions import get_session
+from ..models import TaskRun
+from ..services.scheduler_service import scheduler_status
+
+system_activity_bp = Blueprint(
+    "system_activity", __name__, url_prefix="/api/system/activity",
+)
+
+# Canonical queue order — must match celery_app.task_routes. UI renders
+# in this order; queues with no LLEN response show as null rather than
+# absent.
+_QUEUE_NAMES = (
+    "default", "import", "thumbnail", "ml",
+    "download", "scan", "maintenance",
+)
+
+# Cache module-level so all requests share the cache between polls.
+# Tests can reset via direct dict mutation if needed.
+_QUEUE_CACHE: dict = {"ts": 0.0, "data": None}
+_WORKER_CACHE: dict = {"ts": 0.0, "data": None}
+_QUEUE_CACHE_TTL = 2.0
+_WORKER_CACHE_TTL = 5.0
+
+
+def _read_queues_sync() -> dict:
+    """Reads each queue's LLEN from the broker. Sync — caller wraps in
+    asyncio.to_thread. Per-queue try/except returns None on failure so
+    one bad queue doesn't break the whole response."""
+    import redis  # local import; only this endpoint needs it
+
+    cfg = get_config()
+    client = redis.Redis.from_url(cfg.celery_broker_url)
+    out: dict = {}
+    for name in _QUEUE_NAMES:
+        try:
+            out[name] = int(client.llen(name))
+        except Exception:  # noqa: BLE001 — broker hiccup shouldn't break UI
+            out[name] = None
+    return {
+        "queues": out,
+        "fetched_at": datetime.now(UTC).isoformat(),
+    }
+
+
+def _read_workers_sync() -> dict:
+    """celery inspect active_queues + active. Returns per-worker info."""
+    from ..celery_app import celery as celery_app
+
+    insp = celery_app.control.inspect(timeout=2.0)
+    active_queues = insp.active_queues() or {}
+    active_tasks = insp.active() or {}
+
+    workers: dict = {}
+    for hostname, queues in active_queues.items():
+        workers[hostname] = {
+            "queues": sorted({q["name"] for q in queues}),
+            "active_count": len(active_tasks.get(hostname, [])),
+        }
+    return {
+        "workers": workers,
+        "fetched_at": datetime.now(UTC).isoformat(),
+    }
+
+
+async def _queues_cached() -> dict:
+    """Per-queue Redis LLEN, cached 2s. Shared by /queues and /summary."""
+    now = time.time()
+    if _QUEUE_CACHE["data"] is None or (now - _QUEUE_CACHE["ts"]) > _QUEUE_CACHE_TTL:
+        _QUEUE_CACHE["data"] = await asyncio.to_thread(_read_queues_sync)
+        _QUEUE_CACHE["ts"] = now
+    return _QUEUE_CACHE["data"]
+
+
+@system_activity_bp.route("/queues", methods=["GET"])
+async def get_queues():
+    """Per-queue Redis LLEN. Cached 2s.
+
+    Response: {queues: {name: depth_or_null}, fetched_at: iso8601}
+    """
+    return jsonify(await _queues_cached())
+
+
+@system_activity_bp.route("/workers", methods=["GET"])
+async def get_workers():
+    """Live celery inspect. Cached 5s.
+
+    Response: {workers: {hostname: {queues, active_count}}, fetched_at}
+    """
+    now = time.time()
+    if _WORKER_CACHE["data"] is None or (now - _WORKER_CACHE["ts"]) > _WORKER_CACHE_TTL:
+        _WORKER_CACHE["data"] = await asyncio.to_thread(_read_workers_sync)
+        _WORKER_CACHE["ts"] = now
+    return jsonify(_WORKER_CACHE["data"])
+
+
+@system_activity_bp.route("/summary", methods=["GET"])
+async def get_summary():
+    """One-call rollup for the always-on TopNav pipeline indicator:
+    scheduler health, per-queue pending depths, currently-running count, and
+    recent (24h) failure count. Cheap — cached queue LLENs + two TaskRun
+    counts — so it's safe to poll app-wide."""
+    queues_data = await _queues_cached()
+    depths = queues_data.get("queues", {})
+    queued_total = sum(v for v in depths.values() if isinstance(v, int))
+    since = datetime.now(UTC) - timedelta(hours=24)
+    async with get_session() as session:
+        scheduler = await scheduler_status(session)
+        running = (await session.execute(
+            select(func.count(TaskRun.id)).where(TaskRun.status == "running")
+        )).scalar_one()
+        failing = (await session.execute(
+            select(func.count(TaskRun.id))
+            .where(TaskRun.status.in_(["error", "timeout"]))
+            .where(TaskRun.finished_at >= since)
+        )).scalar_one()
+    return jsonify({
+        "scheduler": scheduler,
+        "queues": depths,
+        "queued_total": queued_total,
+        "running": int(running),
+        "failing": int(failing),
+    })
+
+
+@system_activity_bp.route("/runs", methods=["GET"])
+async def list_runs():
+    """Paginated task_run history. Query params:
+      queue=<name>     filter to one queue
+      status=<status>  filter to one status (running/ok/error/timeout/retry)
+      limit=<int>      default 50, max 200
+      before_id=<int>  cursor for keyset pagination
+
+    Response: {runs: [...], next_cursor: id|null}
+    """
+    try:
+        limit = min(int(request.args.get("limit", "50")), 200)
+    except ValueError:
+        return jsonify({"error": "invalid_limit"}), 400
+    if limit < 1:
+        return jsonify({"error": "invalid_limit"}), 400
+
+    queue = request.args.get("queue")
+    status = request.args.get("status")
+    before_id_raw = request.args.get("before_id")
+    before_id = int(before_id_raw) if before_id_raw else None
+
+    async with get_session() as session:
+        stmt = select(TaskRun).order_by(desc(TaskRun.id))
+        if queue:
+            stmt = stmt.where(TaskRun.queue == queue)
+        if status:
+            stmt = stmt.where(TaskRun.status == status)
+        if before_id is not None:
+            stmt = stmt.where(TaskRun.id < before_id)
+        stmt = stmt.limit(limit + 1)
+        rows = (await session.execute(stmt)).scalars().all()
+
+    has_more = len(rows) > limit
+    rows = rows[:limit]
+    return jsonify({
+        "runs": [_row_to_dict(r) for r in rows],
+        "next_cursor": rows[-1].id if has_more and rows else None,
+    })
+
+
+@system_activity_bp.route("/failures", methods=["GET"])
+async def list_failures():
+    """Recent failures across all lanes (24h window).
+
+    Response: {recent: [...], count_by_type: {ErrorClass: n}, since}
+    """
+    try:
+        limit = min(int(request.args.get("limit", "50")), 200)
+    except ValueError:
+        return jsonify({"error": "invalid_limit"}), 400
+
+    since = datetime.now(UTC) - timedelta(hours=24)
+
+    async with get_session() as session:
+        recent_stmt = (
+            select(TaskRun)
+            .where(TaskRun.status.in_(["error", "timeout"]))
+            .where(TaskRun.finished_at >= since)
+            .order_by(desc(TaskRun.finished_at))
+            .limit(limit)
+        )
+        recent = (await session.execute(recent_stmt)).scalars().all()
+
+        count_stmt = (
+            select(TaskRun.error_type, func.count(TaskRun.id))
+            .where(TaskRun.status.in_(["error", "timeout"]))
+            .where(TaskRun.finished_at >= since)
+            .group_by(TaskRun.error_type)
+            .order_by(desc(func.count(TaskRun.id)))
+        )
+        counts = (await session.execute(count_stmt)).all()
+
+    return jsonify({
+        "recent": [_row_to_dict(r) for r in recent],
+        "count_by_type": {
+            (row[0] or "Unknown"): row[1]
+            for row in counts
+        },
+        "since": since.isoformat(),
+    })
+
+
+def _row_to_dict(r: TaskRun) -> dict:
+    return {
+        "id": r.id,
+        "queue": r.queue,
+        "task_name": r.task_name,
+        "target_id": r.target_id,
+        "celery_task_id": r.celery_task_id,
+        "started_at": r.started_at.isoformat() if r.started_at else None,
+        "finished_at": r.finished_at.isoformat() if r.finished_at else None,
+        "duration_ms": r.duration_ms,
+        "status": r.status,
+        "error_type": r.error_type,
+        "error_message": r.error_message,
+        "retry_count": r.retry_count,
+        "worker_hostname": r.worker_hostname,
+        "args_summary": r.args_summary,
+    }

backend/app/api/system_backup.py

@@ -0,0 +1,255 @@
+"""FC-3h: /api/system/backup — create/list/restore/delete/tag for
+DB + image backups.
+
+Read endpoints are public on FC (operator-facing internal API; same
+posture as /api/system/activity). Write endpoints take a typed
+`confirm` body field that must match a server-generated token for
+that backup row, to prevent click-to-destroy by stale browser tabs
+or accidental cURL.
+"""
+from __future__ import annotations
+
+from quart import Blueprint, jsonify, request
+from sqlalchemy import desc, select
+
+from ..extensions import get_session
+from ..models import BackupRun, ImportSettings
+from ._responses import error_response as _bad
+
+system_backup_bp = Blueprint(
+    "system_backup", __name__, url_prefix="/api/system/backup",
+)
+
+_KINDS = frozenset({"db", "images"})
+_TAG_MAX_LEN = 64
+_BACKUP_SETTINGS_FIELDS = (
+    "backup_db_nightly_enabled",
+    "backup_db_nightly_hour_utc",
+    "backup_db_keep_last_n",
+    "backup_images_keep_last_n",
+)
+
+
+def _row_to_dict(r: BackupRun) -> dict:
+    return {
+        "id": r.id,
+        "kind": r.kind,
+        "status": r.status,
+        "tag": r.tag,
+        "triggered_by": r.triggered_by,
+        "started_at": r.started_at.isoformat() if r.started_at else None,
+        "finished_at": r.finished_at.isoformat() if r.finished_at else None,
+        "duration_seconds": (
+            int((r.finished_at - r.started_at).total_seconds())
+            if r.finished_at and r.started_at else None
+        ),
+        "sql_path": r.sql_path,
+        "tar_path": r.tar_path,
+        "size_bytes": r.size_bytes,
+        "error": r.error,
+        "restored_from_id": r.restored_from_id,
+        "manifest": r.manifest or {},
+    }
+
+
+def _validate_tag(tag):
+    if tag is None:
+        return None
+    if not isinstance(tag, str):
+        return _bad("invalid_tag", detail="tag must be string or null")
+    tag = tag.strip()
+    if not tag:
+        return None
+    if len(tag) > _TAG_MAX_LEN:
+        return _bad("invalid_tag", detail=f"tag too long (max {_TAG_MAX_LEN})")
+    return tag
+
+
+def _validate_backup_settings_patch(body: dict):
+    if "backup_db_nightly_enabled" in body and not isinstance(
+        body["backup_db_nightly_enabled"], bool,
+    ):
+        return _bad("invalid_value", detail="backup_db_nightly_enabled must be bool")
+    if "backup_db_nightly_hour_utc" in body:
+        v = body["backup_db_nightly_hour_utc"]
+        if not isinstance(v, int) or isinstance(v, bool) or not (0 <= v <= 23):
+            return _bad("invalid_value", detail="backup_db_nightly_hour_utc must be 0..23")
+    if "backup_db_keep_last_n" in body:
+        v = body["backup_db_keep_last_n"]
+        if not isinstance(v, int) or isinstance(v, bool) or not (1 <= v <= 365):
+            return _bad("invalid_value", detail="backup_db_keep_last_n must be 1..365")
+    if "backup_images_keep_last_n" in body:
+        v = body["backup_images_keep_last_n"]
+        if not isinstance(v, int) or isinstance(v, bool) or not (1 <= v <= 100):
+            return _bad("invalid_value", detail="backup_images_keep_last_n must be 1..100")
+    return None
+
+
+@system_backup_bp.route("/db", methods=["POST"])
+async def trigger_db_backup():
+    body = await request.get_json(silent=True) or {}
+    tag = _validate_tag(body.get("tag"))
+    if isinstance(tag, tuple):
+        return tag
+    from ..tasks.backup import backup_db_task
+    backup_db_task.delay(tag=tag, triggered_by="manual")
+    return jsonify({"status": "dispatched"}), 202
+
+
+@system_backup_bp.route("/images", methods=["POST"])
+async def trigger_images_backup():
+    body = await request.get_json(silent=True) or {}
+    tag = _validate_tag(body.get("tag"))
+    if isinstance(tag, tuple):
+        return tag
+    from ..tasks.backup import backup_images_task
+    backup_images_task.delay(tag=tag, triggered_by="manual")
+    return jsonify({"status": "dispatched"}), 202
+
+
+@system_backup_bp.route("/runs", methods=["GET"])
+async def list_runs():
+    try:
+        limit = min(int(request.args.get("limit", "50")), 200)
+    except ValueError:
+        return _bad("invalid_limit")
+    if limit < 1:
+        return _bad("invalid_limit")
+    kind = request.args.get("kind")
+    if kind is not None and kind not in _KINDS:
+        return _bad("invalid_kind", detail=f"kind must be one of {sorted(_KINDS)}")
+    before_id_raw = request.args.get("before_id")
+    before_id = int(before_id_raw) if before_id_raw else None
+
+    async with get_session() as session:
+        stmt = select(BackupRun).order_by(desc(BackupRun.id))
+        if kind:
+            stmt = stmt.where(BackupRun.kind == kind)
+        if before_id is not None:
+            stmt = stmt.where(BackupRun.id < before_id)
+        stmt = stmt.limit(limit + 1)
+        rows = (await session.execute(stmt)).scalars().all()
+
+    has_more = len(rows) > limit
+    rows = rows[:limit]
+    return jsonify({
+        "runs": [_row_to_dict(r) for r in rows],
+        "next_cursor": rows[-1].id if has_more and rows else None,
+    })
+
+
+@system_backup_bp.route("/runs/<int:run_id>", methods=["GET"])
+async def get_run(run_id: int):
+    async with get_session() as session:
+        row = await session.get(BackupRun, run_id)
+    if row is None:
+        return _bad("not_found", status=404)
+    return jsonify(_row_to_dict(row))
+
+
+@system_backup_bp.route("/runs/<int:run_id>", methods=["PATCH"])
+async def patch_run(run_id: int):
+    body = await request.get_json(silent=True) or {}
+    if "tag" not in body:
+        return _bad("invalid_body", detail="tag required")
+    tag = _validate_tag(body["tag"])
+    if isinstance(tag, tuple):
+        return tag
+
+    async with get_session() as session:
+        row = await session.get(BackupRun, run_id)
+        if row is None:
+            return _bad("not_found", status=404)
+        row.tag = tag
+        await session.commit()
+        await session.refresh(row)
+    return jsonify(_row_to_dict(row))
+
+
+@system_backup_bp.route("/runs/<int:run_id>/restore", methods=["POST"])
+async def trigger_restore(run_id: int):
+    body = await request.get_json(silent=True) or {}
+    supplied = body.get("confirm", "")
+
+    async with get_session() as session:
+        row = await session.get(BackupRun, run_id)
+        if row is None:
+            return _bad("not_found", status=404)
+        if row.status != "ok":
+            return _bad(
+                "not_restorable",
+                detail=f"source backup status={row.status!r}; only 'ok' rows are restorable",
+            )
+        expected = f"restore-{row.kind}-{row.id}"
+        if supplied != expected:
+            return _bad(
+                "confirm_mismatch",
+                detail=f"confirm must equal {expected!r}",
+                expected=expected,
+            )
+        kind = row.kind
+
+    if kind == "db":
+        from ..tasks.backup import restore_db_task
+        restore_db_task.delay(source_backup_run_id=run_id)
+    else:  # 'images' (the only other value _KINDS allows via the trigger path)
+        from ..tasks.backup import restore_images_task
+        restore_images_task.delay(source_backup_run_id=run_id)
+    return jsonify({"status": "dispatched", "kind": kind}), 202
+
+
+@system_backup_bp.route("/runs/<int:run_id>", methods=["DELETE"])
+async def delete_run(run_id: int):
+    body = await request.get_json(silent=True) or {}
+    supplied = body.get("confirm", "")
+
+    async with get_session() as session:
+        row = await session.get(BackupRun, run_id)
+        if row is None:
+            return _bad("not_found", status=404)
+        expected = f"delete-{row.kind}-{row.id}"
+        if supplied != expected:
+            return _bad(
+                "confirm_mismatch",
+                detail=f"confirm must equal {expected!r}",
+                expected=expected,
+            )
+        from ..services import backup_service
+        backup_service.unlink_artifact_files(
+            sql_path=row.sql_path, tar_path=row.tar_path,
+            manifest_path=(row.manifest or {}).get("manifest_path"),
+        )
+        await session.delete(row)
+        await session.commit()
+    return "", 204
+
+
+@system_backup_bp.route("/settings", methods=["GET"])
+async def get_settings():
+    async with get_session() as session:
+        row = await ImportSettings.load(session)
+    return jsonify({
+        "backup_db_nightly_enabled": row.backup_db_nightly_enabled,
+        "backup_db_nightly_hour_utc": row.backup_db_nightly_hour_utc,
+        "backup_db_keep_last_n": row.backup_db_keep_last_n,
+        "backup_images_keep_last_n": row.backup_images_keep_last_n,
+    })
+
+
+@system_backup_bp.route("/settings", methods=["PATCH"])
+async def patch_settings():
+    body = await request.get_json(silent=True)
+    if not isinstance(body, dict):
+        return _bad("invalid_body", detail="body must be a JSON object")
+
+    err = _validate_backup_settings_patch(body)
+    if err is not None:
+        return err
+
+    async with get_session() as session:
+        row = await ImportSettings.load(session)
+        for field in _BACKUP_SETTINGS_FIELDS:
+            if field in body:
+                setattr(row, field, body[field])
+        await session.commit()
+    return await get_settings()

backend/app/api/tags.py

@@ -15,6 +15,7 @@ from ..services.tag_service import (
     TagService,
     TagValidationError,
 )
+from ..utils.tag_prefix import parse_kind_prefix
 
 tags_bp = Blueprint("tags", __name__, url_prefix="/api")
 
@@ -105,13 +106,39 @@ async def directory():
 
 @tags_bp.route("/tags", methods=["POST"])
 async def create_tag():
+    """Create a tag. Two input shapes accepted:
+      1. Explicit: {name, kind, fandom_id?} — caller already split, kind wins.
+      2. IR-suffix: {name} where name = "kind:Name" (e.g. "artist:Eric").
+         The server runs parse_kind_prefix(name) to derive kind; the colon
+         and prefix are stripped from the stored tag name. If no recognized
+         prefix is present, the kind defaults to `general`.
+    Explicit kind ALWAYS wins (backward-compat for existing callers).
+    """
     body = await request.get_json()
-    if not body or "name" not in body or "kind" not in body:
-        return jsonify({"error": "name and kind required"}), 400
+    if not body or "name" not in body:
+        return jsonify({"error": "name required"}), 400
     name = body["name"]
-    kind = _coerce_kind(body["kind"])
-    if kind is None:
-        return jsonify({"error": f"invalid kind {body['kind']!r}"}), 400
+    explicit_kind_raw = body.get("kind")
+
+    if explicit_kind_raw is not None:
+        # Caller provided kind — honor it; don't re-parse.
+        kind = _coerce_kind(explicit_kind_raw)
+        if kind is None:
+            return jsonify({"error": f"invalid kind {explicit_kind_raw!r}"}), 400
+    else:
+        # IR-style: parse "kind:Name" from the raw name.
+        parsed_kind, parsed_name = parse_kind_prefix(name)
+        if parsed_kind is not None:
+            name = parsed_name
+            kind = _coerce_kind(parsed_kind)
+            # parse_kind_prefix only returns kinds from KNOWN_KINDS which
+            # are all valid TagKind members, so _coerce_kind can't return
+            # None here — but defensive.
+            if kind is None:
+                return jsonify({"error": f"invalid kind {parsed_kind!r}"}), 400
+        else:
+            kind = TagKind.general
+
     fandom_id = body.get("fandom_id")
 
     async with get_session() as session:

backend/app/api/thumbnails.py

@@ -0,0 +1,13 @@
+"""Thumbnail admin API: backfill trigger."""
+
+from quart import Blueprint, jsonify
+
+thumbnails_bp = Blueprint("thumbnails", __name__, url_prefix="/api/thumbnails")
+
+
+@thumbnails_bp.route("/backfill", methods=["POST"])
+async def trigger_backfill():
+    from ..tasks.thumbnail import backfill_thumbnails
+
+    r = backfill_thumbnails.delay()
+    return jsonify({"celery_task_id": r.id}), 202

backend/app/celery_app.py

@@ -28,9 +28,11 @@ def make_celery() -> Celery:
             "backend.app.tasks.import_file",
             "backend.app.tasks.thumbnail",
             "backend.app.tasks.maintenance",
-            "backend.app.tasks.migration",
             "backend.app.tasks.ml",
             "backend.app.tasks.download",
+            "backend.app.tasks.backup",
+            "backend.app.tasks.admin",
+            "backend.app.tasks.library_audit",
         ],
     )
     app.conf.update(
@@ -42,7 +44,9 @@ def make_celery() -> Celery:
             "backend.app.tasks.download.*": {"queue": "download"},
             "backend.app.tasks.scan.*": {"queue": "scan"},
             "backend.app.tasks.maintenance.*": {"queue": "maintenance"},
-            "backend.app.tasks.migration.*": {"queue": "maintenance"},
+            "backend.app.tasks.backup.*": {"queue": "maintenance"},
+            "backend.app.tasks.admin.*": {"queue": "maintenance"},
+            "backend.app.tasks.library_audit.*": {"queue": "maintenance"},
         },
         # Heavy ML tasks need fair dispatch — see ImageRepo's precedent.
         task_acks_late=True,
@@ -81,9 +85,32 @@ def make_celery() -> Celery:
                 "task": "backend.app.tasks.maintenance.cleanup_old_download_events",
                 "schedule": 86400.0,  # daily
             },
+            "recover-stalled-download-events": {
+                "task": "backend.app.tasks.maintenance.recover_stalled_download_events",
+                "schedule": 300.0,  # every 5 min, matches recover-interrupted-tasks
+            },
+            "recover-stalled-task-runs": {
+                "task": "backend.app.tasks.maintenance.recover_stalled_task_runs",
+                "schedule": 300.0,  # every 5 min, matches recover-interrupted-tasks
+            },
+            "prune-task-runs": {
+                "task": "backend.app.tasks.maintenance.prune_task_runs",
+                "schedule": 86400.0,  # daily
+            },
+            "fc3h-backup-db-nightly": {
+                "task": "backend.app.tasks.backup.backup_db_nightly",
+                "schedule": 3600.0,  # hourly tick; task self-gates on configured UTC hour
+            },
+            "fc3h-prune-backups": {
+                "task": "backend.app.tasks.backup.prune_backups",
+                "schedule": 86400.0,  # daily
+            },
         },
         timezone="UTC",
     )
+    # FC-3i: register task_run signal handlers (side-effect import).
+    from . import celery_signals  # noqa: F401
+
     return app
 
 

backend/app/celery_signals.py

@@ -0,0 +1,207 @@
+"""FC-3i: task_run lifecycle via Celery signals.
+
+Subscribes to task_prerun / task_postrun / task_failure / task_retry
+and persists one task_run row per task attempt. Drop-in for every
+existing and future Celery task — no per-task instrumentation.
+
+Signal handlers run inside the worker process (sync context); DB
+writes go through the existing shared sync engine
+(backend.app.tasks._sync_engine.sync_session_factory) — one engine
+per worker process, not per-task, so we don't blow Postgres
+max_connections under load (the reason FC-3g shared-engine fix
+existed).
+
+Failure-mode discipline (operator-pressed point): every handler is
+wrapped in try/except that swallows + logs. If the DB is down or the
+handler has a bug, the real task still runs — the dashboard goes
+dark for that interval. Monitoring NEVER breaks the thing it's
+monitoring.
+"""
+
+import logging
+from datetime import UTC, datetime
+
+from celery.exceptions import SoftTimeLimitExceeded
+from celery.signals import task_failure, task_postrun, task_prerun, task_retry
+
+from .models import TaskRun
+from .tasks._sync_engine import sync_session_factory
+
+log = logging.getLogger(__name__)
+
+# Celery-internal tasks that would generate dashboard noise without
+# operational value. Conservative list; extend only when a specific
+# task proves noisy.
+_UNTRACKED_TASK_NAMES = frozenset({
+    "celery.chord_unlock",
+    "celery.backend_cleanup",
+    "celery.chunks",
+})
+
+_MAX_ERROR_MESSAGE_LEN = 2000
+_MAX_ARGS_SUMMARY_LEN = 255
+_MAX_WORKER_HOSTNAME_LEN = 128
+
+# PostgreSQL Integer is signed 32-bit. Tasks called with a first-arg
+# int outside this range (e.g. an absurdly large mock value, or a
+# string-of-digits coercible to int but bigger than 2^31-1) would crash
+# the INSERT with NumericValueOutOfRange. Bound the recorded value to
+# the column's range; values outside become None (target_id is
+# nullable, so this is safe).
+_INT32_MAX = 2_147_483_647
+_INT32_MIN = -2_147_483_648
+
+
+def _queue_for(task) -> str:
+    """Reverse the task→queue routing from celery_app.task_routes.
+    Keep in sync if task_routes is reordered."""
+    name = getattr(task, "name", "") or ""
+    if name.startswith("backend.app.tasks.import_file."):
+        return "import"
+    if name.startswith("backend.app.tasks.ml."):
+        return "ml"
+    if name.startswith("backend.app.tasks.thumbnail."):
+        return "thumbnail"
+    if name.startswith("backend.app.tasks.download."):
+        return "download"
+    if name.startswith("backend.app.tasks.scan."):
+        return "scan"
+    if name.startswith("backend.app.tasks.maintenance."):
+        return "maintenance"
+    return "default"
+
+
+def _target_id_from_args(args) -> int | None:
+    """Best-effort: if the first positional arg parses as int AND fits
+    in the column's signed-32-bit range, record it as target_id
+    (image_id, source_id, etc.). Never raises."""
+    if not args:
+        return None
+    try:
+        value = int(args[0])
+    except (TypeError, ValueError):
+        return None
+    if value < _INT32_MIN or value > _INT32_MAX:
+        return None
+    return value
+
+
+def _truncate(s, limit: int) -> str | None:
+    if s is None:
+        return None
+    text = str(s)
+    return text if len(text) <= limit else text[:limit]
+
+
+def _is_tracked(task_name: str | None) -> bool:
+    return bool(task_name) and task_name not in _UNTRACKED_TASK_NAMES
+
+
+@task_prerun.connect
+def _on_prerun(sender=None, task_id=None, task=None, args=None,
+               kwargs=None, **_):
+    if not _is_tracked(getattr(task, "name", None)):
+        return
+    try:
+        Session = sync_session_factory()
+        with Session() as session:
+            session.add(TaskRun(
+                celery_task_id=task_id or "",
+                queue=_queue_for(task),
+                task_name=task.name,
+                target_id=_target_id_from_args(args),
+                started_at=datetime.now(UTC),
+                status="running",
+                args_summary=_truncate(repr(args), _MAX_ARGS_SUMMARY_LEN),
+                worker_hostname=_truncate(
+                    getattr(sender, "hostname", None),
+                    _MAX_WORKER_HOSTNAME_LEN,
+                ),
+            ))
+            session.commit()
+    except Exception:  # noqa: BLE001 — never break the worker
+        log.exception("task_run prerun insert failed (task=%s)",
+                      getattr(task, "name", "?"))
+
+
+def _finalize(task_id: str, *, status: str,
+              error_type: str | None = None,
+              error_message: str | None = None,
+              retry_count: int | None = None) -> None:
+    """Shared write path for postrun/failure/retry. Picks the most-
+    recent task_run row for this celery_task_id that's still 'running'
+    (retries reuse the same celery_task_id; each new attempt's prerun
+    inserts a fresh row, so finalize targets the latest running row)."""
+    try:
+        from sqlalchemy import select
+
+        Session = sync_session_factory()
+        now = datetime.now(UTC)
+        with Session() as session:
+            row = session.execute(
+                select(TaskRun)
+                .where(TaskRun.celery_task_id == task_id)
+                .where(TaskRun.status == "running")
+                .order_by(TaskRun.id.desc())
+                .limit(1)
+            ).scalar_one_or_none()
+            if row is None:
+                return  # no prerun row (untracked, insert failed, or already finalized)
+            row.finished_at = now
+            row.duration_ms = int(
+                (now - row.started_at).total_seconds() * 1000
+            )
+            row.status = status
+            if error_type is not None:
+                row.error_type = _truncate(error_type, 128)
+            if error_message is not None:
+                row.error_message = _truncate(error_message, _MAX_ERROR_MESSAGE_LEN)
+            if retry_count is not None:
+                row.retry_count = retry_count
+            session.commit()
+    except Exception:  # noqa: BLE001
+        log.exception("task_run finalize failed (task_id=%s)", task_id)
+
+
+@task_postrun.connect
+def _on_postrun(sender=None, task_id=None, task=None, args=None,
+                kwargs=None, retval=None, state=None, **_):
+    if not _is_tracked(getattr(task, "name", None)):
+        return
+    # state is one of SUCCESS/FAILURE/RETRY/etc. Only handle SUCCESS;
+    # task_failure handles FAILURE explicitly (with the exception).
+    if state != "SUCCESS":
+        return
+    _finalize(task_id, status="ok")
+
+
+@task_failure.connect
+def _on_failure(sender=None, task_id=None, exception=None,
+                args=None, kwargs=None, einfo=None, **_):
+    if not _is_tracked(getattr(sender, "name", None)):
+        return
+    status = ("timeout"
+              if isinstance(exception, SoftTimeLimitExceeded)
+              else "error")
+    _finalize(
+        task_id, status=status,
+        error_type=type(exception).__name__ if exception else "Unknown",
+        error_message=str(exception) if exception else None,
+    )
+
+
+@task_retry.connect
+def _on_retry(sender=None, request=None, reason=None, einfo=None, **_):
+    if not _is_tracked(getattr(sender, "name", None)):
+        return
+    task_id = getattr(request, "id", None)
+    if not task_id:
+        return
+    # Mark current attempt's row as 'retry' (terminal for this row).
+    # The next attempt's task_prerun inserts a fresh row.
+    _finalize(
+        task_id, status="retry",
+        error_type=type(reason).__name__ if reason else "Retry",
+        error_message=str(reason) if reason else None,
+        retry_count=getattr(request, "retries", 0),
+    )

backend/app/frontend.py

@@ -1,14 +1,78 @@
-"""Serves the built Vue SPA from frontend/dist/ with history-mode fallback."""
+"""Serves the built Vue SPA from frontend/dist/ with history-mode fallback,
+the on-disk image library + thumbnails from /images, and the signed
+Firefox extension XPI from frontend/dist/extension/.
+"""
 
+import re
 from pathlib import Path
 
-from quart import Blueprint, send_from_directory
+from quart import Blueprint, abort, send_file, send_from_directory
 
 FRONTEND_DIST = Path(__file__).resolve().parent.parent.parent / "frontend" / "dist"
+IMAGES_ROOT = Path("/images")
+XPI_DIR = FRONTEND_DIST / "extension"
+
+_XPI_NAME_RE = re.compile(r"^fabledcurator-[\w.-]+\.xpi$")
 
 frontend_bp = Blueprint("frontend", __name__)
 
 
+@frontend_bp.route("/images/<path:subpath>")
+async def serve_image(subpath: str):
+    """Serve a file from the /images volume (originals + thumbnails).
+
+    Without this route the SPA catch-all below would swallow image
+    requests and return index.html, leaving the browser to render the
+    aspect-ratio-shaped grey placeholder.
+    """
+    target = (IMAGES_ROOT / subpath).resolve()
+    # Defend against path-traversal: refuse anything that escapes /images.
+    try:
+        target.relative_to(IMAGES_ROOT)
+    except ValueError:
+        abort(404)
+    if not target.is_file():
+        abort(404)
+    return await send_from_directory(IMAGES_ROOT, subpath)
+
+
+@frontend_bp.route("/extension/<filename>")
+async def serve_extension(filename: str):
+    """Serve the signed FC Firefox extension XPI.
+
+    Path whitelist: filename must match fabledcurator-*.xpi. The special
+    name fabledcurator-latest.xpi serves the most-recently-modified XPI
+    in the directory.
+
+    The application/x-xpinstall MIME tells Firefox to show its native
+    install prompt instead of downloading the file as a blob.
+    """
+    if not _XPI_NAME_RE.fullmatch(filename):
+        abort(404)
+    if not XPI_DIR.is_dir():
+        abort(404)
+    if filename == "fabledcurator-latest.xpi":
+        xpis = sorted(XPI_DIR.glob("fabledcurator-*.xpi"), key=lambda p: p.stat().st_mtime)
+        if not xpis:
+            abort(404)
+        latest = xpis[-1]
+        return await send_file(
+            latest, mimetype="application/x-xpinstall",
+            attachment_filename=latest.name,
+        )
+    target = (XPI_DIR / filename).resolve()
+    try:
+        target.relative_to(XPI_DIR)
+    except ValueError:
+        abort(404)
+    if not target.is_file():
+        abort(404)
+    return await send_file(
+        target, mimetype="application/x-xpinstall",
+        attachment_filename=filename,
+    )
+
+
 @frontend_bp.route("/")
 @frontend_bp.route("/<path:subpath>")
 async def serve_spa(subpath: str = ""):

backend/app/models/__init__.py

@@ -2,6 +2,7 @@
 
 from .app_setting import AppSetting
 from .artist import Artist
+from .backup_run import BackupRun
 from .base import Base
 from .credential import Credential
 from .download_event import DownloadEvent
@@ -10,7 +11,7 @@ from .image_record import ImageRecord
 from .import_batch import ImportBatch
 from .import_settings import ImportSettings
 from .import_task import ImportTask
-from .migration_run import MigrationRun
+from .library_audit_run import LibraryAuditRun
 from .ml_settings import MLSettings
 from .post import Post
 from .post_attachment import PostAttachment
@@ -21,11 +22,13 @@ from .tag_alias import TagAlias
 from .tag_allowlist import TagAllowlist
 from .tag_reference_embedding import TagReferenceEmbedding
 from .tag_suggestion_rejection import TagSuggestionRejection
+from .task_run import TaskRun
 
 __all__ = [
     "Base",
     "AppSetting",
     "Artist",
+    "BackupRun",
     "Source",
     "Credential",
     "Post",
@@ -40,10 +43,11 @@ __all__ = [
     "ImportBatch",
     "ImportTask",
     "ImportSettings",
+    "LibraryAuditRun",
     "MLSettings",
-    "MigrationRun",
     "TagAlias",
     "TagAllowlist",
     "TagReferenceEmbedding",
     "TagSuggestionRejection",
+    "TaskRun",
 ]

backend/app/models/backup_run.py

@@ -0,0 +1,55 @@
+"""FC-3h: backup_run — operator-facing artifact record for a backup run.
+
+One row per backup attempt (kind='db' or 'images'). Lifecycle
+tracking (started_at/finished_at/duration_ms/exception text) lives
+in task_run from FC-3i — this row records artifact metadata: file
+paths, sizes, tag (retention protection), and restore lineage via
+restored_from_id.
+
+Status values (String, not Postgres ENUM — per
+feedback_check_existing_enums):
+  pending   — created but task hasn't started yet (rare; usually
+              status starts as 'running' from the task body).
+  running   — backup task is in flight.
+  ok        — artifact successfully written.
+  error     — task raised; error column populated.
+  restoring — this row represents a restore attempt (kind = restored
+              kind); linked to source via restored_from_id.
+  restored  — restore completed successfully.
+"""
+
+from datetime import datetime
+
+from sqlalchemy import JSON, BigInteger, DateTime, ForeignKey, Integer, String, Text
+from sqlalchemy.orm import Mapped, mapped_column
+
+from .base import Base
+
+
+class BackupRun(Base):
+    __tablename__ = "backup_run"
+
+    id: Mapped[int] = mapped_column(Integer, primary_key=True)
+    kind: Mapped[str] = mapped_column(String(16), nullable=False, index=True)
+    status: Mapped[str] = mapped_column(
+        String(16), nullable=False, default="pending", index=True,
+    )
+    tag: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
+    triggered_by: Mapped[str] = mapped_column(String(32), nullable=False)
+    started_at: Mapped[datetime] = mapped_column(
+        DateTime(timezone=True), nullable=False, index=True,
+    )
+    finished_at: Mapped[datetime | None] = mapped_column(
+        DateTime(timezone=True), nullable=True, index=True,
+    )
+    sql_path: Mapped[str | None] = mapped_column(Text, nullable=True)
+    tar_path: Mapped[str | None] = mapped_column(Text, nullable=True)
+    size_bytes: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
+    error: Mapped[str | None] = mapped_column(Text, nullable=True)
+    manifest: Mapped[dict] = mapped_column(
+        JSON, nullable=False, default=dict, server_default="{}",
+    )
+    restored_from_id: Mapped[int | None] = mapped_column(
+        ForeignKey("backup_run.id", ondelete="SET NULL"),
+        nullable=True,
+    )

backend/app/models/image_provenance.py

@@ -1,14 +1,18 @@
 """ImageProvenance — links an ImageRecord to a Post.
 
-Many-to-one (one image, many provenance rows) enables the enrich-on-duplicate
-rule (spec §3): when a downloaded image is a pHash dupe of an existing
-record, we append a new provenance row to the existing record rather than
-dropping the metadata.
+One image can have many provenance rows — different posts each contribute
+metadata (enrich-on-duplicate rule, spec §3: a downloaded image that is a
+pHash dupe of an existing record gets a NEW provenance row for the new post
+appended, rather than the metadata being dropped). But the (image, post)
+pair is unique — alembic 0021 enforces uq_image_provenance_image_post
+after operator-flagged 2026-05-26 saw _apply_sidecar's existence-check +
+INSERT race plant duplicates that then broke .scalar_one_or_none() on
+every later deep-scan rederive (MultipleResultsFound).
 """
 
 from datetime import datetime
 
-from sqlalchemy import JSON, DateTime, ForeignKey, Integer, func
+from sqlalchemy import JSON, DateTime, ForeignKey, Integer, UniqueConstraint, func
 from sqlalchemy.orm import Mapped, mapped_column
 
 from .base import Base
@@ -16,6 +20,12 @@ from .base import Base
 
 class ImageProvenance(Base):
     __tablename__ = "image_provenance"
+    __table_args__ = (
+        UniqueConstraint(
+            "image_record_id", "post_id",
+            name="uq_image_provenance_image_post",
+        ),
+    )
 
     id: Mapped[int] = mapped_column(Integer, primary_key=True)
     image_record_id: Mapped[int] = mapped_column(

backend/app/models/import_batch.py

@@ -26,6 +26,10 @@ class ImportBatch(Base):
     skipped: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
     failed: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
     attachments: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
+    # Deep-scan only: count of already-imported files whose sidecar metadata
+    # got re-applied this run (post/source/provenance upsert). Stays 0 on
+    # quick-scan batches. See `Importer.import_one(deep_scan=True)`.
+    refreshed: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
 
     status: Mapped[str] = mapped_column(String(16), nullable=False, default="running", index=True)
     # running | complete | cancelled

backend/app/models/import_settings.py

@@ -4,7 +4,7 @@ Enforced as a single row via a CHECK (id = 1) constraint. The application
 always SELECTs id=1 and never inserts/deletes after the initial migration.
 """
 
-from sqlalchemy import Boolean, CheckConstraint, Float, Integer, Text
+from sqlalchemy import Boolean, CheckConstraint, Float, Integer, Text, select
 from sqlalchemy.orm import Mapped, mapped_column
 
 from .base import Base
@@ -49,3 +49,27 @@ class ImportSettings(Base):
     download_failure_warning_threshold: Mapped[int] = mapped_column(
         Integer, nullable=False, default=5
     )
+
+    # FC-3h backup knobs.
+    backup_db_nightly_enabled: Mapped[bool] = mapped_column(
+        Boolean, nullable=False, default=False,
+    )
+    backup_db_nightly_hour_utc: Mapped[int] = mapped_column(
+        Integer, nullable=False, default=3,
+    )
+    backup_db_keep_last_n: Mapped[int] = mapped_column(
+        Integer, nullable=False, default=14,
+    )
+    backup_images_keep_last_n: Mapped[int] = mapped_column(
+        Integer, nullable=False, default=3,
+    )
+
+    @classmethod
+    async def load(cls, session) -> ImportSettings:
+        """The singleton settings row (id=1), via an async session."""
+        return (await session.execute(select(cls).where(cls.id == 1))).scalar_one()
+
+    @classmethod
+    def load_sync(cls, session) -> ImportSettings:
+        """The singleton settings row (id=1), via a sync session."""
+        return session.execute(select(cls).where(cls.id == 1)).scalar_one()

backend/app/models/import_task.py

@@ -8,7 +8,16 @@ been processing longer than the stuck-task threshold.
 
 from datetime import datetime
 
-from sqlalchemy import BigInteger, DateTime, ForeignKey, Integer, String, Text, func
+from sqlalchemy import (
+    BigInteger,
+    Boolean,
+    DateTime,
+    ForeignKey,
+    Integer,
+    String,
+    Text,
+    func,
+)
 from sqlalchemy.orm import Mapped, mapped_column, relationship
 
 from .base import Base
@@ -26,6 +35,13 @@ class ImportTask(Base):
     task_type: Mapped[str] = mapped_column(String(16), nullable=False)  # media|archive
     status: Mapped[str] = mapped_column(String(16), nullable=False, default="pending", index=True)
 
+    # Poison-pill circuit breaker (alembic 0026). recovery_count tracks
+    # how many times the stuck-task sweep has re-queued this row; after
+    # the cap it's failed with a diagnostic instead of looping. refetched
+    # bounds the one-shot re-download remediation to a single attempt.
+    recovery_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
+    refetched: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
+
     result_image_id: Mapped[int | None] = mapped_column(
         ForeignKey("image_record.id", ondelete="SET NULL"), nullable=True
     )

backend/app/models/library_audit_run.py

@@ -0,0 +1,37 @@
+"""LibraryAuditRun — async transparency / single_color audit lifecycle.
+
+State machine: running → ready → applied / cancelled / error.
+matched_ids JSONB is appended-to by scan_library_for_rule; apply_audit_run
+reads it and routes through cleanup_service.delete_images.
+"""
+
+from datetime import datetime
+from typing import Any
+
+from sqlalchemy import DateTime, Integer, String, Text, func
+from sqlalchemy.dialects.postgresql import JSONB
+from sqlalchemy.orm import Mapped, mapped_column
+
+from .base import Base
+
+
+class LibraryAuditRun(Base):
+    __tablename__ = "library_audit_run"
+
+    id: Mapped[int] = mapped_column(Integer, primary_key=True)
+    rule: Mapped[str] = mapped_column(String(32), nullable=False, index=True)
+    params: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
+    status: Mapped[str] = mapped_column(
+        String(16), nullable=False, default="running", index=True,
+    )
+    # running | ready | applied | cancelled | error
+    started_at: Mapped[datetime] = mapped_column(
+        DateTime(timezone=True), nullable=False, server_default=func.now(),
+    )
+    finished_at: Mapped[datetime | None] = mapped_column(
+        DateTime(timezone=True), nullable=True,
+    )
+    scanned_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
+    matched_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
+    matched_ids: Mapped[list[int]] = mapped_column(JSONB, nullable=False, default=list)
+    error: Mapped[str | None] = mapped_column(Text, nullable=True)

backend/app/models/migration_run.py

@@ -1,37 +0,0 @@
-"""MigrationRun — tracks each FC-5 migration invocation (backup/gs/ir/etc).
-
-kind/status are String(32) not Postgres ENUM so adding kinds later
-doesn't need a schema migration. The API layer validates values.
-"""
-
-from datetime import datetime
-
-import sqlalchemy as sa
-from sqlalchemy import Boolean, DateTime, Integer, String, Text, func
-from sqlalchemy.dialects.postgresql import JSONB
-from sqlalchemy.orm import Mapped, mapped_column
-
-from .base import Base
-
-
-class MigrationRun(Base):
-    __tablename__ = "migration_run"
-
-    id: Mapped[int] = mapped_column(Integer, primary_key=True)
-    kind: Mapped[str] = mapped_column(String(32), nullable=False, index=True)
-    status: Mapped[str] = mapped_column(String(32), nullable=False, index=True)
-    dry_run: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
-    started_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now(),
-    )
-    finished_at: Mapped[datetime | None] = mapped_column(
-        DateTime(timezone=True), nullable=True,
-    )
-    counts: Mapped[dict] = mapped_column(
-        JSONB, nullable=False, default=dict, server_default=sa.text("'{}'::jsonb"),
-    )
-    error: Mapped[str | None] = mapped_column(Text, nullable=True)
-    metadata_: Mapped[dict] = mapped_column(
-        "metadata", JSONB, nullable=False, default=dict,
-        server_default=sa.text("'{}'::jsonb"),
-    )

backend/app/models/tag.py

@@ -35,8 +35,10 @@ class TagKind(StrEnum):
     series = "series"
     archive = "archive"
     post = "post"
-    meta = "meta"
-    rating = "rating"
+    # `meta` and `rating` retired by operator 2026-05-26 (alembic 0023).
+    # `artist` retired in FC-2d-vii-c — artists are first-class entities
+    # via Artist/Source rows now, not tags — but the enum value stays
+    # to keep historic tag rows queryable.
 
 
 image_tag = Table(

backend/app/models/task_run.py

@@ -0,0 +1,48 @@
+"""FC-3i: task_run — per-Celery-task lifecycle audit row.
+
+One row inserted by the task_prerun signal at task start, updated by
+task_postrun / task_failure / task_retry. The shape supports the
+SystemActivity dashboard's three panes: per-queue queue+worker summary,
+recent failures (24h, grouped by error_type), and full paginated
+activity history.
+
+Retention: ok rows pruned after 24h, error/timeout after 7d (see
+backend.app.tasks.maintenance.prune_task_runs).
+
+Recovery: rows stuck in 'running' for >5 min flipped to 'error' by
+backend.app.tasks.maintenance.recover_stalled_task_runs (Beat 5 min).
+"""
+
+from datetime import datetime
+
+from sqlalchemy import DateTime, Integer, String, Text
+from sqlalchemy.orm import Mapped, mapped_column
+
+from .base import Base
+
+
+class TaskRun(Base):
+    __tablename__ = "task_run"
+
+    id: Mapped[int] = mapped_column(Integer, primary_key=True)
+    celery_task_id: Mapped[str] = mapped_column(
+        String(64), nullable=False, index=True,
+    )
+    queue: Mapped[str] = mapped_column(String(32), nullable=False, index=True)
+    task_name: Mapped[str] = mapped_column(String(128), nullable=False, index=True)
+    target_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
+    started_at: Mapped[datetime] = mapped_column(
+        DateTime(timezone=True), nullable=False, index=True,
+    )
+    finished_at: Mapped[datetime | None] = mapped_column(
+        DateTime(timezone=True), nullable=True, index=True,
+    )
+    duration_ms: Mapped[int | None] = mapped_column(Integer, nullable=True)
+    status: Mapped[str] = mapped_column(
+        String(16), nullable=False, default="running", index=True,
+    )
+    error_type: Mapped[str | None] = mapped_column(String(128), nullable=True)
+    error_message: Mapped[str | None] = mapped_column(Text, nullable=True)
+    retry_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
+    worker_hostname: Mapped[str | None] = mapped_column(String(128), nullable=True)
+    args_summary: Mapped[str | None] = mapped_column(String(255), nullable=True)

backend/app/scripts/download_models.py

@@ -25,12 +25,31 @@ def _snapshot(repo_id: str, dest: Path, allow_patterns: list[str] | None) -> Non
 
 
 def ensure_camie() -> None:
+    """Fetch Camie v2 weights + metadata.
+
+    v2 layout (HuggingFace Camais03/camie-tagger-v2): the ONNX file is
+    named camie-tagger-v2.onnx (not model.onnx) and tags ship inside
+    camie-tagger-v2-metadata.json (not selected_tags.csv). Both at root.
+    The repo also contains app/, game/, training/, images/ subdirs full
+    of setup/demo files we don't need — allow_patterns scopes the fetch
+    to just the inference essentials (~790 MB instead of ~2 GB).
+    """
     dest = MODEL_ROOT / "camie"
-    if (dest / "model.onnx").is_file() and (dest / "selected_tags.csv").is_file():
+    model_file = dest / "camie-tagger-v2.onnx"
+    meta_file = dest / "camie-tagger-v2-metadata.json"
+    if model_file.is_file() and meta_file.is_file():
         print(f"[download_models] Camie present at {dest}")
         return
     print(f"[download_models] Fetching {CAMIE_REPO} -> {dest}")
-    _snapshot(CAMIE_REPO, dest, ["model.onnx", "selected_tags.csv", "*.json"])
+    _snapshot(
+        CAMIE_REPO, dest,
+        [
+            "camie-tagger-v2.onnx",
+            "camie-tagger-v2-metadata.json",
+            "config.json",
+            "config.yaml",
+        ],
+    )
 
 
 def ensure_siglip() -> None:

backend/app/services/artist_directory_service.py

@@ -127,17 +127,20 @@ class ArtistDirectoryService:
                 ImageRecord.artist_id.label("artist_id"),
                 ImageRecord.sha256.label("sha256"),
                 ImageRecord.mime.label("mime"),
+                ImageRecord.thumbnail_path.label("thumbnail_path"),
                 rn,
             )
             .where(ImageRecord.artist_id.in_(artist_ids))
             .subquery()
         )
         stmt = (
-            select(sub.c.artist_id, sub.c.sha256, sub.c.mime)
+            select(
+                sub.c.artist_id, sub.c.sha256, sub.c.mime, sub.c.thumbnail_path,
+            )
             .where(sub.c.rn <= _PREVIEW_COUNT)
             .order_by(sub.c.artist_id, sub.c.rn)
         )
         out: dict[int, list[str]] = {}
-        for aid, sha, mime in (await self.session.execute(stmt)).all():
-            out.setdefault(aid, []).append(thumbnail_url(sha, mime))
+        for aid, sha, mime, tp in (await self.session.execute(stmt)).all():
+            out.setdefault(aid, []).append(thumbnail_url(tp, sha, mime))
         return out

backend/app/services/artist_service.py

@@ -111,12 +111,22 @@ class ArtistService:
             )
         ).all()
 
+        post_count = (
+            await self.session.execute(
+                select(func.count(func.distinct(Post.id)))
+                .select_from(Post)
+                .join(Source, Source.id == Post.source_id)
+                .where(Source.artist_id == aid)
+            )
+        ).scalar_one()
+
         return {
             "id": artist.id,
             "name": artist.name,
             "slug": artist.slug,
             "is_subscription": bool(artist.is_subscription),
             "image_count": int(image_count),
+            "post_count": int(post_count),
             "date_range": {
                 "min": dmin.isoformat() if dmin else None,
                 "max": dmax.isoformat() if dmax else None,
@@ -188,7 +198,7 @@ class ArtistService:
                     "mime": r.mime,
                     "width": r.width,
                     "height": r.height,
-                    "thumbnail_url": thumbnail_url(r.sha256, r.mime),
+                    "thumbnail_url": thumbnail_url(r.thumbnail_path, r.sha256, r.mime),
                 }
                 for r in rows
             ],

backend/app/services/audits/__init__.py

@@ -0,0 +1,6 @@
+"""Audit rule modules. Each module exposes evaluate(pil_image, **params) -> bool.
+
+The retroactive library-cleanup tab and (future) import-time filter logic
+both consume these. Importers should NOT inline rule logic going forward;
+add the rule here and call from both sides.
+"""

backend/app/services/audits/single_color.py

@@ -0,0 +1,53 @@
+"""Single-color audit: matches images where one color dominates beyond
+the threshold (within the given Euclidean RGB tolerance). The first
+canonical implementation — the import-side filter (SkipReason.single_color)
+was never wired; FC-Cleanup's audit module is the source of truth and a
+future spec can adopt it on the import path too.
+"""
+
+from PIL import Image
+
+_THUMB_SIZE = (64, 64)
+
+
+def evaluate(
+    pil_image,
+    *,
+    threshold: float,
+    tolerance: int,
+) -> bool:
+    """True iff the fraction of pixels within `tolerance` (Euclidean RGB
+    distance) of the dominant color exceeds `threshold`.
+
+    Downsamples to 64x64 for speed (~4ms regardless of source size).
+    Alpha channels are stripped; only RGB is considered. Animated images
+    use frame 0 (PIL's default after Image.open without seek).
+    """
+    im = pil_image
+    if im.mode == "RGBA":
+        im = im.convert("RGB")
+    elif im.mode not in ("RGB", "L"):
+        im = im.convert("RGB")
+    if im.size != _THUMB_SIZE:
+        im = im.resize(_THUMB_SIZE, Image.Resampling.BILINEAR)
+    pixels = list(im.getdata())
+    if not pixels:
+        return False
+    # Normalize L-mode pixels to RGB tuples for distance math.
+    if isinstance(pixels[0], int):
+        pixels = [(p, p, p) for p in pixels]
+    # Dominant color = mean RGB.
+    n = len(pixels)
+    sum_r = sum(p[0] for p in pixels)
+    sum_g = sum(p[1] for p in pixels)
+    sum_b = sum(p[2] for p in pixels)
+    dom = (sum_r / n, sum_g / n, sum_b / n)
+    tol_sq = tolerance * tolerance
+    within = 0
+    for r, g, b in pixels:
+        dr = r - dom[0]
+        dg = g - dom[1]
+        db = b - dom[2]
+        if dr * dr + dg * dg + db * db <= tol_sq:
+            within += 1
+    return (within / n) > threshold

backend/app/services/audits/transparency.py

@@ -0,0 +1,27 @@
+"""Transparency audit: matches images whose transparent-pixel fraction
+exceeds the threshold. Animated images short-circuit (skipped) to avoid
+the multi-frame PIL decode that hits Celery's hard time limit."""
+
+
+def evaluate(pil_image, *, threshold: float) -> bool:
+    """True iff the image's transparent-pixel fraction exceeds threshold.
+
+    False for non-alpha modes and animated images. Mirrors the import-side
+    Importer._transparency_pct logic so retroactive enforcement matches
+    prospective filtering.
+    """
+    if getattr(pil_image, "is_animated", False):
+        return False
+    if pil_image.mode not in ("RGBA", "LA") and not (
+        pil_image.mode == "P" and "transparency" in pil_image.info
+    ):
+        return False
+    im = pil_image
+    if im.mode != "RGBA":
+        im = im.convert("RGBA")
+    alpha = im.getchannel("A")
+    histogram = alpha.histogram()
+    transparent = histogram[0]
+    total = sum(histogram)
+    pct = transparent / total if total else 0.0
+    return pct > threshold

backend/app/services/backup_service.py

@@ -0,0 +1,196 @@
+"""FC-3h: first-class backup/restore service for FC.
+
+Two independent backup kinds:
+  - 'db'     — pg_dump only; fast; nightly via Beat (settings-gated)
+  - 'images' — tar+zstd of /images; slow; manual trigger only
+
+Files live under <images_root>/_backups/. Each backup writes:
+  fc_<kind>_<ts>.{sql|tar.zst}        — the artifact
+  fc_<kind>_<ts>.json                 — manifest (kind/tag/triggered_by)
+
+Service functions are sync (subprocess-bound). Celery tasks in
+backend.app.tasks.backup wrap each one with task_run-tracked
+lifecycle + soft/hard time limits + retention bookkeeping.
+"""
+from __future__ import annotations
+
+import json
+import subprocess
+from datetime import UTC, datetime
+from pathlib import Path
+
+_BACKUPS_DIRNAME = "_backups"
+
+# Subprocess-level guardrails BEYOND the Celery soft_time_limit. The
+# Celery soft limit signals the Python process; subprocess.Popen in a
+# blocking syscall ignores that signal. These bound the worst case.
+_DB_SUBPROCESS_TIMEOUT_S = 12 * 60        # 12 min (Celery soft is 10 min)
+_IMAGES_SUBPROCESS_TIMEOUT_S = 7 * 60 * 60  # 7 hr (Celery soft is 6 hr)
+
+
+def _libpq_url(sa_url: str) -> str:
+    """Strip SQLAlchemy +psycopg/+asyncpg driver suffix for pg_dump/psql."""
+    for driver in (
+        "postgresql+psycopg",
+        "postgresql+asyncpg",
+        "postgresql+psycopg2",
+    ):
+        if sa_url.startswith(driver + "://"):
+            return "postgresql://" + sa_url[len(driver) + 3:]
+    return sa_url
+
+
+def _backups_dir(images_root: Path) -> Path:
+    p = images_root / _BACKUPS_DIRNAME
+    p.mkdir(parents=True, exist_ok=True)
+    return p
+
+
+def _now_ts() -> str:
+    return datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ")
+
+
+def _file_size_or_none(path: Path) -> int | None:
+    try:
+        return path.stat().st_size
+    except OSError:
+        return None
+
+
+def _write_manifest(
+    out_dir: Path, *, kind: str, ts: str,
+    tag: str | None, triggered_by: str,
+    artifact_path: Path,
+) -> Path:
+    manifest = {
+        "kind": kind,
+        "backup_id": f"fc_{kind}_{ts}",
+        "tag": tag,
+        "triggered_by": triggered_by,
+        "created_at": datetime.now(UTC).isoformat(),
+        "artifact_path": str(artifact_path),
+    }
+    mf = out_dir / f"fc_{kind}_{ts}.json"
+    mf.write_text(json.dumps(manifest, indent=2))
+    return mf
+
+
+def backup_db(
+    *, db_url: str, images_root: Path,
+    tag: str | None = None, triggered_by: str = "manual",
+) -> dict:
+    """Run pg_dump; write .sql + manifest; return dict for the caller
+    to persist into BackupRun. Raises on subprocess failure."""
+    ts = _now_ts()
+    out_dir = _backups_dir(images_root)
+    sql_path = out_dir / f"fc_db_{ts}.sql"
+    subprocess.run(
+        [
+            "pg_dump", "--no-owner", "--no-acl",
+            "-f", str(sql_path), _libpq_url(db_url),
+        ],
+        capture_output=True, check=True,
+        timeout=_DB_SUBPROCESS_TIMEOUT_S,
+    )
+    manifest_path = _write_manifest(
+        out_dir, kind="db", ts=ts, tag=tag, triggered_by=triggered_by,
+        artifact_path=sql_path,
+    )
+    return {
+        "kind": "db",
+        "ts": ts,
+        "sql_path": str(sql_path),
+        "tar_path": None,
+        "manifest_path": str(manifest_path),
+        "size_bytes": _file_size_or_none(sql_path),
+    }
+
+
+def backup_images(
+    *, images_root: Path,
+    tag: str | None = None, triggered_by: str = "manual",
+) -> dict:
+    """Run tar --zstd over images_root; write .tar.zst + manifest."""
+    ts = _now_ts()
+    out_dir = _backups_dir(images_root)
+    tar_path = out_dir / f"fc_images_{ts}.tar.zst"
+    subprocess.run(
+        [
+            "tar", "--zstd", "-cf", str(tar_path),
+            "-C", str(images_root.parent), images_root.name,
+            f"--exclude={images_root.name}/_backups",
+            f"--exclude={images_root.name}/_quarantine",
+        ],
+        capture_output=True, check=True,
+        timeout=_IMAGES_SUBPROCESS_TIMEOUT_S,
+    )
+    manifest_path = _write_manifest(
+        out_dir, kind="images", ts=ts, tag=tag, triggered_by=triggered_by,
+        artifact_path=tar_path,
+    )
+    return {
+        "kind": "images",
+        "ts": ts,
+        "sql_path": None,
+        "tar_path": str(tar_path),
+        "manifest_path": str(manifest_path),
+        "size_bytes": _file_size_or_none(tar_path),
+    }
+
+
+def restore_db(*, db_url: str, sql_path: Path) -> None:
+    """Wipe public schema, then load from .sql. Raises on subprocess
+    failure; partial-restore state is the caller's concern."""
+    libpq = _libpq_url(db_url)
+    subprocess.run(
+        [
+            "psql", libpq, "-c",
+            "DROP SCHEMA IF EXISTS public CASCADE; CREATE SCHEMA public;",
+        ],
+        capture_output=True, check=True, timeout=120,
+    )
+    subprocess.run(
+        ["psql", libpq, "-f", str(sql_path)],
+        capture_output=True, check=True,
+        timeout=_DB_SUBPROCESS_TIMEOUT_S,
+    )
+
+
+def restore_images(*, images_root: Path, tar_path: Path) -> None:
+    """Untar over images_root.parent. Additive — files NOT in the
+    tarball are NOT removed. Caller wipes first if a clean restore
+    is needed."""
+    subprocess.run(
+        [
+            "tar", "--zstd", "-xf", str(tar_path),
+            "-C", str(images_root.parent),
+        ],
+        capture_output=True, check=True,
+        timeout=_IMAGES_SUBPROCESS_TIMEOUT_S,
+    )
+
+
+def unlink_artifact_files(
+    *,
+    sql_path: str | None,
+    tar_path: str | None,
+    manifest_path: str | None,
+) -> dict:
+    """Best-effort unlink of all on-disk files for a BackupRun row.
+    Returns dict keyed by label with True/False per file. Missing
+    files count as success (missing_ok semantics)."""
+    deleted: dict = {}
+    for label, p in (
+        ("sql", sql_path),
+        ("tar", tar_path),
+        ("manifest", manifest_path),
+    ):
+        if not p:
+            continue
+        path = Path(p)
+        try:
+            path.unlink(missing_ok=True)
+            deleted[label] = True
+        except OSError:
+            deleted[label] = False
+    return deleted

backend/app/services/cleanup_service.py

@@ -0,0 +1,577 @@
+"""FC-3k: first-class admin destructive operations.
+
+Projections are pure SELECTs used by both dry-run preview endpoints
+and Tier-B count prompts. Mutations (Task 2) are called from sync
+HTTP handlers (small ops) and from Celery tasks in
+backend.app.tasks.admin (long ops).
+
+This module is the PERMANENT home of artist-cascade + image-unlink
+logic. (The legacy migrators/cleanup.py copy was removed with the rest of
+the one-and-done GS/IR migration tooling.)
+"""
+from __future__ import annotations
+
+from datetime import UTC, datetime
+from pathlib import Path
+from typing import Any
+
+from sqlalchemy import func, or_, select, update
+from sqlalchemy.orm import Session
+
+from ..models import Artist, ImageRecord, LibraryAuditRun, Tag
+from ..models.series_page import SeriesPage
+from ..models.tag import image_tag
+
+
+def project_artist_cascade(session: Session, *, slug: str) -> dict:
+    """Read-only projection of what delete_artist_cascade would touch.
+
+    Returns:
+        {
+          "artist": {"id": int, "name": str, "slug": str},
+          "projected": {
+            "images": int,
+            "sources": int,
+            "thumbs": int,        # images with a thumbnail_path set
+            "import_tasks": int,  # ImportTask rows referencing the artist's images
+            "bytes_on_disk": int, # SUM(image_record.size_bytes) — column is NOT NULL
+          },
+        }
+    Raises LookupError if slug not found. No mutations.
+    """
+    from ..models.import_task import ImportTask
+    from ..models.source import Source
+
+    artist = session.execute(
+        select(Artist).where(Artist.slug == slug)
+    ).scalar_one_or_none()
+    if artist is None:
+        raise LookupError(f"artist slug not found: {slug!r}")
+
+    images_count = session.execute(
+        select(func.count(ImageRecord.id))
+        .where(ImageRecord.artist_id == artist.id)
+    ).scalar_one()
+    sources_count = session.execute(
+        select(func.count(Source.id))
+        .where(Source.artist_id == artist.id)
+    ).scalar_one()
+    thumbs_count = session.execute(
+        select(func.count(ImageRecord.id))
+        .where(ImageRecord.artist_id == artist.id)
+        .where(ImageRecord.thumbnail_path.is_not(None))
+    ).scalar_one()
+    import_tasks_count = session.execute(
+        select(func.count(ImportTask.id))
+        .where(
+            ImportTask.result_image_id.in_(
+                select(ImageRecord.id).where(ImageRecord.artist_id == artist.id)
+            )
+        )
+    ).scalar_one()
+    bytes_on_disk = session.execute(
+        select(func.coalesce(func.sum(ImageRecord.size_bytes), 0))
+        .where(ImageRecord.artist_id == artist.id)
+    ).scalar_one()
+
+    return {
+        "artist": {"id": artist.id, "name": artist.name, "slug": artist.slug},
+        "projected": {
+            "images": images_count,
+            "sources": sources_count,
+            "thumbs": thumbs_count,
+            "import_tasks": import_tasks_count,
+            "bytes_on_disk": int(bytes_on_disk),
+        },
+    }
+
+
+def project_bulk_image_delete(
+    session: Session, *, image_ids: list[int],
+) -> dict:
+    """Read-only projection of what delete_images would touch.
+
+    Returns:
+        {
+          "images_found": int,
+          "thumbs_to_unlink": int,
+          "bytes_on_disk": int,
+          "missing_ids": list[int],   # ids passed in that don't exist
+        }
+    No mutations.
+    """
+    if not image_ids:
+        return {
+            "images_found": 0,
+            "thumbs_to_unlink": 0,
+            "bytes_on_disk": 0,
+            "missing_ids": [],
+        }
+
+    rows = session.execute(
+        select(
+            ImageRecord.id,
+            ImageRecord.thumbnail_path,
+            ImageRecord.size_bytes,
+        ).where(ImageRecord.id.in_(image_ids))
+    ).all()
+    found_ids = {r.id for r in rows}
+    missing = sorted(set(image_ids) - found_ids)
+    return {
+        "images_found": len(rows),
+        "thumbs_to_unlink": sum(1 for r in rows if r.thumbnail_path),
+        "bytes_on_disk": sum(r.size_bytes for r in rows),
+        "missing_ids": missing,
+    }
+
+
+def count_tag_associations(session: Session, *, tag_id: int) -> int:
+    """COUNT(*) FROM image_tag WHERE tag_id=?. For Tier-B prompt."""
+    return session.execute(
+        select(func.count())
+        .select_from(image_tag)
+        .where(image_tag.c.tag_id == tag_id)
+    ).scalar_one()
+
+
+def find_unused_tags(
+    session: Session, *, limit: int | None = None,
+) -> list[Tag]:
+    """Tags with no image_tag rows AND no series_page rows.
+
+    Sorted by name. Used by both dry-run preview and the live prune.
+    A tag is "unused" iff it has zero rows in image_tag AND zero rows
+    in series_page (so we don't accidentally prune a series tag that
+    happens to have no images yet).
+    """
+    used_via_image_tag = select(image_tag.c.tag_id).distinct()
+    used_via_series = select(SeriesPage.series_tag_id).where(
+        SeriesPage.series_tag_id.is_not(None)
+    ).distinct()
+    stmt = (
+        select(Tag)
+        .where(Tag.id.not_in(used_via_image_tag))
+        .where(Tag.id.not_in(used_via_series))
+        .order_by(Tag.name)
+    )
+    if limit is not None:
+        stmt = stmt.limit(limit)
+    return list(session.execute(stmt).scalars().all())
+
+
+def unlink_image_files(
+    image: ImageRecord, images_root: Path,
+) -> dict:
+    """Best-effort unlink of all on-disk files for an ImageRecord.
+
+    Targets: image.path (original), image.thumbnail_path (cached
+    thumbnail), and the computed thumbs path at
+    /images/thumbs/<sha256[:3]>/<sha256>.(jpg|png|webp) (tries all
+    three extensions; missing extension is silently OK).
+
+    Returns {"original": bool, "thumbnail": bool}. Missing files
+    count as success (missing_ok semantics). OSErrors are swallowed
+    and reported as False so the calling DB delete still proceeds.
+    """
+    out = {"original": False, "thumbnail": False}
+    if image.path:
+        try:
+            Path(image.path).unlink(missing_ok=True)
+            out["original"] = True
+        except OSError:
+            out["original"] = False
+    # Custom thumbnail_path (when set) — try it first.
+    if image.thumbnail_path:
+        try:
+            Path(image.thumbnail_path).unlink(missing_ok=True)
+            out["thumbnail"] = True
+        except OSError:
+            out["thumbnail"] = False
+    # Convention thumbs dir — try all extensions; missing OK.
+    if image.sha256:
+        bucket = image.sha256[:3]
+        for ext in ("jpg", "png", "webp"):
+            try:
+                (images_root / "thumbs" / bucket / f"{image.sha256}.{ext}").unlink(
+                    missing_ok=True,
+                )
+            except OSError:
+                pass
+    return out
+
+
+def delete_artist_cascade(
+    session: Session, *, artist_id: int, images_root: Path,
+) -> dict:
+    """Batched delete of an artist's images + the artist row.
+
+    Mirrors the cleanup_artist_async pattern: 500-row batches,
+    commit between batches so partial progress survives a worker
+    kill. Idempotent on missing artist (returns zeroed counts).
+    Postgres cascades handle image_tag / image_provenance /
+    series_page / tag_suggestion_rejection from ImageRecord delete,
+    and source / post / download_event / etc. from Artist delete
+    (via Artist.sources cascade="all, delete-orphan").
+    """
+    artist = session.get(Artist, artist_id)
+    if artist is None:
+        return {
+            "artist": None,
+            "summary": {
+                "images_deleted": 0,
+                "files_deleted": 0,
+                "thumbs_deleted": 0,
+                "import_tasks_nulled": 0,
+                "files_failed": 0,
+            },
+        }
+    artist_info = {"id": artist.id, "name": artist.name, "slug": artist.slug}
+
+    images_deleted = 0
+    files_deleted = 0
+    thumbs_deleted = 0
+    files_failed = 0
+
+    while True:
+        rows = session.execute(
+            select(ImageRecord)
+            .where(ImageRecord.artist_id == artist.id)
+            .limit(500)
+        ).scalars().all()
+        if not rows:
+            break
+        for img in rows:
+            unlinked = unlink_image_files(img, images_root)
+            if unlinked["original"]:
+                files_deleted += 1
+            else:
+                files_failed += 1
+            if unlinked["thumbnail"]:
+                thumbs_deleted += 1
+            session.delete(img)
+            images_deleted += 1
+        session.commit()
+
+    # ImportTask.result_image_id FK is SET NULL on image delete (Postgres
+    # handles this in the cascade above). We don't separately count those
+    # in FC-3k — the legacy cleanup_artist_async did it via
+    # source_path_prefix matching that's out of scope here.
+    import_tasks_nulled = 0
+
+    session.delete(artist)
+    session.commit()
+
+    return {
+        "artist": artist_info,
+        "summary": {
+            "images_deleted": images_deleted,
+            "files_deleted": files_deleted,
+            "thumbs_deleted": thumbs_deleted,
+            "import_tasks_nulled": import_tasks_nulled,
+            "files_failed": files_failed,
+        },
+    }
+
+
+def delete_images(
+    session: Session, *, image_ids: list[int], images_root: Path,
+) -> dict:
+    """Delete a list of images in 500-row batches with commit between.
+
+    Postgres CASCADE on image_tag / image_provenance / series_page /
+    tag_suggestion_rejection / post_attachment(FK SET NULL) handles
+    the DB side; this function handles file unlinks first then row
+    deletes. Idempotent on missing IDs (returned as missing_ids;
+    no error). On partial OSError, the row is still deleted and
+    files_failed is incremented.
+    """
+    if not image_ids:
+        return {
+            "images_deleted": 0,
+            "files_deleted": 0,
+            "thumbs_deleted": 0,
+            "files_failed": 0,
+            "missing_ids": [],
+        }
+
+    seen_ids: set[int] = set()
+    images_deleted = 0
+    files_deleted = 0
+    thumbs_deleted = 0
+    files_failed = 0
+
+    pending = list(image_ids)
+    while pending:
+        batch_ids = pending[:500]
+        pending = pending[500:]
+        rows = session.execute(
+            select(ImageRecord).where(ImageRecord.id.in_(batch_ids))
+        ).scalars().all()
+        for img in rows:
+            seen_ids.add(img.id)
+            unlinked = unlink_image_files(img, images_root)
+            if unlinked["original"]:
+                files_deleted += 1
+            else:
+                files_failed += 1
+            if unlinked["thumbnail"]:
+                thumbs_deleted += 1
+            session.delete(img)
+            images_deleted += 1
+        session.commit()
+
+    missing = sorted(set(image_ids) - seen_ids)
+    return {
+        "images_deleted": images_deleted,
+        "files_deleted": files_deleted,
+        "thumbs_deleted": thumbs_deleted,
+        "files_failed": files_failed,
+        "missing_ids": missing,
+    }
+
+
+def delete_tag(session: Session, *, tag_id: int) -> dict:
+    """Simple DELETE FROM tag WHERE id=?.
+
+    Postgres cascades the rest (image_tag, tag_alias, tag_allowlist,
+    tag_reference_embedding, tag_suggestion_rejection, series_page).
+    Returns counts BEFORE delete so the caller can surface them.
+    Raises LookupError if tag_id not found.
+    """
+    tag = session.get(Tag, tag_id)
+    if tag is None:
+        raise LookupError(f"tag id not found: {tag_id}")
+    associations_count = count_tag_associations(session, tag_id=tag_id)
+    info = {"id": tag.id, "name": tag.name, "kind": tag.kind.value}
+    session.delete(tag)
+    session.commit()
+    return {"deleted": info, "associations_removed": associations_count}
+
+
+def prune_unused_tags(session: Session, *, dry_run: bool = False) -> dict:
+    """Find tags with zero references and (unless dry_run) delete them.
+
+    Returns:
+      dry_run=True:  {"count": N, "sample_names": [first 50]}
+      dry_run=False: {"deleted": N, "sample_names": [first 50]}
+    """
+    unused = find_unused_tags(session)
+    sample = [t.name for t in unused[:50]]
+    if dry_run:
+        return {"count": len(unused), "sample_names": sample}
+    ids = [t.id for t in unused]
+    if ids:
+        session.execute(
+            Tag.__table__.delete().where(Tag.id.in_(ids))
+        )
+        session.commit()
+    return {"deleted": len(ids), "sample_names": sample}
+
+
+# Legacy tags FC no longer uses, in two shapes:
+#   (1) kinds the tag input never produces — archive/post/artist.
+#       provenance (post grouping) + archive membership are their own
+#       systems now, and artists are first-class Artist/Source rows.
+#       meta/rating were already hard-deleted by alembic 0023.
+#   (2) name prefixes from IR kinds FC never adopted — `source:*`.
+#       ImageRepo had a `source` kind; FC's enum doesn't, so ir_ingest
+#       fell those back to `general` (kind=general, name="source:patreon"
+#       etc.). They can't be caught by kind, so we match the name prefix.
+PURGEABLE_TAG_KINDS = ("archive", "post", "artist")
+LEGACY_NAME_PREFIXES = ("source:",)
+
+
+def _legacy_tag_predicate():
+    name_clauses = [Tag.name.like(f"{p}%") for p in LEGACY_NAME_PREFIXES]
+    return or_(Tag.kind.in_(PURGEABLE_TAG_KINDS), *name_clauses)
+
+
+def purge_legacy_tags(session: Session, *, dry_run: bool = False) -> dict:
+    """Count (dry_run) or delete legacy IR-migration tags: archive/post/
+    artist-kind tags PLUS general tags whose name matches a legacy
+    prefix (source:*).
+
+    CASCADE on image_tag / tag_alias / tag_allowlist /
+    tag_reference_embedding / tag_suggestion_rejection / series_page
+    clears the related rows on the parent DELETE.
+
+    Returns:
+      {"by_kind": {kind: count, ...},        # kind-matched rows
+       "by_prefix": {"source:*": count},     # name-prefix-matched rows
+       "count": total, "sample_names": [first 50],
+       and on live runs "deleted": total}
+    """
+    predicate = _legacy_tag_predicate()
+    rows = session.execute(
+        select(Tag.id, Tag.name, Tag.kind).where(predicate)
+    ).all()
+    by_kind: dict[str, int] = {}
+    by_prefix: dict[str, int] = {}
+    for _id, name, kind in rows:
+        # Classify by name-prefix first so a source:* row counts once,
+        # under the prefix bucket, regardless of its (general) kind.
+        matched_prefix = next(
+            (p for p in LEGACY_NAME_PREFIXES if name.startswith(p)), None,
+        )
+        if matched_prefix is not None:
+            label = f"{matched_prefix}*"
+            by_prefix[label] = by_prefix.get(label, 0) + 1
+        else:
+            key = kind.value if hasattr(kind, "value") else str(kind)
+            by_kind[key] = by_kind.get(key, 0) + 1
+    sample = [name for _id, name, _kind in rows[:50]]
+    total = len(rows)
+    result = {
+        "by_kind": by_kind, "by_prefix": by_prefix,
+        "count": total, "sample_names": sample,
+    }
+    if dry_run:
+        return result
+    if total:
+        session.execute(Tag.__table__.delete().where(predicate))
+        session.commit()
+    result["deleted"] = total
+    return result
+
+
+# ---------------------------------------------------------------------------
+# FC-Cleanup additions (2026-05-26): retroactive audit of import-filter rules.
+# ---------------------------------------------------------------------------
+
+_MIN_DIM_SAMPLE_CAP = 50
+
+
+def project_min_dimension_violations(
+    session: Session, *, min_width: int, min_height: int,
+) -> dict:
+    """Return {count, sample_ids} for image_record rows with width or
+    height below the thresholds. Synchronous SQL — no PIL inspection
+    needed since width/height are stored columns."""
+    base = select(ImageRecord.id).where(
+        (ImageRecord.width < min_width) | (ImageRecord.height < min_height)
+    )
+    count = session.execute(
+        select(func.count()).select_from(base.subquery())
+    ).scalar_one()
+    sample_ids = session.execute(
+        base.order_by(ImageRecord.id).limit(_MIN_DIM_SAMPLE_CAP)
+    ).scalars().all()
+    return {"count": count, "sample_ids": list(sample_ids)}
+
+
+def delete_min_dimension_violations(
+    session: Session, *, min_width: int, min_height: int, images_root: Path,
+) -> int:
+    """Delete every image_record where width<min_w OR height<min_h.
+    Routes through delete_images so file-unlink + cascading FKs
+    (image_tag / image_provenance / etc.) are handled uniformly."""
+    ids = session.execute(
+        select(ImageRecord.id).where(
+            (ImageRecord.width < min_width) | (ImageRecord.height < min_height)
+        )
+    ).scalars().all()
+    if not ids:
+        return 0
+    result = delete_images(
+        session, image_ids=list(ids), images_root=images_root,
+    )
+    return result["images_deleted"]
+
+
+# ---------------------------------------------------------------------------
+# Audit lifecycle (transparency + single_color async scans).
+# ---------------------------------------------------------------------------
+
+
+class AuditAlreadyRunning(Exception):
+    """Another audit_run is currently in status='running' — wait or
+    cancel it before starting a new one. Surfaces as HTTP 409 in the
+    /api/cleanup/audit POST endpoint."""
+
+
+class AuditNotReady(Exception):
+    """apply_audit_run called on an audit whose status is not 'ready'."""
+
+
+class ConfirmTokenMismatch(Exception):
+    """Operator-supplied confirm token did not match server-recomputed token."""
+
+
+_VALID_RULES = ("transparency", "single_color")
+
+
+def start_audit_run(
+    session: Session, *, rule: str, params: dict[str, Any],
+) -> int:
+    """Create a LibraryAuditRun row in status='running' and dispatch the
+    scan_library_for_rule Celery task. Returns the new audit_id.
+
+    Concurrent-runs guard: raises AuditAlreadyRunning if any audit_run
+    has status='running'. Operator must cancel or wait."""
+    if rule not in _VALID_RULES:
+        raise ValueError(f"unknown rule {rule!r}; expected one of {_VALID_RULES}")
+    existing = session.execute(
+        select(LibraryAuditRun.id).where(LibraryAuditRun.status == "running")
+    ).scalar_one_or_none()
+    if existing is not None:
+        raise AuditAlreadyRunning(existing)
+    audit = LibraryAuditRun(
+        rule=rule,
+        params=params,
+        status="running",
+        scanned_count=0,
+        matched_count=0,
+        matched_ids=[],
+    )
+    session.add(audit)
+    session.flush()
+    audit_id = audit.id
+    # Dispatch after flush so audit_id is populated; commit happens in
+    # the API handler so the audit row + dispatch are visible together.
+    from ..tasks.library_audit import scan_library_for_rule
+    scan_library_for_rule.delay(audit_id)
+    return audit_id
+
+
+def apply_audit_run(
+    session: Session, *, audit_id: int, confirm_token: str, images_root: Path,
+) -> int:
+    """Delete all images in audit_run.matched_ids after confirming token.
+    Marks audit status='applied'. Routes through delete_images so files
+    + cascading FK rows are handled uniformly."""
+    audit = session.execute(
+        select(LibraryAuditRun).where(LibraryAuditRun.id == audit_id)
+    ).scalar_one_or_none()
+    if audit is None:
+        raise ValueError(f"audit_run {audit_id} not found")
+    if audit.status != "ready":
+        raise AuditNotReady(audit.status)
+    # Token format matches modal/DestructiveConfirmModal.vue convention:
+    # ${action}-${kind}-${runId}. The modal hardcodes action ∈ {'restore',
+    # 'delete'}; "apply audit" is semantically a delete of the matched
+    # images, so we use 'delete-audit-<id>' (not 'apply-audit-<id>').
+    expected = f"delete-audit-{audit_id}"
+    if confirm_token != expected:
+        raise ConfirmTokenMismatch(expected)
+    ids = list(audit.matched_ids or [])
+    deleted = 0
+    if ids:
+        result = delete_images(session, image_ids=ids, images_root=images_root)
+        deleted = result["images_deleted"]
+    session.execute(
+        update(LibraryAuditRun)
+        .where(LibraryAuditRun.id == audit_id)
+        .values(status="applied", finished_at=datetime.now(UTC))
+    )
+    return deleted
+
+
+def cancel_audit_run(session: Session, *, audit_id: int) -> None:
+    """Flip a running audit_run to 'cancelled'. The scan task checks
+    for status=='cancelled' between batches and exits cleanly."""
+    session.execute(
+        update(LibraryAuditRun)
+        .where(LibraryAuditRun.id == audit_id)
+        .where(LibraryAuditRun.status == "running")
+        .values(status="cancelled", finished_at=datetime.now(UTC))
+    )

backend/app/services/credential_service.py

@@ -7,7 +7,7 @@ from __future__ import annotations
 import json
 import os
 from dataclasses import dataclass
-from datetime import datetime
+from datetime import UTC, datetime
 from pathlib import Path
 
 from sqlalchemy import select
@@ -148,6 +148,7 @@ class CredentialService:
             return None
         plaintext = self.crypto.decrypt(row.encrypted_blob)
         netscape = _to_netscape(plaintext)
+        netscape = _augment_cookies(platform, netscape)
         self.cookies_dir.mkdir(parents=True, exist_ok=True)
         out = self.cookies_dir / f"{platform}_cookies.txt"
         out.write_text(netscape)
@@ -162,6 +163,32 @@ class CredentialService:
             return None
         return self.crypto.decrypt(row.encrypted_blob)
 
+    async def mark_verified(self, platform: str) -> datetime | None:
+        """Stamp last_verified=now after a successful verify. Returns the
+        timestamp, or None if the credential is gone."""
+        row = (await self.session.execute(
+            select(Credential).where(Credential.platform == platform)
+        )).scalar_one_or_none()
+        if row is None:
+            return None
+        ts = datetime.now(UTC)
+        row.last_verified = ts
+        await self.session.commit()
+        return ts
+
+
+def _augment_cookies(platform: str, netscape: str) -> str:
+    """Delegate to the platform's `augment_cookies` hook if one is
+    registered (subscribestar, hentaifoundry, etc. — see
+    `services/platforms/<name>.py`). No-op when the platform doesn't
+    register a hook (Patreon, DeviantArt). Centralizing the
+    quirks-per-platform in the platforms package means adding a new
+    platform's cookie quirks doesn't require touching this file."""
+    info = PLATFORMS.get(platform)
+    if info is None or info.augment_cookies is None:
+        return netscape
+    return info.augment_cookies(netscape)
+
 
 def _to_netscape(plaintext: str) -> str:
     """Accept either Netscape-format text (the extension's output) or a

backend/app/services/download_service.py

@@ -28,6 +28,7 @@ from .credential_service import CredentialService
 from .gallery_dl import GalleryDLService, SourceConfig
 from .importer import Importer
 from .patreon_resolver import resolve_campaign_id
+from .scheduler_service import set_platform_cooldown
 
 log = logging.getLogger(__name__)
 
@@ -276,18 +277,27 @@ class DownloadService:
         }
         await self._update_source_health(
             source_id=ctx["source_id"], status=status, error_message=ev.error,
+            error_type=dl_result.error_type.value if dl_result.error_type else None,
         )
         await self.async_session.commit()
         return event_id
 
     async def _update_source_health(
         self, *, source_id: int, status: str, error_message: str | None,
+        error_type: str | None = None,
     ) -> None:
         """FC-3d: update Source.{consecutive_failures, last_error, last_checked_at}.
 
         ok      -> failures = 0, error = None, checked_at = now
         error   -> failures += 1, error = error_message, checked_at = now
         skipped -> failures unchanged, error = None, checked_at = now
+
+        When error_type == 'rate_limited', also stamps a platform-wide
+        cooldown via scheduler_service.set_platform_cooldown so the next
+        scan tick skips every source on this platform until the cooldown
+        expires. Preventive half of the burst-prevention pair —
+        consecutive_failures still backs the offending source off across
+        ticks.
         """
         source = (await self.async_session.execute(
             select(Source).where(Source.id == source_id)
@@ -299,6 +309,8 @@ class DownloadService:
         elif status == "error":
             source.consecutive_failures = (source.consecutive_failures or 0) + 1
             source.last_error = error_message
+            if error_type == "rate_limited":
+                await set_platform_cooldown(self.async_session, source.platform)
         elif status == "skipped":
             source.last_error = None
         source.last_checked_at = now

backend/app/services/extension_service.py

@@ -0,0 +1,164 @@
+"""FC-3g: backend support for the Firefox extension.
+
+`ExtensionService.quick_add_source(url)` derives platform + artist
+slug from a URL using regex patterns mirrored from
+extension/lib/platforms.js, then find-or-creates Artist + Source rows
+and returns a JSON-shaped dict for the API layer.
+"""
+
+from __future__ import annotations
+
+import re
+
+from sqlalchemy import select
+from sqlalchemy.exc import IntegrityError
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from ..models import Artist, Source
+from ..utils.slug import slugify
+
+
+class UnknownPlatformError(Exception):
+    """URL didn't match any platform pattern."""
+
+
+class InvalidUrlError(Exception):
+    """URL was empty or missing a scheme."""
+
+
+# Mirrored byte-for-byte from extension/lib/platforms.js
+# PLATFORM_ARTIST_PATTERNS. Keep these two copies in sync by hand —
+# reviewers catch drift.
+_PLATFORM_PATTERNS: list[tuple[str, re.Pattern[str]]] = [
+    ("patreon", re.compile(
+        r"^https?://(?:www\.)?patreon\.com/"
+        r"(?!home$|search\b|messages\b|notifications\b|library\b|settings\b|posts\b|c/)"
+        r"(?P<slug>[^/?#]+)/?$",
+        re.IGNORECASE,
+    )),
+    ("subscribestar", re.compile(
+        r"^https?://(?:www\.)?subscribestar\.(?:com|adult)/"
+        r"(?!feed$|messages$|library$)"
+        r"(?P<slug>[^/?#]+)/?$",
+        re.IGNORECASE,
+    )),
+    ("hentaifoundry", re.compile(
+        r"^https?://(?:www\.)?hentai-foundry\.com/user/(?P<slug>[^/?#]+)",
+        re.IGNORECASE,
+    )),
+    ("deviantart", re.compile(
+        r"^https?://(?:www\.)?deviantart\.com/"
+        r"(?!home$|watch\b|tag\b|browse\b)"
+        r"(?P<slug>[^/?#]+)/?$",
+        re.IGNORECASE,
+    )),
+    ("pixiv", re.compile(
+        r"^https?://(?:www\.)?pixiv\.net/(?:en/)?users/(?P<slug>\d+)",
+        re.IGNORECASE,
+    )),
+]
+
+
+class ExtensionService:
+    def __init__(self, session: AsyncSession) -> None:
+        self.session = session
+
+    async def quick_add_source(self, url: str) -> dict:
+        platform, raw_slug = self._derive(url)
+        artist, created_artist = await self._find_or_create_artist(raw_slug)
+        source, created_source = await self._find_or_create_source(
+            artist_id=artist.id, platform=platform, url=url,
+        )
+        return {
+            "source": {
+                "id": source.id,
+                "artist_id": source.artist_id,
+                "platform": source.platform,
+                "url": source.url,
+                "enabled": source.enabled,
+            },
+            "artist": {
+                "id": artist.id,
+                "name": artist.name,
+                "slug": artist.slug,
+            },
+            "created_source": created_source,
+            "created_artist": created_artist,
+        }
+
+    def _derive(self, url: str) -> tuple[str, str]:
+        if not isinstance(url, str) or not url.strip():
+            raise InvalidUrlError("url is empty")
+        if not url.startswith(("http://", "https://")):
+            raise InvalidUrlError(f"url must include http:// or https:// scheme: {url!r}")
+        for platform, pattern in _PLATFORM_PATTERNS:
+            m = pattern.match(url)
+            if m:
+                return platform, m.group("slug")
+        raise UnknownPlatformError(f"no platform pattern matched {url!r}")
+
+    async def _find_or_create_artist(self, raw_name: str) -> tuple[Artist, bool]:
+        """Race-safe find-or-create on Artist by slug. Mirrors the
+        savepoint + IntegrityError recovery pattern used in
+        Importer._find_or_create_source/post (see
+        reference_scalar_one_or_none_duplicates memory). Without this,
+        two concurrent quick-add-source calls hitting the same artist
+        would both miss the existence check and the second INSERT would
+        500 against uq_artist_slug.
+        """
+        slug = slugify(raw_name)
+        existing = (await self.session.execute(
+            select(Artist).where(Artist.slug == slug)
+        )).scalar_one_or_none()
+        if existing is not None:
+            return existing, False
+        sp = await self.session.begin_nested()
+        try:
+            artist = Artist(name=raw_name, slug=slug, is_subscription=True)
+            self.session.add(artist)
+            await self.session.flush()
+            await sp.commit()
+            return artist, True
+        except IntegrityError:
+            await sp.rollback()
+            recovered = (await self.session.execute(
+                select(Artist).where(Artist.slug == slug)
+            )).scalar_one()
+            return recovered, False
+
+    async def _find_or_create_source(
+        self, *, artist_id: int, platform: str, url: str,
+    ) -> tuple[Source, bool]:
+        """Race-safe — same pattern as _find_or_create_artist above. The
+        uq_source_artist_platform_url constraint catches the duplicate
+        insert; we roll the savepoint back and re-select."""
+        existing = (await self.session.execute(
+            select(Source).where(
+                Source.artist_id == artist_id,
+                Source.platform == platform,
+                Source.url == url,
+            )
+        )).scalar_one_or_none()
+        if existing is not None:
+            return existing, False
+        sp = await self.session.begin_nested()
+        try:
+            src = Source(
+                artist_id=artist_id, platform=platform,
+                url=url, enabled=True,
+            )
+            self.session.add(src)
+            await self.session.flush()
+            await sp.commit()
+        except IntegrityError:
+            await sp.rollback()
+            recovered = (await self.session.execute(
+                select(Source).where(
+                    Source.artist_id == artist_id,
+                    Source.platform == platform,
+                    Source.url == url,
+                )
+            )).scalar_one()
+            return recovered, False
+        await self.session.commit()
+        return src, True

backend/app/services/gallery_dl.py

@@ -42,6 +42,18 @@ class ErrorType(StrEnum):
     UNKNOWN_ERROR = "unknown_error"
 
 
+# 30 seconds shy of download_source's Celery soft_time_limit (900s, see
+# tasks/download.py:32). subprocess.run MUST raise TimeoutExpired before
+# Celery raises SoftTimeLimitExceeded — otherwise Celery wins the race,
+# SIGKILLs the worker, in-memory stdout/stderr is lost, and the
+# DownloadEvent ends up empty-logged with "stranded by recovery sweep"
+# 18 minutes later (operator-flagged 2026-05-31, Knuxy event #38275).
+# The 30s buffer absorbs scheduler jitter / GC pauses without making
+# legitimately-long-running syncs timeout-friendlier. Per-source bumps
+# still live in source.config_overrides for legitimately long syncs.
+_DEFAULT_GDL_TIMEOUT_SECONDS = 870
+
+
 @dataclass
 class SourceConfig:
     content_types: list[str] = field(default_factory=lambda: ["all"])
@@ -51,7 +63,7 @@ class SourceConfig:
     filename_pattern: str | None = None
     skip_existing: bool = True
     save_metadata: bool = True
-    timeout: int = 3600
+    timeout: int = _DEFAULT_GDL_TIMEOUT_SECONDS
 
     @classmethod
     def from_dict(cls, data: dict) -> SourceConfig:
@@ -63,7 +75,7 @@ class SourceConfig:
             filename_pattern=data.get("filename_pattern"),
             skip_existing=data.get("skip_existing", True),
             save_metadata=data.get("save_metadata", True),
-            timeout=data.get("timeout", 3600),
+            timeout=data.get("timeout", _DEFAULT_GDL_TIMEOUT_SECONDS),
         )
 
 
@@ -360,7 +372,17 @@ class GalleryDLService:
         if return_code in (1, 4) and (skip_line_count > 0 or has_skip_text) and not has_actual_error:
             return ErrorType.NO_NEW_CONTENT, "No new content to download"
 
-        if return_code in (1, 4) and not has_actual_error:
+        # Tier-gated classification used to require `return_code in (1, 4)`,
+        # which silently fell through to UNKNOWN_ERROR when gallery-dl
+        # returned a different exit code for mixed-failure runs (e.g.
+        # paywall warnings + a missing yt-dlp dep flipping the exit bits).
+        # The artist then surfaced as "needs attention" purely because a
+        # paywall blocked posts the operator wasn't paying to see —
+        # operator-flagged 2026-05-31. Now: if no source-level error
+        # category fired AND tier-gated warnings are present, classify
+        # as TIER_LIMITED regardless of return code. Same priority order
+        # as before (auth/rate/access/not_found/network/http still win).
+        if not has_actual_error:
             tier_gated_lines = [
                 line for line in combined.split("\n")
                 if "][warning]" in line and "not allowed to view post" in line
@@ -632,13 +654,57 @@ class GalleryDLService:
                 started_at=started_at, completed_at=completed_at,
             )
 
-        except subprocess.TimeoutExpired:
+        except subprocess.TimeoutExpired as e:
             duration = time.time() - start_time
-            log.error("Download timeout for %s/%s after %.1fs", artist_slug, platform, duration)
+            # subprocess.run(text=True) makes these str if non-None, but the
+            # caller may have raised TimeoutExpired manually with None or
+            # bytes (tests do); coerce both cases to str.
+            partial_stdout = e.stdout or ""
+            partial_stderr = e.stderr or ""
+            if isinstance(partial_stdout, bytes):
+                partial_stdout = partial_stdout.decode("utf-8", "replace")
+            if isinstance(partial_stderr, bytes):
+                partial_stderr = partial_stderr.decode("utf-8", "replace")
+
+            files_so_far = self._count_downloaded_files(partial_stdout)
+            written_so_far = [str(p) for p in self._written_paths(partial_stdout)]
+            stderr_lines = partial_stderr.strip().splitlines()
+            tail_hint = stderr_lines[-1] if stderr_lines else "no stderr output"
+
+            # If the partial output already shows a rate-limit pattern, the
+            # timeout was almost certainly gallery-dl spinning on retries —
+            # promote to RATE_LIMITED so _update_source_health stamps the
+            # platform cooldown (same code path as a clean-exit rate limit).
+            # Otherwise stay TIMEOUT and let the captured stdout/stderr +
+            # files_so_far tell the operator whether it was "lots of
+            # content" vs "stuck retrying" vs "hung silent".
+            combined = (partial_stdout + "\n" + partial_stderr).lower()
+            if any(p in combined for p in self.RATE_LIMIT_PATTERNS):
+                error_type = ErrorType.RATE_LIMITED
+                error_message = (
+                    f"Rate-limited and never completed within "
+                    f"{source_config.timeout}s ({files_so_far} files written)"
+                )
+            else:
+                error_type = ErrorType.TIMEOUT
+                error_message = (
+                    f"Download timed out after {source_config.timeout}s — "
+                    f"{files_so_far} file(s) written; last stderr: {tail_hint}"
+                )
+
+            log.error(
+                "Download timeout for %s/%s after %.1fs (%d files written, "
+                "last stderr: %s)",
+                artist_slug, platform, duration, files_so_far, tail_hint,
+            )
+
             return DownloadResult(
                 success=False, url=url, artist_slug=artist_slug, platform=platform,
-                error_type=ErrorType.TIMEOUT,
-                error_message=f"Download timed out after {source_config.timeout} seconds",
+                files_downloaded=files_so_far,
+                written_paths=written_so_far,
+                stdout=partial_stdout, stderr=partial_stderr,
+                return_code=-1,  # killed by timeout, no real exit code
+                error_type=error_type, error_message=error_message,
                 duration_seconds=duration,
                 started_at=started_at,
                 completed_at=datetime.now(UTC).isoformat(),
@@ -658,3 +724,64 @@ class GalleryDLService:
                 Path(temp_config_path).unlink()  # noqa: ASYNC240
             except Exception:
                 pass
+
+    async def verify(
+        self,
+        url: str,
+        artist_slug: str,
+        platform: str,
+        source_config: SourceConfig | None = None,
+        cookies_path: str | None = None,
+        auth_token: str | None = None,
+        timeout: float = 45.0,  # noqa: ASYNC109 — subprocess.run timeout, not a coroutine deadline
+    ) -> tuple[bool, str]:
+        """Test that credentials authenticate against `url` WITHOUT
+        downloading anything. Runs gallery-dl in --simulate mode limited
+        to the first item; if auth is bad the extractor errors before it
+        can list, which _categorize_error flags as AUTH_ERROR. Returns
+        (ok, message). Used by the credential Verify button."""
+        if source_config is None:
+            source_config = SourceConfig()
+        config = self._build_config_for_source(platform, source_config, artist_slug)
+        if cookies_path:
+            config["extractor"]["cookies"] = cookies_path
+        if auth_token and platform == "discord":
+            config["extractor"].setdefault("discord", {})["token"] = auth_token
+        if auth_token and platform == "pixiv":
+            config["extractor"].setdefault("pixiv", {})["refresh-token"] = auth_token
+
+        with tempfile.NamedTemporaryFile(
+            mode="w", suffix=".json", delete=False, dir=str(self._config_dir),
+        ) as fh:
+            json.dump(config, fh, indent=2)
+            temp_config_path = fh.name
+        try:
+            cmd = [
+                sys.executable, "-m", "gallery_dl",
+                "--config", temp_config_path,
+                "--simulate", "--range", "1-1", "--verbose", url,
+            ]
+            loop = asyncio.get_running_loop()
+            proc = await loop.run_in_executor(
+                None,
+                lambda: subprocess.run(
+                    cmd, capture_output=True, text=True, timeout=timeout,
+                ),
+            )
+            etype, msg = self._categorize_error(proc.returncode, proc.stdout, proc.stderr)
+            if proc.returncode == 0 or etype == ErrorType.NO_NEW_CONTENT:
+                return True, "Credentials valid — the feed authenticated."
+            if etype == ErrorType.AUTH_ERROR:
+                return False, msg
+            # Network / not-found / rate-limit / unknown: inconclusive,
+            # not a definitive credential failure. Surface the reason.
+            return False, f"Could not confirm ({etype.value}): {msg}"
+        except subprocess.TimeoutExpired:
+            return False, f"Verification timed out after {timeout:.0f}s"
+        except Exception as exc:  # noqa: BLE001
+            return False, f"Verification error: {exc}"
+        finally:
+            try:
+                Path(temp_config_path).unlink()  # noqa: ASYNC240
+            except Exception:
+                pass

backend/app/services/gallery_service.py

@@ -1,26 +1,34 @@
 """Cursor-paginated gallery queries.
 
-Cursor format: opaque base64-encoded "<iso8601_created_at>:<image_id>".
-Pagination key is (created_at DESC, id DESC) so we don't drift when new
-imports arrive between page loads. Decoding rejects malformed cursors with
-a ValueError; the API layer translates that to HTTP 400.
+Cursor format: opaque base64-encoded "<iso8601_effective_date>:<image_id>".
+
+Pagination key is (effective_date DESC, id DESC) where effective_date is
+COALESCE(post.post_date, image_record.created_at) so the gallery surfaces
+images by ORIGINAL publish date when known, falling back to FC's scan
+date. Important for migrated content: ~57k IR images scanned in a single
+week would otherwise all share the same created_at and pile up in one
+month bucket. The effective_date spreads them across the years they
+were originally published.
+
+Decoding rejects malformed cursors with a ValueError; the API layer
+translates that to HTTP 400.
 """
 
 import base64
 from dataclasses import dataclass
 from datetime import datetime
 
-from sqlalchemy import and_, exists, func, or_, select
+from sqlalchemy import Select, and_, exists, func, or_, select
 from sqlalchemy.ext.asyncio import AsyncSession
 
-from ..models import Artist, ImageProvenance, ImageRecord, Source, Tag
+from ..models import Artist, ImageProvenance, ImageRecord, Post, Source, Tag
 from ..models.tag import image_tag
 
 CURSOR_SEPARATOR = "|"
 
 
-def encode_cursor(created_at: datetime, image_id: int) -> str:
-    raw = f"{created_at.isoformat()}{CURSOR_SEPARATOR}{image_id}"
+def encode_cursor(effective_date: datetime, image_id: int) -> str:
+    raw = f"{effective_date.isoformat()}{CURSOR_SEPARATOR}{image_id}"
     return base64.urlsafe_b64encode(raw.encode()).decode()
 
 
@@ -33,6 +41,26 @@ def decode_cursor(cursor: str) -> tuple[datetime, int]:
         raise ValueError(f"invalid cursor: {cursor!r}") from exc
 
 
+def _effective_date_col():
+    """SQL expression: COALESCE(post.post_date, image_record.created_at).
+
+    Used as the canonical sort/group/filter key across the gallery so
+    images backfilled with primary_post_id (e.g. via tag_apply phase 4)
+    surface at their original publish date, not their FC import date.
+    Images without a Post (or with Post.post_date NULL) fall back to
+    image_record.created_at and still order coherently against
+    post-attached ones.
+    """
+    return func.coalesce(Post.post_date, ImageRecord.created_at)
+
+
+def _outer_join_primary_post(stmt: Select) -> Select:
+    """LEFT JOIN Post on ImageRecord.primary_post_id so the COALESCE
+    above sees Post.post_date when available. Images without a post
+    survive the join as NULL on the Post side; COALESCE handles it."""
+    return stmt.outerjoin(Post, Post.id == ImageRecord.primary_post_id)
+
+
 @dataclass(frozen=True)
 class GalleryImage:
     id: int
@@ -41,7 +69,9 @@ class GalleryImage:
     mime: str
     width: int | None
     height: int | None
-    created_at: datetime
+    created_at: datetime          # FC's row-insert time
+    effective_date: datetime      # COALESCE(post.post_date, created_at)
+    posted_at: datetime | None    # post.post_date if known, else None
     thumbnail_url: str
     artist: dict | None = None
 
@@ -60,9 +90,27 @@ class TimelineBucket:
     count: int
 
 
-def thumbnail_url(sha256_hex: str, mime: str) -> str:
-    # Quart serves /images/* via the frontend blueprint (FC-1); thumbnails go
-    # under /images/thumbs/. The MIME determines the extension.
+def thumbnail_url(thumbnail_path: str | None, sha256_hex: str, mime: str) -> str:
+    """Return the URL to fetch a thumbnail.
+
+    Prefers the stored thumbnail_path verbatim — Quart serves /images/*
+    1:1 from the volume (frontend.py:20-36), so the URL IS the disk
+    path. Falls back to deriving from (sha256, mime) only when the
+    record's thumbnail_path is NULL (thumbnailer hasn't run yet); that
+    URL will 404 until backfill catches it, same as before the path
+    was tracked.
+
+    Pre-2026-05-30 this was derived only from (sha256, mime), which
+    disagreed with the actual on-disk extension when the thumbnailer
+    chose its format from transparency rather than MIME — every PNG
+    source without alpha (extension was .jpg on disk) and every WebP
+    source with alpha (extension was .png on disk) silently 404'd
+    despite the thumbnail file existing.
+    """
+    if thumbnail_path:
+        return thumbnail_path
+    # Fallback for records with no thumbnail recorded yet — preserves
+    # prior behavior (URL exists but 404s until backfill regenerates).
     ext = ".png" if mime in ("image/png", "image/gif") else ".jpg"
     bucket = sha256_hex[:3]
     return f"/images/thumbs/{bucket}/{sha256_hex}{ext}"
@@ -78,7 +126,7 @@ def _require_single_filter(tag_id, post_id, artist_id) -> None:
 def _provenance_clause(post_id, artist_id):
     """Correlated EXISTS clause (NOT a join) so an image with multiple
     matching provenance rows is returned exactly once and the
-    (created_at DESC, id DESC) cursor ordering is unaffected."""
+    (effective_date DESC, id DESC) cursor ordering is unaffected."""
     if post_id is not None:
         return exists().where(
             ImageProvenance.image_record_id == ImageRecord.id,
@@ -125,7 +173,9 @@ class GalleryService:
             raise ValueError("limit must be between 1 and 200")
         _require_single_filter(tag_id, post_id, artist_id)
 
-        stmt = select(ImageRecord)
+        eff = _effective_date_col()
+        stmt = select(ImageRecord, Post.post_date, eff.label("eff"))
+        stmt = _outer_join_primary_post(stmt)
         if tag_id is not None:
             stmt = stmt.join(image_tag, image_tag.c.image_record_id == ImageRecord.id).where(
                 image_tag.c.tag_id == tag_id
@@ -138,34 +188,38 @@ class GalleryService:
             cur_ts, cur_id = decode_cursor(cursor)
             stmt = stmt.where(
                 or_(
-                    ImageRecord.created_at < cur_ts,
-                    and_(ImageRecord.created_at == cur_ts, ImageRecord.id < cur_id),
+                    eff < cur_ts,
+                    and_(eff == cur_ts, ImageRecord.id < cur_id),
                 )
             )
 
-        stmt = stmt.order_by(ImageRecord.created_at.desc(), ImageRecord.id.desc()).limit(limit + 1)
-        rows = (await self.session.execute(stmt)).scalars().all()
+        stmt = stmt.order_by(eff.desc(), ImageRecord.id.desc()).limit(limit + 1)
+        rows = (await self.session.execute(stmt)).all()
 
         next_cursor = None
         if len(rows) > limit:
-            last = rows[limit - 1]
-            next_cursor = encode_cursor(last.created_at, last.id)
+            last_record, _last_posted_at, last_eff = rows[limit - 1]
+            next_cursor = encode_cursor(last_eff, last_record.id)
             rows = rows[:limit]
 
-        artists = await _artists_for(self.session, [r.id for r in rows])
+        artists = await _artists_for(
+            self.session, [r[0].id for r in rows]
+        )
         images = [
             GalleryImage(
-                id=r.id,
-                path=r.path,
-                sha256=r.sha256,
-                mime=r.mime,
-                width=r.width,
-                height=r.height,
-                created_at=r.created_at,
-                thumbnail_url=thumbnail_url(r.sha256, r.mime),
-                artist=artists.get(r.id),
+                id=record.id,
+                path=record.path,
+                sha256=record.sha256,
+                mime=record.mime,
+                width=record.width,
+                height=record.height,
+                created_at=record.created_at,
+                effective_date=eff_date,
+                posted_at=posted_at,
+                thumbnail_url=thumbnail_url(record.thumbnail_path, record.sha256, record.mime),
+                artist=artists.get(record.id),
             )
-            for r in rows
+            for record, posted_at, eff_date in rows
         ]
         return GalleryPage(
             images=images,
@@ -179,11 +233,13 @@ class GalleryService:
         post_id: int | None = None,
         artist_id: int | None = None,
     ) -> list[TimelineBucket]:
-        year_col = func.date_part("year", ImageRecord.created_at).label("yr")
-        month_col = func.date_part("month", ImageRecord.created_at).label("mo")
+        eff = _effective_date_col()
+        year_col = func.date_part("year", eff).label("yr")
+        month_col = func.date_part("month", eff).label("mo")
         stmt = select(
             year_col, month_col, func.count(ImageRecord.id).label("cnt")
         )
+        stmt = _outer_join_primary_post(stmt)
         _require_single_filter(tag_id, post_id, artist_id)
         if tag_id is not None:
             stmt = stmt.join(image_tag, image_tag.c.image_record_id == ImageRecord.id).where(
@@ -201,14 +257,17 @@ class GalleryService:
         post_id: int | None = None, artist_id: int | None = None,
     ) -> str | None:
         """Returns a cursor that, when passed to scroll(), positions at the
-        first image of the given year-month. None if the bucket is empty.
+        first image of the given year-month (by effective_date, not
+        created_at). None if the bucket is empty.
         """
         from sqlalchemy import extract
 
-        stmt = select(ImageRecord).where(
-            extract("year", ImageRecord.created_at) == year,
-            extract("month", ImageRecord.created_at) == month,
+        eff = _effective_date_col()
+        stmt = select(ImageRecord, eff.label("eff")).where(
+            extract("year", eff) == year,
+            extract("month", eff) == month,
         )
+        stmt = _outer_join_primary_post(stmt)
         _require_single_filter(tag_id, post_id, artist_id)
         if tag_id is not None:
             stmt = stmt.join(image_tag, image_tag.c.image_record_id == ImageRecord.id).where(
@@ -217,13 +276,14 @@ class GalleryService:
         prov = _provenance_clause(post_id, artist_id)
         if prov is not None:
             stmt = stmt.where(prov)
-        stmt = stmt.order_by(ImageRecord.created_at.desc(), ImageRecord.id.desc()).limit(1)
-        first = (await self.session.execute(stmt)).scalar_one_or_none()
+        stmt = stmt.order_by(eff.desc(), ImageRecord.id.desc()).limit(1)
+        first = (await self.session.execute(stmt)).first()
         if first is None:
             return None
+        record, eff_date = first
         # Cursor is exclusive; we encode a cursor with id+1 so the row itself
         # is the first result in the next scroll().
-        return encode_cursor(first.created_at, first.id + 1)
+        return encode_cursor(eff_date, record.id + 1)
 
     async def get_image_with_tags(self, image_id: int) -> dict | None:
         record = await self.session.get(ImageRecord, image_id)
@@ -236,7 +296,23 @@ class GalleryService:
             .order_by(Tag.kind.asc(), Tag.name.asc())
         )
         tags = (await self.session.execute(tag_stmt)).scalars().all()
+        # Fetch the canonical post.post_date for this image (if any) so
+        # the modal can show "Posted on <date>" alongside import date.
+        posted_at = None
+        if record.primary_post_id is not None:
+            posted_at = (await self.session.execute(
+                select(Post.post_date).where(Post.id == record.primary_post_id)
+            )).scalar_one_or_none()
         neighbors = await self._neighbors(record)
+        # Direct artist FK — used by the modal's ProvenancePanel as a
+        # fallback when ImageProvenance is empty (i.e., filesystem-
+        # imported images without a post-track provenance row). The
+        # source of truth for richer post-level data is still
+        # ImageProvenance/Post; this is just the "we at least know who
+        # made it" line.
+        artist = None
+        if record.artist_id is not None:
+            artist = await self.session.get(Artist, record.artist_id)
         return {
             "id": record.id,
             "path": record.path,
@@ -247,8 +323,13 @@ class GalleryService:
             "size_bytes": record.size_bytes,
             "integrity_status": record.integrity_status,
             "created_at": record.created_at.isoformat(),
-            "thumbnail_url": thumbnail_url(record.sha256, record.mime),
+            "posted_at": posted_at.isoformat() if posted_at else None,
+            "thumbnail_url": thumbnail_url(record.thumbnail_path, record.sha256, record.mime),
             "image_url": f"/images/{record.path.split('/images/', 1)[-1]}",
+            "artist": (
+                {"id": artist.id, "name": artist.name, "slug": artist.slug}
+                if artist is not None else None
+            ),
             "tags": [
                 {
                     "id": t.id,
@@ -262,34 +343,41 @@ class GalleryService:
         }
 
     async def _neighbors(self, record: ImageRecord) -> dict:
-        prev_stmt = (
-            select(ImageRecord.id)
-            .where(
+        # Compute the boundary image's effective_date in Python (one query
+        # below + the SELECT we already have on `record`) and use it for
+        # the neighbor comparison. Cheaper than re-deriving in SQL via
+        # correlated subquery.
+        boundary_eff = record.created_at
+        if record.primary_post_id is not None:
+            post_date = (await self.session.execute(
+                select(Post.post_date).where(Post.id == record.primary_post_id)
+            )).scalar_one_or_none()
+            if post_date is not None:
+                boundary_eff = post_date
+
+        eff = _effective_date_col()
+        prev_stmt = _outer_join_primary_post(
+            select(ImageRecord.id).where(
                 or_(
-                    ImageRecord.created_at > record.created_at,
+                    eff > boundary_eff,
                     and_(
-                        ImageRecord.created_at == record.created_at,
+                        eff == boundary_eff,
                         ImageRecord.id > record.id,
                     ),
                 )
             )
-            .order_by(ImageRecord.created_at.asc(), ImageRecord.id.asc())
-            .limit(1)
-        )
-        next_stmt = (
-            select(ImageRecord.id)
-            .where(
+        ).order_by(eff.asc(), ImageRecord.id.asc()).limit(1)
+        next_stmt = _outer_join_primary_post(
+            select(ImageRecord.id).where(
                 or_(
-                    ImageRecord.created_at < record.created_at,
+                    eff < boundary_eff,
                     and_(
-                        ImageRecord.created_at == record.created_at,
+                        eff == boundary_eff,
                         ImageRecord.id < record.id,
                     ),
                 )
             )
-            .order_by(ImageRecord.created_at.desc(), ImageRecord.id.desc())
-            .limit(1)
-        )
+        ).order_by(eff.desc(), ImageRecord.id.desc()).limit(1)
         prev_id = (await self.session.execute(prev_stmt)).scalar_one_or_none()
         next_id = (await self.session.execute(next_stmt)).scalar_one_or_none()
         return {"prev_id": prev_id, "next_id": next_id}
@@ -298,9 +386,11 @@ class GalleryService:
 def _group_by_year_month(
     images: list[GalleryImage],
 ) -> list[tuple[int, int, list[int]]]:
+    """Group by effective_date's year/month so migrated content surfaces
+    in the publish-date buckets, not the FC-scan-date bucket."""
     groups: list[tuple[int, int, list[int]]] = []
     for img in images:
-        y, m = img.created_at.year, img.created_at.month
+        y, m = img.effective_date.year, img.effective_date.month
         if groups and groups[-1][0] == y and groups[-1][1] == m:
             groups[-1][2].append(img.id)
         else:

backend/app/services/importer.py

@@ -18,6 +18,7 @@ from pathlib import Path
 
 from PIL import Image
 from sqlalchemy import select
+from sqlalchemy.exc import IntegrityError
 from sqlalchemy.orm import Session
 
 from ..models import (
@@ -29,6 +30,7 @@ from ..models import (
     PostAttachment,
     Source,
 )
+from ..utils import safe_probe
 from ..utils.paths import derive_subdir, derive_top_level_artist, hash_suffixed_name
 from ..utils.phash import compute_phash, find_similar
 from ..utils.sidecar import find_sidecar, parse_sidecar
@@ -51,7 +53,14 @@ class SkipReason(StrEnum):
 
 @dataclass(frozen=True)
 class ImportResult:
-    status: str  # 'imported'|'skipped'|'failed'|'superseded'|'attached'
+    # 'imported'   — new ImageRecord row created
+    # 'superseded' — existing ImageRecord row got the new file (larger) + sidecar
+    # 'attached'   — non-media saved as PostAttachment
+    # 'refreshed'  — deep scan re-applied sidecar / filled NULL phash / NULL
+    #                artist on an already-imported row (no new ImageRecord)
+    # 'skipped'    — no work done (true duplicate, too small, etc.)
+    # 'failed'     — pipeline error
+    status: str
     image_id: int | None = None
     skip_reason: SkipReason | None = None
     error: str | None = None
@@ -71,6 +80,31 @@ def is_video(path: Path) -> bool:
     return path.suffix.lower() in VIDEO_EXTS
 
 
+def _safe_ext(path: Path) -> str:
+    """Conservatively extract a file extension for PostAttachment.ext
+    (varchar(32)).
+
+    gallery-dl produces some filenames with URL-encoded query-string
+    artifacts embedded into the basename (e.g.
+    `79507046_media_..._https___www.patreon.com_media-u_Z0FBQUFBQm5q...`).
+    `Path.suffix` finds the LAST dot and returns everything after, which
+    in those cases yields a 50+ char "extension" of mostly base64-ish
+    junk. That blows the column. Operator-flagged 2026-05-25.
+
+    Real extensions are short and alphanumeric. We accept anything ≤ 16
+    chars where every post-dot character is alphanumeric; anything else
+    means the input wasn't a real extension and we return the empty
+    string. ext is nullable-ish (empty string still satisfies NOT NULL)
+    and consumers should treat "" as "no known extension".
+    """
+    suffix = path.suffix.lower()
+    if not suffix or len(suffix) > 16:
+        return ""
+    if not all(c.isalnum() for c in suffix[1:]):
+        return ""
+    return suffix
+
+
 def _mime_for(path: Path) -> str:
     suffix = path.suffix.lower()
     image_mimes = {
@@ -126,6 +160,177 @@ class Importer:
         self.settings = settings
         self.deep = deep
         self.attachments = AttachmentStore(images_root)
+        # phash near-dup candidate cache. Archive imports call _import_media
+        # per-member; without this cache the per-member SELECT *FROM
+        # image_record WHERE phash IS NOT NULL fetch repeats N times and a
+        # large library × many-member archive blew past soft_time_limit
+        # (300s) — operator-flagged 2026-05-25. Loaded lazily on first
+        # need, appended to on every imported/superseded outcome, never
+        # invalidated mid-Importer (Importer instances are per-task /
+        # per-archive-import so cross-instance staleness is harmless).
+        self._phash_candidates: list[tuple] | None = None
+
+    def _phash_candidates_cache(self) -> list[tuple]:
+        """Cached `(phash, width, height, id)` rows from image_record.
+        Loaded on first call, appended-to on subsequent imported/
+        superseded outcomes. Soft-timeout pattern: an archive with N
+        members + a library of M existing rows used to do N × M-row
+        fetches (operator-flagged 2026-05-25); now it's exactly one.
+
+        The per-task lifecycle of Importer (instantiated fresh by
+        import_media_file) bounds the cache's staleness window: cross-
+        process changes (other workers importing concurrently) won't
+        be reflected, but that's the same race the un-cached version
+        had — `find_similar` is best-effort anyway."""
+        if self._phash_candidates is None:
+            rows = self.session.execute(
+                select(
+                    ImageRecord.phash,
+                    ImageRecord.width,
+                    ImageRecord.height,
+                    ImageRecord.id,
+                ).where(ImageRecord.phash.is_not(None))
+            ).all()
+            self._phash_candidates = [
+                (r.phash, r.width or 0, r.height or 0, r.id) for r in rows
+            ]
+        return self._phash_candidates
+
+    def _phash_cache_append(self, phash, width, height, image_id) -> None:
+        """Append a freshly-imported row to the cache so subsequent
+        members of the same archive can match against it."""
+        if self._phash_candidates is not None and phash is not None:
+            self._phash_candidates.append(
+                (phash, width or 0, height or 0, image_id)
+            )
+
+    def _get_or_create(self, stmt, factory):
+        """Race-safe find-or-create. Run `stmt` (scalar_one_or_none); if a
+        row exists, return it. Otherwise open a savepoint and INSERT
+        ``factory()``; on IntegrityError (a concurrent worker inserted the
+        same row first) roll the savepoint back — NOT the outer transaction,
+        which would lose the surrounding scan's progress — and re-run `stmt`
+        (scalar_one) to return the row the other worker created.
+
+        Centralizes the pattern shared by _find_or_create_source,
+        _source_for_sidecar, and _find_or_create_post. The plain
+        SELECT-then-INSERT version lost races under the 5-min recovery sweep
+        (operator-flagged 2026-05-26)."""
+        existing = self.session.execute(stmt).scalar_one_or_none()
+        if existing is not None:
+            return existing
+        sp = self.session.begin_nested()
+        try:
+            row = factory()
+            self.session.add(row)
+            self.session.flush()
+            sp.commit()
+            return row
+        except IntegrityError:
+            sp.rollback()
+            return self.session.execute(stmt).scalar_one()
+
+    def _find_or_create_source(
+        self, *, artist_id: int, platform: str, url: str,
+    ) -> Source:
+        """Race-safe find-or-create on `source` keyed by
+        (artist_id, platform, url) — the same key as the
+        `uq_source_artist_platform_url` constraint.
+
+        Two concurrent workers processing different files in the same
+        post can both find no existing Source row then both INSERT,
+        which trips the unique constraint and poisons the session with
+        `psycopg.errors.UniqueViolation`. Operator-flagged 2026-05-26.
+
+        Pattern: select; if absent, open a savepoint and INSERT.
+        On IntegrityError, roll the savepoint back (NOT the outer
+        transaction, which would lose the surrounding scan's progress)
+        and re-select — the concurrent op just created the row we
+        wanted, so the second select will find it.
+        """
+        stmt = select(Source).where(
+            Source.artist_id == artist_id,
+            Source.platform == platform,
+            Source.url == url,
+        )
+        return self._get_or_create(
+            stmt,
+            lambda: Source(artist_id=artist_id, platform=platform, url=url),
+        )
+
+    def _source_for_sidecar(
+        self, *, artist_id: int, platform: str, artist_slug: str,
+    ) -> Source:
+        """Sidecar-import Source resolver. Used by both filesystem imports
+        and gallery-dl downloads (both write sidecar JSON, both flow through
+        _apply_sidecar / _capture_attachment).
+
+        Source represents a subscription feed (one per artist+platform — the
+        URL polled by the FC-3 downloader). The filesystem importer used to
+        call _find_or_create_source(url=sd.post_url), creating one Source
+        row per post URL — 100s of junk Sources per artist, all with
+        enabled=True, polluting the artist detail page and tricking the
+        subscription checker into trying to poll patreon post URLs as feeds.
+        Operator-flagged 2026-05-26; consolidated via alembic 0022.
+
+        Resolution order: prefer a real (non-sidecar) Source over a
+        synthetic anchor. When alembic 0022 ran, it may have rewritten
+        per-post Sources into `sidecar:<platform>:<slug>` synthetic
+        anchors. If the operator later added the real subscription, both
+        rows now coexist. A naive `ORDER BY id ASC LIMIT 1` lookup would
+        pick the older synthetic and silently attach every gallery-dl
+        download to the wrong Source — operator-flagged 2026-05-31 after
+        the Subscriptions UI surfaced the phantom anchors. Pick the real
+        one when one exists; fall back to the synthetic; only create a
+        new synthetic when nothing exists for (artist, platform).
+        """
+        real_stmt = (
+            select(Source)
+            .where(
+                Source.artist_id == artist_id,
+                Source.platform == platform,
+                ~Source.url.like("sidecar:%"),
+            )
+            .order_by(Source.id.asc())
+            .limit(1)
+        )
+        real = self.session.execute(real_stmt).scalar_one_or_none()
+        if real is not None:
+            return real
+
+        any_stmt = (
+            select(Source)
+            .where(
+                Source.artist_id == artist_id,
+                Source.platform == platform,
+            )
+            .order_by(Source.id.asc())
+            .limit(1)
+        )
+        return self._get_or_create(
+            any_stmt,
+            lambda: Source(
+                artist_id=artist_id,
+                platform=platform,
+                url=f"sidecar:{platform}:{artist_slug}",
+                enabled=False,
+            ),
+        )
+
+    def _find_or_create_post(
+        self, *, source_id: int, external_post_id: str,
+    ) -> Post:
+        """Race-safe find-or-create on `post` keyed by
+        (source_id, external_post_id). Mirrors `_find_or_create_source`
+        — same savepoint + IntegrityError-recovery pattern."""
+        stmt = select(Post).where(
+            Post.source_id == source_id,
+            Post.external_post_id == external_post_id,
+        )
+        return self._get_or_create(
+            stmt,
+            lambda: Post(source_id=source_id, external_post_id=external_post_id),
+        )
 
     def import_one(self, source: Path) -> ImportResult:
         """Dispatch by kind. Media → normal pipeline. Archive → extract
@@ -165,30 +370,13 @@ class Importer:
             return None
         sd = parse_sidecar(data)
         platform = sd.platform or "unknown"
-        url = sd.post_url or f"sidecar:{platform}"
-        src = self.session.execute(
-            select(Source).where(
-                Source.artist_id == artist.id,
-                Source.platform == platform,
-                Source.url == url,
-            )
-        ).scalar_one_or_none()
-        if src is None:
-            src = Source(artist_id=artist.id, platform=platform, url=url)
-            self.session.add(src)
-            self.session.flush()
+        src = self._source_for_sidecar(
+            artist_id=artist.id, platform=platform, artist_slug=artist.slug,
+        )
         epid = sd.external_post_id or sc.stem
-        post = self.session.execute(
-            select(Post).where(
-                Post.source_id == src.id,
-                Post.external_post_id == epid,
-            )
-        ).scalar_one_or_none()
-        if post is None:
-            post = Post(source_id=src.id, external_post_id=epid)
-            self.session.add(post)
-            self.session.flush()
-        return post
+        return self._find_or_create_post(
+            source_id=src.id, external_post_id=epid,
+        )
 
     def _capture_attachment(
         self, source: Path, *, post: Post | None = None,
@@ -209,7 +397,7 @@ class Importer:
                 sha256=sha,
                 path=stored,
                 original_filename=source.name,
-                ext=source.suffix.lower(),
+                ext=_safe_ext(source),
                 mime=_mime_for(source),
                 size_bytes=source.stat().st_size,
             ))
@@ -218,6 +406,29 @@ class Importer:
         return ImportResult(status="attached")
 
     def _import_archive(self, source: Path) -> ImportResult:
+        # Layer-3 isolation: bomb-size guard + integrity test in a
+        # spawned child BEFORE extracting in this process. A
+        # decompression bomb or a native-lib crash on a malformed
+        # archive is contained to the child; we reject the file cleanly
+        # instead of OOMing/segfaulting the import worker. extract_archive
+        # is already fail-soft for plain exceptions, so this only adds
+        # the hard-crash protection.
+        probe = safe_probe.probe_archive(source)
+        if not probe.ok:
+            if probe.crashed:
+                return ImportResult(
+                    status="failed",
+                    error=f"archive probe crashed/timed out: {probe.reason}",
+                )
+            # Clean rejection (bomb cap exceeded, integrity mismatch):
+            # still preserve the archive file itself as an attachment so
+            # nothing silently vanishes, matching extract_archive's
+            # fail-soft contract.
+            artist = self._resolve_artist(source)
+            post = self._post_for_sidecar(source, artist)
+            self._capture_attachment(source, post=post, artist=artist, resolved=True)
+            return ImportResult(status="attached")
+
         artist = self._resolve_artist(source)
         post = self._post_for_sidecar(source, artist)
         member_ids: list[int] = []
@@ -257,7 +468,25 @@ class Importer:
         # Compute file dimensions (images only) and apply filters.
         width = height = None
         has_alpha = False
-        if not is_video(source):
+        if is_video(source):
+            # Layer-3 isolation: validate the container via ffprobe (a
+            # separate process) before the rest of the pipeline touches
+            # it. A corrupt video that would crash a decoder is rejected
+            # cleanly here, and we capture width/height for free (the
+            # importer didn't previously record video dimensions).
+            probe = safe_probe.probe_video(source)
+            if not probe.ok:
+                if probe.crashed:
+                    return ImportResult(
+                        status="failed",
+                        error=f"video probe crashed/timed out: {probe.reason}",
+                    )
+                return ImportResult(
+                    status="skipped", skip_reason=SkipReason.invalid_image,
+                    error=probe.reason,
+                )
+            width, height = probe.width, probe.height
+        else:
             try:
                 with Image.open(source) as im:
                     im.verify()
@@ -280,7 +509,18 @@ class Importer:
                 )
 
             if self.settings.skip_transparent and has_alpha:
-                pct = self._transparency_pct(source)
+                try:
+                    pct = self._transparency_pct(source)
+                except OSError as exc:
+                    # PIL.verify() at line 263 only validates header structure;
+                    # truncated/corrupt pixel data only surfaces when load()
+                    # actually decodes (here via getchannel('A')). Convert to
+                    # invalid_image skip so the Celery autoretry loop doesn't
+                    # bounce the same broken file forever.
+                    return ImportResult(
+                        status="skipped", skip_reason=SkipReason.invalid_image,
+                        error=f"PIL load failed during transparency check: {exc}",
+                    )
                 if pct >= self.settings.transparency_threshold:
                     return ImportResult(
                         status="skipped", skip_reason=SkipReason.too_transparent,
@@ -302,21 +542,18 @@ class Importer:
         # Perceptual near-dup (images only; videos keep phash NULL).
         phash = None
         if not is_video(source):
-            with Image.open(source) as im:
-                phash = compute_phash(im)
+            try:
+                with Image.open(source) as im:
+                    phash = compute_phash(im)
+            except OSError as exc:
+                # Same rationale as the transparency-check guard above:
+                # broken-pixel-data files pass verify() but blow up here.
+                return ImportResult(
+                    status="skipped", skip_reason=SkipReason.invalid_image,
+                    error=f"PIL load failed during phash compute: {exc}",
+                )
             if phash is not None:
-                cand_rows = self.session.execute(
-                    select(
-                        ImageRecord.phash,
-                        ImageRecord.width,
-                        ImageRecord.height,
-                        ImageRecord.id,
-                    ).where(ImageRecord.phash.is_not(None))
-                ).all()
-                candidates = [
-                    (c.phash, c.width or 0, c.height or 0, c.id)
-                    for c in cand_rows
-                ]
+                candidates = self._phash_candidates_cache()
                 rel, match_id = find_similar(
                     phash, width or 0, height or 0,
                     candidates, self.settings.phash_threshold,
@@ -350,6 +587,7 @@ class Importer:
         )
         self.session.add(record)
         self.session.flush()
+        self._phash_cache_append(phash, width, height, record.id)
 
         # Folder→artist (anchored to attribution_path).
         artist = None
@@ -373,13 +611,27 @@ class Importer:
     ) -> ImportResult:
         """Deep scan: backfill phash/provenance/artist on an
         already-imported record. METADATA ONLY — never re-runs the pHash
-        near-dup / supersede path. NULL-only, idempotent."""
+        near-dup / supersede path. NULL-only on phash/artist, additive on
+        sidecar Post/Source/ImageProvenance (via _apply_sidecar).
+        Idempotent: a second deep-scan over the same file finds nothing
+        to refresh and is a no-op.
+
+        Returns status="refreshed" so the UI can surface the work done
+        instead of the prior misleading "skipped/duplicate_hash" reading.
+        Operator-flagged 2026-05-25 — IR has had this; FC inherited it
+        as a no-op skip during the original port and the UI showed deep
+        scan as "completed with no changes" even when sidecar metadata
+        actually got re-applied to N existing rows.
+        """
         if existing.phash is None and not is_video(source):
             try:
                 with Image.open(source) as im:
                     ph = compute_phash(im)
                 if ph is not None:
                     existing.phash = ph
+                    # Promoted from NULL to non-NULL → cache is now stale
+                    # (this row would newly qualify for the candidates set).
+                    self._phash_candidates = None
             except Exception as exc:
                 log.warning("deep rephash failed for %s: %s", source, exc)
 
@@ -392,10 +644,7 @@ class Importer:
 
         self._apply_sidecar(existing, attribution_path, artist)
         self.session.commit()
-        return ImportResult(
-            status="skipped", skip_reason=SkipReason.duplicate_hash,
-            image_id=existing.id, error="deep: re-derived",
-        )
+        return ImportResult(status="refreshed", image_id=existing.id)
 
     def attach_in_place(
         self,
@@ -479,18 +728,7 @@ class Importer:
             except Exception:
                 phash = None
             if phash is not None:
-                cand_rows = self.session.execute(
-                    select(
-                        ImageRecord.phash,
-                        ImageRecord.width,
-                        ImageRecord.height,
-                        ImageRecord.id,
-                    ).where(ImageRecord.phash.is_not(None))
-                ).all()
-                candidates = [
-                    (c.phash, c.width or 0, c.height or 0, c.id)
-                    for c in cand_rows
-                ]
+                candidates = self._phash_candidates_cache()
                 rel, match_id = find_similar(
                     phash, width or 0, height or 0,
                     candidates, self.settings.phash_threshold,
@@ -525,6 +763,7 @@ class Importer:
             record.artist_id = artist.id
         self.session.add(record)
         self.session.flush()
+        self._phash_cache_append(phash, width, height, record.id)
 
         # Sidecar provenance (best-effort). When `source` is passed, link
         # the post to that subscription Source instead of creating a new
@@ -619,30 +858,15 @@ class Importer:
             src = explicit_source
         else:
             platform = sd.platform or "unknown"
-            url = sd.post_url or f"sidecar:{platform}"
-            src = self.session.execute(
-                select(Source).where(
-                    Source.artist_id == artist.id,
-                    Source.platform == platform,
-                    Source.url == url,
-                )
-            ).scalar_one_or_none()
-            if src is None:
-                src = Source(artist_id=artist.id, platform=platform, url=url)
-                self.session.add(src)
-                self.session.flush()
+            src = self._source_for_sidecar(
+                artist_id=artist.id, platform=platform,
+                artist_slug=artist.slug,
+            )
 
         epid = sd.external_post_id or sc.stem
-        post = self.session.execute(
-            select(Post).where(
-                Post.source_id == src.id,
-                Post.external_post_id == epid,
-            )
-        ).scalar_one_or_none()
-        if post is None:
-            post = Post(source_id=src.id, external_post_id=epid)
-            self.session.add(post)
-            self.session.flush()
+        post = self._find_or_create_post(
+            source_id=src.id, external_post_id=epid,
+        )
         if sd.post_url is not None:
             post.post_url = sd.post_url
         if sd.post_title is not None:
@@ -655,6 +879,15 @@ class Importer:
             post.attachment_count = sd.attachment_count
         post.raw_metadata = sd.raw
 
+        # Race-safe (image_record_id, post_id) upsert — mirrors the
+        # _find_or_create_source/post savepoint pattern. The plain
+        # SELECT-then-INSERT pattern lost a race when two workers ran
+        # _apply_sidecar on the same (image, post) pair (e.g. the 5-min
+        # recovery sweep re-enqueued a still-running long import), planting
+        # duplicates that then broke .scalar_one_or_none() on every later
+        # deep-scan rederive (MultipleResultsFound). Alembic 0021 adds the
+        # uq_image_provenance_image_post UNIQUE so this savepoint actually
+        # trips on collision.
         exists = self.session.execute(
             select(ImageProvenance.id).where(
                 ImageProvenance.image_record_id == record.id,
@@ -662,14 +895,20 @@ class Importer:
             )
         ).scalar_one_or_none()
         if exists is None:
-            self.session.add(
-                ImageProvenance(
-                    image_record_id=record.id,
-                    post_id=post.id,
-                    source_id=src.id,
-                    captured_metadata=sd.raw,
+            sp = self.session.begin_nested()
+            try:
+                self.session.add(
+                    ImageProvenance(
+                        image_record_id=record.id,
+                        post_id=post.id,
+                        source_id=src.id,
+                        captured_metadata=sd.raw,
+                    )
                 )
-            )
+                self.session.flush()
+                sp.commit()
+            except IntegrityError:
+                sp.rollback()
         if record.primary_post_id is None:
             record.primary_post_id = post.id
         self.session.flush()
@@ -703,6 +942,15 @@ class Importer:
         row id (so tags/series/curation stay attached). ML is cleared so
         the import task re-derives it on the new pixels.
 
+        After the file swap, the new file's adjacent gallery-dl sidecar
+        (if any) is applied via _apply_sidecar — operator-flagged
+        2026-05-25: scanning a GS download dir with smaller IR-migrated
+        images on the receiving end used to swap files but lose the GS
+        sidecar's post metadata entirely. _apply_sidecar is additive
+        (find-or-create Post / Source / ImageProvenance, NULL-only
+        primary_post_id update) so any pre-existing Post linkage
+        survives untouched.
+
         If `new_path` is provided, `source` is assumed to ALREADY be at
         that path (FC-3c attach_in_place case) — skip the copy step.
         Otherwise the file is copied via _copy_to_library."""
@@ -731,6 +979,26 @@ class Importer:
         # created_at intentionally preserved; updated_at auto-bumps.
         self.session.flush()
         self.session.commit()
+        # The phash candidate cache (used to avoid N+1 selects during
+        # archive imports) is now stale for `existing.id` — the row's
+        # phash/dimensions changed. Invalidate; the next call re-fetches.
+        self._phash_candidates = None
+
+        # Sidecar enrichment from the new (larger) file's location.
+        # _apply_sidecar resolves artist from the sidecar itself if the
+        # existing row has none, and is internally guarded against
+        # missing-or-malformed sidecars (silent return).
+        try:
+            self._apply_sidecar(existing, source, None)
+        except Exception as exc:
+            # Don't unwind the supersede DB swap if sidecar parsing
+            # blows up unexpectedly — the file replacement is the
+            # critical operation, sidecar is enrichment.
+            log.warning(
+                "sidecar enrichment failed during supersede of "
+                "image_record.id=%s from %s: %s",
+                existing.id, source, exc,
+            )
 
         for stale in (old_path, old_thumb):
             if not stale or stale == str(dest):
@@ -746,8 +1014,26 @@ class Importer:
                 pass
 
     def _transparency_pct(self, source: Path) -> float:
-        """Fraction of fully-transparent pixels in the image. 0.0 if no alpha."""
+        """Fraction of fully-transparent pixels in the image. 0.0 if no alpha.
+
+        For animated formats (multi-frame WebP / GIF / APNG), short-circuit
+        to 0.0 instead of decoding every frame. PIL's `getchannel("A")`
+        forces a full decode of all frames in an animated image, which for
+        a large animated WebP takes 5+ minutes and blows past the Celery
+        soft+hard time limits (300s/360s → SIGKILL). Operator-flagged
+        2026-05-26. Transparency analysis on a multi-frame image isn't
+        meaningful for art-curation purposes anyway — different frames
+        have different alpha — so the existing too_transparent skip rule
+        is bypassed entirely for animated content.
+        """
         with Image.open(source) as im:
+            if getattr(im, "is_animated", False):
+                log.info(
+                    "skipping transparency check for animated image %s "
+                    "(n_frames=%d) — avoids multi-frame decode timeout",
+                    source, getattr(im, "n_frames", 0),
+                )
+                return 0.0
             if im.mode not in ("RGBA", "LA") and not (
                 im.mode == "P" and "transparency" in im.info
             ):

backend/app/services/migrators/__init__.py

@@ -1,6 +0,0 @@
-"""FC-5 migration tooling.
-
-One module per concern (backup/rollback/gs/ir/overlap/ml_queue/verify).
-Each migrator returns a counts dict; the run_migration task wires
-that dict into MigrationRun.counts so the UI polling shows progress.
-"""

backend/app/services/migrators/backup.py

@@ -1,137 +0,0 @@
-"""pg_dump + tar.zst-based backup, restorable via pair of subprocess calls.
-
-Backups live under <images_root>/_backups/. Each backup is two files
-(SQL + tarball) plus a manifest JSON. Tagged backups (e.g. tag='pre_migration')
-are how rollback.py finds the most recent restorable snapshot.
-"""
-from __future__ import annotations
-
-import json
-import shutil
-import subprocess
-from datetime import UTC, datetime
-from pathlib import Path
-from typing import Any
-
-_BACKUPS_DIRNAME = "_backups"
-
-
-def _libpq_url(sa_url: str) -> str:
-    """Strip SQLAlchemy driver suffix so pg_dump/psql accept the URL.
-
-    SQLAlchemy uses URLs like `postgresql+psycopg://...` or
-    `postgresql+asyncpg://...`. libpq tools (pg_dump, psql) only know
-    the plain `postgresql://` scheme.
-    """
-    for driver in ("postgresql+psycopg", "postgresql+asyncpg", "postgresql+psycopg2"):
-        if sa_url.startswith(driver + "://"):
-            return "postgresql://" + sa_url[len(driver) + 3:]
-    return sa_url
-
-
-def _backups_dir(images_root: Path | None = None) -> Path:
-    # Overridable for tests via monkeypatch.
-    root = images_root if images_root is not None else Path("/images")
-    p = root / _BACKUPS_DIRNAME
-    p.mkdir(parents=True, exist_ok=True)
-    return p
-
-
-def _run_subprocess(cmd: list[str], **kwargs: Any):
-    # Overridable for tests via monkeypatch.
-    return subprocess.run(
-        cmd,
-        capture_output=True,
-        check=True,
-        **{k: v for k, v in kwargs.items() if not k.startswith("_")},
-    )
-
-
-def create_backup(
-    *, db_url: str, images_root: Path, tag: str = "manual",
-) -> dict:
-    """Create a backup: pg_dump SQL + tar.zst of images.
-
-    Returns a manifest dict. Writes <ts>.sql, <ts>.tar.zst, <ts>.json into
-    <images_root>/_backups/.
-    """
-    ts = datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ")
-    out_dir = _backups_dir(images_root)
-    sql_path = out_dir / f"fc_{ts}.sql"
-    tar_path = out_dir / f"fc_{ts}.tar.zst"
-    manifest_path = out_dir / f"fc_{ts}.json"
-
-    _run_subprocess(
-        ["pg_dump", "--no-owner", "--no-acl", "-f", str(sql_path), _libpq_url(db_url)],
-        _test_ts=ts,
-    )
-    _run_subprocess(
-        [
-            "tar", "--zstd", "-cf", str(tar_path),
-            "-C", str(images_root.parent), images_root.name,
-            f"--exclude={images_root.name}/_backups",
-            f"--exclude={images_root.name}/_quarantine",
-        ],
-        _test_ts=ts,
-    )
-
-    manifest = {
-        "backup_id": ts,
-        "tag": tag,
-        "created_at": datetime.now(UTC).isoformat(),
-        "sql_path": str(sql_path),
-        "tar_path": str(tar_path),
-    }
-    manifest_path.write_text(json.dumps(manifest, indent=2))
-    return manifest
-
-
-def list_backups(images_root: Path) -> list[dict]:
-    out_dir = _backups_dir(images_root)
-    items = []
-    for mf in sorted(out_dir.glob("fc_*.json"), reverse=True):
-        try:
-            items.append(json.loads(mf.read_text()))
-        except Exception:
-            continue
-    return items
-
-
-def find_latest_backup(images_root: Path, *, tag: str) -> dict | None:
-    for mf in list_backups(images_root):
-        if mf.get("tag") == tag:
-            return mf
-    return None
-
-
-def restore_backup(
-    *, manifest: dict, db_url: str, images_root: Path,
-) -> dict:
-    """Restore from a backup manifest.
-
-    1. Replay the .sql via psql.
-    2. Wipe /images/ contents (except _backups/, which holds the file we're using).
-    3. Untar the .tar.zst into /images/.
-    """
-    sql_path = Path(manifest["sql_path"])
-    tar_path = Path(manifest["tar_path"])
-
-    _run_subprocess(
-        ["psql", "-d", _libpq_url(db_url), "-f", str(sql_path)],
-    )
-
-    # Wipe everything in images_root EXCEPT _backups/ (we'd delete the backup
-    # we're restoring from!).
-    for entry in images_root.iterdir():
-        if entry.name == _BACKUPS_DIRNAME:
-            continue
-        if entry.is_dir():
-            shutil.rmtree(entry)
-        else:
-            entry.unlink()
-
-    _run_subprocess(
-        ["tar", "--zstd", "-xf", str(tar_path), "-C", str(images_root.parent)],
-    )
-
-    return {"restored_from": manifest["backup_id"]}

backend/app/services/migrators/gs_ingest.py

@@ -1,126 +0,0 @@
-"""GallerySubscriber export → FabledCurator ingest.
-
-Reads a parsed gallerysubscriber-export-v1.json dict (no DB connection
-to GS). Creates Artist (from subscriptions) + Source (nested under each
-subscription) + Credential (re-encrypted with FC's key). Idempotent on
-natural keys: Artist.slug, (artist_id, platform, url), Credential.platform.
-
-Credentials arrive plaintext in the export — GS's export script
-decrypts using GS's Fernet key in GS's own process. FC re-encrypts
-with FC's CredentialCrypto.
-"""
-from __future__ import annotations
-
-import json
-
-from sqlalchemy import select
-from sqlalchemy.ext.asyncio import AsyncSession
-
-from ...models import Artist, Credential, Source
-from ...utils.slug import slugify
-from ..credential_crypto import CredentialCrypto
-
-
-def _zero_counts() -> dict:
-    return {
-        "rows_processed": 0, "rows_inserted": 0, "rows_skipped": 0,
-        "files_copied": 0, "bytes_copied": 0, "conflicts": 0,
-    }
-
-
-async def migrate_async(
-    db: AsyncSession,
-    *,
-    data: dict,
-    fc_crypto: CredentialCrypto | None = None,
-    dry_run: bool = False,
-) -> dict:
-    """Ingest a parsed gallerysubscriber-export-v1.json dict."""
-    if data.get("source_app") != "gallerysubscriber":
-        raise ValueError("export source_app must be 'gallerysubscriber'")
-    if data.get("schema_version") != 1:
-        raise ValueError(f"unsupported schema_version: {data.get('schema_version')}")
-
-    counts = _zero_counts()
-
-    # Phase 1: subscriptions → Artist; nested sources within each.
-    for sub in data.get("subscriptions", []):
-        counts["rows_processed"] += 1
-        slug = slugify(sub["name"])
-        artist = (await db.execute(
-            select(Artist).where(Artist.slug == slug)
-        )).scalar_one_or_none()
-        if artist is None:
-            if dry_run:
-                counts["rows_inserted"] += 1
-                # Continue to nested sources, but they can't link without an artist row.
-                continue
-            notes = json.dumps(sub.get("metadata"), indent=2) if sub.get("metadata") else None
-            artist = Artist(
-                name=sub["name"], slug=slug,
-                is_subscription=True,
-                auto_check=bool(sub.get("enabled", True)),
-                notes=notes,
-            )
-            db.add(artist)
-            await db.flush()
-            counts["rows_inserted"] += 1
-        else:
-            counts["rows_skipped"] += 1
-
-        # Nested sources under this subscription.
-        for src in sub.get("sources", []):
-            counts["rows_processed"] += 1
-            existing = (await db.execute(
-                select(Source).where(
-                    Source.artist_id == artist.id,
-                    Source.platform == src["platform"],
-                    Source.url == src["url"],
-                )
-            )).scalar_one_or_none()
-            if existing is not None:
-                counts["rows_skipped"] += 1
-                continue
-            if dry_run:
-                counts["rows_inserted"] += 1
-                continue
-            db.add(Source(
-                artist_id=artist.id,
-                platform=src["platform"],
-                url=src["url"],
-                enabled=bool(src.get("enabled", True)),
-                check_interval_override=src.get("check_interval"),
-                config_overrides=src.get("metadata") or {},
-            ))
-            counts["rows_inserted"] += 1
-
-    # Phase 2: credentials.
-    for cred in data.get("credentials", []):
-        counts["rows_processed"] += 1
-        existing = (await db.execute(
-            select(Credential).where(Credential.platform == cred["platform"])
-        )).scalar_one_or_none()
-        if existing is not None:
-            counts["rows_skipped"] += 1
-            continue
-        if dry_run:
-            counts["rows_inserted"] += 1
-            continue
-        if fc_crypto is None:
-            # Without a crypto helper we can't encrypt — skip rather than
-            # store plaintext.
-            counts["rows_skipped"] += 1
-            counts["conflicts"] += 1
-            continue
-        encrypted = fc_crypto.encrypt(cred["plaintext"])
-        db.add(Credential(
-            platform=cred["platform"],
-            credential_type=cred.get("credential_type") or "cookies",
-            encrypted_blob=encrypted,
-            expires_at=cred.get("expires_at"),
-        ))
-        counts["rows_inserted"] += 1
-
-    if not dry_run:
-        await db.commit()
-    return counts

backend/app/services/migrators/ir_ingest.py

@@ -1,141 +0,0 @@
-"""ImageRepo export → FabledCurator ingest.
-
-Reads a parsed imagerepo-export-v1.json dict (no DB connection to IR).
-Creates Tag rows (skipping artist/post kinds, resolving fandom_name to
-FK). Writes the per-image-sha256 artist assignments + tag associations
-+ series page assignments to /images/_migration_state/ir_tag_manifest.json
-so tag_apply.py can join them to ImageRecord rows AFTER the operator
-runs FC's filesystem scan.
-"""
-from __future__ import annotations
-
-import json
-from pathlib import Path
-
-from sqlalchemy import select
-from sqlalchemy.ext.asyncio import AsyncSession
-
-from ...models import Tag, TagKind
-
-_SKIP_KINDS = frozenset({"artist", "post"})
-_MIGRATION_STATE_DIRNAME = "_migration_state"
-_IR_MANIFEST_FILENAME = "ir_tag_manifest.json"
-
-
-def _zero_counts() -> dict:
-    return {
-        "rows_processed": 0, "rows_inserted": 0, "rows_skipped": 0,
-        "files_copied": 0, "bytes_copied": 0, "conflicts": 0,
-    }
-
-
-def manifest_path(images_root: Path | None = None) -> Path:
-    root = images_root if images_root is not None else Path("/images")
-    p = root / _MIGRATION_STATE_DIRNAME
-    p.mkdir(parents=True, exist_ok=True)
-    return p / _IR_MANIFEST_FILENAME
-
-
-async def _resolve_fandom_id(
-    db: AsyncSession, fandom_name: str | None, dry_run: bool,
-) -> int | None:
-    """Find-or-create a fandom-kind Tag by name."""
-    if not fandom_name:
-        return None
-    existing = (await db.execute(
-        select(Tag).where(Tag.name == fandom_name, Tag.kind == "fandom")
-    )).scalar_one_or_none()
-    if existing is not None:
-        return existing.id
-    if dry_run:
-        return None
-    t = Tag(name=fandom_name, kind=TagKind.fandom)
-    db.add(t)
-    await db.flush()
-    return t.id
-
-
-async def migrate_async(
-    db: AsyncSession,
-    *,
-    data: dict,
-    images_root: Path | None = None,
-    dry_run: bool = False,
-) -> dict:
-    """Ingest a parsed imagerepo-export-v1.json dict.
-
-    Creates Tag rows + writes the IR tag manifest file. Tag-to-image
-    binding happens later in tag_apply.py (after FC's filesystem scan
-    populates image_record.sha256 → id).
-    """
-    if data.get("source_app") != "imagerepo":
-        raise ValueError("export source_app must be 'imagerepo'")
-    if data.get("schema_version") != 1:
-        raise ValueError(f"unsupported schema_version: {data.get('schema_version')}")
-
-    counts = _zero_counts()
-
-    # Phase 1: tags (skip artist + post kinds; resolve fandom_name → fandom_id).
-    # First pass: create all fandom-kind tags so they're available for FK resolution.
-    for tag in data.get("tags", []):
-        kind = tag.get("kind") or "general"
-        if kind != "fandom":
-            continue
-        counts["rows_processed"] += 1
-        existing = (await db.execute(
-            select(Tag).where(Tag.name == tag["name"], Tag.kind == "fandom")
-        )).scalar_one_or_none()
-        if existing is not None:
-            counts["rows_skipped"] += 1
-            continue
-        if dry_run:
-            counts["rows_inserted"] += 1
-            continue
-        db.add(Tag(name=tag["name"], kind=TagKind.fandom))
-        counts["rows_inserted"] += 1
-    if not dry_run:
-        await db.flush()
-
-    # Second pass: every other kind.
-    for tag in data.get("tags", []):
-        kind_str = tag.get("kind") or "general"
-        if kind_str in _SKIP_KINDS:
-            counts["rows_skipped"] += 1
-            continue
-        if kind_str == "fandom":
-            continue  # handled above
-        counts["rows_processed"] += 1
-        try:
-            kind = TagKind(kind_str)
-        except ValueError:
-            kind = TagKind.general
-        fandom_id = await _resolve_fandom_id(db, tag.get("fandom_name"), dry_run)
-        existing = (await db.execute(
-            select(Tag).where(Tag.name == tag["name"], Tag.kind == kind)
-        )).scalar_one_or_none()
-        if existing is not None:
-            counts["rows_skipped"] += 1
-            continue
-        if dry_run:
-            counts["rows_inserted"] += 1
-            continue
-        db.add(Tag(name=tag["name"], kind=kind, fandom_id=fandom_id))
-        counts["rows_inserted"] += 1
-
-    if not dry_run:
-        await db.commit()
-
-    # Phase 2: write the per-image manifest for tag_apply.py to consume later.
-    manifest = {
-        "schema_version": 1,
-        "image_artist_assignments": data.get("image_artist_assignments", []),
-        "image_tag_associations": data.get("image_tag_associations", []),
-        "series_pages": data.get("series_pages", []),
-    }
-    counts["rows_processed"] += len(manifest["image_artist_assignments"])
-    counts["rows_processed"] += len(manifest["image_tag_associations"])
-    counts["rows_processed"] += len(manifest["series_pages"])
-    if not dry_run:
-        manifest_path(images_root).write_text(json.dumps(manifest, indent=2))
-
-    return counts

backend/app/services/migrators/ml_queue.py

@@ -1,21 +0,0 @@
-"""Queue every migrated image_record with no embedding for ML re-processing."""
-from __future__ import annotations
-
-from sqlalchemy import select
-from sqlalchemy.ext.asyncio import AsyncSession
-
-from ...models import ImageRecord
-
-
-async def queue_all_unprocessed_async(db: AsyncSession) -> int:
-    """Find every ImageRecord with siglip_embedding IS NULL, fire
-    tag_and_embed.delay(id) for each. Returns count queued.
-    """
-    from ...tasks.ml import tag_and_embed
-
-    rows = (await db.execute(
-        select(ImageRecord.id).where(ImageRecord.siglip_embedding.is_(None))
-    )).scalars().all()
-    for image_id in rows:
-        tag_and_embed.delay(image_id)
-    return len(rows)

backend/app/services/migrators/rollback.py

@@ -1,21 +0,0 @@
-"""Restore from the most recent 'pre_migration'-tagged backup."""
-from __future__ import annotations
-
-from pathlib import Path
-
-from . import backup as backup_mod
-
-
-class NoBackupFoundError(Exception):
-    """Raised when rollback() is called with no pre_migration backup on disk."""
-
-
-def rollback_to_pre_migration(*, db_url: str, images_root: Path) -> dict:
-    manifest = backup_mod.find_latest_backup(images_root, tag="pre_migration")
-    if manifest is None:
-        raise NoBackupFoundError(
-            "no pre_migration-tagged backup found under <images_root>/_backups/"
-        )
-    return backup_mod.restore_backup(
-        manifest=manifest, db_url=db_url, images_root=images_root,
-    )

backend/app/services/migrators/tag_apply.py

@@ -1,172 +0,0 @@
-"""Apply the IR tag manifest after FC's filesystem scan.
-
-Reads /images/_migration_state/ir_tag_manifest.json and joins each entry
-to an ImageRecord row by sha256 (which exists after the operator runs
-FC's filesystem scan over the mounted IR images dir).
-
-- image_artist_assignments → ImageRecord.artist_id (find_or_create Artist by slug).
-- image_tag_associations  → image_tag insert (idempotent).
-- series_pages            → series_page insert (idempotent on image_id unique).
-
-Unmatched sha256s are logged into the result's `unmatched` list so the
-Celery task can drop them into MigrationRun.metadata for the operator
-to inspect.
-"""
-from __future__ import annotations
-
-import json
-from pathlib import Path
-
-from sqlalchemy import select
-from sqlalchemy.ext.asyncio import AsyncSession
-
-from ...models import Artist, ImageRecord, SeriesPage, Tag, TagKind, image_tag
-from ...utils.slug import slugify
-from .ir_ingest import manifest_path
-
-
-def _zero_counts() -> dict:
-    return {
-        "rows_processed": 0, "rows_inserted": 0, "rows_skipped": 0,
-        "files_copied": 0, "bytes_copied": 0, "conflicts": 0,
-    }
-
-
-async def _ensure_artist_id(
-    db: AsyncSession, artist_name: str, dry_run: bool,
-) -> int | None:
-    if not artist_name or not artist_name.strip():
-        return None
-    slug = slugify(artist_name)
-    existing = (await db.execute(
-        select(Artist).where(Artist.slug == slug)
-    )).scalar_one_or_none()
-    if existing is not None:
-        return existing.id
-    if dry_run:
-        return None
-    a = Artist(name=artist_name, slug=slug, is_subscription=False)
-    db.add(a)
-    await db.flush()
-    return a.id
-
-
-async def _resolve_tag_id(
-    db: AsyncSession, tag_name: str, tag_kind: str,
-) -> int | None:
-    try:
-        kind = TagKind(tag_kind)
-    except ValueError:
-        kind = TagKind.general
-    row = (await db.execute(
-        select(Tag.id).where(Tag.name == tag_name, Tag.kind == kind)
-    )).scalar_one_or_none()
-    return row
-
-
-async def _sha_to_image_id(db: AsyncSession, sha: str) -> int | None:
-    return (await db.execute(
-        select(ImageRecord.id).where(ImageRecord.sha256 == sha)
-    )).scalar_one_or_none()
-
-
-async def apply_async(
-    db: AsyncSession,
-    *,
-    images_root: Path | None = None,
-    dry_run: bool = False,
-) -> dict:
-    """Apply the manifest. Returns counts + an `unmatched` list of sha256s."""
-    mf_path = manifest_path(images_root)
-    if not mf_path.exists():
-        raise FileNotFoundError(f"no IR tag manifest at {mf_path}")
-
-    manifest = json.loads(mf_path.read_text())
-    counts = _zero_counts()
-    unmatched: list[dict] = []
-
-    # 1. Artist assignments.
-    for entry in manifest.get("image_artist_assignments", []):
-        counts["rows_processed"] += 1
-        img_id = await _sha_to_image_id(db, entry["sha256"])
-        if img_id is None:
-            unmatched.append({"kind": "artist", **entry})
-            counts["rows_skipped"] += 1
-            continue
-        aid = await _ensure_artist_id(db, entry["artist_name"], dry_run)
-        if aid is None:
-            counts["rows_skipped"] += 1
-            continue
-        if dry_run:
-            counts["rows_inserted"] += 1
-            continue
-        img = await db.get(ImageRecord, img_id)
-        if img is not None and img.artist_id != aid:
-            img.artist_id = aid
-            counts["rows_inserted"] += 1
-        else:
-            counts["rows_skipped"] += 1
-
-    # 2. Tag associations.
-    for entry in manifest.get("image_tag_associations", []):
-        counts["rows_processed"] += 1
-        img_id = await _sha_to_image_id(db, entry["sha256"])
-        if img_id is None:
-            unmatched.append({"kind": "tag", **entry})
-            counts["rows_skipped"] += 1
-            continue
-        tag_id = await _resolve_tag_id(
-            db, entry["tag_name"], entry.get("tag_kind") or "general",
-        )
-        if tag_id is None:
-            counts["rows_skipped"] += 1
-            continue
-        # Skip if association already exists.
-        already = (await db.execute(
-            select(image_tag.c.image_record_id).where(
-                image_tag.c.image_record_id == img_id,
-                image_tag.c.tag_id == tag_id,
-            )
-        )).first()
-        if already is not None:
-            counts["rows_skipped"] += 1
-            continue
-        if dry_run:
-            counts["rows_inserted"] += 1
-            continue
-        await db.execute(image_tag.insert().values(
-            image_record_id=img_id, tag_id=tag_id, source="manual",
-        ))
-        counts["rows_inserted"] += 1
-
-    # 3. Series pages.
-    for entry in manifest.get("series_pages", []):
-        counts["rows_processed"] += 1
-        img_id = await _sha_to_image_id(db, entry["sha256"])
-        if img_id is None:
-            unmatched.append({"kind": "series", **entry})
-            counts["rows_skipped"] += 1
-            continue
-        series_tag_id = await _resolve_tag_id(db, entry["series_tag_name"], "series")
-        if series_tag_id is None:
-            counts["rows_skipped"] += 1
-            continue
-        existing = (await db.execute(
-            select(SeriesPage).where(SeriesPage.image_id == img_id)
-        )).scalar_one_or_none()
-        if existing is not None:
-            counts["rows_skipped"] += 1
-            continue
-        if dry_run:
-            counts["rows_inserted"] += 1
-            continue
-        db.add(SeriesPage(
-            series_tag_id=series_tag_id,
-            image_id=img_id,
-            page_number=entry["page_number"],
-        ))
-        counts["rows_inserted"] += 1
-
-    if not dry_run:
-        await db.commit()
-    return {"counts": counts, "unmatched": unmatched}

backend/app/services/migrators/verify.py

@@ -1,80 +0,0 @@
-"""Post-migration verification: row counts + sha256 sampling."""
-from __future__ import annotations
-
-import hashlib
-from pathlib import Path
-
-from sqlalchemy import func, select
-from sqlalchemy.ext.asyncio import AsyncSession
-
-from ...models import Artist, Credential, ImageRecord, Source, Tag
-
-
-async def verify_async(db: AsyncSession, *, expected: dict | None = None) -> dict:
-    """Return per-check status dicts. `expected` is optional row-count
-    assertions; checks default to status='ok' when no expected provided."""
-    expected = expected or {}
-    results: dict[str, dict] = {}
-
-    checks = {
-        "artist_subscriptions": (
-            select(func.count(Artist.id)).where(Artist.is_subscription.is_(True))
-        ),
-        "source_count": select(func.count(Source.id)),
-        "credential_count": select(func.count(Credential.id)),
-        "tag_count": select(func.count(Tag.id)),
-        "image_record_imported_or_downloaded": (
-            select(func.count(ImageRecord.id))
-            .where(ImageRecord.origin.in_(["imported_filesystem", "downloaded"]))
-        ),
-    }
-    for name, stmt in checks.items():
-        actual = (await db.execute(stmt)).scalar_one()
-        exp = expected.get(name)
-        status = "ok" if exp is None or exp == actual else "mismatch"
-        results[name] = {"status": status, "actual": int(actual), "expected": exp}
-    return results
-
-
-async def verify_sha256_sample(
-    db: AsyncSession, *, sample_size: int = 20,
-) -> dict:
-    """Sample N image_records; verify file exists + sha256 matches."""
-    rows = (await db.execute(
-        select(ImageRecord.id, ImageRecord.path, ImageRecord.sha256)
-        .order_by(func.random()).limit(sample_size)
-    )).all()
-
-    matched = 0
-    mismatched = 0
-    missing = 0
-    samples: list[dict] = []
-    for img_id, path, expected_sha in rows:
-        p = Path(path)
-        # Sync stdlib filesystem ops are intentional: this verify pass runs
-        # inside a Celery task under asyncio.run; no other awaitables compete
-        # for the loop. Same pattern as download_service.py.
-        if not p.exists():  # noqa: ASYNC240
-            missing += 1
-            samples.append({"id": img_id, "path": path, "result": "missing"})
-            continue
-        h = hashlib.sha256()
-        with p.open("rb") as f:  # noqa: ASYNC230
-            for chunk in iter(lambda: f.read(65536), b""):
-                h.update(chunk)
-        if h.hexdigest() == expected_sha:
-            matched += 1
-            samples.append({"id": img_id, "result": "ok"})
-        else:
-            mismatched += 1
-            samples.append({
-                "id": img_id, "path": path, "result": "mismatch",
-                "expected_sha": expected_sha, "actual_sha": h.hexdigest(),
-            })
-    return {
-        "sample_size": len(rows),
-        "matched": matched,
-        "mismatched": mismatched,
-        "missing": missing,
-        "samples": samples,
-    }

backend/app/services/ml/embedder.py

@@ -34,10 +34,21 @@ class Embedder:
         if self._model is not None:
             return
         import torch
-        from transformers import AutoModel, AutoProcessor
+        from transformers import AutoModel, SiglipImageProcessor
 
         self._torch = torch
-        self._processor = AutoProcessor.from_pretrained(str(self._model_dir))
+        # FC's embedder only does IMAGE inference — never text. AutoProcessor
+        # loads the full processor including SiglipTokenizer, which requires
+        # the sentencepiece library at import time even if we never call it.
+        # SiglipImageProcessor loads ONLY preprocessor_config.json (image
+        # side) and skips the tokenizer config entirely. Operator hit the
+        # ImportError 2026-05-25 once the ml-worker started actually running
+        # tag_and_embed; switching to the image-only loader avoids the
+        # tokenizer dep without adding ~30 MB of unused C++ build to the
+        # lean ml-worker image.
+        self._processor = SiglipImageProcessor.from_pretrained(
+            str(self._model_dir)
+        )
         self._model = AutoModel.from_pretrained(str(self._model_dir))
         self._model.eval()
 

backend/app/services/ml/tagger.py

@@ -4,12 +4,14 @@ CPU-only, single-image at a time. Loaded lazily inside the ml-worker
 process; NOT thread-safe — the ml queue worker must run --concurrency=1
 (set by the FC-1 entrypoint).
 
-Camie's selected_tags.csv columns: tag_id,name,category,count
-where category is a string: general|character|copyright|artist|meta|rating|year
-(unlike WD14's integer Danbooru category ids).
+v2 layout reference: HuggingFace Camais03/camie-tagger-v2 root has
+camie-tagger-v2.onnx (789 MB) + camie-tagger-v2-metadata.json (7.77 MB)
++ config.json. Tags ship as nested JSON, not CSV. Preprocessing and
+output handling follow the published onnx_inference.py reference:
+ImageNet normalize, NCHW layout, sigmoid on refined logits (output[1]).
 """
 
-import csv
+import json
 import os
 from dataclasses import dataclass
 from pathlib import Path
@@ -28,6 +30,8 @@ ImageFile.LOAD_TRUNCATED_IMAGES = True
 
 MODEL_NAME = os.environ.get("CAMIE_MODEL_NAME", "camie-tagger-v2")
 _MODEL_DIR = Path(os.environ.get("ML_MODEL_DIR", "/models")) / "camie"
+_MODEL_FILE = f"{MODEL_NAME}.onnx"
+_METADATA_FILE = f"{MODEL_NAME}-metadata.json"
 
 # Below this confidence, predictions aren't stored (keeps the JSON compact).
 STORE_FLOOR = float(os.environ.get("TAGGER_STORE_FLOOR", "0.05"))
@@ -39,6 +43,12 @@ STORE_FLOOR = float(os.environ.get("TAGGER_STORE_FLOOR", "0.05"))
 # stored at STORE_FLOOR but artist never surfaces.
 SURFACED_CATEGORIES = {"character", "copyright", "general"}
 
+# ImageNet preprocessing constants (per Camie v2 onnx_inference.py).
+_IMAGENET_MEAN = np.array([0.485, 0.456, 0.406], dtype=np.float32)
+_IMAGENET_STD = np.array([0.229, 0.224, 0.225], dtype=np.float32)
+# Square-pad color ≈ ImageNet mean × 255 (matches reference inference).
+_PAD_COLOR = (124, 116, 104)
+
 
 @dataclass(frozen=True)
 class TagPrediction:
@@ -51,34 +61,48 @@ class Tagger:
     def __init__(self, model_dir: Path | None = None):
         self._model_dir = model_dir or _MODEL_DIR
         self._session = None  # onnxruntime.InferenceSession once load()ed
-        self._tag_meta: list[dict] | None = None
+        self._tag_names: list[str] | None = None
+        self._tag_categories: list[str] | None = None
         self._input_name: str | None = None
-        self._output_name: str | None = None
-        self._input_size: int = 448
+        self._input_size: int = 512
 
     def load(self) -> None:
         if self._session is not None:
             return
-        model_path = self._model_dir / "model.onnx"
-        tags_path = self._model_dir / "selected_tags.csv"
+        model_path = self._model_dir / _MODEL_FILE
+        meta_path = self._model_dir / _METADATA_FILE
         if not model_path.is_file():
             raise RuntimeError(
-                f"Camie model.onnx missing at {model_path}. "
+                f"Camie {_MODEL_FILE} missing at {model_path}. "
                 f"Populate /models via the ml-worker downloader."
             )
-        if not tags_path.is_file():
+        if not meta_path.is_file():
             raise RuntimeError(
-                f"Camie selected_tags.csv missing at {tags_path}. "
+                f"Camie {_METADATA_FILE} missing at {meta_path}. "
                 f"Populate /models via the ml-worker downloader."
             )
 
-        tag_meta: list[dict] = []
-        with open(tags_path, newline="") as f:
-            reader = csv.DictReader(f)
-            for row in reader:
-                tag_meta.append(
-                    {"name": row["name"], "category": row["category"]}
-                )
+        with open(meta_path) as f:
+            metadata = json.load(f)
+
+        # Per Camie v2 onnx_inference.py: idx_to_tag is keyed by str(idx);
+        # tag_to_category maps tag_name -> category. Project to two parallel
+        # lists indexed by output position for O(1) lookup in the hot path.
+        ds = metadata["dataset_info"]
+        idx_to_tag = ds["tag_mapping"]["idx_to_tag"]
+        tag_to_category = ds["tag_mapping"]["tag_to_category"]
+        total = ds["total_tags"]
+        names: list[str] = []
+        cats: list[str] = []
+        for i in range(total):
+            name = idx_to_tag.get(str(i), f"unknown-{i}")
+            names.append(name)
+            cats.append(tag_to_category.get(name, "general"))
+
+        # Input size from metadata; fall back to 512 (the v2 default).
+        self._input_size = int(
+            metadata.get("model_info", {}).get("img_size", 512)
+        )
 
         # Lazy import — kept after the file-existence checks so the
         # missing-model RuntimeError still fires first in environments
@@ -89,51 +113,65 @@ class Tagger:
             str(model_path), providers=["CPUExecutionProvider"]
         )
         self._input_name = session.get_inputs()[0].name
-        self._output_name = session.get_outputs()[0].name
-        input_shape = session.get_inputs()[0].shape
-        for dim in input_shape:
-            if isinstance(dim, int) and dim > 1:
-                self._input_size = dim
-                break
         # Assign sentinels last so a partial load isn't observable.
-        self._tag_meta = tag_meta
+        self._tag_names = names
+        self._tag_categories = cats
         self._session = session
 
     def _preprocess(self, image_path: Path) -> np.ndarray:
         img = Image.open(image_path)
-        # Camie handles RGBA natively but we still composite onto white so
-        # transparency doesn't bias the model (same as IR's WD14 path).
-        if img.mode != "RGBA":
-            img = img.convert("RGBA")
-        bg = Image.new("RGBA", img.size, (255, 255, 255, 255))
-        bg.paste(img, mask=img.split()[3])
-        img = bg.convert("RGB")
+        # Composite RGBA onto neutral so transparency doesn't bias the model.
+        if img.mode == "RGBA":
+            bg = Image.new("RGBA", img.size, (255, 255, 255, 255))
+            bg.paste(img, mask=img.split()[3])
+            img = bg.convert("RGB")
+        elif img.mode != "RGB":
+            img = img.convert("RGB")
 
+        # Pad to square with ImageNet-mean color, then bicubic resize.
         w, h = img.size
         side = max(w, h)
-        square = Image.new("RGB", (side, side), (255, 255, 255))
+        square = Image.new("RGB", (side, side), _PAD_COLOR)
         square.paste(img, ((side - w) // 2, (side - h) // 2))
         square = square.resize(
             (self._input_size, self._input_size), Image.BICUBIC
         )
-        arr = np.array(square, dtype=np.float32)
-        return arr[np.newaxis, :, :, :]  # NHWC
+
+        arr = np.array(square, dtype=np.float32) / 255.0   # HWC, [0,1]
+        arr = (arr - _IMAGENET_MEAN) / _IMAGENET_STD        # ImageNet normalize
+        arr = arr.transpose(2, 0, 1)                        # HWC -> CHW
+        return arr[np.newaxis, :, :, :]                     # NCHW
 
     def infer(self, image_path: Path) -> dict[str, TagPrediction]:
-        """Run Camie on one image. Returns {name: TagPrediction}, only
-        entries with confidence >= STORE_FLOOR (across all categories —
-        the suggestion service does category filtering later)."""
+        """Run Camie v2 on one image. Returns {name: TagPrediction} with
+        confidence >= STORE_FLOOR (across all categories — the suggestion
+        service does category filtering later).
+
+        v2 emits multiple outputs; we use the refined predictions
+        (output[1] per onnx_inference.py). Sigmoid is applied to raw
+        logits to produce [0,1] confidence scores.
+        """
         self.load()
         x = self._preprocess(image_path)
-        out = self._session.run([self._output_name], {self._input_name: x})[0][0]
+        outputs = self._session.run(None, {self._input_name: x})
+        # Refined predictions if present (v2 emits initial + refined),
+        # fall back to initial for single-output forks.
+        logits = outputs[1] if len(outputs) > 1 else outputs[0]
+        # Squeeze batch dim, apply sigmoid.
+        probs = 1.0 / (1.0 + np.exp(-logits[0]))
         results: dict[str, TagPrediction] = {}
-        for idx, score in enumerate(out):
+        names = self._tag_names
+        cats = self._tag_categories
+        for idx, score in enumerate(probs):
             conf = float(score)
             if conf < STORE_FLOOR:
                 continue
-            meta = self._tag_meta[idx]
-            results[meta["name"]] = TagPrediction(
-                name=meta["name"], category=meta["category"], confidence=conf
+            if idx >= len(names):
+                # Output longer than metadata declared — shouldn't happen but
+                # don't crash the import pipeline if v2 metadata desynchronizes.
+                continue
+            results[names[idx]] = TagPrediction(
+                name=names[idx], category=cats[idx], confidence=conf
             )
         return results
 

backend/app/services/platforms.py

@@ -1,140 +0,0 @@
-"""FC-3b platforms registry — the single source of truth for what
-FabledCurator supports.
-
-Lifted from GallerySubscriber's
-~/Nextcloud/Projects/GallerySubscriber/backend/app/api/platforms.py
-and ~/.../extension/lib/platforms.js. Six platforms; auth_type and
-URL patterns match GS exactly so the existing browser extension
-hits FC unmodified.
-"""
-
-from dataclasses import dataclass
-from typing import Literal
-
-
-@dataclass(frozen=True)
-class PlatformInfo:
-    key: str
-    name: str
-    description: str
-    auth_type: Literal["cookies", "token"]
-    requires_auth: bool
-    url_pattern: str
-    url_examples: list[str]
-    default_config: dict
-    notes: str | None = None
-
-
-# Common defaults used across most platforms; embedded per-platform
-# below so per-platform overrides remain explicit.
-_DEFAULTS = {
-    "sleep": 3.0,
-    "sleep_request": 1.5,
-    "skip_existing": True,
-    "save_metadata": True,
-    "timeout": 3600,
-}
-
-
-PLATFORMS: dict[str, PlatformInfo] = {
-    "patreon": PlatformInfo(
-        key="patreon",
-        name="Patreon",
-        description="Download posts from Patreon creators",
-        auth_type="cookies",
-        requires_auth=True,
-        url_pattern=r"^https?://(www\.)?patreon\.com/",
-        url_examples=[
-            "https://www.patreon.com/example_artist",
-            "https://www.patreon.com/user?u=12345678",
-        ],
-        default_config={**_DEFAULTS, "content_types": ["images", "attachments"]},
-    ),
-    "subscribestar": PlatformInfo(
-        key="subscribestar",
-        name="SubscribeStar",
-        description="Download posts from SubscribeStar creators",
-        auth_type="cookies",
-        requires_auth=True,
-        url_pattern=r"^https?://(www\.)?subscribestar\.(com|adult)/",
-        url_examples=[
-            "https://subscribestar.adult/example_artist",
-            "https://www.subscribestar.com/example_artist",
-        ],
-        default_config={**_DEFAULTS, "content_types": ["all"]},
-    ),
-    "hentaifoundry": PlatformInfo(
-        key="hentaifoundry",
-        name="Hentai Foundry",
-        description="Download artwork from Hentai Foundry artists",
-        auth_type="cookies",
-        requires_auth=False,
-        url_pattern=r"^https?://(www\.)?hentai-foundry\.com/",
-        url_examples=[
-            "https://www.hentai-foundry.com/user/example_artist",
-            "https://www.hentai-foundry.com/pictures/user/example_artist",
-        ],
-        default_config={**_DEFAULTS, "content_types": ["pictures"]},
-    ),
-    "discord": PlatformInfo(
-        key="discord",
-        name="Discord",
-        description="Download attachments from Discord channels",
-        auth_type="token",
-        requires_auth=True,
-        url_pattern=r"^https?://(www\.)?discord\.com/channels/",
-        url_examples=["https://discord.com/channels/123456789/987654321"],
-        default_config={**_DEFAULTS, "content_types": ["all"]},
-        notes="Requires Discord user token (not bot token).",
-    ),
-    "pixiv": PlatformInfo(
-        key="pixiv",
-        name="Pixiv",
-        description="Download artwork from Pixiv artists",
-        auth_type="token",
-        requires_auth=True,
-        url_pattern=r"^https?://(www\.)?pixiv\.net/",
-        url_examples=[
-            "https://www.pixiv.net/users/12345678",
-            "https://www.pixiv.net/en/users/12345678",
-        ],
-        default_config={**_DEFAULTS, "content_types": ["all"]},
-        notes="Requires OAuth refresh token. Run `gallery-dl oauth:pixiv` to obtain one.",
-    ),
-    "deviantart": PlatformInfo(
-        key="deviantart",
-        name="DeviantArt",
-        description="Download artwork from DeviantArt artists",
-        auth_type="cookies",
-        requires_auth=False,
-        url_pattern=r"^https?://(www\.)?deviantart\.com/",
-        url_examples=[
-            "https://www.deviantart.com/example-artist",
-            "https://www.deviantart.com/example-artist/gallery",
-        ],
-        default_config={**_DEFAULTS, "content_types": ["gallery"]},
-    ),
-}
-
-
-def known_platform_keys() -> frozenset[str]:
-    return frozenset(PLATFORMS.keys())
-
-
-def auth_type_for(platform: str) -> str | None:
-    info = PLATFORMS.get(platform)
-    return info.auth_type if info else None
-
-
-def to_dict(info: PlatformInfo) -> dict:
-    return {
-        "key": info.key,
-        "name": info.name,
-        "description": info.description,
-        "auth_type": info.auth_type,
-        "requires_auth": info.requires_auth,
-        "url_pattern": info.url_pattern,
-        "url_examples": info.url_examples,
-        "default_config": info.default_config,
-        "notes": info.notes,
-    }

backend/app/services/platforms/__init__.py

@@ -0,0 +1,95 @@
+"""FC-3b platforms registry — single source of truth for what
+FabledCurator supports + where each platform's quirks live.
+
+Adding a new platform: drop a new module `<platform>.py` next to this
+one, declare an `INFO = PlatformInfo(...)`, add the import + entry in
+PLATFORMS below. Sidecar parsing, cookie materialization, and
+`/api/platforms` pick it up automatically.
+
+Lifted from GallerySubscriber's
+~/Nextcloud/Projects/GallerySubscriber/backend/app/api/platforms.py
+and ~/.../extension/lib/platforms.js. Six platforms; auth_type and
+URL patterns match GS exactly so the existing browser extension
+hits FC unmodified.
+"""
+
+from .base import (
+    DEFAULT_DESCRIPTION_KEYS,
+    DEFAULT_EXTERNAL_POST_ID_KEYS,
+    PlatformInfo,
+)
+from .deviantart import INFO as _DEVIANTART
+from .discord import INFO as _DISCORD
+from .hentaifoundry import INFO as _HENTAIFOUNDRY
+from .patreon import INFO as _PATREON
+from .pixiv import INFO as _PIXIV
+from .subscribestar import INFO as _SUBSCRIBESTAR
+
+PLATFORMS: dict[str, PlatformInfo] = {
+    info.key: info
+    for info in (
+        _PATREON,
+        _SUBSCRIBESTAR,
+        _HENTAIFOUNDRY,
+        _DISCORD,
+        _PIXIV,
+        _DEVIANTART,
+    )
+}
+
+
+def known_platform_keys() -> frozenset[str]:
+    return frozenset(PLATFORMS.keys())
+
+
+def auth_type_for(platform: str) -> str | None:
+    info = PLATFORMS.get(platform)
+    return info.auth_type if info else None
+
+
+def to_dict(info: PlatformInfo) -> dict:
+    """Serialize a PlatformInfo to a JSON-safe dict for /api/platforms.
+
+    Behavioral fields (callables, sidecar-chain overrides) are
+    intentionally omitted — they aren't useful to API consumers.
+    """
+    return {
+        "key": info.key,
+        "name": info.name,
+        "description": info.description,
+        "auth_type": info.auth_type,
+        "requires_auth": info.requires_auth,
+        "url_pattern": info.url_pattern,
+        "url_examples": info.url_examples,
+        "default_config": info.default_config,
+        "notes": info.notes,
+    }
+
+
+def external_post_id_keys_for(platform: str | None) -> tuple[str, ...]:
+    """Resolve the external_post_id lookup chain for a given platform,
+    falling back to the module default when the platform isn't
+    registered or hasn't overridden the chain."""
+    info = PLATFORMS.get(platform) if platform else None
+    if info is not None and info.external_post_id_keys is not None:
+        return info.external_post_id_keys
+    return DEFAULT_EXTERNAL_POST_ID_KEYS
+
+
+def description_keys_for(platform: str | None) -> tuple[str, ...]:
+    """Resolve the description body lookup chain for a given platform."""
+    info = PLATFORMS.get(platform) if platform else None
+    if info is not None and info.description_keys is not None:
+        return info.description_keys
+    return DEFAULT_DESCRIPTION_KEYS
+
+
+__all__ = [
+    "PLATFORMS",
+    "PlatformInfo",
+    "auth_type_for",
+    "description_keys_for",
+    "external_post_id_keys_for",
+    "known_platform_keys",
+    "to_dict",
+]

backend/app/services/platforms/base.py

@@ -0,0 +1,105 @@
+"""PlatformInfo dataclass + shared defaults + small helpers.
+
+Per-platform modules import from here, register their PlatformInfo via
+INFO, optionally attaching `derive_post_url` and/or `augment_cookies`
+callables for behavior that diverges from gallery-dl's mainline shape
+(Patreon).
+
+Adding a new platform: drop a new module under `services/platforms/`,
+declare an INFO, and add it to the import list in
+`services/platforms/__init__.py`. Sidecar parsing, cookie
+materialization, and the /api/platforms response pick it up
+automatically.
+"""
+
+from collections.abc import Callable
+from dataclasses import dataclass
+from typing import Literal
+
+# Sidecar parsing defaults. Per-platform PlatformInfo entries can
+# override these by setting `external_post_id_keys=` /
+# `description_keys=`. Most don't need to — the defaults already cover
+# every platform FC supports.
+#
+# external_post_id chain: `post_id` MUST come before `id` because
+# SubscribeStar gallery-dl puts the per-attachment id in `id` and the
+# actual post id in `post_id`; picking `id` first fragments
+# multi-image SubscribeStar posts into N Post rows. Patreon/Pixiv have
+# no `post_id` so `id` still wins for them; HF uses `index`, Discord
+# uses `message_id` — all reached via the remaining chain entries.
+# (Banked 2026-05-27 during the sidecar audit.)
+DEFAULT_EXTERNAL_POST_ID_KEYS: tuple[str, ...] = (
+    "post_id", "id", "index", "message_id",
+)
+
+# Description body chain: Discord's gallery-dl extractor uses `message`
+# (no `content`); appended to the chain so Discord posts surface body
+# text.
+DEFAULT_DESCRIPTION_KEYS: tuple[str, ...] = (
+    "content", "description", "caption", "message",
+)
+
+
+@dataclass(frozen=True)
+class PlatformInfo:
+    # --- Identity / metadata ---
+    key: str
+    name: str
+    description: str
+    auth_type: Literal["cookies", "token"]
+    requires_auth: bool
+    url_pattern: str
+    url_examples: list[str]
+    default_config: dict
+    notes: str | None = None
+
+    # --- Sidecar parsing overrides ---
+    # Each is None to mean "use the module default above"; a platform
+    # only sets one of these when its sidecar shape genuinely differs.
+    external_post_id_keys: tuple[str, ...] | None = None
+    description_keys: tuple[str, ...] | None = None
+
+    # --- Behavioral hooks ---
+    # Synthesize a post permalink from sidecar data. Required when
+    # gallery-dl's `url` field is the file/CDN URL rather than the post
+    # permalink (subscribestar/pixiv/hf/discord). None = trust the bare
+    # `url` field (patreon, deviantart).
+    derive_post_url: Callable[[dict], str | None] | None = None
+
+    # Post-process the materialized cookies.txt for gallery-dl. Used by
+    # platforms whose server gates or extractor quirks need synthetic
+    # cookies the extension can't capture (subscribestar age cookie, HF
+    # host-only PHPSESSID duplicate). None = no-op.
+    augment_cookies: Callable[[str], str] | None = None
+
+
+def str_id_value(v) -> str | None:
+    """Coerce a JSON scalar id into a non-empty string, rejecting bool
+    (Python's bool is an int subclass so `isinstance(True, int)` is
+    True; without this guard a sidecar with `"id": true` would produce
+    external_post_id="True")."""
+    if isinstance(v, bool):
+        return None
+    if isinstance(v, (str, int)) and str(v).strip():
+        return str(v).strip()
+    return None
+
+
+def str_field(v) -> str | None:
+    """Same idea as str_id_value but for plain string fields (no int
+    coercion)."""
+    if isinstance(v, str) and v.strip():
+        return v.strip()
+    return None
+
+
+# Shared gallery-dl invocation defaults. Embedded in each platform's
+# default_config (with platform-specific overrides) so per-platform
+# choices stay explicit.
+GD_DEFAULTS = {
+    "sleep": 3.0,
+    "sleep_request": 1.5,
+    "skip_existing": True,
+    "save_metadata": True,
+    "timeout": 3600,
+}

backend/app/services/platforms/deviantart.py

@@ -0,0 +1,23 @@
+"""DeviantArt — no exercised quirks yet.
+
+No operator-owned DeviantArt archive existed at the 2026-05-27 sidecar
+audit, so we don't know yet whether DA's gallery-dl sidecars are
+well-behaved or have their own quirks. When DA gets exercised for the
+first time, add `derive_post_url` / `augment_cookies` here as needed.
+"""
+
+from .base import GD_DEFAULTS, PlatformInfo
+
+INFO = PlatformInfo(
+    key="deviantart",
+    name="DeviantArt",
+    description="Download artwork from DeviantArt artists",
+    auth_type="cookies",
+    requires_auth=False,
+    url_pattern=r"^https?://(www\.)?deviantart\.com/",
+    url_examples=[
+        "https://www.deviantart.com/example-artist",
+        "https://www.deviantart.com/example-artist/gallery",
+    ],
+    default_config={**GD_DEFAULTS, "content_types": ["gallery"]},
+)

backend/app/services/platforms/discord.py

@@ -0,0 +1,38 @@
+"""Discord — one quirk + one already-default.
+
+post_url: gallery-dl's `url` is the CDN attachment URL. The "permalink"
+for a Discord message uses the (server, channel, message) triple via
+`discord.com/channels/<server>/<channel>/<message>`. Note that
+permalinks are only resolvable for users in the same server — public
+access doesn't work — but the URL is still useful to the operator
+in-app.
+
+Description body is in `message` not `content`. That's already covered
+by the default description chain in base.py (DEFAULT_DESCRIPTION_KEYS
+ends with `message`). No description_keys override needed.
+"""
+
+from .base import GD_DEFAULTS, PlatformInfo, str_id_value
+
+
+def derive_post_url(data: dict) -> str | None:
+    sid = str_id_value(data.get("server_id"))
+    cid = str_id_value(data.get("channel_id"))
+    mid = str_id_value(data.get("message_id"))
+    if sid and cid and mid:
+        return f"https://discord.com/channels/{sid}/{cid}/{mid}"
+    return None
+
+
+INFO = PlatformInfo(
+    key="discord",
+    name="Discord",
+    description="Download attachments from Discord channels",
+    auth_type="token",
+    requires_auth=True,
+    url_pattern=r"^https?://(www\.)?discord\.com/channels/",
+    url_examples=["https://discord.com/channels/123456789/987654321"],
+    default_config={**GD_DEFAULTS, "content_types": ["all"]},
+    notes="Requires Discord user token (not bot token).",
+    derive_post_url=derive_post_url,
+)

backend/app/services/platforms/hentaifoundry.py

@@ -0,0 +1,83 @@
+"""HentaiFoundry — two quirks colocated.
+
+1. post_url: HF sidecars omit `url` entirely; `src` is the image URL.
+   Synthesize the permalink from `user` + `index`
+   (/pictures/user/<user>/<index>).
+
+2. augment_cookies: gallery-dl's HF extractor checks
+   `self.cookies.get("PHPSESSID", domain="www.hentai-foundry.com")` with
+   `requests`' EXACT domain matching. The extension's pre-v1.0.5
+   `cookies.js` aggressively rewrote every captured cookie to the
+   leading-dot subdomain-wide form (`.hentai-foundry.com`), which fails
+   the exact lookup even though the cookie IS sent on actual HTTP
+   requests (RFC 6265 subdomain matching). The extractor falls into
+   an unauthenticated `?enterAgree=1` HEAD that 401s. Inject host-only
+   duplicates of PHPSESSID + YII_CSRF_TOKEN so the lookup succeeds.
+"""
+
+from .base import GD_DEFAULTS, PlatformInfo, str_field, str_id_value
+
+_HOST_ONLY_NAMES = ("PHPSESSID", "YII_CSRF_TOKEN")
+
+
+def derive_post_url(data: dict) -> str | None:
+    user = str_field(data.get("user")) or str_field(data.get("artist"))
+    idx = str_id_value(data.get("index"))
+    if user and idx:
+        return f"https://www.hentai-foundry.com/pictures/user/{user}/{idx}"
+    return None
+
+
+def augment_cookies(netscape: str) -> str:
+    body = netscape.rstrip("\n")
+    if not body:
+        return netscape
+    lines = body.split("\n")
+    existing_host_only: set[str] = set()
+    by_name: dict[str, list[str]] = {}
+    for raw in lines:
+        if not raw or raw.startswith("#"):
+            continue
+        parts = raw.split("\t")
+        if len(parts) < 7:
+            continue
+        domain, _flag, _path, _secure, _exp, name, _value = parts[:7]
+        if name not in _HOST_ONLY_NAMES:
+            continue
+        if domain == "www.hentai-foundry.com":
+            existing_host_only.add(name)
+        elif domain in (".hentai-foundry.com", "hentai-foundry.com"):
+            by_name.setdefault(name, []).append(raw)
+
+    appended: list[str] = []
+    for name in _HOST_ONLY_NAMES:
+        if name in existing_host_only or name not in by_name:
+            continue
+        # Duplicate the first subdomain-wide line as host-only on
+        # www.hentai-foundry.com. Same value + expiry; flag=FALSE marks
+        # the entry host-only in netscape format.
+        parts = by_name[name][0].split("\t")
+        parts[0] = "www.hentai-foundry.com"
+        parts[1] = "FALSE"
+        appended.append("\t".join(parts[:7]))
+
+    if not appended:
+        return netscape
+    return body + "\n" + "\n".join(appended) + "\n"
+
+
+INFO = PlatformInfo(
+    key="hentaifoundry",
+    name="Hentai Foundry",
+    description="Download artwork from Hentai Foundry artists",
+    auth_type="cookies",
+    requires_auth=False,
+    url_pattern=r"^https?://(www\.)?hentai-foundry\.com/",
+    url_examples=[
+        "https://www.hentai-foundry.com/user/example_artist",
+        "https://www.hentai-foundry.com/pictures/user/example_artist",
+    ],
+    default_config={**GD_DEFAULTS, "content_types": ["pictures"]},
+    derive_post_url=derive_post_url,
+    augment_cookies=augment_cookies,
+)

backend/app/services/platforms/patreon.py

@@ -0,0 +1,23 @@
+"""Patreon — no quirks. The reference platform.
+
+Patreon's gallery-dl sidecars are the well-behaved baseline: `url` is a
+real permalink, `id` is the post id, `title` and `content` are
+populated. No cookie quirks (session cookies are domain-wide). No
+derivation overrides.
+"""
+
+from .base import GD_DEFAULTS, PlatformInfo
+
+INFO = PlatformInfo(
+    key="patreon",
+    name="Patreon",
+    description="Download posts from Patreon creators",
+    auth_type="cookies",
+    requires_auth=True,
+    url_pattern=r"^https?://(www\.)?patreon\.com/",
+    url_examples=[
+        "https://www.patreon.com/example_artist",
+        "https://www.patreon.com/user?u=12345678",
+    ],
+    default_config={**GD_DEFAULTS, "content_types": ["images", "attachments"]},
+)

backend/app/services/platforms/pixiv.py

@@ -0,0 +1,32 @@
+"""Pixiv — one quirk.
+
+post_url: gallery-dl's `url` is the image URL on `i.pximg.net`. The
+post permalink follows /artworks/<id>. external_post_id (= `id`) was
+already correct, so no override there.
+"""
+
+from .base import GD_DEFAULTS, PlatformInfo, str_id_value
+
+
+def derive_post_url(data: dict) -> str | None:
+    pid = str_id_value(data.get("id"))
+    if pid:
+        return f"https://www.pixiv.net/artworks/{pid}"
+    return None
+
+
+INFO = PlatformInfo(
+    key="pixiv",
+    name="Pixiv",
+    description="Download artwork from Pixiv artists",
+    auth_type="token",
+    requires_auth=True,
+    url_pattern=r"^https?://(www\.)?pixiv\.net/",
+    url_examples=[
+        "https://www.pixiv.net/users/12345678",
+        "https://www.pixiv.net/en/users/12345678",
+    ],
+    default_config={**GD_DEFAULTS, "content_types": ["all"]},
+    notes="Requires OAuth refresh token. Run `gallery-dl oauth:pixiv` to obtain one.",
+    derive_post_url=derive_post_url,
+)

backend/app/services/platforms/subscribestar.py

@@ -0,0 +1,62 @@
+"""SubscribeStar — three quirks colocated.
+
+1. external_post_id: gallery-dl puts the per-attachment id in `id`
+   (e.g. 711509) and the actual post id in `post_id` (e.g. 360360).
+   The default chain in base.py already prefers `post_id`; this module
+   doesn't need to override it but the comment lives here too so a
+   future reader knows the chain's order was driven by this platform.
+
+2. post_url: gallery-dl's `url` is the file CDN URL
+   (`/post_uploads?payload=...`). Synthesize the post permalink from
+   `post_id`.
+
+3. augment_cookies: the server gates artist pages behind a
+   `_personalization_id` age-confirmation cookie that the user can't
+   easily refresh — SubscribeStar's frontend JS uses localStorage to
+   suppress the age popup once dismissed. gallery-dl's own login flow
+   sidesteps this by setting `18_plus_agreement_generic=true` on
+   `.subscribestar.adult`; we mirror that for cookies captured via the
+   extension.
+"""
+
+from .base import GD_DEFAULTS, PlatformInfo, str_id_value
+
+
+def derive_post_url(data: dict) -> str | None:
+    pid = str_id_value(data.get("post_id"))
+    if pid:
+        return f"https://www.subscribestar.com/posts/{pid}"
+    return None
+
+
+def augment_cookies(netscape: str) -> str:
+    if "18_plus_agreement_generic" in netscape:
+        return netscape
+    # Far-future expiry — gallery-dl's own login flow sets this with no
+    # explicit expiry; the server only checks presence/value.
+    expiry = 4102444800  # 2100-01-01 UTC
+    line = "\t".join([
+        ".subscribestar.adult", "TRUE", "/", "TRUE",
+        str(expiry), "18_plus_agreement_generic", "true",
+    ])
+    body = netscape.rstrip("\n")
+    if not body:
+        body = "# Netscape HTTP Cookie File"
+    return body + "\n" + line + "\n"
+
+
+INFO = PlatformInfo(
+    key="subscribestar",
+    name="SubscribeStar",
+    description="Download posts from SubscribeStar creators",
+    auth_type="cookies",
+    requires_auth=True,
+    url_pattern=r"^https?://(www\.)?subscribestar\.(com|adult)/",
+    url_examples=[
+        "https://subscribestar.adult/example_artist",
+        "https://www.subscribestar.com/example_artist",
+    ],
+    default_config={**GD_DEFAULTS, "content_types": ["all"]},
+    derive_post_url=derive_post_url,
+    augment_cookies=augment_cookies,
+)

backend/app/services/post_feed_service.py

@@ -56,9 +56,17 @@ class PostFeedService:
         artist_id: int | None = None,
         platform: str | None = None,
         limit: int = 24,
+        direction: str = "older",
     ) -> dict:
+        """Paginate the feed from `cursor`. direction='older' walks back in
+        time (default, infinite-scroll down); direction='newer' walks forward
+        (scroll up in an anchored view). Items are always returned in feed
+        (descending) order; `next_cursor` points to the far edge in the
+        requested direction (null when exhausted)."""
         if limit < 1 or limit > 100:
             raise ValueError("limit must be between 1 and 100")
+        if direction not in ("older", "newer"):
+            raise ValueError("direction must be 'older' or 'newer'")
 
         sort_key = _sort_key()
         stmt = (
@@ -72,22 +80,37 @@ class PostFeedService:
             stmt = stmt.where(Source.platform == platform)
         if cursor:
             cur_ts, cur_id = decode_cursor(cursor)
-            stmt = stmt.where(
-                or_(
+            if direction == "older":
+                stmt = stmt.where(or_(
                     sort_key < cur_ts,
                     and_(sort_key == cur_ts, Post.id < cur_id),
-                )
-            )
+                ))
+            else:
+                stmt = stmt.where(or_(
+                    sort_key > cur_ts,
+                    and_(sort_key == cur_ts, Post.id > cur_id),
+                ))
 
-        stmt = stmt.order_by(sort_key.desc(), Post.id.desc()).limit(limit + 1)
+        if direction == "older":
+            stmt = stmt.order_by(sort_key.desc(), Post.id.desc())
+        else:
+            stmt = stmt.order_by(sort_key.asc(), Post.id.asc())
+        stmt = stmt.limit(limit + 1)
         rows = (await self.session.execute(stmt)).all()
 
+        has_more = len(rows) > limit
+        rows = rows[:limit]
+        if direction == "newer":
+            # Fetched ascending (closest-newer first); flip to feed order.
+            rows = list(reversed(rows))
+
         next_cursor: str | None = None
-        if len(rows) > limit:
-            last_post, _, _ = rows[limit - 1]
-            last_key = last_post.post_date or last_post.downloaded_at
-            next_cursor = encode_cursor(last_key, last_post.id)
-            rows = rows[:limit]
+        if has_more and rows:
+            # Far edge in the travel direction: oldest row going older,
+            # newest row going newer (rows is descending for display).
+            edge_post = rows[-1][0] if direction == "older" else rows[0][0]
+            edge_key = edge_post.post_date or edge_post.downloaded_at
+            next_cursor = encode_cursor(edge_key, edge_post.id)
 
         post_ids = [p.id for p, _, _ in rows]
         thumbs_map = await self._thumbnails_for(post_ids)
@@ -99,6 +122,49 @@ class PostFeedService:
         ]
         return {"items": items, "next_cursor": next_cursor}
 
+    async def around(
+        self,
+        *,
+        post_id: int,
+        artist_id: int | None = None,
+        platform: str | None = None,
+        limit: int = 12,
+    ) -> dict | None:
+        """A window centered on `post_id`: up to `limit` newer posts + the
+        post + up to `limit` older posts, in feed (descending) order, with a
+        cursor for each end. Returns None if the post doesn't exist."""
+        anchor = (await self.session.execute(
+            select(Post, Artist, Source)
+            .join(Source, Post.source_id == Source.id)
+            .join(Artist, Source.artist_id == Artist.id)
+            .where(Post.id == post_id)
+        )).one_or_none()
+        if anchor is None:
+            return None
+        anchor_post, anchor_artist, anchor_source = anchor
+        anchor_key = anchor_post.post_date or anchor_post.downloaded_at
+        anchor_cursor = encode_cursor(anchor_key, anchor_post.id)
+
+        older = await self.scroll(
+            cursor=anchor_cursor, artist_id=artist_id, platform=platform,
+            limit=limit, direction="older",
+        )
+        newer = await self.scroll(
+            cursor=anchor_cursor, artist_id=artist_id, platform=platform,
+            limit=limit, direction="newer",
+        )
+        thumbs_map = await self._thumbnails_for([anchor_post.id])
+        atts_map = await self._attachments_for([anchor_post.id])
+        anchor_item = self._to_dict(
+            anchor_post, anchor_artist, anchor_source, thumbs_map, atts_map,
+        )
+        return {
+            "items": newer["items"] + [anchor_item] + older["items"],
+            "cursor_older": older["next_cursor"],
+            "cursor_newer": newer["next_cursor"],
+            "anchor_id": anchor_post.id,
+        }
+
     async def get_post(self, post_id: int) -> dict | None:
         row = (await self.session.execute(
             select(Post, Artist, Source)
@@ -109,7 +175,10 @@ class PostFeedService:
         if row is None:
             return None
         post, artist, source = row
-        thumbs_map = await self._thumbnails_for([post.id])
+        # Detail endpoint returns the FULL image list for PostModal's
+        # masonry grid — feed query still caps at THUMBNAIL_LIMIT via
+        # the default arg.
+        thumbs_map = await self._thumbnails_for([post.id], limit=None)
         atts_map = await self._attachments_for([post.id])
         item = self._to_dict(post, artist, source, thumbs_map, atts_map)
         item["description_full"] = html_to_plain(post.description)
@@ -117,21 +186,28 @@ class PostFeedService:
 
     # --- composition helpers ---------------------------------------------
 
-    async def _thumbnails_for(self, post_ids: list[int]) -> dict[int, dict]:
-        """post_id -> {"thumbs": [...up to 6], "more": int}.
+    async def _thumbnails_for(
+        self, post_ids: list[int], *, limit: int | None = THUMBNAIL_LIMIT,
+    ) -> dict[int, dict]:
+        """post_id -> {"thumbs": [...up to limit], "more": int}.
 
-        Selects THUMBNAIL_LIMIT+1 images per post via window function so we
-        can detect overflow in a single query.
+        Selects up to `limit` images per post via window function so we
+        can detect overflow in a single query. Pass `limit=None` to
+        return ALL thumbnails per post (used by `get_post` for PostModal's
+        masonry grid; the feed pass keeps the default cap so payloads
+        stay small).
         """
         if not post_ids:
             return {}
-        # Rank images within each post and fetch only the top THUMBNAIL_LIMIT+1.
+        # Rank images within each post; cap at `limit` rows per post when
+        # limit is set, return all when limit is None.
         ranked = (
             select(
                 ImageRecord.id,
                 ImageRecord.primary_post_id,
                 ImageRecord.sha256,
                 ImageRecord.mime,
+                ImageRecord.thumbnail_path,
                 func.row_number().over(
                     partition_by=ImageRecord.primary_post_id,
                     order_by=ImageRecord.id.asc(),
@@ -143,19 +219,20 @@ class PostFeedService:
             .where(ImageRecord.primary_post_id.in_(post_ids))
             .subquery()
         )
-        rows = (await self.session.execute(
-            select(
-                ranked.c.id, ranked.c.primary_post_id,
-                ranked.c.sha256, ranked.c.mime, ranked.c.total,
-            ).where(ranked.c.rn <= THUMBNAIL_LIMIT)
-        )).all()
+        stmt = select(
+            ranked.c.id, ranked.c.primary_post_id,
+            ranked.c.sha256, ranked.c.mime, ranked.c.thumbnail_path, ranked.c.total,
+        )
+        if limit is not None:
+            stmt = stmt.where(ranked.c.rn <= limit)
+        rows = (await self.session.execute(stmt)).all()
 
         out: dict[int, dict] = {pid: {"thumbs": [], "more": 0} for pid in post_ids}
-        for img_id, pid, sha, mime, total in rows:
+        for img_id, pid, sha, mime, tp, total in rows:
             entry = out.setdefault(pid, {"thumbs": [], "more": 0})
             entry["thumbs"].append({
                 "image_id": img_id,
-                "thumbnail_url": thumbnail_url(sha, mime),
+                "thumbnail_url": thumbnail_url(tp, sha, mime),
                 "mime": mime,
             })
             # `total` is constant per partition; overflow = total - THUMBNAIL_LIMIT.

backend/app/services/refetch_service.py

@@ -0,0 +1,106 @@
+"""Layer-2 one-shot re-download remediation for corrupt imported files.
+
+When an import fails on a file that came from a known, pollable
+subscription Source, deleting the bad copy and re-running the source's
+downloader can fetch a fresh, unblemished copy. This only helps when:
+
+  - the corruption is in transit / on disk (not at the source), AND
+  - the file resolves to an ENABLED Source with a real feed URL
+    (a `sidecar:<platform>:<slug>` synthetic anchor is not pollable),
+    AND
+  - we haven't already re-fetched this task once (bounded by
+    ImportTask.refetched so source-side corruption can't loop).
+
+Filesystem-only imports with no resolvable Source return 'no_source' —
+the operator's only remediation there is to replace the file on disk.
+
+Operator-requested 2026-05-28 (Layer 2).
+"""
+
+import json
+import logging
+from pathlib import Path
+
+from sqlalchemy import select
+from sqlalchemy.orm import Session
+
+from ..models import Artist, ImportTask, Source
+from ..utils.paths import derive_top_level_artist
+from ..utils.sidecar import find_sidecar, parse_sidecar
+from ..utils.slug import slugify
+
+log = logging.getLogger(__name__)
+
+
+def resolve_refetch_source(
+    session: Session, source_path: str, import_root: Path,
+) -> Source | None:
+    """Find an enabled, real-URL Source for the file's (artist, platform),
+    or None when nothing re-pollable resolves."""
+    path = Path(source_path)
+    sc = find_sidecar(path)
+    if sc is None:
+        return None
+    try:
+        data = json.loads(sc.read_text("utf-8"))
+    except (OSError, json.JSONDecodeError):
+        return None
+    if not isinstance(data, dict):
+        return None
+    sd = parse_sidecar(data)
+    if not sd.platform:
+        return None
+    artist_name = derive_top_level_artist(path, import_root)
+    if not artist_name:
+        return None
+    artist = session.execute(
+        select(Artist).where(Artist.slug == slugify(artist_name))
+    ).scalar_one_or_none()
+    if artist is None:
+        return None
+    src = session.execute(
+        select(Source)
+        .where(
+            Source.artist_id == artist.id,
+            Source.platform == sd.platform,
+            Source.enabled.is_(True),
+        )
+        .order_by(Source.id.asc())
+    ).scalars().first()
+    if src is None:
+        return None
+    if (src.url or "").startswith("sidecar:"):
+        return None  # synthetic anchor — not a pollable feed
+    return src
+
+
+def attempt_refetch(
+    session: Session, task: ImportTask, import_root: Path,
+) -> dict:
+    """Delete the corrupt file, mark the task refetched, and trigger ONE
+    source re-check. Idempotent/bounded: a task already refetched (or
+    with no resolvable Source) is a no-op. Commits."""
+    if task.refetched:
+        return {"status": "already_refetched"}
+    src = resolve_refetch_source(session, task.source_path, import_root)
+    if src is None:
+        return {"status": "no_source"}
+
+    # Remove the bad copy so gallery-dl (skip_existing) re-fetches it on
+    # the source re-check instead of skipping the still-present corrupt
+    # file.
+    try:
+        Path(task.source_path).unlink(missing_ok=True)
+    except OSError as exc:
+        log.warning("refetch unlink failed for %s: %s", task.source_path, exc)
+
+    task.refetched = True
+    session.add(task)
+    session.commit()
+
+    # Lazy import to avoid a tasks→services→tasks import cycle at module
+    # load. download_source.delay() is sync-safe in any context.
+    from ..tasks.download import download_source
+
+    download_source.delay(src.id)
+    return {"status": "refetch_queued", "source_id": src.id}

backend/app/services/scheduler_service.py

@@ -9,15 +9,33 @@ from __future__ import annotations
 from datetime import UTC, datetime, timedelta
 
 from sqlalchemy import select
+from sqlalchemy.dialects.postgresql import insert as pg_insert
 from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy.orm import selectinload
 
-from ..models import Artist, ImportSettings, Source
+from ..models import AppSetting, Artist, ImportSettings, Source
 
 MIN_INTERVAL_SECONDS = 60
 MAX_INTERVAL_SECONDS = 86400
 MAX_BACKOFF_EXPONENT = 6
 
+# AppSetting key stamped every time the Beat tick fires (see scan.py). The
+# tick runs every 60s; the UI flags the scheduler as stalled if the last
+# stamp is older than a few minutes.
+SCHEDULER_LAST_TICK_KEY = "scheduler_last_tick_at"
+
+# AppSetting key prefix for per-platform rate-limit cooldowns. When a
+# download surfaces ErrorType.RATE_LIMITED, every other source on the same
+# platform is deferred for PLATFORM_RATE_LIMIT_COOLDOWN_SECONDS so the next
+# scan tick doesn't fire a burst of due same-platform sources back into the
+# same limit. Per-source consecutive_failures backoff still applies on top
+# of this — but this is PREVENTIVE (kills the same-tick burst from N due
+# sources hammering the platform at once), while consecutive_failures is
+# REACTIVE (slows the offender down over many cycles). Operator-confirmed
+# 2026-05-30.
+PLATFORM_COOLDOWN_KEY_PREFIX = "platform_cooldown:"
+PLATFORM_RATE_LIMIT_COOLDOWN_SECONDS = 900  # 15 min
+
 
 def compute_effective_interval(
     source: Source, artist: Artist, settings: ImportSettings,
@@ -40,10 +58,76 @@ def compute_effective_interval(
     return max(MIN_INTERVAL_SECONDS, min(MAX_INTERVAL_SECONDS, raw))
 
 
+async def set_platform_cooldown(
+    session: AsyncSession, platform: str,
+    seconds: int = PLATFORM_RATE_LIMIT_COOLDOWN_SECONDS,
+) -> None:
+    """Stamp a cooldown expiry on the given platform so select_due_sources
+    skips every source on that platform until it expires.
+
+    Called when a download surfaces ErrorType.RATE_LIMITED so the other
+    sources on the same platform don't all retry into the same rate limit.
+    Caller is responsible for committing the session.
+
+    Uses INSERT...ON CONFLICT DO UPDATE so two concurrent workers hitting
+    the same platform's rate limit don't race: a SELECT-then-INSERT pattern
+    would let the loser's whole transaction (including the source-health
+    update + event finalize) roll back on a unique-violation, stranding
+    that event. Atomic upsert avoids that.
+    """
+    now = datetime.now(UTC)
+    expires_at = (now + timedelta(seconds=seconds)).isoformat()
+    key = f"{PLATFORM_COOLDOWN_KEY_PREFIX}{platform}"
+    stmt = pg_insert(AppSetting.__table__).values(
+        key=key, value=expires_at, updated_at=now,
+    ).on_conflict_do_update(
+        index_elements=["key"],
+        set_={"value": expires_at, "updated_at": now},
+    )
+    await session.execute(stmt)
+
+
+async def active_platform_cooldowns(session: AsyncSession) -> dict[str, datetime]:
+    """Return {platform: expires_at} for platforms whose cooldown is still
+    in the future. Expired rows are ignored (a future maintenance sweep can
+    delete them; they don't affect routing decisions on their own).
+
+    Exposed beyond scheduler_service so the manual check endpoint
+    (`/api/sources/<id>/check`) can defer bulk retries that would bowl
+    into the same rate limit the cooldown is preventing.
+    """
+    rows = (await session.execute(
+        select(AppSetting.key, AppSetting.value)
+        .where(AppSetting.key.startswith(PLATFORM_COOLDOWN_KEY_PREFIX))
+    )).all()
+    if not rows:
+        return {}
+    now = datetime.now(UTC)
+    active: dict[str, datetime] = {}
+    for key, value in rows:
+        try:
+            expires_at = datetime.fromisoformat(value)
+        except (ValueError, TypeError):
+            continue
+        if expires_at > now:
+            active[key[len(PLATFORM_COOLDOWN_KEY_PREFIX):]] = expires_at
+    return active
+
+
 async def select_due_sources(session: AsyncSession) -> list[Source]:
     """Sources where (enabled, artist.auto_check) and now >= last_checked_at + effective_interval.
 
-    Never-checked sources (last_checked_at IS NULL) are always due.
+    Never-checked sources (last_checked_at IS NULL) are always due. Sources
+    whose platform is currently in a rate-limit cooldown are excluded — the
+    cooldown is the preventive half of the burst-prevention pair (per-source
+    consecutive_failures backoff handles the offending source itself).
+
+    Ordering: last_checked_at ASC NULLS FIRST, then id. Never-checked
+    sources go first, then the longest-since-checked, so the most overdue
+    sources hit Celery's FIFO download queue first. Anti-starvation: if
+    queue throughput ever falls below the tick rate, a freshly-rerun source
+    can't keep cutting in line ahead of one that hasn't been checked at all.
+    Operator-confirmed 2026-05-30.
     """
     rows = (await session.execute(
         select(Source)
@@ -51,15 +135,17 @@ async def select_due_sources(session: AsyncSession) -> list[Source]:
         .join(Artist, Source.artist_id == Artist.id)
         .where(Source.enabled.is_(True))
         .where(Artist.auto_check.is_(True))
+        .order_by(Source.last_checked_at.asc().nulls_first(), Source.id)
     )).scalars().all()
 
-    settings = (await session.execute(
-        select(ImportSettings).where(ImportSettings.id == 1)
-    )).scalar_one()
+    cooldowns = await active_platform_cooldowns(session)
+    settings = await ImportSettings.load(session)
 
     now = datetime.now(UTC)
     due: list[Source] = []
     for s in rows:
+        if s.platform in cooldowns:
+            continue
         interval = compute_effective_interval(s, s.artist, settings)
         if s.last_checked_at is None:
             due.append(s)
@@ -78,3 +164,65 @@ def compute_next_check_at(
         return None
     interval = compute_effective_interval(source, artist, settings)
     return source.last_checked_at + timedelta(seconds=interval)
+
+
+async def record_tick(session: AsyncSession) -> None:
+    """Stamp the current time on the SCHEDULER_LAST_TICK_KEY AppSetting.
+
+    Called once per Beat tick so the UI can prove the scheduler is alive.
+    Commits its own write so the stamp survives even if the rest of the
+    tick errors out.
+    """
+    now_iso = datetime.now(UTC).isoformat()
+    row = (await session.execute(
+        select(AppSetting).where(AppSetting.key == SCHEDULER_LAST_TICK_KEY)
+    )).scalar_one_or_none()
+    if row is None:
+        session.add(AppSetting(key=SCHEDULER_LAST_TICK_KEY, value=now_iso))
+    else:
+        row.value = now_iso
+    await session.commit()
+
+
+async def scheduler_status(session: AsyncSession) -> dict:
+    """Summarise scheduler health for the dashboard.
+
+    Returns last_tick_at (when Beat last fired), next_due_at (earliest
+    upcoming scheduled check across enabled auto-check sources), due_now
+    (how many are due right now), and auto_sources (total under schedule).
+    """
+    last_tick_at = (await session.execute(
+        select(AppSetting.value).where(AppSetting.key == SCHEDULER_LAST_TICK_KEY)
+    )).scalar_one_or_none()
+
+    rows = (await session.execute(
+        select(Source)
+        .options(selectinload(Source.artist))
+        .join(Artist, Source.artist_id == Artist.id)
+        .where(Source.enabled.is_(True))
+        .where(Artist.auto_check.is_(True))
+    )).scalars().all()
+    settings = await ImportSettings.load(session)
+
+    now = datetime.now(UTC)
+    due_now = 0
+    next_due_at: datetime | None = None
+    for s in rows:
+        if s.last_checked_at is None:
+            due_now += 1
+            continue
+        nca = compute_next_check_at(s, s.artist, settings)
+        if nca is None or nca <= now:
+            due_now += 1
+        elif next_due_at is None or nca < next_due_at:
+            next_due_at = nca
+
+    cooldowns = await active_platform_cooldowns(session)
+
+    return {
+        "last_tick_at": last_tick_at,
+        "next_due_at": next_due_at.isoformat() if next_due_at else None,
+        "due_now": due_now,
+        "auto_sources": len(rows),
+        "platform_cooldowns": {p: dt.isoformat() for p, dt in cooldowns.items()},
+    }

backend/app/services/series_service.py

@@ -63,6 +63,7 @@ class SeriesService:
                     ImageRecord.sha256,
                     ImageRecord.mime,
                     ImageRecord.path,
+                    ImageRecord.thumbnail_path,
                 )
                 .join(ImageRecord, ImageRecord.id == SeriesPage.image_id)
                 .where(SeriesPage.series_tag_id == series_tag_id)
@@ -75,7 +76,7 @@ class SeriesService:
                 {
                     "image_id": r.image_id,
                     "page_number": r.page_number,
-                    "thumbnail_url": thumbnail_url(r.sha256, r.mime),
+                    "thumbnail_url": thumbnail_url(r.thumbnail_path, r.sha256, r.mime),
                     "image_url": f"/images/{r.path.split('/images/', 1)[-1]}",
                 }
                 for r in rows

backend/app/services/showcase_service.py

@@ -34,7 +34,7 @@ class ShowcaseService:
                 "mime": r.mime,
                 "width": r.width,
                 "height": r.height,
-                "thumbnail_url": thumbnail_url(r.sha256, r.mime),
+                "thumbnail_url": thumbnail_url(r.thumbnail_path, r.sha256, r.mime),
             }
             for r in rows
         ]

backend/app/services/source_service.py

@@ -120,9 +120,7 @@ class SourceService:
         return config
 
     async def _load_settings(self) -> ImportSettings:
-        return (await self.session.execute(
-            select(ImportSettings).where(ImportSettings.id == 1)
-        )).scalar_one()
+        return await ImportSettings.load(self.session)
 
     def _build_record(
         self, source: Source, artist: Artist, settings: ImportSettings,
@@ -151,14 +149,27 @@ class SourceService:
         settings = await self._load_settings()
         return self._build_record(source, artist, settings)
 
-    async def list(self, artist_id: int | None = None) -> list[SourceRecord]:
-        stmt = (
-            select(Source, Artist)
-            .join(Artist, Artist.id == Source.artist_id)
-            .order_by(Artist.name.asc(), Source.id.asc())
-        )
+    async def list(
+        self, artist_id: int | None = None, failing: bool = False,
+        include_synthetic: bool = False,
+    ) -> list[SourceRecord]:
+        stmt = select(Source, Artist).join(Artist, Artist.id == Source.artist_id)
         if artist_id is not None:
             stmt = stmt.where(Source.artist_id == artist_id)
+        if not include_synthetic:
+            # Filesystem-import sidecar anchors (importer._source_for_sidecar)
+            # have url='sidecar:<platform>:<slug>' and exist only to give
+            # imported Posts a NOT-NULL Source FK. They aren't pollable
+            # feeds; the Subscriptions UI used to render them as phantom
+            # subscriptions. Hide by default.
+            stmt = stmt.where(~Source.url.like("sidecar:%"))
+        if failing:
+            # Worst-first so the rollup card surfaces the loudest failures.
+            stmt = stmt.where(Source.consecutive_failures > 0).order_by(
+                Source.consecutive_failures.desc(), Artist.name.asc(),
+            )
+        else:
+            stmt = stmt.order_by(Artist.name.asc(), Source.id.asc())
         rows = (await self.session.execute(stmt)).all()
         settings = await self._load_settings()
         return [self._build_record(s, a, settings) for s, a in rows]

backend/app/services/tag_directory_service.py

@@ -115,12 +115,17 @@ class TagDirectoryService:
             .subquery()
         )
         stmt = (
-            select(sub.c.tag_id, ImageRecord.sha256, ImageRecord.mime)
+            select(
+                sub.c.tag_id,
+                ImageRecord.sha256,
+                ImageRecord.mime,
+                ImageRecord.thumbnail_path,
+            )
             .join(ImageRecord, ImageRecord.id == sub.c.image_record_id)
             .where(sub.c.rn <= 3)
             .order_by(sub.c.tag_id, sub.c.rn)
         )
         out: dict[int, list[str]] = {}
-        for tag_id, sha, mime in (await self.session.execute(stmt)).all():
-            out.setdefault(tag_id, []).append(thumbnail_url(sha, mime))
+        for tag_id, sha, mime, tp in (await self.session.execute(stmt)).all():
+            out.setdefault(tag_id, []).append(thumbnail_url(tp, sha, mime))
         return out

backend/app/tasks/_async_session.py

@@ -0,0 +1,21 @@
+"""Per-invocation async session factory for Celery task modules.
+
+Async engine connections are bound to the event loop. Each Celery task
+runs its async body under a fresh ``asyncio.run()`` loop, so it needs its
+own engine created (and disposed) within that loop — a process-wide async
+engine would reuse loop-bound connections across tasks and raise "attached
+to a different loop". So unlike the process-wide sync engine in
+``_sync_engine.py``, this returns a fresh engine per call; the caller
+disposes it (``await engine.dispose()``) when its loop ends.
+"""
+
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
+
+from ..config import get_config
+
+
+def async_session_factory():
+    """Return ``(sessionmaker, engine)`` bound to a fresh async engine."""
+    cfg = get_config()
+    engine = create_async_engine(cfg.database_url, future=True, pool_pre_ping=True)
+    return async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False), engine

backend/app/tasks/admin.py

@@ -0,0 +1,57 @@
+"""FC-3k: admin destructive Celery tasks.
+
+Two long-running ops on the maintenance queue. task_run lifecycle is
+captured automatically by FC-3i signals — these tasks just return
+their summary dict so it lands in task_run.metadata (via Celery's
+result backend) for the dashboard to surface.
+
+Soft/hard time limits inherit the FC-3i recovery sweep: a runaway
+task gets killed and flipped to status='timeout' by
+recover_stalled_task_runs.
+"""
+from __future__ import annotations
+
+import logging
+from pathlib import Path
+
+from sqlalchemy.exc import DBAPIError, OperationalError
+
+from ..celery_app import celery
+from ..services import cleanup_service
+from ._sync_engine import sync_session_factory as _sync_session_factory
+
+log = logging.getLogger(__name__)
+IMAGES_ROOT = Path("/images")
+
+
+@celery.task(
+    name="backend.app.tasks.admin.delete_artist_cascade_task",
+    bind=True,
+    autoretry_for=(OperationalError, DBAPIError),
+    retry_backoff=15, retry_backoff_max=180, max_retries=1,
+    soft_time_limit=1800, time_limit=2400,  # 30 min / 40 min
+)
+def delete_artist_cascade_task(self, *, artist_id: int) -> dict:
+    """Wraps cleanup_service.delete_artist_cascade. Returns the
+    service's summary dict for FC-3i task_run.metadata capture."""
+    SessionLocal = _sync_session_factory()
+    with SessionLocal() as session:
+        return cleanup_service.delete_artist_cascade(
+            session, artist_id=artist_id, images_root=IMAGES_ROOT,
+        )
+
+
+@celery.task(
+    name="backend.app.tasks.admin.bulk_delete_images_task",
+    bind=True,
+    autoretry_for=(OperationalError, DBAPIError),
+    retry_backoff=15, retry_backoff_max=180, max_retries=1,
+    soft_time_limit=900, time_limit=1200,  # 15 min / 20 min
+)
+def bulk_delete_images_task(self, *, image_ids: list[int]) -> dict:
+    """Wraps cleanup_service.delete_images."""
+    SessionLocal = _sync_session_factory()
+    with SessionLocal() as session:
+        return cleanup_service.delete_images(
+            session, image_ids=image_ids, images_root=IMAGES_ROOT,
+        )

backend/app/tasks/backup.py

@@ -0,0 +1,296 @@
+"""FC-3h: backup/restore Celery tasks.
+
+All tasks live on the maintenance queue (per celery_app.task_routes).
+task_run lifecycle tracking is automatic via FC-3i signals — these
+tasks just record the operator-facing artifact metadata into
+BackupRun.
+"""
+from __future__ import annotations
+
+import logging
+from datetime import UTC, datetime
+from pathlib import Path
+
+from celery.exceptions import SoftTimeLimitExceeded
+from sqlalchemy import select
+from sqlalchemy.exc import DBAPIError, OperationalError
+
+from ..celery_app import celery
+from ..config import get_config
+from ..models import BackupRun, ImportSettings
+from ..services import backup_service
+from ._sync_engine import sync_session_factory as _sync_session_factory
+
+log = logging.getLogger(__name__)
+IMAGES_ROOT = Path("/images")
+
+
+def _mark_failed(session, row: BackupRun, exc: BaseException) -> None:
+    """Flip a BackupRun row from running/restoring to error with a
+    truncated error message and finished_at. Caller already holds the
+    session open."""
+    row.status = "error"
+    row.error = f"{type(exc).__name__}: {exc}"[:2000]
+    row.finished_at = datetime.now(UTC)
+    session.add(row)
+    session.commit()
+
+
+@celery.task(
+    name="backend.app.tasks.backup.backup_db_task",
+    bind=True,
+    autoretry_for=(OperationalError, DBAPIError),
+    retry_backoff=10, retry_backoff_max=120, max_retries=2,
+    soft_time_limit=600, time_limit=720,
+)
+def backup_db_task(self, *, tag: str | None = None,
+                   triggered_by: str = "manual") -> dict:
+    """Create one DB backup. Returns {'backup_run_id': N}."""
+    SessionLocal = _sync_session_factory()
+    cfg = get_config()
+    now = datetime.now(UTC)
+    with SessionLocal() as session:
+        row = BackupRun(
+            kind="db", status="running", tag=tag,
+            triggered_by=triggered_by, started_at=now, manifest={},
+        )
+        session.add(row)
+        session.commit()
+        session.refresh(row)
+        run_id = row.id
+
+    try:
+        result = backup_service.backup_db(
+            db_url=cfg.database_url_sync, images_root=IMAGES_ROOT,
+            tag=tag, triggered_by=triggered_by,
+        )
+    except (SoftTimeLimitExceeded, Exception) as exc:
+        with SessionLocal() as session:
+            row = session.get(BackupRun, run_id)
+            if row is not None:
+                _mark_failed(session, row, exc)
+        raise
+
+    with SessionLocal() as session:
+        row = session.get(BackupRun, run_id)
+        row.status = "ok"
+        row.finished_at = datetime.now(UTC)
+        row.sql_path = result["sql_path"]
+        row.size_bytes = result["size_bytes"]
+        row.manifest = {
+            "manifest_path": result["manifest_path"],
+            "ts": result["ts"],
+        }
+        session.commit()
+    return {"backup_run_id": run_id}
+
+
+@celery.task(
+    name="backend.app.tasks.backup.backup_images_task",
+    bind=True,
+    autoretry_for=(OperationalError, DBAPIError),
+    retry_backoff=30, retry_backoff_max=300, max_retries=1,
+    soft_time_limit=21600, time_limit=23400,
+)
+def backup_images_task(self, *, tag: str | None = None,
+                       triggered_by: str = "manual") -> dict:
+    """Create one images backup. Same shape as backup_db_task; uses
+    tar_path instead of sql_path."""
+    SessionLocal = _sync_session_factory()
+    now = datetime.now(UTC)
+    with SessionLocal() as session:
+        row = BackupRun(
+            kind="images", status="running", tag=tag,
+            triggered_by=triggered_by, started_at=now, manifest={},
+        )
+        session.add(row)
+        session.commit()
+        session.refresh(row)
+        run_id = row.id
+
+    try:
+        result = backup_service.backup_images(
+            images_root=IMAGES_ROOT,
+            tag=tag, triggered_by=triggered_by,
+        )
+    except (SoftTimeLimitExceeded, Exception) as exc:
+        with SessionLocal() as session:
+            row = session.get(BackupRun, run_id)
+            if row is not None:
+                _mark_failed(session, row, exc)
+        raise
+
+    with SessionLocal() as session:
+        row = session.get(BackupRun, run_id)
+        row.status = "ok"
+        row.finished_at = datetime.now(UTC)
+        row.tar_path = result["tar_path"]
+        row.size_bytes = result["size_bytes"]
+        row.manifest = {
+            "manifest_path": result["manifest_path"],
+            "ts": result["ts"],
+        }
+        session.commit()
+    return {"backup_run_id": run_id}
+
+
+@celery.task(
+    name="backend.app.tasks.backup.restore_db_task",
+    bind=True,
+    max_retries=0,  # NEVER auto-retry a half-applied restore.
+    soft_time_limit=1200, time_limit=1800,
+)
+def restore_db_task(self, *, source_backup_run_id: int) -> dict:
+    """Restore from a previous DB backup. Inserts a NEW BackupRun row
+    (kind='db', status='restoring') linked to the source via
+    restored_from_id; flips to 'restored' on success or 'error' on
+    failure. Operator sees the restore as a row in the dashboard."""
+    SessionLocal = _sync_session_factory()
+    cfg = get_config()
+    now = datetime.now(UTC)
+    with SessionLocal() as session:
+        src = session.get(BackupRun, source_backup_run_id)
+        if src is None or src.kind != "db" or not src.sql_path:
+            raise ValueError(
+                f"BackupRun id={source_backup_run_id} is not a valid DB backup"
+            )
+        marker = BackupRun(
+            kind="db", status="restoring",
+            triggered_by="restore", started_at=now,
+            restored_from_id=src.id,
+            manifest={"source_sql_path": src.sql_path},
+        )
+        session.add(marker)
+        session.commit()
+        session.refresh(marker)
+        marker_id = marker.id
+        sql_path = src.sql_path
+
+    try:
+        backup_service.restore_db(
+            db_url=cfg.database_url_sync, sql_path=Path(sql_path),
+        )
+    except (SoftTimeLimitExceeded, Exception) as exc:
+        with SessionLocal() as session:
+            row = session.get(BackupRun, marker_id)
+            if row is not None:
+                _mark_failed(session, row, exc)
+        raise
+
+    with SessionLocal() as session:
+        row = session.get(BackupRun, marker_id)
+        row.status = "restored"
+        row.finished_at = datetime.now(UTC)
+        session.commit()
+    return {"backup_run_id": marker_id}
+
+
+@celery.task(
+    name="backend.app.tasks.backup.restore_images_task",
+    bind=True, max_retries=0,
+    soft_time_limit=21600, time_limit=23400,
+)
+def restore_images_task(self, *, source_backup_run_id: int) -> dict:
+    """Mirrors restore_db_task; uses backup_service.restore_images."""
+    SessionLocal = _sync_session_factory()
+    now = datetime.now(UTC)
+    with SessionLocal() as session:
+        src = session.get(BackupRun, source_backup_run_id)
+        if src is None or src.kind != "images" or not src.tar_path:
+            raise ValueError(
+                f"BackupRun id={source_backup_run_id} is not a valid images backup"
+            )
+        marker = BackupRun(
+            kind="images", status="restoring",
+            triggered_by="restore", started_at=now,
+            restored_from_id=src.id,
+            manifest={"source_tar_path": src.tar_path},
+        )
+        session.add(marker)
+        session.commit()
+        session.refresh(marker)
+        marker_id = marker.id
+        tar_path = src.tar_path
+
+    try:
+        backup_service.restore_images(
+            images_root=IMAGES_ROOT, tar_path=Path(tar_path),
+        )
+    except (SoftTimeLimitExceeded, Exception) as exc:
+        with SessionLocal() as session:
+            row = session.get(BackupRun, marker_id)
+            if row is not None:
+                _mark_failed(session, row, exc)
+        raise
+
+    with SessionLocal() as session:
+        row = session.get(BackupRun, marker_id)
+        row.status = "restored"
+        row.finished_at = datetime.now(UTC)
+        session.commit()
+    return {"backup_run_id": marker_id}
+
+
+@celery.task(
+    name="backend.app.tasks.backup.prune_backups",
+    soft_time_limit=300, time_limit=600,
+)
+def prune_backups() -> dict:
+    """Daily Beat. Per-kind retention from ImportSettings.
+
+    Returns {"db_deleted": N, "images_deleted": M, "files_unlinked": K}.
+    Tagged rows (tag IS NOT NULL) are never pruned.
+    Status='running' / 'restoring' rows are never pruned (recovery
+    sweep from FC-3i handles those via task_run).
+    """
+    SessionLocal = _sync_session_factory()
+    counts = {"db_deleted": 0, "images_deleted": 0, "files_unlinked": 0}
+    with SessionLocal() as session:
+        s = ImportSettings.load_sync(session)
+        for kind, keep in (
+            ("db", s.backup_db_keep_last_n),
+            ("images", s.backup_images_keep_last_n),
+        ):
+            candidates = session.execute(
+                select(BackupRun)
+                .where(BackupRun.kind == kind)
+                .where(BackupRun.tag.is_(None))
+                .where(BackupRun.status.in_(["ok", "error"]))
+                .order_by(BackupRun.started_at.desc())
+                .offset(keep)
+            ).scalars().all()
+            for row in candidates:
+                result = backup_service.unlink_artifact_files(
+                    sql_path=row.sql_path,
+                    tar_path=row.tar_path,
+                    manifest_path=(row.manifest or {}).get("manifest_path"),
+                )
+                counts["files_unlinked"] += sum(
+                    1 for v in result.values() if v
+                )
+                session.delete(row)
+                counts[f"{kind}_deleted"] += 1
+        session.commit()
+    return counts
+
+
+@celery.task(
+    name="backend.app.tasks.backup.backup_db_nightly",
+    soft_time_limit=60, time_limit=120,
+)
+def backup_db_nightly() -> dict:
+    """Hourly tick. Dispatches a real backup ONLY if the configured
+    UTC hour matches and the nightly setting is enabled. Returns
+    either {'skipped': '<reason>'} or {'dispatched': '<task_id>'}."""
+    SessionLocal = _sync_session_factory()
+    with SessionLocal() as session:
+        s = ImportSettings.load_sync(session)
+        nightly_enabled = s.backup_db_nightly_enabled
+        configured_hour = s.backup_db_nightly_hour_utc
+    if not nightly_enabled:
+        return {"skipped": "nightly disabled"}
+    now_hour = datetime.now(UTC).hour
+    if now_hour != configured_hour:
+        return {"skipped": f"hour={now_hour} != configured={configured_hour}"}
+    res = backup_db_task.delay(triggered_by="nightly")
+    return {"dispatched": res.id}

backend/app/tasks/download.py

@@ -3,11 +3,9 @@
 import asyncio
 from pathlib import Path
 
-from sqlalchemy import select
-from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
+from sqlalchemy.exc import DBAPIError, OperationalError
 
 from ..celery_app import celery
-from ..config import get_config
 from ..models import ImportSettings
 from ..services.credential_crypto import CredentialCrypto
 from ..services.credential_service import CredentialService
@@ -15,30 +13,34 @@ from ..services.download_service import DownloadService
 from ..services.gallery_dl import GalleryDLService
 from ..services.importer import Importer
 from ..services.thumbnailer import Thumbnailer
+from ._async_session import async_session_factory
 from .import_file import _sync_session_factory
 
 IMAGES_ROOT = Path("/images")
 _KEY_PATH = IMAGES_ROOT / "secrets" / "credential_key.b64"
 
 
-def _async_session_factory():
-    cfg = get_config()
-    engine = create_async_engine(cfg.database_url, future=True, pool_pre_ping=True)
-    return async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False), engine
-
-
-@celery.task(name="backend.app.tasks.download.download_source", bind=True, acks_late=True)
+@celery.task(
+    name="backend.app.tasks.download.download_source",
+    bind=True,
+    acks_late=True,
+    autoretry_for=(OperationalError, DBAPIError, OSError),
+    retry_backoff=10,
+    retry_backoff_max=120,
+    retry_jitter=True,
+    max_retries=3,
+    soft_time_limit=900,
+    time_limit=1200,
+)
 def download_source(self, source_id: int) -> int:
     """Returns the DownloadEvent.id."""
 
     async def _run():
-        async_factory, async_engine = _async_session_factory()
+        async_factory, async_engine = async_session_factory()
         SyncFactory = _sync_session_factory()
         try:
             with SyncFactory() as sync_session:
-                settings = sync_session.execute(
-                    select(ImportSettings).where(ImportSettings.id == 1)
-                ).scalar_one()
+                settings = ImportSettings.load_sync(sync_session)
                 rate_limit = settings.download_rate_limit_seconds
                 validate_files = settings.download_validate_files
 
@@ -52,9 +54,7 @@ def download_source(self, source_id: int) -> int:
             async with async_factory() as async_session:
                 cred_service = CredentialService(async_session, crypto)
                 with SyncFactory() as sync_session:
-                    sync_settings = sync_session.execute(
-                        select(ImportSettings).where(ImportSettings.id == 1)
-                    ).scalar_one()
+                    sync_settings = ImportSettings.load_sync(sync_session)
                     importer = Importer(
                         session=sync_session,
                         images_root=IMAGES_ROOT,

backend/app/tasks/import_file.py

@@ -1,11 +1,21 @@
 """import_media_file task: imports one file via the Importer service and
 updates the ImportTask state machine + ImportBatch counters atomically.
+
+Resilience contract (2026-05-24, operator-mandated): once a row has been
+flipped to 'processing' inside this task, EVERY exit path MUST flip it
+to a terminal state (complete / skipped / failed) or rely on Celery's
+autoretry to attempt the work again. No exit path is allowed to leave
+the row stuck in 'processing'. The `recover_interrupted_tasks`
+maintenance sweep is the safety net (5 min threshold) for the case
+where even the failure-marking commit can't be written.
 """
 
 from datetime import UTC, datetime
 from pathlib import Path
 
+from celery.exceptions import SoftTimeLimitExceeded
 from sqlalchemy import select, update
+from sqlalchemy.exc import DBAPIError, OperationalError
 
 from ..celery_app import celery
 from ..models import ImportBatch, ImportSettings, ImportTask
@@ -19,19 +29,49 @@ IMAGES_ROOT = Path("/images")
 def _map_result_to_status(result):
     """(ImportTask.status, should_requeue_ml_and_thumb) for an ImportResult.
     'superseded' = the kept row's file/ML changed → complete + re-derive.
-    'attached' = a non-art file preserved → complete, no ML/thumb."""
+    'attached' = a non-art file preserved → complete, no ML/thumb.
+    'refreshed' = deep scan refreshed sidecar/phash on an existing row →
+    complete, no ML/thumb re-derive (file/pixels unchanged)."""
     if result.status in ("imported", "superseded"):
         return ("complete", True)
-    if result.status == "attached":
+    if result.status in ("attached", "refreshed"):
         return ("complete", False)
     if result.status == "skipped":
         return ("skipped", False)
     return ("failed", False)
 
 
-@celery.task(name="backend.app.tasks.import_file.import_media_file", bind=True)
-def import_media_file(self, import_task_id: int) -> dict:
-    """Returns a dict so the eager-mode tests can assert without DB."""
+def _mark_failed(session, task, error_msg: str) -> None:
+    """Best-effort flip of a 'processing' row to 'failed' + batch counter
+    increment. Wrapped in its own try because if the DB is what just
+    broke, this commit will also fail — that's why the maintenance sweep
+    exists as a backstop."""
+    try:
+        task.status = "failed"
+        task.error = error_msg
+        task.finished_at = datetime.now(UTC)
+        session.add(task)
+        session.execute(
+            update(ImportBatch)
+            .where(ImportBatch.id == task.batch_id)
+            .values(failed=ImportBatch.failed + 1)
+        )
+        session.commit()
+    except Exception:  # noqa: BLE001 — best-effort, sweep catches the rest
+        try:
+            session.rollback()
+        except Exception:  # noqa: BLE001
+            pass
+
+
+def _run_import_task(import_task_id: int) -> dict:
+    """Shared body for import_media_file + import_archive_file. The two
+    tasks differ ONLY in their Celery time limits (a single media file
+    is sub-second; an archive runs the full per-member pipeline inline
+    for every member and can take many minutes). Both flip the row to
+    'processing', dispatch to `_do_import`, and honor the
+    flip-to-terminal resilience contract.
+    """
     SessionLocal = _sync_session_factory()
     with SessionLocal() as session:
         task = session.get(ImportTask, import_task_id)
@@ -43,99 +83,177 @@ def import_media_file(self, import_task_id: int) -> dict:
         session.add(task)
         session.commit()
 
-        settings = session.execute(
-            select(ImportSettings).where(ImportSettings.id == 1)
-        ).scalar_one()
-        import_root = Path(settings.import_scan_path)
-        batch = session.get(ImportBatch, task.batch_id)
-        deep = bool(batch and batch.scan_mode == "deep")
-        importer = Importer(
-            session=session,
-            images_root=IMAGES_ROOT,
-            import_root=import_root,
-            thumbnailer=Thumbnailer(images_root=IMAGES_ROOT),
-            settings=settings,
-            deep=deep,
-        )
-
         try:
-            result = importer.import_one(Path(task.source_path))
-        except Exception as exc:  # pragma: no cover — pipeline crash
-            task.status = "failed"
-            task.error = f"{type(exc).__name__}: {exc}"
-            task.finished_at = datetime.now(UTC)
-            session.execute(
-                update(ImportBatch)
-                .where(ImportBatch.id == task.batch_id)
-                .values(failed=ImportBatch.failed + 1)
-            )
-            session.add(task)
-            session.commit()
+            return _do_import(session, task, import_task_id)
+        except SoftTimeLimitExceeded:
+            _mark_failed(session, task, "soft_time_limit exceeded")
+            raise
+        except (OperationalError, DBAPIError, OSError):
+            # Retryable per the decorator; do NOT mark failed (let
+            # autoretry have a clean go at it). If autoretry exhausts,
+            # the row stays 'processing' and the maintenance sweep
+            # flips it.
+            raise
+        except Exception as exc:  # noqa: BLE001 — pipeline crash, mark + re-raise
+            _mark_failed(session, task, f"{type(exc).__name__}: {exc}")
             raise
 
-        if result.status in ("imported", "superseded"):
-            task.status = "complete"
-            task.result_image_id = result.image_id
-            counter_col_name = "imported"
-            counter_col = ImportBatch.imported
-        elif result.status == "attached":
-            task.status = "complete"
-            counter_col_name = "attachments"
-            counter_col = ImportBatch.attachments
-        elif result.status == "skipped":
-            task.status = "skipped"
-            task.error = (
-                f"{result.skip_reason.value}: {result.error}"
-                if result.skip_reason
-                else result.error
-            )
-            task.result_image_id = result.image_id
-            counter_col_name = "skipped"
-            counter_col = ImportBatch.skipped
-        else:
-            task.status = "failed"
-            task.error = result.error
-            counter_col_name = "failed"
-            counter_col = ImportBatch.failed
 
-        task.finished_at = datetime.now(UTC)
+@celery.task(
+    name="backend.app.tasks.import_file.import_media_file",
+    bind=True,
+    autoretry_for=(OperationalError, DBAPIError, OSError),
+    retry_backoff=5,
+    retry_backoff_max=60,
+    retry_jitter=True,
+    max_retries=3,
+    soft_time_limit=300,
+    time_limit=360,
+)
+def import_media_file(self, import_task_id: int) -> dict:
+    """Import ONE media file (or non-media → PostAttachment). Sub-second
+    for the common case; the tight 5-min soft limit keeps a genuinely
+    stuck single-file import detectable fast.
+
+    Decorator notes:
+      - autoretry_for: transient DB / filesystem errors retry with
+        exponential backoff (5s base, jitter, max 3 attempts). On final
+        give-up the task raises and acks_late=True (set globally on the
+        Celery app) does NOT redeliver — the recovery sweep catches the
+        row instead.
+      - soft_time_limit (300s) raises SoftTimeLimitExceeded in-process
+        so the task can mark its row failed before being killed.
+      - time_limit (360s) is the hard SIGKILL cap.
+    """
+    return _run_import_task(import_task_id)
+
+
+@celery.task(
+    name="backend.app.tasks.import_file.import_archive_file",
+    bind=True,
+    autoretry_for=(OperationalError, DBAPIError, OSError),
+    retry_backoff=5,
+    retry_backoff_max=60,
+    retry_jitter=True,
+    max_retries=3,
+    # Archives run the full per-member pipeline (sha256 + pHash + dedup
+    # query + copy + provenance) for EVERY media member inline, under a
+    # single task budget. A multi-hundred-member archive blows the
+    # 5-min media limit. soft=30min / hard=35min sizes for a large
+    # archive. Operator-flagged 2026-05-28 (target 1645019 hit the old
+    # shared 300s soft limit). The recovery sweep gives this task its
+    # own 40-min threshold via maintenance.TASK_STUCK_THRESHOLD_MINUTES
+    # so it isn't preempted while legitimately grinding through members.
+    soft_time_limit=1800,
+    time_limit=2100,
+)
+def import_archive_file(self, import_task_id: int) -> dict:
+    """Import an archive: extract + run the per-member media pipeline for
+    every member inline, then preserve the archive as a PostAttachment.
+    Same body as import_media_file (dispatch is by file kind inside
+    Importer.import_one); split out purely for the larger time budget."""
+    return _run_import_task(import_task_id)
+
+
+def enqueue_import(task_id: int, task_type: str) -> None:
+    """Route an ImportTask to the right Celery task by its task_type.
+    Single source of truth for the media-vs-archive dispatch so the
+    scan, retry, and recovery-requeue paths stay in sync."""
+    if task_type == "archive":
+        import_archive_file.delay(task_id)
+    else:
+        import_media_file.delay(task_id)
+
+
+def _do_import(session, task, import_task_id: int) -> dict:
+    """Actual work, called from inside the resilience wrapper."""
+    settings = ImportSettings.load_sync(session)
+    import_root = Path(settings.import_scan_path)
+    batch = session.get(ImportBatch, task.batch_id)
+    deep = bool(batch and batch.scan_mode == "deep")
+    importer = Importer(
+        session=session,
+        images_root=IMAGES_ROOT,
+        import_root=import_root,
+        thumbnailer=Thumbnailer(images_root=IMAGES_ROOT),
+        settings=settings,
+        deep=deep,
+    )
+
+    result = importer.import_one(Path(task.source_path))
+
+    if result.status in ("imported", "superseded"):
+        task.status = "complete"
+        task.result_image_id = result.image_id
+        counter_col_name = "imported"
+        counter_col = ImportBatch.imported
+    elif result.status == "refreshed":
+        # Deep-scan rederive: existing row got phash/artist/sidecar
+        # refreshed. Task is complete (no further work), but counted in
+        # `refreshed` not `imported` so the UI can surface the actual
+        # work done. operator-flagged 2026-05-25.
+        task.status = "complete"
+        task.result_image_id = result.image_id
+        counter_col_name = "refreshed"
+        counter_col = ImportBatch.refreshed
+    elif result.status == "attached":
+        task.status = "complete"
+        counter_col_name = "attachments"
+        counter_col = ImportBatch.attachments
+    elif result.status == "skipped":
+        task.status = "skipped"
+        task.error = (
+            f"{result.skip_reason.value}: {result.error}"
+            if result.skip_reason
+            else result.error
+        )
+        task.result_image_id = result.image_id
+        counter_col_name = "skipped"
+        counter_col = ImportBatch.skipped
+    else:
+        task.status = "failed"
+        task.error = result.error
+        counter_col_name = "failed"
+        counter_col = ImportBatch.failed
+
+    task.finished_at = datetime.now(UTC)
+    session.execute(
+        update(ImportBatch)
+        .where(ImportBatch.id == task.batch_id)
+        .values({counter_col_name: counter_col + 1})
+    )
+    session.add(task)
+    session.commit()
+
+    # Enqueue thumbnail + ML for newly imported AND superseded images
+    # (a superseded row has cleared ML + no thumbnail).
+    if result.status in ("imported", "superseded"):
+        from .ml import tag_and_embed
+        from .thumbnail import generate_thumbnail
+
+        ids = list(result.member_image_ids)
+        if result.image_id is not None and result.image_id not in ids:
+            ids.append(result.image_id)
+        for img_id in ids:
+            generate_thumbnail.delay(img_id)
+            tag_and_embed.delay(img_id)
+
+    # If this was the last task in the batch, mark the batch complete.
+    remaining = session.execute(
+        select(ImportTask.id)
+        .where(ImportTask.batch_id == task.batch_id)
+        .where(ImportTask.status.in_(["pending", "queued", "processing"]))
+        .limit(1)
+    ).scalar_one_or_none()
+    if remaining is None:
         session.execute(
             update(ImportBatch)
             .where(ImportBatch.id == task.batch_id)
-            .values({counter_col_name: counter_col + 1})
+            .values(
+                status="complete",
+                finished_at=datetime.now(UTC),
+            )
         )
-        session.add(task)
         session.commit()
 
-        # Enqueue thumbnail + ML for newly imported AND superseded images
-        # (a superseded row has cleared ML + no thumbnail).
-        if result.status in ("imported", "superseded"):
-            from .ml import tag_and_embed
-            from .thumbnail import generate_thumbnail
-
-            ids = list(result.member_image_ids)
-            if result.image_id is not None and result.image_id not in ids:
-                ids.append(result.image_id)
-            for img_id in ids:
-                generate_thumbnail.delay(img_id)
-                tag_and_embed.delay(img_id)
-
-        # If this was the last task in the batch, mark the batch complete.
-        remaining = session.execute(
-            select(ImportTask.id)
-            .where(ImportTask.batch_id == task.batch_id)
-            .where(ImportTask.status.in_(["pending", "queued", "processing"]))
-            .limit(1)
-        ).scalar_one_or_none()
-        if remaining is None:
-            session.execute(
-                update(ImportBatch)
-                .where(ImportBatch.id == task.batch_id)
-                .values(
-                    status="complete",
-                    finished_at=datetime.now(UTC),
-                )
-            )
-            session.commit()
-
-        return {"task_id": import_task_id, "status": task.status}
+    return {"task_id": import_task_id, "status": task.status}

backend/app/tasks/library_audit.py

@@ -0,0 +1,167 @@
+"""scan_library_for_rule Celery task — iterates image_record in keyset-
+paginated batches, evaluates the audit rule per image, populates
+LibraryAuditRun.matched_ids. Runs on the maintenance queue with a 2h soft
+time limit (plenty of margin for 100k+ image libraries at ~100ms PIL
+decode + histogram per image).
+
+State machine:
+  start: status='running'
+  end success: status='ready'
+  end error: status='error', error=traceback
+  oversize: status='error', error='matched too many images; tighten threshold'
+  external cancel: scan sees status='cancelled' between batches, exits.
+"""
+
+import logging
+import traceback
+from datetime import UTC, datetime
+
+from celery.exceptions import SoftTimeLimitExceeded
+from PIL import Image
+from sqlalchemy import select, update
+from sqlalchemy.exc import DBAPIError, OperationalError
+
+from ..celery_app import celery
+from ..models import ImageRecord, LibraryAuditRun
+from ..services.audits import single_color, transparency
+from ._sync_engine import sync_session_factory as _sync_session_factory
+
+log = logging.getLogger(__name__)
+
+_BATCH = 500
+_PROGRESS_TICK = 100
+_MAX_MATCHED = 50_000
+
+_RULES = {
+    "transparency": transparency.evaluate,
+    "single_color": single_color.evaluate,
+}
+
+
+@celery.task(
+    name="backend.app.tasks.library_audit.scan_library_for_rule",
+    bind=True,
+    autoretry_for=(OperationalError, DBAPIError),
+    retry_backoff=5,
+    retry_backoff_max=60,
+    retry_jitter=True,
+    max_retries=3,
+    soft_time_limit=7200,
+    time_limit=7500,
+)
+def scan_library_for_rule(self, audit_id: int) -> dict:
+    """See module docstring. Returns a small summary dict for eager-mode
+    test assertions (real workers ignore the return value)."""
+    SessionLocal = _sync_session_factory()
+    try:
+        with SessionLocal() as session:
+            audit = session.get(LibraryAuditRun, audit_id)
+            if audit is None:
+                return {"audit_id": audit_id, "status": "missing"}
+            evaluate = _RULES.get(audit.rule)
+            if evaluate is None:
+                _mark_error(session, audit_id, f"unknown rule {audit.rule!r}")
+                return {"audit_id": audit_id, "status": "error"}
+            params = dict(audit.params or {})
+            matched: list[int] = []
+            scanned = 0
+            last_id = 0
+            while True:
+                # Cancellation check between batches.
+                current_status = session.execute(
+                    select(LibraryAuditRun.status)
+                    .where(LibraryAuditRun.id == audit_id)
+                ).scalar_one()
+                if current_status == "cancelled":
+                    return {"audit_id": audit_id, "status": "cancelled"}
+                rows = session.execute(
+                    select(ImageRecord.id, ImageRecord.path)
+                    .where(ImageRecord.id > last_id)
+                    .where(ImageRecord.mime.like("image/%"))
+                    .order_by(ImageRecord.id.asc())
+                    .limit(_BATCH)
+                ).all()
+                if not rows:
+                    break
+                for image_id, image_path in rows:
+                    last_id = image_id
+                    scanned += 1
+                    try:
+                        with Image.open(image_path) as im:
+                            try:
+                                if evaluate(im, **params):
+                                    matched.append(image_id)
+                            except Exception as exc:  # noqa: BLE001
+                                log.warning(
+                                    "audit %s: rule evaluate failed on %s: %s",
+                                    audit_id, image_path, exc,
+                                )
+                    except FileNotFoundError:
+                        log.warning(
+                            "audit %s: image_record %s file missing at %s; skipping",
+                            audit_id, image_id, image_path,
+                        )
+                    except OSError as exc:
+                        log.warning(
+                            "audit %s: PIL load failed for %s: %s",
+                            audit_id, image_path, exc,
+                        )
+                    if len(matched) > _MAX_MATCHED:
+                        _mark_error(
+                            session, audit_id,
+                            f"matched > {_MAX_MATCHED} images; "
+                            "tighten threshold and re-run",
+                        )
+                        return {"audit_id": audit_id, "status": "error"}
+                    if scanned % _PROGRESS_TICK == 0:
+                        session.execute(
+                            update(LibraryAuditRun)
+                            .where(LibraryAuditRun.id == audit_id)
+                            .values(scanned_count=scanned)
+                        )
+                        session.commit()
+            # Final state.
+            session.execute(
+                update(LibraryAuditRun)
+                .where(LibraryAuditRun.id == audit_id)
+                .values(
+                    scanned_count=scanned,
+                    matched_count=len(matched),
+                    matched_ids=matched,
+                    status="ready",
+                    finished_at=datetime.now(UTC),
+                )
+            )
+            session.commit()
+            return {
+                "audit_id": audit_id,
+                "status": "ready",
+                "scanned": scanned,
+                "matched": len(matched),
+            }
+    except SoftTimeLimitExceeded:
+        with SessionLocal() as session:
+            _mark_error(session, audit_id, "soft_time_limit exceeded (>7200s)")
+        raise
+    except (OperationalError, DBAPIError):
+        # Retryable per the decorator; leave row in 'running' and let
+        # autoretry try again. Recovery sweep catches if all retries fail.
+        raise
+    except Exception:  # noqa: BLE001
+        tb = traceback.format_exc()
+        with SessionLocal() as session:
+            _mark_error(session, audit_id, tb)
+        raise
+
+
+def _mark_error(session, audit_id: int, error_msg: str) -> None:
+    session.execute(
+        update(LibraryAuditRun)
+        .where(LibraryAuditRun.id == audit_id)
+        .values(
+            status="error",
+            error=error_msg,
+            finished_at=datetime.now(UTC),
+        )
+    )
+    session.commit()

backend/app/tasks/maintenance.py

@@ -1,59 +1,226 @@
 """Periodic maintenance: recover stuck import tasks, garbage-collect old finished tasks."""
 
 import logging
+import os
 import subprocess
 from datetime import UTC, datetime, timedelta
 from pathlib import Path
 
 from PIL import Image
-from sqlalchemy import delete, select, update
+from sqlalchemy import and_, delete, or_, select, update
 
 from ..celery_app import celery
-from ..models import DownloadEvent, ImageRecord, ImportSettings, ImportTask
+from ..models import (
+    DownloadEvent,
+    ImageRecord,
+    ImportSettings,
+    ImportTask,
+    Source,
+    TaskRun,
+)
 from ..utils.phash import compute_phash
 from ._sync_engine import sync_session_factory as _sync_session_factory
 
 log = logging.getLogger(__name__)
 
-STUCK_THRESHOLD_MINUTES = 30
+STUCK_THRESHOLD_MINUTES = 5
+# Archive ImportTasks run the per-member pipeline inline for every
+# member (import_archive_file: soft=30min/hard=35min). The ImportTask
+# 'processing' recovery sweep must give them a longer threshold or it
+# re-queues a legitimately-running archive mid-import (double-process).
+# 40 min = 5-min buffer past the archive task's hard kill.
+# Operator-flagged 2026-05-28 (target 1645019, a big archive).
+ARCHIVE_STUCK_THRESHOLD_MINUTES = 40
+
+# Poison-pill cap. After being recovered (re-queued from a stuck
+# 'processing' state) MAX_RECOVERY_ATTEMPTS-1 times, the next sweep
+# marks the row 'failed' instead of looping. 3 = two recoveries then
+# give up. A row reaches this only if it leaves NO terminal flip each
+# run — i.e. it hard-crashes the worker (OOM/segfault/SIGKILL), the
+# signature of a corrupt or oversized input. Caught exceptions already
+# flip to terminal 'failed' and never enter this loop.
+MAX_RECOVERY_ATTEMPTS = 3
+ORPHAN_PENDING_THRESHOLD_MINUTES = 30
+
+# DownloadEvent (pending|running) recovery threshold. download_source has
+# time_limit=1200s (20 min); 30 min is 10 min past that, so a legitimately-
+# running task is never killed by the sweep. Operator-confirmed 2026-05-29
+# after 43 sources stranded at "last check never" by the in-flight guard.
+DOWNLOAD_STALL_THRESHOLD_MINUTES = 30
+
 OLD_TASK_DAYS = 7
 PHASH_PAGE = 500
 VERIFY_PAGE = 200
 FFPROBE_TIMEOUT_SECONDS = 10
+TASK_RUN_KEEP_OK_SECONDS = 24 * 3600       # 24 h
+TASK_RUN_KEEP_FAILURE_SECONDS = 7 * 24 * 3600  # 7 days
+
+# Overrides for recover_stalled_task_runs (the TaskRun 'running' sweep).
+# Tasks/queues that legitimately run longer than the default 5-min
+# threshold need their own larger value, else the sweep marks in-flight
+# work 'error' before it finishes. Each value MUST be ≥ the relevant
+# task.time_limit + a small buffer. task_name overrides take precedence
+# over queue overrides.
+#
+# ml queue: tag_and_embed video branch (≈20 GPU ops); time_limit=1200.
+# import_archive_file: shares the 'import' queue with the fast
+#   single-file import_media_file, so it needs a task-name override
+#   (the import queue itself stays at the 5-min default for single
+#   files); time_limit=2100.
+QUEUE_STUCK_THRESHOLD_MINUTES: dict[str, int] = {
+    "ml": 25,
+}
+TASK_STUCK_THRESHOLD_MINUTES: dict[str, int] = {
+    "backend.app.tasks.import_file.import_archive_file": 40,
+}
 
 
 @celery.task(name="backend.app.tasks.maintenance.recover_interrupted_tasks")
 def recover_interrupted_tasks() -> int:
-    """Find ImportTask rows stuck in 'processing' for >30 min and re-queue them.
+    """Recover stuck ImportTask rows. Two distinct stuck states:
 
-    Why 30 min: large videos can legitimately take many minutes to import;
-    30 is a safe gate that catches actual crashes (which leave the row stuck
-    forever) without resetting slow-but-still-running jobs.
+    1. 'processing' too long — worker crash mid-import. Re-queue via
+       enqueue_import (routing media vs archive) and let the import
+       retry. Threshold is task-type-aware: media files are sub-second
+       and capped at the 5-min soft limit, so STUCK_THRESHOLD_MINUTES
+       (5) means a confirmed crash; archives run the per-member
+       pipeline inline (import_archive_file, 35-min hard limit) so they
+       get ARCHIVE_STUCK_THRESHOLD_MINUTES (40) to avoid re-queueing a
+       still-running archive. (Media was tightened from 30 min to 5
+       2026-05-24 after a 2224-row zombie pile; archive split out
+       2026-05-28.)
+
+    2. 'pending' or 'queued' > 30 min — enqueue-phase crash. scan_directory
+       creates rows with status='pending' (commit), then in a second pass
+       transitions to 'queued' and calls .delay() (commit). If the scanner
+       crashes between those two commits, rows are orphaned in 'pending'
+       (never enqueued) with no recovery path — invisible to the
+       'processing' sweep above. Flagged 2026-05-25 by operator hitting a
+       5490-row orphan pile. Flip these to 'failed' (not re-enqueue) so
+       the operator drains them via /api/import/retry-failed at their own
+       pace; bulk-re-enqueueing 5000+ rows would thundering-herd the
+       import worker.
+
+    Returns total rows touched (recovered + marked failed).
     """
     SessionLocal = _sync_session_factory()
-    cutoff = datetime.now(UTC) - timedelta(minutes=STUCK_THRESHOLD_MINUTES)
+    now = datetime.now(UTC)
+    media_cutoff = now - timedelta(minutes=STUCK_THRESHOLD_MINUTES)
+    archive_cutoff = now - timedelta(minutes=ARCHIVE_STUCK_THRESHOLD_MINUTES)
+    orphan_cutoff = now - timedelta(minutes=ORPHAN_PENDING_THRESHOLD_MINUTES)
     with SessionLocal() as session:
-        stuck_ids = session.execute(
-            select(ImportTask.id)
-            .where(ImportTask.status == "processing")
-            .where(ImportTask.started_at < cutoff)
-        ).scalars().all()
-
-        if not stuck_ids:
-            return 0
-
-        session.execute(
-            update(ImportTask)
-            .where(ImportTask.id.in_(stuck_ids))
-            .values(status="queued", started_at=None, error="recovered from stuck state")
+        # Both sweeps used to be SELECT ids → UPDATE WHERE id IN (...) which
+        # blew past psycopg's 65535-parameter ceiling once a sweep covered
+        # tens of thousands of rows (operator hit it 2026-05-26 after the
+        # /import deep scan piled up orphans). Folding the SELECT into the
+        # UPDATE eliminates the IN-list entirely. RETURNING gives us back
+        # exactly the (id, task_type) pairs that flipped so the requeue
+        # can route media vs archive correctly.
+        #
+        # Media + archive get separate cutoffs: a single media file is
+        # sub-second so 5 min means crash; an archive runs the per-member
+        # pipeline inline and can legitimately take up to its 35-min hard
+        # limit, so it gets ARCHIVE_STUCK_THRESHOLD_MINUTES (40) to avoid
+        # re-queueing a still-running archive.
+        stuck_predicate = and_(
+            ImportTask.status == "processing",
+            or_(
+                and_(ImportTask.task_type != "archive",
+                     ImportTask.started_at < media_cutoff),
+                and_(ImportTask.task_type == "archive",
+                     ImportTask.started_at < archive_cutoff),
+            ),
         )
+
+        # POISON-PILL CIRCUIT BREAKER (Layer 1, 2026-05-28). A row that
+        # leaves no terminal flip (hard worker crash: OOM/segfault/SIGKILL
+        # on a corrupt or oversized input) gets re-queued by this sweep —
+        # and would loop forever, re-crashing the worker each pass,
+        # without a cap. Once a row has already been recovered
+        # MAX_RECOVERY_ATTEMPTS-1 times, stop re-queueing it and mark it
+        # 'failed' with a diagnostic so the operator can find + replace
+        # the offending file. This UPDATE runs FIRST so the rows it
+        # claims drop out of 'processing' before the re-queue pass.
+        poison_result = session.execute(
+            update(ImportTask)
+            .where(stuck_predicate)
+            .where(ImportTask.recovery_count >= MAX_RECOVERY_ATTEMPTS - 1)
+            .values(
+                status="failed",
+                finished_at=now,
+                error=(
+                    f"crashed or stalled the worker {MAX_RECOVERY_ATTEMPTS} "
+                    f"times without completing — likely a corrupt or "
+                    f"oversized input. Not re-queued. Inspect/replace the "
+                    f"file, then retry via /api/import/retry-failed."
+                ),
+            )
+            .returning(ImportTask.id)
+        )
+        poison_ids = [r[0] for r in poison_result.all()]
+
+        # Re-queue the remaining stuck rows (under the cap) and bump
+        # their recovery_count. RETURNING (id, task_type) so the requeue
+        # routes media vs archive correctly.
+        stuck_result = session.execute(
+            update(ImportTask)
+            .where(stuck_predicate)
+            .where(ImportTask.recovery_count < MAX_RECOVERY_ATTEMPTS - 1)
+            .values(
+                status="queued",
+                started_at=None,
+                recovery_count=ImportTask.recovery_count + 1,
+                error="recovered from stuck state",
+            )
+            .returning(ImportTask.id, ImportTask.task_type)
+        )
+        stuck = stuck_result.all()
+
+        orphan_result = session.execute(
+            update(ImportTask)
+            .where(ImportTask.status.in_(["pending", "queued"]))
+            .where(ImportTask.created_at < orphan_cutoff)
+            .values(
+                status="failed",
+                error=(
+                    "orphan pending/queued swept by recover_interrupted_tasks "
+                    "(scanner likely crashed mid-enqueue); retry via "
+                    "/api/import/retry-failed"
+                ),
+            )
+        )
+        orphan_count = orphan_result.rowcount or 0
+
         session.commit()
 
-        from .import_file import import_media_file
-        for tid in stuck_ids:
-            import_media_file.delay(tid)
+        if stuck:
+            from .import_file import enqueue_import
+            for tid, task_type in stuck:
+                enqueue_import(tid, task_type)
 
-        return len(stuck_ids)
+        # Layer-2 auto re-download (env-gated, default OFF). For each
+        # poison-pill row that resolves to a pollable Source, delete the
+        # bad file and trigger ONE source re-check to fetch a fresh
+        # copy. Bounded by ImportTask.refetched so source-side
+        # corruption can't loop. The 'failed' row stays as history; the
+        # re-downloaded file re-imports as a fresh task on the next scan.
+        if poison_ids and os.environ.get("FC_AUTO_REFETCH_CORRUPT", "0") == "1":
+            from ..models import ImportSettings
+            from ..services.refetch_service import attempt_refetch
+            import_root = Path(session.execute(
+                select(ImportSettings.import_scan_path)
+                .where(ImportSettings.id == 1)
+            ).scalar_one())
+            for pid in poison_ids:
+                ptask = session.get(ImportTask, pid)
+                if ptask is None:
+                    continue
+                try:
+                    attempt_refetch(session, ptask, import_root)
+                except Exception as exc:  # noqa: BLE001 — best-effort
+                    log.warning("auto-refetch failed for task %s: %s", pid, exc)
+
+        return len(stuck) + len(poison_ids) + orphan_count
 
 
 @celery.task(name="backend.app.tasks.maintenance.cleanup_old_tasks")
@@ -76,6 +243,110 @@ def cleanup_old_tasks() -> int:
         return result.rowcount or 0
 
 
+@celery.task(name="backend.app.tasks.maintenance.recover_stalled_task_runs")
+def recover_stalled_task_runs() -> int:
+    """Flip task_run rows stuck in 'running' past their queue-specific
+    threshold to 'error'. FC-3i.
+
+    A row gets stuck when the worker dies without emitting
+    task_postrun / task_failure (e.g. OOM, container restart between
+    signals, signal handler raised+logged). The default 5-min threshold
+    fits short-lived queues (import/thumbnail/download); queues that
+    legitimately run longer tasks (ml-video, deep scans) get their
+    own larger threshold via QUEUE_STUCK_THRESHOLD_MINUTES so the
+    sweep doesn't preempt them.
+
+    Runs once per distinct threshold value: each pass updates rows
+    whose queue maps to that threshold.
+    """
+    SessionLocal = _sync_session_factory()
+    now = datetime.now(UTC)
+    override_tasks = set(TASK_STUCK_THRESHOLD_MINUTES.keys())
+    override_queues = set(QUEUE_STUCK_THRESHOLD_MINUTES.keys())
+    total = 0
+
+    def _flag(minutes, *extra_where):
+        cutoff = now - timedelta(minutes=minutes)
+        stmt = (
+            update(TaskRun)
+            .where(TaskRun.status == "running")
+            .where(TaskRun.started_at < cutoff)
+            .values(
+                status="error",
+                error_type="RecoverySweep",
+                error_message=(
+                    f"no completion signal received within {minutes} min"
+                ),
+                finished_at=now,
+            )
+        )
+        for w in extra_where:
+            stmt = stmt.where(w)
+        return session.execute(stmt).rowcount or 0
+
+    with SessionLocal() as session:
+        # Precedence: task_name override → queue override → default.
+        # Each pass excludes rows claimed by a higher-precedence pass so
+        # every row is touched at most once.
+
+        # 1. Per-task-name overrides (e.g. import_archive_file, which
+        #    shares the 'import' queue with fast single-file imports).
+        for task_name, minutes in TASK_STUCK_THRESHOLD_MINUTES.items():
+            total += _flag(minutes, TaskRun.task_name == task_name)
+
+        # 2. Per-queue overrides, excluding the override task-names.
+        for queue, minutes in QUEUE_STUCK_THRESHOLD_MINUTES.items():
+            wheres = [TaskRun.queue == queue]
+            if override_tasks:
+                wheres.append(TaskRun.task_name.notin_(override_tasks))
+            total += _flag(minutes, *wheres)
+
+        # 3. Default — everything not claimed above.
+        default_wheres = []
+        if override_queues:
+            default_wheres.append(TaskRun.queue.notin_(override_queues))
+        if override_tasks:
+            default_wheres.append(TaskRun.task_name.notin_(override_tasks))
+        total += _flag(STUCK_THRESHOLD_MINUTES, *default_wheres)
+
+        session.commit()
+        return total
+
+
+@celery.task(name="backend.app.tasks.maintenance.prune_task_runs")
+def prune_task_runs() -> dict:
+    """Daily retention for task_run rows. FC-3i.
+
+    - 'ok' rows: deleted after TASK_RUN_KEEP_OK_SECONDS (24h default).
+      Success is high-volume, not interesting after a day.
+    - 'error' / 'timeout' rows: deleted after TASK_RUN_KEEP_FAILURE_SECONDS
+      (7 days default). Failures are operationally interesting longer.
+    - 'running' rows: NEVER deleted by this task. The recovery sweep
+      (recover_stalled_task_runs) is the mechanism that flips them to
+      terminal state; prune doesn't touch in-flight state.
+    - 'retry' rows: treated as failures (>7d).
+
+    Returns dict of how many rows were deleted in each bucket.
+    """
+    SessionLocal = _sync_session_factory()
+    now = datetime.now(UTC)
+    ok_cutoff = now - timedelta(seconds=TASK_RUN_KEEP_OK_SECONDS)
+    fail_cutoff = now - timedelta(seconds=TASK_RUN_KEEP_FAILURE_SECONDS)
+    with SessionLocal() as session:
+        ok_deleted = session.execute(
+            delete(TaskRun)
+            .where(TaskRun.status == "ok")
+            .where(TaskRun.finished_at < ok_cutoff)
+        ).rowcount or 0
+        fail_deleted = session.execute(
+            delete(TaskRun)
+            .where(TaskRun.status.in_(["error", "timeout", "retry"]))
+            .where(TaskRun.finished_at < fail_cutoff)
+        ).rowcount or 0
+        session.commit()
+        return {"ok_deleted": ok_deleted, "failures_deleted": fail_deleted}
+
+
 @celery.task(name="backend.app.tasks.maintenance.backfill_phash")
 def backfill_phash() -> int:
     """Recompute phash for stored images that have none (imported before
@@ -191,6 +462,65 @@ def verify_integrity() -> int:
     return total
 
 
+@celery.task(name="backend.app.tasks.maintenance.recover_stalled_download_events")
+def recover_stalled_download_events() -> int:
+    """Recover DownloadEvent rows stuck pending/running past the worker hard kill.
+
+    The scan tick (scheduler_service.select_due_sources →
+    tasks.scan._tick_due_sources_async) inserts DownloadEvent(status='pending')
+    and fires download_source.delay(). If that task dies before finalizing the
+    event — worker OOM/SIGKILL, lost task, or a gallery-dl that didn't unwind
+    on the 1200s hard time_limit — the event stays in-flight forever. The next
+    tick then skips that source because of the in-flight guard (scan.py:168)
+    and Source.last_checked_at never updates; the operator sees "last check
+    never" in the Subscriptions health column, permanently.
+
+    This sweep flips matching events to 'error', stamps each affected Source's
+    last_checked_at + last_error and bumps consecutive_failures (once per
+    source, not per event — backoff is exponential on that count so an N-event
+    bump would inflate the next interval by 2^N for no reason). The source
+    becomes re-queueable on the next tick and the health dot goes amber.
+
+    Operator-confirmed 2026-05-29 (43-row strand pile in production).
+    """
+    SessionLocal = _sync_session_factory()
+    now = datetime.now(UTC)
+    cutoff = now - timedelta(minutes=DOWNLOAD_STALL_THRESHOLD_MINUTES)
+    msg = "stranded by recovery sweep (no terminal status after time_limit)"
+    with SessionLocal() as session:
+        # UPDATE...RETURNING the source_ids in one round trip — keeps us off
+        # the psycopg 65535-param ceiling that SELECT-then-UPDATE-WHERE-IN
+        # would hit on a large strand pile.
+        result = session.execute(
+            update(DownloadEvent)
+            .where(DownloadEvent.status.in_(["pending", "running"]))
+            .where(DownloadEvent.started_at < cutoff)
+            .values(status="error", finished_at=now, error=msg)
+            .returning(DownloadEvent.source_id)
+        )
+        returned = result.all()
+        if not returned:
+            session.commit()
+            return 0
+        events_recovered = len(returned)
+        source_ids = list({row.source_id for row in returned})
+        session.execute(
+            update(Source)
+            .where(Source.id.in_(source_ids))
+            .values(
+                consecutive_failures=Source.consecutive_failures + 1,
+                last_error=msg,
+                last_checked_at=now,
+            )
+        )
+        session.commit()
+        log.info(
+            "recover_stalled_download_events: recovered %d events across %d sources",
+            events_recovered, len(source_ids),
+        )
+        return events_recovered
+
+
 @celery.task(name="backend.app.tasks.maintenance.cleanup_old_download_events")
 def cleanup_old_download_events() -> int:
     """FC-3d: delete terminal DownloadEvent rows older than the configured
@@ -203,9 +533,7 @@ def cleanup_old_download_events() -> int:
     """
     SessionLocal = _sync_session_factory()
     with SessionLocal() as session:
-        settings = session.execute(
-            select(ImportSettings).where(ImportSettings.id == 1)
-        ).scalar_one()
+        settings = ImportSettings.load_sync(session)
         retention_days = settings.download_event_retention_days
         cutoff = datetime.now(UTC) - timedelta(days=retention_days)
         result = session.execute(

backend/app/tasks/migration.py

@@ -1,180 +0,0 @@
-"""FC-5 run_migration Celery task.
-
-Dispatches to the right migrator based on `kind`. Updates MigrationRun
-row's status/counts/finished_at as it runs. Failures set status='error'
-with the error message preserved.
-
-kinds: backup, gs_ingest, ir_ingest, tag_apply, ml_queue, verify, rollback
-"""
-from __future__ import annotations
-
-import asyncio
-import logging
-from datetime import UTC, datetime
-from pathlib import Path
-
-from sqlalchemy import select
-from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
-
-from ..celery_app import celery
-from ..config import get_config
-from ..models import MigrationRun
-from ..services.credential_crypto import CredentialCrypto
-from ..services.migrators import backup as backup_mod
-from ..services.migrators import gs_ingest, ir_ingest, ml_queue, tag_apply, verify
-from ..services.migrators import rollback as rollback_mod
-
-log = logging.getLogger(__name__)
-
-IMAGES_ROOT = Path("/images")
-_KEY_PATH = IMAGES_ROOT / "secrets" / "credential_key.b64"
-
-
-def _async_session_factory():
-    cfg = get_config()
-    engine = create_async_engine(cfg.database_url, future=True, pool_pre_ping=True)
-    return async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False), engine
-
-
-async def _update_run(
-    db: AsyncSession, run_id: int, *,
-    status: str | None = None, counts: dict | None = None,
-    error: str | None = None, finished_at: datetime | None = None,
-    metadata_patch: dict | None = None,
-) -> None:
-    run = (await db.execute(
-        select(MigrationRun).where(MigrationRun.id == run_id)
-    )).scalar_one()
-    if status is not None:
-        run.status = status
-    if counts is not None:
-        run.counts = counts
-    if error is not None:
-        run.error = error
-    if finished_at is not None:
-        run.finished_at = finished_at
-    if metadata_patch:
-        run.metadata_ = {**(run.metadata_ or {}), **metadata_patch}
-    await db.commit()
-
-
-async def _run_async(run_id: int, kind: str, params: dict) -> dict:
-    factory, engine = _async_session_factory()
-    try:
-        async with factory() as db:
-            await _update_run(db, run_id, status="running")
-            try:
-                if kind == "backup":
-                    manifest = backup_mod.create_backup(
-                        db_url=get_config().database_url_sync,
-                        images_root=IMAGES_ROOT,
-                        tag=params.get("tag", "manual"),
-                    )
-                    await _update_run(
-                        db, run_id, status="ok",
-                        counts={"rows_processed": 0, "rows_inserted": 0,
-                                "rows_skipped": 0, "files_copied": 0,
-                                "bytes_copied": 0, "conflicts": 0},
-                        finished_at=datetime.now(UTC),
-                        metadata_patch={"manifest": manifest},
-                    )
-                    return manifest
-
-                elif kind == "gs_ingest":
-                    fc_crypto = CredentialCrypto(_KEY_PATH)
-                    counts = await gs_ingest.migrate_async(
-                        db, data=params["data"],
-                        fc_crypto=fc_crypto,
-                        dry_run=params.get("dry_run", False),
-                    )
-                    await _update_run(
-                        db, run_id, status="ok", counts=counts,
-                        finished_at=datetime.now(UTC),
-                    )
-                    return counts
-
-                elif kind == "ir_ingest":
-                    counts = await ir_ingest.migrate_async(
-                        db, data=params["data"],
-                        images_root=IMAGES_ROOT,
-                        dry_run=params.get("dry_run", False),
-                    )
-                    await _update_run(
-                        db, run_id, status="ok", counts=counts,
-                        finished_at=datetime.now(UTC),
-                    )
-                    return counts
-
-                elif kind == "tag_apply":
-                    result = await tag_apply.apply_async(
-                        db, images_root=IMAGES_ROOT,
-                        dry_run=params.get("dry_run", False),
-                    )
-                    await _update_run(
-                        db, run_id, status="ok",
-                        counts=result["counts"],
-                        finished_at=datetime.now(UTC),
-                        metadata_patch={"unmatched": result["unmatched"]},
-                    )
-                    return result
-
-                elif kind == "ml_queue":
-                    count = await ml_queue.queue_all_unprocessed_async(db)
-                    await _update_run(
-                        db, run_id, status="ok",
-                        counts={"rows_processed": count, "rows_inserted": 0,
-                                "rows_skipped": 0, "files_copied": 0,
-                                "bytes_copied": 0, "conflicts": 0},
-                        finished_at=datetime.now(UTC),
-                    )
-                    return {"queued": count}
-
-                elif kind == "verify":
-                    checks = await verify.verify_async(db, expected=params.get("expected"))
-                    sample = await verify.verify_sha256_sample(
-                        db, sample_size=params.get("sample_size", 20),
-                    )
-                    await _update_run(
-                        db, run_id, status="ok",
-                        counts={"rows_processed": sample["sample_size"],
-                                "rows_inserted": 0, "rows_skipped": 0,
-                                "files_copied": 0, "bytes_copied": 0,
-                                "conflicts": sample["mismatched"] + sample["missing"]},
-                        finished_at=datetime.now(UTC),
-                        metadata_patch={"checks": checks, "sample": sample},
-                    )
-                    return {"checks": checks, "sample": sample}
-
-                elif kind == "rollback":
-                    result = rollback_mod.rollback_to_pre_migration(
-                        db_url=get_config().database_url_sync,
-                        images_root=IMAGES_ROOT,
-                    )
-                    await _update_run(
-                        db, run_id, status="ok",
-                        counts={"rows_processed": 0, "rows_inserted": 0,
-                                "rows_skipped": 0, "files_copied": 0,
-                                "bytes_copied": 0, "conflicts": 0},
-                        finished_at=datetime.now(UTC),
-                        metadata_patch={"rollback_result": result},
-                    )
-                    return result
-
-                else:
-                    raise ValueError(f"unknown kind: {kind}")
-
-            except Exception as exc:
-                log.exception("migration kind=%s failed", kind)
-                await _update_run(
-                    db, run_id, status="error", error=str(exc),
-                    finished_at=datetime.now(UTC),
-                )
-                raise
-    finally:
-        await engine.dispose()
-
-
-@celery.task(name="backend.app.tasks.migration.run_migration", bind=True, acks_late=True)
-def run_migration(self, run_id: int, kind: str, params: dict) -> dict:
-    """FC-5: dispatch a migration kind. Updates MigrationRun row as it goes."""
-    return asyncio.run(_run_async(run_id, kind, params))

backend/app/tasks/ml.py

@@ -9,6 +9,7 @@ apply_allowlist_tags sweeps which are 'maintenance' lane. Sync sessions
 from pathlib import Path
 
 from sqlalchemy import select
+from sqlalchemy.exc import DBAPIError, OperationalError
 
 from ..celery_app import celery
 from ..models import ImageRecord, MLSettings
@@ -22,7 +23,24 @@ def _is_video(path: Path) -> bool:
     return path.suffix.lower() in VIDEO_EXTS
 
 
-@celery.task(name="backend.app.tasks.ml.tag_and_embed", bind=True)
+@celery.task(
+    name="backend.app.tasks.ml.tag_and_embed",
+    bind=True,
+    autoretry_for=(OperationalError, DBAPIError, OSError),
+    retry_backoff=5,
+    retry_backoff_max=60,
+    retry_jitter=True,
+    max_retries=3,
+    # Sized for the video branch: sample 10 frames, run tagger +
+    # embedder on each (≈20 GPU ops vs 2 for an image). A loaded
+    # ml-worker can take 5-10 min on a long video; bumped from
+    # 5min/7min on 2026-05-28 after operator-flagged image 6288 (a
+    # .mp4) hit the recovery sweep at 5 min while still legitimately
+    # processing. Image runs return in seconds; the bump doesn't
+    # affect their UX.
+    soft_time_limit=900,   # 15 min
+    time_limit=1200,       # 20 min hard
+)
 def tag_and_embed(self, image_id: int) -> dict:
     """Run Camie + SigLIP on one image; store predictions + embedding;
     then enqueue per-image allowlist application.
@@ -53,6 +71,18 @@ def tag_and_embed(self, image_id: int) -> dict:
         embedder = get_embedder()
 
         if _is_video(src):
+            # Layer-3 isolation: ffprobe (a separate process) validates
+            # the container before we burn ~20 GPU ops sampling frames
+            # from it. A corrupt video that would crash the frame
+            # decoder is rejected cleanly here instead of taking down
+            # the ml-worker. Operator-flagged 2026-05-28.
+            from ..utils import safe_probe
+            vprobe = safe_probe.probe_video(src)
+            if not vprobe.ok:
+                return {
+                    "status": "bad_video", "image_id": image_id,
+                    "reason": vprobe.reason,
+                }
             frames = _sample_video_frames(
                 src, int(os.environ.get("VIDEO_ML_FRAMES", "10"))
             )

backend/app/tasks/scan.py

@@ -12,12 +12,12 @@ from datetime import UTC, datetime
 from pathlib import Path
 
 from sqlalchemy import select
-from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
 
 from ..celery_app import celery
-from ..config import get_config
 from ..models import DownloadEvent, ImportBatch, ImportSettings, ImportTask
-from ..services.scheduler_service import select_due_sources
+from ..services.archive_extractor import is_archive
+from ..services.scheduler_service import record_tick, select_due_sources
+from ._async_session import async_session_factory
 from ._sync_engine import sync_session_factory as _sync_session_factory
 
 
@@ -44,9 +44,7 @@ def scan_directory(self, triggered_by: str = "manual",
     batch id."""
     SessionLocal = _sync_session_factory()
     with SessionLocal() as session:
-        settings = session.execute(
-            select(ImportSettings).where(ImportSettings.id == 1)
-        ).scalar_one()
+        settings = ImportSettings.load_sync(session)
         import_root = Path(settings.import_scan_path)
 
         batch = ImportBatch(
@@ -59,16 +57,25 @@ def scan_directory(self, triggered_by: str = "manual",
         session.flush()
         batch_id = batch.id
 
-        # Skip-set: any source_path that already has a non-failed ImportTask
-        # row. Re-running scan_directory must not re-enqueue files the
-        # importer has already handled (or is currently handling); doing so
-        # creates duplicate work and inflates the queue. Failed prior tasks
-        # are eligible for retry.
+        # Skip-set behavior splits by mode (operator-flagged 2026-05-25):
+        #
+        # quick: any non-failed prior ImportTask (active OR finished) is
+        #   skipped — quick scan only does new-file enqueue, so re-touching
+        #   already-imported files is wasted work.
+        #
+        # deep: ONLY currently-in-flight tasks (pending/queued/processing)
+        #   are skipped. Completed and skipped tasks ARE re-queued because
+        #   deep scan exists precisely to re-touch already-imported files
+        #   (refresh sidecar metadata, fill NULL phash, fill NULL artist
+        #   via Importer._deep_rederive). Matches IR's deep-scan behavior.
+        active_statuses = ["pending", "queued", "processing"]
+        if mode == "deep":
+            skip_statuses = active_statuses
+        else:
+            skip_statuses = active_statuses + ["complete", "skipped"]
         non_failed_existing = set(session.execute(
             select(ImportTask.source_path).where(
-                ImportTask.status.in_(
-                    ["pending", "queued", "processing", "complete", "skipped"]
-                ),
+                ImportTask.status.in_(skip_statuses),
             )
         ).scalars().all())
 
@@ -87,7 +94,9 @@ def scan_directory(self, triggered_by: str = "manual",
             task = ImportTask(
                 batch_id=batch_id,
                 source_path=entry_str,
-                task_type="media",
+                # Archives route to import_archive_file (larger time
+                # budget) — they run the per-member pipeline inline.
+                task_type="archive" if is_archive(entry) else "media",
                 status="pending",
                 size_bytes=size,
             )
@@ -95,17 +104,27 @@ def scan_directory(self, triggered_by: str = "manual",
             files_seen += 1
 
         batch.total_files = files_seen
+        # If the walk enqueued nothing (every file was already on a
+        # non-failed ImportTask from a prior scan), there's no
+        # import_media_file message that would ever flip this batch to
+        # 'complete' — finalize it now so the active-batch query in
+        # /api/system/stats doesn't get stuck reporting all-zero
+        # counters from an empty scan.
+        if files_seen == 0:
+            batch.status = "complete"
+            batch.finished_at = datetime.now(UTC)
         session.commit()
 
-        # Now enqueue import_media_file for each pending task.
+        # Now enqueue each pending task on the right Celery task
+        # (media vs archive) via the shared router.
+        from .import_file import enqueue_import
+
         for task in session.execute(
             select(ImportTask).where(ImportTask.batch_id == batch_id)
         ).scalars():
             task.status = "queued"
             session.add(task)
-            from .import_file import import_media_file
-
-            import_media_file.delay(task.id)
+            enqueue_import(task.id, task.task_type)
         session.commit()
 
         if mode == "deep":
@@ -119,16 +138,12 @@ def scan_directory(self, triggered_by: str = "manual",
 # --- FC-3d: periodic source-check tick ------------------------------------
 
 
-def _async_session_factory():
-    cfg = get_config()
-    engine = create_async_engine(cfg.database_url, future=True, pool_pre_ping=True)
-    return async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False), engine
-
-
 async def _tick_due_sources_async() -> dict:
-    factory, engine = _async_session_factory()
+    factory, engine = async_session_factory()
     try:
         async with factory() as session:
+            # Prove the scheduler is alive even on empty ticks (UI reads this).
+            await record_tick(session)
             due = await select_due_sources(session)
             if not due:
                 return {

backend/app/tasks/thumbnail.py

@@ -7,6 +7,8 @@ so they deserve their own queue lane.
 
 from pathlib import Path
 
+from sqlalchemy.exc import DBAPIError, OperationalError
+
 from ..celery_app import celery
 from ..models import ImageRecord
 from ..services.importer import is_video
@@ -15,8 +17,42 @@ from ._sync_engine import sync_session_factory as _sync_session_factory
 
 IMAGES_ROOT = Path("/images")
 
+THUMB_MAGIC_JPEG = b"\xff\xd8\xff"
+THUMB_MAGIC_PNG = b"\x89PNG\r\n\x1a\n"
 
-@celery.task(name="backend.app.tasks.thumbnail.generate_thumbnail", bind=True)
+
+def _thumb_is_valid(path: Path) -> bool:
+    """Return True iff `path` exists and starts with a JPEG or PNG magic header.
+
+    The on-disk thumbnail format is set by services/thumbnailer.py — JPEG for
+    opaque sources, PNG for alpha sources. Anything else (missing file, OSError,
+    truncated, wrong magic) is invalid.
+    """
+    try:
+        with path.open("rb") as f:
+            head = f.read(12)
+    except OSError:
+        return False
+    if len(head) < 8:
+        return False
+    if head[:3] == THUMB_MAGIC_JPEG:
+        return True
+    if head[:8] == THUMB_MAGIC_PNG:
+        return True
+    return False
+
+
+@celery.task(
+    name="backend.app.tasks.thumbnail.generate_thumbnail",
+    bind=True,
+    autoretry_for=(OperationalError, DBAPIError, OSError),
+    retry_backoff=5,
+    retry_backoff_max=60,
+    retry_jitter=True,
+    max_retries=3,
+    soft_time_limit=120,
+    time_limit=180,
+)
 def generate_thumbnail(self, image_id: int) -> dict:
     SessionLocal = _sync_session_factory()
     with SessionLocal() as session:
@@ -38,3 +74,58 @@ def generate_thumbnail(self, image_id: int) -> dict:
         session.add(record)
         session.commit()
         return {"status": "ok", "image_id": image_id, "path": str(result.path)}
+
+
+@celery.task(
+    name="backend.app.tasks.thumbnail.backfill_thumbnails",
+    bind=True,
+)
+def backfill_thumbnails(self) -> dict:
+    """Scan ImageRecord and enqueue generate_thumbnail for rows whose
+    thumbnail is missing, gone from disk, or has wrong magic bytes.
+
+    Keyset paginates by id ASC, page size 500. NULLs out thumbnail_path for
+    rows that point at a missing or corrupt file before enqueueing — keeps
+    the DB self-consistent on partial runs and makes re-runs safe.
+
+    Returns {"enqueued": N, "ok": M, "regenerated": K} where:
+      - enqueued = total generate_thumbnail.delay() calls
+      - ok       = rows whose existing thumbnail file is valid (skipped)
+      - regenerated = subset of enqueued that had a non-NULL thumbnail_path
+                      cleared (i.e. missing + corrupt)
+    """
+    from sqlalchemy import select, update
+
+    SessionLocal = _sync_session_factory()
+    enqueued = 0
+    ok = 0
+    regenerated = 0
+    last_id = 0
+    with SessionLocal() as session:
+        while True:
+            rows = session.execute(
+                select(ImageRecord.id, ImageRecord.thumbnail_path)
+                .where(ImageRecord.id > last_id)
+                .order_by(ImageRecord.id.asc())
+                .limit(500)
+            ).all()
+            if not rows:
+                break
+            for image_id, thumb_path in rows:
+                if thumb_path is None:
+                    generate_thumbnail.delay(image_id)
+                    enqueued += 1
+                elif _thumb_is_valid(Path(thumb_path)):
+                    ok += 1
+                else:
+                    session.execute(
+                        update(ImageRecord)
+                        .where(ImageRecord.id == image_id)
+                        .values(thumbnail_path=None)
+                    )
+                    generate_thumbnail.delay(image_id)
+                    enqueued += 1
+                    regenerated += 1
+            session.commit()
+            last_id = rows[-1][0]
+    return {"enqueued": enqueued, "ok": ok, "regenerated": regenerated}

backend/app/utils/_archive_probe_runner.py

@@ -0,0 +1,33 @@
+"""Subprocess entrypoint for safe_probe.probe_archive — see safe_probe.py.
+
+probe_archive spawns this via subprocess (not multiprocessing.Process)
+because Celery's prefork worker pool runs tasks in DAEMON processes and
+Python's multiprocessing forbids daemon processes from spawning children
+("AssertionError: daemonic processes are not allowed to have children",
+operator-flagged 2026-05-30 — every archive import failed at task
+startup). subprocess has no such restriction; we still get crash-
+isolation because a probe segfault/OOM exits non-zero rather than
+killing the worker.
+
+Prints a single JSON line on stdout: {"status": "ok"|"error",
+"detail": "..."?}. Exit code 0 for clean outcomes; non-zero exit
+(signal / OOM-kill / unhandled exception) is the poison-pill signature
+the parent maps to ProbeResult(crashed=True).
+"""
+import json
+import sys
+
+from .safe_probe import _run_probe
+
+
+def main() -> int:
+    if len(sys.argv) != 2:
+        print(json.dumps({"status": "error", "detail": "usage: <path>"}))
+        return 2
+    status, detail = _run_probe(sys.argv[1])
+    print(json.dumps({"status": status, "detail": detail}))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

backend/app/utils/phash.py

@@ -13,8 +13,21 @@ HASH_SIZE = 8
 
 def compute_phash(pil_image) -> str | None:
     """Perceptual hash of an opened PIL image, as a hex string. None on any
-    failure (videos/unreadable/non-image)."""
+    failure (videos/unreadable/non-image).
+
+    For animated images (multi-frame WebP/GIF/APNG), explicitly seek to
+    frame 0 first. Without this, some PIL operations downstream of
+    imagehash.phash (convert("L"), resize) can iterate all frames and
+    blow past Celery's hard time limit on large animations
+    (operator-flagged 2026-05-26 against animated WebPs). The pHash of
+    frame 0 is the conventional choice for animated content.
+    """
     try:
+        if getattr(pil_image, "is_animated", False):
+            try:
+                pil_image.seek(0)
+            except Exception:
+                pass
         return str(imagehash.phash(pil_image, hash_size=HASH_SIZE))
     except Exception:
         return None