Merge pull request 'The weekly refresh now publishes only what a gate has proven — and a new install can start' (#249) from dev into main

The gate held. Run 5320, dispatched with the forced failure in place:

  build-web    success   (candidate published)
  build-ml     success
  build-agent  success
  smoke-web    FAILED
  promote      skipped
  run          failure

And the three channel tags did not move:

  fabledcurator        33d3d8332f74 -> 33d3d8332f74
  fabledcurator-ml     e94a5435cb45 -> e94a5435cb45
  fabledcurator-agent  bae27d34d811 -> bae27d34d811

So a refresh that breaks something now leaves :latest naming the build that
works, which is the property milestone 362 exists to establish. The rejected
candidate is still published under :refresh-candidate, so whoever reads the
red job on Monday can pull the exact image that failed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA

.env.example

@@ -34,6 +34,34 @@ DB_PASSWORD=
 SECRET_KEY=
 
 
+# ---------------------------------------------------------------------------
+# FIRST BOOT ONLY — then delete this line
+# ---------------------------------------------------------------------------
+
+# FabledCurator encrypts your stored platform credentials with a Fernet key it
+# keeps at /images/secrets/credential_key.b64 — inside the ./images bind mount,
+# so it outlives the container. On a brand-new install that file does not exist
+# yet, and the app REFUSES TO START rather than quietly create one:
+#
+#     MissingCredentialKey: Fernet key file not found at
+#     /images/secrets/credential_key.b64
+#
+# That refusal is deliberate. Auto-creating a key is indistinguishable from the
+# disaster case — a restore that brought the database back but lost
+# ./images/secrets — and there it would mint a key that cannot decrypt anything,
+# leaving an instance that looks healthy while every paywalled download fails.
+# So the choice is yours to make explicitly, once.
+#
+# Set this for your first `up`, watch the container come up, then DELETE THE
+# LINE. Leaving it set disarms the protection permanently, on an instance that
+# by then has credentials worth protecting.
+#
+# BACK UP ./images/secrets/ ALONGSIDE YOUR DATABASE. The key is the only thing
+# that can read your stored credentials; a database restored without it needs
+# every credential re-entered by hand.
+CURATOR_BOOTSTRAP_NEW_KEY=1
+
+
 # ---------------------------------------------------------------------------
 # Optional — defaults are fine
 # ---------------------------------------------------------------------------

.forgejo/workflows/build.yml

@@ -452,6 +452,18 @@ jobs:
        # to. Same source of truth; no double-store.
 
   build-web:
+    # Consumed by smoke-web's job-level `if:`. It cannot read `env` — the env
+    # context is available to STEP `if:` and step bodies, never to a job's own
+    # condition, and an unresolvable context there is empty rather than an
+    # error. `smoke-web` skipped silently on run 5290 for exactly that reason.
+    #
+    # Keying off the reuse step's own output is better than re-deriving the
+    # trigger anyway: it is the same single decision the build, the XPI
+    # download and the promote all take (build.yml's "one decision drives
+    # everything downstream"), and it says the thing smoke-web actually needs
+    # to know — a candidate was published — rather than restating why.
+    outputs:
+      candidate: ${{ steps.reuse.outputs.promote }}
     # A plain `needs` — no `always()`. That expression existed to let a
     # SKIPPED sign-extension through on a tag push while still blocking a
     # FAILED one. With no tag trigger, sign-extension always runs, so the
@@ -722,10 +734,16 @@ jobs:
           # already allows for :buildcache, not the per-build tag family that
           # milestone 318 withdrew.
           #
-          # Both values are decided HERE, beside `hit`, for the reason the
+          # Decided HERE, beside `hit`, for the reason the force/schedule
-          # force/schedule branch below gives: one step decides what this job
+          # branch below gives: one step decides what this job does. A
-          # does. A promote condition derived independently could disagree with
+          # condition derived independently could disagree with the tag the
-          # the tag the build actually wrote.
+          # build actually wrote.
+          #
+          # build-web additionally exposes this as `outputs.candidate`, which is
+          # what gates the `promote` job — a job's `if:` cannot read `env`, and
+          # one flag is enough because all three derive it from the same
+          # IS_REFRESH. ml and agent do not re-emit it; a second copy nothing
+          # reads is the kind of thing that later reads as load-bearing.
           if [ "${IS_REFRESH:-}" = "true" ]; then
             echo "build_ref=$IMAGE:refresh-candidate" >> "$GITHUB_OUTPUT"
             echo "promote=true" >> "$GITHUB_OUTPUT"
@@ -947,77 +965,6 @@ jobs:
             FC_CHANNEL=${{ steps.tag.outputs.channel }}
             FC_VERSION=${{ steps.reuse.outputs.version }}
 
-      # Point the channel tag at the candidate the refresh just built.
-      #
-      # Unconditional TODAY, so this milestone never leaves the refresh in a
-      # state where it builds and publishes nothing. Step 4 wraps it in the
-      # smoke suite's verdict; until then the scheduled path behaves exactly
-      # as it did, just via two operations instead of one.
-      #
-      # NOT `imagetools create`. That wraps its source in an INDEX, and an
-      # indexed channel tag is the one thing this pipeline cannot survive:
-      # `.Image.Config.Labels` does not resolve through an index, so the
-      # fc.revision the reuse check reads off the channel tag would come back
-      # empty, every subsequent push would miss and rebuild, and nothing would
-      # go red. That is #3183, observed on run 4751 — reuse worked exactly once
-      # and the only symptom was the bill. The repoint step below excludes its
-      # own source tag for precisely this reason; a promote that re-introduced
-      # the wrap through a different door would undo that care.
-      #
-      # A manifest PUT is what "make this tag name that image" means at the
-      # registry level: the same bytes under the same media type, so the digest
-      # is identical, the media type is preserved, and no layer moves.
-      - name: Promote the refresh candidate to the channel
-        if: steps.reuse.outputs.promote == 'true'
-        env:
-          IMAGE: git.fabledsword.com/bvandeusen/fabledcurator
-          CHANNEL_REF: ${{ steps.reuse.outputs.channel_ref }}
-          TOKEN: ${{ secrets.RELEASE_TOKEN }}
-          ACTOR: ${{ github.actor }}
-        run: |
-          set -eu
-          REPO=${IMAGE#git.fabledsword.com/}
-          TAG=${CHANNEL_REF##*:}
-
-          # Registry auth is its own token exchange — the `docker login` above
-          # authenticates the docker client, not curl. Deadline on every call
-          # (rule 156): a registry that stops answering must fail this step,
-          # not hang the weekly refresh until the job times out.
-          BEARER=$(curl -fsS --max-time 30 -u "$ACTOR:$TOKEN" \
-            "https://git.fabledsword.com/v2/token?scope=repository:$REPO:pull,push&service=git.fabledsword.com" \
-            | python3 -c 'import sys,json; print(json.load(sys.stdin)["token"])')
-
-          # Ask for the image manifest media types ONLY. Offering the index
-          # types too would let the registry hand back an index if one ever
-          # existed at this tag, and we would faithfully copy the thing we are
-          # trying not to create.
-          ACCEPT='application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json'
-          CT=$(curl -fsS --max-time 60 -o manifest.json -D headers.txt \
-                 -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \
-                 "https://git.fabledsword.com/v2/$REPO/manifests/refresh-candidate" \
-               && tr -d '\r' < headers.txt | awk -F': ' '/^[Cc]ontent-[Tt]ype:/{print $2}')
-          test -n "$CT"
-          SRC_DIGEST=$(tr -d '\r' < headers.txt | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}')
-          echo "promote: candidate is $SRC_DIGEST ($CT)"
-
-          curl -fsS --max-time 120 -X PUT \
-            -H "Authorization: Bearer $BEARER" -H "Content-Type: $CT" \
-            --data-binary @manifest.json \
-            "https://git.fabledsword.com/v2/$REPO/manifests/$TAG"
-
-          # Read it back. A PUT that returned 2xx but landed something else is
-          # exactly the silent-and-plausible failure this pipeline keeps
-          # producing, and the check costs one request.
-          NOW=$(curl -fsS --max-time 30 -o /dev/null -D - \
-                  -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \
-                  "https://git.fabledsword.com/v2/$REPO/manifests/$TAG" \
-                | tr -d '\r' | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}')
-          if [ "$NOW" != "$SRC_DIGEST" ]; then
-            echo "promote: $IMAGE:$TAG is $NOW, expected $SRC_DIGEST" >&2
-            exit 1
-          fi
-          echo "promote: $IMAGE:$TAG now names $NOW"
-
       # Every tag but the channel's own is written HERE, registry-side,
       # whether or not a build ran. Each -t becomes another reference to the
       # SAME manifest the channel tag holds, so :c-<sha> is byte-identical to
@@ -1123,8 +1070,8 @@ jobs:
   # time anyone saw this job run would also be the first time it could stop a
   # publish.
   smoke-web:
-    if: env.IS_REFRESH == 'true'
     needs: [build-web]
+    if: needs.build-web.outputs.candidate == 'true'
     runs-on: python-ci
     container:
       image: git.fabledsword.com/bvandeusen/ci-python:3.14
@@ -1201,6 +1148,14 @@ jobs:
           ENVOPTS="$ENVOPTS -e DB_PORT=5432 -e DB_NAME=$DB_NAME -e SECRET_KEY=$SECRET_KEY"
           ENVOPTS="$ENVOPTS -e CELERY_BROKER_URL=redis://$RD_IP:6379/0"
           ENVOPTS="$ENVOPTS -e CELERY_RESULT_BACKEND=redis://$RD_IP:6379/0"
+          # A throwaway CI instance IS first-time setup, which is the one case
+          # credential_crypto allows a key to be minted in. Without it the web
+          # role refuses to boot — deliberately, since silently generating a
+          # key on a restored-DB-but-lost-secrets deployment would leave every
+          # Credential row undecryptable (the 2026-06-02 audit). Discovered by
+          # this job on its first real run; see #3422 for the fact that no
+          # user-facing file mentions this variable at all.
+          ENVOPTS="$ENVOPTS -e CURATOR_BOOTSTRAP_NEW_KEY=1"
 
           # 1. The schema builds from empty, using the image's OWN libpq and
           #    psycopg. This is the same call entrypoint.sh makes before it
@@ -1235,18 +1190,141 @@ jobs:
               healthy=1
               break
             fi
+            # A container that has EXITED will never answer, so stop asking.
+            # Without this the loop spent 3m35s polling a dead container on
+            # this job's first run, and — because docker recycles the IP — got
+            # a confusing mix of connection-refused and 5s timeouts from
+            # whatever took the address next. The trap's log dump had the real
+            # answer the whole time; this just stops burying it.
+            if [ "$(docker inspect -f '{{.State.Running}}' "$CID" 2>/dev/null)" != "true" ]; then
+              echo "smoke: FAILED — the web container exited during boot." >&2
+              echo "smoke: its log follows; entrypoint runs alembic BEFORE" >&2
+              echo "smoke: serving, so a startup exception lands here." >&2
+              exit 1
+            fi
             sleep 2
           done
           if [ -z "$healthy" ]; then
-            echo "smoke: FAILED — web did not answer /api/health within 120s." >&2
+            # 60 iterations of (up to 5s connect + 2s sleep) — up to ~7min, not
-            echo "smoke: entrypoint runs alembic BEFORE serving, and step 1" >&2
+            # the 120s an earlier version of this message claimed.
-            echo "smoke: passed, so look at hypercorn and the python base." >&2
+            echo "smoke: FAILED — web is running but never answered" >&2
+            echo "smoke: /api/health. It is up, so look at hypercorn and the" >&2
+            echo "smoke: python base rather than at startup." >&2
             exit 1
           fi
           curl -fsS --max-time 5 "http://$WEB_IP:8080/api/health"
           echo
+
           echo "smoke: all checks passed against $CANDIDATE"
 
+  # Move the channel tags — the whole point of the gate.
+  #
+  # Lives in its own job because the verdict it depends on cannot exist until
+  # after build-web has finished, and the promote used to run INSIDE build-web.
+  #
+  # `needs` on smoke-web is the gate. A failed smoke skips this job, so a
+  # refresh that broke something leaves :latest naming the build that works —
+  # "the refresh failed" and "production is broken" must not be the same event.
+  # A SKIPPED smoke also skips this job, which is the behaviour that matters
+  # most: on run 5290 the gate silently skipped itself, and a design where only
+  # a FAILED gate blocks would have published unverified images while reporting
+  # success. Not running is not the same as passing.
+  #
+  # All three images promote TOGETHER, or none do. They are one stack: build.yml
+  # already refuses to publish a :dev web image beside a stale :dev ml, because
+  # the mismatch only shows up as a runtime failure. A refresh that published ml
+  # and withheld web would be that same trap, arrived at through the gate.
+  #
+  # The gate covers the web image only (milestone 362 step 3 scoped it there),
+  # so ml and agent are being held to web's verdict rather than their own. That
+  # is deliberate and it is the conservative direction — they ship together, so
+  # the weakest evidence should govern all three — but it is not the same as
+  # having smoked them, and it should not be read as if it were.
+  promote:
+    needs: [build-web, build-ml, build-agent, smoke-web]
+    # Only a refresh publishes through a candidate; a push writes its channel
+    # tag directly from the build. Reads the same reuse-step decision the build
+    # took, via a job output — a job's `if:` cannot see the `env` context.
+    if: needs.build-web.outputs.candidate == 'true'
+    runs-on: python-ci
+    container:
+      image: git.fabledsword.com/bvandeusen/ci-python:3.14
+    steps:
+      - name: Point the channel tags at the smoked candidates
+        env:
+          TOKEN: ${{ secrets.RELEASE_TOKEN }}
+          ACTOR: ${{ github.actor }}
+        run: |
+          set -eu
+          # `latest` is not a guess: a refresh always builds `main` (BUILD_REF),
+          # and the "must have checked out main" guard in every build job fails
+          # the run if that did not hold. So the channel is main's.
+          TAG=latest
+          FAILED=""
+
+          for NAME in fabledcurator fabledcurator-ml fabledcurator-agent; do
+            REPO="bvandeusen/$NAME"
+            echo "promote: $REPO"
+
+            # Registry auth is its own token exchange — `docker login`
+            # authenticates the docker client, not curl. Deadline on every call
+            # (rule 156): a registry that stops answering must fail this step,
+            # not hang the weekly refresh until the job times out.
+            BEARER=$(curl -fsS --max-time 30 -u "$ACTOR:$TOKEN" \
+              "https://git.fabledsword.com/v2/token?scope=repository:$REPO:pull,push&service=git.fabledsword.com" \
+              | python3 -c 'import sys,json; print(json.load(sys.stdin)["token"])')
+
+            # Ask for the IMAGE manifest media types only. Offering the index
+            # types too would let the registry hand back an index if one ever
+            # existed at this tag, and we would faithfully copy the thing this
+            # whole approach exists to avoid creating.
+            ACCEPT='application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json'
+            CT=$(curl -fsS --max-time 60 -o manifest.json -D headers.txt \
+                   -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \
+                   "https://git.fabledsword.com/v2/$REPO/manifests/refresh-candidate" \
+                 && tr -d '\r' < headers.txt | awk -F': ' '/^[Cc]ontent-[Tt]ype:/{print $2}')
+            test -n "$CT"
+            SRC=$(tr -d '\r' < headers.txt | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}')
+            echo "promote:   candidate $SRC ($CT)"
+
+            # NOT `imagetools create`. That wraps its source in an INDEX, and
+            # `.Image.Config.Labels` does not resolve through one — the
+            # fc.revision the reuse check reads off the channel tag would come
+            # back empty, every later push would miss and rebuild, and nothing
+            # would go red (#3183, run 4751). A manifest PUT is what "make this
+            # tag name that image" means at the registry: same bytes, same media
+            # type, same digest, no layer transfer.
+            curl -fsS --max-time 120 -X PUT \
+              -H "Authorization: Bearer $BEARER" -H "Content-Type: $CT" \
+              --data-binary @manifest.json \
+              "https://git.fabledsword.com/v2/$REPO/manifests/$TAG"
+
+            # Read it back. A PUT that returned 2xx but landed something else is
+            # exactly the silent-and-plausible failure this pipeline keeps
+            # producing, and the check costs one request.
+            NOW=$(curl -fsS --max-time 30 -o /dev/null -D - \
+                    -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \
+                    "https://git.fabledsword.com/v2/$REPO/manifests/$TAG" \
+                  | tr -d '\r' | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}')
+            if [ "$NOW" != "$SRC" ]; then
+              echo "promote:   FAILED — $NAME:$TAG is $NOW, expected $SRC" >&2
+              FAILED="$FAILED $NAME"
+              continue
+            fi
+            echo "promote:   $NAME:$TAG now names $NOW"
+          done
+
+          if [ -n "$FAILED" ]; then
+            echo "" >&2
+            echo "promote: FAILED for:$FAILED" >&2
+            echo "promote: the channel tags are now INCONSISTENT — some images" >&2
+            echo "promote: moved and some did not. Re-run this refresh; the" >&2
+            echo "promote: candidates are still published and the promote is" >&2
+            echo "promote: idempotent." >&2
+            exit 1
+          fi
+          echo "promote: all three channel tags moved"
+
   build-ml:
     runs-on: python-ci
     container:
@@ -1452,16 +1530,20 @@ jobs:
           # already allows for :buildcache, not the per-build tag family that
           # milestone 318 withdrew.
           #
-          # Both values are decided HERE, beside `hit`, for the reason the
+          # Decided HERE, beside `hit`, for the reason the force/schedule
-          # force/schedule branch below gives: one step decides what this job
+          # branch below gives: one step decides what this job does. A
-          # does. A promote condition derived independently could disagree with
+          # condition derived independently could disagree with the tag the
-          # the tag the build actually wrote.
+          # build actually wrote.
+          #
+          # build-web additionally exposes this as `outputs.candidate`, which is
+          # what gates the `promote` job — a job's `if:` cannot read `env`, and
+          # one flag is enough because all three derive it from the same
+          # IS_REFRESH. ml and agent do not re-emit it; a second copy nothing
+          # reads is the kind of thing that later reads as load-bearing.
           if [ "${IS_REFRESH:-}" = "true" ]; then
             echo "build_ref=$IMAGE:refresh-candidate" >> "$GITHUB_OUTPUT"
-            echo "promote=true" >> "$GITHUB_OUTPUT"
           else
             echo "build_ref=$IMAGE:$T" >> "$GITHUB_OUTPUT"
-            echo "promote=false" >> "$GITHUB_OUTPUT"
           fi
 
           # Compare VALUES, never exit codes. Measured on buildx v0.36.1
@@ -1595,77 +1677,6 @@ jobs:
           cache-from: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-ml:buildcache
           cache-to: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-ml:buildcache,mode=max
 
-      # Point the channel tag at the candidate the refresh just built.
-      #
-      # Unconditional TODAY, so this milestone never leaves the refresh in a
-      # state where it builds and publishes nothing. Step 4 wraps it in the
-      # smoke suite's verdict; until then the scheduled path behaves exactly
-      # as it did, just via two operations instead of one.
-      #
-      # NOT `imagetools create`. That wraps its source in an INDEX, and an
-      # indexed channel tag is the one thing this pipeline cannot survive:
-      # `.Image.Config.Labels` does not resolve through an index, so the
-      # fc.revision the reuse check reads off the channel tag would come back
-      # empty, every subsequent push would miss and rebuild, and nothing would
-      # go red. That is #3183, observed on run 4751 — reuse worked exactly once
-      # and the only symptom was the bill. The repoint step below excludes its
-      # own source tag for precisely this reason; a promote that re-introduced
-      # the wrap through a different door would undo that care.
-      #
-      # A manifest PUT is what "make this tag name that image" means at the
-      # registry level: the same bytes under the same media type, so the digest
-      # is identical, the media type is preserved, and no layer moves.
-      - name: Promote the refresh candidate to the channel
-        if: steps.reuse.outputs.promote == 'true'
-        env:
-          IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-ml
-          CHANNEL_REF: ${{ steps.reuse.outputs.channel_ref }}
-          TOKEN: ${{ secrets.RELEASE_TOKEN }}
-          ACTOR: ${{ github.actor }}
-        run: |
-          set -eu
-          REPO=${IMAGE#git.fabledsword.com/}
-          TAG=${CHANNEL_REF##*:}
-
-          # Registry auth is its own token exchange — the `docker login` above
-          # authenticates the docker client, not curl. Deadline on every call
-          # (rule 156): a registry that stops answering must fail this step,
-          # not hang the weekly refresh until the job times out.
-          BEARER=$(curl -fsS --max-time 30 -u "$ACTOR:$TOKEN" \
-            "https://git.fabledsword.com/v2/token?scope=repository:$REPO:pull,push&service=git.fabledsword.com" \
-            | python3 -c 'import sys,json; print(json.load(sys.stdin)["token"])')
-
-          # Ask for the image manifest media types ONLY. Offering the index
-          # types too would let the registry hand back an index if one ever
-          # existed at this tag, and we would faithfully copy the thing we are
-          # trying not to create.
-          ACCEPT='application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json'
-          CT=$(curl -fsS --max-time 60 -o manifest.json -D headers.txt \
-                 -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \
-                 "https://git.fabledsword.com/v2/$REPO/manifests/refresh-candidate" \
-               && tr -d '\r' < headers.txt | awk -F': ' '/^[Cc]ontent-[Tt]ype:/{print $2}')
-          test -n "$CT"
-          SRC_DIGEST=$(tr -d '\r' < headers.txt | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}')
-          echo "promote: candidate is $SRC_DIGEST ($CT)"
-
-          curl -fsS --max-time 120 -X PUT \
-            -H "Authorization: Bearer $BEARER" -H "Content-Type: $CT" \
-            --data-binary @manifest.json \
-            "https://git.fabledsword.com/v2/$REPO/manifests/$TAG"
-
-          # Read it back. A PUT that returned 2xx but landed something else is
-          # exactly the silent-and-plausible failure this pipeline keeps
-          # producing, and the check costs one request.
-          NOW=$(curl -fsS --max-time 30 -o /dev/null -D - \
-                  -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \
-                  "https://git.fabledsword.com/v2/$REPO/manifests/$TAG" \
-                | tr -d '\r' | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}')
-          if [ "$NOW" != "$SRC_DIGEST" ]; then
-            echo "promote: $IMAGE:$TAG is $NOW, expected $SRC_DIGEST" >&2
-            exit 1
-          fi
-          echo "promote: $IMAGE:$TAG now names $NOW"
-
       # Every tag but the channel's own is written HERE, registry-side,
       # whether or not a build ran. Each -t becomes another reference to the
       # SAME manifest the channel tag holds, so :c-<sha> is byte-identical to
@@ -1947,16 +1958,20 @@ jobs:
           # already allows for :buildcache, not the per-build tag family that
           # milestone 318 withdrew.
           #
-          # Both values are decided HERE, beside `hit`, for the reason the
+          # Decided HERE, beside `hit`, for the reason the force/schedule
-          # force/schedule branch below gives: one step decides what this job
+          # branch below gives: one step decides what this job does. A
-          # does. A promote condition derived independently could disagree with
+          # condition derived independently could disagree with the tag the
-          # the tag the build actually wrote.
+          # build actually wrote.
+          #
+          # build-web additionally exposes this as `outputs.candidate`, which is
+          # what gates the `promote` job — a job's `if:` cannot read `env`, and
+          # one flag is enough because all three derive it from the same
+          # IS_REFRESH. ml and agent do not re-emit it; a second copy nothing
+          # reads is the kind of thing that later reads as load-bearing.
           if [ "${IS_REFRESH:-}" = "true" ]; then
             echo "build_ref=$IMAGE:refresh-candidate" >> "$GITHUB_OUTPUT"
-            echo "promote=true" >> "$GITHUB_OUTPUT"
           else
             echo "build_ref=$IMAGE:$T" >> "$GITHUB_OUTPUT"
-            echo "promote=false" >> "$GITHUB_OUTPUT"
           fi
 
           # Compare VALUES, never exit codes. Measured on buildx v0.36.1
@@ -2090,77 +2105,6 @@ jobs:
           cache-from: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-agent:buildcache
           cache-to: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-agent:buildcache,mode=max
 
-      # Point the channel tag at the candidate the refresh just built.
-      #
-      # Unconditional TODAY, so this milestone never leaves the refresh in a
-      # state where it builds and publishes nothing. Step 4 wraps it in the
-      # smoke suite's verdict; until then the scheduled path behaves exactly
-      # as it did, just via two operations instead of one.
-      #
-      # NOT `imagetools create`. That wraps its source in an INDEX, and an
-      # indexed channel tag is the one thing this pipeline cannot survive:
-      # `.Image.Config.Labels` does not resolve through an index, so the
-      # fc.revision the reuse check reads off the channel tag would come back
-      # empty, every subsequent push would miss and rebuild, and nothing would
-      # go red. That is #3183, observed on run 4751 — reuse worked exactly once
-      # and the only symptom was the bill. The repoint step below excludes its
-      # own source tag for precisely this reason; a promote that re-introduced
-      # the wrap through a different door would undo that care.
-      #
-      # A manifest PUT is what "make this tag name that image" means at the
-      # registry level: the same bytes under the same media type, so the digest
-      # is identical, the media type is preserved, and no layer moves.
-      - name: Promote the refresh candidate to the channel
-        if: steps.reuse.outputs.promote == 'true'
-        env:
-          IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-agent
-          CHANNEL_REF: ${{ steps.reuse.outputs.channel_ref }}
-          TOKEN: ${{ secrets.RELEASE_TOKEN }}
-          ACTOR: ${{ github.actor }}
-        run: |
-          set -eu
-          REPO=${IMAGE#git.fabledsword.com/}
-          TAG=${CHANNEL_REF##*:}
-
-          # Registry auth is its own token exchange — the `docker login` above
-          # authenticates the docker client, not curl. Deadline on every call
-          # (rule 156): a registry that stops answering must fail this step,
-          # not hang the weekly refresh until the job times out.
-          BEARER=$(curl -fsS --max-time 30 -u "$ACTOR:$TOKEN" \
-            "https://git.fabledsword.com/v2/token?scope=repository:$REPO:pull,push&service=git.fabledsword.com" \
-            | python3 -c 'import sys,json; print(json.load(sys.stdin)["token"])')
-
-          # Ask for the image manifest media types ONLY. Offering the index
-          # types too would let the registry hand back an index if one ever
-          # existed at this tag, and we would faithfully copy the thing we are
-          # trying not to create.
-          ACCEPT='application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json'
-          CT=$(curl -fsS --max-time 60 -o manifest.json -D headers.txt \
-                 -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \
-                 "https://git.fabledsword.com/v2/$REPO/manifests/refresh-candidate" \
-               && tr -d '\r' < headers.txt | awk -F': ' '/^[Cc]ontent-[Tt]ype:/{print $2}')
-          test -n "$CT"
-          SRC_DIGEST=$(tr -d '\r' < headers.txt | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}')
-          echo "promote: candidate is $SRC_DIGEST ($CT)"
-
-          curl -fsS --max-time 120 -X PUT \
-            -H "Authorization: Bearer $BEARER" -H "Content-Type: $CT" \
-            --data-binary @manifest.json \
-            "https://git.fabledsword.com/v2/$REPO/manifests/$TAG"
-
-          # Read it back. A PUT that returned 2xx but landed something else is
-          # exactly the silent-and-plausible failure this pipeline keeps
-          # producing, and the check costs one request.
-          NOW=$(curl -fsS --max-time 30 -o /dev/null -D - \
-                  -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \
-                  "https://git.fabledsword.com/v2/$REPO/manifests/$TAG" \
-                | tr -d '\r' | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}')
-          if [ "$NOW" != "$SRC_DIGEST" ]; then
-            echo "promote: $IMAGE:$TAG is $NOW, expected $SRC_DIGEST" >&2
-            exit 1
-          fi
-          echo "promote: $IMAGE:$TAG now names $NOW"
-
       # Every tag but the channel's own is written HERE, registry-side,
       # whether or not a build ran. Each -t becomes another reference to the
       # SAME manifest the channel tag holds, so :c-<sha> is byte-identical to

README.md

@@ -90,7 +90,31 @@ If you forget it, the symptom is a long build instead of a quick pull.
 The database schema is created automatically on first start — the web container
 runs its migrations before serving. Nothing to initialise by hand.
 
-A few things are worth knowing about the first few minutes:
+**One thing does need a deliberate act, and the app will not start without it.**
+FabledCurator encrypts your stored platform credentials with a key it keeps at
+`./images/secrets/credential_key.b64`. On a brand-new install that file does not
+exist, and rather than quietly creating one the app stops:
+
+```
+MissingCredentialKey: Fernet key file not found at /images/secrets/credential_key.b64
+```
+
+Set `CURATOR_BOOTSTRAP_NEW_KEY=1` in your `.env` for the first `up`, then delete
+the line once the container is running. `.env.example` ships it with that
+instruction attached.
+
+The refusal is deliberate, and worth understanding rather than working around:
+auto-creating a key is indistinguishable from the disaster case — a restore that
+brought the database back but lost `./images/secrets` — where it would mint a key
+that cannot decrypt anything, leaving an instance that looks healthy while every
+paywalled download fails. Making you say so once, on an empty install, is the
+price of that not happening silently later.
+
+**Which means: back up `./images/secrets/` alongside your database.** It is the
+only thing that can read your stored credentials. A database restored without it
+needs every credential entered again by hand.
+
+A few other things are worth knowing about the first few minutes:
 
 - **The ML worker downloads its model weights on first boot**, several GB from
   HuggingFace into `./models`. Until that finishes, tagging is queued rather

docker-compose.yml

@@ -133,6 +133,20 @@ services:
       CELERY_RESULT_BACKEND: redis://redis:6379/0
       SECRET_KEY: ${SECRET_KEY:-dev_secret_key_not_for_production_change_me}
       LOG_LEVEL: ${LOG_LEVEL:-INFO}
+      # First boot only. FabledCurator refuses to start until the credential
+      # encryption key at /images/secrets/credential_key.b64 exists, and
+      # refuses to create one unless told to — auto-creating is
+      # indistinguishable from a restore that lost ./images/secrets, where it
+      # would mint a key that decrypts nothing and leave an instance that looks
+      # healthy while every paywalled download fails.
+      #
+      # Passed through EXPLICITLY because a variable in `.env` is only used for
+      # ${...} interpolation; it does not reach the container unless it is
+      # named here. Defaulted to empty so the refusal stands for everyone who
+      # has not opted in — the app tests for exactly "1".
+      #
+      # Set it in .env for one `up`, then remove it. See .env.example.
+      CURATOR_BOOTSTRAP_NEW_KEY: ${CURATOR_BOOTSTRAP_NEW_KEY:-}
     volumes:
       - ./images:/images
       - ./import:/import