diff --git a/.env.example b/.env.example index cbb94d4..fe2cb12 100644 --- a/.env.example +++ b/.env.example @@ -34,6 +34,34 @@ DB_PASSWORD= SECRET_KEY= +# --------------------------------------------------------------------------- +# FIRST BOOT ONLY — then delete this line +# --------------------------------------------------------------------------- + +# FabledCurator encrypts your stored platform credentials with a Fernet key it +# keeps at /images/secrets/credential_key.b64 — inside the ./images bind mount, +# so it outlives the container. On a brand-new install that file does not exist +# yet, and the app REFUSES TO START rather than quietly create one: +# +# MissingCredentialKey: Fernet key file not found at +# /images/secrets/credential_key.b64 +# +# That refusal is deliberate. Auto-creating a key is indistinguishable from the +# disaster case — a restore that brought the database back but lost +# ./images/secrets — and there it would mint a key that cannot decrypt anything, +# leaving an instance that looks healthy while every paywalled download fails. +# So the choice is yours to make explicitly, once. +# +# Set this for your first `up`, watch the container come up, then DELETE THE +# LINE. Leaving it set disarms the protection permanently, on an instance that +# by then has credentials worth protecting. +# +# BACK UP ./images/secrets/ ALONGSIDE YOUR DATABASE. The key is the only thing +# that can read your stored credentials; a database restored without it needs +# every credential re-entered by hand. +CURATOR_BOOTSTRAP_NEW_KEY=1 + + # --------------------------------------------------------------------------- # Optional — defaults are fine # --------------------------------------------------------------------------- diff --git a/.forgejo/workflows/build.yml b/.forgejo/workflows/build.yml index a1c115f..b37e0ba 100644 --- a/.forgejo/workflows/build.yml +++ b/.forgejo/workflows/build.yml @@ -452,6 +452,18 @@ jobs: # to. Same source of truth; no double-store. build-web: + # Consumed by smoke-web's job-level `if:`. It cannot read `env` — the env + # context is available to STEP `if:` and step bodies, never to a job's own + # condition, and an unresolvable context there is empty rather than an + # error. `smoke-web` skipped silently on run 5290 for exactly that reason. + # + # Keying off the reuse step's own output is better than re-deriving the + # trigger anyway: it is the same single decision the build, the XPI + # download and the promote all take (build.yml's "one decision drives + # everything downstream"), and it says the thing smoke-web actually needs + # to know — a candidate was published — rather than restating why. + outputs: + candidate: ${{ steps.reuse.outputs.promote }} # A plain `needs` — no `always()`. That expression existed to let a # SKIPPED sign-extension through on a tag push while still blocking a # FAILED one. With no tag trigger, sign-extension always runs, so the @@ -722,10 +734,16 @@ jobs: # already allows for :buildcache, not the per-build tag family that # milestone 318 withdrew. # - # Both values are decided HERE, beside `hit`, for the reason the - # force/schedule branch below gives: one step decides what this job - # does. A promote condition derived independently could disagree with - # the tag the build actually wrote. + # Decided HERE, beside `hit`, for the reason the force/schedule + # branch below gives: one step decides what this job does. A + # condition derived independently could disagree with the tag the + # build actually wrote. + # + # build-web additionally exposes this as `outputs.candidate`, which is + # what gates the `promote` job — a job's `if:` cannot read `env`, and + # one flag is enough because all three derive it from the same + # IS_REFRESH. ml and agent do not re-emit it; a second copy nothing + # reads is the kind of thing that later reads as load-bearing. if [ "${IS_REFRESH:-}" = "true" ]; then echo "build_ref=$IMAGE:refresh-candidate" >> "$GITHUB_OUTPUT" echo "promote=true" >> "$GITHUB_OUTPUT" @@ -947,77 +965,6 @@ jobs: FC_CHANNEL=${{ steps.tag.outputs.channel }} FC_VERSION=${{ steps.reuse.outputs.version }} - # Point the channel tag at the candidate the refresh just built. - # - # Unconditional TODAY, so this milestone never leaves the refresh in a - # state where it builds and publishes nothing. Step 4 wraps it in the - # smoke suite's verdict; until then the scheduled path behaves exactly - # as it did, just via two operations instead of one. - # - # NOT `imagetools create`. That wraps its source in an INDEX, and an - # indexed channel tag is the one thing this pipeline cannot survive: - # `.Image.Config.Labels` does not resolve through an index, so the - # fc.revision the reuse check reads off the channel tag would come back - # empty, every subsequent push would miss and rebuild, and nothing would - # go red. That is #3183, observed on run 4751 — reuse worked exactly once - # and the only symptom was the bill. The repoint step below excludes its - # own source tag for precisely this reason; a promote that re-introduced - # the wrap through a different door would undo that care. - # - # A manifest PUT is what "make this tag name that image" means at the - # registry level: the same bytes under the same media type, so the digest - # is identical, the media type is preserved, and no layer moves. - - name: Promote the refresh candidate to the channel - if: steps.reuse.outputs.promote == 'true' - env: - IMAGE: git.fabledsword.com/bvandeusen/fabledcurator - CHANNEL_REF: ${{ steps.reuse.outputs.channel_ref }} - TOKEN: ${{ secrets.RELEASE_TOKEN }} - ACTOR: ${{ github.actor }} - run: | - set -eu - REPO=${IMAGE#git.fabledsword.com/} - TAG=${CHANNEL_REF##*:} - - # Registry auth is its own token exchange — the `docker login` above - # authenticates the docker client, not curl. Deadline on every call - # (rule 156): a registry that stops answering must fail this step, - # not hang the weekly refresh until the job times out. - BEARER=$(curl -fsS --max-time 30 -u "$ACTOR:$TOKEN" \ - "https://git.fabledsword.com/v2/token?scope=repository:$REPO:pull,push&service=git.fabledsword.com" \ - | python3 -c 'import sys,json; print(json.load(sys.stdin)["token"])') - - # Ask for the image manifest media types ONLY. Offering the index - # types too would let the registry hand back an index if one ever - # existed at this tag, and we would faithfully copy the thing we are - # trying not to create. - ACCEPT='application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json' - CT=$(curl -fsS --max-time 60 -o manifest.json -D headers.txt \ - -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \ - "https://git.fabledsword.com/v2/$REPO/manifests/refresh-candidate" \ - && tr -d '\r' < headers.txt | awk -F': ' '/^[Cc]ontent-[Tt]ype:/{print $2}') - test -n "$CT" - SRC_DIGEST=$(tr -d '\r' < headers.txt | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}') - echo "promote: candidate is $SRC_DIGEST ($CT)" - - curl -fsS --max-time 120 -X PUT \ - -H "Authorization: Bearer $BEARER" -H "Content-Type: $CT" \ - --data-binary @manifest.json \ - "https://git.fabledsword.com/v2/$REPO/manifests/$TAG" - - # Read it back. A PUT that returned 2xx but landed something else is - # exactly the silent-and-plausible failure this pipeline keeps - # producing, and the check costs one request. - NOW=$(curl -fsS --max-time 30 -o /dev/null -D - \ - -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \ - "https://git.fabledsword.com/v2/$REPO/manifests/$TAG" \ - | tr -d '\r' | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}') - if [ "$NOW" != "$SRC_DIGEST" ]; then - echo "promote: $IMAGE:$TAG is $NOW, expected $SRC_DIGEST" >&2 - exit 1 - fi - echo "promote: $IMAGE:$TAG now names $NOW" - # Every tag but the channel's own is written HERE, registry-side, # whether or not a build ran. Each -t becomes another reference to the # SAME manifest the channel tag holds, so :c- is byte-identical to @@ -1123,8 +1070,8 @@ jobs: # time anyone saw this job run would also be the first time it could stop a # publish. smoke-web: - if: env.IS_REFRESH == 'true' needs: [build-web] + if: needs.build-web.outputs.candidate == 'true' runs-on: python-ci container: image: git.fabledsword.com/bvandeusen/ci-python:3.14 @@ -1201,6 +1148,14 @@ jobs: ENVOPTS="$ENVOPTS -e DB_PORT=5432 -e DB_NAME=$DB_NAME -e SECRET_KEY=$SECRET_KEY" ENVOPTS="$ENVOPTS -e CELERY_BROKER_URL=redis://$RD_IP:6379/0" ENVOPTS="$ENVOPTS -e CELERY_RESULT_BACKEND=redis://$RD_IP:6379/0" + # A throwaway CI instance IS first-time setup, which is the one case + # credential_crypto allows a key to be minted in. Without it the web + # role refuses to boot — deliberately, since silently generating a + # key on a restored-DB-but-lost-secrets deployment would leave every + # Credential row undecryptable (the 2026-06-02 audit). Discovered by + # this job on its first real run; see #3422 for the fact that no + # user-facing file mentions this variable at all. + ENVOPTS="$ENVOPTS -e CURATOR_BOOTSTRAP_NEW_KEY=1" # 1. The schema builds from empty, using the image's OWN libpq and # psycopg. This is the same call entrypoint.sh makes before it @@ -1235,18 +1190,141 @@ jobs: healthy=1 break fi + # A container that has EXITED will never answer, so stop asking. + # Without this the loop spent 3m35s polling a dead container on + # this job's first run, and — because docker recycles the IP — got + # a confusing mix of connection-refused and 5s timeouts from + # whatever took the address next. The trap's log dump had the real + # answer the whole time; this just stops burying it. + if [ "$(docker inspect -f '{{.State.Running}}' "$CID" 2>/dev/null)" != "true" ]; then + echo "smoke: FAILED — the web container exited during boot." >&2 + echo "smoke: its log follows; entrypoint runs alembic BEFORE" >&2 + echo "smoke: serving, so a startup exception lands here." >&2 + exit 1 + fi sleep 2 done if [ -z "$healthy" ]; then - echo "smoke: FAILED — web did not answer /api/health within 120s." >&2 - echo "smoke: entrypoint runs alembic BEFORE serving, and step 1" >&2 - echo "smoke: passed, so look at hypercorn and the python base." >&2 + # 60 iterations of (up to 5s connect + 2s sleep) — up to ~7min, not + # the 120s an earlier version of this message claimed. + echo "smoke: FAILED — web is running but never answered" >&2 + echo "smoke: /api/health. It is up, so look at hypercorn and the" >&2 + echo "smoke: python base rather than at startup." >&2 exit 1 fi curl -fsS --max-time 5 "http://$WEB_IP:8080/api/health" echo + echo "smoke: all checks passed against $CANDIDATE" + # Move the channel tags — the whole point of the gate. + # + # Lives in its own job because the verdict it depends on cannot exist until + # after build-web has finished, and the promote used to run INSIDE build-web. + # + # `needs` on smoke-web is the gate. A failed smoke skips this job, so a + # refresh that broke something leaves :latest naming the build that works — + # "the refresh failed" and "production is broken" must not be the same event. + # A SKIPPED smoke also skips this job, which is the behaviour that matters + # most: on run 5290 the gate silently skipped itself, and a design where only + # a FAILED gate blocks would have published unverified images while reporting + # success. Not running is not the same as passing. + # + # All three images promote TOGETHER, or none do. They are one stack: build.yml + # already refuses to publish a :dev web image beside a stale :dev ml, because + # the mismatch only shows up as a runtime failure. A refresh that published ml + # and withheld web would be that same trap, arrived at through the gate. + # + # The gate covers the web image only (milestone 362 step 3 scoped it there), + # so ml and agent are being held to web's verdict rather than their own. That + # is deliberate and it is the conservative direction — they ship together, so + # the weakest evidence should govern all three — but it is not the same as + # having smoked them, and it should not be read as if it were. + promote: + needs: [build-web, build-ml, build-agent, smoke-web] + # Only a refresh publishes through a candidate; a push writes its channel + # tag directly from the build. Reads the same reuse-step decision the build + # took, via a job output — a job's `if:` cannot see the `env` context. + if: needs.build-web.outputs.candidate == 'true' + runs-on: python-ci + container: + image: git.fabledsword.com/bvandeusen/ci-python:3.14 + steps: + - name: Point the channel tags at the smoked candidates + env: + TOKEN: ${{ secrets.RELEASE_TOKEN }} + ACTOR: ${{ github.actor }} + run: | + set -eu + # `latest` is not a guess: a refresh always builds `main` (BUILD_REF), + # and the "must have checked out main" guard in every build job fails + # the run if that did not hold. So the channel is main's. + TAG=latest + FAILED="" + + for NAME in fabledcurator fabledcurator-ml fabledcurator-agent; do + REPO="bvandeusen/$NAME" + echo "promote: $REPO" + + # Registry auth is its own token exchange — `docker login` + # authenticates the docker client, not curl. Deadline on every call + # (rule 156): a registry that stops answering must fail this step, + # not hang the weekly refresh until the job times out. + BEARER=$(curl -fsS --max-time 30 -u "$ACTOR:$TOKEN" \ + "https://git.fabledsword.com/v2/token?scope=repository:$REPO:pull,push&service=git.fabledsword.com" \ + | python3 -c 'import sys,json; print(json.load(sys.stdin)["token"])') + + # Ask for the IMAGE manifest media types only. Offering the index + # types too would let the registry hand back an index if one ever + # existed at this tag, and we would faithfully copy the thing this + # whole approach exists to avoid creating. + ACCEPT='application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json' + CT=$(curl -fsS --max-time 60 -o manifest.json -D headers.txt \ + -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \ + "https://git.fabledsword.com/v2/$REPO/manifests/refresh-candidate" \ + && tr -d '\r' < headers.txt | awk -F': ' '/^[Cc]ontent-[Tt]ype:/{print $2}') + test -n "$CT" + SRC=$(tr -d '\r' < headers.txt | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}') + echo "promote: candidate $SRC ($CT)" + + # NOT `imagetools create`. That wraps its source in an INDEX, and + # `.Image.Config.Labels` does not resolve through one — the + # fc.revision the reuse check reads off the channel tag would come + # back empty, every later push would miss and rebuild, and nothing + # would go red (#3183, run 4751). A manifest PUT is what "make this + # tag name that image" means at the registry: same bytes, same media + # type, same digest, no layer transfer. + curl -fsS --max-time 120 -X PUT \ + -H "Authorization: Bearer $BEARER" -H "Content-Type: $CT" \ + --data-binary @manifest.json \ + "https://git.fabledsword.com/v2/$REPO/manifests/$TAG" + + # Read it back. A PUT that returned 2xx but landed something else is + # exactly the silent-and-plausible failure this pipeline keeps + # producing, and the check costs one request. + NOW=$(curl -fsS --max-time 30 -o /dev/null -D - \ + -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \ + "https://git.fabledsword.com/v2/$REPO/manifests/$TAG" \ + | tr -d '\r' | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}') + if [ "$NOW" != "$SRC" ]; then + echo "promote: FAILED — $NAME:$TAG is $NOW, expected $SRC" >&2 + FAILED="$FAILED $NAME" + continue + fi + echo "promote: $NAME:$TAG now names $NOW" + done + + if [ -n "$FAILED" ]; then + echo "" >&2 + echo "promote: FAILED for:$FAILED" >&2 + echo "promote: the channel tags are now INCONSISTENT — some images" >&2 + echo "promote: moved and some did not. Re-run this refresh; the" >&2 + echo "promote: candidates are still published and the promote is" >&2 + echo "promote: idempotent." >&2 + exit 1 + fi + echo "promote: all three channel tags moved" + build-ml: runs-on: python-ci container: @@ -1452,16 +1530,20 @@ jobs: # already allows for :buildcache, not the per-build tag family that # milestone 318 withdrew. # - # Both values are decided HERE, beside `hit`, for the reason the - # force/schedule branch below gives: one step decides what this job - # does. A promote condition derived independently could disagree with - # the tag the build actually wrote. + # Decided HERE, beside `hit`, for the reason the force/schedule + # branch below gives: one step decides what this job does. A + # condition derived independently could disagree with the tag the + # build actually wrote. + # + # build-web additionally exposes this as `outputs.candidate`, which is + # what gates the `promote` job — a job's `if:` cannot read `env`, and + # one flag is enough because all three derive it from the same + # IS_REFRESH. ml and agent do not re-emit it; a second copy nothing + # reads is the kind of thing that later reads as load-bearing. if [ "${IS_REFRESH:-}" = "true" ]; then echo "build_ref=$IMAGE:refresh-candidate" >> "$GITHUB_OUTPUT" - echo "promote=true" >> "$GITHUB_OUTPUT" else echo "build_ref=$IMAGE:$T" >> "$GITHUB_OUTPUT" - echo "promote=false" >> "$GITHUB_OUTPUT" fi # Compare VALUES, never exit codes. Measured on buildx v0.36.1 @@ -1595,77 +1677,6 @@ jobs: cache-from: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-ml:buildcache cache-to: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-ml:buildcache,mode=max - # Point the channel tag at the candidate the refresh just built. - # - # Unconditional TODAY, so this milestone never leaves the refresh in a - # state where it builds and publishes nothing. Step 4 wraps it in the - # smoke suite's verdict; until then the scheduled path behaves exactly - # as it did, just via two operations instead of one. - # - # NOT `imagetools create`. That wraps its source in an INDEX, and an - # indexed channel tag is the one thing this pipeline cannot survive: - # `.Image.Config.Labels` does not resolve through an index, so the - # fc.revision the reuse check reads off the channel tag would come back - # empty, every subsequent push would miss and rebuild, and nothing would - # go red. That is #3183, observed on run 4751 — reuse worked exactly once - # and the only symptom was the bill. The repoint step below excludes its - # own source tag for precisely this reason; a promote that re-introduced - # the wrap through a different door would undo that care. - # - # A manifest PUT is what "make this tag name that image" means at the - # registry level: the same bytes under the same media type, so the digest - # is identical, the media type is preserved, and no layer moves. - - name: Promote the refresh candidate to the channel - if: steps.reuse.outputs.promote == 'true' - env: - IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-ml - CHANNEL_REF: ${{ steps.reuse.outputs.channel_ref }} - TOKEN: ${{ secrets.RELEASE_TOKEN }} - ACTOR: ${{ github.actor }} - run: | - set -eu - REPO=${IMAGE#git.fabledsword.com/} - TAG=${CHANNEL_REF##*:} - - # Registry auth is its own token exchange — the `docker login` above - # authenticates the docker client, not curl. Deadline on every call - # (rule 156): a registry that stops answering must fail this step, - # not hang the weekly refresh until the job times out. - BEARER=$(curl -fsS --max-time 30 -u "$ACTOR:$TOKEN" \ - "https://git.fabledsword.com/v2/token?scope=repository:$REPO:pull,push&service=git.fabledsword.com" \ - | python3 -c 'import sys,json; print(json.load(sys.stdin)["token"])') - - # Ask for the image manifest media types ONLY. Offering the index - # types too would let the registry hand back an index if one ever - # existed at this tag, and we would faithfully copy the thing we are - # trying not to create. - ACCEPT='application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json' - CT=$(curl -fsS --max-time 60 -o manifest.json -D headers.txt \ - -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \ - "https://git.fabledsword.com/v2/$REPO/manifests/refresh-candidate" \ - && tr -d '\r' < headers.txt | awk -F': ' '/^[Cc]ontent-[Tt]ype:/{print $2}') - test -n "$CT" - SRC_DIGEST=$(tr -d '\r' < headers.txt | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}') - echo "promote: candidate is $SRC_DIGEST ($CT)" - - curl -fsS --max-time 120 -X PUT \ - -H "Authorization: Bearer $BEARER" -H "Content-Type: $CT" \ - --data-binary @manifest.json \ - "https://git.fabledsword.com/v2/$REPO/manifests/$TAG" - - # Read it back. A PUT that returned 2xx but landed something else is - # exactly the silent-and-plausible failure this pipeline keeps - # producing, and the check costs one request. - NOW=$(curl -fsS --max-time 30 -o /dev/null -D - \ - -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \ - "https://git.fabledsword.com/v2/$REPO/manifests/$TAG" \ - | tr -d '\r' | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}') - if [ "$NOW" != "$SRC_DIGEST" ]; then - echo "promote: $IMAGE:$TAG is $NOW, expected $SRC_DIGEST" >&2 - exit 1 - fi - echo "promote: $IMAGE:$TAG now names $NOW" - # Every tag but the channel's own is written HERE, registry-side, # whether or not a build ran. Each -t becomes another reference to the # SAME manifest the channel tag holds, so :c- is byte-identical to @@ -1947,16 +1958,20 @@ jobs: # already allows for :buildcache, not the per-build tag family that # milestone 318 withdrew. # - # Both values are decided HERE, beside `hit`, for the reason the - # force/schedule branch below gives: one step decides what this job - # does. A promote condition derived independently could disagree with - # the tag the build actually wrote. + # Decided HERE, beside `hit`, for the reason the force/schedule + # branch below gives: one step decides what this job does. A + # condition derived independently could disagree with the tag the + # build actually wrote. + # + # build-web additionally exposes this as `outputs.candidate`, which is + # what gates the `promote` job — a job's `if:` cannot read `env`, and + # one flag is enough because all three derive it from the same + # IS_REFRESH. ml and agent do not re-emit it; a second copy nothing + # reads is the kind of thing that later reads as load-bearing. if [ "${IS_REFRESH:-}" = "true" ]; then echo "build_ref=$IMAGE:refresh-candidate" >> "$GITHUB_OUTPUT" - echo "promote=true" >> "$GITHUB_OUTPUT" else echo "build_ref=$IMAGE:$T" >> "$GITHUB_OUTPUT" - echo "promote=false" >> "$GITHUB_OUTPUT" fi # Compare VALUES, never exit codes. Measured on buildx v0.36.1 @@ -2090,77 +2105,6 @@ jobs: cache-from: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-agent:buildcache cache-to: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-agent:buildcache,mode=max - # Point the channel tag at the candidate the refresh just built. - # - # Unconditional TODAY, so this milestone never leaves the refresh in a - # state where it builds and publishes nothing. Step 4 wraps it in the - # smoke suite's verdict; until then the scheduled path behaves exactly - # as it did, just via two operations instead of one. - # - # NOT `imagetools create`. That wraps its source in an INDEX, and an - # indexed channel tag is the one thing this pipeline cannot survive: - # `.Image.Config.Labels` does not resolve through an index, so the - # fc.revision the reuse check reads off the channel tag would come back - # empty, every subsequent push would miss and rebuild, and nothing would - # go red. That is #3183, observed on run 4751 — reuse worked exactly once - # and the only symptom was the bill. The repoint step below excludes its - # own source tag for precisely this reason; a promote that re-introduced - # the wrap through a different door would undo that care. - # - # A manifest PUT is what "make this tag name that image" means at the - # registry level: the same bytes under the same media type, so the digest - # is identical, the media type is preserved, and no layer moves. - - name: Promote the refresh candidate to the channel - if: steps.reuse.outputs.promote == 'true' - env: - IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-agent - CHANNEL_REF: ${{ steps.reuse.outputs.channel_ref }} - TOKEN: ${{ secrets.RELEASE_TOKEN }} - ACTOR: ${{ github.actor }} - run: | - set -eu - REPO=${IMAGE#git.fabledsword.com/} - TAG=${CHANNEL_REF##*:} - - # Registry auth is its own token exchange — the `docker login` above - # authenticates the docker client, not curl. Deadline on every call - # (rule 156): a registry that stops answering must fail this step, - # not hang the weekly refresh until the job times out. - BEARER=$(curl -fsS --max-time 30 -u "$ACTOR:$TOKEN" \ - "https://git.fabledsword.com/v2/token?scope=repository:$REPO:pull,push&service=git.fabledsword.com" \ - | python3 -c 'import sys,json; print(json.load(sys.stdin)["token"])') - - # Ask for the image manifest media types ONLY. Offering the index - # types too would let the registry hand back an index if one ever - # existed at this tag, and we would faithfully copy the thing we are - # trying not to create. - ACCEPT='application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json' - CT=$(curl -fsS --max-time 60 -o manifest.json -D headers.txt \ - -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \ - "https://git.fabledsword.com/v2/$REPO/manifests/refresh-candidate" \ - && tr -d '\r' < headers.txt | awk -F': ' '/^[Cc]ontent-[Tt]ype:/{print $2}') - test -n "$CT" - SRC_DIGEST=$(tr -d '\r' < headers.txt | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}') - echo "promote: candidate is $SRC_DIGEST ($CT)" - - curl -fsS --max-time 120 -X PUT \ - -H "Authorization: Bearer $BEARER" -H "Content-Type: $CT" \ - --data-binary @manifest.json \ - "https://git.fabledsword.com/v2/$REPO/manifests/$TAG" - - # Read it back. A PUT that returned 2xx but landed something else is - # exactly the silent-and-plausible failure this pipeline keeps - # producing, and the check costs one request. - NOW=$(curl -fsS --max-time 30 -o /dev/null -D - \ - -H "Authorization: Bearer $BEARER" -H "Accept: $ACCEPT" \ - "https://git.fabledsword.com/v2/$REPO/manifests/$TAG" \ - | tr -d '\r' | awk -F': ' '/^[Dd]ocker-[Cc]ontent-[Dd]igest:/{print $2}') - if [ "$NOW" != "$SRC_DIGEST" ]; then - echo "promote: $IMAGE:$TAG is $NOW, expected $SRC_DIGEST" >&2 - exit 1 - fi - echo "promote: $IMAGE:$TAG now names $NOW" - # Every tag but the channel's own is written HERE, registry-side, # whether or not a build ran. Each -t becomes another reference to the # SAME manifest the channel tag holds, so :c- is byte-identical to diff --git a/README.md b/README.md index 365032d..2770fd4 100644 --- a/README.md +++ b/README.md @@ -90,7 +90,31 @@ If you forget it, the symptom is a long build instead of a quick pull. The database schema is created automatically on first start — the web container runs its migrations before serving. Nothing to initialise by hand. -A few things are worth knowing about the first few minutes: +**One thing does need a deliberate act, and the app will not start without it.** +FabledCurator encrypts your stored platform credentials with a key it keeps at +`./images/secrets/credential_key.b64`. On a brand-new install that file does not +exist, and rather than quietly creating one the app stops: + +``` +MissingCredentialKey: Fernet key file not found at /images/secrets/credential_key.b64 +``` + +Set `CURATOR_BOOTSTRAP_NEW_KEY=1` in your `.env` for the first `up`, then delete +the line once the container is running. `.env.example` ships it with that +instruction attached. + +The refusal is deliberate, and worth understanding rather than working around: +auto-creating a key is indistinguishable from the disaster case — a restore that +brought the database back but lost `./images/secrets` — where it would mint a key +that cannot decrypt anything, leaving an instance that looks healthy while every +paywalled download fails. Making you say so once, on an empty install, is the +price of that not happening silently later. + +**Which means: back up `./images/secrets/` alongside your database.** It is the +only thing that can read your stored credentials. A database restored without it +needs every credential entered again by hand. + +A few other things are worth knowing about the first few minutes: - **The ML worker downloads its model weights on first boot**, several GB from HuggingFace into `./models`. Until that finishes, tagging is queued rather diff --git a/docker-compose.yml b/docker-compose.yml index fd68189..0bca4f2 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -133,6 +133,20 @@ services: CELERY_RESULT_BACKEND: redis://redis:6379/0 SECRET_KEY: ${SECRET_KEY:-dev_secret_key_not_for_production_change_me} LOG_LEVEL: ${LOG_LEVEL:-INFO} + # First boot only. FabledCurator refuses to start until the credential + # encryption key at /images/secrets/credential_key.b64 exists, and + # refuses to create one unless told to — auto-creating is + # indistinguishable from a restore that lost ./images/secrets, where it + # would mint a key that decrypts nothing and leave an instance that looks + # healthy while every paywalled download fails. + # + # Passed through EXPLICITLY because a variable in `.env` is only used for + # ${...} interpolation; it does not reach the container unless it is + # named here. Defaulted to empty so the refusal stands for everyone who + # has not opted in — the app tests for exactly "1". + # + # Set it in .env for one `up`, then remove it. See .env.example. + CURATOR_BOOTSTRAP_NEW_KEY: ${CURATOR_BOOTSTRAP_NEW_KEY:-} volumes: - ./images:/images - ./import:/import