refactor: retire the fabledcurator-ml image name entirely (4311)
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 18s
extension / lint (push) Successful in 19s
CI / backend-lint-and-test (push) Failing after 32s
Build images / build-web (push) Successful in 1m37s
CI / integration (push) Successful in 2m11s
Build images / smoke-web (push) Successful in 42s
Build images / promote (push) Skipped
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 18s
extension / lint (push) Successful in 19s
CI / backend-lint-and-test (push) Failing after 32s
Build images / build-web (push) Successful in 1m37s
CI / integration (push) Successful in 2m11s
Build images / smoke-web (push) Successful in 42s
Build images / promote (push) Skipped
Operator, 2026-09-23: *"we don't need to keep this as I'll be collapsing my
stack to the same consolidated version that we're building here."*
That was the only thing holding the name up. `fabledcurator-ml` has been the
same bytes as `fabledcurator` since milestone 422 step 6, and 8152684 had
already stopped rebuilding it — this removes the name.
Gone: the whole `build-ml` job (137 more lines), its entry in promote's
`needs` and in promote's tag loop, `ML_PATHS` and the `ml)` case in
artifacts.sh, the `ml` artifact in test_artifact_paths, the image in
release_notes.py and its test, and the README row. `docker-compose.yml`'s
ml-worker service now runs `fabledcurator:latest` with `command:
["ml-worker"]` — the service still exists for anyone who wants lane
separation, it just no longer needs a second image name.
Prose that said "all three images" is now "both", except where it means
three VALUES (revision/version/epoch) or records what happened on 2026-08-30,
when there genuinely were three.
Published tags are left alone. Nothing new goes to that name; what is already
in the registry stays pullable, so a stack that has not moved yet keeps
running rather than losing its image mid-flight.
## A latent trap this made reachable, found by running the script
`artifacts.sh revision ml` did not fail. It printed the usage line to stderr
and answered with **the newest commit in the whole repository** — a
real-looking 12-char sha on stdout, exit 0.
`newest()` inlined the path set as `git log ... -- $(cmd_paths "$1")`, and
`usage` exits from the command SUBSHELL, so the substitution came back empty
and `git log HEAD --` had no pathspec left to filter by. The reuse check
would have compared that answer against a published label, missed, and
rebuilt on every push forever with nothing going red.
It could not be fixed inside `newest`, which was my first attempt: every
caller wraps it in a substitution too, so its `exit` also died in a subshell —
stdout went empty but the status stayed 0. The guard has to run in the main
shell, so it validates the artifact name at dispatch. `newest` keeps a
defensive `|| exit 2` for a future caller that reaches it another way, and
the comment says which of the two is the real check.
Latent while every name callers passed was valid. Removing `ml` from the set
is what made a name that used to work start taking the silent path, so the
test that pins it lands here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
+24
-157
@@ -10,9 +10,12 @@ on:
|
|||||||
# pressure to merge in order to try something does not come from
|
# pressure to merge in order to try something does not come from
|
||||||
# carelessness; it comes from `:dev` being unable to carry the build.
|
# carelessness; it comes from `:dev` being unable to carry the build.
|
||||||
#
|
#
|
||||||
# All three images build on dev, deliberately: a `:dev` web image paired
|
# BOTH images build on dev, deliberately: a `:dev` web image paired
|
||||||
# with a stale `:dev` ml or agent is a worse trap than no dev channel at
|
# with a stale `:dev` agent is a worse trap than no dev channel at all,
|
||||||
# all, since the mismatch only shows up as a runtime failure.
|
# since the mismatch only shows up as a runtime failure. (It was three
|
||||||
|
# until #4311 — `fabledcurator-ml` was the same bytes as `fabledcurator`
|
||||||
|
# under a second name, and the stack that needed the second name is being
|
||||||
|
# collapsed onto the consolidated image.)
|
||||||
branches: [main, dev]
|
branches: [main, dev]
|
||||||
#
|
#
|
||||||
# NO tag trigger (milestone 318 step 2). A `v*` tag names a commit `main`
|
# NO tag trigger (milestone 318 step 2). A `v*` tag names a commit `main`
|
||||||
@@ -35,8 +38,8 @@ on:
|
|||||||
# workflow is not shipped bytes, so it is in no artifact's path set. Putting
|
# workflow is not shipped bytes, so it is in no artifact's path set. Putting
|
||||||
# it in one would re-version every artifact for a comment change.
|
# it in one would re-version every artifact for a comment change.
|
||||||
#
|
#
|
||||||
# ONE input, not one per artifact. Forcing all three is cheap once the
|
# ONE input, not one per artifact. Forcing both is cheap once the registry
|
||||||
# registry cache is warm (#3114), and three booleans is an interface nobody
|
# cache is warm (#3114), and a boolean per artifact is an interface nobody
|
||||||
# remembers the meaning of.
|
# remembers the meaning of.
|
||||||
workflow_dispatch:
|
workflow_dispatch:
|
||||||
inputs:
|
inputs:
|
||||||
@@ -787,8 +790,8 @@ jobs:
|
|||||||
#
|
#
|
||||||
# build-web additionally exposes this as `outputs.candidate`, which is
|
# build-web additionally exposes this as `outputs.candidate`, which is
|
||||||
# what gates the `promote` job — a job's `if:` cannot read `env`, and
|
# what gates the `promote` job — a job's `if:` cannot read `env`, and
|
||||||
# one flag is enough because all three derive it from the same
|
# one flag is enough because both jobs derive it from the same
|
||||||
# IS_REFRESH. ml and agent do not re-emit it; a second copy nothing
|
# IS_REFRESH. build-agent does not re-emit it; a second copy nothing
|
||||||
# reads is the kind of thing that later reads as load-bearing.
|
# reads is the kind of thing that later reads as load-bearing.
|
||||||
if [ "${IS_REFRESH:-}" = "true" ]; then
|
if [ "${IS_REFRESH:-}" = "true" ]; then
|
||||||
echo "build_ref=$IMAGE:refresh-candidate" >> "$GITHUB_OUTPUT"
|
echo "build_ref=$IMAGE:refresh-candidate" >> "$GITHUB_OUTPUT"
|
||||||
@@ -1523,18 +1526,19 @@ jobs:
|
|||||||
# a FAILED gate blocks would have published unverified images while reporting
|
# a FAILED gate blocks would have published unverified images while reporting
|
||||||
# success. Not running is not the same as passing.
|
# success. Not running is not the same as passing.
|
||||||
#
|
#
|
||||||
# All three images promote TOGETHER, or none do. They are one stack: build.yml
|
# BOTH images promote TOGETHER, or neither does. They are one stack:
|
||||||
# already refuses to publish a :dev web image beside a stale :dev ml, because
|
# build.yml already refuses to publish a :dev web image beside a stale :dev
|
||||||
# the mismatch only shows up as a runtime failure. A refresh that published ml
|
# agent, because the mismatch only shows up as a runtime failure. A refresh
|
||||||
# and withheld web would be that same trap, arrived at through the gate.
|
# that published the agent and withheld web would be that same trap, arrived
|
||||||
|
# at through the gate.
|
||||||
#
|
#
|
||||||
# The gate covers the web image only (milestone 362 step 3 scoped it there),
|
# The gate covers the web image only (milestone 362 step 3 scoped it there),
|
||||||
# so ml and agent are being held to web's verdict rather than their own. That
|
# so the agent is being held to web's verdict rather than its own. That is
|
||||||
# is deliberate and it is the conservative direction — they ship together, so
|
# deliberate and it is the conservative direction — they ship together, so
|
||||||
# the weakest evidence should govern all three — but it is not the same as
|
# the weakest evidence should govern both — but it is not the same as having
|
||||||
# having smoked them, and it should not be read as if it were.
|
# smoked it, and it should not be read as if it were.
|
||||||
promote:
|
promote:
|
||||||
needs: [build-web, build-ml, build-agent, smoke-web]
|
needs: [build-web, build-agent, smoke-web]
|
||||||
# Only a refresh publishes through a candidate; a push writes its channel
|
# Only a refresh publishes through a candidate; a push writes its channel
|
||||||
# tag directly from the build. Reads the same reuse-step decision the build
|
# tag directly from the build. Reads the same reuse-step decision the build
|
||||||
# took, via a job output — a job's `if:` cannot see the `env` context.
|
# took, via a job output — a job's `if:` cannot see the `env` context.
|
||||||
@@ -1555,7 +1559,7 @@ jobs:
|
|||||||
TAG=latest
|
TAG=latest
|
||||||
FAILED=""
|
FAILED=""
|
||||||
|
|
||||||
for NAME in fabledcurator fabledcurator-ml fabledcurator-agent; do
|
for NAME in fabledcurator fabledcurator-agent; do
|
||||||
REPO="bvandeusen/$NAME"
|
REPO="bvandeusen/$NAME"
|
||||||
echo "promote: $REPO"
|
echo "promote: $REPO"
|
||||||
|
|
||||||
@@ -1616,144 +1620,7 @@ jobs:
|
|||||||
echo "promote: idempotent." >&2
|
echo "promote: idempotent." >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
echo "promote: all three channel tags moved"
|
echo "promote: both channel tags moved"
|
||||||
# NOT A BUILD. `fabledcurator-ml` is the SAME IMAGE as `fabledcurator` and
|
|
||||||
# has been since milestone 422 step 6 merged the ML layers into the one
|
|
||||||
# Dockerfile — this job built `file: Dockerfile, context: .`, byte for byte
|
|
||||||
# what build-web builds, and published it under a second name.
|
|
||||||
#
|
|
||||||
# So it was doing the whole build twice. Measured on run 7282, a cold cache:
|
|
||||||
# build-web 1m54s, build-ml 1m55s, for identical output — plus a second
|
|
||||||
# push of a few hundred MB. Operator, 2026-09-22: *"please fix the CI so it
|
|
||||||
# doesn't do this superfluous work."*
|
|
||||||
#
|
|
||||||
# It now publishes NOTHING OF ITS OWN. It re-tags the manifest build-web
|
|
||||||
# already pushed, so the second name goes on working for the operator's
|
|
||||||
# Swarm stack — which still references it — while CI stops building twice.
|
|
||||||
#
|
|
||||||
# The name is retired entirely in #4311, once that stack points its
|
|
||||||
# ml-worker service at `fabledcurator:latest` with `command: ["ml-worker"]`.
|
|
||||||
# Nothing here is load-bearing after that: the whole job goes, rather than
|
|
||||||
# this comment growing another paragraph.
|
|
||||||
build-ml:
|
|
||||||
# Was parallel with build-web. It cannot be any more — there is nothing to
|
|
||||||
# copy until build-web has decided what this run publishes. That is the
|
|
||||||
# cost of not building twice, and it is seconds: this job transfers no
|
|
||||||
# layers the runner does not already have.
|
|
||||||
needs: [build-web]
|
|
||||||
runs-on: python-ci
|
|
||||||
container:
|
|
||||||
image: git.fabledsword.com/bvandeusen/ci-python:3.14
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
with:
|
|
||||||
ref: ${{ env.BUILD_REF }}
|
|
||||||
# Shallow is enough now. This job derives no version from history:
|
|
||||||
# it publishes no content, so it has no artifact identity of its own
|
|
||||||
# and `artifacts.sh revision ml` decides nothing here any more.
|
|
||||||
|
|
||||||
# See sign-extension's copy for why this guard exists.
|
|
||||||
- name: Guard — a scheduled run must have checked out main
|
|
||||||
if: env.IS_REFRESH == 'true'
|
|
||||||
run: |
|
|
||||||
set -eu
|
|
||||||
BRANCH=$(git rev-parse --abbrev-ref HEAD)
|
|
||||||
echo "schedule: HEAD is $BRANCH ($(git rev-parse --short HEAD))"
|
|
||||||
if [ "$BRANCH" != "main" ]; then
|
|
||||||
echo "schedule: expected main, got '$BRANCH'." >&2
|
|
||||||
echo "schedule: BUILD_REF was not honoured by the runner." >&2
|
|
||||||
echo "schedule: refusing to publish a channel tag from it." >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Shell step rather than docker/login-action — see build-web's note on
|
|
||||||
# the shared action-cache race (#3118).
|
|
||||||
- name: Login to Forgejo registry
|
|
||||||
env:
|
|
||||||
TOKEN: ${{ secrets.RELEASE_TOKEN }}
|
|
||||||
ACTOR: ${{ github.actor }}
|
|
||||||
run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin
|
|
||||||
|
|
||||||
- name: Point fabledcurator-ml at the image fabledcurator published
|
|
||||||
env:
|
|
||||||
SRC: git.fabledsword.com/bvandeusen/fabledcurator
|
|
||||||
DST: git.fabledsword.com/bvandeusen/fabledcurator-ml
|
|
||||||
BUILT_DIGEST: ${{ needs.build-web.outputs.digest }}
|
|
||||||
PUBLISHED_DIGEST: ${{ needs.build-web.outputs.published_digest }}
|
|
||||||
run: |
|
|
||||||
set -eu
|
|
||||||
# WHERE THE BYTES COME FROM. build-web either pushed a manifest this
|
|
||||||
# run, or it hit reuse — in which case the digest its channel tag
|
|
||||||
# already names is the right source, because that is what "hit"
|
|
||||||
# MEANS: the reuse step read this commit's fc.revision off that tag.
|
|
||||||
DIGEST="${BUILT_DIGEST:-}"
|
|
||||||
[ -n "$DIGEST" ] || DIGEST="${PUBLISHED_DIGEST:-}"
|
|
||||||
if [ -z "$DIGEST" ]; then
|
|
||||||
echo "alias: FAILED — build-web neither built an image nor" >&2
|
|
||||||
echo "alias: resolved a published one, so there is nothing for" >&2
|
|
||||||
echo "alias: fabledcurator-ml to be pointed at." >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# WHICH TAGS, mirroring exactly what build-web wrote for itself.
|
|
||||||
#
|
|
||||||
# A refresh writes the CANDIDATE tag and nothing else, so `promote`
|
|
||||||
# moves it to :latest after smoke-web passes. That keeps this name
|
|
||||||
# under the same gate it has today and needs no change to promote,
|
|
||||||
# which already loops over all three image names.
|
|
||||||
SHORT_SHA=$(printf '%s' "$GITHUB_SHA" | cut -c1-7)
|
|
||||||
if [ "${IS_REFRESH:-}" = "true" ]; then
|
|
||||||
TAGS="refresh-candidate"
|
|
||||||
elif [ "${GITHUB_REF##*/}" = "main" ]; then
|
|
||||||
TAGS="latest c-${SHORT_SHA}"
|
|
||||||
else
|
|
||||||
TAGS="dev"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Pull, tag, push — deliberately NOT `imagetools create`.
|
|
||||||
#
|
|
||||||
# That wraps its source in an INDEX, and `.Image.Config.Labels` does
|
|
||||||
# not resolve through one (#3183, run 4751). Worse here than it was
|
|
||||||
# there: promote asks the registry for IMAGE manifest media types
|
|
||||||
# ONLY, so an index sitting at :refresh-candidate would fail its
|
|
||||||
# `test -n "$CT"` and break the weekly refresh for this name.
|
|
||||||
#
|
|
||||||
# MEASURED on run 7300, the first execution: the push moved ZERO
|
|
||||||
# layer bytes. Every blob came back either "Layer already exists" or
|
|
||||||
# "Mounted from bvandeusen/fabledcurator" — the registry does
|
|
||||||
# cross-repository blob mounts between two repos of one owner, so
|
|
||||||
# this copies a manifest and nothing else. 22s, against the ~2min
|
|
||||||
# the duplicate build cost (run 7282: build-ml 1m55s).
|
|
||||||
#
|
|
||||||
# The pushed manifest gets its OWN digest (b979b145… where the
|
|
||||||
# source was aca4c75…) because docker re-serialises it. Same layers,
|
|
||||||
# same config, same fc.revision; only the manifest bytes differ.
|
|
||||||
# Nothing compares the two, and promote reads whatever is at the tag
|
|
||||||
# — but it is worth saying, because the digests looking different is
|
|
||||||
# the first thing someone will notice and misread as a second build.
|
|
||||||
docker pull "$SRC@$DIGEST"
|
|
||||||
for t in $TAGS; do
|
|
||||||
echo "alias: $DST:$t -> $SRC@$DIGEST"
|
|
||||||
docker tag "$SRC@$DIGEST" "$DST:$t"
|
|
||||||
docker push "$DST:$t"
|
|
||||||
done
|
|
||||||
|
|
||||||
# Read it back. A push that reported success but left the tag
|
|
||||||
# elsewhere is exactly the silent-and-plausible failure this
|
|
||||||
# pipeline keeps producing, and the check costs one request.
|
|
||||||
for t in $TAGS; do
|
|
||||||
NOW=$(docker buildx imagetools inspect "$DST:$t" \
|
|
||||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' 2>/dev/null || echo "")
|
|
||||||
echo "alias: $DST:$t now carries fc.revision=${NOW:-<none>}"
|
|
||||||
if [ -z "$NOW" ]; then
|
|
||||||
echo "alias: FAILED — $DST:$t has no readable fc.revision." >&2
|
|
||||||
echo "alias: The label resolves through a plain image manifest" >&2
|
|
||||||
echo "alias: and not through an index, so this means the tag is" >&2
|
|
||||||
echo "alias: index-wrapped — which breaks promote's refresh path." >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
build-agent:
|
build-agent:
|
||||||
runs-on: python-ci
|
runs-on: python-ci
|
||||||
container:
|
container:
|
||||||
@@ -1961,8 +1828,8 @@ jobs:
|
|||||||
#
|
#
|
||||||
# build-web additionally exposes this as `outputs.candidate`, which is
|
# build-web additionally exposes this as `outputs.candidate`, which is
|
||||||
# what gates the `promote` job — a job's `if:` cannot read `env`, and
|
# what gates the `promote` job — a job's `if:` cannot read `env`, and
|
||||||
# one flag is enough because all three derive it from the same
|
# one flag is enough because both jobs derive it from the same
|
||||||
# IS_REFRESH. ml and agent do not re-emit it; a second copy nothing
|
# IS_REFRESH. build-agent does not re-emit it; a second copy nothing
|
||||||
# reads is the kind of thing that later reads as load-bearing.
|
# reads is the kind of thing that later reads as load-bearing.
|
||||||
if [ "${IS_REFRESH:-}" = "true" ]; then
|
if [ "${IS_REFRESH:-}" = "true" ]; then
|
||||||
echo "build_ref=$IMAGE:refresh-candidate" >> "$GITHUB_OUTPUT"
|
echo "build_ref=$IMAGE:refresh-candidate" >> "$GITHUB_OUTPUT"
|
||||||
|
|||||||
@@ -260,12 +260,11 @@ commits since the previous tag; it builds no image.
|
|||||||
|
|
||||||
## What's in here
|
## What's in here
|
||||||
|
|
||||||
Five deployable pieces, built by `.forgejo/workflows/build.yml`:
|
Four deployable pieces, built by `.forgejo/workflows/build.yml`:
|
||||||
|
|
||||||
| Piece | Built from | Image | Role |
|
| Piece | Built from | Image | Role |
|
||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| **Web / workers** | `Dockerfile` | `fabledcurator` | Quart API + the built Vue SPA in one image. `entrypoint.sh` picks the role: `web`, `worker`, `scheduler`. The `maintenance-long` service is a second `worker` pinned to the long-running maintenance queue. |
|
| **Web / workers** | `Dockerfile` | `fabledcurator` | Quart API + the built Vue SPA in one image. `entrypoint.sh` picks the role: `web`, `worker`, `scheduler`, `ml-worker`, or `all` (every lane under supervisord, the single-container layout). The `maintenance-long` service is a second `worker` pinned to the long-running maintenance queue. |
|
||||||
| **ML worker** | `Dockerfile` | `fabledcurator-ml` | The same image as the web service since milestone 422 — one image serves every lane. Published under this name too, for stacks that still reference it. |
|
|
||||||
| **GPU agent** | `agent/Dockerfile` | `fabledcurator-agent` | Optional desktop-GPU worker (`agent/`). Leases jobs over **HTTP only** — never touches the database or Redis. See `agent/README.md`. |
|
| **GPU agent** | `agent/Dockerfile` | `fabledcurator-agent` | Optional desktop-GPU worker (`agent/`). Leases jobs over **HTTP only** — never touches the database or Redis. See `agent/README.md`. |
|
||||||
| **Firefox extension** | `extension/` | signed XPI | MV3 extension: pushes platform session cookies into FC and adds a creator as a Source in one click. AMO-signed on both `dev` and `main` (one signature per extension change, shared by the two channels), bundled into that channel's web image and served from Settings → Maintenance. See `extension/README.md`. |
|
| **Firefox extension** | `extension/` | signed XPI | MV3 extension: pushes platform session cookies into FC and adds a creator as a Source in one click. AMO-signed on both `dev` and `main` (one signature per extension change, shared by the two channels), bundled into that channel's web image and served from Settings → Maintenance. See `extension/README.md`. |
|
||||||
| **Data** | — | `pgvector/pgvector:pg16`, `redis:7-alpine` | Postgres with pgvector for embeddings; Redis as the Celery broker. |
|
| **Data** | — | `pgvector/pgvector:pg16`, `redis:7-alpine` | Postgres with pgvector for embeddings; Redis as the Celery broker. |
|
||||||
|
|||||||
+1
-1
@@ -233,7 +233,7 @@ services:
|
|||||||
redis: { condition: service_healthy }
|
redis: { condition: service_healthy }
|
||||||
|
|
||||||
ml-worker:
|
ml-worker:
|
||||||
image: git.fabledsword.com/bvandeusen/fabledcurator-ml:latest
|
image: git.fabledsword.com/bvandeusen/fabledcurator:latest
|
||||||
command: ["ml-worker"]
|
command: ["ml-worker"]
|
||||||
# A single GPU inference pass can run tens of seconds — let it finish.
|
# A single GPU inference pass can run tens of seconds — let it finish.
|
||||||
stop_grace_period: 120s
|
stop_grace_period: 120s
|
||||||
|
|||||||
+40
-23
@@ -8,7 +8,7 @@
|
|||||||
# never reaches the image — see DERIVER below, and #3156 for the same finding
|
# never reaches the image — see DERIVER below, and #3156 for the same finding
|
||||||
# about packaging.sh.
|
# about packaging.sh.
|
||||||
#
|
#
|
||||||
# Four artifacts, four independent versions. An artifact whose shipped files
|
# Three artifacts, three independent versions. An artifact whose shipped files
|
||||||
# did not change keeps its version and does not rebuild — that is the whole
|
# did not change keeps its version and does not rebuild — that is the whole
|
||||||
# point, and it is why each path set must match its Dockerfile rather than
|
# point, and it is why each path set must match its Dockerfile rather than
|
||||||
# being a plausible guess. Getting a set wrong is quiet in BOTH directions:
|
# being a plausible guess. Getting a set wrong is quiet in BOTH directions:
|
||||||
@@ -51,21 +51,6 @@ ROOT=$(git rev-parse --show-toplevel)
|
|||||||
# rather than restated — one definition, per #2397.
|
# rather than restated — one definition, per #2397.
|
||||||
WEB_PATHS='Dockerfile requirements.txt requirements-ml.txt backend alembic alembic.ini entrypoint.sh frontend :(exclude)frontend/test :(exclude)frontend/test/**'
|
WEB_PATHS='Dockerfile requirements.txt requirements-ml.txt backend alembic alembic.ini entrypoint.sh frontend :(exclude)frontend/test :(exclude)frontend/test/**'
|
||||||
|
|
||||||
# ml — THE SAME IMAGE as web since milestone 422 step 6, built from the same
|
|
||||||
# Dockerfile with the same context and published under a second name only
|
|
||||||
# because the operator's Swarm stack still references it.
|
|
||||||
#
|
|
||||||
# So it is the same path set, by assignment rather than by a copy that would
|
|
||||||
# drift. This was two lists describing one image for about an hour, and the
|
|
||||||
# artifact-paths guard caught it immediately: the merged Dockerfile carries
|
|
||||||
# the frontend-builder stage, so the ml image copies `frontend/package.json`,
|
|
||||||
# and the old ml list did not cover it — its version would not have moved when
|
|
||||||
# the frontend changed, and a pinned build would serve stale bytes (#3202).
|
|
||||||
#
|
|
||||||
# `requirements-ml.txt` is in WEB_PATHS for the mirror-image reason: the web
|
|
||||||
# image now installs the ML requirements, so changing them changes it.
|
|
||||||
ML_PATHS="$WEB_PATHS"
|
|
||||||
|
|
||||||
# agent (agent/Dockerfile, context `agent`) — copies requirements.txt and
|
# agent (agent/Dockerfile, context `agent`) — copies requirements.txt and
|
||||||
# fc_agent only. agent/README.md, agent/docker-compose.yml and agent/ruff.toml
|
# fc_agent only. agent/README.md, agent/docker-compose.yml and agent/ruff.toml
|
||||||
# live in the directory but never reach the image, so they must not re-version
|
# live in the directory but never reach the image, so they must not re-version
|
||||||
@@ -78,7 +63,7 @@ AGENT_PATHS='agent/Dockerfile agent/requirements.txt agent/fc_agent'
|
|||||||
#
|
#
|
||||||
# Why web and nothing else. Every artifact stamps `fc.revision`, but only web
|
# Why web and nothing else. Every artifact stamps `fc.revision`, but only web
|
||||||
# also stamps a version (build.yml line ~488 feeds `version web` to the
|
# also stamps a version (build.yml line ~488 feeds `version web` to the
|
||||||
# FC_VERSION build arg; ml and agent ask for `revision` alone, and the
|
# FC_VERSION build arg; the agent asks for `revision` alone, and the
|
||||||
# extension takes its version from packaging.sh). For a revision-only artifact
|
# extension takes its version from packaging.sh). For a revision-only artifact
|
||||||
# this file needs no entry: any change to how the revision is COMPUTED changes
|
# this file needs no entry: any change to how the revision is COMPUTED changes
|
||||||
# the derived value, which then disagrees with the label on the published image
|
# the derived value, which then disagrees with the label on the published image
|
||||||
@@ -101,7 +86,7 @@ DERIVER='scripts/artifacts.sh'
|
|||||||
|
|
||||||
|
|
||||||
usage() {
|
usage() {
|
||||||
echo "usage: artifacts.sh {paths|revision|version|epoch} {web|ml|agent|extension}" >&2
|
echo "usage: artifacts.sh {paths|revision|version|epoch} {web|agent|extension}" >&2
|
||||||
exit 2
|
exit 2
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -115,9 +100,6 @@ ext_paths() {
|
|||||||
cmd_paths() {
|
cmd_paths() {
|
||||||
case "$1" in
|
case "$1" in
|
||||||
web) echo "$WEB_PATHS $DERIVER $(ext_paths)" ;;
|
web) echo "$WEB_PATHS $DERIVER $(ext_paths)" ;;
|
||||||
# Identical to web, deliberately: it IS web's image. That includes the
|
|
||||||
# deriver and the bundled extension — the XPI is in these bytes too.
|
|
||||||
ml) echo "$ML_PATHS $DERIVER $(ext_paths)" ;;
|
|
||||||
agent) echo "$AGENT_PATHS" ;;
|
agent) echo "$AGENT_PATHS" ;;
|
||||||
extension) ext_paths ;;
|
extension) ext_paths ;;
|
||||||
*) usage ;;
|
*) usage ;;
|
||||||
@@ -128,8 +110,21 @@ cmd_paths() {
|
|||||||
# Unquoted on purpose: the pathspec must word-split into separate args.
|
# Unquoted on purpose: the pathspec must word-split into separate args.
|
||||||
# Globbing is already off script-wide.
|
# Globbing is already off script-wide.
|
||||||
newest() {
|
newest() {
|
||||||
# shellcheck disable=SC2046
|
# Resolve the path set into a variable rather than inlining it as
|
||||||
set -- "$(cd "$ROOT" && git log --format='%ct %H' HEAD -- $(cmd_paths "$1") \
|
# `git log ... -- $(cmd_paths "$1")`. An empty substitution there leaves
|
||||||
|
# `git log HEAD --` with NO pathspec, which answers with the newest commit
|
||||||
|
# in the whole repository instead of failing — see the dispatch guard at the
|
||||||
|
# foot of this file for what that cost and why the real check lives there.
|
||||||
|
#
|
||||||
|
# This `|| exit 2` cannot be that check (it exits a subshell, since every
|
||||||
|
# caller wraps `newest` in one) — it is belt and braces for a future caller
|
||||||
|
# that reaches `newest` without passing the guard.
|
||||||
|
paths=$(cmd_paths "$1") || exit 2
|
||||||
|
|
||||||
|
# Unquoted on purpose: the pathspec must word-split into separate args.
|
||||||
|
# Globbing is off script-wide (`set -f`).
|
||||||
|
# shellcheck disable=SC2086
|
||||||
|
set -- "$(cd "$ROOT" && git log --format='%ct %H' HEAD -- $paths \
|
||||||
| sort -n | tail -1)"
|
| sort -n | tail -1)"
|
||||||
if [ -z "$1" ]; then
|
if [ -z "$1" ]; then
|
||||||
echo "artifacts.sh: no commit touches this artifact's shipped files" >&2
|
echo "artifacts.sh: no commit touches this artifact's shipped files" >&2
|
||||||
@@ -226,6 +221,28 @@ cmd_version() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
[ $# -ge 2 ] || usage
|
[ $# -ge 2 ] || usage
|
||||||
|
|
||||||
|
# Validate the ARTIFACT here, in the main shell, before anything dispatches.
|
||||||
|
#
|
||||||
|
# It cannot be done deeper down. `cmd_revision`, `cmd_version` and `cmd_epoch`
|
||||||
|
# all call `newest` inside a command substitution, and `newest` resolves the
|
||||||
|
# path set inside another one — so an `exit` from either kills only that
|
||||||
|
# subshell. The script carried on, printed nothing, and exited 0.
|
||||||
|
#
|
||||||
|
# Before this, `revision ml` was worse than nothing: the empty substitution
|
||||||
|
# left `git log HEAD --` with no pathspec, so it answered with the newest
|
||||||
|
# commit in the WHOLE REPOSITORY — a real-looking 12-char sha on stdout, exit
|
||||||
|
# 0, and the usage line on stderr where no caller reads it. The reuse check
|
||||||
|
# would have compared that against a published label, missed, and rebuilt
|
||||||
|
# every push forever without a red run anywhere.
|
||||||
|
#
|
||||||
|
# Latent while every name callers passed was valid. #4311 removed `ml` from
|
||||||
|
# the set, which made a name that used to work start taking that path.
|
||||||
|
case "$2" in
|
||||||
|
web|agent|extension) ;;
|
||||||
|
*) usage ;;
|
||||||
|
esac
|
||||||
|
|
||||||
case "$1" in
|
case "$1" in
|
||||||
paths) cmd_paths "$2" ;;
|
paths) cmd_paths "$2" ;;
|
||||||
revision) cmd_revision "$2" ;;
|
revision) cmd_revision "$2" ;;
|
||||||
|
|||||||
@@ -83,7 +83,6 @@ API = "https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator"
|
|||||||
|
|
||||||
IMAGES = (
|
IMAGES = (
|
||||||
"git.fabledsword.com/bvandeusen/fabledcurator",
|
"git.fabledsword.com/bvandeusen/fabledcurator",
|
||||||
"git.fabledsword.com/bvandeusen/fabledcurator-ml",
|
|
||||||
"git.fabledsword.com/bvandeusen/fabledcurator-agent",
|
"git.fabledsword.com/bvandeusen/fabledcurator-agent",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -31,7 +31,6 @@ ROOT = Path(__file__).resolve().parent.parent
|
|||||||
# artifact -> (dockerfile, build context relative to the repo root)
|
# artifact -> (dockerfile, build context relative to the repo root)
|
||||||
ARTIFACTS = {
|
ARTIFACTS = {
|
||||||
"web": ("Dockerfile", ""),
|
"web": ("Dockerfile", ""),
|
||||||
"ml": ("Dockerfile", ""),
|
|
||||||
"agent": ("agent/Dockerfile", "agent"),
|
"agent": ("agent/Dockerfile", "agent"),
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -133,7 +132,7 @@ DERIVERS = [
|
|||||||
# orphaned. Guarded for web too, since web bundles what the extension makes.
|
# orphaned. Guarded for web too, since web bundles what the extension makes.
|
||||||
("extension/scripts/packaging.sh", ("extension", "web")),
|
("extension/scripts/packaging.sh", ("extension", "web")),
|
||||||
# artifacts.sh decides the FC_VERSION baked into the web image (#3202).
|
# artifacts.sh decides the FC_VERSION baked into the web image (#3202).
|
||||||
# Web only, and deliberately: ml and agent ask this script for `revision`
|
# Web only, and deliberately: the agent asks this script for `revision`
|
||||||
# alone, so they are covered by the self-correcting path above, and the
|
# alone, so they are covered by the self-correcting path above, and the
|
||||||
# extension takes its version from packaging.sh. Milestone 318 step 5 is
|
# extension takes its version from packaging.sh. Milestone 318 step 5 is
|
||||||
# the worked instance — b3989d0 and 5771fd5 share revision fb2c4d5b80be
|
# the worked instance — b3989d0 and 5771fd5 share revision fb2c4d5b80be
|
||||||
@@ -197,3 +196,30 @@ def test_files_that_never_reach_an_image_do_not_version_it(artifact, path):
|
|||||||
f"image — it would re-version and rebuild {artifact} for a change it "
|
f"image — it would re-version and rebuild {artifact} for a change it "
|
||||||
f"does not ship."
|
f"does not ship."
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_unknown_artifact_fails_instead_of_answering():
|
||||||
|
"""It used to answer — with the newest commit in the whole repository.
|
||||||
|
|
||||||
|
`newest()` inlined the path set as `git log ... -- $(cmd_paths "$1")`.
|
||||||
|
`usage` exits from the command SUBSHELL, so an unknown name made the
|
||||||
|
substitution come back empty and `git log HEAD --` walked everything:
|
||||||
|
stdout got a real-looking 12-char sha, the exit code was 0, and the usage
|
||||||
|
line went to stderr where no caller reads it. A wrong answer shaped
|
||||||
|
exactly like a right one, which the reuse check would have compared
|
||||||
|
against a published label and quietly rebuilt on forever.
|
||||||
|
|
||||||
|
Latent while every name callers passed was valid. #4311 removed `ml` from
|
||||||
|
that set, which is what made a name that used to work start taking the
|
||||||
|
silent path.
|
||||||
|
"""
|
||||||
|
out = subprocess.run(
|
||||||
|
["sh", str(ROOT / "scripts" / "artifacts.sh"), "revision", "ml"],
|
||||||
|
capture_output=True, text=True, cwd=ROOT,
|
||||||
|
)
|
||||||
|
assert out.returncode != 0, (
|
||||||
|
f"an unknown artifact exited 0 and printed {out.stdout!r}"
|
||||||
|
)
|
||||||
|
assert not out.stdout.strip(), (
|
||||||
|
f"an unknown artifact printed {out.stdout!r} on stdout"
|
||||||
|
)
|
||||||
|
|||||||
@@ -195,13 +195,13 @@ def test_a_non_tag_ref_renders_but_refuses_to_claim_it_published():
|
|||||||
body_of(out)
|
body_of(out)
|
||||||
|
|
||||||
|
|
||||||
def test_the_rollback_refs_name_all_three_images():
|
def test_the_rollback_refs_name_both_images():
|
||||||
"""Rule 145: `:c-<sha>` is the rollback unit, and the three images move
|
"""Rule 145: `:c-<sha>` is the rollback unit, and both images move
|
||||||
together. A release listing only the web image sends an operator into a
|
together. A release listing only the web image sends an operator into a
|
||||||
rollback that leaves ml and agent on the newer build — the exact mismatch
|
rollback that leaves the agent on the newer build — the exact mismatch
|
||||||
build.yml builds all three on every push to avoid."""
|
build.yml builds both on every push to avoid."""
|
||||||
body = body_of(notes("HEAD"))
|
body = body_of(notes("HEAD"))
|
||||||
for image in ("fabledcurator", "fabledcurator-ml", "fabledcurator-agent"):
|
for image in ("fabledcurator", "fabledcurator-agent"):
|
||||||
assert f"bvandeusen/{image}:c-" in body, f"{image} missing from the rollback refs"
|
assert f"bvandeusen/{image}:c-" in body, f"{image} missing from the rollback refs"
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user