bvandeusen/FabledCurator
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 13 Wiki Activity

perf: stop building the same image twice under two names (4311)
CI / lint (push) Successful in 3s
Details
CI / extension-version (push) Successful in 3s
Details
Build images / sign-extension (push) Successful in 4s
Details
Build images / build-agent (push) Successful in 6s
Details
Build images / build-web (push) Successful in 5s
Details
CI / frontend-build (push) Successful in 23s
Details
extension / lint (push) Successful in 23s
Details
CI / backend-lint-and-test (push) Successful in 32s
Details
Build images / build-ml (push) Successful in 22s
Details
Build images / smoke-web (push) Successful in 30s
Details
Build images / promote (push) Skipped
Details
CI / integration (push) Successful in 2m17s
Details

Browse Source 
`fabledcurator-ml` has been the SAME IMAGE as `fabledcurator` since milestone
422 step 6 merged the ML layers into the one Dockerfile. `build-ml` was still
running `file: Dockerfile, context: .` — byte for byte what `build-web`
builds — and publishing the result under a second name. So CI did the whole
build twice. Measured on run 7282, cold cache: build-web 1m54s, build-ml
1m55s, for identical output, plus a second push of a few hundred MB.

Operator, 2026-09-22: *"please fix the CI so it doesn't do this superfluous
work."*

build-ml now publishes nothing of its own. It pulls the manifest build-web
just pushed, re-tags it, and pushes that — so the second name goes on working
for the operator's Swarm stack, which still references it, while CI stops
compiling anything twice. 437 lines of duplicated build machinery go with it:
its own buildx setup, its own reuse check, its own version derivation, its
own tag repoint.

It now `needs: [build-web]` rather than running in parallel. That is the one
cost, and it is seconds: there is nothing to copy until build-web has decided
what this run publishes, and the layers are already on the runner.

Deliberately NOT `imagetools create`, which would have been the obvious
one-liner. That wraps its source in an INDEX and `.Image.Config.Labels` does
not resolve through one (#3183, run 4751) — and it would bite harder here
than it did there, because `promote` asks the registry for IMAGE manifest
media types ONLY, so an index at :refresh-candidate would fail its
`test -n "$CT"` and break the weekly refresh for this name. Plain
pull/tag/push keeps a plain manifest, which rule 40 also asks for. The step
reads the tag back and fails if `fc.revision` stops resolving, so a future
change that reintroduces index-wrapping says so instead of quietly costing
the refresh.

The tag list mirrors build-web's exactly, refresh included: a refresh writes
:refresh-candidate alone and `promote` moves it to :latest after smoke-web
passes. So this name keeps the gate it has today and `promote` needs no
change — it already loops over all three image names.

Source is build-web's `digest` when it built, else its `published_digest` on
a reuse hit — the digest its channel tag already names, which is what a hit
MEANS. With neither, the job fails with a reason rather than publishing
something arbitrary.

This does not retire the name; #4311 still does that, and is still gated on
the operator's stack pointing ml-worker at `fabledcurator:latest` with
`command: ["ml-worker"]`. Until then nothing of theirs changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
bvandeusen
2026-09-23 07:20:30 -04:00
co-authored by Claude Opus 5
parent 187b6d2cdf
commit 815268418e
1 changed files with 85 additions and 436 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.forgejo/workflows/build.yml
+85 -436
View File
@@ -1617,24 +1617,40 @@ jobs:
exit 1 exit 1
fi fi
echo "promote: all three channel tags moved" echo "promote: all three channel tags moved"
# NOT A BUILD. `fabledcurator-ml` is the SAME IMAGE as `fabledcurator` and
# has been since milestone 422 step 6 merged the ML layers into the one
# Dockerfile — this job built `file: Dockerfile, context: .`, byte for byte
# what build-web builds, and published it under a second name.
#
# So it was doing the whole build twice. Measured on run 7282, a cold cache:
# build-web 1m54s, build-ml 1m55s, for identical output — plus a second
# push of a few hundred MB. Operator, 2026-09-22: *"please fix the CI so it
# doesn't do this superfluous work."*
#
# It now publishes NOTHING OF ITS OWN. It re-tags the manifest build-web
# already pushed, so the second name goes on working for the operator's
# Swarm stack — which still references it — while CI stops building twice.
#
# The name is retired entirely in #4311, once that stack points its
# ml-worker service at `fabledcurator:latest` with `command: ["ml-worker"]`.
# Nothing here is load-bearing after that: the whole job goes, rather than
# this comment growing another paragraph.
build-ml: build-ml:
# Was parallel with build-web. It cannot be any more — there is nothing to
# copy until build-web has decided what this run publishes. That is the
# cost of not building twice, and it is seconds: this job transfers no
# layers the runner does not already have.
needs: [build-web]
runs-on: python-ci runs-on: python-ci
container: container:
image: git.fabledsword.com/bvandeusen/ci-python:3.14 image: git.fabledsword.com/bvandeusen/ci-python:3.14
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with: with:
# Not the triggering ref — see the `env:` block at the top. On a
# scheduled refresh this is `main`; on everything else it is the ref
# that fired, so this is a no-op on every ordinary path.
ref: ${{ env.BUILD_REF }} ref: ${{ env.BUILD_REF }}
# Full history: this job derives its artifact's version from the # Shallow is enough now. This job derives no version from history:
# commit its shipped files last changed in (milestone 313). A # it publishes no content, so it has no artifact identity of its own
# depth-1 clone cannot see that commit — it either derives a wrong, # and `artifacts.sh revision ml` decides nothing here any more.
# too-low value or finds nothing at all, and neither is a failure
# the build would otherwise notice.
fetch-depth: 0
# See sign-extension's copy for why this guard exists. # See sign-extension's copy for why this guard exists.
- name: Guard — a scheduled run must have checked out main - name: Guard — a scheduled run must have checked out main
@@ -1650,66 +1666,6 @@ jobs:
exit 1 exit 1
fi fi
# --- derived values, one line (milestone 313) ------------------------
# These stopped being shadow output at step 3. `revision` decides
# whether the build below runs at all and `version` is what the image
# reports about itself; the load-bearing steps each print only the one
# they use, so this is the only place the pair appears together. When a
# build is skipped, this is the line that says what the commit derived.
#
# Still diagnostic, so it still must not fail the build — no `set -e`,
# and every derivation falls back to UNAVAILABLE. A broken echo must
# never be the reason an image does not ship.
#
# What it should say:
# * a push touching only agent/ moves the agent and leaves web and ml
# STILL. If web moves, its path set is too wide.
# * a push touching only docs moves nothing.
# * a push touching the extension moves the extension AND web, since
# web bakes in the XPI. If web does not move, its set is too narrow:
# the reuse check hits, and the channel serves a web image bundling
# the PREVIOUS XPI while the freshly signed one is orphaned (#3156).
# * dev and main derive the same values for the same source.
- name: Report the derived artifact version
env:
# Diagnostic for the trigger normalisation. `refresh` is reported RAW
# as well as normalised, because the two disagreeing is the whole
# failure mode: a dispatch input whose type does not compare the way
# the expression assumes evaluates to false silently, and the only
# symptom is a refresh that quietly behaves like an ordinary push.
RAW_REFRESH: ${{ github.event.inputs.refresh }}
RAW_FORCE: ${{ github.event.inputs.force_build }}
run: |
set -u
echo "trigger: event=$GITHUB_EVENT_NAME IS_REFRESH='${IS_REFRESH:-<unset>}' BUILD_REF='${BUILD_REF:-<unset>}'"
echo "trigger: raw inputs refresh='${RAW_REFRESH:-<unset>}' force_build='${RAW_FORCE:-<unset>}'"
A=ml
V=$(sh scripts/artifacts.sh version "$A" 2>&1 || echo UNAVAILABLE)
R=$(sh scripts/artifacts.sh revision "$A" 2>&1 || echo UNAVAILABLE)
echo "derived: artifact=$A version=$V revision=$R sha=$GITHUB_SHA"
- name: Determine tag
id: tag
run: |
# Mirrors build-web's tag list; see the comment there.
# POSIX-safe substring (the runner shell is dash/BusyBox sh, not
# bash — `${var:0:7}` errors with "Bad substitution"; cut works
# everywhere). Operator-flagged 2026-06-01 after first :c-<sha>
# main-push build failed at this step.
SHORT_SHA=$(printf '%s' "$GITHUB_SHA" | cut -c1-7)
# Mirrors build-web's tag list and its schedule handling; see
# the comments there.
if [ "${IS_REFRESH:-}" = "true" ]; then
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:latest" >> "$GITHUB_OUTPUT"
echo "channel=main" >> "$GITHUB_OUTPUT"
elif [ "${GITHUB_REF##*/}" = "main" ]; then
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:latest,git.fabledsword.com/bvandeusen/fabledcurator-ml:c-${SHORT_SHA}" >> "$GITHUB_OUTPUT"
echo "channel=main" >> "$GITHUB_OUTPUT"
else
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:dev" >> "$GITHUB_OUTPUT"
echo "channel=dev" >> "$GITHUB_OUTPUT"
fi
# Shell step rather than docker/login-action — see build-web's note on # Shell step rather than docker/login-action — see build-web's note on
# the shared action-cache race (#3118). # the shared action-cache race (#3118).
- name: Login to Forgejo registry - name: Login to Forgejo registry
@@ -1718,386 +1674,79 @@ jobs:
ACTOR: ${{ github.actor }} ACTOR: ${{ github.actor }}
run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin
# A REAL buildx builder, not the default `docker` driver (#3114, #3190). - name: Point fabledcurator-ml at the image fabledcurator published
#
# The default driver builds through the local dockerd. It cannot export a
# registry cache at all — which is why the agent rebuilds a ~6.3 GB CUDA
# + torch image from scratch whenever the runner's local cache is cold,
# measured at 9m26s against 7s warm. It is also #3190's leading suspect:
# after a registry-direct push it resolves image metadata against a local
# store the push never filled, and reports `No such image` on an image
# that published perfectly well three seconds earlier.
#
# These jobs run INSIDE a container against a mounted docker socket, so
# the buildkit container this starts is a SIBLING of the job container,
# not a child. That works over the socket mount; it had never been tried
# here before milestone 326 step 1.
- name: Set up buildx
uses: docker/setup-buildx-action@v3
# --- reuse-if-published (milestone 313, step 4) ----------------------
# Does the image the channel tag already points at carry THIS commit's
# revision? If so the bytes this job would produce are already published
# and the build is pure waste: the remaining tags get repointed at that
# existing manifest instead, registry-side, in seconds.
#
# Keyed on an `fc.revision` LABEL rather than on a tag of its own
# (milestone 318 step 3). A tag would be a name minted per build that one
# thing reads — what rule 145 narrowed against — and would be prunable
# under the registry's keep_pattern (#3157), silently expiring the cache.
# A label rides inside a tag that has to exist anyway.
#
# An image with no such label reads as a miss and rebuilds. That is the
# migration, not a fault: labels cannot be backfilled, since the reuse
# path copies a manifest and config labels are not manifest annotations.
# Each artifact pays one rebuild, once.
#
# This is what stops a push that touched only `agent/` from rebuilding
# web and ml, and a merge to main from rebuilding what dev already built.
#
# The failure direction is deliberate. An inspect that errors for ANY
# reason — network, auth, a registry hiccup — reads as a miss and the
# build runs. Only a genuine 200 skips one, so there is no path here
# that skips a build that was actually needed; the worst case is paying
# for a build we could have avoided.
#
# BASE-IMAGE FRESHNESS: an artifact whose source stops moving stops
# picking up base-image updates. Milestone 318 removed the argument this
# used to need rather than answering it — with no version tags there is
# no immutable name a refresh could contradict, and rule 145 already
# allows a rebuild with different contents to republish a MOVING tag.
# So a refresh is just a build. A scheduled channel-only one is tracked
# separately (#3154); it does not belong in the push path.
- name: Is this content already published?
id: reuse
env: env:
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-ml SRC: git.fabledsword.com/bvandeusen/fabledcurator
CHANNEL: ${{ steps.tag.outputs.channel }} DST: git.fabledsword.com/bvandeusen/fabledcurator-ml
# Empty on a push; the string "true" only from a workflow_dispatch BUILT_DIGEST: ${{ needs.build-web.outputs.digest }}
# that asked for it. `github.event.inputs` rather than the `inputs` PUBLISHED_DIGEST: ${{ needs.build-web.outputs.published_digest }}
# context — release.yml already uses that form, and it is the one
# this runner is known to evaluate. Read through env rather than
# interpolated into the run block, same rule as release.yml's TAG.
FORCE: ${{ github.event.inputs.force_build }}
# A scheduled refresh has to bypass reuse by construction: it
# rebuilds the SAME source, so fc.revision always matches and the
# check would skip every refresh there has ever been.
run: | run: |
set -eu set -eu
DERIVED=$(sh scripts/artifacts.sh revision ml) # WHERE THE BYTES COME FROM. build-web either pushed a manifest this
echo "revision=$DERIVED" >> "$GITHUB_OUTPUT" # run, or it hit reuse — in which case the digest its channel tag
# already names is the right source, because that is what "hit"
# MEANS: the reuse step read this commit's fc.revision off that tag.
DIGEST="${BUILT_DIGEST:-}"
[ -n "$DIGEST" ] || DIGEST="${PUBLISHED_DIGEST:-}"
if [ -z "$DIGEST" ]; then
echo "alias: FAILED — build-web neither built an image nor" >&2
echo "alias: resolved a published one, so there is nothing for" >&2
echo "alias: fabledcurator-ml to be pointed at." >&2
exit 1
fi
# The build clock, pinned to the same commit (#3265). Without it # WHICH TAGS, mirroring exactly what build-web wrote for itself.
# buildkit stamps the image config with the wall clock of the build,
# so identical layers republish under a new config blob and the
# channel tag gets a new manifest digest for no reason. Derived from
# `newest()` like revision and version, so all three name one commit
# and cannot drift apart.
echo "epoch=$(sh scripts/artifacts.sh epoch ml)" >> "$GITHUB_OUTPUT"
# The moving tag for this channel. Which tag we ask IS the channel —
# that is why the revision needs no -main/-dev qualifier any more.
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
echo "channel_ref=$IMAGE:$T" >> "$GITHUB_OUTPUT"
# WHERE THE BUILD PUBLISHES, which is not always the channel — and
# whether the channel then has to be written separately.
# #
# On a push the build writes the channel tag directly: the bytes came # A refresh writes the CANDIDATE tag and nothing else, so `promote`
# from a commit, and a commit is the thing CI tests. Nothing to hold # moves it to :latest after smoke-web passes. That keeps this name
# it behind. # under the same gate it has today and needs no change to promote,
# # which already loops over all three image names.
# On the scheduled refresh it writes a CANDIDATE tag instead. A SHORT_SHA=$(printf '%s' "$GITHUB_SHA" | cut -c1-7)
# refresh rebuilds against freshly resolved base images, and the web
# image's runtime is a line of UNPINNED Debian packages (ffmpeg,
# libjpeg62-turbo, libpq5, megatools…) re-resolved on every build.
# Nothing in ci.yml can see that: its lanes run on ci-python:3.14 and
# install requirements.txt, and a base bump changes neither. So
# refreshed bytes have to be proven before :latest names them, and
# proving needs a moment between "built" and "published" to occupy.
# This is that moment; :latest goes on naming the build that works
# until something says otherwise.
#
# `:refresh-candidate` is one moving ref per image, overwritten in
# place, holding a build nobody is told to pull — the shape rule 145
# already allows for :buildcache, not the per-build tag family that
# milestone 318 withdrew.
#
# Decided HERE, beside `hit`, for the reason the force/schedule
# branch below gives: one step decides what this job does. A
# condition derived independently could disagree with the tag the
# build actually wrote.
#
# build-web additionally exposes this as `outputs.candidate`, which is
# what gates the `promote` job — a job's `if:` cannot read `env`, and
# one flag is enough because all three derive it from the same
# IS_REFRESH. ml and agent do not re-emit it; a second copy nothing
# reads is the kind of thing that later reads as load-bearing.
if [ "${IS_REFRESH:-}" = "true" ]; then if [ "${IS_REFRESH:-}" = "true" ]; then
echo "build_ref=$IMAGE:refresh-candidate" >> "$GITHUB_OUTPUT" TAGS="refresh-candidate"
elif [ "${GITHUB_REF##*/}" = "main" ]; then
TAGS="latest c-${SHORT_SHA}"
else else
echo "build_ref=$IMAGE:$T" >> "$GITHUB_OUTPUT" TAGS="dev"
fi fi
# Compare VALUES, never exit codes. Measured on buildx v0.36.1 # Pull, tag, push — deliberately NOT `imagetools create`.
# (run 4732): a missing key returns an empty string and exits 0, so
# branching on the exit code would read "no label yet" as success.
# An unreachable tag also lands here as empty via the `|| echo`.
# Empty never equals a 12-char revision, so every uncertain case
# falls through to a build — the safe direction, with no special
# casing for it.
# #
# Read the SPECIFIC key. The map also carries whatever the base image # That wraps its source in an INDEX, and `.Image.Config.Labels` does
# set, and `org.opencontainers.image.version` sits right beside ours # not resolve through one (#3183, run 4751). Worse here than it was
# looking like a plausible answer (it reads 24.04 on the agent). # there: promote asks the registry for IMAGE manifest media types
PUBLISHED=$(docker buildx imagetools inspect "$IMAGE:$T" \ # ONLY, so an index sitting at :refresh-candidate would fail its
--format '{{ index .Image.Config.Labels "fc.revision" }}' \ # `test -n "$CT"` and break the weekly refresh for this name.
2>/dev/null || echo "")
echo "reuse: $IMAGE:$T carries fc.revision=${PUBLISHED:-<none>}; derived=$DERIVED"
if [ -z "$PUBLISHED" ] && docker buildx imagetools inspect "$IMAGE:$T" >/dev/null 2>&1; then
# The tag resolves but carries no readable label. Expected exactly
# once per artifact, during the migration onto labels. If it recurs
# every push, something is rewriting the channel tag as a manifest
# index — see the repoint step's note.
echo "reuse: NOTE $IMAGE:$T exists but has no readable fc.revision."
echo "reuse: NOTE Fine once, while migrating. Every push means the"
echo "reuse: NOTE tag is being index-wrapped and reuse is dead."
fi
# FORCE is checked here rather than in the build step's `if:`, so
# that one decision drives everything downstream. The repoint step
# keys off `hit` too, and a force that bypassed only the build would
# leave the two disagreeing about what just happened.
if [ "${FORCE:-false}" = "true" ]; then
echo "hit=false" >> "$GITHUB_OUTPUT"
echo "reuse: force_build set — building regardless"
elif [ "${IS_REFRESH:-}" = "true" ]; then
echo "hit=false" >> "$GITHUB_OUTPUT"
echo "reuse: scheduled base refresh — building regardless"
elif [ -n "$PUBLISHED" ] && [ "$PUBLISHED" = "$DERIVED" ]; then
echo "hit=true" >> "$GITHUB_OUTPUT"
echo "reuse: already published — skipping the build"
else
echo "hit=false" >> "$GITHUB_OUTPUT"
echo "reuse: not published — building"
fi
- name: Build and push ml image
# `id:` so the repoint step below can read `outputs.digest` — the
# manifest THIS run published, as opposed to whatever the channel tag
# happens to name by the time that step runs (#4290).
id: build
if: steps.reuse.outputs.hit != 'true'
# Read by buildx out of the ENVIRONMENT, not passed as a build-arg —
# it normalises the image config's `created` field and the history
# timestamps rather than being consumed by the Dockerfile. See #3265
# and the reuse step's `epoch` output.
env:
SOURCE_DATE_EPOCH: ${{ steps.reuse.outputs.epoch }}
uses: docker/build-push-action@v5
with:
context: .
# The merged image (milestone 422 step 6). `fabledcurator-ml` keeps
# publishing from it — same bytes under both names — because the
# operator's Swarm stack references fabledcurator-ml:latest and
# lives outside this repo. Dropping the name here would not break
# their deploy, it would freeze it silently at the last publish.
# Retiring the NAME is its own task, gated on that stack moving.
file: Dockerfile
push: true
# Re-resolve the FROM references against the registry instead of
# trusting whatever digest the cache was built against. This is the
# whole mechanism of the scheduled refresh (#3154): if the base tag
# moved, the FROM layer's cache key changes, every layer above it
# invalidates, and the image genuinely rebuilds.
# #
# MEASURED on the first real fire, run 4934 (#3265): when the base # The layers were just built on this runner pool, so the pull is
# did NOT move, the build was ~13s with every content step CACHED — # normally local. The push sends only blobs the destination lacks —
# and the channel tag STILL got a new manifest digest, because # whether the registry shares them across two repositories of one
# buildkit stamps a fresh image config per run and republishes the # owner is not something this file should assume, and the first run's
# identical layers under it. All three images moved that way on # output answers it. Either way it moves no more bytes than the
# 2026-08-30 with nothing whatsoever changed in them. # duplicate build's own push did, and it compiles nothing.
# docker pull "$SRC@$DIGEST"
# SOURCE_DATE_EPOCH (below) is the fix: pinned to the commit the
# content came from, the config is byte-identical across runs, so
# the manifest digest is too and the push is a registry no-op. A
# digest change means the content changed again, which is the only
# thing a digest is any use for.
#
# What `pull` does NOT catch either: a Debian package update inside
# the `apt-get install` layer while the base tag itself stands
# still. The official python/cuda images rebuild with those updates
# baked in, so this is a lag rather than a hole; closing it needs
# `no-cache: true`, which is a much larger version of the same
# churn #3265 is about.
#
# Only on the schedule. An ordinary push wants the cached base.
pull: ${{ env.IS_REFRESH == 'true' }}
# ONE tag, the channel's. Every other tag is written by the step
# below, registry-side. buildx here pushes the first tag to the
# registry and then re-pushes the rest through the DOCKER driver,
# out of a local image store a registry-direct build never filled —
# #3190, which cost `main` its :c-<sha> on 2026-08-29 while :latest
# published perfectly well.
tags: ${{ steps.reuse.outputs.build_ref }}
# The reuse key. Read back off the channel tag on the next push to
# decide whether that push needs to build at all, so this is not
# decoration — an unstamped image is one that will always rebuild.
labels: |
fc.revision=${{ steps.reuse.outputs.revision }}
# LOAD-BEARING, not a preference. On the default docker driver these
# were no-ops; on the docker-container driver above,
# build-push-action@v5 defaults provenance to TRUE when pushing.
# Provenance attaches an attestation manifest, which makes the pushed
# tag a manifest INDEX — and `.Image.Config.Labels` does not resolve
# through an index.
#
# The label directly above IS the reuse key. Wrap the channel tag in
# an index and the next push reads fc.revision=<none>, misses, and
# rebuilds. Then so does the one after that, forever. Nothing fails,
# nothing goes red, and the only symptom is the bill. That is #3183
# arriving through a different door, and note #3127 §4 records the
# same shape for `platforms:`.
provenance: false
sbom: false
# The ONLY cache this driver can have. `docker-container` gets a
# FRESH buildkit instance per job, so unlike the default docker
# driver it has no local layer store to fall back on — measured on
# run 4896, the first builds after the driver change: web 3m44s
# (was 2m23s), ml 3m49s (was 3m20s), agent 11m12s (was 9m26s). The
# driver change ALONE is a regression; this is the other half of it.
#
# mode=max so intermediate stages cache too. web's frontend-builder
# stage and the agent's two ~150s pip layers are the whole cost, and
# they are exactly what a min-mode cache would drop.
#
# A `:buildcache` tag is NOT the withdrawn tag scheme coming back.
# Rule 145 narrowed against names NOTHING reads; this one is read by
# every build that runs, is one moving ref per image rather than one
# per build, holds cache blobs rather than a shippable artifact, and
# is overwritten in place rather than accumulating. It is closer to
# :dev than to the :2026.8.28 tags milestone 318 deleted. (#3114.)
cache-from: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-ml:buildcache
cache-to: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-ml:buildcache,mode=max
# Every tag but the channel's own is written HERE, registry-side,
# whether or not a build ran. Each -t becomes another reference to the
# SAME manifest the channel tag holds, so :c-<sha> is byte-identical to
# what is published rather than a lookalike rebuild.
#
# Owning the build path too is #3190's fix, not a tidy-up:
#
# #27 pushing …/fabledcurator:latest DONE 15.8s
# #28 pushing …/fabledcurator:c-0e15c44 with docker
# #28 ERROR: tag does not exist: …:c-0e15c44
#
# Intermittent — build-ml made the identical two-tag push seconds later
# and succeeded — and worse than it looks. `:latest` had already
# published, so production was correct while the immutable rollback tag
# rule 145 requires of every main push simply did not exist. Nothing but
# the red job would ever have noticed: a missing :c-<sha> has no
# consumer that fails, so it surfaces when somebody needs to roll back.
#
# `imagetools create` is a registry-side manifest copy — no layer
# transfer, no local daemon, nothing that can be absent. The reuse case
# has always gone this way, so this puts the build case on the code that
# was already proven rather than on a second path.
#
# Running on every path also keeps family rule 146 true: a rolling
# channel refreshes itself, so skipping a build must never leave :dev or
# :latest pointing at something older than the commit just pushed.
#
# The cost, accepted knowingly: `imagetools create` wraps its source in
# an index, so :c-<sha> becomes an index and fc.revision does not
# resolve through it. Nothing reads that label off :c-<sha> — the reuse
# check only ever inspects the CHANNEL tag — and the index names the
# same manifest, so a pull is byte-identical. The reuse path already
# produced :c-<sha> this way; this only makes it uniform.
- name: Write the remaining tags from the published image
env:
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-ml
CHANNEL_REF: ${{ steps.reuse.outputs.channel_ref }}
# Empty when no build ran this job (a reuse hit, or the step's `if:`
# skipped it). Non-empty means THIS run pushed that manifest.
BUILT_DIGEST: ${{ steps.build.outputs.digest }}
TAGS: ${{ steps.tag.outputs.tags }}
run: |
set -euf
# WHAT WE COPY FROM, which is not what we EXCLUDE (#4290).
#
# This step used to copy from the channel tag by NAME. Nothing
# serialises builds — there is no `concurrency:` key anywhere in
# .forgejo/workflows/ — so two pushes to one branch run in full
# parallel, both miss the reuse check, and both build. If the OLDER
# one finishes last it wins the channel tag; and then its repoint
# step, reading that tag by name, wrote :c-<sha> from whatever the
# other run had just published. An immutable rollback tag (rule 145)
# naming a different commit's bytes, wrong from birth — and
# immutability then guarantees nobody ever corrects it. Nothing goes
# red; it surfaces the day someone needs to roll back.
#
# So when this job built, copy from the DIGEST it pushed. Correct
# whatever a concurrent run does to the tag, and it does not depend
# on the runner honouring a `concurrency:` key — which this file has
# already been burned by once (the `format()` note at the top: an
# expression that evaluated false with no symptom at all).
#
# On a reuse hit there is no digest, and the channel tag is still the
# right source: "hit" MEANS that tag already carries this commit's
# fc.revision, which the reuse step verified by reading it.
if [ -n "${BUILT_DIGEST:-}" ]; then
SOURCE="$IMAGE@$BUILT_DIGEST"
echo "repoint: copying the digest this run published: $SOURCE"
else
SOURCE="$CHANNEL_REF"
echo "repoint: no build this run (reuse hit) — copying from $SOURCE"
fi
# The source tag is EXCLUDED from the targets, and that is load-
# bearing rather than an optimisation.
#
# `imagetools create` wraps the source manifest in an INDEX. Point it
# at the channel tag with that same tag as a target and the tag stops
# being a plain image — after which `.Image.Config.Labels` no longer
# resolves through it and the fc.revision label reads as absent. The
# next push then misses and rebuilds, so reuse worked exactly once
# and every subsequent push paid full price. Observed on run 4751:
# ml:dev reported fc.revision=<none> one push after run 4749 had read
# a7e626a67a79 off it. Nothing failed; the savings just evaporated.
#
# Excluding the source means the channel tag is only ever written
# by a real build, so it stays a plain image and stays readable.
# On dev that leaves nothing to do either way: the build pushed :dev
# itself, or the hit established it was already right. On main it
# leaves :c-<sha>, which rule 145 requires of every main push whether
# or not a build ran.
#
# steps.tag emits ONE comma-separated list; imagetools wants a -t per
# ref. (That list used to feed docker/build-push-action directly —
# which is exactly what #3190 made unsafe.)
ARGS=""
IFS=,
for t in $TAGS; do for t in $TAGS; do
# Keyed on CHANNEL_REF, never on SOURCE. SOURCE may now be a digest echo "alias: $DST:$t -> $SRC@$DIGEST"
# ref, which never equals a tag string — testing against it would docker tag "$SRC@$DIGEST" "$DST:$t"
# stop excluding the channel tag, imagetools would index-wrap it, docker push "$DST:$t"
# and `.Image.Config.Labels` would stop resolving through it. That done
# kills the reuse label permanently (see the note just below).
[ "$t" = "$CHANNEL_REF" ] && continue # Read it back. A push that reported success but left the tag
ARGS="$ARGS -t $t" # elsewhere is exactly the silent-and-plausible failure this
# pipeline keeps producing, and the check costs one request.
for t in $TAGS; do
NOW=$(docker buildx imagetools inspect "$DST:$t" \
--format '{{ index .Image.Config.Labels "fc.revision" }}' 2>/dev/null || echo "")
echo "alias: $DST:$t now carries fc.revision=${NOW:-<none>}"
if [ -z "$NOW" ]; then
echo "alias: FAILED — $DST:$t has no readable fc.revision." >&2
echo "alias: The label resolves through a plain image manifest" >&2
echo "alias: and not through an index, so this means the tag is" >&2
echo "alias: index-wrapped — which breaks promote's refresh path." >&2
exit 1
fi
done done
unset IFS
if [ -z "$ARGS" ]; then
echo "repoint: $CHANNEL_REF is the only tag for this channel and"
echo "repoint: already holds this revision — nothing to write."
exit 0
fi
# shellcheck disable=SC2086
docker buildx imagetools create $ARGS "$SOURCE"
echo "repointed from $SOURCE:$ARGS"
# The desktop GPU agent (#114) — published so the operator pulls + runs it on
# the GPU machine instead of building locally. Independent of web/ml (its own
# CUDA + onnxruntime-gpu image, context = agent/). Same tag cadence.
build-agent: build-agent:
runs-on: python-ci runs-on: python-ci
container: container: