ci: key the reuse check on an image label, not a tag (318 step 3)
CI / extension-version (push) Successful in 4s
CI / lint (push) Failing after 4s
Build images / sign-extension (push) Successful in 4s
CI / backend-lint-and-test (push) Failing after 13s
CI / frontend-build (push) Successful in 20s
extension / lint (push) Successful in 22s
CI / integration (push) Failing after 2m24s
Build images / build-web (push) Successful in 2m44s
Build images / build-ml (push) Successful in 3m13s
Build images / build-agent (push) Successful in 8m56s
CI / extension-version (push) Successful in 4s
CI / lint (push) Failing after 4s
Build images / sign-extension (push) Successful in 4s
CI / backend-lint-and-test (push) Failing after 13s
CI / frontend-build (push) Successful in 20s
extension / lint (push) Successful in 22s
CI / integration (push) Failing after 2m24s
Build images / build-web (push) Successful in 2m44s
Build images / build-ml (push) Successful in 3m13s
Build images / build-agent (push) Successful in 8m56s
The shadow (dee93fa, run 4732) answered the gate: `imagetools inspect
--format` reads `.Image.Config.Labels` against this registry on buildx
v0.36.1. So the reuse check now asks the moving channel tag whether the image
it already points at carries this commit's `fc.revision`, and the r-<rev>
identity tags stop being published.
Three things this removes rather than manages:
A name minted per build that one thing read. Rule 145's narrowing is aimed
exactly there — "a third name for the same thing is upkeep for a model we do
not run."
The -main/-dev qualifier, and the CHANNELLED list behind it. Which tag you
inspect IS the channel, so the distinction has nowhere to live. cmd_identity
goes with it.
A silent expiry nobody wrote down. r-<rev> matches no branch of the
registry's keep_pattern (#3157), so identity tags were prunable past the
newest 10 — a pruned one costs a rebuild, in the safe direction and entirely
invisibly. A label rides inside a tag that has to exist anyway.
It also dissolves #3154 instead of deferring it: a scheduled base refresh
rebuilds :latest with the same revision label, the next unrelated push sees a
match and skips, and the refreshed base survives. Under the tag scheme that
push repointed :latest back to the older base.
The measured detail that shapes the code: a missing label returns an EMPTY
STRING and exits 0. Branching on the exit code would read "no label yet" as
success and skip a build that was needed. So it compares values, and every
uncertain case — absent label, unreachable tag, older image — lands as empty,
never equals a 12-char revision, and falls through to a build.
Reading the specific key matters too. The map carries the base image's labels,
and org.opencontainers.image.version sits right beside ours reading 24.04 on
the agent — a plausible-looking wrong answer.
Expect every artifact to rebuild once on this push: nothing carries a label
yet and it cannot be backfilled, since the reuse path copies a manifest and
config labels are not manifest annotations. One rebuild per artifact, ever,
self-healing after.
test_artifact_identity.py is rewritten around what is now load-bearing. The
CHANNELLED drift test had nothing left to guard; in its place the revision is
asked of git directly, so the file fails if the derivation ever stops being
"the commit this artifact's own shipped files last changed in".
This commit is contained in:
+159
-267
@@ -452,12 +452,21 @@ jobs:
|
||||
run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin
|
||||
|
||||
# --- reuse-if-published (milestone 313, step 4) ----------------------
|
||||
# The identity tag names this artifact's CONTENT — r-<revision>, the
|
||||
# commit its shipped files last changed in, plus the channel for images
|
||||
# that bake one in. If the registry already carries it, the bytes this
|
||||
# job would produce are already published and the build is pure waste:
|
||||
# the channel and date tags get repointed at the existing manifest
|
||||
# instead, registry-side, in seconds.
|
||||
# Does the image the channel tag already points at carry THIS commit's
|
||||
# revision? If so the bytes this job would produce are already published
|
||||
# and the build is pure waste: the remaining tags get repointed at that
|
||||
# existing manifest instead, registry-side, in seconds.
|
||||
#
|
||||
# Keyed on an `fc.revision` LABEL rather than on a tag of its own
|
||||
# (milestone 318 step 3). A tag would be a name minted per build that one
|
||||
# thing reads — what rule 145 narrowed against — and would be prunable
|
||||
# under the registry's keep_pattern (#3157), silently expiring the cache.
|
||||
# A label rides inside a tag that has to exist anyway.
|
||||
#
|
||||
# An image with no such label reads as a miss and rebuilds. That is the
|
||||
# migration, not a fault: labels cannot be backfilled, since the reuse
|
||||
# path copies a manifest and config labels are not manifest annotations.
|
||||
# Each artifact pays one rebuild, once.
|
||||
#
|
||||
# This is what stops a push that touched only `agent/` from rebuilding
|
||||
# web and ml, and a merge to main from rebuilding what dev already built.
|
||||
@@ -485,88 +494,39 @@ jobs:
|
||||
TAGS: ${{ steps.tag.outputs.tags }}
|
||||
run: |
|
||||
set -eu
|
||||
ID=$(sh scripts/artifacts.sh identity web "$CHANNEL")
|
||||
echo "identity=$ID" >> "$GITHUB_OUTPUT"
|
||||
# The value step 3 will key on once the label mechanism is proven.
|
||||
# Emitted now so the build below can stamp it — stamping is harmless
|
||||
# and is what gives the shadow read something to find next push.
|
||||
REV=$(sh scripts/artifacts.sh revision web)
|
||||
echo "revision=$REV" >> "$GITHUB_OUTPUT"
|
||||
DERIVED=$(sh scripts/artifacts.sh revision web)
|
||||
echo "revision=$DERIVED" >> "$GITHUB_OUTPUT"
|
||||
echo "build_tags=$TAGS" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Every build that runs now claims the identity. The carve-out here
|
||||
# existed only for tag pushes, which rebuilt an already-published
|
||||
# revision and so had to be stopped from re-pointing an immutable
|
||||
# tag at fresh bytes. No tag trigger, nothing to carve out.
|
||||
echo "build_tags=$TAGS,$IMAGE:$ID" >> "$GITHUB_OUTPUT"
|
||||
# The moving tag for this channel. Which tag we ask IS the channel —
|
||||
# that is why the revision needs no -main/-dev qualifier any more.
|
||||
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
|
||||
echo "channel_ref=$IMAGE:$T" >> "$GITHUB_OUTPUT"
|
||||
|
||||
if docker buildx imagetools inspect "$IMAGE:$ID" >/dev/null 2>&1; then
|
||||
# Compare VALUES, never exit codes. Measured on buildx v0.36.1
|
||||
# (run 4732): a missing key returns an empty string and exits 0, so
|
||||
# branching on the exit code would read "no label yet" as success.
|
||||
# An unreachable tag also lands here as empty via the `|| echo`.
|
||||
# Empty never equals a 12-char revision, so every uncertain case
|
||||
# falls through to a build — the safe direction, with no special
|
||||
# casing for it.
|
||||
#
|
||||
# Read the SPECIFIC key. The map also carries whatever the base image
|
||||
# set, and `org.opencontainers.image.version` sits right beside ours
|
||||
# looking like a plausible answer (it reads 24.04 on the agent).
|
||||
PUBLISHED=$(docker buildx imagetools inspect "$IMAGE:$T" \
|
||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' \
|
||||
2>/dev/null || echo "")
|
||||
echo "reuse: $IMAGE:$T carries fc.revision=${PUBLISHED:-<none>}; derived=$DERIVED"
|
||||
|
||||
if [ -n "$PUBLISHED" ] && [ "$PUBLISHED" = "$DERIVED" ]; then
|
||||
echo "hit=true" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse: $IMAGE:$ID is already published — skipping the build"
|
||||
echo "reuse: already published — skipping the build"
|
||||
else
|
||||
echo "hit=false" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse: $IMAGE:$ID is not published — building"
|
||||
echo "reuse: not published — building"
|
||||
fi
|
||||
|
||||
# --- shadow: can we read a label off the channel tag? (milestone 318 step 3)
|
||||
# Informational ONLY — no `set -e`, every probe falls back to
|
||||
# UNAVAILABLE, and nothing downstream reads this. It must never fail a
|
||||
# build; that is the whole reason it runs before the mechanism does.
|
||||
#
|
||||
# The question: can the reuse check be keyed on an image LABEL read off
|
||||
# the channel tag, instead of on a separate r-<rev> tag? If yes, nothing
|
||||
# is minted per build, the -main/-dev identity split disappears, and the
|
||||
# base-refresh clobber (#3154) stops existing.
|
||||
#
|
||||
# Two questions, answered at different times — worth separating, because
|
||||
# reading "no value" as "the mechanism does not work" would abandon a
|
||||
# working design:
|
||||
#
|
||||
# 1. Do the TEMPLATE MECHANICS work here? Answered on the very first
|
||||
# run, by probes B and C: if they print a labels map (even an empty
|
||||
# one) or a config JSON, `--format` over .Image resolves on this
|
||||
# buildx and this registry. That is the actual gate for step 3.
|
||||
# 2. Does a stamped value ROUND-TRIP? Only answerable once an artifact
|
||||
# genuinely rebuilds. A push that changes no shipped file reuses,
|
||||
# the build is skipped, and nothing gets stamped — so an artifact
|
||||
# keeps reporting no label until its own source next moves.
|
||||
#
|
||||
# That second point is not a defect, it is the migration path: under the
|
||||
# real mechanism "no label" reads as a MISS, so the artifact rebuilds
|
||||
# once and is stamped from then on. One extra build per artifact, ever,
|
||||
# in the safe direction. `imagetools create` cannot add the label on the
|
||||
# reuse path — it copies a manifest and config labels are not manifest
|
||||
# annotations — so there is no way to backfill, and no need to.
|
||||
#
|
||||
# Several probes on purpose: `--format` templating over .Image varies by
|
||||
# buildx version, and one push should tell us which spelling works
|
||||
# rather than costing a round trip per guess.
|
||||
- name: Shadow — read fc.revision off the channel tag (informational)
|
||||
env:
|
||||
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator
|
||||
CHANNEL: ${{ steps.tag.outputs.channel }}
|
||||
DERIVED: ${{ steps.reuse.outputs.revision }}
|
||||
run: |
|
||||
set -u
|
||||
# The moving tag this channel publishes. `main` is asked about
|
||||
# :latest, which is what step 4 keeps; :main is on its way out.
|
||||
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
|
||||
REF="$IMAGE:$T"
|
||||
echo "label-shadow: ref=$REF derived=$DERIVED"
|
||||
|
||||
A=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' 2>&1 || echo FAILED)
|
||||
echo "label-shadow: probe-index = $A"
|
||||
|
||||
B=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ .Image.Config.Labels }}' 2>&1 | head -c 300 || echo FAILED)
|
||||
echo "label-shadow: probe-labels = $B"
|
||||
|
||||
C=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ json .Image }}' 2>&1 | head -c 400 || echo FAILED)
|
||||
echo "label-shadow: probe-json = $C"
|
||||
|
||||
echo "label-shadow: buildx = $(docker buildx version 2>&1 || echo UNAVAILABLE)"
|
||||
|
||||
- name: Download signed XPI from Forgejo release asset
|
||||
# dev and main each bundle the XPI their own sign-extension just
|
||||
# published — the point of the channel work (milestone 271 step 6): the
|
||||
@@ -651,9 +611,9 @@ jobs:
|
||||
file: Dockerfile
|
||||
push: true
|
||||
tags: ${{ steps.reuse.outputs.build_tags }}
|
||||
# Stamped now, read later. Additive and inert this step: no
|
||||
# decision depends on it until the shadow above proves the
|
||||
# label can actually be read back (milestone 318 step 3).
|
||||
# The reuse key. Read back off the channel tag on the next push to
|
||||
# decide whether that push needs to build at all, so this is not
|
||||
# decoration — an unstamped image is one that will always rebuild.
|
||||
labels: |
|
||||
fc.revision=${{ steps.reuse.outputs.revision }}
|
||||
# Only the web image carries a channel: it is the one that serves
|
||||
@@ -664,8 +624,8 @@ jobs:
|
||||
|
||||
# Registry-side manifest copy: no layer transfer, no local daemon, no
|
||||
# rebuild. Each -t becomes another reference to the SAME manifest the
|
||||
# identity tag holds, so :latest and the date pin are byte-identical to
|
||||
# what was published rather than a lookalike rebuild.
|
||||
# channel tag already holds, so :c-<sha> and the date pin are
|
||||
# byte-identical to what is published rather than a lookalike rebuild.
|
||||
#
|
||||
# Runs on EVERY reuse, which is what keeps family rule 146 true: a
|
||||
# rolling channel refreshes itself, so skipping a build must never mean
|
||||
@@ -675,7 +635,7 @@ jobs:
|
||||
if: steps.reuse.outputs.hit == 'true'
|
||||
env:
|
||||
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator
|
||||
IDENTITY: ${{ steps.reuse.outputs.identity }}
|
||||
SOURCE: ${{ steps.reuse.outputs.channel_ref }}
|
||||
TAGS: ${{ steps.tag.outputs.tags }}
|
||||
run: |
|
||||
set -euf
|
||||
@@ -685,9 +645,13 @@ jobs:
|
||||
IFS=,
|
||||
for t in $TAGS; do ARGS="$ARGS -t $t"; done
|
||||
unset IFS
|
||||
# Source is the channel tag itself — the image we just confirmed
|
||||
# carries this revision. One of the targets is that same tag, which
|
||||
# makes its copy a no-op; the others (:main, :c-<sha>, the date pin)
|
||||
# are what this exists for.
|
||||
# shellcheck disable=SC2086
|
||||
docker buildx imagetools create $ARGS "$IMAGE:$IDENTITY"
|
||||
echo "repointed to $IMAGE:$IDENTITY: $TAGS"
|
||||
docker buildx imagetools create $ARGS "$SOURCE"
|
||||
echo "repointed to $SOURCE: $TAGS"
|
||||
|
||||
build-ml:
|
||||
runs-on: python-ci
|
||||
@@ -780,12 +744,21 @@ jobs:
|
||||
run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin
|
||||
|
||||
# --- reuse-if-published (milestone 313, step 4) ----------------------
|
||||
# The identity tag names this artifact's CONTENT — r-<revision>, the
|
||||
# commit its shipped files last changed in, plus the channel for images
|
||||
# that bake one in. If the registry already carries it, the bytes this
|
||||
# job would produce are already published and the build is pure waste:
|
||||
# the channel and date tags get repointed at the existing manifest
|
||||
# instead, registry-side, in seconds.
|
||||
# Does the image the channel tag already points at carry THIS commit's
|
||||
# revision? If so the bytes this job would produce are already published
|
||||
# and the build is pure waste: the remaining tags get repointed at that
|
||||
# existing manifest instead, registry-side, in seconds.
|
||||
#
|
||||
# Keyed on an `fc.revision` LABEL rather than on a tag of its own
|
||||
# (milestone 318 step 3). A tag would be a name minted per build that one
|
||||
# thing reads — what rule 145 narrowed against — and would be prunable
|
||||
# under the registry's keep_pattern (#3157), silently expiring the cache.
|
||||
# A label rides inside a tag that has to exist anyway.
|
||||
#
|
||||
# An image with no such label reads as a miss and rebuilds. That is the
|
||||
# migration, not a fault: labels cannot be backfilled, since the reuse
|
||||
# path copies a manifest and config labels are not manifest annotations.
|
||||
# Each artifact pays one rebuild, once.
|
||||
#
|
||||
# This is what stops a push that touched only `agent/` from rebuilding
|
||||
# web and ml, and a merge to main from rebuilding what dev already built.
|
||||
@@ -813,88 +786,39 @@ jobs:
|
||||
TAGS: ${{ steps.tag.outputs.tags }}
|
||||
run: |
|
||||
set -eu
|
||||
ID=$(sh scripts/artifacts.sh identity ml "$CHANNEL")
|
||||
echo "identity=$ID" >> "$GITHUB_OUTPUT"
|
||||
# The value step 3 will key on once the label mechanism is proven.
|
||||
# Emitted now so the build below can stamp it — stamping is harmless
|
||||
# and is what gives the shadow read something to find next push.
|
||||
REV=$(sh scripts/artifacts.sh revision ml)
|
||||
echo "revision=$REV" >> "$GITHUB_OUTPUT"
|
||||
DERIVED=$(sh scripts/artifacts.sh revision ml)
|
||||
echo "revision=$DERIVED" >> "$GITHUB_OUTPUT"
|
||||
echo "build_tags=$TAGS" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Every build that runs now claims the identity. The carve-out here
|
||||
# existed only for tag pushes, which rebuilt an already-published
|
||||
# revision and so had to be stopped from re-pointing an immutable
|
||||
# tag at fresh bytes. No tag trigger, nothing to carve out.
|
||||
echo "build_tags=$TAGS,$IMAGE:$ID" >> "$GITHUB_OUTPUT"
|
||||
# The moving tag for this channel. Which tag we ask IS the channel —
|
||||
# that is why the revision needs no -main/-dev qualifier any more.
|
||||
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
|
||||
echo "channel_ref=$IMAGE:$T" >> "$GITHUB_OUTPUT"
|
||||
|
||||
if docker buildx imagetools inspect "$IMAGE:$ID" >/dev/null 2>&1; then
|
||||
# Compare VALUES, never exit codes. Measured on buildx v0.36.1
|
||||
# (run 4732): a missing key returns an empty string and exits 0, so
|
||||
# branching on the exit code would read "no label yet" as success.
|
||||
# An unreachable tag also lands here as empty via the `|| echo`.
|
||||
# Empty never equals a 12-char revision, so every uncertain case
|
||||
# falls through to a build — the safe direction, with no special
|
||||
# casing for it.
|
||||
#
|
||||
# Read the SPECIFIC key. The map also carries whatever the base image
|
||||
# set, and `org.opencontainers.image.version` sits right beside ours
|
||||
# looking like a plausible answer (it reads 24.04 on the agent).
|
||||
PUBLISHED=$(docker buildx imagetools inspect "$IMAGE:$T" \
|
||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' \
|
||||
2>/dev/null || echo "")
|
||||
echo "reuse: $IMAGE:$T carries fc.revision=${PUBLISHED:-<none>}; derived=$DERIVED"
|
||||
|
||||
if [ -n "$PUBLISHED" ] && [ "$PUBLISHED" = "$DERIVED" ]; then
|
||||
echo "hit=true" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse: $IMAGE:$ID is already published — skipping the build"
|
||||
echo "reuse: already published — skipping the build"
|
||||
else
|
||||
echo "hit=false" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse: $IMAGE:$ID is not published — building"
|
||||
echo "reuse: not published — building"
|
||||
fi
|
||||
|
||||
# --- shadow: can we read a label off the channel tag? (milestone 318 step 3)
|
||||
# Informational ONLY — no `set -e`, every probe falls back to
|
||||
# UNAVAILABLE, and nothing downstream reads this. It must never fail a
|
||||
# build; that is the whole reason it runs before the mechanism does.
|
||||
#
|
||||
# The question: can the reuse check be keyed on an image LABEL read off
|
||||
# the channel tag, instead of on a separate r-<rev> tag? If yes, nothing
|
||||
# is minted per build, the -main/-dev identity split disappears, and the
|
||||
# base-refresh clobber (#3154) stops existing.
|
||||
#
|
||||
# Two questions, answered at different times — worth separating, because
|
||||
# reading "no value" as "the mechanism does not work" would abandon a
|
||||
# working design:
|
||||
#
|
||||
# 1. Do the TEMPLATE MECHANICS work here? Answered on the very first
|
||||
# run, by probes B and C: if they print a labels map (even an empty
|
||||
# one) or a config JSON, `--format` over .Image resolves on this
|
||||
# buildx and this registry. That is the actual gate for step 3.
|
||||
# 2. Does a stamped value ROUND-TRIP? Only answerable once an artifact
|
||||
# genuinely rebuilds. A push that changes no shipped file reuses,
|
||||
# the build is skipped, and nothing gets stamped — so an artifact
|
||||
# keeps reporting no label until its own source next moves.
|
||||
#
|
||||
# That second point is not a defect, it is the migration path: under the
|
||||
# real mechanism "no label" reads as a MISS, so the artifact rebuilds
|
||||
# once and is stamped from then on. One extra build per artifact, ever,
|
||||
# in the safe direction. `imagetools create` cannot add the label on the
|
||||
# reuse path — it copies a manifest and config labels are not manifest
|
||||
# annotations — so there is no way to backfill, and no need to.
|
||||
#
|
||||
# Several probes on purpose: `--format` templating over .Image varies by
|
||||
# buildx version, and one push should tell us which spelling works
|
||||
# rather than costing a round trip per guess.
|
||||
- name: Shadow — read fc.revision off the channel tag (informational)
|
||||
env:
|
||||
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-ml
|
||||
CHANNEL: ${{ steps.tag.outputs.channel }}
|
||||
DERIVED: ${{ steps.reuse.outputs.revision }}
|
||||
run: |
|
||||
set -u
|
||||
# The moving tag this channel publishes. `main` is asked about
|
||||
# :latest, which is what step 4 keeps; :main is on its way out.
|
||||
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
|
||||
REF="$IMAGE:$T"
|
||||
echo "label-shadow: ref=$REF derived=$DERIVED"
|
||||
|
||||
A=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' 2>&1 || echo FAILED)
|
||||
echo "label-shadow: probe-index = $A"
|
||||
|
||||
B=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ .Image.Config.Labels }}' 2>&1 | head -c 300 || echo FAILED)
|
||||
echo "label-shadow: probe-labels = $B"
|
||||
|
||||
C=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ json .Image }}' 2>&1 | head -c 400 || echo FAILED)
|
||||
echo "label-shadow: probe-json = $C"
|
||||
|
||||
echo "label-shadow: buildx = $(docker buildx version 2>&1 || echo UNAVAILABLE)"
|
||||
|
||||
- name: Build and push ml image
|
||||
if: steps.reuse.outputs.hit != 'true'
|
||||
uses: docker/build-push-action@v5
|
||||
@@ -903,16 +827,16 @@ jobs:
|
||||
file: Dockerfile.ml
|
||||
push: true
|
||||
tags: ${{ steps.reuse.outputs.build_tags }}
|
||||
# Stamped now, read later. Additive and inert this step: no
|
||||
# decision depends on it until the shadow above proves the
|
||||
# label can actually be read back (milestone 318 step 3).
|
||||
# The reuse key. Read back off the channel tag on the next push to
|
||||
# decide whether that push needs to build at all, so this is not
|
||||
# decoration — an unstamped image is one that will always rebuild.
|
||||
labels: |
|
||||
fc.revision=${{ steps.reuse.outputs.revision }}
|
||||
|
||||
# Registry-side manifest copy: no layer transfer, no local daemon, no
|
||||
# rebuild. Each -t becomes another reference to the SAME manifest the
|
||||
# identity tag holds, so :latest and the date pin are byte-identical to
|
||||
# what was published rather than a lookalike rebuild.
|
||||
# channel tag already holds, so :c-<sha> and the date pin are
|
||||
# byte-identical to what is published rather than a lookalike rebuild.
|
||||
#
|
||||
# Runs on EVERY reuse, which is what keeps family rule 146 true: a
|
||||
# rolling channel refreshes itself, so skipping a build must never mean
|
||||
@@ -922,7 +846,7 @@ jobs:
|
||||
if: steps.reuse.outputs.hit == 'true'
|
||||
env:
|
||||
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-ml
|
||||
IDENTITY: ${{ steps.reuse.outputs.identity }}
|
||||
SOURCE: ${{ steps.reuse.outputs.channel_ref }}
|
||||
TAGS: ${{ steps.tag.outputs.tags }}
|
||||
run: |
|
||||
set -euf
|
||||
@@ -932,9 +856,13 @@ jobs:
|
||||
IFS=,
|
||||
for t in $TAGS; do ARGS="$ARGS -t $t"; done
|
||||
unset IFS
|
||||
# Source is the channel tag itself — the image we just confirmed
|
||||
# carries this revision. One of the targets is that same tag, which
|
||||
# makes its copy a no-op; the others (:main, :c-<sha>, the date pin)
|
||||
# are what this exists for.
|
||||
# shellcheck disable=SC2086
|
||||
docker buildx imagetools create $ARGS "$IMAGE:$IDENTITY"
|
||||
echo "repointed to $IMAGE:$IDENTITY: $TAGS"
|
||||
docker buildx imagetools create $ARGS "$SOURCE"
|
||||
echo "repointed to $SOURCE: $TAGS"
|
||||
|
||||
# The desktop GPU agent (#114) — published so the operator pulls + runs it on
|
||||
# the GPU machine instead of building locally. Independent of web/ml (its own
|
||||
@@ -1023,12 +951,21 @@ jobs:
|
||||
run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin
|
||||
|
||||
# --- reuse-if-published (milestone 313, step 4) ----------------------
|
||||
# The identity tag names this artifact's CONTENT — r-<revision>, the
|
||||
# commit its shipped files last changed in, plus the channel for images
|
||||
# that bake one in. If the registry already carries it, the bytes this
|
||||
# job would produce are already published and the build is pure waste:
|
||||
# the channel and date tags get repointed at the existing manifest
|
||||
# instead, registry-side, in seconds.
|
||||
# Does the image the channel tag already points at carry THIS commit's
|
||||
# revision? If so the bytes this job would produce are already published
|
||||
# and the build is pure waste: the remaining tags get repointed at that
|
||||
# existing manifest instead, registry-side, in seconds.
|
||||
#
|
||||
# Keyed on an `fc.revision` LABEL rather than on a tag of its own
|
||||
# (milestone 318 step 3). A tag would be a name minted per build that one
|
||||
# thing reads — what rule 145 narrowed against — and would be prunable
|
||||
# under the registry's keep_pattern (#3157), silently expiring the cache.
|
||||
# A label rides inside a tag that has to exist anyway.
|
||||
#
|
||||
# An image with no such label reads as a miss and rebuilds. That is the
|
||||
# migration, not a fault: labels cannot be backfilled, since the reuse
|
||||
# path copies a manifest and config labels are not manifest annotations.
|
||||
# Each artifact pays one rebuild, once.
|
||||
#
|
||||
# This is what stops a push that touched only `agent/` from rebuilding
|
||||
# web and ml, and a merge to main from rebuilding what dev already built.
|
||||
@@ -1056,88 +993,39 @@ jobs:
|
||||
TAGS: ${{ steps.tag.outputs.tags }}
|
||||
run: |
|
||||
set -eu
|
||||
ID=$(sh scripts/artifacts.sh identity agent "$CHANNEL")
|
||||
echo "identity=$ID" >> "$GITHUB_OUTPUT"
|
||||
# The value step 3 will key on once the label mechanism is proven.
|
||||
# Emitted now so the build below can stamp it — stamping is harmless
|
||||
# and is what gives the shadow read something to find next push.
|
||||
REV=$(sh scripts/artifacts.sh revision agent)
|
||||
echo "revision=$REV" >> "$GITHUB_OUTPUT"
|
||||
DERIVED=$(sh scripts/artifacts.sh revision agent)
|
||||
echo "revision=$DERIVED" >> "$GITHUB_OUTPUT"
|
||||
echo "build_tags=$TAGS" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Every build that runs now claims the identity. The carve-out here
|
||||
# existed only for tag pushes, which rebuilt an already-published
|
||||
# revision and so had to be stopped from re-pointing an immutable
|
||||
# tag at fresh bytes. No tag trigger, nothing to carve out.
|
||||
echo "build_tags=$TAGS,$IMAGE:$ID" >> "$GITHUB_OUTPUT"
|
||||
# The moving tag for this channel. Which tag we ask IS the channel —
|
||||
# that is why the revision needs no -main/-dev qualifier any more.
|
||||
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
|
||||
echo "channel_ref=$IMAGE:$T" >> "$GITHUB_OUTPUT"
|
||||
|
||||
if docker buildx imagetools inspect "$IMAGE:$ID" >/dev/null 2>&1; then
|
||||
# Compare VALUES, never exit codes. Measured on buildx v0.36.1
|
||||
# (run 4732): a missing key returns an empty string and exits 0, so
|
||||
# branching on the exit code would read "no label yet" as success.
|
||||
# An unreachable tag also lands here as empty via the `|| echo`.
|
||||
# Empty never equals a 12-char revision, so every uncertain case
|
||||
# falls through to a build — the safe direction, with no special
|
||||
# casing for it.
|
||||
#
|
||||
# Read the SPECIFIC key. The map also carries whatever the base image
|
||||
# set, and `org.opencontainers.image.version` sits right beside ours
|
||||
# looking like a plausible answer (it reads 24.04 on the agent).
|
||||
PUBLISHED=$(docker buildx imagetools inspect "$IMAGE:$T" \
|
||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' \
|
||||
2>/dev/null || echo "")
|
||||
echo "reuse: $IMAGE:$T carries fc.revision=${PUBLISHED:-<none>}; derived=$DERIVED"
|
||||
|
||||
if [ -n "$PUBLISHED" ] && [ "$PUBLISHED" = "$DERIVED" ]; then
|
||||
echo "hit=true" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse: $IMAGE:$ID is already published — skipping the build"
|
||||
echo "reuse: already published — skipping the build"
|
||||
else
|
||||
echo "hit=false" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse: $IMAGE:$ID is not published — building"
|
||||
echo "reuse: not published — building"
|
||||
fi
|
||||
|
||||
# --- shadow: can we read a label off the channel tag? (milestone 318 step 3)
|
||||
# Informational ONLY — no `set -e`, every probe falls back to
|
||||
# UNAVAILABLE, and nothing downstream reads this. It must never fail a
|
||||
# build; that is the whole reason it runs before the mechanism does.
|
||||
#
|
||||
# The question: can the reuse check be keyed on an image LABEL read off
|
||||
# the channel tag, instead of on a separate r-<rev> tag? If yes, nothing
|
||||
# is minted per build, the -main/-dev identity split disappears, and the
|
||||
# base-refresh clobber (#3154) stops existing.
|
||||
#
|
||||
# Two questions, answered at different times — worth separating, because
|
||||
# reading "no value" as "the mechanism does not work" would abandon a
|
||||
# working design:
|
||||
#
|
||||
# 1. Do the TEMPLATE MECHANICS work here? Answered on the very first
|
||||
# run, by probes B and C: if they print a labels map (even an empty
|
||||
# one) or a config JSON, `--format` over .Image resolves on this
|
||||
# buildx and this registry. That is the actual gate for step 3.
|
||||
# 2. Does a stamped value ROUND-TRIP? Only answerable once an artifact
|
||||
# genuinely rebuilds. A push that changes no shipped file reuses,
|
||||
# the build is skipped, and nothing gets stamped — so an artifact
|
||||
# keeps reporting no label until its own source next moves.
|
||||
#
|
||||
# That second point is not a defect, it is the migration path: under the
|
||||
# real mechanism "no label" reads as a MISS, so the artifact rebuilds
|
||||
# once and is stamped from then on. One extra build per artifact, ever,
|
||||
# in the safe direction. `imagetools create` cannot add the label on the
|
||||
# reuse path — it copies a manifest and config labels are not manifest
|
||||
# annotations — so there is no way to backfill, and no need to.
|
||||
#
|
||||
# Several probes on purpose: `--format` templating over .Image varies by
|
||||
# buildx version, and one push should tell us which spelling works
|
||||
# rather than costing a round trip per guess.
|
||||
- name: Shadow — read fc.revision off the channel tag (informational)
|
||||
env:
|
||||
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-agent
|
||||
CHANNEL: ${{ steps.tag.outputs.channel }}
|
||||
DERIVED: ${{ steps.reuse.outputs.revision }}
|
||||
run: |
|
||||
set -u
|
||||
# The moving tag this channel publishes. `main` is asked about
|
||||
# :latest, which is what step 4 keeps; :main is on its way out.
|
||||
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
|
||||
REF="$IMAGE:$T"
|
||||
echo "label-shadow: ref=$REF derived=$DERIVED"
|
||||
|
||||
A=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' 2>&1 || echo FAILED)
|
||||
echo "label-shadow: probe-index = $A"
|
||||
|
||||
B=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ .Image.Config.Labels }}' 2>&1 | head -c 300 || echo FAILED)
|
||||
echo "label-shadow: probe-labels = $B"
|
||||
|
||||
C=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ json .Image }}' 2>&1 | head -c 400 || echo FAILED)
|
||||
echo "label-shadow: probe-json = $C"
|
||||
|
||||
echo "label-shadow: buildx = $(docker buildx version 2>&1 || echo UNAVAILABLE)"
|
||||
|
||||
- name: Build and push agent image
|
||||
if: steps.reuse.outputs.hit != 'true'
|
||||
uses: docker/build-push-action@v5
|
||||
@@ -1146,16 +1034,16 @@ jobs:
|
||||
file: agent/Dockerfile
|
||||
push: true
|
||||
tags: ${{ steps.reuse.outputs.build_tags }}
|
||||
# Stamped now, read later. Additive and inert this step: no
|
||||
# decision depends on it until the shadow above proves the
|
||||
# label can actually be read back (milestone 318 step 3).
|
||||
# The reuse key. Read back off the channel tag on the next push to
|
||||
# decide whether that push needs to build at all, so this is not
|
||||
# decoration — an unstamped image is one that will always rebuild.
|
||||
labels: |
|
||||
fc.revision=${{ steps.reuse.outputs.revision }}
|
||||
|
||||
# Registry-side manifest copy: no layer transfer, no local daemon, no
|
||||
# rebuild. Each -t becomes another reference to the SAME manifest the
|
||||
# identity tag holds, so :latest and the date pin are byte-identical to
|
||||
# what was published rather than a lookalike rebuild.
|
||||
# channel tag already holds, so :c-<sha> and the date pin are
|
||||
# byte-identical to what is published rather than a lookalike rebuild.
|
||||
#
|
||||
# Runs on EVERY reuse, which is what keeps family rule 146 true: a
|
||||
# rolling channel refreshes itself, so skipping a build must never mean
|
||||
@@ -1165,7 +1053,7 @@ jobs:
|
||||
if: steps.reuse.outputs.hit == 'true'
|
||||
env:
|
||||
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-agent
|
||||
IDENTITY: ${{ steps.reuse.outputs.identity }}
|
||||
SOURCE: ${{ steps.reuse.outputs.channel_ref }}
|
||||
TAGS: ${{ steps.tag.outputs.tags }}
|
||||
run: |
|
||||
set -euf
|
||||
@@ -1175,6 +1063,10 @@ jobs:
|
||||
IFS=,
|
||||
for t in $TAGS; do ARGS="$ARGS -t $t"; done
|
||||
unset IFS
|
||||
# Source is the channel tag itself — the image we just confirmed
|
||||
# carries this revision. One of the targets is that same tag, which
|
||||
# makes its copy a no-op; the others (:main, :c-<sha>, the date pin)
|
||||
# are what this exists for.
|
||||
# shellcheck disable=SC2086
|
||||
docker buildx imagetools create $ARGS "$IMAGE:$IDENTITY"
|
||||
echo "repointed to $IMAGE:$IDENTITY: $TAGS"
|
||||
docker buildx imagetools create $ARGS "$SOURCE"
|
||||
echo "repointed to $SOURCE: $TAGS"
|
||||
|
||||
Reference in New Issue
Block a user