ci: key the reuse check on an image label, not a tag (318 step 3)
CI / extension-version (push) Successful in 4s
CI / lint (push) Failing after 4s
Build images / sign-extension (push) Successful in 4s
CI / backend-lint-and-test (push) Failing after 13s
CI / frontend-build (push) Successful in 20s
extension / lint (push) Successful in 22s
CI / integration (push) Failing after 2m24s
Build images / build-web (push) Successful in 2m44s
Build images / build-ml (push) Successful in 3m13s
Build images / build-agent (push) Successful in 8m56s
CI / extension-version (push) Successful in 4s
CI / lint (push) Failing after 4s
Build images / sign-extension (push) Successful in 4s
CI / backend-lint-and-test (push) Failing after 13s
CI / frontend-build (push) Successful in 20s
extension / lint (push) Successful in 22s
CI / integration (push) Failing after 2m24s
Build images / build-web (push) Successful in 2m44s
Build images / build-ml (push) Successful in 3m13s
Build images / build-agent (push) Successful in 8m56s
The shadow (dee93fa, run 4732) answered the gate: `imagetools inspect
--format` reads `.Image.Config.Labels` against this registry on buildx
v0.36.1. So the reuse check now asks the moving channel tag whether the image
it already points at carries this commit's `fc.revision`, and the r-<rev>
identity tags stop being published.
Three things this removes rather than manages:
A name minted per build that one thing read. Rule 145's narrowing is aimed
exactly there — "a third name for the same thing is upkeep for a model we do
not run."
The -main/-dev qualifier, and the CHANNELLED list behind it. Which tag you
inspect IS the channel, so the distinction has nowhere to live. cmd_identity
goes with it.
A silent expiry nobody wrote down. r-<rev> matches no branch of the
registry's keep_pattern (#3157), so identity tags were prunable past the
newest 10 — a pruned one costs a rebuild, in the safe direction and entirely
invisibly. A label rides inside a tag that has to exist anyway.
It also dissolves #3154 instead of deferring it: a scheduled base refresh
rebuilds :latest with the same revision label, the next unrelated push sees a
match and skips, and the refreshed base survives. Under the tag scheme that
push repointed :latest back to the older base.
The measured detail that shapes the code: a missing label returns an EMPTY
STRING and exits 0. Branching on the exit code would read "no label yet" as
success and skip a build that was needed. So it compares values, and every
uncertain case — absent label, unreachable tag, older image — lands as empty,
never equals a 12-char revision, and falls through to a build.
Reading the specific key matters too. The map carries the base image's labels,
and org.opencontainers.image.version sits right beside ours reading 24.04 on
the agent — a plausible-looking wrong answer.
Expect every artifact to rebuild once on this push: nothing carries a label
yet and it cannot be backfilled, since the reuse path copies a manifest and
config labels are not manifest annotations. One rebuild per artifact, ever,
self-healing after.
test_artifact_identity.py is rewritten around what is now load-bearing. The
CHANNELLED drift test had nothing left to guard; in its place the revision is
asked of git directly, so the file fails if the derivation ever stops being
"the commit this artifact's own shipped files last changed in".
This commit is contained in:
+159
-267
@@ -452,12 +452,21 @@ jobs:
|
||||
run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin
|
||||
|
||||
# --- reuse-if-published (milestone 313, step 4) ----------------------
|
||||
# The identity tag names this artifact's CONTENT — r-<revision>, the
|
||||
# commit its shipped files last changed in, plus the channel for images
|
||||
# that bake one in. If the registry already carries it, the bytes this
|
||||
# job would produce are already published and the build is pure waste:
|
||||
# the channel and date tags get repointed at the existing manifest
|
||||
# instead, registry-side, in seconds.
|
||||
# Does the image the channel tag already points at carry THIS commit's
|
||||
# revision? If so the bytes this job would produce are already published
|
||||
# and the build is pure waste: the remaining tags get repointed at that
|
||||
# existing manifest instead, registry-side, in seconds.
|
||||
#
|
||||
# Keyed on an `fc.revision` LABEL rather than on a tag of its own
|
||||
# (milestone 318 step 3). A tag would be a name minted per build that one
|
||||
# thing reads — what rule 145 narrowed against — and would be prunable
|
||||
# under the registry's keep_pattern (#3157), silently expiring the cache.
|
||||
# A label rides inside a tag that has to exist anyway.
|
||||
#
|
||||
# An image with no such label reads as a miss and rebuilds. That is the
|
||||
# migration, not a fault: labels cannot be backfilled, since the reuse
|
||||
# path copies a manifest and config labels are not manifest annotations.
|
||||
# Each artifact pays one rebuild, once.
|
||||
#
|
||||
# This is what stops a push that touched only `agent/` from rebuilding
|
||||
# web and ml, and a merge to main from rebuilding what dev already built.
|
||||
@@ -485,88 +494,39 @@ jobs:
|
||||
TAGS: ${{ steps.tag.outputs.tags }}
|
||||
run: |
|
||||
set -eu
|
||||
ID=$(sh scripts/artifacts.sh identity web "$CHANNEL")
|
||||
echo "identity=$ID" >> "$GITHUB_OUTPUT"
|
||||
# The value step 3 will key on once the label mechanism is proven.
|
||||
# Emitted now so the build below can stamp it — stamping is harmless
|
||||
# and is what gives the shadow read something to find next push.
|
||||
REV=$(sh scripts/artifacts.sh revision web)
|
||||
echo "revision=$REV" >> "$GITHUB_OUTPUT"
|
||||
DERIVED=$(sh scripts/artifacts.sh revision web)
|
||||
echo "revision=$DERIVED" >> "$GITHUB_OUTPUT"
|
||||
echo "build_tags=$TAGS" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Every build that runs now claims the identity. The carve-out here
|
||||
# existed only for tag pushes, which rebuilt an already-published
|
||||
# revision and so had to be stopped from re-pointing an immutable
|
||||
# tag at fresh bytes. No tag trigger, nothing to carve out.
|
||||
echo "build_tags=$TAGS,$IMAGE:$ID" >> "$GITHUB_OUTPUT"
|
||||
# The moving tag for this channel. Which tag we ask IS the channel —
|
||||
# that is why the revision needs no -main/-dev qualifier any more.
|
||||
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
|
||||
echo "channel_ref=$IMAGE:$T" >> "$GITHUB_OUTPUT"
|
||||
|
||||
if docker buildx imagetools inspect "$IMAGE:$ID" >/dev/null 2>&1; then
|
||||
# Compare VALUES, never exit codes. Measured on buildx v0.36.1
|
||||
# (run 4732): a missing key returns an empty string and exits 0, so
|
||||
# branching on the exit code would read "no label yet" as success.
|
||||
# An unreachable tag also lands here as empty via the `|| echo`.
|
||||
# Empty never equals a 12-char revision, so every uncertain case
|
||||
# falls through to a build — the safe direction, with no special
|
||||
# casing for it.
|
||||
#
|
||||
# Read the SPECIFIC key. The map also carries whatever the base image
|
||||
# set, and `org.opencontainers.image.version` sits right beside ours
|
||||
# looking like a plausible answer (it reads 24.04 on the agent).
|
||||
PUBLISHED=$(docker buildx imagetools inspect "$IMAGE:$T" \
|
||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' \
|
||||
2>/dev/null || echo "")
|
||||
echo "reuse: $IMAGE:$T carries fc.revision=${PUBLISHED:-<none>}; derived=$DERIVED"
|
||||
|
||||
if [ -n "$PUBLISHED" ] && [ "$PUBLISHED" = "$DERIVED" ]; then
|
||||
echo "hit=true" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse: $IMAGE:$ID is already published — skipping the build"
|
||||
echo "reuse: already published — skipping the build"
|
||||
else
|
||||
echo "hit=false" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse: $IMAGE:$ID is not published — building"
|
||||
echo "reuse: not published — building"
|
||||
fi
|
||||
|
||||
# --- shadow: can we read a label off the channel tag? (milestone 318 step 3)
|
||||
# Informational ONLY — no `set -e`, every probe falls back to
|
||||
# UNAVAILABLE, and nothing downstream reads this. It must never fail a
|
||||
# build; that is the whole reason it runs before the mechanism does.
|
||||
#
|
||||
# The question: can the reuse check be keyed on an image LABEL read off
|
||||
# the channel tag, instead of on a separate r-<rev> tag? If yes, nothing
|
||||
# is minted per build, the -main/-dev identity split disappears, and the
|
||||
# base-refresh clobber (#3154) stops existing.
|
||||
#
|
||||
# Two questions, answered at different times — worth separating, because
|
||||
# reading "no value" as "the mechanism does not work" would abandon a
|
||||
# working design:
|
||||
#
|
||||
# 1. Do the TEMPLATE MECHANICS work here? Answered on the very first
|
||||
# run, by probes B and C: if they print a labels map (even an empty
|
||||
# one) or a config JSON, `--format` over .Image resolves on this
|
||||
# buildx and this registry. That is the actual gate for step 3.
|
||||
# 2. Does a stamped value ROUND-TRIP? Only answerable once an artifact
|
||||
# genuinely rebuilds. A push that changes no shipped file reuses,
|
||||
# the build is skipped, and nothing gets stamped — so an artifact
|
||||
# keeps reporting no label until its own source next moves.
|
||||
#
|
||||
# That second point is not a defect, it is the migration path: under the
|
||||
# real mechanism "no label" reads as a MISS, so the artifact rebuilds
|
||||
# once and is stamped from then on. One extra build per artifact, ever,
|
||||
# in the safe direction. `imagetools create` cannot add the label on the
|
||||
# reuse path — it copies a manifest and config labels are not manifest
|
||||
# annotations — so there is no way to backfill, and no need to.
|
||||
#
|
||||
# Several probes on purpose: `--format` templating over .Image varies by
|
||||
# buildx version, and one push should tell us which spelling works
|
||||
# rather than costing a round trip per guess.
|
||||
- name: Shadow — read fc.revision off the channel tag (informational)
|
||||
env:
|
||||
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator
|
||||
CHANNEL: ${{ steps.tag.outputs.channel }}
|
||||
DERIVED: ${{ steps.reuse.outputs.revision }}
|
||||
run: |
|
||||
set -u
|
||||
# The moving tag this channel publishes. `main` is asked about
|
||||
# :latest, which is what step 4 keeps; :main is on its way out.
|
||||
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
|
||||
REF="$IMAGE:$T"
|
||||
echo "label-shadow: ref=$REF derived=$DERIVED"
|
||||
|
||||
A=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' 2>&1 || echo FAILED)
|
||||
echo "label-shadow: probe-index = $A"
|
||||
|
||||
B=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ .Image.Config.Labels }}' 2>&1 | head -c 300 || echo FAILED)
|
||||
echo "label-shadow: probe-labels = $B"
|
||||
|
||||
C=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ json .Image }}' 2>&1 | head -c 400 || echo FAILED)
|
||||
echo "label-shadow: probe-json = $C"
|
||||
|
||||
echo "label-shadow: buildx = $(docker buildx version 2>&1 || echo UNAVAILABLE)"
|
||||
|
||||
- name: Download signed XPI from Forgejo release asset
|
||||
# dev and main each bundle the XPI their own sign-extension just
|
||||
# published — the point of the channel work (milestone 271 step 6): the
|
||||
@@ -651,9 +611,9 @@ jobs:
|
||||
file: Dockerfile
|
||||
push: true
|
||||
tags: ${{ steps.reuse.outputs.build_tags }}
|
||||
# Stamped now, read later. Additive and inert this step: no
|
||||
# decision depends on it until the shadow above proves the
|
||||
# label can actually be read back (milestone 318 step 3).
|
||||
# The reuse key. Read back off the channel tag on the next push to
|
||||
# decide whether that push needs to build at all, so this is not
|
||||
# decoration — an unstamped image is one that will always rebuild.
|
||||
labels: |
|
||||
fc.revision=${{ steps.reuse.outputs.revision }}
|
||||
# Only the web image carries a channel: it is the one that serves
|
||||
@@ -664,8 +624,8 @@ jobs:
|
||||
|
||||
# Registry-side manifest copy: no layer transfer, no local daemon, no
|
||||
# rebuild. Each -t becomes another reference to the SAME manifest the
|
||||
# identity tag holds, so :latest and the date pin are byte-identical to
|
||||
# what was published rather than a lookalike rebuild.
|
||||
# channel tag already holds, so :c-<sha> and the date pin are
|
||||
# byte-identical to what is published rather than a lookalike rebuild.
|
||||
#
|
||||
# Runs on EVERY reuse, which is what keeps family rule 146 true: a
|
||||
# rolling channel refreshes itself, so skipping a build must never mean
|
||||
@@ -675,7 +635,7 @@ jobs:
|
||||
if: steps.reuse.outputs.hit == 'true'
|
||||
env:
|
||||
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator
|
||||
IDENTITY: ${{ steps.reuse.outputs.identity }}
|
||||
SOURCE: ${{ steps.reuse.outputs.channel_ref }}
|
||||
TAGS: ${{ steps.tag.outputs.tags }}
|
||||
run: |
|
||||
set -euf
|
||||
@@ -685,9 +645,13 @@ jobs:
|
||||
IFS=,
|
||||
for t in $TAGS; do ARGS="$ARGS -t $t"; done
|
||||
unset IFS
|
||||
# Source is the channel tag itself — the image we just confirmed
|
||||
# carries this revision. One of the targets is that same tag, which
|
||||
# makes its copy a no-op; the others (:main, :c-<sha>, the date pin)
|
||||
# are what this exists for.
|
||||
# shellcheck disable=SC2086
|
||||
docker buildx imagetools create $ARGS "$IMAGE:$IDENTITY"
|
||||
echo "repointed to $IMAGE:$IDENTITY: $TAGS"
|
||||
docker buildx imagetools create $ARGS "$SOURCE"
|
||||
echo "repointed to $SOURCE: $TAGS"
|
||||
|
||||
build-ml:
|
||||
runs-on: python-ci
|
||||
@@ -780,12 +744,21 @@ jobs:
|
||||
run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin
|
||||
|
||||
# --- reuse-if-published (milestone 313, step 4) ----------------------
|
||||
# The identity tag names this artifact's CONTENT — r-<revision>, the
|
||||
# commit its shipped files last changed in, plus the channel for images
|
||||
# that bake one in. If the registry already carries it, the bytes this
|
||||
# job would produce are already published and the build is pure waste:
|
||||
# the channel and date tags get repointed at the existing manifest
|
||||
# instead, registry-side, in seconds.
|
||||
# Does the image the channel tag already points at carry THIS commit's
|
||||
# revision? If so the bytes this job would produce are already published
|
||||
# and the build is pure waste: the remaining tags get repointed at that
|
||||
# existing manifest instead, registry-side, in seconds.
|
||||
#
|
||||
# Keyed on an `fc.revision` LABEL rather than on a tag of its own
|
||||
# (milestone 318 step 3). A tag would be a name minted per build that one
|
||||
# thing reads — what rule 145 narrowed against — and would be prunable
|
||||
# under the registry's keep_pattern (#3157), silently expiring the cache.
|
||||
# A label rides inside a tag that has to exist anyway.
|
||||
#
|
||||
# An image with no such label reads as a miss and rebuilds. That is the
|
||||
# migration, not a fault: labels cannot be backfilled, since the reuse
|
||||
# path copies a manifest and config labels are not manifest annotations.
|
||||
# Each artifact pays one rebuild, once.
|
||||
#
|
||||
# This is what stops a push that touched only `agent/` from rebuilding
|
||||
# web and ml, and a merge to main from rebuilding what dev already built.
|
||||
@@ -813,88 +786,39 @@ jobs:
|
||||
TAGS: ${{ steps.tag.outputs.tags }}
|
||||
run: |
|
||||
set -eu
|
||||
ID=$(sh scripts/artifacts.sh identity ml "$CHANNEL")
|
||||
echo "identity=$ID" >> "$GITHUB_OUTPUT"
|
||||
# The value step 3 will key on once the label mechanism is proven.
|
||||
# Emitted now so the build below can stamp it — stamping is harmless
|
||||
# and is what gives the shadow read something to find next push.
|
||||
REV=$(sh scripts/artifacts.sh revision ml)
|
||||
echo "revision=$REV" >> "$GITHUB_OUTPUT"
|
||||
DERIVED=$(sh scripts/artifacts.sh revision ml)
|
||||
echo "revision=$DERIVED" >> "$GITHUB_OUTPUT"
|
||||
echo "build_tags=$TAGS" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Every build that runs now claims the identity. The carve-out here
|
||||
# existed only for tag pushes, which rebuilt an already-published
|
||||
# revision and so had to be stopped from re-pointing an immutable
|
||||
# tag at fresh bytes. No tag trigger, nothing to carve out.
|
||||
echo "build_tags=$TAGS,$IMAGE:$ID" >> "$GITHUB_OUTPUT"
|
||||
# The moving tag for this channel. Which tag we ask IS the channel —
|
||||
# that is why the revision needs no -main/-dev qualifier any more.
|
||||
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
|
||||
echo "channel_ref=$IMAGE:$T" >> "$GITHUB_OUTPUT"
|
||||
|
||||
if docker buildx imagetools inspect "$IMAGE:$ID" >/dev/null 2>&1; then
|
||||
# Compare VALUES, never exit codes. Measured on buildx v0.36.1
|
||||
# (run 4732): a missing key returns an empty string and exits 0, so
|
||||
# branching on the exit code would read "no label yet" as success.
|
||||
# An unreachable tag also lands here as empty via the `|| echo`.
|
||||
# Empty never equals a 12-char revision, so every uncertain case
|
||||
# falls through to a build — the safe direction, with no special
|
||||
# casing for it.
|
||||
#
|
||||
# Read the SPECIFIC key. The map also carries whatever the base image
|
||||
# set, and `org.opencontainers.image.version` sits right beside ours
|
||||
# looking like a plausible answer (it reads 24.04 on the agent).
|
||||
PUBLISHED=$(docker buildx imagetools inspect "$IMAGE:$T" \
|
||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' \
|
||||
2>/dev/null || echo "")
|
||||
echo "reuse: $IMAGE:$T carries fc.revision=${PUBLISHED:-<none>}; derived=$DERIVED"
|
||||
|
||||
if [ -n "$PUBLISHED" ] && [ "$PUBLISHED" = "$DERIVED" ]; then
|
||||
echo "hit=true" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse: $IMAGE:$ID is already published — skipping the build"
|
||||
echo "reuse: already published — skipping the build"
|
||||
else
|
||||
echo "hit=false" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse: $IMAGE:$ID is not published — building"
|
||||
echo "reuse: not published — building"
|
||||
fi
|
||||
|
||||
# --- shadow: can we read a label off the channel tag? (milestone 318 step 3)
|
||||
# Informational ONLY — no `set -e`, every probe falls back to
|
||||
# UNAVAILABLE, and nothing downstream reads this. It must never fail a
|
||||
# build; that is the whole reason it runs before the mechanism does.
|
||||
#
|
||||
# The question: can the reuse check be keyed on an image LABEL read off
|
||||
# the channel tag, instead of on a separate r-<rev> tag? If yes, nothing
|
||||
# is minted per build, the -main/-dev identity split disappears, and the
|
||||
# base-refresh clobber (#3154) stops existing.
|
||||
#
|
||||
# Two questions, answered at different times — worth separating, because
|
||||
# reading "no value" as "the mechanism does not work" would abandon a
|
||||
# working design:
|
||||
#
|
||||
# 1. Do the TEMPLATE MECHANICS work here? Answered on the very first
|
||||
# run, by probes B and C: if they print a labels map (even an empty
|
||||
# one) or a config JSON, `--format` over .Image resolves on this
|
||||
# buildx and this registry. That is the actual gate for step 3.
|
||||
# 2. Does a stamped value ROUND-TRIP? Only answerable once an artifact
|
||||
# genuinely rebuilds. A push that changes no shipped file reuses,
|
||||
# the build is skipped, and nothing gets stamped — so an artifact
|
||||
# keeps reporting no label until its own source next moves.
|
||||
#
|
||||
# That second point is not a defect, it is the migration path: under the
|
||||
# real mechanism "no label" reads as a MISS, so the artifact rebuilds
|
||||
# once and is stamped from then on. One extra build per artifact, ever,
|
||||
# in the safe direction. `imagetools create` cannot add the label on the
|
||||
# reuse path — it copies a manifest and config labels are not manifest
|
||||
# annotations — so there is no way to backfill, and no need to.
|
||||
#
|
||||
# Several probes on purpose: `--format` templating over .Image varies by
|
||||
# buildx version, and one push should tell us which spelling works
|
||||
# rather than costing a round trip per guess.
|
||||
- name: Shadow — read fc.revision off the channel tag (informational)
|
||||
env:
|
||||
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-ml
|
||||
CHANNEL: ${{ steps.tag.outputs.channel }}
|
||||
DERIVED: ${{ steps.reuse.outputs.revision }}
|
||||
run: |
|
||||
set -u
|
||||
# The moving tag this channel publishes. `main` is asked about
|
||||
# :latest, which is what step 4 keeps; :main is on its way out.
|
||||
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
|
||||
REF="$IMAGE:$T"
|
||||
echo "label-shadow: ref=$REF derived=$DERIVED"
|
||||
|
||||
A=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' 2>&1 || echo FAILED)
|
||||
echo "label-shadow: probe-index = $A"
|
||||
|
||||
B=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ .Image.Config.Labels }}' 2>&1 | head -c 300 || echo FAILED)
|
||||
echo "label-shadow: probe-labels = $B"
|
||||
|
||||
C=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ json .Image }}' 2>&1 | head -c 400 || echo FAILED)
|
||||
echo "label-shadow: probe-json = $C"
|
||||
|
||||
echo "label-shadow: buildx = $(docker buildx version 2>&1 || echo UNAVAILABLE)"
|
||||
|
||||
- name: Build and push ml image
|
||||
if: steps.reuse.outputs.hit != 'true'
|
||||
uses: docker/build-push-action@v5
|
||||
@@ -903,16 +827,16 @@ jobs:
|
||||
file: Dockerfile.ml
|
||||
push: true
|
||||
tags: ${{ steps.reuse.outputs.build_tags }}
|
||||
# Stamped now, read later. Additive and inert this step: no
|
||||
# decision depends on it until the shadow above proves the
|
||||
# label can actually be read back (milestone 318 step 3).
|
||||
# The reuse key. Read back off the channel tag on the next push to
|
||||
# decide whether that push needs to build at all, so this is not
|
||||
# decoration — an unstamped image is one that will always rebuild.
|
||||
labels: |
|
||||
fc.revision=${{ steps.reuse.outputs.revision }}
|
||||
|
||||
# Registry-side manifest copy: no layer transfer, no local daemon, no
|
||||
# rebuild. Each -t becomes another reference to the SAME manifest the
|
||||
# identity tag holds, so :latest and the date pin are byte-identical to
|
||||
# what was published rather than a lookalike rebuild.
|
||||
# channel tag already holds, so :c-<sha> and the date pin are
|
||||
# byte-identical to what is published rather than a lookalike rebuild.
|
||||
#
|
||||
# Runs on EVERY reuse, which is what keeps family rule 146 true: a
|
||||
# rolling channel refreshes itself, so skipping a build must never mean
|
||||
@@ -922,7 +846,7 @@ jobs:
|
||||
if: steps.reuse.outputs.hit == 'true'
|
||||
env:
|
||||
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-ml
|
||||
IDENTITY: ${{ steps.reuse.outputs.identity }}
|
||||
SOURCE: ${{ steps.reuse.outputs.channel_ref }}
|
||||
TAGS: ${{ steps.tag.outputs.tags }}
|
||||
run: |
|
||||
set -euf
|
||||
@@ -932,9 +856,13 @@ jobs:
|
||||
IFS=,
|
||||
for t in $TAGS; do ARGS="$ARGS -t $t"; done
|
||||
unset IFS
|
||||
# Source is the channel tag itself — the image we just confirmed
|
||||
# carries this revision. One of the targets is that same tag, which
|
||||
# makes its copy a no-op; the others (:main, :c-<sha>, the date pin)
|
||||
# are what this exists for.
|
||||
# shellcheck disable=SC2086
|
||||
docker buildx imagetools create $ARGS "$IMAGE:$IDENTITY"
|
||||
echo "repointed to $IMAGE:$IDENTITY: $TAGS"
|
||||
docker buildx imagetools create $ARGS "$SOURCE"
|
||||
echo "repointed to $SOURCE: $TAGS"
|
||||
|
||||
# The desktop GPU agent (#114) — published so the operator pulls + runs it on
|
||||
# the GPU machine instead of building locally. Independent of web/ml (its own
|
||||
@@ -1023,12 +951,21 @@ jobs:
|
||||
run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin
|
||||
|
||||
# --- reuse-if-published (milestone 313, step 4) ----------------------
|
||||
# The identity tag names this artifact's CONTENT — r-<revision>, the
|
||||
# commit its shipped files last changed in, plus the channel for images
|
||||
# that bake one in. If the registry already carries it, the bytes this
|
||||
# job would produce are already published and the build is pure waste:
|
||||
# the channel and date tags get repointed at the existing manifest
|
||||
# instead, registry-side, in seconds.
|
||||
# Does the image the channel tag already points at carry THIS commit's
|
||||
# revision? If so the bytes this job would produce are already published
|
||||
# and the build is pure waste: the remaining tags get repointed at that
|
||||
# existing manifest instead, registry-side, in seconds.
|
||||
#
|
||||
# Keyed on an `fc.revision` LABEL rather than on a tag of its own
|
||||
# (milestone 318 step 3). A tag would be a name minted per build that one
|
||||
# thing reads — what rule 145 narrowed against — and would be prunable
|
||||
# under the registry's keep_pattern (#3157), silently expiring the cache.
|
||||
# A label rides inside a tag that has to exist anyway.
|
||||
#
|
||||
# An image with no such label reads as a miss and rebuilds. That is the
|
||||
# migration, not a fault: labels cannot be backfilled, since the reuse
|
||||
# path copies a manifest and config labels are not manifest annotations.
|
||||
# Each artifact pays one rebuild, once.
|
||||
#
|
||||
# This is what stops a push that touched only `agent/` from rebuilding
|
||||
# web and ml, and a merge to main from rebuilding what dev already built.
|
||||
@@ -1056,88 +993,39 @@ jobs:
|
||||
TAGS: ${{ steps.tag.outputs.tags }}
|
||||
run: |
|
||||
set -eu
|
||||
ID=$(sh scripts/artifacts.sh identity agent "$CHANNEL")
|
||||
echo "identity=$ID" >> "$GITHUB_OUTPUT"
|
||||
# The value step 3 will key on once the label mechanism is proven.
|
||||
# Emitted now so the build below can stamp it — stamping is harmless
|
||||
# and is what gives the shadow read something to find next push.
|
||||
REV=$(sh scripts/artifacts.sh revision agent)
|
||||
echo "revision=$REV" >> "$GITHUB_OUTPUT"
|
||||
DERIVED=$(sh scripts/artifacts.sh revision agent)
|
||||
echo "revision=$DERIVED" >> "$GITHUB_OUTPUT"
|
||||
echo "build_tags=$TAGS" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Every build that runs now claims the identity. The carve-out here
|
||||
# existed only for tag pushes, which rebuilt an already-published
|
||||
# revision and so had to be stopped from re-pointing an immutable
|
||||
# tag at fresh bytes. No tag trigger, nothing to carve out.
|
||||
echo "build_tags=$TAGS,$IMAGE:$ID" >> "$GITHUB_OUTPUT"
|
||||
# The moving tag for this channel. Which tag we ask IS the channel —
|
||||
# that is why the revision needs no -main/-dev qualifier any more.
|
||||
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
|
||||
echo "channel_ref=$IMAGE:$T" >> "$GITHUB_OUTPUT"
|
||||
|
||||
if docker buildx imagetools inspect "$IMAGE:$ID" >/dev/null 2>&1; then
|
||||
# Compare VALUES, never exit codes. Measured on buildx v0.36.1
|
||||
# (run 4732): a missing key returns an empty string and exits 0, so
|
||||
# branching on the exit code would read "no label yet" as success.
|
||||
# An unreachable tag also lands here as empty via the `|| echo`.
|
||||
# Empty never equals a 12-char revision, so every uncertain case
|
||||
# falls through to a build — the safe direction, with no special
|
||||
# casing for it.
|
||||
#
|
||||
# Read the SPECIFIC key. The map also carries whatever the base image
|
||||
# set, and `org.opencontainers.image.version` sits right beside ours
|
||||
# looking like a plausible answer (it reads 24.04 on the agent).
|
||||
PUBLISHED=$(docker buildx imagetools inspect "$IMAGE:$T" \
|
||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' \
|
||||
2>/dev/null || echo "")
|
||||
echo "reuse: $IMAGE:$T carries fc.revision=${PUBLISHED:-<none>}; derived=$DERIVED"
|
||||
|
||||
if [ -n "$PUBLISHED" ] && [ "$PUBLISHED" = "$DERIVED" ]; then
|
||||
echo "hit=true" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse: $IMAGE:$ID is already published — skipping the build"
|
||||
echo "reuse: already published — skipping the build"
|
||||
else
|
||||
echo "hit=false" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse: $IMAGE:$ID is not published — building"
|
||||
echo "reuse: not published — building"
|
||||
fi
|
||||
|
||||
# --- shadow: can we read a label off the channel tag? (milestone 318 step 3)
|
||||
# Informational ONLY — no `set -e`, every probe falls back to
|
||||
# UNAVAILABLE, and nothing downstream reads this. It must never fail a
|
||||
# build; that is the whole reason it runs before the mechanism does.
|
||||
#
|
||||
# The question: can the reuse check be keyed on an image LABEL read off
|
||||
# the channel tag, instead of on a separate r-<rev> tag? If yes, nothing
|
||||
# is minted per build, the -main/-dev identity split disappears, and the
|
||||
# base-refresh clobber (#3154) stops existing.
|
||||
#
|
||||
# Two questions, answered at different times — worth separating, because
|
||||
# reading "no value" as "the mechanism does not work" would abandon a
|
||||
# working design:
|
||||
#
|
||||
# 1. Do the TEMPLATE MECHANICS work here? Answered on the very first
|
||||
# run, by probes B and C: if they print a labels map (even an empty
|
||||
# one) or a config JSON, `--format` over .Image resolves on this
|
||||
# buildx and this registry. That is the actual gate for step 3.
|
||||
# 2. Does a stamped value ROUND-TRIP? Only answerable once an artifact
|
||||
# genuinely rebuilds. A push that changes no shipped file reuses,
|
||||
# the build is skipped, and nothing gets stamped — so an artifact
|
||||
# keeps reporting no label until its own source next moves.
|
||||
#
|
||||
# That second point is not a defect, it is the migration path: under the
|
||||
# real mechanism "no label" reads as a MISS, so the artifact rebuilds
|
||||
# once and is stamped from then on. One extra build per artifact, ever,
|
||||
# in the safe direction. `imagetools create` cannot add the label on the
|
||||
# reuse path — it copies a manifest and config labels are not manifest
|
||||
# annotations — so there is no way to backfill, and no need to.
|
||||
#
|
||||
# Several probes on purpose: `--format` templating over .Image varies by
|
||||
# buildx version, and one push should tell us which spelling works
|
||||
# rather than costing a round trip per guess.
|
||||
- name: Shadow — read fc.revision off the channel tag (informational)
|
||||
env:
|
||||
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-agent
|
||||
CHANNEL: ${{ steps.tag.outputs.channel }}
|
||||
DERIVED: ${{ steps.reuse.outputs.revision }}
|
||||
run: |
|
||||
set -u
|
||||
# The moving tag this channel publishes. `main` is asked about
|
||||
# :latest, which is what step 4 keeps; :main is on its way out.
|
||||
if [ "$CHANNEL" = "main" ]; then T=latest; else T=dev; fi
|
||||
REF="$IMAGE:$T"
|
||||
echo "label-shadow: ref=$REF derived=$DERIVED"
|
||||
|
||||
A=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ index .Image.Config.Labels "fc.revision" }}' 2>&1 || echo FAILED)
|
||||
echo "label-shadow: probe-index = $A"
|
||||
|
||||
B=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ .Image.Config.Labels }}' 2>&1 | head -c 300 || echo FAILED)
|
||||
echo "label-shadow: probe-labels = $B"
|
||||
|
||||
C=$(docker buildx imagetools inspect "$REF" \
|
||||
--format '{{ json .Image }}' 2>&1 | head -c 400 || echo FAILED)
|
||||
echo "label-shadow: probe-json = $C"
|
||||
|
||||
echo "label-shadow: buildx = $(docker buildx version 2>&1 || echo UNAVAILABLE)"
|
||||
|
||||
- name: Build and push agent image
|
||||
if: steps.reuse.outputs.hit != 'true'
|
||||
uses: docker/build-push-action@v5
|
||||
@@ -1146,16 +1034,16 @@ jobs:
|
||||
file: agent/Dockerfile
|
||||
push: true
|
||||
tags: ${{ steps.reuse.outputs.build_tags }}
|
||||
# Stamped now, read later. Additive and inert this step: no
|
||||
# decision depends on it until the shadow above proves the
|
||||
# label can actually be read back (milestone 318 step 3).
|
||||
# The reuse key. Read back off the channel tag on the next push to
|
||||
# decide whether that push needs to build at all, so this is not
|
||||
# decoration — an unstamped image is one that will always rebuild.
|
||||
labels: |
|
||||
fc.revision=${{ steps.reuse.outputs.revision }}
|
||||
|
||||
# Registry-side manifest copy: no layer transfer, no local daemon, no
|
||||
# rebuild. Each -t becomes another reference to the SAME manifest the
|
||||
# identity tag holds, so :latest and the date pin are byte-identical to
|
||||
# what was published rather than a lookalike rebuild.
|
||||
# channel tag already holds, so :c-<sha> and the date pin are
|
||||
# byte-identical to what is published rather than a lookalike rebuild.
|
||||
#
|
||||
# Runs on EVERY reuse, which is what keeps family rule 146 true: a
|
||||
# rolling channel refreshes itself, so skipping a build must never mean
|
||||
@@ -1165,7 +1053,7 @@ jobs:
|
||||
if: steps.reuse.outputs.hit == 'true'
|
||||
env:
|
||||
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-agent
|
||||
IDENTITY: ${{ steps.reuse.outputs.identity }}
|
||||
SOURCE: ${{ steps.reuse.outputs.channel_ref }}
|
||||
TAGS: ${{ steps.tag.outputs.tags }}
|
||||
run: |
|
||||
set -euf
|
||||
@@ -1175,6 +1063,10 @@ jobs:
|
||||
IFS=,
|
||||
for t in $TAGS; do ARGS="$ARGS -t $t"; done
|
||||
unset IFS
|
||||
# Source is the channel tag itself — the image we just confirmed
|
||||
# carries this revision. One of the targets is that same tag, which
|
||||
# makes its copy a no-op; the others (:main, :c-<sha>, the date pin)
|
||||
# are what this exists for.
|
||||
# shellcheck disable=SC2086
|
||||
docker buildx imagetools create $ARGS "$IMAGE:$IDENTITY"
|
||||
echo "repointed to $IMAGE:$IDENTITY: $TAGS"
|
||||
docker buildx imagetools create $ARGS "$SOURCE"
|
||||
echo "repointed to $SOURCE: $TAGS"
|
||||
|
||||
+12
-58
@@ -56,25 +56,9 @@ ML_PATHS='Dockerfile.ml requirements-ml.txt requirements.txt backend alembic ale
|
||||
# it: this is deliberately NOT `agent/`.
|
||||
AGENT_PATHS='agent/Dockerfile agent/requirements.txt agent/fc_agent'
|
||||
|
||||
# Which artifacts bake the BUILD CHANNEL into the image, and therefore cannot
|
||||
# share a content identity across channels. The web image takes FC_CHANNEL as
|
||||
# a build-arg and reports it from /api/extension/manifest (milestone 271 step
|
||||
# 7), so `main` and `dev` builds of one revision are genuinely different
|
||||
# images — reusing the dev one on main would ship an instance that names
|
||||
# itself `dev` forever.
|
||||
#
|
||||
# ml and agent take no build-args at all: one revision, one image, and a merge
|
||||
# to main can reuse exactly what dev already built. That is not a detail, it is
|
||||
# most of what step 4 saves — merges would otherwise rebuild the agent's CUDA
|
||||
# image to produce bytes that already exist.
|
||||
#
|
||||
# Extend this list if a second artifact ever gains a build-arg;
|
||||
# tests/test_artifact_identity.py reads the Dockerfiles and fails if it drifts.
|
||||
CHANNELLED='web'
|
||||
|
||||
usage() {
|
||||
echo "usage: artifacts.sh {paths|revision|version|tag} {web|ml|agent|extension}" >&2
|
||||
echo " artifacts.sh identity {web|ml|agent} [channel]" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
@@ -124,9 +108,18 @@ strip0() {
|
||||
}
|
||||
|
||||
# The IDENTITY of an artifact's content: the commit its shipped files last
|
||||
# changed in. This — not the tag — is what decides whether a build can be
|
||||
# skipped, because the published tag is only day-precise and two different
|
||||
# builds can share it.
|
||||
# changed in. This is what decides whether a build can be skipped.
|
||||
#
|
||||
# It is published as the `fc.revision` LABEL on the image itself, and read
|
||||
# back off the moving channel tag — not as a tag of its own (milestone 318
|
||||
# step 3). A tag would be a name minted per build that only one thing reads,
|
||||
# which is what rule 145 narrowed against; it would also be prunable under the
|
||||
# registry's keep_pattern (#3157), so the cache would silently expire.
|
||||
#
|
||||
# A published image with no such label reads as a MISS and rebuilds. That is
|
||||
# the migration path, not a fault: `imagetools create` copies a manifest and
|
||||
# config labels are not manifest annotations, so the reuse path cannot stamp
|
||||
# one and there is nothing to backfill. Each artifact pays one rebuild, once.
|
||||
cmd_revision() {
|
||||
echo "$(newest "$1")" | cut -d' ' -f2 | cut -c1-12
|
||||
}
|
||||
@@ -157,50 +150,11 @@ cmd_tag() {
|
||||
"$(strip0 "$(fmt "$sha" %d)")"
|
||||
}
|
||||
|
||||
# The CONTENT IDENTITY of a published image: an immutable tag naming exactly
|
||||
# what a build of this commit would produce. build.yml asks the registry for it
|
||||
# and, on a hit, skips the build entirely and repoints the channel and date
|
||||
# tags at the manifest that is already there (milestone 313 step 4).
|
||||
#
|
||||
# It is deliberately NOT either of the other two values:
|
||||
# * the date tag is day-precise and last-one-wins, so two different builds
|
||||
# share it — it cannot answer "is this content published?".
|
||||
# * the commit sha moves on every push, so it would never hit, which is the
|
||||
# redundant rebuild this exists to remove.
|
||||
#
|
||||
# The revision does both jobs: it is content-unique AND stable across pushes
|
||||
# that did not touch the artifact.
|
||||
cmd_identity() {
|
||||
_art=$1
|
||||
_chan=${2:-}
|
||||
case "$_art" in
|
||||
web|ml|agent) ;;
|
||||
extension)
|
||||
echo "artifacts.sh: the extension is cached as an ext-<version> Forgejo release, not an image tag — use \`version\`" >&2
|
||||
exit 2 ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
for _c in $CHANNELLED; do
|
||||
if [ "$_art" = "$_c" ]; then
|
||||
# Refused rather than defaulted: an unqualified identity for a
|
||||
# channelled artifact would let a dev image be reused as the main one.
|
||||
if [ -z "$_chan" ]; then
|
||||
echo "artifacts.sh: $_art bakes the channel into the image — identity needs one" >&2
|
||||
exit 2
|
||||
fi
|
||||
printf 'r-%s-%s\n' "$(cmd_revision "$_art")" "$_chan"
|
||||
return
|
||||
fi
|
||||
done
|
||||
printf 'r-%s\n' "$(cmd_revision "$_art")"
|
||||
}
|
||||
|
||||
[ $# -ge 2 ] || usage
|
||||
case "$1" in
|
||||
paths) cmd_paths "$2" ;;
|
||||
revision) cmd_revision "$2" ;;
|
||||
version) cmd_version "$2" ;;
|
||||
tag) cmd_tag "$2" ;;
|
||||
identity) cmd_identity "$2" "${3:-}" ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
|
||||
+78
-116
@@ -1,140 +1,102 @@
|
||||
"""`artifacts.sh identity` is what decides whether a build gets skipped.
|
||||
"""`artifacts.sh revision` is what decides whether a build gets skipped.
|
||||
|
||||
Milestone 313 step 4: build.yml asks the registry for `<image>:<identity>` and,
|
||||
on a hit, publishes NO new bytes — it repoints the channel and date tags at the
|
||||
manifest already there. So the identity has to be a true name for the content.
|
||||
Both ways of getting it wrong are silent at build time and only surface in
|
||||
production:
|
||||
Milestone 318 step 3: each image carries its revision as an `fc.revision`
|
||||
label, and build.yml reads that label back off the moving channel tag. Equal
|
||||
to the derived revision means the bytes this push would produce are already
|
||||
published, so the build is skipped.
|
||||
|
||||
* **too coarse** — two genuinely different images share an identity, so the
|
||||
second one never gets built and its tags point at the first one's bytes. The
|
||||
live case is FC_CHANNEL: a `dev` and a `main` build of one revision differ,
|
||||
and collapsing them ships an instance that reports the wrong channel forever.
|
||||
* **too fine** — the identity moves when the content did not, nothing ever
|
||||
hits, and step 4 buys nothing. A commit sha would do exactly this.
|
||||
That makes the revision load-bearing in a way a version string is not — it is
|
||||
compared for equality against a value stamped into a real published artifact.
|
||||
Both ways of getting it wrong are silent:
|
||||
|
||||
The Dockerfiles are read here rather than trusted, because the coarse direction
|
||||
appears the moment someone adds a build-arg without touching `CHANNELLED`.
|
||||
* **it does not identify the content** — a revision that moves when the source
|
||||
did not (a HEAD-derived value, say) never matches, nothing is ever skipped,
|
||||
and the mechanism quietly buys nothing while every lane stays green.
|
||||
* **it identifies the wrong content** — a revision that holds still when the
|
||||
source DID change matches a stale label, the build is skipped, and the
|
||||
channel serves bytes that do not correspond to the commit. This is the
|
||||
dangerous direction, and it is what `test_artifact_paths.py` guards from the
|
||||
other side by pinning the path sets.
|
||||
|
||||
This module owns the narrower claim: whatever the path sets say, the revision
|
||||
is genuinely the commit those paths last changed in.
|
||||
|
||||
The identity-TAG tests this file used to hold are gone with the tag. There is
|
||||
no longer a `CHANNELLED` list to drift (the channel is which tag you inspect),
|
||||
and no `identity` subcommand to refuse an unqualified call.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
from test_artifact_paths import ROOT, declared_paths
|
||||
|
||||
# Only image artifacts have an identity — the extension is cached as an
|
||||
# ext-<version> Forgejo release, not a registry tag.
|
||||
IMAGE_ARTIFACTS = {
|
||||
"web": "Dockerfile",
|
||||
"ml": "Dockerfile.ml",
|
||||
"agent": "agent/Dockerfile",
|
||||
}
|
||||
ARTIFACTS = ("web", "ml", "agent", "extension")
|
||||
|
||||
CHANNELS = ("main", "dev")
|
||||
|
||||
# docker's own tag grammar: [A-Za-z0-9_][A-Za-z0-9._-]{0,127}
|
||||
_TAG = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9._-]{0,127}$")
|
||||
|
||||
# `ARG FC_CHANNEL` in a Dockerfile means build.yml passes a per-channel value
|
||||
# in, so the channel is part of what the image IS.
|
||||
_ARG_CHANNEL = re.compile(r"^\s*ARG\s+FC_CHANNEL\b", re.MULTILINE)
|
||||
# 12 hex chars — the prefix build.yml stamps and compares.
|
||||
_REVISION = re.compile(r"^[0-9a-f]{12}$")
|
||||
|
||||
|
||||
def identity(artifact: str, channel: str | None = None) -> subprocess.CompletedProcess:
|
||||
cmd = ["sh", str(ROOT / "scripts" / "artifacts.sh"), "identity", artifact]
|
||||
if channel is not None:
|
||||
cmd.append(channel)
|
||||
return subprocess.run(cmd, capture_output=True, text=True, cwd=ROOT)
|
||||
|
||||
|
||||
def ok(artifact: str, channel: str | None = None) -> str:
|
||||
proc = identity(artifact, channel)
|
||||
assert proc.returncode == 0, f"identity {artifact} {channel}: {proc.stderr}"
|
||||
return proc.stdout.strip()
|
||||
|
||||
|
||||
def bakes_the_channel(artifact: str) -> bool:
|
||||
return bool(_ARG_CHANNEL.search((ROOT / IMAGE_ARTIFACTS[artifact]).read_text()))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("artifact", sorted(IMAGE_ARTIFACTS))
|
||||
def test_channel_dependence_matches_the_dockerfile(artifact):
|
||||
"""The coarse direction, caught at its source.
|
||||
|
||||
Whether the channel belongs in the identity is not a preference — it is
|
||||
dictated by whether the Dockerfile takes it as a build-arg. Adding an
|
||||
`ARG FC_CHANNEL` to another image without adding it to `CHANNELLED` would
|
||||
make its dev and main builds collide, and nothing else would notice.
|
||||
"""
|
||||
per_channel = {c: ok(artifact, c) for c in CHANNELS}
|
||||
differs = len(set(per_channel.values())) > 1
|
||||
|
||||
if bakes_the_channel(artifact):
|
||||
assert differs, (
|
||||
f"{IMAGE_ARTIFACTS[artifact]} declares ARG FC_CHANNEL, so a dev "
|
||||
f"build and a main build of one revision are different images — "
|
||||
f"but both derive the identity {per_channel['main']!r}. The main "
|
||||
f"build would reuse the dev image and report the wrong channel. "
|
||||
f"Add {artifact!r} to CHANNELLED in scripts/artifacts.sh."
|
||||
)
|
||||
else:
|
||||
assert not differs, (
|
||||
f"{IMAGE_ARTIFACTS[artifact]} takes no channel build-arg, so one "
|
||||
f"revision is one image and a merge to main should reuse what dev "
|
||||
f"already built — but the identity differs per channel "
|
||||
f"({per_channel}), so every merge rebuilds it for nothing. Remove "
|
||||
f"{artifact!r} from CHANNELLED in scripts/artifacts.sh."
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("artifact", sorted(IMAGE_ARTIFACTS))
|
||||
def test_identity_tracks_the_artifacts_own_revision(artifact):
|
||||
"""The fine direction: the identity must be the revision, not the push.
|
||||
|
||||
`revision` is the commit this artifact's shipped files last changed in, so
|
||||
it holds still across pushes that did not touch it. Anything derived from
|
||||
HEAD instead would move every push and never hit the registry.
|
||||
"""
|
||||
rev = subprocess.run(
|
||||
def revision(artifact: str) -> str:
|
||||
return subprocess.run(
|
||||
["sh", str(ROOT / "scripts" / "artifacts.sh"), "revision", artifact],
|
||||
capture_output=True, text=True, check=True, cwd=ROOT,
|
||||
).stdout.strip()
|
||||
value = ok(artifact, "main")
|
||||
assert rev and rev in value, (
|
||||
f"identity {value!r} does not contain the {artifact} revision {rev!r}"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("artifact", ARTIFACTS)
|
||||
def test_revision_is_the_commit_its_own_shipped_files_last_changed_in(artifact):
|
||||
"""The claim the whole skip decision rests on.
|
||||
|
||||
Asked of git directly rather than of the script, so this fails if the
|
||||
derivation ever stops meaning what it says — deriving from HEAD, from a
|
||||
build clock, or from a path set it did not actually use.
|
||||
"""
|
||||
paths = declared_paths(artifact)
|
||||
expected = subprocess.run(
|
||||
["git", "log", "--format=%H", "-1", "HEAD", "--", *paths],
|
||||
capture_output=True, text=True, check=True, cwd=ROOT,
|
||||
).stdout.strip()
|
||||
|
||||
assert expected, (
|
||||
f"no commit in this history touches the {artifact} path set — the "
|
||||
f"derivation has nothing to stand on"
|
||||
)
|
||||
assert expected.startswith(revision(artifact)), (
|
||||
f"{artifact} derives {revision(artifact)!r}, but the newest commit "
|
||||
f"touching its shipped files is {expected[:12]!r}. The label stamped "
|
||||
f"into the image would not identify its own content."
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("artifact", sorted(IMAGE_ARTIFACTS))
|
||||
def test_identity_is_a_legal_docker_tag(artifact):
|
||||
"""It is pushed as a tag, so an illegal one fails at the registry — after
|
||||
the build has already run."""
|
||||
for channel in CHANNELS:
|
||||
value = ok(artifact, channel)
|
||||
assert _TAG.match(value), f"{value!r} is not a valid docker tag"
|
||||
@pytest.mark.parametrize("artifact", ARTIFACTS)
|
||||
def test_revision_is_a_legal_label_value_and_is_stable(artifact):
|
||||
"""It is stamped as a docker label and compared for string equality, so a
|
||||
stray newline or a varying value breaks the comparison rather than the
|
||||
build — the mechanism would simply stop hitting, silently."""
|
||||
first = revision(artifact)
|
||||
assert _REVISION.match(first), f"{first!r} is not a 12-char hex revision"
|
||||
assert first == revision(artifact), "revision is not stable across calls"
|
||||
|
||||
|
||||
def test_a_channelled_artifact_refuses_an_unqualified_identity():
|
||||
"""Refusing beats defaulting. If `identity web` quietly returned the
|
||||
unqualified `r-<rev>`, a workflow that forgot to pass the channel would
|
||||
publish one image under a name both channels then reuse — the exact
|
||||
collision the CHANNELLED list exists to prevent, reintroduced by an
|
||||
omission rather than by an edit."""
|
||||
proc = identity("web")
|
||||
assert proc.returncode != 0, (
|
||||
"identity web returned a value with no channel: "
|
||||
f"{proc.stdout.strip()!r}"
|
||||
)
|
||||
def test_an_artifact_whose_paths_did_not_change_keeps_its_revision():
|
||||
"""The property that makes skipping possible at all.
|
||||
|
||||
|
||||
def test_the_extension_has_no_image_identity():
|
||||
"""It is cached as an ext-<version> release asset, and its cache key is the
|
||||
version. Answering with a plausible image tag would invite a second,
|
||||
divergent cache."""
|
||||
proc = identity("extension", "main")
|
||||
assert proc.returncode != 0
|
||||
assert "ext-" in proc.stderr
|
||||
The agent's set is disjoint from web's, so the two must be free to differ.
|
||||
Asserting they *are* different today would pin an accident of history —
|
||||
what matters is that the derivation is per-artifact rather than global, so
|
||||
this asserts each artifact's revision is drawn from its own path set.
|
||||
"""
|
||||
seen = {a: revision(a) for a in ARTIFACTS}
|
||||
for artifact, rev in seen.items():
|
||||
touched = subprocess.run(
|
||||
["git", "log", "--format=%H", "-1", "HEAD", "--", *declared_paths(artifact)],
|
||||
capture_output=True, text=True, check=True, cwd=ROOT,
|
||||
).stdout.strip()
|
||||
assert touched.startswith(rev), (
|
||||
f"{artifact}'s revision {rev!r} is not the newest commit touching "
|
||||
f"its own paths — the derivation is not per-artifact"
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user