ci: reuse the published image instead of rebuilding it (milestone 313 step 4)
Build images / sign-extension (push) Successful in 4s
CI / extension-version (push) Successful in 4s
CI / lint (push) Successful in 4s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 17s
extension / lint (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m43s
Build images / sign-extension (push) Successful in 4s
CI / extension-version (push) Successful in 4s
CI / lint (push) Successful in 4s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 17s
extension / lint (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m43s
Before building, each job asks the registry whether this artifact's content is already published. On a hit it skips the build entirely and repoints the channel and date tags at the existing manifest with `imagetools create` — registry-side, no layer transfer, seconds. This is the step that stops a push touching only `agent/` from rebuilding web and ml, and stops a merge to main rebuilding what dev already built. The question is asked with a new `artifacts.sh identity`, not with the date tag: the date tag is day-precise and last-one-wins, so two different builds share it and it cannot answer "is this content published?". The commit sha would move on every push and never hit, which is the redundant rebuild being removed. The revision does both jobs — content-unique, and stable across pushes that did not touch the artifact. Identity is channel-qualified for web and only for web, because web is the only image that takes a build-arg: FC_CHANNEL is baked in and reported by /api/extension/manifest, so its dev and main builds of one revision are genuinely different images. ml and agent take none, which is what lets a merge reuse dev's build rather than rebuilding the agent's CUDA image to produce bytes that already exist. tests/test_artifact_identity.py reads the Dockerfiles and fails if that list drifts from the ARG declarations, in either direction — collapsing the channels ships an instance that reports the wrong one, and splitting them needlessly rebuilds every merge. Failure direction is deliberate: an inspect that errors for any reason reads as a miss and the build runs. Only a real 200 skips one. A tag-push never claims the identity. It rebuilds a revision main already published, and image configs are not bit-reproducible, so re-pushing r-<rev> would point an immutable tag at fresh bytes — rule 145's exact prohibition. It publishes only its own :v... label and otherwise reuses. Base-image freshness, decided rather than left implicit: an artifact whose source stops moving stops picking up base updates under its pinned tag. That is what a pin means, and rule 145 already says the refresh belongs on the moving tag instead. Filed as #3154 rather than folded in here, because the naive version regresses :latest on the next unrelated push. ci.yml's backend lane gains fetch-depth: 0 — the new tests derive real revisions, and on a depth-1 clone that derivation returns the tip sha or fails, so the lane would go green while asserting nothing. The three build jobs' shadow steps are renamed and re-commented: those values stopped being informational at step 3, and a step captioned "nothing reads this" beside steps that do is worse than no caption.
This commit is contained in:
@@ -56,8 +56,25 @@ ML_PATHS='Dockerfile.ml requirements-ml.txt requirements.txt backend alembic ale
|
||||
# it: this is deliberately NOT `agent/`.
|
||||
AGENT_PATHS='agent/Dockerfile agent/requirements.txt agent/fc_agent'
|
||||
|
||||
# Which artifacts bake the BUILD CHANNEL into the image, and therefore cannot
|
||||
# share a content identity across channels. The web image takes FC_CHANNEL as
|
||||
# a build-arg and reports it from /api/extension/manifest (milestone 271 step
|
||||
# 7), so `main` and `dev` builds of one revision are genuinely different
|
||||
# images — reusing the dev one on main would ship an instance that names
|
||||
# itself `dev` forever.
|
||||
#
|
||||
# ml and agent take no build-args at all: one revision, one image, and a merge
|
||||
# to main can reuse exactly what dev already built. That is not a detail, it is
|
||||
# most of what step 4 saves — merges would otherwise rebuild the agent's CUDA
|
||||
# image to produce bytes that already exist.
|
||||
#
|
||||
# Extend this list if a second artifact ever gains a build-arg;
|
||||
# tests/test_artifact_identity.py reads the Dockerfiles and fails if it drifts.
|
||||
CHANNELLED='web'
|
||||
|
||||
usage() {
|
||||
echo "usage: artifacts.sh {paths|revision|version|tag} {web|ml|agent|extension}" >&2
|
||||
echo " artifacts.sh identity {web|ml|agent} [channel]" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
@@ -140,11 +157,50 @@ cmd_tag() {
|
||||
"$(strip0 "$(fmt "$sha" %d)")"
|
||||
}
|
||||
|
||||
# The CONTENT IDENTITY of a published image: an immutable tag naming exactly
|
||||
# what a build of this commit would produce. build.yml asks the registry for it
|
||||
# and, on a hit, skips the build entirely and repoints the channel and date
|
||||
# tags at the manifest that is already there (milestone 313 step 4).
|
||||
#
|
||||
# It is deliberately NOT either of the other two values:
|
||||
# * the date tag is day-precise and last-one-wins, so two different builds
|
||||
# share it — it cannot answer "is this content published?".
|
||||
# * the commit sha moves on every push, so it would never hit, which is the
|
||||
# redundant rebuild this exists to remove.
|
||||
#
|
||||
# The revision does both jobs: it is content-unique AND stable across pushes
|
||||
# that did not touch the artifact.
|
||||
cmd_identity() {
|
||||
_art=$1
|
||||
_chan=${2:-}
|
||||
case "$_art" in
|
||||
web|ml|agent) ;;
|
||||
extension)
|
||||
echo "artifacts.sh: the extension is cached as an ext-<version> Forgejo release, not an image tag — use \`version\`" >&2
|
||||
exit 2 ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
for _c in $CHANNELLED; do
|
||||
if [ "$_art" = "$_c" ]; then
|
||||
# Refused rather than defaulted: an unqualified identity for a
|
||||
# channelled artifact would let a dev image be reused as the main one.
|
||||
if [ -z "$_chan" ]; then
|
||||
echo "artifacts.sh: $_art bakes the channel into the image — identity needs one" >&2
|
||||
exit 2
|
||||
fi
|
||||
printf 'r-%s-%s\n' "$(cmd_revision "$_art")" "$_chan"
|
||||
return
|
||||
fi
|
||||
done
|
||||
printf 'r-%s\n' "$(cmd_revision "$_art")"
|
||||
}
|
||||
|
||||
[ $# -ge 2 ] || usage
|
||||
case "$1" in
|
||||
paths) cmd_paths "$2" ;;
|
||||
revision) cmd_revision "$2" ;;
|
||||
version) cmd_version "$2" ;;
|
||||
tag) cmd_tag "$2" ;;
|
||||
identity) cmd_identity "$2" "${3:-}" ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
|
||||
Reference in New Issue
Block a user