CI / extension-version (push) Successful in 4s
CI / lint (push) Failing after 4s
Build images / sign-extension (push) Successful in 4s
CI / backend-lint-and-test (push) Failing after 13s
CI / frontend-build (push) Successful in 20s
extension / lint (push) Successful in 22s
CI / integration (push) Failing after 2m24s
Build images / build-web (push) Successful in 2m44s
Build images / build-ml (push) Successful in 3m13s
Build images / build-agent (push) Successful in 8m56s
The shadow (dee93fa, run 4732) answered the gate: `imagetools inspect
--format` reads `.Image.Config.Labels` against this registry on buildx
v0.36.1. So the reuse check now asks the moving channel tag whether the image
it already points at carries this commit's `fc.revision`, and the r-<rev>
identity tags stop being published.
Three things this removes rather than manages:
A name minted per build that one thing read. Rule 145's narrowing is aimed
exactly there — "a third name for the same thing is upkeep for a model we do
not run."
The -main/-dev qualifier, and the CHANNELLED list behind it. Which tag you
inspect IS the channel, so the distinction has nowhere to live. cmd_identity
goes with it.
A silent expiry nobody wrote down. r-<rev> matches no branch of the
registry's keep_pattern (#3157), so identity tags were prunable past the
newest 10 — a pruned one costs a rebuild, in the safe direction and entirely
invisibly. A label rides inside a tag that has to exist anyway.
It also dissolves #3154 instead of deferring it: a scheduled base refresh
rebuilds :latest with the same revision label, the next unrelated push sees a
match and skips, and the refreshed base survives. Under the tag scheme that
push repointed :latest back to the older base.
The measured detail that shapes the code: a missing label returns an EMPTY
STRING and exits 0. Branching on the exit code would read "no label yet" as
success and skip a build that was needed. So it compares values, and every
uncertain case — absent label, unreachable tag, older image — lands as empty,
never equals a 12-char revision, and falls through to a build.
Reading the specific key matters too. The map carries the base image's labels,
and org.opencontainers.image.version sits right beside ours reading 24.04 on
the agent — a plausible-looking wrong answer.
Expect every artifact to rebuild once on this push: nothing carries a label
yet and it cannot be backfilled, since the reuse path copies a manifest and
config labels are not manifest annotations. One rebuild per artifact, ever,
self-healing after.
test_artifact_identity.py is rewritten around what is now load-bearing. The
CHANNELLED drift test had nothing left to guard; in its place the revision is
asked of git directly, so the file fails if the derivation ever stops being
"the commit this artifact's own shipped files last changed in".
161 lines
6.7 KiB
Bash
Executable File
161 lines
6.7 KiB
Bash
Executable File
#!/bin/sh
|
|
# Single definition of WHAT EACH PUBLISHED ARTIFACT IS BUILT FROM, and the
|
|
# version derived from it. Milestone 313; generalises the shape
|
|
# extension/scripts/packaging.sh established for the extension alone.
|
|
#
|
|
# Four artifacts, four independent versions. An artifact whose shipped files
|
|
# did not change keeps its version and does not rebuild — that is the whole
|
|
# point, and it is why each path set must match its Dockerfile rather than
|
|
# being a plausible guess. Getting a set wrong is quiet in BOTH directions:
|
|
#
|
|
# too narrow -> a pin serves stale bytes, because the version did not move
|
|
# when the content did. This is the dangerous one.
|
|
# too wide -> the artifact re-versions and rebuilds for a change it does
|
|
# not ship. Merely wasteful.
|
|
#
|
|
# tests/test_artifact_paths.py asserts every COPY source in each Dockerfile is
|
|
# covered here, so adding a COPY without updating this file fails CI.
|
|
#
|
|
# POSIX sh only — CI's run shell is busybox on some paths.
|
|
#
|
|
# -f (no pathname expansion) is load-bearing for the whole script: the lists
|
|
# below are iterated with deliberate word-splitting, and without it the shell
|
|
# would glob `frontend/test/**` against the working tree and silently narrow
|
|
# the pattern. Callers substituting the output need their own `set -f` too;
|
|
# the two guards protect different expansions.
|
|
set -euf
|
|
|
|
ROOT=$(git rev-parse --show-toplevel)
|
|
|
|
# --- what each artifact ships ------------------------------------------------
|
|
#
|
|
# Each set includes its own Dockerfile and requirements: changing a base image
|
|
# or a pin changes the artifact just as surely as changing a source file.
|
|
#
|
|
# web (Dockerfile, context `.`) — the runtime stage copies backend/, alembic/,
|
|
# alembic.ini, entrypoint.sh and requirements.txt; the frontend-builder stage
|
|
# copies frontend/ and the runtime takes its `dist` output.
|
|
#
|
|
# frontend/test is excluded: `npm run build` is vite, which builds from src/,
|
|
# index.html and public/ and never reads test/. It lands in the builder layer
|
|
# but not in `dist`, so it cannot reach the shipped image.
|
|
#
|
|
# The web image ALSO bundles the signed XPI (build.yml downloads it into
|
|
# frontend/public/extension/ before the docker build), so an extension change
|
|
# changes the web image. The extension's packaged set is appended in cmd_paths
|
|
# rather than restated — one definition, per #2397.
|
|
WEB_PATHS='Dockerfile requirements.txt backend alembic alembic.ini entrypoint.sh frontend :(exclude)frontend/test :(exclude)frontend/test/**'
|
|
|
|
# ml (Dockerfile.ml, context `.`) — no frontend, no extension. Note it copies
|
|
# BOTH requirements-ml.txt and requirements.txt.
|
|
ML_PATHS='Dockerfile.ml requirements-ml.txt requirements.txt backend alembic alembic.ini entrypoint.sh'
|
|
|
|
# agent (agent/Dockerfile, context `agent`) — copies requirements.txt and
|
|
# fc_agent only. agent/README.md, agent/docker-compose.yml and agent/ruff.toml
|
|
# live in the directory but never reach the image, so they must not re-version
|
|
# it: this is deliberately NOT `agent/`.
|
|
AGENT_PATHS='agent/Dockerfile agent/requirements.txt agent/fc_agent'
|
|
|
|
|
|
usage() {
|
|
echo "usage: artifacts.sh {paths|revision|version|tag} {web|ml|agent|extension}" >&2
|
|
exit 2
|
|
}
|
|
|
|
# The extension's packaged set, read from its own definition rather than
|
|
# copied. packaging.sh emits `:(exclude)extension/...` entries, so the bare
|
|
# `extension` include has to come with them.
|
|
ext_paths() {
|
|
echo "extension $(sh "$ROOT/extension/scripts/packaging.sh" pathspec)"
|
|
}
|
|
|
|
cmd_paths() {
|
|
case "$1" in
|
|
web) echo "$WEB_PATHS $(ext_paths)" ;;
|
|
ml) echo "$ML_PATHS" ;;
|
|
agent) echo "$AGENT_PATHS" ;;
|
|
extension) ext_paths ;;
|
|
*) usage ;;
|
|
esac
|
|
}
|
|
|
|
# "<unix ts> <sha>" of the newest commit touching this artifact's shipped set.
|
|
# Unquoted on purpose: the pathspec must word-split into separate args.
|
|
# Globbing is already off script-wide.
|
|
newest() {
|
|
# shellcheck disable=SC2046
|
|
set -- "$(cd "$ROOT" && git log --format='%ct %H' HEAD -- $(cmd_paths "$1") \
|
|
| sort -n | tail -1)"
|
|
if [ -z "$1" ]; then
|
|
echo "artifacts.sh: no commit touches this artifact's shipped files" >&2
|
|
exit 1
|
|
fi
|
|
echo "$1"
|
|
}
|
|
|
|
# Formatted through git rather than date(1): busybox date does not reliably
|
|
# accept `-d @<epoch>`, and git's own --date=format-local is available wherever
|
|
# git is. TZ=UTC so the value does not depend on the runner's timezone.
|
|
fmt() {
|
|
(cd "$ROOT" && TZ=UTC git show -s --format=%cd --date="format-local:$2" "$1")
|
|
}
|
|
|
|
# Leading zeros stripped so every segment is a plain integer — some version
|
|
# validators reject `08`, and a leading zero buys nothing. `0000` (midnight)
|
|
# must survive as `0`, not as the empty string.
|
|
strip0() {
|
|
printf '%s' "$1" | sed -e 's/^0*//' -e 's/^$/0/'
|
|
}
|
|
|
|
# The IDENTITY of an artifact's content: the commit its shipped files last
|
|
# changed in. This is what decides whether a build can be skipped.
|
|
#
|
|
# It is published as the `fc.revision` LABEL on the image itself, and read
|
|
# back off the moving channel tag — not as a tag of its own (milestone 318
|
|
# step 3). A tag would be a name minted per build that only one thing reads,
|
|
# which is what rule 145 narrowed against; it would also be prunable under the
|
|
# registry's keep_pattern (#3157), so the cache would silently expire.
|
|
#
|
|
# A published image with no such label reads as a MISS and rebuilds. That is
|
|
# the migration path, not a fault: `imagetools create` copies a manifest and
|
|
# config labels are not manifest annotations, so the reuse path cannot stamp
|
|
# one and there is nothing to backfill. Each artifact pays one rebuild, once.
|
|
cmd_revision() {
|
|
echo "$(newest "$1")" | cut -d' ' -f2 | cut -c1-12
|
|
}
|
|
|
|
# The ORDERING KEY: full precision, YYYY.M.D.HHMM. Used by the extension,
|
|
# where the value is what Firefox compares to decide whether an update exists
|
|
# — two same-day builds MUST be distinguishable or the second never reaches
|
|
# anyone.
|
|
cmd_version() {
|
|
sha=$(echo "$(newest "$1")" | cut -d' ' -f2)
|
|
printf '%s.%s.%s.%s\n' \
|
|
"$(fmt "$sha" %Y)" \
|
|
"$(strip0 "$(fmt "$sha" %m)")" \
|
|
"$(strip0 "$(fmt "$sha" %d)")" \
|
|
"$(strip0 "$(fmt "$sha" %H%M)")"
|
|
}
|
|
|
|
# The PUBLISHED IMAGE TAG: day precision, YYYY.M.D. Deliberately coarser than
|
|
# the ordering key, per the operator 2026-08-28 — same-day work is not
|
|
# something worth pinning, so a second build the same day replaces the first
|
|
# rather than accumulating a tag nobody would roll back to. Safe only because
|
|
# skip decisions key on cmd_revision, never on this.
|
|
cmd_tag() {
|
|
sha=$(echo "$(newest "$1")" | cut -d' ' -f2)
|
|
printf '%s.%s.%s\n' \
|
|
"$(fmt "$sha" %Y)" \
|
|
"$(strip0 "$(fmt "$sha" %m)")" \
|
|
"$(strip0 "$(fmt "$sha" %d)")"
|
|
}
|
|
|
|
[ $# -ge 2 ] || usage
|
|
case "$1" in
|
|
paths) cmd_paths "$2" ;;
|
|
revision) cmd_revision "$2" ;;
|
|
version) cmd_version "$2" ;;
|
|
tag) cmd_tag "$2" ;;
|
|
*) usage ;;
|
|
esac
|