build: one definition per artifact of what it ships (milestone 313 step 1)

scripts/artifacts.sh generalises what packaging.sh established for the extension: four published artifacts, four path sets, four independent versions derived from the newest commit touching each set. Measured on this commit, and this is the point of the whole thing: web tag=2026.8.27 version=2026.8.27.1547 rev=a7e626a ml tag=2026.8.27 version=2026.8.27.1547 rev=a7e626a agent tag=2026.7.17 version=2026.7.17.1657 rev=57e5243 extension tag=2026.8.27 version=2026.8.27.1547 rev=a7e626a The agent is six weeks behind because agent/fc_agent has not changed since 57e5243. Today it rebuilds and re-tags on every push regardless; from step 4 it will not. Three outputs, because they answer different questions and conflating them is how this goes wrong: tag YYYY.M.D the published image tag. Day precision, per the operator: same-day work is not worth pinning, so a second build that day replaces the first. version YYYY.M.D.HHMM the ordering key. The extension needs this and cannot use `tag`: Firefox compares it to decide whether an update exists, so two same-day builds must be distinguishable or the second hits the ext-<version> cache and ships stale bytes. That is issue #2397's failure mode exactly. revision <sha> content identity. Because `tag` is only day-precise, "does this tag already exist" cannot decide whether a build can be skipped — two different builds legitimately share a tag. Step 4 keys on this instead. Path sets read from the Dockerfiles rather than guessed. Notable calls: - web includes the extension's packaged set, because build.yml bakes the signed XPI into frontend/public/extension/ before the docker build. Miss that and :latest serves a NEW extension under an unchanged web version. - web excludes frontend/test: vite builds from src/, index.html and public/, so a spec change lands in the builder layer but never in dist. - agent is agent/Dockerfile + agent/requirements.txt + agent/fc_agent, NOT agent/. README.md, ruff.toml and docker-compose.yml sit in that directory and never reach the image. - every set includes its own Dockerfile and requirements: a base-image bump changes the artifact as surely as a source edit does. - the extension's set is read from packaging.sh, not restated. One definition, per #2397. tests/test_artifact_paths.py guards both directions of being wrong, since both are silent. Too narrow — a COPY'd file missing from the set — means the version does not move when the content does, and a pin serves stale bytes. Too wide means re-versioning for a change the artifact does not ship. The test parses each Dockerfile's COPY lines and compares them against the declaration, so adding a COPY without updating the set fails the lane. No workflow reads any of this yet. Step 2 shadows it.
2026-08-27 21:36:17 -04:00
parent 0db38cc111
commit cf06c81db9
2 changed files with 288 additions and 0 deletions
@@ -0,0 +1,150 @@
+#!/bin/sh
+# Single definition of WHAT EACH PUBLISHED ARTIFACT IS BUILT FROM, and the
+# version derived from it. Milestone 313; generalises the shape
+# extension/scripts/packaging.sh established for the extension alone.
+#
+# Four artifacts, four independent versions. An artifact whose shipped files
+# did not change keeps its version and does not rebuild — that is the whole
+# point, and it is why each path set must match its Dockerfile rather than
+# being a plausible guess. Getting a set wrong is quiet in BOTH directions:
+#
+#   too narrow -> a pin serves stale bytes, because the version did not move
+#                 when the content did. This is the dangerous one.
+#   too wide   -> the artifact re-versions and rebuilds for a change it does
+#                 not ship. Merely wasteful.
+#
+# tests/test_artifact_paths.py asserts every COPY source in each Dockerfile is
+# covered here, so adding a COPY without updating this file fails CI.
+#
+# POSIX sh only — CI's run shell is busybox on some paths.
+#
+# -f (no pathname expansion) is load-bearing for the whole script: the lists
+# below are iterated with deliberate word-splitting, and without it the shell
+# would glob `frontend/test/**` against the working tree and silently narrow
+# the pattern. Callers substituting the output need their own `set -f` too;
+# the two guards protect different expansions.
+set -euf
+
+ROOT=$(git rev-parse --show-toplevel)
+
+# --- what each artifact ships ------------------------------------------------
+#
+# Each set includes its own Dockerfile and requirements: changing a base image
+# or a pin changes the artifact just as surely as changing a source file.
+#
+# web (Dockerfile, context `.`) — the runtime stage copies backend/, alembic/,
+# alembic.ini, entrypoint.sh and requirements.txt; the frontend-builder stage
+# copies frontend/ and the runtime takes its `dist` output.
+#
+# frontend/test is excluded: `npm run build` is vite, which builds from src/,
+# index.html and public/ and never reads test/. It lands in the builder layer
+# but not in `dist`, so it cannot reach the shipped image.
+#
+# The web image ALSO bundles the signed XPI (build.yml downloads it into
+# frontend/public/extension/ before the docker build), so an extension change
+# changes the web image. The extension's packaged set is appended in cmd_paths
+# rather than restated — one definition, per #2397.
+WEB_PATHS='Dockerfile requirements.txt backend alembic alembic.ini entrypoint.sh frontend :(exclude)frontend/test :(exclude)frontend/test/**'
+
+# ml (Dockerfile.ml, context `.`) — no frontend, no extension. Note it copies
+# BOTH requirements-ml.txt and requirements.txt.
+ML_PATHS='Dockerfile.ml requirements-ml.txt requirements.txt backend alembic alembic.ini entrypoint.sh'
+
+# agent (agent/Dockerfile, context `agent`) — copies requirements.txt and
+# fc_agent only. agent/README.md, agent/docker-compose.yml and agent/ruff.toml
+# live in the directory but never reach the image, so they must not re-version
+# it: this is deliberately NOT `agent/`.
+AGENT_PATHS='agent/Dockerfile agent/requirements.txt agent/fc_agent'
+
+usage() {
+  echo "usage: artifacts.sh {paths|revision|version|tag} {web|ml|agent|extension}" >&2
+  exit 2
+}
+
+# The extension's packaged set, read from its own definition rather than
+# copied. packaging.sh emits `:(exclude)extension/...` entries, so the bare
+# `extension` include has to come with them.
+ext_paths() {
+  echo "extension $(sh "$ROOT/extension/scripts/packaging.sh" pathspec)"
+}
+
+cmd_paths() {
+  case "$1" in
+    web)       echo "$WEB_PATHS $(ext_paths)" ;;
+    ml)        echo "$ML_PATHS" ;;
+    agent)     echo "$AGENT_PATHS" ;;
+    extension) ext_paths ;;
+    *)         usage ;;
+  esac
+}
+
+# "<unix ts> <sha>" of the newest commit touching this artifact's shipped set.
+# Unquoted on purpose: the pathspec must word-split into separate args.
+# Globbing is already off script-wide.
+newest() {
+  # shellcheck disable=SC2046
+  set -- "$(cd "$ROOT" && git log --format='%ct %H' HEAD -- $(cmd_paths "$1") \
+            | sort -n | tail -1)"
+  if [ -z "$1" ]; then
+    echo "artifacts.sh: no commit touches this artifact's shipped files" >&2
+    exit 1
+  fi
+  echo "$1"
+}
+
+# Formatted through git rather than date(1): busybox date does not reliably
+# accept `-d @<epoch>`, and git's own --date=format-local is available wherever
+# git is. TZ=UTC so the value does not depend on the runner's timezone.
+fmt() {
+  (cd "$ROOT" && TZ=UTC git show -s --format=%cd --date="format-local:$2" "$1")
+}
+
+# Leading zeros stripped so every segment is a plain integer — some version
+# validators reject `08`, and a leading zero buys nothing. `0000` (midnight)
+# must survive as `0`, not as the empty string.
+strip0() {
+  printf '%s' "$1" | sed -e 's/^0*//' -e 's/^$/0/'
+}
+
+# The IDENTITY of an artifact's content: the commit its shipped files last
+# changed in. This — not the tag — is what decides whether a build can be
+# skipped, because the published tag is only day-precise and two different
+# builds can share it.
+cmd_revision() {
+  echo "$(newest "$1")" | cut -d' ' -f2 | cut -c1-12
+}
+
+# The ORDERING KEY: full precision, YYYY.M.D.HHMM. Used by the extension,
+# where the value is what Firefox compares to decide whether an update exists
+# — two same-day builds MUST be distinguishable or the second never reaches
+# anyone.
+cmd_version() {
+  sha=$(echo "$(newest "$1")" | cut -d' ' -f2)
+  printf '%s.%s.%s.%s\n' \
+    "$(fmt "$sha" %Y)" \
+    "$(strip0 "$(fmt "$sha" %m)")" \
+    "$(strip0 "$(fmt "$sha" %d)")" \
+    "$(strip0 "$(fmt "$sha" %H%M)")"
+}
+
+# The PUBLISHED IMAGE TAG: day precision, YYYY.M.D. Deliberately coarser than
+# the ordering key, per the operator 2026-08-28 — same-day work is not
+# something worth pinning, so a second build the same day replaces the first
+# rather than accumulating a tag nobody would roll back to. Safe only because
+# skip decisions key on cmd_revision, never on this.
+cmd_tag() {
+  sha=$(echo "$(newest "$1")" | cut -d' ' -f2)
+  printf '%s.%s.%s\n' \
+    "$(fmt "$sha" %Y)" \
+    "$(strip0 "$(fmt "$sha" %m)")" \
+    "$(strip0 "$(fmt "$sha" %d)")"
+}
+
+[ $# -ge 2 ] || usage
+case "$1" in
+  paths)    cmd_paths "$2" ;;
+  revision) cmd_revision "$2" ;;
+  version)  cmd_version "$2" ;;
+  tag)      cmd_tag "$2" ;;
+  *)        usage ;;
+esac