Per-artifact commit-derived versions: pinnable date tags, and no rebuild when nothing changed #238

Merged

bvandeusen merged 5 commits from dev into main 2026-08-28 10:14:15 -04:00

Conversation 0 Commits 5 Files Changed 7

+1106 -79

5 Commits

Author	SHA1	Message	Date
bvandeusen	fb2c4d5b80	fix(extension): packaging.sh decides the version, so it must move the version ... `packaging.sh pathspec` excluded `extension/scripts/**` — the same list web-ext ignores. But the two lists answer different questions, and this is the one place they disagree. packaging.sh is not packaged into the XPI. It does decide the version string, and build.yml stamps that string into the manifest.json that IS packaged. Changing how the version is computed therefore changes the shipped bytes, and the derivation has to see it. Harmless while every push rebuilt the web image. Step 4 made the rebuild conditional on the derived revision moving, which turns it into a silent failure: a packaging.sh change yields a NEW version, so sign-extension misses its ext-<version> cache and signs — while build-web sees an unmoved revision, reuses the published image and ships the OLD XPI. One orphaned AMO signature, and an instance serving code the registry calls current. Found while checking the ground under step 5, which changes the version format and is exactly the commit that would have hit it. Split the list rather than widening the shared one: NOT_VERSION_RELEVANT drives the pathspec, NOT_PACKAGED_TRACKED still drives web-ext's ignore list, and scripts/ stays out of the XPI. The two directions are not symmetric, which is why the version list is the narrower one — too wide costs a re-sign and a rebuild for a change that ships nothing new, too narrow serves stale bytes and says nothing. No version churn: the last packaging.sh commit predates the current extension revision, so the derived version is unchanged at 1.0.3500147 and web's revision stays a7e626a67a. Both suites now pin the disagreement from their own side, because the tempting fix for either half is to make the lists one again — version.spec.js asserts the pathspec does NOT exclude scripts while the ignore list still does, and test_artifact_paths.py asserts packaging.sh is inside the extension and web path sets.	2026-08-28 08:49:06 -04:00
bvandeusen	609bc82acc	ci: reuse the published image instead of rebuilding it (milestone 313 step 4) ... Before building, each job asks the registry whether this artifact's content is already published. On a hit it skips the build entirely and repoints the channel and date tags at the existing manifest with `imagetools create` — registry-side, no layer transfer, seconds. This is the step that stops a push touching only `agent/` from rebuilding web and ml, and stops a merge to main rebuilding what dev already built. The question is asked with a new `artifacts.sh identity`, not with the date tag: the date tag is day-precise and last-one-wins, so two different builds share it and it cannot answer "is this content published?". The commit sha would move on every push and never hit, which is the redundant rebuild being removed. The revision does both jobs — content-unique, and stable across pushes that did not touch the artifact. Identity is channel-qualified for web and only for web, because web is the only image that takes a build-arg: FC_CHANNEL is baked in and reported by /api/extension/manifest, so its dev and main builds of one revision are genuinely different images. ml and agent take none, which is what lets a merge reuse dev's build rather than rebuilding the agent's CUDA image to produce bytes that already exist. tests/test_artifact_identity.py reads the Dockerfiles and fails if that list drifts from the ARG declarations, in either direction — collapsing the channels ships an instance that reports the wrong one, and splitting them needlessly rebuilds every merge. Failure direction is deliberate: an inspect that errors for any reason reads as a miss and the build runs. Only a real 200 skips one. A tag-push never claims the identity. It rebuilds a revision main already published, and image configs are not bit-reproducible, so re-pushing r-<rev> would point an immutable tag at fresh bytes — rule 145's exact prohibition. It publishes only its own :v... label and otherwise reuses. Base-image freshness, decided rather than left implicit: an artifact whose source stops moving stops picking up base updates under its pinned tag. That is what a pin means, and rule 145 already says the refresh belongs on the moving tag instead. Filed as #3154 rather than folded in here, because the naive version regresses :latest on the next unrelated push. ci.yml's backend lane gains fetch-depth: 0 — the new tests derive real revisions, and on a depth-1 clone that derivation returns the tip sha or fails, so the lane would go green while asserting nothing. The three build jobs' shadow steps are renamed and re-commented: those values stopped being informational at step 3, and a step captioned "nothing reads this" beside steps that do is worse than no caption.	2026-08-28 08:24:58 -04:00
bvandeusen	7a20c55441	ci: publish a per-artifact date tag on main builds (milestone 313 step 3) ... Each main build now also publishes <image>:YYYY.M.D — the date of the commit that artifact's shipped files last changed in. Purely additive; :main, :latest, :c-<sha> and the dev path are untouched, so reverting this commit reverts the behaviour. main push -> :main, :latest, :c-<sha>, :2026.8.27 dev push -> :dev Per artifact, so an image whose files did not change keeps the tag it already had. On this commit the agent reads 2026.7.17 while web and ml read 2026.8.27 — six weeks apart, from one push. Step 4 turns that into not rebuilding it. Day precision, and a second main build the same day replaces the first. Operator's call, and the reasoning is theirs: same-day work is not something worth pinning. A rollback goes to a day, not to the fourth merge of a Tuesday afternoon. It also makes retention mean "the last N days" rather than "the last N pushes". CALVER is computed inside the main branch rather than at the top of the step, and hard-fails when empty. There is no `set -e` here, so an unconditional assignment that failed would have left it empty and published the tag `fabledcurator:` — an invalid name, from a step that still reported success. It is also simply unused on the dev and tag paths. Fixed a stale comment while in this block rather than leaving it for step 7: it claimed release tags carry "no `.N` per family release-posture rule". Rule 148 was amended on 2026-08-24 to REQUIRE the suffix, after the ban caused a same-day tag to be retargeted and a release deleted to make room (note 2813). FC's own tags already carry suffixes; only the comment was asserting the superseded rule. Verified before pushing: the derivation holds across 200 commits of real history — a derived revision always touches its own path set, the version never decreases along any parent->child edge, and web tracks all three extension-only commits in the log. That last one is the direction that would serve stale bytes on a pin.	2026-08-27 21:56:52 -04:00
bvandeusen	0c43fa3eb2	ci: shadow the per-artifact derived versions (milestone 313 step 2) ... Every build job now logs the tag, version and revision its artifact would get. Nothing reads them; no `set -e`, and each derivation falls back to UNAVAILABLE, so a broken script cannot fail a build. Same discipline as milestone 271 step 2, which is what made that cutover safe to do in one commit. Also fixes a landmine the plan named but had not checked: build-ml and build-agent were checking out at depth 1. Both now use fetch-depth: 0. That mattered more than it looks. A depth-1 clone sees one commit, so `git log HEAD -- <shipped paths>` either returns that commit's timestamp — plausible, and wrong — or returns nothing. For build-ml on this push it would have returned today's date, because HEAD touches backend/, and nothing downstream would have questioned it. For build-agent it would have returned nothing at all, since no single commit here touches agent/, and artifacts.sh exits non-zero rather than guessing. One direction is silent and one is loud; only the loud one was ever going to get noticed. What to read from the shadow lines over the next few pushes, in order of how badly each would bite: * a push touching the extension must move BOTH the extension and web, because build-web bakes the XPI in. If web does not move, its path set is too narrow and a pinned web image will serve an extension it does not name. * a push touching only agent/ must leave web and ml still. If they move, their sets are too wide and they will rebuild for changes they do not ship. * a docs-only push must move nothing. * dev and main must derive the same values for the same source. Step 3 only lets these values name a tag once those hold.	2026-08-27 21:41:14 -04:00
bvandeusen	cf06c81db9	build: one definition per artifact of what it ships (milestone 313 step 1) ... scripts/artifacts.sh generalises what packaging.sh established for the extension: four published artifacts, four path sets, four independent versions derived from the newest commit touching each set. Measured on this commit, and this is the point of the whole thing: web tag=2026.8.27 version=2026.8.27.1547 rev=a7e626a ml tag=2026.8.27 version=2026.8.27.1547 rev=a7e626a agent tag=2026.7.17 version=2026.7.17.1657 rev=57e5243 extension tag=2026.8.27 version=2026.8.27.1547 rev=a7e626a The agent is six weeks behind because agent/fc_agent has not changed since 57e5243. Today it rebuilds and re-tags on every push regardless; from step 4 it will not. Three outputs, because they answer different questions and conflating them is how this goes wrong: tag YYYY.M.D the published image tag. Day precision, per the operator: same-day work is not worth pinning, so a second build that day replaces the first. version YYYY.M.D.HHMM the ordering key. The extension needs this and cannot use `tag`: Firefox compares it to decide whether an update exists, so two same-day builds must be distinguishable or the second hits the ext-<version> cache and ships stale bytes. That is issue #2397's failure mode exactly. revision <sha> content identity. Because `tag` is only day-precise, "does this tag already exist" cannot decide whether a build can be skipped — two different builds legitimately share a tag. Step 4 keys on this instead. Path sets read from the Dockerfiles rather than guessed. Notable calls: - web includes the extension's packaged set, because build.yml bakes the signed XPI into frontend/public/extension/ before the docker build. Miss that and :latest serves a NEW extension under an unchanged web version. - web excludes frontend/test: vite builds from src/, index.html and public/, so a spec change lands in the builder layer but never in dist. - agent is agent/Dockerfile + agent/requirements.txt + agent/fc_agent, NOT agent/. README.md, ruff.toml and docker-compose.yml sit in that directory and never reach the image. - every set includes its own Dockerfile and requirements: a base-image bump changes the artifact as surely as a source edit does. - the extension's set is read from packaging.sh, not restated. One definition, per #2397. tests/test_artifact_paths.py guards both directions of being wrong, since both are silent. Too narrow — a COPY'd file missing from the set — means the version does not move when the content does, and a pin serves stale bytes. Too wide means re-versioning for a change the artifact does not ship. The test parses each Dockerfile's COPY lines and compares them against the declaration, so adding a COPY without updating the set fails the lane. No workflow reads any of this yet. Step 2 shadows it.	2026-08-27 21:36:17 -04:00