ci: a weekly base-image refresh on the channel tags (milestone 326 step 4)
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
CI / extension-version (push) Successful in 3s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 6s
extension / lint (push) Successful in 20s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m50s

Skip-if-exists is keyed on our own source, so an artifact whose source
stops moving stops picking up base-image updates. `agent/` last changed
2026-07-17; every push since has correctly declined to rebuild it, which
also means it will serve that day's nvidia/cuda layers indefinitely.

A `schedule:` trigger, Sunday 06:00 UTC, away from CI-runner's Monday
security sweep so the two are never diagnosing each other.

#3154's blocking open question is dissolved rather than answered. It was
written when the identity was a `r-<revision>` TAG, and asked how the
next ordinary push could avoid repointing :latest back off the refresh.
Milestone 318 replaced that tag with a LABEL, and #3183 made the repoint
step exclude its source tag so the label stays readable. Excluding the
source is what also keeps a refresh from being undone: on the next main
push the reuse check hits, :latest is not rewritten, and the new :c-<sha>
is written FROM the refreshed :latest. To be verified by digest, not by
this argument.

Four decisions, each commented where it lives:

* It builds `main`, not the branch that triggered it. Forgejo registers a
  cron from the default branch — `dev` here — so a scheduled run arrives
  with github.ref on dev, and a refresh of :dev would be refreshing the
  one channel that is rebuilt constantly anyway. The ref is decided once
  in a top-level `env: BUILD_REF` that all four checkouts take. Deriving
  it per job would let the halves disagree: sign-extension would derive
  dev's extension version while build-web bundled main's, and the release
  download would 404 on a version that exists perfectly well.

* It publishes only the channel tag. :c-<sha> for main's HEAD already
  names the bytes that commit built; re-pushing it over refreshed layers
  would break the one tag rule 145 makes immutable, and it is the
  rollback unit — so the breakage would surface on the day somebody
  needed it. The repoint step needs no schedule case: the tag list is the
  channel tag alone, SOURCE is the only entry, it is excluded as always,
  and the step correctly does nothing.

* It bypasses reuse by construction, since it rebuilds the same source
  and fc.revision always matches. Checked in the reuse step beside
  force_build, so one decision still drives both the build and the
  repoint.

* `pull: true`, on the scheduled path only, is the actual mechanism. A
  moved base tag changes the FROM layer's cache key and everything above
  it rebuilds; an unmoved one is satisfied by the registry cache and the
  refresh is a ~13s no-op that republishes nothing. That no-op is the
  point — :latest should change when there is something new in it, not
  every Sunday. The known lag, left deliberately: an apt package update
  while the base tag stands still is not caught, and closing it needs
  no-cache: true, which buys weekly churn for it.
This commit is contained in:
2026-08-29 22:53:48 -04:00
parent 5e72076298
commit 63e0a423d7
2 changed files with 216 additions and 6 deletions
+35
View File
@@ -167,6 +167,41 @@ per `docs/process.md`'s "add deps to the image when used by >1 project".
`github.event.inputs` into an env var rather than interpolated into a run
block, and it is checked inside the reuse step so that one decision drives
both the build and the repoint.
- **A weekly `schedule` rebuilds all three images against fresh base layers**
(Sunday 06:00 UTC, milestone 326 step 4, #3154). Skip-if-exists is keyed on
OUR source, so an artifact whose source stops moving stops picking up base
updates — `agent/` has not changed since 2026-07-17 and would otherwise serve
that day's `nvidia/cuda` layers forever. Four things make it work:
- It **builds `main`, not the branch that triggered it.** Forgejo registers a
cron from the DEFAULT branch (`dev` here), so a scheduled run arrives with
`github.ref` on dev. The ref is decided once in a top-level `env:
BUILD_REF` that every checkout in the file takes, rather than per job —
otherwise `sign-extension` would derive dev's extension version while
`build-web` bundled main's, and the release download would 404 on a version
that exists perfectly well.
- It **publishes only `:latest`.** `:c-<sha>` for main's HEAD already names
the bytes that commit built; re-pushing it over refreshed layers would
break the one tag rule 145 makes immutable, and it is the rollback unit.
The repoint step needs no schedule case for this — the tag list is the
channel tag alone, so SOURCE is the only entry, it is excluded as always,
and the step correctly does nothing.
- **`:latest` and `:c-<sha>` therefore diverge between a refresh and the next
`main` push, by design.** They re-converge on that push: it hits reuse (a
refresh does not move `fc.revision`, because it does not touch the source),
and the repoint writes the NEW `:c-<sha>` from the refreshed `:latest`. The
push path needed no change for this, because the repoint already excluded
the source tag — the same rule that keeps the label readable also keeps a
refresh from being undone.
- **`pull: true` on the scheduled path only** is the actual mechanism. If a
base tag moved, the `FROM` layer's cache key changes and everything above
it rebuilds; if it did not, the registry cache satisfies the whole graph
and the refresh is a ~13s no-op that republishes nothing. That no-op is the
point — `:latest` should change when there is something new in it, not
every Sunday. The known lag: a Debian package update inside the `apt-get
install` layer while the base tag stands still is not caught. Closing it
needs `no-cache: true`, which buys weekly churn for it; the official
python/cuda images rebuild with those updates baked in, so this is a lag
rather than a hole.
- **`FC_CHANNEL` and `FC_VERSION` are build args, not runtime settings.**
`build.yml` passes them to the web image only — the ml and agent images have
nothing to report them to. `/api/health` returns both, the foot of Settings