ci: registry-backed layer cache for all three images (milestone 326 step 2)
Build images / sign-extension (push) Successful in 4s
CI / lint (push) Successful in 5s
CI / extension-version (push) Successful in 5s
Build images / build-ml (push) Successful in 8s
Build images / build-agent (push) Successful in 9s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 17s
extension / lint (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m44s
extension / lint (pull_request) Successful in 29s
Build images / sign-extension (push) Successful in 4s
CI / lint (push) Successful in 5s
CI / extension-version (push) Successful in 5s
Build images / build-ml (push) Successful in 8s
Build images / build-agent (push) Successful in 9s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 17s
extension / lint (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m44s
extension / lint (pull_request) Successful in 29s
`cache-from`/`cache-to` on `<image>:buildcache`, `mode=max`, on all three
build steps. Closes the half of the driver change that step 1 left open.
Step 1 measured worse, not better, and that was expected but is worth stating
with numbers. Run 4896, first builds after moving to `docker-container`:
build-web 3m44s (cold baseline 2m23s)
build-ml 3m49s (cold baseline 3m20s)
build-agent 11m12s (cold baseline 9m26s)
The container driver gets a FRESH buildkit instance per job, so it has no
local layer store to fall back on — where the old docker driver at least
reused whatever the runner's dockerd happened to hold. That is why a registry
cache is the only cache this driver can have, and why step 1 on its own is a
regression rather than a win.
`mode=max` so intermediate stages cache too. The agent's two ~150s pip layers
and web's frontend-builder stage are the entire cost, and a min-mode cache
would drop exactly those.
A `:buildcache` tag is not the withdrawn tag scheme returning. Rule 145
narrowed against names NOTHING reads; this one is read by every build that
runs, is one moving ref per image rather than one per build, holds cache blobs
rather than a shippable artifact, and is overwritten in place rather than
accumulating. Closer to `:dev` than to the `:2026.8.28` tags 318 deleted — and
said so in the workflow, so it is not "cleaned up" by a later reader.
Expect the next build to be slower again, once: it is still cold AND now pays
the cache export. The measurement that matters is the one after that.
Scribe #3114.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -688,6 +688,25 @@ jobs:
|
||||
# same shape for `platforms:`.
|
||||
provenance: false
|
||||
sbom: false
|
||||
# The ONLY cache this driver can have. `docker-container` gets a
|
||||
# FRESH buildkit instance per job, so unlike the default docker
|
||||
# driver it has no local layer store to fall back on — measured on
|
||||
# run 4896, the first builds after the driver change: web 3m44s
|
||||
# (was 2m23s), ml 3m49s (was 3m20s), agent 11m12s (was 9m26s). The
|
||||
# driver change ALONE is a regression; this is the other half of it.
|
||||
#
|
||||
# mode=max so intermediate stages cache too. web's frontend-builder
|
||||
# stage and the agent's two ~150s pip layers are the whole cost, and
|
||||
# they are exactly what a min-mode cache would drop.
|
||||
#
|
||||
# A `:buildcache` tag is NOT the withdrawn tag scheme coming back.
|
||||
# Rule 145 narrowed against names NOTHING reads; this one is read by
|
||||
# every build that runs, is one moving ref per image rather than one
|
||||
# per build, holds cache blobs rather than a shippable artifact, and
|
||||
# is overwritten in place rather than accumulating. It is closer to
|
||||
# :dev than to the :2026.8.28 tags milestone 318 deleted. (#3114.)
|
||||
cache-from: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator:buildcache
|
||||
cache-to: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator:buildcache,mode=max
|
||||
# Only the web image carries these: it is the one with a UI and an
|
||||
# HTTP surface to report them on. The ml and agent images have
|
||||
# nothing to tell.
|
||||
@@ -986,6 +1005,25 @@ jobs:
|
||||
# same shape for `platforms:`.
|
||||
provenance: false
|
||||
sbom: false
|
||||
# The ONLY cache this driver can have. `docker-container` gets a
|
||||
# FRESH buildkit instance per job, so unlike the default docker
|
||||
# driver it has no local layer store to fall back on — measured on
|
||||
# run 4896, the first builds after the driver change: web 3m44s
|
||||
# (was 2m23s), ml 3m49s (was 3m20s), agent 11m12s (was 9m26s). The
|
||||
# driver change ALONE is a regression; this is the other half of it.
|
||||
#
|
||||
# mode=max so intermediate stages cache too. web's frontend-builder
|
||||
# stage and the agent's two ~150s pip layers are the whole cost, and
|
||||
# they are exactly what a min-mode cache would drop.
|
||||
#
|
||||
# A `:buildcache` tag is NOT the withdrawn tag scheme coming back.
|
||||
# Rule 145 narrowed against names NOTHING reads; this one is read by
|
||||
# every build that runs, is one moving ref per image rather than one
|
||||
# per build, holds cache blobs rather than a shippable artifact, and
|
||||
# is overwritten in place rather than accumulating. It is closer to
|
||||
# :dev than to the :2026.8.28 tags milestone 318 deleted. (#3114.)
|
||||
cache-from: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-ml:buildcache
|
||||
cache-to: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-ml:buildcache,mode=max
|
||||
|
||||
# Every tag but the channel's own is written HERE, registry-side,
|
||||
# whether or not a build ran. Each -t becomes another reference to the
|
||||
@@ -1276,6 +1314,25 @@ jobs:
|
||||
# same shape for `platforms:`.
|
||||
provenance: false
|
||||
sbom: false
|
||||
# The ONLY cache this driver can have. `docker-container` gets a
|
||||
# FRESH buildkit instance per job, so unlike the default docker
|
||||
# driver it has no local layer store to fall back on — measured on
|
||||
# run 4896, the first builds after the driver change: web 3m44s
|
||||
# (was 2m23s), ml 3m49s (was 3m20s), agent 11m12s (was 9m26s). The
|
||||
# driver change ALONE is a regression; this is the other half of it.
|
||||
#
|
||||
# mode=max so intermediate stages cache too. web's frontend-builder
|
||||
# stage and the agent's two ~150s pip layers are the whole cost, and
|
||||
# they are exactly what a min-mode cache would drop.
|
||||
#
|
||||
# A `:buildcache` tag is NOT the withdrawn tag scheme coming back.
|
||||
# Rule 145 narrowed against names NOTHING reads; this one is read by
|
||||
# every build that runs, is one moving ref per image rather than one
|
||||
# per build, holds cache blobs rather than a shippable artifact, and
|
||||
# is overwritten in place rather than accumulating. It is closer to
|
||||
# :dev than to the :2026.8.28 tags milestone 318 deleted. (#3114.)
|
||||
cache-from: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-agent:buildcache
|
||||
cache-to: type=registry,ref=git.fabledsword.com/bvandeusen/fabledcurator-agent:buildcache,mode=max
|
||||
|
||||
# Every tag but the channel's own is written HERE, registry-side,
|
||||
# whether or not a build ran. Each -t becomes another reference to the
|
||||
|
||||
@@ -146,6 +146,17 @@ per `docs/process.md`'s "add deps to the image when used by >1 project".
|
||||
rebuild forever with every lane green. Same failure as #3183, different door.
|
||||
These jobs run inside a container against a mounted docker socket, so the
|
||||
buildkit container is a sibling rather than a child.
|
||||
- **All three images import and export a registry layer cache**
|
||||
(`<image>:buildcache`, `mode=max`). This is not an optimisation bolted onto
|
||||
the driver change — it is the other half of it. The `docker-container`
|
||||
driver gets a fresh buildkit instance per job and therefore has **no local
|
||||
layer store at all**, where the old `docker` driver at least reused whatever
|
||||
the runner's dockerd happened to hold. Measured on run 4896, the first builds
|
||||
after the driver moved: web 3m44s (was 2m23s), ml 3m49s (was 3m20s), agent
|
||||
11m12s (was 9m26s) — every one slower. A `:buildcache` tag is read by every
|
||||
build that runs, is one moving ref per image, holds cache blobs rather than a
|
||||
shippable artifact, and is overwritten in place, so it is not a return of the
|
||||
per-version tags milestone 318 withdrew (#3114).
|
||||
- **`build.yml` accepts a `workflow_dispatch` with `force_build`**, which
|
||||
bypasses the reuse check for all three images. It exists because
|
||||
skip-if-exists makes its own build path untestable: `agent/` has not changed
|
||||
|
||||
Reference in New Issue
Block a user