diff --git a/.forgejo/workflows/build.yml b/.forgejo/workflows/build.yml index 42a9a72..18a45bb 100644 --- a/.forgejo/workflows/build.yml +++ b/.forgejo/workflows/build.yml @@ -443,6 +443,23 @@ jobs: ACTOR: ${{ github.actor }} run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin + # A REAL buildx builder, not the default `docker` driver (#3114, #3190). + # + # The default driver builds through the local dockerd. It cannot export a + # registry cache at all — which is why the agent rebuilds a ~6.3 GB CUDA + # + torch image from scratch whenever the runner's local cache is cold, + # measured at 9m26s against 7s warm. It is also #3190's leading suspect: + # after a registry-direct push it resolves image metadata against a local + # store the push never filled, and reports `No such image` on an image + # that published perfectly well three seconds earlier. + # + # These jobs run INSIDE a container against a mounted docker socket, so + # the buildkit container this starts is a SIBLING of the job container, + # not a child. That works over the socket mount; it had never been tried + # here before milestone 326 step 1. + - name: Set up buildx + uses: docker/setup-buildx-action@v3 + # --- reuse-if-published (milestone 313, step 4) ---------------------- # Does the image the channel tag already points at carry THIS commit's # revision? If so the bytes this job would produce are already published @@ -623,6 +640,21 @@ jobs: # decoration — an unstamped image is one that will always rebuild. labels: | fc.revision=${{ steps.reuse.outputs.revision }} + # LOAD-BEARING, not a preference. On the default docker driver these + # were no-ops; on the docker-container driver above, + # build-push-action@v5 defaults provenance to TRUE when pushing. + # Provenance attaches an attestation manifest, which makes the pushed + # tag a manifest INDEX — and `.Image.Config.Labels` does not resolve + # through an index. + # + # The label directly above IS the reuse key. Wrap the channel tag in + # an index and the next push reads fc.revision=, misses, and + # rebuilds. Then so does the one after that, forever. Nothing fails, + # nothing goes red, and the only symptom is the bill. That is #3183 + # arriving through a different door, and note #3127 §4 records the + # same shape for `platforms:`. + provenance: false + sbom: false # Only the web image carries these: it is the one with a UI and an # HTTP surface to report them on. The ml and agent images have # nothing to tell. @@ -776,6 +808,23 @@ jobs: ACTOR: ${{ github.actor }} run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin + # A REAL buildx builder, not the default `docker` driver (#3114, #3190). + # + # The default driver builds through the local dockerd. It cannot export a + # registry cache at all — which is why the agent rebuilds a ~6.3 GB CUDA + # + torch image from scratch whenever the runner's local cache is cold, + # measured at 9m26s against 7s warm. It is also #3190's leading suspect: + # after a registry-direct push it resolves image metadata against a local + # store the push never filled, and reports `No such image` on an image + # that published perfectly well three seconds earlier. + # + # These jobs run INSIDE a container against a mounted docker socket, so + # the buildkit container this starts is a SIBLING of the job container, + # not a child. That works over the socket mount; it had never been tried + # here before milestone 326 step 1. + - name: Set up buildx + uses: docker/setup-buildx-action@v3 + # --- reuse-if-published (milestone 313, step 4) ---------------------- # Does the image the channel tag already points at carry THIS commit's # revision? If so the bytes this job would produce are already published @@ -876,6 +925,21 @@ jobs: # decoration — an unstamped image is one that will always rebuild. labels: | fc.revision=${{ steps.reuse.outputs.revision }} + # LOAD-BEARING, not a preference. On the default docker driver these + # were no-ops; on the docker-container driver above, + # build-push-action@v5 defaults provenance to TRUE when pushing. + # Provenance attaches an attestation manifest, which makes the pushed + # tag a manifest INDEX — and `.Image.Config.Labels` does not resolve + # through an index. + # + # The label directly above IS the reuse key. Wrap the channel tag in + # an index and the next push reads fc.revision=, misses, and + # rebuilds. Then so does the one after that, forever. Nothing fails, + # nothing goes red, and the only symptom is the bill. That is #3183 + # arriving through a different door, and note #3127 §4 records the + # same shape for `platforms:`. + provenance: false + sbom: false # Every tag but the channel's own is written HERE, registry-side, # whether or not a build ran. Each -t becomes another reference to the @@ -1021,6 +1085,23 @@ jobs: ACTOR: ${{ github.actor }} run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin + # A REAL buildx builder, not the default `docker` driver (#3114, #3190). + # + # The default driver builds through the local dockerd. It cannot export a + # registry cache at all — which is why the agent rebuilds a ~6.3 GB CUDA + # + torch image from scratch whenever the runner's local cache is cold, + # measured at 9m26s against 7s warm. It is also #3190's leading suspect: + # after a registry-direct push it resolves image metadata against a local + # store the push never filled, and reports `No such image` on an image + # that published perfectly well three seconds earlier. + # + # These jobs run INSIDE a container against a mounted docker socket, so + # the buildkit container this starts is a SIBLING of the job container, + # not a child. That works over the socket mount; it had never been tried + # here before milestone 326 step 1. + - name: Set up buildx + uses: docker/setup-buildx-action@v3 + # --- reuse-if-published (milestone 313, step 4) ---------------------- # Does the image the channel tag already points at carry THIS commit's # revision? If so the bytes this job would produce are already published @@ -1121,6 +1202,21 @@ jobs: # decoration — an unstamped image is one that will always rebuild. labels: | fc.revision=${{ steps.reuse.outputs.revision }} + # LOAD-BEARING, not a preference. On the default docker driver these + # were no-ops; on the docker-container driver above, + # build-push-action@v5 defaults provenance to TRUE when pushing. + # Provenance attaches an attestation manifest, which makes the pushed + # tag a manifest INDEX — and `.Image.Config.Labels` does not resolve + # through an index. + # + # The label directly above IS the reuse key. Wrap the channel tag in + # an index and the next push reads fc.revision=, misses, and + # rebuilds. Then so does the one after that, forever. Nothing fails, + # nothing goes red, and the only symptom is the bill. That is #3183 + # arriving through a different door, and note #3127 §4 records the + # same shape for `platforms:`. + provenance: false + sbom: false # Every tag but the channel's own is written HERE, registry-side, # whether or not a build ran. Each -t becomes another reference to the diff --git a/ci-requirements.md b/ci-requirements.md index 2744b40..643cde6 100644 --- a/ci-requirements.md +++ b/ci-requirements.md @@ -133,6 +133,19 @@ per `docs/process.md`'s "add deps to the image when used by >1 project". already ran, so both paths now share one proven route. The cost: `:c-` is an index rather than a plain image, so `fc.revision` does not resolve through it — nothing reads it there, and the index names the same manifest. +- **The image builds run on a `docker-container` buildx builder, and + `provenance`/`sbom` are explicitly OFF** (milestone 326 step 1). The builder + is what makes a registry layer cache possible at all — the default `docker` + driver cannot export one (#3114) — and it is #3190's leading suspect, since + it is the driver that resolves image metadata against a local store a + registry-direct push never fills. **The attestation flags are load-bearing, + not tidiness:** on the container driver `build-push-action@v5` defaults + `provenance` to true when pushing, an attestation manifest makes the pushed + tag a manifest INDEX, and config labels do not resolve through an index — so + leaving them on would make every push read `fc.revision=`, miss, and + rebuild forever with every lane green. Same failure as #3183, different door. + These jobs run inside a container against a mounted docker socket, so the + buildkit container is a sibling rather than a child. - **`FC_CHANNEL` and `FC_VERSION` are build args, not runtime settings.** `build.yml` passes them to the web image only — the ml and agent images have nothing to report them to. `/api/health` returns both, the foot of Settings