From 7175ace67aeee03e76c73ba1fa4501c538ccc351 Mon Sep 17 00:00:00 2001 From: Bryan Van Deusen Date: Tue, 22 Sep 2026 10:16:17 -0400 Subject: [PATCH] fix: the smoke's health probe has to run inside the sandbox it created (4319) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first real execution of the egress-blocked smoke (run 7282) failed with "web is running but never answered /api/health" — and the application was perfect. Its own log shows all four hypercorn workers serving three seconds after start and still up seven minutes later, with no internet: [entrypoint] Starting hypercorn on :8080 [14:08:58] [10] [INFO] Running on http://0.0.0.0:8080 [14:08:58] [11] [INFO] Running on http://0.0.0.0:8080 [14:08:58] [12] [INFO] Running on http://0.0.0.0:8080 [14:08:58] [13] [INFO] Running on http://0.0.0.0:8080 5ca1058 put the app container on an `--internal` network. Docker gives such a network isolation rules that DROP traffic entering it from any other interface, and this job's own container sits on the runner's default bridge — so its curl to the app was discarded before arrival. Dropped rather than refused, so every attempt burned the full --max-time and the loop read as a wedged app instead of an unroutable address. Steps 0-2 were right only by accident: each already runs its check inside a container on $NET. Step 3 was the one place that reached in from outside, and so the one place that could not work. It now probes from inside too, using the image's own python3 over `shell -c` — the same shape as the egress guard above it, and necessary because the runtime stage ships no curl. Attaching the job container to $NET would also work in one line. Rejected: it puts an internet-connected container on the network whose whole purpose is being offline, and it would rest on `hostname` equalling the container id. Same family as #3374 — a CI check that could never pass, failing in a way that accuses the thing it was meant to protect. Worth stating plainly: the egress property itself PASSED on 7282 ("smoke: no route out, as required") and the schema built from empty through 0104 with no network. Only the harness's last step was broken. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR --- .forgejo/workflows/build.yml | 32 +++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/.forgejo/workflows/build.yml b/.forgejo/workflows/build.yml index ce3bf76..cbbd559 100644 --- a/.forgejo/workflows/build.yml +++ b/.forgejo/workflows/build.yml @@ -1353,9 +1353,34 @@ jobs: trap 'rc=$?; [ $rc -eq 0 ] || docker logs "$CID" 2>&1 | tail -40; docker rm -f "$CID" >/dev/null 2>&1 || true; exit $rc' EXIT WEB_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$CID") test -n "$WEB_IP" + + # The probe runs INSIDE the sandbox, like every check above it. + # + # It has to. Docker gives an `--internal` network isolation rules + # that DROP traffic entering it from any other interface, and this + # job's own container sits on the runner's default bridge — so a + # curl from here to $WEB_IP is discarded before it arrives. Because + # the packets are dropped rather than refused, every attempt burns + # the full --max-time and the job reports a web container that + # "never answered" while the app is running perfectly. That is how + # this read on its first real execution (run 7282): a false failure + # blaming the application for the harness's own blind spot. + # + # Steps 0-2 were already right by accident — each runs a container + # ON $NET. Only this one reached in from outside, and it was the + # only one that could not work. + # + # Same shape as the egress probe above: the image's own python3 over + # `shell -c`, since the runtime stage ships no curl. + probe() { + docker run --rm --network "$NET" "$CANDIDATE" shell -c \ + "python3 -c \"import urllib.request; urllib.request.urlopen('http://$WEB_IP:8080/api/health', timeout=5)\"" \ + >/dev/null 2>&1 + } + healthy="" for i in $(seq 1 60); do - if curl -fsS --max-time 5 "http://$WEB_IP:8080/api/health" >/dev/null 2>&1; then + if probe; then healthy=1 break fi @@ -1381,8 +1406,9 @@ jobs: echo "smoke: python base rather than at startup." >&2 exit 1 fi - curl -fsS --max-time 5 "http://$WEB_IP:8080/api/health" - echo + # Print what it actually answered — from inside, for the same reason. + docker run --rm --network "$NET" "$CANDIDATE" shell -c \ + "python3 -c \"import urllib.request; print(urllib.request.urlopen('http://$WEB_IP:8080/api/health', timeout=5).read().decode())\"" echo "smoke: all checks passed against $CANDIDATE, with egress blocked"