fix: the smoke's health probe has to run inside the sandbox it created (4319)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 5s
Build images / build-agent (push) Successful in 6s
Build images / build-ml (push) Successful in 6s
Build images / build-web (push) Successful in 5s
Build images / smoke-web (push) Skipped
Build images / promote (push) Skipped
CI / frontend-build (push) Successful in 19s
extension / lint (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 2m11s
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 5s
Build images / build-agent (push) Successful in 6s
Build images / build-ml (push) Successful in 6s
Build images / build-web (push) Successful in 5s
Build images / smoke-web (push) Skipped
Build images / promote (push) Skipped
CI / frontend-build (push) Successful in 19s
extension / lint (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 2m11s
The first real execution of the egress-blocked smoke (run 7282) failed with
"web is running but never answered /api/health" — and the application was
perfect. Its own log shows all four hypercorn workers serving three seconds
after start and still up seven minutes later, with no internet:
[entrypoint] Starting hypercorn on :8080
[14:08:58] [10] [INFO] Running on http://0.0.0.0:8080
[14:08:58] [11] [INFO] Running on http://0.0.0.0:8080
[14:08:58] [12] [INFO] Running on http://0.0.0.0:8080
[14:08:58] [13] [INFO] Running on http://0.0.0.0:8080
5ca1058 put the app container on an `--internal` network. Docker gives such
a network isolation rules that DROP traffic entering it from any other
interface, and this job's own container sits on the runner's default bridge
— so its curl to the app was discarded before arrival. Dropped rather than
refused, so every attempt burned the full --max-time and the loop read as a
wedged app instead of an unroutable address.
Steps 0-2 were right only by accident: each already runs its check inside a
container on $NET. Step 3 was the one place that reached in from outside,
and so the one place that could not work. It now probes from inside too,
using the image's own python3 over `shell -c` — the same shape as the egress
guard above it, and necessary because the runtime stage ships no curl.
Attaching the job container to $NET would also work in one line. Rejected:
it puts an internet-connected container on the network whose whole purpose
is being offline, and it would rest on `hostname` equalling the container id.
Same family as #3374 — a CI check that could never pass, failing in a way
that accuses the thing it was meant to protect. Worth stating plainly: the
egress property itself PASSED on 7282 ("smoke: no route out, as required")
and the schema built from empty through 0104 with no network. Only the
harness's last step was broken.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
@@ -1353,9 +1353,34 @@ jobs:
|
||||
trap 'rc=$?; [ $rc -eq 0 ] || docker logs "$CID" 2>&1 | tail -40; docker rm -f "$CID" >/dev/null 2>&1 || true; exit $rc' EXIT
|
||||
WEB_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$CID")
|
||||
test -n "$WEB_IP"
|
||||
|
||||
# The probe runs INSIDE the sandbox, like every check above it.
|
||||
#
|
||||
# It has to. Docker gives an `--internal` network isolation rules
|
||||
# that DROP traffic entering it from any other interface, and this
|
||||
# job's own container sits on the runner's default bridge — so a
|
||||
# curl from here to $WEB_IP is discarded before it arrives. Because
|
||||
# the packets are dropped rather than refused, every attempt burns
|
||||
# the full --max-time and the job reports a web container that
|
||||
# "never answered" while the app is running perfectly. That is how
|
||||
# this read on its first real execution (run 7282): a false failure
|
||||
# blaming the application for the harness's own blind spot.
|
||||
#
|
||||
# Steps 0-2 were already right by accident — each runs a container
|
||||
# ON $NET. Only this one reached in from outside, and it was the
|
||||
# only one that could not work.
|
||||
#
|
||||
# Same shape as the egress probe above: the image's own python3 over
|
||||
# `shell -c`, since the runtime stage ships no curl.
|
||||
probe() {
|
||||
docker run --rm --network "$NET" "$CANDIDATE" shell -c \
|
||||
"python3 -c \"import urllib.request; urllib.request.urlopen('http://$WEB_IP:8080/api/health', timeout=5)\"" \
|
||||
>/dev/null 2>&1
|
||||
}
|
||||
|
||||
healthy=""
|
||||
for i in $(seq 1 60); do
|
||||
if curl -fsS --max-time 5 "http://$WEB_IP:8080/api/health" >/dev/null 2>&1; then
|
||||
if probe; then
|
||||
healthy=1
|
||||
break
|
||||
fi
|
||||
@@ -1381,8 +1406,9 @@ jobs:
|
||||
echo "smoke: python base rather than at startup." >&2
|
||||
exit 1
|
||||
fi
|
||||
curl -fsS --max-time 5 "http://$WEB_IP:8080/api/health"
|
||||
echo
|
||||
# Print what it actually answered — from inside, for the same reason.
|
||||
docker run --rm --network "$NET" "$CANDIDATE" shell -c \
|
||||
"python3 -c \"import urllib.request; print(urllib.request.urlopen('http://$WEB_IP:8080/api/health', timeout=5).read().decode())\""
|
||||
|
||||
echo "smoke: all checks passed against $CANDIDATE, with egress blocked"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user