Files
FabledCurator/scripts/smoke_image.py
T
bvandeusenandClaude Opus 5 bfa9fd678b
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
CI / extension-version (push) Successful in 4s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 9s
Build images / build-web (push) Successful in 7s
Build images / smoke-web (push) Skipped
extension / lint (push) Successful in 20s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 1m48s
extension / lint (pull_request) Successful in 20s
ci: smoke the refreshed image against real Postgres and Redis
Milestone 362 step 3. This is the gate the weekly base refresh never had.

`ci.yml` cannot be that gate, and the reason matters more than the fix. Its
lanes run on ci-python:3.14 and install requirements.txt — a base refresh
changes neither, so all five stay green through a bump that breaks the product.
What a refresh re-resolves is the Dockerfile's apt layer:

    ffmpeg unar libpq5 postgresql-client zstd megatools
    libjpeg62-turbo libwebp7 libpng16-16 ca-certificates

Unpinned, every build, and nothing else in this repo looks at it. That line is
the dependency creep; it is also precisely what the test suite structurally
cannot observe, since the suite never runs inside the image and the image
carries no tests and no pytest.

So `smoke-web` runs the CANDIDATE IMAGE against real service containers:

  1. `alembic upgrade head` on an empty database — the image's own libpq and
     psycopg, and the same call entrypoint.sh makes before it serves anything,
     so a failure here is a failure to boot.
  2. The apt binaries, then the application's own `Thumbnailer` — JPEG, PNG
     with alpha, WebP, and a video frame through ffmpeg. `Thumbnailer` needs no
     database and no app context, so the check exercises real product code
     rather than a proxy for it. `ffmpeg -version` exiting 0 would pass while a
     codec removal broke every thumbnail in the library.
  3. The web role boots and answers /api/health.

Every failure names the package it implicates. This fires on a Sunday,
unattended, about a change nobody made deliberately — "assertion failed" a week
later teaches nobody anything.

The script is piped over stdin rather than bind-mounted: the workspace is a
docker volume belonging to the job's own container, so a host bind of $PWD does
not resolve for a sibling. Container logs are dumped only on failure, and the
trap re-exits with the real status rather than the status of `docker rm`.

Deliberately NOT gating the promote yet — that is step 4. Landing the gate and
the thing it gates together would mean the first time anyone saw this job run
would also be the first time it could stop a publish.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA
2026-09-02 15:45:10 -04:00

156 lines
6.1 KiB
Python

"""Prove a freshly built image can still do the things its OS packages provide.
Run INSIDE the image, not against the source tree. That distinction is the
entire reason this file exists.
`ci.yml`'s lanes run on `ci-python:3.14` and install `requirements.txt`. A base
refresh changes neither, so all five lanes stay green through a base bump that
breaks the product. What a refresh actually re-resolves is this, from the
Dockerfile:
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg unar libpq5 postgresql-client zstd megatools \
libjpeg62-turbo libwebp7 libpng16-16 ca-certificates
Unpinned, every build. Nothing else in this repo looks at it.
So the checks below run the APPLICATION'S OWN code — `Thumbnailer`, which needs
no database and no app context — against whatever Pillow and ffmpeg have
become. `ffmpeg -version` exiting 0 would pass while a codec removal or an
soname bump broke every thumbnail in the library; producing a thumbnail would
not.
Every failure names the package it implicates. This fires on a Sunday,
unattended, about a change nobody made deliberately — "assertion failed" a week
later teaches nobody anything.
Usage: docker run --rm -i <image> shell -c 'python3 -' < scripts/smoke_image.py
"""
from __future__ import annotations
import shutil
import subprocess
import tempfile
from pathlib import Path
try:
from PIL import Image
from backend.app.services.thumbnailer import Thumbnailer
except Exception as exc: # noqa: BLE001 — a smoke test reports, it never raises
print(f"smoke: FAILED — could not import the thumbnail path at all: {exc}")
print(" Implicates Pillow or its shared libraries (libjpeg62-turbo,")
print(" libpng16-16, libwebp7), or the python base image itself.")
raise SystemExit(1) from exc
# Binary → what stops working without it. Listed individually because
# `--no-install-recommends` means any one of them can vanish on its own when a
# dependency chain higher up changes.
REQUIRED_BINARIES = {
"ffmpeg": "video thumbnails and transcoding (Dockerfile: ffmpeg)",
"unar": "archive import — cbz/zip/rar members (Dockerfile: unar)",
"pg_dump": "database backup (Dockerfile: postgresql-client)",
"zstd": "backup compression, pg_dump | tar --zstd (Dockerfile: zstd)",
"megatools": "mega.nz public-link downloads, #830 (Dockerfile: megatools)",
}
def check_jpeg(thumbs: Thumbnailer, src: Path) -> None:
path = src / "flat.jpg"
Image.new("RGB", (900, 400), (30, 90, 160)).save(path, "JPEG")
result = thumbs.generate_image_thumbnail(path, "a" * 64)
assert result.mime == "image/jpeg", f"mime was {result.mime}"
assert result.path.stat().st_size > 0, "no bytes written"
# Re-open it. A file that writes but cannot be read back is the shape a
# half-broken codec produces, and size alone would not catch it.
with Image.open(result.path) as im:
im.load()
def check_png_alpha(thumbs: Thumbnailer, src: Path) -> None:
path = src / "alpha.png"
Image.new("RGBA", (400, 900), (200, 40, 40, 128)).save(path, "PNG")
result = thumbs.generate_image_thumbnail(path, "b" * 64)
assert result.mime == "image/png", f"mime was {result.mime}"
with Image.open(result.path) as im:
im.load()
assert im.mode in ("RGBA", "LA", "P"), f"alpha lost, mode={im.mode}"
def check_webp(thumbs: Thumbnailer, src: Path) -> None:
path = src / "sample.webp"
Image.new("RGB", (500, 500), (10, 140, 70)).save(path, "WEBP")
result = thumbs.generate_image_thumbnail(path, "c" * 64)
assert result.path.stat().st_size > 0, "no bytes written"
def check_video(thumbs: Thumbnailer, src: Path) -> None:
# Synthesised rather than committed as a fixture: a checked-in video is a
# binary blob nobody can review, and lavfi ships with every ffmpeg build.
#
# 3 seconds, not 2. The seek lands at max(1.0, duration * 0.05) = 1.0s, and
# a clip barely longer than its own seek is how #1231 produced zero frames.
# This check exists to exercise ffmpeg, not to re-litigate that edge.
clip = src / "clip.mp4"
subprocess.run(
["ffmpeg", "-nostdin", "-f", "lavfi", "-i", "testsrc=size=640x360:rate=10",
"-t", "3", "-pix_fmt", "yuv420p", "-y", str(clip)],
check=True, capture_output=True, timeout=120,
)
result = thumbs.generate_video_thumbnail(clip, "d" * 64, duration_seconds=3.0)
assert result.path.stat().st_size > 0, "no bytes written"
with Image.open(result.path) as im:
im.load()
CHECKS = (
("JPEG thumbnail", "libjpeg62-turbo / Pillow", check_jpeg),
("PNG thumbnail (alpha)", "libpng16-16 / Pillow", check_png_alpha),
("WebP decode", "libwebp7 / Pillow", check_webp),
("video thumbnail", "ffmpeg", check_video),
)
def main() -> int:
failures: list[str] = []
print("smoke: binaries the apt layer provides")
for binary, purpose in REQUIRED_BINARIES.items():
if shutil.which(binary) is None:
print(f" FAIL {binary}: not on PATH")
failures.append(f"{binary}{purpose}")
else:
print(f" ok {binary}")
print("smoke: the application's own thumbnail path, against this image's libraries")
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
src = root / "src"
src.mkdir()
thumbs = Thumbnailer(root)
for name, implicates, fn in CHECKS:
try:
fn(thumbs, src)
print(f" ok {name}")
except Exception as exc: # noqa: BLE001 — report every check, then fail once
print(f" FAIL {name}: {exc}")
failures.append(f"{name}{implicates}")
if failures:
print(f"\nsmoke: FAILED — {len(failures)} check(s)")
for failure in failures:
print(f" - {failure}")
print("\nThis image was built against freshly resolved base layers. The")
print("named packages are where to look: compare this build's apt versions")
print("against the previous :latest before assuming the app changed.")
return 1
print("\nsmoke: all checks passed")
return 0
if __name__ == "__main__":
raise SystemExit(main())