bvandeusen
e3cdd0f92b
Layer 3 — prevent the hard worker crash rather than just recovering from it. The realistic process-crash vectors (operator's observed slow/heavy tasks) are video decode and archive extraction; images decode in-process and Pillow raises-and-skips cleanly, and a subprocess per image would wreck deep-scan throughput, so images are intentionally not probed. New backend/app/utils/safe_probe.py (leaf module, lazy heavy imports so the spawned child stays light): - probe_video(path): validates the container + first video stream via ffprobe (a separate binary — a decoder crash kills only ffprobe, not the worker). Returns width/height, which the importer didn't capture for videos before. crashed=True only on ffprobe timeout. - probe_archive(path): an uncompressed-size bomb guard (MAX_ARCHIVE_UNCOMPRESSED_BYTES = 4 GiB) plus the format integrity test (zipfile.testzip / rarfile.testrar / py7zr.test) run in a spawned child process. A decompression-bomb OOM or native-lib segfault on a malformed archive shows up as a non-zero child exit code → crashed=True, never a dead worker. ProbeResult.crashed distinguishes a HARD failure (subprocess killed / timed out — the poison-pill signature → caller returns terminal 'failed') from a CLEAN rejection (corrupt-but-handled, bomb cap, integrity mismatch → caller's choice of skipped/attached). Wired: - importer._import_media video branch: probe_video before the pipeline; crash → failed, clean reject → invalid_image skip, ok → capture dims. - importer._import_archive: probe_archive before extract_archive; crash → failed, clean reject → still preserve the archive as a PostAttachment (matches extract_archive's fail-soft contract). - ml.tag_and_embed video branch: probe_video before sampling 10 frames, so a corrupt video is rejected (status='bad_video') instead of crashing the ml-worker on frame decode. Tests (test_safe_probe.py): valid/corrupt zip via probe_archive, direct _inspect_archive size+integrity, in-process _archive_probe_target bomb guard (monkeypatch can't reach a spawned child, so the target is called directly), and a non-video → ok=False that's robust to ffprobe presence in CI.
72 lines
2.5 KiB
Python
72 lines
2.5 KiB
Python
"""Layer-3 subprocess-isolated probe tests.
|
|
|
|
The bomb-guard cap is exercised against `_archive_probe_target` directly
|
|
(in-process, where a monkeypatch on the module constant takes effect) —
|
|
spawn re-imports the module in the child, so a parent-process
|
|
monkeypatch wouldn't reach the spawned worker.
|
|
"""
|
|
|
|
import multiprocessing as mp
|
|
import zipfile
|
|
from pathlib import Path
|
|
|
|
from backend.app.utils import safe_probe
|
|
|
|
|
|
def _zip(path, entries):
|
|
with zipfile.ZipFile(path, "w") as zf:
|
|
for name, data in entries.items():
|
|
zf.writestr(name, data)
|
|
|
|
|
|
def test_probe_archive_valid_zip(tmp_path):
|
|
z = tmp_path / "ok.zip"
|
|
_zip(z, {"a.jpg": b"hello", "b.png": b"world"})
|
|
res = safe_probe.probe_archive(z)
|
|
assert res.ok is True
|
|
assert res.crashed is False
|
|
|
|
|
|
def test_probe_archive_corrupt_zip_clean_rejection(tmp_path):
|
|
z = tmp_path / "broken.zip"
|
|
z.write_bytes(b"PK\x03\x04 not really a zip past here")
|
|
res = safe_probe.probe_archive(z)
|
|
assert res.ok is False
|
|
# Corrupt-but-handled (zipfile raises BadZipFile in the child) — a
|
|
# clean rejection, not a hard crash.
|
|
assert res.crashed is False
|
|
assert res.reason
|
|
|
|
|
|
def test_inspect_archive_reports_size_and_clean_integrity(tmp_path):
|
|
z = tmp_path / "sized.zip"
|
|
_zip(z, {"a.txt": b"x" * 100, "b.txt": b"y" * 50})
|
|
total, bad = safe_probe._inspect_archive(z, ".zip")
|
|
assert total == 150
|
|
assert bad is None
|
|
|
|
|
|
def test_archive_probe_target_bomb_guard(tmp_path, monkeypatch):
|
|
"""In-process call to the child target so the monkeypatched cap
|
|
takes effect. A normal zip whose uncompressed size exceeds the
|
|
(lowered) cap is rejected with the bomb-guard reason."""
|
|
monkeypatch.setattr(safe_probe, "MAX_ARCHIVE_UNCOMPRESSED_BYTES", 10)
|
|
z = tmp_path / "bomb.zip"
|
|
_zip(z, {"big.txt": b"x" * 5000}) # 5000 uncompressed > 10-byte cap
|
|
q = mp.get_context("spawn").Queue()
|
|
safe_probe._archive_probe_target(str(z), q)
|
|
status, detail = q.get(timeout=5)
|
|
assert status == "error"
|
|
assert "bomb-guard cap" in detail
|
|
|
|
|
|
def test_probe_video_non_video_is_not_ok(tmp_path):
|
|
"""A text file is not a decodable video. Whether ffprobe is present
|
|
(returncode != 0) or absent (OSError → 'unavailable'), the result is
|
|
ok=False. We don't assert on crashed/reason so the test is robust to
|
|
ffprobe presence in CI."""
|
|
f = tmp_path / "nope.txt"
|
|
f.write_text("definitely not a video container")
|
|
res = safe_probe.probe_video(f)
|
|
assert res.ok is False
|