feat(import-resilience L3): subprocess-isolated probes for video + archive

Layer 3 — prevent the hard worker crash rather than just recovering from
it. The realistic process-crash vectors (operator's observed slow/heavy
tasks) are video decode and archive extraction; images decode in-process
and Pillow raises-and-skips cleanly, and a subprocess per image would
wreck deep-scan throughput, so images are intentionally not probed.

New backend/app/utils/safe_probe.py (leaf module, lazy heavy imports so
the spawned child stays light):

- probe_video(path): validates the container + first video stream via
  ffprobe (a separate binary — a decoder crash kills only ffprobe, not
  the worker). Returns width/height, which the importer didn't capture
  for videos before. crashed=True only on ffprobe timeout.
- probe_archive(path): an uncompressed-size bomb guard
  (MAX_ARCHIVE_UNCOMPRESSED_BYTES = 4 GiB) plus the format integrity
  test (zipfile.testzip / rarfile.testrar / py7zr.test) run in a
  spawned child process. A decompression-bomb OOM or native-lib
  segfault on a malformed archive shows up as a non-zero child exit
  code → crashed=True, never a dead worker.

ProbeResult.crashed distinguishes a HARD failure (subprocess killed /
timed out — the poison-pill signature → caller returns terminal
'failed') from a CLEAN rejection (corrupt-but-handled, bomb cap,
integrity mismatch → caller's choice of skipped/attached).

Wired:
- importer._import_media video branch: probe_video before the pipeline;
  crash → failed, clean reject → invalid_image skip, ok → capture dims.
- importer._import_archive: probe_archive before extract_archive; crash
  → failed, clean reject → still preserve the archive as a
  PostAttachment (matches extract_archive's fail-soft contract).
- ml.tag_and_embed video branch: probe_video before sampling 10 frames,
  so a corrupt video is rejected (status='bad_video') instead of
  crashing the ml-worker on frame decode.

Tests (test_safe_probe.py): valid/corrupt zip via probe_archive, direct
_inspect_archive size+integrity, in-process _archive_probe_target bomb
guard (monkeypatch can't reach a spawned child, so the target is called
directly), and a non-video → ok=False that's robust to ffprobe presence
in CI.
This commit is contained in:
2026-05-28 00:01:32 -04:00
parent e77afe8295
commit e3cdd0f92b
4 changed files with 296 additions and 1 deletions
+12
View File
@@ -71,6 +71,18 @@ def tag_and_embed(self, image_id: int) -> dict:
embedder = get_embedder()
if _is_video(src):
# Layer-3 isolation: ffprobe (a separate process) validates
# the container before we burn ~20 GPU ops sampling frames
# from it. A corrupt video that would crash the frame
# decoder is rejected cleanly here instead of taking down
# the ml-worker. Operator-flagged 2026-05-28.
from ..utils import safe_probe
vprobe = safe_probe.probe_video(src)
if not vprobe.ok:
return {
"status": "bad_video", "image_id": image_id,
"reason": vprobe.reason,
}
frames = _sample_video_frames(
src, int(os.environ.get("VIDEO_ML_FRAMES", "10"))
)