fix(import): archive probe → subprocess (was multiprocessing.Process)
Every archive import was failing immediately with "AssertionError:
daemonic processes are not allowed to have children" (operator-flagged
2026-05-30 — import_archive_file crashes in 49ms with the assertion).
Celery's prefork pool runs tasks in daemon processes; Python's
multiprocessing module refuses to let daemons spawn children, which is
exactly what probe_archive was doing via mp.get_context("spawn")
.Process. The Layer-3 crash-isolation feature added 2026-05-28 was
effectively a hard-blocker on the very import path it was meant to
protect.
Switched probe_archive to subprocess.run — no daemon restriction, still
isolates the probe (a probe segfault/OOM exits non-zero, doesn't kill
the worker). The probe body lifted to a tiny runner module
(_archive_probe_runner) that imports the unchanged _run_probe helper
and prints a single JSON line; parent parses stdout, returns the
ProbeResult exactly as before (timeout, signal, OOM, clean-rejection,
ok — all preserved).
cwd for the subprocess is the repo root derived from __file__ parents
so `python -m backend.app.utils._archive_probe_runner` resolves both in
the Celery container and pytest, regardless of where the worker was
launched.
Test refactor: test_archive_probe_target_bomb_guard →
test_run_probe_bomb_guard. Same in-process call to the (renamed) probe
body so the monkeypatched cap still takes effect; the real subprocess
path is exercised by the existing test_probe_archive_valid_zip /
test_probe_archive_corrupt_zip_clean_rejection tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,12 +1,11 @@
|
||||
"""Layer-3 subprocess-isolated probe tests.
|
||||
|
||||
The bomb-guard cap is exercised against `_archive_probe_target` directly
|
||||
(in-process, where a monkeypatch on the module constant takes effect) —
|
||||
spawn re-imports the module in the child, so a parent-process
|
||||
monkeypatch wouldn't reach the spawned worker.
|
||||
The bomb-guard cap is exercised against `_run_probe` directly (in-process,
|
||||
where a monkeypatch on the module constant takes effect) — the real probe
|
||||
runs in a subprocess that re-imports the module, so a parent-process
|
||||
monkeypatch wouldn't reach the spawned interpreter.
|
||||
"""
|
||||
|
||||
import multiprocessing as mp
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
@@ -46,16 +45,14 @@ def test_inspect_archive_reports_size_and_clean_integrity(tmp_path):
|
||||
assert bad is None
|
||||
|
||||
|
||||
def test_archive_probe_target_bomb_guard(tmp_path, monkeypatch):
|
||||
"""In-process call to the child target so the monkeypatched cap
|
||||
takes effect. A normal zip whose uncompressed size exceeds the
|
||||
(lowered) cap is rejected with the bomb-guard reason."""
|
||||
def test_run_probe_bomb_guard(tmp_path, monkeypatch):
|
||||
"""In-process call to _run_probe so the monkeypatched cap takes effect.
|
||||
The public probe_archive runs in a subprocess which re-imports the
|
||||
module and wouldn't see the patched constant."""
|
||||
monkeypatch.setattr(safe_probe, "MAX_ARCHIVE_UNCOMPRESSED_BYTES", 10)
|
||||
z = tmp_path / "bomb.zip"
|
||||
_zip(z, {"big.txt": b"x" * 5000}) # 5000 uncompressed > 10-byte cap
|
||||
q = mp.get_context("spawn").Queue()
|
||||
safe_probe._archive_probe_target(str(z), q)
|
||||
status, detail = q.get(timeout=5)
|
||||
status, detail = safe_probe._run_probe(str(z))
|
||||
assert status == "error"
|
||||
assert "bomb-guard cap" in detail
|
||||
|
||||
|
||||
Reference in New Issue
Block a user