bvandeusen
810baf63ac
Every archive import was failing immediately with "AssertionError:
daemonic processes are not allowed to have children" (operator-flagged
2026-05-30 — import_archive_file crashes in 49ms with the assertion).
Celery's prefork pool runs tasks in daemon processes; Python's
multiprocessing module refuses to let daemons spawn children, which is
exactly what probe_archive was doing via mp.get_context("spawn")
.Process. The Layer-3 crash-isolation feature added 2026-05-28 was
effectively a hard-blocker on the very import path it was meant to
protect.
Switched probe_archive to subprocess.run — no daemon restriction, still
isolates the probe (a probe segfault/OOM exits non-zero, doesn't kill
the worker). The probe body lifted to a tiny runner module
(_archive_probe_runner) that imports the unchanged _run_probe helper
and prints a single JSON line; parent parses stdout, returns the
ProbeResult exactly as before (timeout, signal, OOM, clean-rejection,
ok — all preserved).
cwd for the subprocess is the repo root derived from __file__ parents
so `python -m backend.app.utils._archive_probe_runner` resolves both in
the Celery container and pytest, regardless of where the worker was
launched.
Test refactor: test_archive_probe_target_bomb_guard →
test_run_probe_bomb_guard. Same in-process call to the (renamed) probe
body so the monkeypatched cap still takes effect; the real subprocess
path is exercised by the existing test_probe_archive_valid_zip /
test_probe_archive_corrupt_zip_clean_rejection tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
34 lines
1.2 KiB
Python
34 lines
1.2 KiB
Python
"""Subprocess entrypoint for safe_probe.probe_archive — see safe_probe.py.
|
|
|
|
probe_archive spawns this via subprocess (not multiprocessing.Process)
|
|
because Celery's prefork worker pool runs tasks in DAEMON processes and
|
|
Python's multiprocessing forbids daemon processes from spawning children
|
|
("AssertionError: daemonic processes are not allowed to have children",
|
|
operator-flagged 2026-05-30 — every archive import failed at task
|
|
startup). subprocess has no such restriction; we still get crash-
|
|
isolation because a probe segfault/OOM exits non-zero rather than
|
|
killing the worker.
|
|
|
|
Prints a single JSON line on stdout: {"status": "ok"|"error",
|
|
"detail": "..."?}. Exit code 0 for clean outcomes; non-zero exit
|
|
(signal / OOM-kill / unhandled exception) is the poison-pill signature
|
|
the parent maps to ProbeResult(crashed=True).
|
|
"""
|
|
import json
|
|
import sys
|
|
|
|
from .safe_probe import _run_probe
|
|
|
|
|
|
def main() -> int:
|
|
if len(sys.argv) != 2:
|
|
print(json.dumps({"status": "error", "detail": "usage: <path>"}))
|
|
return 2
|
|
status, detail = _run_probe(sys.argv[1])
|
|
print(json.dumps({"status": status, "detail": detail}))
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main())
|