fix(archive): magic-byte archive detection so mis-named archives extract — #713 part 1
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 2m59s

Patreon attachment downloads land with sanitized URL-blob filenames
(01_https___www.patreon.com_media-u_v3_<id>) whose Path.suffix is junk, never
.zip — so the extension-only is_archive() filed them as opaque PostAttachments
and never extracted them ("No images attached to this post").

Add archive_extractor.detect_archive_format() — extension first, then magic-byte
sniff (zipfile.is_zipfile + RAR/7z signatures). is_archive(), extract_archive(),
and safe_probe._inspect_archive() (the bomb-guard) all route through it, so a
mis-named/extension-less archive is now detected, bomb-guarded, integrity-tested,
AND extracted regardless of filename. Stops new ones; part 2 re-extracts the
already-imported backlog.

Tests: mis-named zip detected + extracted; non-archive dotted name not
misdetected; _inspect_archive on a mis-named zip; signature updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-06 14:33:24 -04:00
parent 911d535f56
commit 5bb25245a5
4 changed files with 83 additions and 15 deletions
+13 -9
View File
@@ -149,9 +149,8 @@ def _run_probe(path_str: str) -> tuple[str, str | None]:
call the same code path.
"""
path = Path(path_str)
ext = path.suffix.lower()
try:
total, test_bad = _inspect_archive(path, ext)
total, test_bad = _inspect_archive(path)
except Exception as exc: # noqa: BLE001 — clean rejection
return ("error", f"{type(exc).__name__}: {exc}")
if total is not None and total > MAX_ARCHIVE_UNCOMPRESSED_BYTES:
@@ -162,24 +161,29 @@ def _run_probe(path_str: str) -> tuple[str, str | None]:
return ("ok", None)
def _inspect_archive(path: Path, ext: str):
def _inspect_archive(path: Path):
"""Return (total_uncompressed_bytes | None, first_bad_member | None)
for the archive. Format-specific; raises on a structurally-broken
container (caught by the child as a clean rejection)."""
if ext in (".zip", ".cbz"):
for the archive. Format detected by extension OR magic bytes (so a
mis-named archive is still bomb-guarded + integrity-tested, matching the
extractor's gate); raises on a structurally-broken container (caught by the
child as a clean rejection)."""
from ..services.archive_extractor import detect_archive_format
fmt = detect_archive_format(path)
if fmt == "zip":
import zipfile
with zipfile.ZipFile(path) as zf:
total = sum(zi.file_size for zi in zf.infolist())
return total, zf.testzip()
if ext == ".rar":
if fmt == "rar":
import rarfile
with rarfile.RarFile(path) as rf:
total = sum(getattr(ri, "file_size", 0) for ri in rf.infolist())
rf.testrar()
return total, None
if ext == ".7z":
if fmt == "7z":
import py7zr
with py7zr.SevenZipFile(path, "r") as zf:
@@ -187,5 +191,5 @@ def _inspect_archive(path: Path, ext: str):
total = getattr(info, "uncompressed", None)
ok = zf.test() # True / None when all members pass
return total, (None if ok in (True, None) else "7z test reported corruption")
# Unknown extension — nothing to test; treat as clean.
# Not a recognised archive — nothing to test; treat as clean.
return None, None