fix: a game build is recognised before the archive integrity test, so re-extract no longer stalls on it
CI and images / lint (push) Successful in 2s
CI and images / extension-version (push) Successful in 2s
CI and images / frontend-build (push) Successful in 24s
CI and images / extension-test (push) Successful in 18s
CI and images / backend-lint-and-test (push) Successful in 47s
CI and images / integration (push) Successful in 2m44s
CI and images / sign-extension (push) Successful in 3s
CI and images / build-agent (push) Successful in 5s
CI and images / build-web (push) Successful in 2m1s
CI and images / smoke-web (push) Successful in 49s
CI and images / promote (push) Successful in 1s
CI and images / refresh-dev (push) Skipped
CI and images / unattended-failure (push) Skipped

The archive probe read and checksummed every member before the importer
looked at what the archive held. For a multi-gigabyte game build that is
minutes of work on a file that is kept whole and never extracted, and it
held a re-extract chunk far past its time budget.

- The importer checks the archive's directory for a software build first,
  for the outer archive and for one nested inside an art pack, and only
  probes archives it may extract.
- The probe rejects an archive over the size cap from its directory totals,
  before the integrity test decompresses anything.
- Re-extract logs each archive as it starts and finishes, with its size and
  how long it took, so an archive that stalls a chunk is named in the log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-08 07:53:05 -05:00
co-authored by Claude Opus 5.5
parent af8386509a
commit 4cdc851cb5
5 changed files with 93 additions and 28 deletions
+9 -1
View File
@@ -1205,6 +1205,11 @@ def reextract_archive_attachments(
)
if post is None:
summary["without_post"] += 1
# Logged on the way in as well as out, so an archive that stalls a
# chunk is named in the log even if the worker dies holding it.
size_mb = (att.size_bytes or 0) / 1e6
log.info("re-extract: attachment %s (%.0f MB) starting", att.id, size_mb)
archive_start = time.monotonic()
try:
ids = _reextract_archive_to_post(
importer, stored, post, source_row, artist, images_root,
@@ -1214,8 +1219,11 @@ def reextract_archive_attachments(
except Exception as exc: # one bad archive must not strand the rest
session.rollback()
summary["errors"] += 1
log.warning("re-extract failed for attachment %s: %s", att.id, exc)
log.warning("re-extract failed for attachment %s after %.1fs: %s",
att.id, time.monotonic() - archive_start, exc)
continue
log.info("re-extract: attachment %s done in %.1fs, %d new image(s)",
att.id, time.monotonic() - archive_start, len(ids))
if ids:
summary["members_imported"] += len(ids)
if post is not None: