fix: a native chunk stops walking when its import work would overrun the task
CI and images / lint (push) Successful in 2s
CI and images / extension-version (push) Successful in 2s
CI and images / frontend-build (push) Successful in 19s
CI and images / backend-lint-and-test (push) Successful in 31s
CI and images / integration (push) Successful in 2m15s
CI and images / sign-extension (push) Successful in 3s
CI and images / build-agent (push) Successful in 7s
CI and images / build-web (push) Successful in 1m41s
CI and images / smoke-web (push) Successful in 55s
CI and images / promote (push) Skipped
CI and images / lint (push) Successful in 2s
CI and images / extension-version (push) Successful in 2s
CI and images / frontend-build (push) Successful in 19s
CI and images / backend-lint-and-test (push) Successful in 31s
CI and images / integration (push) Successful in 2m15s
CI and images / sign-extension (push) Successful in 3s
CI and images / build-agent (push) Successful in 7s
CI and images / build-web (push) Successful in 1m41s
CI and images / smoke-web (push) Successful in 55s
CI and images / promote (push) Skipped
The walk's time budget covered the walk alone, but phase 3 runs in the same Celery task under the same 1350s soft limit. TamadaHeijun's recapture walked for about two minutes and handed phase 3 431 orphan imports and ~3000 relinks. Phase 3 ran for 20 minutes and died at the soft limit (event 90808). This predates the worker consolidation; the limits are unchanged since June. The walk now also stops when elapsed time plus phase 3's estimated cost (2.5s per import, 0.25s per relink, measured on the live instance) passes CHUNK_TOTAL_SECONDS (1200). Work handed to phase 3 counts as progress, so such a stop is a PARTIAL chunk boundary and the next chunk resumes the page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
@@ -46,6 +46,15 @@ def test_timeout_ladder_keeps_subprocess_budgets_under_soft_limit():
|
||||
assert DOWNLOAD_SOFT_TIME_LIMIT < DOWNLOAD_HARD_TIME_LIMIT
|
||||
|
||||
|
||||
def test_the_native_chunk_leaves_room_to_tear_down_before_the_soft_limit():
|
||||
"""The native walk sizes itself against CHUNK_TOTAL_SECONDS, walk plus
|
||||
phase 3; that total has to leave the task time to finalize its event."""
|
||||
from backend.app.services.ingest_core import CHUNK_TOTAL_SECONDS
|
||||
from backend.app.tasks.download import DOWNLOAD_SOFT_TIME_LIMIT
|
||||
|
||||
assert CHUNK_TOTAL_SECONDS <= DOWNLOAD_SOFT_TIME_LIMIT - 120
|
||||
|
||||
|
||||
def test_decorated_limits_match_module_constants():
|
||||
"""The @celery.task decorator must use the audited constants, not
|
||||
drifted literals."""
|
||||
|
||||
Reference in New Issue
Block a user