fix(external): split fetch timeout into read (60s) + total (30m) budgets (#883)

The single _FETCH_TIMEOUT=3000s meant different things per host: a TOTAL
wall-clock for mega (subprocess), but only a per-read socket timeout for HTTP
hosts (requests' timeout is the idle gap between bytes, never a total). So a
stalled HTTP connection tied up a download-worker slot AND the per-host
serialize lock for ~50 min before failing (operator-flagged 2026-06-17).

Split into two limits in external_fetch:
- read timeout (_READ_TIMEOUT=60s, with _CONNECT_TIMEOUT=30s) → requests gets
  (connect, read); a stalled socket now fails in ~60s.
- total budget (_TOTAL_TIMEOUT=30min) → enforced as a wall-clock deadline
  across chunks in _stream_to_file (HTTP has no total-download timeout), and
  passed as the subprocess total for mega.
fetch_external() signature: timeout= → read_timeout=/total_timeout=. gdrive
(gdown) self-manages; the celery hard limit is the outer backstop.

Also lowered the per-host lock TTL 3600→2400 so a worker that dies holding it
can't wedge a host's links much past one fetch's budget.

Each external link is already one Celery task (sweep enqueues one
fetch_external_link.delay per link), so these budgets are per-link.

Tests: total-budget-exceeded cleans the .part; HTTP gets (connect, read);
mega gets the total. Worker fakes updated to **kwargs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

backend/app/tasks/external.py

@@ -41,16 +41,19 @@ IMAGES_ROOT = Path("/images")
 # After this many failed attempts a link is dead-lettered (skipped by routine
 # sweeps; an operator recovery still re-attempts). Mirrors the ingester ledger.
 DEAD_LETTER_THRESHOLD = 3
-# Per-fetch wall-clock budget — films/packs are large; the celery time_limit is
-# the backstop above this.
-_FETCH_TIMEOUT = 3000.0
+# Per-fetch read + total budgets now live in external_fetch (a short read
+# timeout fails a stalled host fast; a generous total caps a big-but-flowing
+# download). The celery soft/hard time_limit is the outer backstop above those.
 # Links enqueued per sweep — bounds the burst when a big backfill records many.
 _SWEEP_BATCH = 50
 # Dead rows older than this are pruned (retention).
 _RETENTION_DAYS = 30
-# Per-host serialize lock.
+# Per-host serialize lock. TTL is a safety net for a worker that dies holding it
+# (normal completion/error releases it in `finally`); sized just past the fetch
+# total budget (30 min) so a dead worker can't wedge a host's links much longer
+# than one fetch would have taken.
 _LOCK_PREFIX = "fc:extdl_lock:"
-_LOCK_TTL = 3600
+_LOCK_TTL = 2400
 _SERIALIZE_COUNTDOWN = 120
 _MAX_SERIALIZE_WAITS = 30
 
@@ -177,7 +180,7 @@ def fetch_external_link(self, link_id: int, _serialize_waits: int = 0) -> dict:
         / str(post.external_post_id) / str(link_id)
     )
     try:
-        result = fetch_external(host, url, post_dir, timeout=_FETCH_TIMEOUT)
+        result = fetch_external(host, url, post_dir)
         with SessionLocal() as session:
             link = session.get(ExternalLink, link_id)
             if not result.ok: