Backfill events were STILL stranding empty after the timeout-ladder fix.
Worker logs showed the salvage path working ("Download timeout for
anduo/patreon after 1170.0s (18 files written)") but then:
Retry in 3s: DBAPIError(ConnectionDoesNotExistError: connection was
closed in the middle of operation)
...succeeded in 0.149s <- in-flight guard no-op
Root cause: DownloadService held the async + sync DB connections checked
out across the entire (≤19.5-min backfill) gallery-dl subprocess. The
server reaps the idle connection, so phase 3's first query hits a dead
socket. That DBAPIError trips download_source's autoretry_for, the retry
re-enters _phase1_setup, sees the event still 'running', returns
in_flight and no-ops — leaving the event to be stranded empty by the
recovery sweep. pool_pre_ping was already on both engines but can't help
a *held* connection (it only validates on pool checkout).
Fix:
- DownloadService.download_source closes the async + sync sessions after
phase 1, before the subprocess, so phase 3 re-acquires a live
connection (matches the class's "Phase 2 — no DB connection" docstring).
- The per-task async engine switches to NullPool so phase 3 always opens
a fresh connection rather than a pooled one the server may have reaped.
Tests: assert connections are released before gdl.download runs and the
event still finalizes; assert the task engine uses NullPool. Also fixes a
stale 1800s->1170s comment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
download/migration/scan each defined an identical _async_session_factory()
(fresh per-invocation async engine — async connections are event-loop-bound
so each asyncio.run() task needs its own engine, unlike the process-wide
_sync_engine). Moved it to tasks/_async_session.py; the 3 files import it
and drop their now-orphaned sqlalchemy.ext.asyncio / get_config imports
(migration keeps AsyncSession for a type hint). Call-site try/finally
dispose left as-is to avoid re-indenting the critical task bodies.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bvandeusen