feat(downloads): live per-file progress on running events — #709
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Failing after 3m1s

Now that we own the walk, surface live counts on the in-flight download in the
Downloads view. ingest_core.run takes an event_id and does a TIME-THROTTLED
write (~5s, decoupled from page boundaries so it ticks steadily regardless of
how big/slow a page is) of {downloaded, skipped, errors, quarantined, posts} to
the running download_event's metadata.live (jsonb_set; short session; status
guard so a finalized event isn't clobbered). download_backends threads
event_id from ctx; the /api/downloads list surfaces `live`; ActiveDownloadsPanel
renders it beside the elapsed timer. Native (Patreon) only — gallery-dl is an
opaque subprocess; the row only shows when `live` is present. Phase 3 overwrites
metadata with run_stats on finish, dropping `live`.

Test: _write_live_progress updates a running event's metadata.live and leaves a
finalized (status != running) event alone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-06 15:48:31 -04:00
parent 89dfa42e18
commit 62cca64dce
5 changed files with 93 additions and 0 deletions
+28
View File
@@ -421,6 +421,34 @@ async def test_preview_page_limit_caps_walk(source_id, sync_engine, tmp_path):
assert result["has_more"] is True
@pytest.mark.asyncio
async def test_write_live_progress_updates_running_event(
source_id, sync_engine, tmp_path, db,
):
"""plan #709: live counts land in the RUNNING event's metadata.live; a
finalized event is left alone (status guard)."""
from backend.app.models import DownloadEvent
ing = _ingester(sync_engine, tmp_path, _FakeClient([]), _FakeDownloader(tmp_path))
running = DownloadEvent(source_id=source_id, status="running")
done = DownloadEvent(source_id=source_id, status="ok")
db.add_all([running, done])
await db.commit()
counts = {"downloaded": 3, "skipped": 1, "errors": 0, "quarantined": 0, "posts": 4}
ing._write_live_progress(running.id, counts)
ing._write_live_progress(done.id, {"downloaded": 9}) # guarded: status != running
live = (await db.execute(
select(DownloadEvent.metadata_).where(DownloadEvent.id == running.id)
)).scalar_one()
assert live["live"] == counts
done_meta = (await db.execute(
select(DownloadEvent.metadata_).where(DownloadEvent.id == done.id)
)).scalar_one()
assert "live" not in (done_meta or {})
@pytest.mark.asyncio
async def test_still_running_reads_backfill_state(source_id, sync_engine, tmp_path, db):
"""plan #708 B4: _still_running reflects the source's `_backfill_state` — the