feat(sources): pre-flight credential verify on backfill/recovery arm — #703 step 2
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 2m59s

Before arming a deep walk on a native-ingester platform (Patreon — where
verify is one cheap API page), POST /sources/{id}/backfill {start|recover}
runs the shared verify_source_credential first and REFUSES (409 + reason)
only on a definitive rejection (verify→False, e.g. expired cookies). It
proceeds on valid (True) or inconclusive (None — a network blip must not
block). Gated to native platforms: gallery-dl verify is a slow --simulate
subprocess, too heavy for an arm action. The credential read happens in a
session that's CLOSED before the verify network call (no held conn).

Frontend: onBackfill/onRecover now read e.body.detail (ApiError carries the
reason in .body, not .detail) so the rejection text surfaces in the toast.

Tests: arm blocked on rejection (409, source not armed), proceeds on
inconclusive, stop never pre-flights, gallery-dl platform skips pre-flight.
An autouse fixture stubs verify to 'valid' for the existing backfill
endpoint tests so they stay network-free.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-05 23:21:05 -04:00
parent d6c15f4ea0
commit 5b615b7ded
3 changed files with 145 additions and 2 deletions
+40
View File
@@ -130,6 +130,15 @@ async def set_backfill(source_id: int):
current pHash threshold; 'stop' cancels either back to tick mode. Returns the
updated source dict (incl. backfill_state / backfill_chunks /
backfill_bypass_seen)."""
from pathlib import Path
from ..services.credential_service import CredentialService
from ..services.download_backends import (
uses_native_ingester,
verify_source_credential,
)
from .credentials import _get_crypto
payload = await request.get_json(silent=True) or {}
action = payload.get("action", "start")
if action not in ("start", "stop", "recover"):
@@ -137,6 +146,37 @@ async def set_backfill(source_id: int):
"invalid_action",
detail="action must be 'start', 'stop', or 'recover'",
)
# Pre-flight (plan #703 #2): before arming a deep walk on a native-ingester
# platform (where verify is one cheap API page), refuse if the credential is
# DEFINITIVELY rejected — don't burn chunks against expired cookies. Proceed
# on valid OR inconclusive (a network blip shouldn't block). Gated to native
# platforms: gallery-dl verify is a slow --simulate subprocess, too heavy for
# an arm action. The credential read happens in a session that's CLOSED
# before the verify network call (don't hold a DB conn across the request).
if action in ("start", "recover"):
async with get_session() as session:
rec = await SourceService(session).get(source_id)
if rec is None:
return _bad("not_found", status=404)
native = uses_native_ingester(rec.platform)
if native:
cred = CredentialService(session, _get_crypto())
cookies_path = await cred.get_cookies_path(rec.platform)
auth_token = await cred.get_token(rec.platform)
if native:
ok, message = await verify_source_credential(
platform=rec.platform,
url=rec.url,
artist_slug=rec.artist_slug,
config_overrides=rec.config_overrides or {},
cookies_path=str(cookies_path) if cookies_path else None,
auth_token=auth_token,
images_root=Path("/images"),
)
if ok is False:
return _bad("credential_rejected", detail=message, status=409)
async with get_session() as session:
try:
svc = SourceService(session)