feat(downloads): platform cooldown honors server Retry-After — B1 (plan #708)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m1s

Owning the native client means we see the 429 Retry-After header — previously
discarded. PatreonAPIError now carries `retry_after`; on a PERSISTENT page-fetch
429 the client attaches the server's raw Retry-After seconds. New
DownloadResult.retry_after_seconds; patreon_ingester._failure_result sets it on
RATE_LIMITED. download_service._update_source_health passes it to
set_platform_cooldown as `seconds=`, clamped to [60, 3600] (a tiny hint can't
leave the platform effectively un-cooled; a huge one can't strand it for hours);
no hint → the flat 900s default. So a rate-limited platform cools for as long as
the server actually asks, not a fixed guess.

Tests: terminal 429 surfaces retry_after (test_patreon_client); cooldown honors
+ clamps the hint, falls back to default when absent (test_download_service).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-06 11:34:37 -04:00
parent e47fa0cf4b
commit fb7383eea7
6 changed files with 91 additions and 4 deletions
+13 -2
View File
@@ -407,6 +407,7 @@ class DownloadService:
await self._update_source_health(
source_id=ctx["source_id"], status=status, error_message=ev.error,
error_type=dl_result.error_type.value if dl_result.error_type else None,
retry_after_seconds=getattr(dl_result, "retry_after_seconds", None),
)
await self._apply_backfill_lifecycle(ctx, dl_result)
await self.async_session.commit()
@@ -501,7 +502,7 @@ class DownloadService:
async def _update_source_health(
self, *, source_id: int, status: str, error_message: str | None,
error_type: str | None = None,
error_type: str | None = None, retry_after_seconds: float | None = None,
) -> None:
"""FC-3d: update Source.{consecutive_failures, last_error, last_checked_at}.
@@ -533,7 +534,17 @@ class DownloadService:
# by error class without opening Logs per row.
source.error_type = error_type
if error_type == "rate_limited":
await set_platform_cooldown(self.async_session, source.platform)
# plan #708 B1: honor the server's Retry-After when the native
# client surfaced one (clamped to a sane [60, 3600] window so a
# tiny hint can't leave the platform effectively un-cooled and a
# huge one can't strand it for hours); else the flat default.
if retry_after_seconds is not None:
seconds = int(min(max(retry_after_seconds, 60), 3600))
await set_platform_cooldown(
self.async_session, source.platform, seconds=seconds,
)
else:
await set_platform_cooldown(self.async_session, source.platform)
elif status == "skipped":
source.last_error = None
source.last_checked_at = now