feat(downloads): platform cooldown honors server Retry-After — B1 (plan #708)
Owning the native client means we see the 429 Retry-After header — previously discarded. PatreonAPIError now carries `retry_after`; on a PERSISTENT page-fetch 429 the client attaches the server's raw Retry-After seconds. New DownloadResult.retry_after_seconds; patreon_ingester._failure_result sets it on RATE_LIMITED. download_service._update_source_health passes it to set_platform_cooldown as `seconds=`, clamped to [60, 3600] (a tiny hint can't leave the platform effectively un-cooled; a huge one can't strand it for hours); no hint → the flat 900s default. So a rate-limited platform cools for as long as the server actually asks, not a fixed guess. Tests: terminal 429 surfaces retry_after (test_patreon_client); cooldown honors + clamps the hint, falls back to default when absent (test_download_service). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -407,6 +407,7 @@ class DownloadService:
|
||||
await self._update_source_health(
|
||||
source_id=ctx["source_id"], status=status, error_message=ev.error,
|
||||
error_type=dl_result.error_type.value if dl_result.error_type else None,
|
||||
retry_after_seconds=getattr(dl_result, "retry_after_seconds", None),
|
||||
)
|
||||
await self._apply_backfill_lifecycle(ctx, dl_result)
|
||||
await self.async_session.commit()
|
||||
@@ -501,7 +502,7 @@ class DownloadService:
|
||||
|
||||
async def _update_source_health(
|
||||
self, *, source_id: int, status: str, error_message: str | None,
|
||||
error_type: str | None = None,
|
||||
error_type: str | None = None, retry_after_seconds: float | None = None,
|
||||
) -> None:
|
||||
"""FC-3d: update Source.{consecutive_failures, last_error, last_checked_at}.
|
||||
|
||||
@@ -533,7 +534,17 @@ class DownloadService:
|
||||
# by error class without opening Logs per row.
|
||||
source.error_type = error_type
|
||||
if error_type == "rate_limited":
|
||||
await set_platform_cooldown(self.async_session, source.platform)
|
||||
# plan #708 B1: honor the server's Retry-After when the native
|
||||
# client surfaced one (clamped to a sane [60, 3600] window so a
|
||||
# tiny hint can't leave the platform effectively un-cooled and a
|
||||
# huge one can't strand it for hours); else the flat default.
|
||||
if retry_after_seconds is not None:
|
||||
seconds = int(min(max(retry_after_seconds, 60), 3600))
|
||||
await set_platform_cooldown(
|
||||
self.async_session, source.platform, seconds=seconds,
|
||||
)
|
||||
else:
|
||||
await set_platform_cooldown(self.async_session, source.platform)
|
||||
elif status == "skipped":
|
||||
source.last_error = None
|
||||
source.last_checked_at = now
|
||||
|
||||
Reference in New Issue
Block a user