feat(patreon): drift detection + error categorization — build step 4 (plan #697)
Typed, loud failure mapping for the native Patreon ingester so a changed
API shape or expired auth never silently zero-downloads as "success".
- New ErrorType.API_DRIFT (free varchar error_type col → no migration):
distinct from auth so the operator knows the fix is updating the
ingester, not rotating cookies.
- patreon_client: PatreonAPIError carries status_code; new PatreonAuthError
for 401/403 + HTML-login/non-JSON bodies (reclassified from drift —
expired-session is auth, actionable as "rotate cookies").
- patreon_ingester._failure_result maps: PatreonAuthError→AUTH_ERROR,
PatreonDriftError→API_DRIFT ("Patreon API changed — ingester needs
update"), HTTP 429→RATE_LIMITED, 404→NOT_FOUND, other HTTP→HTTP_ERROR,
transport→NETWORK_ERROR. (429 thus drives the platform cooldown.)
- FailingSourcesCard: api_drift chip (red) + hint.
Contract test (new test_patreon_contract.py): the recorded /api/posts
fixture must parse end-to-end (no drift, 5 media across 4 posts) AND the
request params must still carry every field the parser depends on
(file_name, image_urls/download_url, images/attachments_media/media
includes, content/post_file/image post fields) — a trim of either trips a
red build. Plus client HTTP-status classification tests and ingester
error-type mapping tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -77,16 +77,34 @@ _MAX_EXT_LEN = 16
|
||||
|
||||
|
||||
class PatreonAPIError(Exception):
|
||||
"""Base for native Patreon client failures."""
|
||||
"""Base for native Patreon client failures.
|
||||
|
||||
`status_code` carries the HTTP status when the failure was an HTTP response
|
||||
(None for transport-level / parse failures), so the ingester can map it to a
|
||||
DownloadResult.error_type (429 → rate_limited, 404 → not_found, …).
|
||||
"""
|
||||
|
||||
def __init__(self, message: str, *, status_code: int | None = None):
|
||||
super().__init__(message)
|
||||
self.status_code = status_code
|
||||
|
||||
|
||||
class PatreonAuthError(PatreonAPIError):
|
||||
"""Authentication / authorization failure — missing or expired session
|
||||
cookies, an insufficient pledge tier, or an HTML login/challenge page served
|
||||
where JSON was expected. DISTINCT from drift: the fix is rotating the
|
||||
credential, not updating the ingester. Maps to error_type 'auth_error'.
|
||||
"""
|
||||
|
||||
|
||||
class PatreonDriftError(PatreonAPIError):
|
||||
"""A response did not match the JSON:API shape we depend on.
|
||||
"""A JSON response did not match the JSON:API shape we depend on.
|
||||
|
||||
Raised for: a non-JSON / HTML-login response where JSON was expected, a
|
||||
missing top-level `data` list, or a media resource lacking the
|
||||
`file_name`/`url` fields we resolve against. Fail loud so the later import
|
||||
step flags API drift instead of silently importing nothing.
|
||||
Raised for: a missing top-level `data` list, `data` not a list, or a media
|
||||
resource lacking the `file_name`/`url` fields we resolve against. Fail loud
|
||||
so the import step flags API drift (ingester needs update) instead of
|
||||
silently importing nothing. An HTML-login / non-JSON body is auth, not
|
||||
drift — that raises PatreonAuthError.
|
||||
"""
|
||||
|
||||
|
||||
@@ -221,20 +239,32 @@ class PatreonClient:
|
||||
f"Patreon posts request failed (campaign_id={campaign_id}): {exc}"
|
||||
) from exc
|
||||
|
||||
if resp.status_code in (401, 403):
|
||||
# Auth rejected — expired/missing cookies or an insufficient tier.
|
||||
# Actionable as "rotate credentials", so it's auth, not drift/http.
|
||||
raise PatreonAuthError(
|
||||
f"Patreon posts API returned HTTP {resp.status_code} — auth "
|
||||
f"rejected (cookies expired or tier insufficient; "
|
||||
f"campaign_id={campaign_id})",
|
||||
status_code=resp.status_code,
|
||||
)
|
||||
if resp.status_code != 200:
|
||||
raise PatreonAPIError(
|
||||
f"Patreon posts API returned HTTP {resp.status_code} "
|
||||
f"(campaign_id={campaign_id})"
|
||||
f"(campaign_id={campaign_id})",
|
||||
status_code=resp.status_code,
|
||||
)
|
||||
try:
|
||||
payload = resp.json()
|
||||
except ValueError as exc:
|
||||
# A non-JSON body here is almost always the HTML login/challenge
|
||||
# page served when cookies are missing/expired — that is drift
|
||||
# from the JSON:API contract, not a transient network error.
|
||||
raise PatreonDriftError(
|
||||
# page served when cookies are missing/expired — that is an AUTH
|
||||
# failure (rotate cookies), not API drift (update the ingester) and
|
||||
# not a transient network error.
|
||||
raise PatreonAuthError(
|
||||
"Patreon posts API returned a non-JSON response (likely an "
|
||||
f"HTML login/challenge page; campaign_id={campaign_id}): {exc}"
|
||||
f"HTML login/challenge page — session expired; "
|
||||
f"campaign_id={campaign_id}): {exc}"
|
||||
) from exc
|
||||
return payload
|
||||
|
||||
|
||||
Reference in New Issue
Block a user