feat(patreon): drift detection + error categorization — build step 4 (plan #697)
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 12s
CI / frontend-build (push) Successful in 28s
CI / integration (push) Successful in 2m58s

Typed, loud failure mapping for the native Patreon ingester so a changed
API shape or expired auth never silently zero-downloads as "success".

- New ErrorType.API_DRIFT (free varchar error_type col → no migration):
  distinct from auth so the operator knows the fix is updating the
  ingester, not rotating cookies.
- patreon_client: PatreonAPIError carries status_code; new PatreonAuthError
  for 401/403 + HTML-login/non-JSON bodies (reclassified from drift —
  expired-session is auth, actionable as "rotate cookies").
- patreon_ingester._failure_result maps: PatreonAuthError→AUTH_ERROR,
  PatreonDriftError→API_DRIFT ("Patreon API changed — ingester needs
  update"), HTTP 429→RATE_LIMITED, 404→NOT_FOUND, other HTTP→HTTP_ERROR,
  transport→NETWORK_ERROR. (429 thus drives the platform cooldown.)
- FailingSourcesCard: api_drift chip (red) + hint.

Contract test (new test_patreon_contract.py): the recorded /api/posts
fixture must parse end-to-end (no drift, 5 media across 4 posts) AND the
request params must still carry every field the parser depends on
(file_name, image_urls/download_url, images/attachments_media/media
includes, content/post_file/image post fields) — a trim of either trips a
red build. Plus client HTTP-status classification tests and ingester
error-type mapping tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-05 21:56:58 -04:00
parent 96c30eba13
commit 682beafbc5
7 changed files with 261 additions and 35 deletions
+41 -11
View File
@@ -77,16 +77,34 @@ _MAX_EXT_LEN = 16
class PatreonAPIError(Exception):
"""Base for native Patreon client failures."""
"""Base for native Patreon client failures.
`status_code` carries the HTTP status when the failure was an HTTP response
(None for transport-level / parse failures), so the ingester can map it to a
DownloadResult.error_type (429 → rate_limited, 404 → not_found, …).
"""
def __init__(self, message: str, *, status_code: int | None = None):
super().__init__(message)
self.status_code = status_code
class PatreonAuthError(PatreonAPIError):
"""Authentication / authorization failure — missing or expired session
cookies, an insufficient pledge tier, or an HTML login/challenge page served
where JSON was expected. DISTINCT from drift: the fix is rotating the
credential, not updating the ingester. Maps to error_type 'auth_error'.
"""
class PatreonDriftError(PatreonAPIError):
"""A response did not match the JSON:API shape we depend on.
"""A JSON response did not match the JSON:API shape we depend on.
Raised for: a non-JSON / HTML-login response where JSON was expected, a
missing top-level `data` list, or a media resource lacking the
`file_name`/`url` fields we resolve against. Fail loud so the later import
step flags API drift instead of silently importing nothing.
Raised for: a missing top-level `data` list, `data` not a list, or a media
resource lacking the `file_name`/`url` fields we resolve against. Fail loud
so the import step flags API drift (ingester needs update) instead of
silently importing nothing. An HTML-login / non-JSON body is auth, not
drift — that raises PatreonAuthError.
"""
@@ -221,20 +239,32 @@ class PatreonClient:
f"Patreon posts request failed (campaign_id={campaign_id}): {exc}"
) from exc
if resp.status_code in (401, 403):
# Auth rejected — expired/missing cookies or an insufficient tier.
# Actionable as "rotate credentials", so it's auth, not drift/http.
raise PatreonAuthError(
f"Patreon posts API returned HTTP {resp.status_code} — auth "
f"rejected (cookies expired or tier insufficient; "
f"campaign_id={campaign_id})",
status_code=resp.status_code,
)
if resp.status_code != 200:
raise PatreonAPIError(
f"Patreon posts API returned HTTP {resp.status_code} "
f"(campaign_id={campaign_id})"
f"(campaign_id={campaign_id})",
status_code=resp.status_code,
)
try:
payload = resp.json()
except ValueError as exc:
# A non-JSON body here is almost always the HTML login/challenge
# page served when cookies are missing/expired — that is drift
# from the JSON:API contract, not a transient network error.
raise PatreonDriftError(
# page served when cookies are missing/expired — that is an AUTH
# failure (rotate cookies), not API drift (update the ingester) and
# not a transient network error.
raise PatreonAuthError(
"Patreon posts API returned a non-JSON response (likely an "
f"HTML login/challenge page; campaign_id={campaign_id}): {exc}"
f"HTML login/challenge page — session expired; "
f"campaign_id={campaign_id}): {exc}"
) from exc
return payload