feat(patreon): drift detection + error categorization — build step 4 (plan #697)

Typed, loud failure mapping for the native Patreon ingester so a changed
API shape or expired auth never silently zero-downloads as "success".

- New ErrorType.API_DRIFT (free varchar error_type col → no migration):
  distinct from auth so the operator knows the fix is updating the
  ingester, not rotating cookies.
- patreon_client: PatreonAPIError carries status_code; new PatreonAuthError
  for 401/403 + HTML-login/non-JSON bodies (reclassified from drift —
  expired-session is auth, actionable as "rotate cookies").
- patreon_ingester._failure_result maps: PatreonAuthError→AUTH_ERROR,
  PatreonDriftError→API_DRIFT ("Patreon API changed — ingester needs
  update"), HTTP 429→RATE_LIMITED, 404→NOT_FOUND, other HTTP→HTTP_ERROR,
  transport→NETWORK_ERROR. (429 thus drives the platform cooldown.)
- FailingSourcesCard: api_drift chip (red) + hint.

Contract test (new test_patreon_contract.py): the recorded /api/posts
fixture must parse end-to-end (no drift, 5 media across 4 posts) AND the
request params must still carry every field the parser depends on
(file_name, image_urls/download_url, images/attachments_media/media
includes, content/post_file/image post fields) — a trim of either trips a
red build. Plus client HTTP-status classification tests and ingester
error-type mapping tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

backend/app/services/gallery_dl.py

@@ -40,6 +40,12 @@ class ErrorType(StrEnum):
     HTTP_ERROR = "http_error"
     UNSUPPORTED_URL = "unsupported_url"
     VALIDATION_FAILED = "validation_failed"
+    # Native Patreon ingester only (plan #697): a response parsed as JSON but
+    # didn't match the JSON:API shape the ingester depends on (missing `data`,
+    # media lacking `file_name`/`url`, etc.). Distinct from AUTH_ERROR — the fix
+    # is updating the ingester's field-set/parser, not rotating credentials. The
+    # contract test guards against silently shipping this.
+    API_DRIFT = "api_drift"
     # Run made real progress (downloaded ≥1 file) but did not finish in the
     # subprocess budget. Distinct from UNKNOWN_ERROR — the downstream status
     # mapping classifies this as "ok" because the next tick continues.

backend/app/services/patreon_client.py

@@ -77,16 +77,34 @@ _MAX_EXT_LEN = 16
 
 
 class PatreonAPIError(Exception):
-    """Base for native Patreon client failures."""
+    """Base for native Patreon client failures.
+
+    `status_code` carries the HTTP status when the failure was an HTTP response
+    (None for transport-level / parse failures), so the ingester can map it to a
+    DownloadResult.error_type (429 → rate_limited, 404 → not_found, …).
+    """
+
+    def __init__(self, message: str, *, status_code: int | None = None):
+        super().__init__(message)
+        self.status_code = status_code
+
+
+class PatreonAuthError(PatreonAPIError):
+    """Authentication / authorization failure — missing or expired session
+    cookies, an insufficient pledge tier, or an HTML login/challenge page served
+    where JSON was expected. DISTINCT from drift: the fix is rotating the
+    credential, not updating the ingester. Maps to error_type 'auth_error'.
+    """
 
 
 class PatreonDriftError(PatreonAPIError):
-    """A response did not match the JSON:API shape we depend on.
+    """A JSON response did not match the JSON:API shape we depend on.
 
-    Raised for: a non-JSON / HTML-login response where JSON was expected, a
-    missing top-level `data` list, or a media resource lacking the
-    `file_name`/`url` fields we resolve against. Fail loud so the later import
-    step flags API drift instead of silently importing nothing.
+    Raised for: a missing top-level `data` list, `data` not a list, or a media
+    resource lacking the `file_name`/`url` fields we resolve against. Fail loud
+    so the import step flags API drift (ingester needs update) instead of
+    silently importing nothing. An HTML-login / non-JSON body is auth, not
+    drift — that raises PatreonAuthError.
     """
 
 
@@ -221,20 +239,32 @@ class PatreonClient:
                 f"Patreon posts request failed (campaign_id={campaign_id}): {exc}"
             ) from exc
 
+        if resp.status_code in (401, 403):
+            # Auth rejected — expired/missing cookies or an insufficient tier.
+            # Actionable as "rotate credentials", so it's auth, not drift/http.
+            raise PatreonAuthError(
+                f"Patreon posts API returned HTTP {resp.status_code} — auth "
+                f"rejected (cookies expired or tier insufficient; "
+                f"campaign_id={campaign_id})",
+                status_code=resp.status_code,
+            )
         if resp.status_code != 200:
             raise PatreonAPIError(
                 f"Patreon posts API returned HTTP {resp.status_code} "
-                f"(campaign_id={campaign_id})"
+                f"(campaign_id={campaign_id})",
+                status_code=resp.status_code,
             )
         try:
             payload = resp.json()
         except ValueError as exc:
             # A non-JSON body here is almost always the HTML login/challenge
-            # page served when cookies are missing/expired — that is drift
-            # from the JSON:API contract, not a transient network error.
-            raise PatreonDriftError(
+            # page served when cookies are missing/expired — that is an AUTH
+            # failure (rotate cookies), not API drift (update the ingester) and
+            # not a transient network error.
+            raise PatreonAuthError(
                 "Patreon posts API returned a non-JSON response (likely an "
-                f"HTML login/challenge page; campaign_id={campaign_id}): {exc}"
+                f"HTML login/challenge page — session expired; "
+                f"campaign_id={campaign_id}): {exc}"
             ) from exc
         return payload
 

backend/app/services/patreon_ingester.py

@@ -51,6 +51,7 @@ from .gallery_dl import DownloadResult, ErrorType
 from .patreon_client import (
     MediaItem,
     PatreonAPIError,
+    PatreonAuthError,
     PatreonClient,
     PatreonDriftError,
 )
@@ -242,7 +243,9 @@ class PatreonIngester:
                     break
             else:
                 reached_bottom = True
-        except (PatreonDriftError, PatreonAPIError) as exc:
+        except PatreonAPIError as exc:
+            # Base of PatreonAuthError + PatreonDriftError — catches every
+            # client-level failure; _failure_result maps it to a typed error.
             return self._failure_result(exc, _result)
 
         if errors:
@@ -294,24 +297,35 @@ class PatreonIngester:
     def _failure_result(self, exc: Exception, _result) -> DownloadResult:
         """Map a client-level exception to a loud, typed failed DownloadResult.
 
-        Step 3 keeps this deliberately coarse — auth-ish drift vs. everything
-        else — so the integration is correct end-to-end. Step 4 (drift detection
-        + error categorization) refines the typed-error mapping and adds the
-        contract test. Either way we NEVER return a silent zero-download
-        "success": the whole point of the native ingester is to fail red when
-        Patreon's API shape or our auth changes.
+        We NEVER return a silent zero-download "success" — the whole point of the
+        native ingester is to fail RED when Patreon's API shape or our auth
+        changes. The typed mapping lets FailingSourcesCard render the right chip
+        and tells the operator what to do:
+          - PatreonAuthError  → AUTH_ERROR  (rotate cookies)
+          - PatreonDriftError → API_DRIFT   (ingester field-set/parser needs update)
+          - HTTP 429 / 404    → RATE_LIMITED / NOT_FOUND
+          - other HTTP status → HTTP_ERROR;  transport failure → NETWORK_ERROR
+
+        PatreonAuthError and PatreonDriftError both subclass PatreonAPIError, so
+        they must be matched before the generic HTTP/transport fallthrough.
         """
         message = str(exc)
-        lowered = message.lower()
-        if isinstance(exc, PatreonDriftError):
-            if "login" in lowered or "challenge" in lowered or "auth" in lowered:
-                error_type = ErrorType.AUTH_ERROR
-            else:
-                error_type = ErrorType.UNKNOWN_ERROR
+        if isinstance(exc, PatreonAuthError):
+            error_type = ErrorType.AUTH_ERROR
+        elif isinstance(exc, PatreonDriftError):
+            error_type = ErrorType.API_DRIFT
             message = f"Patreon API changed — ingester needs update: {message}"
-        else:
-            error_type = ErrorType.NETWORK_ERROR
-        log.warning("Patreon ingest failed: %s", message)
+        else:  # generic PatreonAPIError: HTTP non-2xx (status_code set) or transport
+            status = getattr(exc, "status_code", None)
+            if status == 429:
+                error_type = ErrorType.RATE_LIMITED
+            elif status == 404:
+                error_type = ErrorType.NOT_FOUND
+            elif status is not None:
+                error_type = ErrorType.HTTP_ERROR
+            else:
+                error_type = ErrorType.NETWORK_ERROR
+        log.warning("Patreon ingest failed (%s): %s", error_type.value, message)
         return _result(
             success=False, return_code=1,
             error_type=error_type, error_message=message,