Commit Graph

3 Commits

Author SHA1 Message Date
bvandeusen 682beafbc5 feat(patreon): drift detection + error categorization — build step 4 (plan #697)
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 12s
CI / frontend-build (push) Successful in 28s
CI / integration (push) Successful in 2m58s
Typed, loud failure mapping for the native Patreon ingester so a changed
API shape or expired auth never silently zero-downloads as "success".

- New ErrorType.API_DRIFT (free varchar error_type col → no migration):
  distinct from auth so the operator knows the fix is updating the
  ingester, not rotating cookies.
- patreon_client: PatreonAPIError carries status_code; new PatreonAuthError
  for 401/403 + HTML-login/non-JSON bodies (reclassified from drift —
  expired-session is auth, actionable as "rotate cookies").
- patreon_ingester._failure_result maps: PatreonAuthError→AUTH_ERROR,
  PatreonDriftError→API_DRIFT ("Patreon API changed — ingester needs
  update"), HTTP 429→RATE_LIMITED, 404→NOT_FOUND, other HTTP→HTTP_ERROR,
  transport→NETWORK_ERROR. (429 thus drives the platform cooldown.)
- FailingSourcesCard: api_drift chip (red) + hint.

Contract test (new test_patreon_contract.py): the recorded /api/posts
fixture must parse end-to-end (no drift, 5 media across 4 posts) AND the
request params must still carry every field the parser depends on
(file_name, image_urls/download_url, images/attachments_media/media
includes, content/post_file/image post fields) — a trim of either trips a
red build. Plus client HTTP-status classification tests and ingester
error-type mapping tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 21:56:58 -04:00
bvandeusen 1bdaa04aa2 test(patreon): fix self-contradictory attachment/postfile dedup assertion
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 13s
CI / frontend-build (push) Successful in 29s
CI / integration (push) Successful in 2m57s
The fixture gives the attachment and post_file the same filehash, so they
correctly collapse to one item; the test asserted both survival and collapse.
Rewrite to verify the cross-kind dedup (postfile kind covered by video test).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 19:17:56 -04:00
bvandeusen 1c2dc7659a feat(patreon): native JSON-API client — ingester build step 1 (plan #697)
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Failing after 12s
CI / frontend-build (push) Successful in 19s
CI / integration (push) Successful in 2m58s
PatreonClient: cookie-auth requests session, /api/posts cursor pagination,
JSON:API included flattening, per-post media extraction (images/image_large/
attachments/postfile/content) with filehash dedup, loud drift detection.
Zero per-file HEADs — every media URL+file_name comes from the API. Not yet
wired into download_service (later step). Pure-parsing unit tests + fixture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 19:16:20 -04:00