Commit Graph

2 Commits

Author SHA1 Message Date
bvandeusen ebe6ab9741 refactor(native-ingest): shared exception trio + base _failure_result (#899 DRY 2/3)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m18s
DRY pass commit 2. The two adapters re-implemented the same auth→drift→429→404
→http→network mapping in _failure_result; only the exception classes + drift
phrasing differed (divergence-bug risk: a new error_type handled in one and not
the other).

- native_ingest_common gains NativeIngestError / NativeAuthError / NativeDriftError
  (status_code + retry_after on the base). Patreon{API,Auth,Drift}Error and
  SubscribeStar{API,Auth,Drift}Error now subclass them via multiple inheritance,
  keeping their isinstance-distinct platform names.
- Ingester._failure_result (base) does the whole mapping via the shared
  NativeAuthError/NativeDriftError taxonomy + status_code; a new platform gets it
  free. New drift_label kwarg supplies the per-platform API_DRIFT phrasing
  ("Patreon API" / "SubscribeStar markup"), preserving the existing message
  (test asserts "Patreon API changed").
- Both adapters drop their near-identical _failure_result overrides and their now
  -unused DownloadResult/ErrorType/*Auth/*Drift imports.

Verified at every consumer (rule 93/§8b): test_patreon_ingester (auth/drift/429/
404/network) and test_subscribestar_native (_failure_result mapping) both exercise
the base method now. Remaining: ingest_core L1/L3 logging (3/3).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:40:36 -04:00
bvandeusen 817a002c2b feat(subscribestar): native client + downloader + ingester (post-first) (#890/#891/#892)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m23s
Phase-1 steps 2-4 of moving SubscribeStar off gallery-dl onto the native core
ingester. SubscribeStar has no JSON:API, so the client scrapes HTML; the
platform-agnostic core (ingest_core) is unchanged.

- subscribestar_client.py: HTML-scrape read path. iter_posts pages via the
  creator page → infinite_scroll-next_page href → JSON {html} fragments
  (campaign_id = creator URL; no resolver). extract_media reads the per-post
  data-gallery JSON manifest (id/original_filename/type/url). post_record_key,
  post_meta, and post_is_gated (best-effort locked-teaser marker, pending a live
  locked sample). Loud auth/drift taxonomy (SubscribeStar{API,Auth,Drift}Error).
  Parser validated against the real Step-0 fixtures.
- subscribestar_downloader.py: mirrors PatreonDownloader minus the Mux/yt-dlp
  branch (SubscribeStar serves files directly via /post_uploads). gallery-dl
  on-disk layout so existing downloads dedup on disk at cutover. Post-first:
  _post.json owns the body/links; per-media sidecar carries image identity only.
- subscribestar_ingester.py: thin adapter wiring client/downloader/the
  SubscribeStar ledgers into the core; ledger_key = filehash else
  post_id:media_id; SubscribeStar failure mapping. verify_subscribestar_credential.
- tests: client parsing/pagination/media/gating/record-key/dates, downloader
  layout/sidecar/post-record/skip-seen, ingester ledger_key + failure mapping.

Not yet wired into dispatch (Step 5) — these modules are inert until then.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:06:02 -04:00