feat(pixiv): ledger models + migration 0076 + PixivIngester adapter (#129 step 3)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Failing after 31s
CI / integration (push) Successful in 3m36s

pixiv_seen_media / pixiv_failed_media mirror the Patreon/SubscribeStar
ledgers (keys are always synthesized <illust_id>:p<num> / <illust_id>:ugoira
— pximg URLs carry no content hash). PixivIngester wires client/downloader/
ledgers into ingest_core with drift label 'Pixiv app API' and the new
body_canary=False opt-out: caption-less pixiv artists are common, so the
zero-bodies #862 alarm would false-positive here — the client's
response-shape drift checks cover that failure class instead. auth_token
joins the uniform adapter constructor (pixiv is the first token-auth native
platform). verify_pixiv_credential = one OAuth refresh, no feed walk.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
This commit is contained in:
2026-07-03 09:45:14 -04:00
parent 7ef2ecd82f
commit 0563b2d750
7 changed files with 607 additions and 1 deletions
+12 -1
View File
@@ -90,6 +90,7 @@ class Ingester:
platform: str,
error_base: type[Exception],
drift_label: str | None = None,
body_canary: bool = True,
):
self.client = client
self.downloader = downloader
@@ -105,6 +106,12 @@ class Ingester:
# update"). Defaults to the platform name; adapters pass a richer phrase
# (e.g. "Patreon API", "SubscribeStar markup").
self._drift_label = drift_label or platform
# #862 canary opt-out: platforms whose posts legitimately have empty
# bodies across large samples (pixiv — caption-less artists are common)
# would false-positive the zero-bodies-means-drift alarm; their clients
# catch drift structurally (response-shape checks) instead. The
# "bodies X/N" summary line still surfaces the ratio either way.
self._body_canary = body_canary
# -- public ------------------------------------------------------------
@@ -536,7 +543,11 @@ class Ingester:
# creds") so the breakage screams instead of silently archiving empties.
# Only reached on an otherwise-clean walk (timeout/stop/error returned
# above), so it never masks a more specific failure.
if posts_recorded >= _CANARY_MIN_SAMPLE and posts_with_body == 0:
if (
self._body_canary
and posts_recorded >= _CANARY_MIN_SAMPLE
and posts_with_body == 0
):
msg = (
f"Post-body canary: extracted a body from 0 of {posts_recorded} "
"posts — Patreon's body field shape likely changed; the ingester "