feat(pixiv): ledger models + migration 0076 + PixivIngester adapter (#129 step 3)
pixiv_seen_media / pixiv_failed_media mirror the Patreon/SubscribeStar ledgers (keys are always synthesized <illust_id>:p<num> / <illust_id>:ugoira — pximg URLs carry no content hash). PixivIngester wires client/downloader/ ledgers into ingest_core with drift label 'Pixiv app API' and the new body_canary=False opt-out: caption-less pixiv artists are common, so the zero-bodies #862 alarm would false-positive here — the client's response-shape drift checks cover that failure class instead. auth_token joins the uniform adapter constructor (pixiv is the first token-auth native platform). verify_pixiv_credential = one OAuth refresh, no feed walk. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
This commit is contained in:
@@ -0,0 +1,42 @@
|
||||
"""PixivSeenMedia — per-source ledger of Pixiv media already
|
||||
downloaded+processed.
|
||||
|
||||
Mirror of PatreonSeenMedia/SubscribeStarSeenMedia for the Pixiv native
|
||||
ingester (replacing gallery-dl). One queryable row per (source, media) so
|
||||
routine walks skip media we've already ingested; recovery mode bypasses the
|
||||
ledger to re-walk.
|
||||
|
||||
Pixiv original URLs carry no content hash, so `filehash` is always the
|
||||
synthesized ``<illust_id>:p<num>`` (page) / ``<illust_id>:ugoira`` (frame
|
||||
zip) key — stable across any URL-shape drift. String(128) matches the sibling
|
||||
ledgers.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import ForeignKey, Integer, String, UniqueConstraint, func
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
from sqlalchemy.types import DateTime
|
||||
|
||||
from .base import Base
|
||||
|
||||
|
||||
class PixivSeenMedia(Base):
|
||||
__tablename__ = "pixiv_seen_media"
|
||||
__table_args__ = (
|
||||
# Dedup key the downloader upserts against: one ledger row per
|
||||
# (source, media). A second sighting of the same media is a no-op.
|
||||
UniqueConstraint(
|
||||
"source_id", "filehash", name="uq_pixiv_seen_media_source_id"
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||
source_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("source.id", ondelete="CASCADE"), nullable=False, index=True
|
||||
)
|
||||
filehash: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||
post_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
seen_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||
)
|
||||
Reference in New Issue
Block a user