bvandeusen
0563b2d750
pixiv_seen_media / pixiv_failed_media mirror the Patreon/SubscribeStar ledgers (keys are always synthesized <illust_id>:p<num> / <illust_id>:ugoira — pximg URLs carry no content hash). PixivIngester wires client/downloader/ ledgers into ingest_core with drift label 'Pixiv app API' and the new body_canary=False opt-out: caption-less pixiv artists are common, so the zero-bodies #862 alarm would false-positive here — the client's response-shape drift checks cover that failure class instead. auth_token joins the uniform adapter constructor (pixiv is the first token-auth native platform). verify_pixiv_credential = one OAuth refresh, no feed walk. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
43 lines
1.6 KiB
Python
43 lines
1.6 KiB
Python
"""PixivSeenMedia — per-source ledger of Pixiv media already
|
|
downloaded+processed.
|
|
|
|
Mirror of PatreonSeenMedia/SubscribeStarSeenMedia for the Pixiv native
|
|
ingester (replacing gallery-dl). One queryable row per (source, media) so
|
|
routine walks skip media we've already ingested; recovery mode bypasses the
|
|
ledger to re-walk.
|
|
|
|
Pixiv original URLs carry no content hash, so `filehash` is always the
|
|
synthesized ``<illust_id>:p<num>`` (page) / ``<illust_id>:ugoira`` (frame
|
|
zip) key — stable across any URL-shape drift. String(128) matches the sibling
|
|
ledgers.
|
|
"""
|
|
|
|
from datetime import datetime
|
|
|
|
from sqlalchemy import ForeignKey, Integer, String, UniqueConstraint, func
|
|
from sqlalchemy.orm import Mapped, mapped_column
|
|
from sqlalchemy.types import DateTime
|
|
|
|
from .base import Base
|
|
|
|
|
|
class PixivSeenMedia(Base):
|
|
__tablename__ = "pixiv_seen_media"
|
|
__table_args__ = (
|
|
# Dedup key the downloader upserts against: one ledger row per
|
|
# (source, media). A second sighting of the same media is a no-op.
|
|
UniqueConstraint(
|
|
"source_id", "filehash", name="uq_pixiv_seen_media_source_id"
|
|
),
|
|
)
|
|
|
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
|
source_id: Mapped[int] = mapped_column(
|
|
ForeignKey("source.id", ondelete="CASCADE"), nullable=False, index=True
|
|
)
|
|
filehash: Mapped[str] = mapped_column(String(128), nullable=False)
|
|
post_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
seen_at: Mapped[datetime] = mapped_column(
|
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
|
)
|