feat: retire pixiv entirely — delete its code, its ledgers, its credential (3977, 3978, 3979)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 4s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 25s
extension / lint (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 34s
Build images / build-web (push) Successful in 1m5s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m54s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m19s

Milestone #406 phase 2, with issue #3980 folded in. Phase 1 (2026-09-13)
unregistered pixiv so nothing could reach it; the code has sat in the tree
uncalled since. DeviantArt is why the second half is not left for later —
#3069 retired it in code on 2026-08-27 and its stored session was still in
the database seven weeks on.

Step 5 — the code. Deletes pixiv_client, pixiv_downloader, pixiv_ingester,
platforms/pixiv and their three test modules and fixture, then edits out
every remaining reference: the dispatch entry, the campaign-id and verify
branches in download_backends, the display-name branch in extension_service,
and the comments that still described pixiv as live.

The consolidation check the step asked for comes back negative:
native_ingest_common has seven non-pixiv callers (patreon, subscribestar,
membership_reconcile, membership_roster, ingest_core), so nothing there
drops to a single user.

Step 6 — the data, alembic 0102. Drops pixiv_seen_media and
pixiv_failed_media, and deletes credential rows whose platform is not
registered. Written as "not registered" rather than "pixiv" at the step's
explicit ask, which is what makes one migration cover two retirements: the
pixiv OAuth refresh token and DeviantArt's leftover session (#3980). It is
also the only way either row can go — the credentials UI renders one card
per platform from /api/platforms and looks the credential up by key, so an
unregistered platform's row has no card and no Remove button.

Pixiv's Source rows are KEPT, changing the milestone's original data table
on the operator's call. `platform` is stored only on Source; neither Post
nor ImageRecord carries it. Both FKs are ON DELETE SET NULL, so a delete
would not lose the art — but it would drop every pixiv image into the
gallery's __unsourced__ bucket and strip the platform chip off every pixiv
post. The rows stay disabled (0097) and unregistered, so nothing schedules
or downloads through them. Keeping them costs nothing and keeps the
attribution that "the art already downloaded from pixiv stays" is about.

Step 7 — the guard. test_pixiv_code_and_tables_are_gone asserts absence from
the module table and from Base.metadata, not from prose (snippet #3352's
trap). The extension and registry negative assertions were already in place
from phase 1. The final sweep found one real residue step 4 missed:
extension/README.md still advertised pixiv support and carried a "Pixiv
OAuth" manual-test item.

Also replaces the two deleted dispatch tests with one over the whole
NATIVE_INGESTER_PLATFORMS set, so adding a platform and forgetting its
ingester class now fails at unit level rather than as a mid-download
KeyError.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
2026-09-21 20:46:15 -04:00
co-authored by Claude Opus 5
parent 5aa8e3d81b
commit 24b10d0ffa
23 changed files with 213 additions and 2340 deletions
@@ -0,0 +1,138 @@
"""Drop pixiv's ledgers, and delete credentials for platforms FC no longer has.
Milestone #406 step 6 (with issue #3980 folded in). Phase 1 unregistered pixiv
and the commit alongside this one deleted its client, downloader, ingester and
models. This removes the data those models described, and the stored secrets of
every platform that has been retired.
## The two ledger tables
`pixiv_seen_media` and `pixiv_failed_media` are the per-source seen / dead-letter
ledgers for a downloader that no longer exists. They were created in
`0089_baseline.py`, so dropping them needs a new revision rather than an edit
there.
## The credentials
Written as *delete every credential whose platform is not registered* rather
than as `platform = 'pixiv'`, at the explicit ask in this step's plan. That is
what makes one migration cover two retirements:
- **pixiv** — a live OAuth refresh token for a service FC no longer talks to.
- **deviantart** — issue #3980. #3069 retired DeviantArt in code on 2026-08-27
and left its stored session behind; seven weeks later it was still there.
And it is the only way either row can go. The credentials UI
(`subscriptions/SettingsTab.vue`) renders one card per platform returned by
`/api/platforms`, then looks the credential up by key — so a row whose platform
is unregistered has no card, no Remove button, and no way for the operator to
reach it. `CredentialService.list()` would return it; nothing asks.
The registered set is written out literally instead of importing
`known_platform_keys()`. A migration is a statement about one moment in the
schema's history: if it imported the live registry, retiring a fifth platform
in 2027 would silently change what this 2026 revision did on a fresh database.
The list below is the registry as of 2026-09-21.
## What is deliberately kept
**Every pixiv `Source` row.** The original plan deleted them; the operator's
call on 2026-09-21 was to keep them, and the reason is that `platform` is
stored ONLY on `Source` — neither `Post` nor `ImageRecord` carries it. Both
FKs are `ON DELETE SET NULL`, so a delete would not lose the art, but it would
drop every pixiv image into the gallery's `__unsourced__` bucket and strip the
platform chip off every pixiv post. The rows stay disabled (0097) and their
platform is unregistered, so nothing schedules them, nothing downloads through
them, and `POST /api/sources` will not make another. Keeping them costs
nothing and keeps the attribution the milestone's goal — *"the art already
downloaded from pixiv stays"* — is actually about.
Every pixiv `Post` and `ImageRecord` is likewise untouched.
Revision ID: 0102
Revises: 0101
Create Date: 2026-09-21
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0102"
down_revision: Union[str, None] = "0101"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
# services/platforms/__init__.py's PLATFORMS as of this revision. See the
# docstring for why this is a literal and not an import.
_REGISTERED_PLATFORMS = ("patreon", "subscribestar", "hentaifoundry", "discord")
def upgrade() -> None:
op.execute(
sa.text(
"DELETE FROM credential WHERE platform NOT IN :registered"
).bindparams(
sa.bindparam("registered", value=_REGISTERED_PLATFORMS, expanding=True)
)
)
op.drop_index("ix_pixiv_failed_media_source_id", table_name="pixiv_failed_media")
op.drop_table("pixiv_failed_media")
op.drop_index("ix_pixiv_seen_media_source_id", table_name="pixiv_seen_media")
op.drop_table("pixiv_seen_media")
def downgrade() -> None:
# The tables come back empty, and the credentials do not come back at all:
# they were encrypted blobs, this migration does not copy them anywhere,
# and restoring a live token for a platform FC cannot talk to would only
# re-create the liability. Rule #22 owes no story backwards.
op.create_table(
"pixiv_seen_media",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("source_id", sa.Integer(), nullable=False),
sa.Column("filehash", sa.String(length=128), nullable=False),
sa.Column("url", sa.Text(), nullable=True),
sa.Column(
"created_at", sa.DateTime(timezone=True),
server_default=sa.text("now()"), nullable=False,
),
sa.ForeignKeyConstraint(
["source_id"], ["source.id"],
name=op.f("fk_pixiv_seen_media_source_id_source"), ondelete="CASCADE",
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_pixiv_seen_media")),
sa.UniqueConstraint(
"source_id", "filehash", name="uq_pixiv_seen_media_source_id",
),
)
op.create_index(
op.f("ix_pixiv_seen_media_source_id"), "pixiv_seen_media", ["source_id"],
unique=False,
)
op.create_table(
"pixiv_failed_media",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("source_id", sa.Integer(), nullable=False),
sa.Column("filehash", sa.String(length=128), nullable=False),
sa.Column("url", sa.Text(), nullable=True),
sa.Column("error", sa.Text(), nullable=True),
sa.Column("attempts", sa.Integer(), server_default="1", nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True),
server_default=sa.text("now()"), nullable=False,
),
sa.ForeignKeyConstraint(
["source_id"], ["source.id"],
name=op.f("fk_pixiv_failed_media_source_id_source"), ondelete="CASCADE",
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_pixiv_failed_media")),
sa.UniqueConstraint(
"source_id", "filehash", name="uq_pixiv_failed_media_source_id",
),
)
op.create_index(
op.f("ix_pixiv_failed_media_source_id"), "pixiv_failed_media", ["source_id"],
unique=False,
)