feat: retire pixiv entirely — delete its code, its ledgers, its credential (3977, 3978, 3979)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 4s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 25s
extension / lint (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 34s
Build images / build-web (push) Successful in 1m5s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m54s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m19s
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 4s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 25s
extension / lint (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 34s
Build images / build-web (push) Successful in 1m5s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m54s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m19s
Milestone #406 phase 2, with issue #3980 folded in. Phase 1 (2026-09-13) unregistered pixiv so nothing could reach it; the code has sat in the tree uncalled since. DeviantArt is why the second half is not left for later — #3069 retired it in code on 2026-08-27 and its stored session was still in the database seven weeks on. Step 5 — the code. Deletes pixiv_client, pixiv_downloader, pixiv_ingester, platforms/pixiv and their three test modules and fixture, then edits out every remaining reference: the dispatch entry, the campaign-id and verify branches in download_backends, the display-name branch in extension_service, and the comments that still described pixiv as live. The consolidation check the step asked for comes back negative: native_ingest_common has seven non-pixiv callers (patreon, subscribestar, membership_reconcile, membership_roster, ingest_core), so nothing there drops to a single user. Step 6 — the data, alembic 0102. Drops pixiv_seen_media and pixiv_failed_media, and deletes credential rows whose platform is not registered. Written as "not registered" rather than "pixiv" at the step's explicit ask, which is what makes one migration cover two retirements: the pixiv OAuth refresh token and DeviantArt's leftover session (#3980). It is also the only way either row can go — the credentials UI renders one card per platform from /api/platforms and looks the credential up by key, so an unregistered platform's row has no card and no Remove button. Pixiv's Source rows are KEPT, changing the milestone's original data table on the operator's call. `platform` is stored only on Source; neither Post nor ImageRecord carries it. Both FKs are ON DELETE SET NULL, so a delete would not lose the art — but it would drop every pixiv image into the gallery's __unsourced__ bucket and strip the platform chip off every pixiv post. The rows stay disabled (0097) and unregistered, so nothing schedules or downloads through them. Keeping them costs nothing and keeps the attribution that "the art already downloaded from pixiv stays" is about. Step 7 — the guard. test_pixiv_code_and_tables_are_gone asserts absence from the module table and from Base.metadata, not from prose (snippet #3352's trap). The extension and registry negative assertions were already in place from phase 1. The final sweep found one real residue step 4 missed: extension/README.md still advertised pixiv support and carried a "Pixiv OAuth" manual-test item. Also replaces the two deleted dispatch tests with one over the whole NATIVE_INGESTER_PLATFORMS set, so adding a platform and forgetting its ingester class now fails at unit level rather than as a mid-download KeyError. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
@@ -26,14 +26,12 @@ from pathlib import Path
|
||||
from .gallery_dl import DownloadResult, ErrorType
|
||||
from .patreon_ingester import PatreonIngester
|
||||
from .patreon_resolver import extract_vanity, resolve_campaign_id_for_source
|
||||
from .pixiv_client import user_id_from_url
|
||||
from .pixiv_ingester import PixivIngester
|
||||
from .platforms import known_platform_keys
|
||||
from .subscribestar_ingester import SubscribeStarIngester
|
||||
|
||||
# Platforms whose download + verify go through the native ingester rather than
|
||||
# gallery-dl. gallery-dl still serves the rest (hentaifoundry, discord) until
|
||||
# they migrate too. pixiv left this set when it was retired (milestone #406).
|
||||
# they migrate too.
|
||||
NATIVE_INGESTER_PLATFORMS = frozenset({"patreon", "subscribestar"})
|
||||
|
||||
|
||||
@@ -67,7 +65,6 @@ def _native_ingester_cls(platform: str):
|
||||
dispatch pick up the replacement."""
|
||||
return {
|
||||
"patreon": PatreonIngester,
|
||||
"pixiv": PixivIngester,
|
||||
"subscribestar": SubscribeStarIngester,
|
||||
}[platform]
|
||||
|
||||
@@ -127,25 +124,17 @@ async def _resolve_native_campaign_id(
|
||||
platform: str, url: str, cookies_path: str | None, overrides: dict,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""`(campaign_id, resolved_campaign_id)` for a native source. SubscribeStar's
|
||||
feed id IS the creator URL; Pixiv's is the numeric user id parsed straight
|
||||
from it (no lookup → resolved None either way). Patreon resolves the
|
||||
feed id IS the creator URL (no lookup → resolved None). Patreon resolves the
|
||||
campaign id from the vanity URL (resolved non-None when a lookup actually ran,
|
||||
so phase 3 caches it)."""
|
||||
if platform == "subscribestar":
|
||||
return url, None
|
||||
if platform == "pixiv":
|
||||
return user_id_from_url(url), None
|
||||
return await resolve_campaign_id_for_source(url, cookies_path, overrides)
|
||||
|
||||
|
||||
def _campaign_resolution_error(platform: str, url: str) -> str:
|
||||
"""Operator-facing message for a native source whose campaign id could not
|
||||
be resolved — names the platform's own lookup mechanism."""
|
||||
if platform == "pixiv":
|
||||
return (
|
||||
f"Could not extract a pixiv user id. source_url={url!r} — expected "
|
||||
"a URL like https://www.pixiv.net/users/<id>."
|
||||
)
|
||||
vanity = extract_vanity(url)
|
||||
return (
|
||||
f"Could not resolve Patreon campaign id. source_url={url!r}; "
|
||||
@@ -172,8 +161,8 @@ async def _run_native_ingester(
|
||||
platform, ctx["url"], ctx["cookies_path"], overrides
|
||||
)
|
||||
if not campaign_id:
|
||||
# Patreon: vanity lookup failed. Pixiv: no numeric user id in the URL.
|
||||
# (SubscribeStar's campaign id is the URL itself — never lands here.)
|
||||
# Patreon: vanity lookup failed. (SubscribeStar's campaign id is the
|
||||
# URL itself — never lands here.)
|
||||
url = ctx["url"]
|
||||
return (
|
||||
DownloadResult(
|
||||
@@ -205,7 +194,7 @@ async def _run_native_ingester(
|
||||
validate=gdl._validate_files,
|
||||
rate_limit=rate_limit,
|
||||
request_sleep=request_sleep,
|
||||
# Uniform across adapters: token platforms (pixiv) authenticate with
|
||||
# Uniform across adapters: a token platform would authenticate with
|
||||
# it, cookie platforms accept-and-ignore — so this construction stays
|
||||
# platform-agnostic.
|
||||
auth_token=ctx["auth_token"],
|
||||
@@ -252,16 +241,11 @@ async def verify_source_credential(
|
||||
if uses_native_ingester(platform):
|
||||
# Native ingester platforms verify via their own lightweight auth probe.
|
||||
# SubscribeStar's probe takes the creator URL directly; Patreon's
|
||||
# resolves the campaign id first; Pixiv's is one OAuth refresh (the
|
||||
# exact call that fails when the token is bad — no feed walk).
|
||||
# resolves the campaign id first.
|
||||
if platform == "subscribestar":
|
||||
from .subscribestar_ingester import verify_subscribestar_credential
|
||||
|
||||
return await verify_subscribestar_credential(url, cookies_path, config_overrides)
|
||||
if platform == "pixiv":
|
||||
from .pixiv_ingester import verify_pixiv_credential
|
||||
|
||||
return await verify_pixiv_credential(auth_token)
|
||||
from .patreon_ingester import verify_patreon_credential
|
||||
|
||||
return await verify_patreon_credential(url, cookies_path, config_overrides)
|
||||
|
||||
Reference in New Issue
Block a user