CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 4s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 25s
extension / lint (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 34s
Build images / build-web (push) Successful in 1m5s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m54s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m19s
Milestone #406 phase 2, with issue #3980 folded in. Phase 1 (2026-09-13) unregistered pixiv so nothing could reach it; the code has sat in the tree uncalled since. DeviantArt is why the second half is not left for later — #3069 retired it in code on 2026-08-27 and its stored session was still in the database seven weeks on. Step 5 — the code. Deletes pixiv_client, pixiv_downloader, pixiv_ingester, platforms/pixiv and their three test modules and fixture, then edits out every remaining reference: the dispatch entry, the campaign-id and verify branches in download_backends, the display-name branch in extension_service, and the comments that still described pixiv as live. The consolidation check the step asked for comes back negative: native_ingest_common has seven non-pixiv callers (patreon, subscribestar, membership_reconcile, membership_roster, ingest_core), so nothing there drops to a single user. Step 6 — the data, alembic 0102. Drops pixiv_seen_media and pixiv_failed_media, and deletes credential rows whose platform is not registered. Written as "not registered" rather than "pixiv" at the step's explicit ask, which is what makes one migration cover two retirements: the pixiv OAuth refresh token and DeviantArt's leftover session (#3980). It is also the only way either row can go — the credentials UI renders one card per platform from /api/platforms and looks the credential up by key, so an unregistered platform's row has no card and no Remove button. Pixiv's Source rows are KEPT, changing the milestone's original data table on the operator's call. `platform` is stored only on Source; neither Post nor ImageRecord carries it. Both FKs are ON DELETE SET NULL, so a delete would not lose the art — but it would drop every pixiv image into the gallery's __unsourced__ bucket and strip the platform chip off every pixiv post. The rows stay disabled (0097) and unregistered, so nothing schedules or downloads through them. Keeping them costs nothing and keeps the attribution that "the art already downloaded from pixiv stays" is about. Step 7 — the guard. test_pixiv_code_and_tables_are_gone asserts absence from the module table and from Base.metadata, not from prose (snippet #3352's trap). The extension and registry negative assertions were already in place from phase 1. The final sweep found one real residue step 4 missed: extension/README.md still advertised pixiv support and carried a "Pixiv OAuth" manual-test item. Also replaces the two deleted dispatch tests with one over the whole NATIVE_INGESTER_PLATFORMS set, so adding a platform and forgetting its ingester class now fails at unit level rather than as a mid-download KeyError. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
126 lines
4.5 KiB
Python
126 lines
4.5 KiB
Python
"""download_backends — the single predicate that routes a platform to the
|
|
native ingester vs. gallery-dl. Pure, no DB."""
|
|
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
from backend.app.services.download_backends import (
|
|
NATIVE_INGESTER_PLATFORMS,
|
|
_native_ingester_cls,
|
|
_unsupported_platform_message,
|
|
run_download,
|
|
uses_native_ingester,
|
|
verify_source_credential,
|
|
)
|
|
from backend.app.services.gallery_dl import ErrorType
|
|
|
|
|
|
def test_native_platforms():
|
|
for platform in ("patreon", "subscribestar"):
|
|
assert uses_native_ingester(platform) is True
|
|
assert platform in NATIVE_INGESTER_PLATFORMS
|
|
|
|
|
|
def test_pixiv_is_no_longer_native():
|
|
"""Retired at milestone #406 — unregistered in phase 1, deleted in phase 2.
|
|
The refusal below is what stops it falling through to gallery-dl now that
|
|
it is neither native nor registered."""
|
|
assert uses_native_ingester("pixiv") is False
|
|
assert "pixiv" not in NATIVE_INGESTER_PLATFORMS
|
|
|
|
|
|
# --- the retired-platform guard (#406 phase 1) -----------------------------
|
|
|
|
|
|
class _RecordingGalleryDL:
|
|
"""Stands in for GalleryDLService: records whether a download was attempted."""
|
|
|
|
def __init__(self):
|
|
self.calls = []
|
|
|
|
async def download(self, **kwargs):
|
|
self.calls.append(kwargs["platform"])
|
|
return "reached gallery-dl"
|
|
|
|
|
|
def _ctx(platform):
|
|
return {
|
|
"platform": platform, "url": f"https://example.invalid/{platform}",
|
|
"artist_slug": "someone", "cookies_path": None, "auth_token": None,
|
|
}
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_retired_platform_never_reaches_a_downloader():
|
|
"""An enabled source on a retired platform is data that survives a deploy.
|
|
Unguarded, pixiv — no longer native — would fall straight through to the
|
|
gallery-dl branch, which still has a pixiv extractor."""
|
|
gdl = _RecordingGalleryDL()
|
|
result, campaign_id = await run_download(
|
|
ctx=_ctx("pixiv"), source_config=None, skip_value=False, mode=None,
|
|
gdl=gdl, sync_session_factory=None,
|
|
)
|
|
assert gdl.calls == []
|
|
assert result.success is False
|
|
assert result.error_type == ErrorType.UNSUPPORTED_URL
|
|
assert "pixiv" in result.error_message
|
|
assert campaign_id is None
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_supported_gallery_dl_platform_still_reaches_gallery_dl():
|
|
"""The positive control. Without it, a guard that refused EVERY platform
|
|
would pass the test above just as well (rule #167)."""
|
|
gdl = _RecordingGalleryDL()
|
|
result, _ = await run_download(
|
|
ctx=_ctx("hentaifoundry"), source_config=None, skip_value=False, mode=None,
|
|
gdl=gdl, sync_session_factory=None,
|
|
)
|
|
assert gdl.calls == ["hentaifoundry"]
|
|
assert result == "reached gallery-dl"
|
|
|
|
|
|
def test_the_guard_discriminates_by_registration():
|
|
assert _unsupported_platform_message("hentaifoundry") is None
|
|
assert _unsupported_platform_message("patreon") is None
|
|
assert _unsupported_platform_message("pixiv") is not None
|
|
assert _unsupported_platform_message("deviantart") is not None
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_verifying_a_retired_platform_is_inconclusive_not_rejected():
|
|
"""Nothing is probed, so nothing is rejected — returning False would tell the
|
|
operator their credential is bad when the platform is simply gone."""
|
|
ok, message = await verify_source_credential(
|
|
platform="pixiv", url="https://www.pixiv.net/users/1", artist_slug="someone",
|
|
config_overrides=None, cookies_path=None, auth_token=None,
|
|
images_root=Path("/nonexistent"),
|
|
)
|
|
assert ok is None
|
|
assert "pixiv" in message
|
|
|
|
|
|
def test_gallery_dl_platforms_are_not_native():
|
|
# The platforms still served by gallery-dl must NOT route to the native
|
|
# ingester — guards an accidental over-broad migration.
|
|
for platform in ("hentaifoundry", "discord"):
|
|
assert uses_native_ingester(platform) is False
|
|
|
|
|
|
def test_unknown_platform_is_not_native():
|
|
assert uses_native_ingester("nonsense") is False
|
|
|
|
|
|
def test_every_native_platform_dispatches_to_an_ingester():
|
|
"""The dispatch table and NATIVE_INGESTER_PLATFORMS have to agree, or a
|
|
platform that routes native raises KeyError mid-download instead of being
|
|
refused up front. Written over the set rather than per-platform so adding
|
|
one to NATIVE_INGESTER_PLATFORMS and forgetting the class fails here.
|
|
|
|
(This replaces the per-platform dispatch tests, one of which was pixiv's;
|
|
it was deleted with pixiv at milestone #406 phase 2.)"""
|
|
for platform in NATIVE_INGESTER_PLATFORMS:
|
|
assert _native_ingester_cls(platform) is not None
|
|
|