feat: Discord on the native core ingester — client, downloader, ledgers, wiring (milestone 428)
CI and images / lint (push) Successful in 3s
CI and images / extension-version (push) Successful in 3s
CI and images / frontend-build (push) Successful in 25s
CI and images / backend-lint-and-test (push) Successful in 32s
CI and images / integration (push) Successful in 2m27s
CI and images / sign-extension (push) Successful in 3s
CI and images / build-agent (push) Successful in 5s
CI and images / build-web (push) Successful in 1m48s
CI and images / smoke-web (push) Successful in 54s
CI and images / promote (push) Successful in 1s

Discord was the last focus platform still on gallery-dl. This adds the native
path, mirrored from gallery-dl 1.32.13's discord extractor:

- discord_client: API v10 with the user token and gallery-dl's request
  profile (dated Firefox UA, Referer). Walks a server, category, forum,
  channel or thread in gallery-dl's order and pages each channel newest-first.
  Files are attachments, then embeds, then forwards, numbered across the
  message. The resume cursor is <channel>:<before>. Text-only messages are
  not posts, since gallery-dl never made them.
- discord_downloader: gallery-dl's on-disk layout, cleaned the way it cleans
  names on Linux (only `/` and control characters change), so existing files
  are skipped_disk rather than fetched again. Sidecars carry identity only.
  The message record keeps gallery-dl's keys, so parse_sidecar,
  derive_post_url and the drop grouping read it unchanged.
- The ledger keys on the attachment id (or a hash of an embed's URL path),
  not the file's position, which an edit can renumber. Migration 0111.
- DiscordIngester: token auth, body canary off (files-only drops are
  normal). Registered as native, verified by token, and serialised
  per-platform, since every source shares one user token.
- ingest_core: optional `skip_feed` client seam (#4413). A tick's early-out
  on a multi-channel source now ends the quiet channel, not the whole walk.
  Clients without the seam behave as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
2026-09-24 19:42:00 -04:00
co-authored by Claude Opus 5.5
parent 058fa85606
commit 84e5448941
15 changed files with 1579 additions and 15 deletions
+15 -3
View File
@@ -17,7 +17,7 @@ from backend.app.services.gallery_dl import ErrorType
def test_native_platforms():
for platform in ("patreon", "subscribestar"):
for platform in ("patreon", "subscribestar", "discord"):
assert uses_native_ingester(platform) is True
assert platform in NATIVE_INGESTER_PLATFORMS
@@ -104,8 +104,20 @@ async def test_verifying_a_retired_platform_is_inconclusive_not_rejected():
def test_gallery_dl_platforms_are_not_native():
# The platforms still served by gallery-dl must NOT route to the native
# ingester — guards an accidental over-broad migration.
for platform in ("hentaifoundry", "discord"):
assert uses_native_ingester(platform) is False
assert uses_native_ingester("hentaifoundry") is False
@pytest.mark.asyncio
async def test_discord_verify_without_a_token_is_a_rejection_not_a_request():
"""Discord authenticates by token (milestone 428); with none saved there is
nothing to send, and saying so beats an HTTP 401 from Discord."""
ok, message = await verify_source_credential(
platform="discord", url="https://discord.com/channels/1/2",
artist_slug="someone", config_overrides=None, cookies_path=None,
auth_token=None, images_root=Path("/nonexistent"),
)
assert ok is False
assert "token" in message.lower()
def test_unknown_platform_is_not_native():