fix: a Discord source takes only messages with an image attached, from the posters it names, and knows its channel's name (#4481)
CI and images / lint (push) Successful in 4s
CI and images / extension-version (push) Successful in 3s
CI and images / frontend-build (push) Successful in 21s
CI and images / extension-test (push) Successful in 20s
CI and images / backend-lint-and-test (push) Successful in 34s
CI and images / integration (push) Successful in 2m24s
CI and images / sign-extension (push) Successful in 3s
CI and images / build-agent (push) Successful in 7s
CI and images / build-web (push) Successful in 1m42s
CI and images / smoke-web (push) Successful in 59s
CI and images / promote (push) Successful in 2s

- A message with no image or video attachment is chat: extract_media returns
  nothing for it, so nothing downloads and no post record is written. A
  message that passes keeps every file, numbered as gallery-dl numbers them.
- `discord_authors` in a source's config limits the walk to those posters
  (id, username or display name); the edit dialog has a field for it and no
  longer drops config keys it has no field for.
- source.display_name (migration 0115), refreshed by every Discord walk, shows
  on Subscriptions in place of the two-id URL; a Discord post card names its
  channel from the record's `channel`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-28 07:03:47 -04:00
co-authored by Claude Opus 5.5
parent 42a60e3e74
commit 29c22afcb7
13 changed files with 348 additions and 25 deletions
+54 -1
View File
@@ -13,6 +13,11 @@ Two things differ from the cookie platforms:
something; a Discord drop is routinely files and nothing else, so on
Discord that is an ordinary backfill, not a broken parser.
And two things only a Discord source has: `discord_authors` in its
config_overrides limits it to the creator's own messages (a server's other
members post pictures too), and it learns its own name — the server and
channel it walks — into `source.display_name`.
`campaign_id` is the source URL (a server, channel, thread or category link).
FC runs on a plain-HTTP homelab; nothing here uses a secure-context Web API.
"""
@@ -20,15 +25,23 @@ FC runs on a plain-HTTP homelab; nothing here uses a secure-context Web API.
from __future__ import annotations
import asyncio
import logging
from collections.abc import Callable
from pathlib import Path
from ..models import DiscordFailedMedia, DiscordSeenMedia
from sqlalchemy import select, update
from ..models import DiscordFailedMedia, DiscordSeenMedia, Source
from .discord_client import DiscordAPIError, DiscordClient, MediaItem
from .discord_downloader import DiscordDownloader
from .ingest_core import Ingester
log = logging.getLogger(__name__)
_LEDGER_KEY_MAX = 128
# `config_overrides` key: the people whose messages this source takes (user
# ids, usernames or display names). Absent or empty takes everyone's.
AUTHORS_KEY = "discord_authors"
def _ledger_key(media: MediaItem) -> str:
@@ -74,6 +87,46 @@ class DiscordIngester(Ingester):
body_canary=False,
)
def run(self, **kwargs):
"""The core walk, bracketed by the two things only a Discord source
has: whose messages it takes, read before the walk, and the name of
what it walks, written after it (the walk is what loads that name)."""
source_id = kwargs["source_id"]
self.client.only_from(self._source_authors(source_id))
try:
return super().run(**kwargs)
finally:
self._record_display_name(source_id, kwargs["campaign_id"])
def _source_authors(self, source_id: int) -> list:
if self.session_factory is None:
return []
with self.session_factory() as session:
overrides = session.execute(
select(Source.config_overrides).where(Source.id == source_id)
).scalar_one_or_none() or {}
authors = overrides.get(AUTHORS_KEY) or []
return authors if isinstance(authors, list) else []
def _record_display_name(self, source_id: int, url: str) -> None:
"""Refreshed on every walk, never written once: a renamed channel
should read as its new name. A name the walk couldn't read leaves the
stored one alone, and a failure here never fails the walk."""
label = self.client.source_label(url)
if not label or self.session_factory is None:
return
try:
with self.session_factory() as session:
session.execute(
update(Source)
.where(Source.id == source_id)
.where(Source.display_name.is_distinct_from(label))
.values(display_name=label)
)
session.commit()
except Exception as exc: # a name is decoration — never fail the walk
log.warning("Discord: couldn't record source %s's name: %s", source_id, exc)
async def verify_discord_credential(url: str, auth_token: str | None) -> tuple[bool | None, str]:
"""The uniform `(ok, message)` probe: is the token valid, and can its