Operator-requested 2026-05-27: centralize the per-platform quirks that
had been accumulating across credential_service, sidecar, and platforms
into a single per-platform module so adding/updating quirks becomes
"edit one file."
**Layout**
services/platforms/
base.py PlatformInfo dataclass + module-default key
chains + shared helpers (str_id_value, str_field)
__init__.py PLATFORMS dict + public API (auth_type_for,
known_platform_keys, to_dict,
external_post_id_keys_for, description_keys_for)
patreon.py metadata only — the reference platform, no quirks
subscribestar.py metadata + augment_cookies (18+ agreement) +
derive_post_url (synthetic /posts/<post_id>)
hentaifoundry.py metadata + augment_cookies (host-only PHPSESSID
duplicate) + derive_post_url (/pictures/user/...)
pixiv.py metadata + derive_post_url (/artworks/<id>)
discord.py metadata + derive_post_url
(channels/<server>/<channel>/<message>)
deviantart.py metadata only — un-audited; quirks to be added
when an operator first exercises DA
**PlatformInfo extensions**
Existing fields preserved. Four new optional fields:
external_post_id_keys: tuple[str, ...] | None
Override the sidecar external_post_id lookup chain. None falls
back to DEFAULT_EXTERNAL_POST_ID_KEYS in base.py
("post_id", "id", "index", "message_id") — covers every current
platform.
description_keys: tuple[str, ...] | None
Override the description body lookup chain. None falls back to
DEFAULT_DESCRIPTION_KEYS ("content", "description", "caption",
"message") — Discord's "message" body field is covered by the
default's trailing entry.
derive_post_url: Callable[[dict], str | None] | None
Synthesize the post permalink from sidecar metadata. None = trust
the bare `url` / `post_url` field (patreon, deviantart).
subscribestar/pixiv/hf/discord override this because their `url`
is the file CDN URL.
augment_cookies: Callable[[str], str] | None
Post-process the materialized cookies.txt before gallery-dl
consumes it. None = no-op. Used by subscribestar (age cookie) and
hentaifoundry (host-only PHPSESSID duplicate).
**Consumer changes**
- credential_service._augment_cookies(platform, netscape) shrunk from a
per-platform-conditional dispatcher (~80 lines of inlined helpers) to
a 5-line lookup: `info.augment_cookies(netscape) if info and
info.augment_cookies else netscape`. The platform-specific helper
bodies moved verbatim into the per-platform modules.
- sidecar.parse_sidecar similarly delegates: external_post_id chain via
external_post_id_keys_for(category), description chain via
description_keys_for(category), post_url via
PLATFORMS[category].derive_post_url. The _DERIVED_URL_PLATFORMS set
and inline _derive_post_url body both gone. Added a shared `_first_id`
helper for bool-safe id coercion.
**Public API preserved**
PLATFORMS, PlatformInfo, auth_type_for, known_platform_keys, to_dict
are all re-exported from the package's __init__.py. test_platforms_registry
test_credential_service, and test_sidecar_util pass without changes
because the behavior is identical; only the implementation moved.
**Adding a new platform**
1. Create services/platforms/<name>.py with `INFO = PlatformInfo(...)`
and any of the four optional hooks.
2. Import it in services/platforms/__init__.py + add to the PLATFORMS
tuple-comprehension.
3. Done. sidecar parsing, cookie materialization, /api/platforms all
pick it up automatically.
Operator-flagged 2026-05-27: HF source check 401'd on
`HEAD /?enterAgree=1` even with valid login cookies. Root cause is the
combination of (1) gallery-dl's HF extractor checking
`self.cookies.get("PHPSESSID", domain="www.hentai-foundry.com")` with
`requests`' EXACT domain matching, and (2) the extension's cookies.js
forcibly rewriting every captured cookie to a leading-dot subdomain-wide
form. HF's PHPSESSID is browser-stored as host-only on
`www.hentai-foundry.com`; the rewrite re-anchored it to
`.hentai-foundry.com`, which `cookies.get(...)` no longer matches even
though the cookie is still sent on actual HTTP requests (RFC 6265
subdomain rules). The extractor falls into its unauthenticated
`?enterAgree=1` fallback, which 401s (Cloudflare or HF's anti-bot HEAD
gating).
Two-part fix, no operator action required for existing stored cookies:
1. **Backend** (`credential_service._augment_cookies`) — refactored from
the subscribestar-only single function into a per-platform dispatcher.
New `_augment_hentaifoundry` parses the materialized netscape file
and, for each `.hentai-foundry.com` entry whose name is PHPSESSID or
YII_CSRF_TOKEN, appends a host-only duplicate
(`www.hentai-foundry.com\tFALSE\t...`). Originals preserved. Three
new tests pin: injection fires + originals preserved; idempotent
when host-only already exists; doesn't touch unrelated cookies
(e.g. `_ga`).
2. **Extension** (`cookies.js`) — `toNetscapeFormat` now respects
`c.hostOnly` from the browser instead of blindly forcing a
leading-dot subdomain-wide form. Host-only cookies are written with
the bare host + FALSE flag; non-host-only cookies retain the
leading-dot + TRUE form. Forward-compat — fresh captures from
v1.0.5+ no longer need the backend's host-only duplication.
Extension bumped 1.0.4 → 1.0.5; manifest + package.json in lockstep.
After deploy: the next HF source check on the operator's already-stored
cookies will succeed because the materialized cookies.txt now contains
host-only PHPSESSID. No browser re-export needed.
Operator-flagged 2026-05-27: subscribestar source check aborted with
`AbortExtraction: HTTP redirect to .../age_confirmation_warning`. The
captured `_personalization_id` cookie in the browser-stored file had
expired (annual rotation), and the user could not realistically refresh
it: SubscribeStar's frontend JS uses localStorage to suppress the
age-confirmation popup once dismissed, so a logged-in revisit doesn't
re-show the popup and the server-side cookie is never re-issued.
gallery-dl's own login flow (which FC doesn't exercise — cookies come
from the extension instead) sidesteps this by manually setting
`18_plus_agreement_generic=true` on `.subscribestar.adult`. The server
accepts that as the age-confirmation marker.
`credential_service._augment_cookies(platform, netscape)` mirrors that
behavior: when the materialized cookies file is for subscribestar and
the age cookie isn't already present, append a synthetic line for
`.subscribestar.adult` with name=`18_plus_agreement_generic` value=`true`
and a far-future expiry. No-op for other platforms; no-op if the cookie
is already present (idempotent for manual pastes / extension captures
that happen to include it).
Three new tests pin: (a) injection fires for subscribestar, preserves
existing cookies; (b) idempotent when already present (no double
injection); (c) does NOT fire for non-subscribestar platforms (Patreon
etc. don't get a foreign-domain cookie).
Not a curator handling bug per se — the extension faithfully captured
what the browser had. This is mirroring a documented gallery-dl
workaround so the cookies-via-extension auth path doesn't degrade as the
server-side cookie expires.
bvandeusen