bvandeusen
abafc3265e
Operator-requested 2026-05-27: centralize the per-platform quirks that
had been accumulating across credential_service, sidecar, and platforms
into a single per-platform module so adding/updating quirks becomes
"edit one file."
**Layout**
services/platforms/
base.py PlatformInfo dataclass + module-default key
chains + shared helpers (str_id_value, str_field)
__init__.py PLATFORMS dict + public API (auth_type_for,
known_platform_keys, to_dict,
external_post_id_keys_for, description_keys_for)
patreon.py metadata only — the reference platform, no quirks
subscribestar.py metadata + augment_cookies (18+ agreement) +
derive_post_url (synthetic /posts/<post_id>)
hentaifoundry.py metadata + augment_cookies (host-only PHPSESSID
duplicate) + derive_post_url (/pictures/user/...)
pixiv.py metadata + derive_post_url (/artworks/<id>)
discord.py metadata + derive_post_url
(channels/<server>/<channel>/<message>)
deviantart.py metadata only — un-audited; quirks to be added
when an operator first exercises DA
**PlatformInfo extensions**
Existing fields preserved. Four new optional fields:
external_post_id_keys: tuple[str, ...] | None
Override the sidecar external_post_id lookup chain. None falls
back to DEFAULT_EXTERNAL_POST_ID_KEYS in base.py
("post_id", "id", "index", "message_id") — covers every current
platform.
description_keys: tuple[str, ...] | None
Override the description body lookup chain. None falls back to
DEFAULT_DESCRIPTION_KEYS ("content", "description", "caption",
"message") — Discord's "message" body field is covered by the
default's trailing entry.
derive_post_url: Callable[[dict], str | None] | None
Synthesize the post permalink from sidecar metadata. None = trust
the bare `url` / `post_url` field (patreon, deviantart).
subscribestar/pixiv/hf/discord override this because their `url`
is the file CDN URL.
augment_cookies: Callable[[str], str] | None
Post-process the materialized cookies.txt before gallery-dl
consumes it. None = no-op. Used by subscribestar (age cookie) and
hentaifoundry (host-only PHPSESSID duplicate).
**Consumer changes**
- credential_service._augment_cookies(platform, netscape) shrunk from a
per-platform-conditional dispatcher (~80 lines of inlined helpers) to
a 5-line lookup: `info.augment_cookies(netscape) if info and
info.augment_cookies else netscape`. The platform-specific helper
bodies moved verbatim into the per-platform modules.
- sidecar.parse_sidecar similarly delegates: external_post_id chain via
external_post_id_keys_for(category), description chain via
description_keys_for(category), post_url via
PLATFORMS[category].derive_post_url. The _DERIVED_URL_PLATFORMS set
and inline _derive_post_url body both gone. Added a shared `_first_id`
helper for bool-safe id coercion.
**Public API preserved**
PLATFORMS, PlatformInfo, auth_type_for, known_platform_keys, to_dict
are all re-exported from the package's __init__.py. test_platforms_registry
test_credential_service, and test_sidecar_util pass without changes
because the behavior is identical; only the implementation moved.
**Adding a new platform**
1. Create services/platforms/<name>.py with `INFO = PlatformInfo(...)`
and any of the four optional hooks.
2. Import it in services/platforms/__init__.py + add to the PLATFORMS
tuple-comprehension.
3. Done. sidecar parsing, cookie materialization, /api/platforms all
pick it up automatically.
84 lines
3.0 KiB
Python
84 lines
3.0 KiB
Python
"""HentaiFoundry — two quirks colocated.
|
|
|
|
1. post_url: HF sidecars omit `url` entirely; `src` is the image URL.
|
|
Synthesize the permalink from `user` + `index`
|
|
(/pictures/user/<user>/<index>).
|
|
|
|
2. augment_cookies: gallery-dl's HF extractor checks
|
|
`self.cookies.get("PHPSESSID", domain="www.hentai-foundry.com")` with
|
|
`requests`' EXACT domain matching. The extension's pre-v1.0.5
|
|
`cookies.js` aggressively rewrote every captured cookie to the
|
|
leading-dot subdomain-wide form (`.hentai-foundry.com`), which fails
|
|
the exact lookup even though the cookie IS sent on actual HTTP
|
|
requests (RFC 6265 subdomain matching). The extractor falls into
|
|
an unauthenticated `?enterAgree=1` HEAD that 401s. Inject host-only
|
|
duplicates of PHPSESSID + YII_CSRF_TOKEN so the lookup succeeds.
|
|
"""
|
|
|
|
from .base import GD_DEFAULTS, PlatformInfo, str_field, str_id_value
|
|
|
|
_HOST_ONLY_NAMES = ("PHPSESSID", "YII_CSRF_TOKEN")
|
|
|
|
|
|
def derive_post_url(data: dict) -> str | None:
|
|
user = str_field(data.get("user")) or str_field(data.get("artist"))
|
|
idx = str_id_value(data.get("index"))
|
|
if user and idx:
|
|
return f"https://www.hentai-foundry.com/pictures/user/{user}/{idx}"
|
|
return None
|
|
|
|
|
|
def augment_cookies(netscape: str) -> str:
|
|
body = netscape.rstrip("\n")
|
|
if not body:
|
|
return netscape
|
|
lines = body.split("\n")
|
|
existing_host_only: set[str] = set()
|
|
by_name: dict[str, list[str]] = {}
|
|
for raw in lines:
|
|
if not raw or raw.startswith("#"):
|
|
continue
|
|
parts = raw.split("\t")
|
|
if len(parts) < 7:
|
|
continue
|
|
domain, _flag, _path, _secure, _exp, name, _value = parts[:7]
|
|
if name not in _HOST_ONLY_NAMES:
|
|
continue
|
|
if domain == "www.hentai-foundry.com":
|
|
existing_host_only.add(name)
|
|
elif domain in (".hentai-foundry.com", "hentai-foundry.com"):
|
|
by_name.setdefault(name, []).append(raw)
|
|
|
|
appended: list[str] = []
|
|
for name in _HOST_ONLY_NAMES:
|
|
if name in existing_host_only or name not in by_name:
|
|
continue
|
|
# Duplicate the first subdomain-wide line as host-only on
|
|
# www.hentai-foundry.com. Same value + expiry; flag=FALSE marks
|
|
# the entry host-only in netscape format.
|
|
parts = by_name[name][0].split("\t")
|
|
parts[0] = "www.hentai-foundry.com"
|
|
parts[1] = "FALSE"
|
|
appended.append("\t".join(parts[:7]))
|
|
|
|
if not appended:
|
|
return netscape
|
|
return body + "\n" + "\n".join(appended) + "\n"
|
|
|
|
|
|
INFO = PlatformInfo(
|
|
key="hentaifoundry",
|
|
name="Hentai Foundry",
|
|
description="Download artwork from Hentai Foundry artists",
|
|
auth_type="cookies",
|
|
requires_auth=False,
|
|
url_pattern=r"^https?://(www\.)?hentai-foundry\.com/",
|
|
url_examples=[
|
|
"https://www.hentai-foundry.com/user/example_artist",
|
|
"https://www.hentai-foundry.com/pictures/user/example_artist",
|
|
],
|
|
default_config={**GD_DEFAULTS, "content_types": ["pictures"]},
|
|
derive_post_url=derive_post_url,
|
|
augment_cookies=augment_cookies,
|
|
)
|