fix(hentaifoundry): inject host-only PHPSESSID/CSRF duplicates + extension preserves browser hostOnly

Operator-flagged 2026-05-27: HF source check 401'd on
`HEAD /?enterAgree=1` even with valid login cookies. Root cause is the
combination of (1) gallery-dl's HF extractor checking
`self.cookies.get("PHPSESSID", domain="www.hentai-foundry.com")` with
`requests`' EXACT domain matching, and (2) the extension's cookies.js
forcibly rewriting every captured cookie to a leading-dot subdomain-wide
form. HF's PHPSESSID is browser-stored as host-only on
`www.hentai-foundry.com`; the rewrite re-anchored it to
`.hentai-foundry.com`, which `cookies.get(...)` no longer matches even
though the cookie is still sent on actual HTTP requests (RFC 6265
subdomain rules). The extractor falls into its unauthenticated
`?enterAgree=1` fallback, which 401s (Cloudflare or HF's anti-bot HEAD
gating).

Two-part fix, no operator action required for existing stored cookies:

1. **Backend** (`credential_service._augment_cookies`) — refactored from
   the subscribestar-only single function into a per-platform dispatcher.
   New `_augment_hentaifoundry` parses the materialized netscape file
   and, for each `.hentai-foundry.com` entry whose name is PHPSESSID or
   YII_CSRF_TOKEN, appends a host-only duplicate
   (`www.hentai-foundry.com\tFALSE\t...`). Originals preserved. Three
   new tests pin: injection fires + originals preserved; idempotent
   when host-only already exists; doesn't touch unrelated cookies
   (e.g. `_ga`).

2. **Extension** (`cookies.js`) — `toNetscapeFormat` now respects
   `c.hostOnly` from the browser instead of blindly forcing a
   leading-dot subdomain-wide form. Host-only cookies are written with
   the bare host + FALSE flag; non-host-only cookies retain the
   leading-dot + TRUE form. Forward-compat — fresh captures from
   v1.0.5+ no longer need the backend's host-only duplication.
   Extension bumped 1.0.4 → 1.0.5; manifest + package.json in lockstep.

After deploy: the next HF source check on the operator's already-stored
cookies will succeed because the materialized cookies.txt now contains
host-only PHPSESSID. No browser re-export needed.
This commit is contained in:
2026-05-27 19:12:51 -04:00
parent 8243740a04
commit 2394e47370
5 changed files with 153 additions and 17 deletions
+68 -11
View File
@@ -166,7 +166,7 @@ class CredentialService:
def _augment_cookies(platform: str, netscape: str) -> str:
"""Inject platform-specific synthetic cookies needed to bypass server
gates that the user can't realistically re-trigger in their browser.
gates or extractor quirks.
subscribestar.adult: the server gates artist pages behind the
`_personalization_id` age-confirmation cookie. The site's frontend JS
@@ -177,21 +177,37 @@ def _augment_cookies(platform: str, netscape: str) -> str:
FC doesn't use; we capture cookies via the extension instead)
sidesteps this by manually setting `18_plus_agreement_generic=true`
on `.subscribestar.adult` — the server accepts that as the
age-confirmation marker. Mirror that behavior here so cookies-only
auth works long-term.
age-confirmation marker.
The injection is a no-op if the cookie is already present (operator
might have it from an earlier login or a manual cookies.txt paste).
hentaifoundry: gallery-dl's extractor uses
`self.cookies.get("PHPSESSID", domain="www.hentai-foundry.com")` to
decide whether the user is logged in. `requests` does EXACT domain
matching on .get(); the extension rewrites every captured cookie to
a leading-dot subdomain-wide form (`.hentai-foundry.com`), which
fails that exact match. The fallback path hits a HEAD
`?enterAgree=1` that 401s. Emit host-only duplicates of PHPSESSID +
YII_CSRF_TOKEN on `www.hentai-foundry.com` so the lookup succeeds.
(The original `.hentai-foundry.com` entries stay — the actual HTTP
requests use RFC 6265 subdomain matching, which works either way.)
Operator-flagged 2026-05-27 after a subscribestar source check
aborted with `HTTP redirect to .../age_confirmation_warning`.
All injections are idempotent (no-op if the target cookie is already
present) and platform-scoped.
Operator-flagged 2026-05-27: subscribestar age-confirmation, then
hentaifoundry 401 on /?enterAgree=1.
"""
if platform != "subscribestar":
return netscape
if platform == "subscribestar":
return _augment_subscribestar(netscape)
if platform == "hentaifoundry":
return _augment_hentaifoundry(netscape)
return netscape
def _augment_subscribestar(netscape: str) -> str:
if "18_plus_agreement_generic" in netscape:
return netscape
# Far-future expiry — 10 years out. The server only checks presence/value;
# gallery-dl's own login flow sets this with no explicit expiry too.
# Far-future expiry — gallery-dl's own login flow sets this with no
# explicit expiry; the server only checks presence/value.
expiry = 4102444800 # 2100-01-01 UTC, opaque "far future"
line = "\t".join([
".subscribestar.adult", "TRUE", "/", "TRUE",
@@ -203,6 +219,47 @@ def _augment_cookies(platform: str, netscape: str) -> str:
return body + "\n" + line + "\n"
_HF_HOST_ONLY_NAMES = ("PHPSESSID", "YII_CSRF_TOKEN")
def _augment_hentaifoundry(netscape: str) -> str:
body = netscape.rstrip("\n")
if not body:
return netscape
lines = body.split("\n")
existing_host_only = set()
by_name: dict[str, list[str]] = {}
for raw in lines:
if not raw or raw.startswith("#"):
continue
parts = raw.split("\t")
if len(parts) < 7:
continue
domain, _flag, _path, _secure, _exp, name, _value = parts[:7]
if name not in _HF_HOST_ONLY_NAMES:
continue
if domain == "www.hentai-foundry.com":
existing_host_only.add(name)
elif domain in (".hentai-foundry.com", "hentai-foundry.com"):
by_name.setdefault(name, []).append(raw)
appended = []
for name in _HF_HOST_ONLY_NAMES:
if name in existing_host_only or name not in by_name:
continue
# Duplicate the FIRST subdomain-wide line as host-only on
# www.hentai-foundry.com. Same value + expiry; flag=FALSE marks
# it host-only in netscape format.
parts = by_name[name][0].split("\t")
parts[0] = "www.hentai-foundry.com"
parts[1] = "FALSE"
appended.append("\t".join(parts[:7]))
if not appended:
return netscape
return body + "\n" + "\n".join(appended) + "\n"
def _to_netscape(plaintext: str) -> str:
"""Accept either Netscape-format text (the extension's output) or a
JSON array of cookie dicts (a manual-paste edge case); produce