fix(subscribestar): mirror gallery-dl's full request profile (verify_subscriber gate) #120

Merged

bvandeusen merged 1 commits from dev into main 2026-06-17 15:32:28 -04:00

Conversation 0 Commits 1 Files Changed 2

+42 -29

bvandeusen commented 2026-06-17 15:32:23 -04:00

Owner

Copy Link

Copy Source

What

After the delimiter fix, the live cheunart fetch 302'd to /cheunart/verify_subscriber even with valid cookies (confirmed via devtools: _personalization_id + _subscribestar_session present on .subscribestar.adult, logged in). This makes our SubscribeStar HTTP request profile identical to gallery-dl's so it stops tripping that gate.

Root cause (walking gallery-dl's entire flow, base Extractor included)

The divergence isn't cookies or parsing — it's the request profile. gallery-dl's _init_session (default cookies-only mode) sends on every request, including the first creator-page GET:

• User-Agent: …Firefox/140.0 (not Chrome)
• Accept: */*
• Accept-Language: en-US,en;q=0.5
• Referer: https://subscribestar.adult/ (same-site)
• no X-Requested-With anywhere (the "load more" endpoint is a plain GET parsed as JSON)

Ours sent a Chrome UA, no Referer, no Accept-Language, and toggled X-Requested-With — unlike a browser, so SubscribeStar gated the adult-creator page.

Changes

• SubscribeStar session now uses Firefox UA + Accept: */* + Accept-Language (via make_session extra_headers).
• Stamp Referer: <base>/ per walk (mirrors gallery-dl's root/).
• Drop the per-request XHR headers; both the feed and load-more use the shared profile (load-more is a plain GET parsed as JSON, exactly like gallery-dl).
• Test updated to assert the gallery-dl-parity profile.

If it still 302s after deploy, it's a genuine session/subscription issue (gallery-dl itself aborts on that redirect too).

CI green on dev (run 1234).

🤖 Generated with Claude Code

## What After the delimiter fix, the live cheunart fetch 302'd to `/cheunart/verify_subscriber` **even with valid cookies** (confirmed via devtools: `_personalization_id` + `_subscribestar_session` present on `.subscribestar.adult`, logged in). This makes our SubscribeStar HTTP request profile identical to gallery-dl's so it stops tripping that gate. ## Root cause (walking gallery-dl's *entire* flow, base `Extractor` included) The divergence isn't cookies or parsing — it's the request profile. gallery-dl's `_init_session` (default cookies-only mode) sends on **every** request, including the first creator-page GET: - `User-Agent: …Firefox/140.0` (not Chrome) - `Accept: */*` - `Accept-Language: en-US,en;q=0.5` - `Referer: https://subscribestar.adult/` (same-site) - **no `X-Requested-With`** anywhere (the "load more" endpoint is a plain GET parsed as JSON) Ours sent a Chrome UA, no `Referer`, no `Accept-Language`, and toggled `X-Requested-With` — unlike a browser, so SubscribeStar gated the adult-creator page. ## Changes - SubscribeStar session now uses Firefox UA + `Accept: */*` + `Accept-Language` (via `make_session` extra_headers). - Stamp `Referer: <base>/` per walk (mirrors gallery-dl's `root/`). - Drop the per-request XHR headers; both the feed and load-more use the shared profile (load-more is a plain GET parsed as JSON, exactly like gallery-dl). - Test updated to assert the gallery-dl-parity profile. If it still 302s after deploy, it's a genuine session/subscription issue (gallery-dl itself aborts on that redirect too). CI green on dev (run 1234). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 1 commit 2026-06-17 15:32:23 -04:00

fix(subscribestar): mirror gallery-dl's full request profile (verify_subscriber gate) ... 559d29fe1c

After the delimiter fix, the live cheunart fetch 302'd to /cheunart/verify_
subscriber even with valid .adult cookies (confirmed present: _personalization_id
+ _subscribestar_session on .subscribestar.adult, logged in). Walking gallery-dl's
ENTIRE flow — including the base Extractor._init_session I'd not read — the
divergence is the HTTP request profile, not the cookies or parser.

gallery-dl's default (cookies-only) mode sends, on EVERY request including the
first creator-page GET: a Firefox UA, Accept: */*, Accept-Language, and a same-
site Referer (root/), with NO X-Requested-With anywhere (the load-more endpoint
is a plain GET parsed as JSON). Our Chrome UA + missing Referer + XHR toggling
looked unlike a browser → SubscribeStar gated the adult-creator page.

Make our SubscribeStar session identical: Firefox UA + Accept */* + Accept-
Language via make_session extra_headers; stamp Referer=<base>/ per walk; drop the
per-request XHR headers (both feed and load-more now use the shared profile).

Test updated to assert the gallery-dl-parity profile instead of the old
navigation-vs-XHR split.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

bvandeusen merged commit a1d19bd96a into main 2026-06-17 15:32:28 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#120