bvandeusen/FabledCurator
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 12 Wiki Activity

fix(subscribestar): mirror gallery-dl's full request profile (verify_subscriber gate) #120

Merged
bvandeusen merged 1 commits from dev into main 2026-06-17 15:32:28 -04:00
Conversation 0 Commits 1 Files Changed 2
+42 -29
bvandeusen commented 2026-06-17 15:32:23 -04:00
Owner
Copy Link
Copy Source

What

After the delimiter fix, the live cheunart fetch 302'd to /cheunart/verify_subscriber even with valid cookies (confirmed via devtools: _personalization_id + _subscribestar_session present on .subscribestar.adult, logged in). This makes our SubscribeStar HTTP request profile identical to gallery-dl's so it stops tripping that gate.

Root cause (walking gallery-dl's entire flow, base Extractor included)

The divergence isn't cookies or parsing — it's the request profile. gallery-dl's _init_session (default cookies-only mode) sends on every request, including the first creator-page GET:

  • User-Agent: …Firefox/140.0 (not Chrome)
  • Accept: */*
  • Accept-Language: en-US,en;q=0.5
  • Referer: https://subscribestar.adult/ (same-site)
  • no X-Requested-With anywhere (the "load more" endpoint is a plain GET parsed as JSON)

Ours sent a Chrome UA, no Referer, no Accept-Language, and toggled X-Requested-With — unlike a browser, so SubscribeStar gated the adult-creator page.

Changes

  • SubscribeStar session now uses Firefox UA + Accept: */* + Accept-Language (via make_session extra_headers).
  • Stamp Referer: <base>/ per walk (mirrors gallery-dl's root/).
  • Drop the per-request XHR headers; both the feed and load-more use the shared profile (load-more is a plain GET parsed as JSON, exactly like gallery-dl).
  • Test updated to assert the gallery-dl-parity profile.

If it still 302s after deploy, it's a genuine session/subscription issue (gallery-dl itself aborts on that redirect too).

CI green on dev (run 1234).

🤖 Generated with Claude Code

## What After the delimiter fix, the live cheunart fetch 302'd to `/cheunart/verify_subscriber` **even with valid cookies** (confirmed via devtools: `_personalization_id` + `_subscribestar_session` present on `.subscribestar.adult`, logged in). This makes our SubscribeStar HTTP request profile identical to gallery-dl's so it stops tripping that gate. ## Root cause (walking gallery-dl's *entire* flow, base `Extractor` included) The divergence isn't cookies or parsing — it's the request profile. gallery-dl's `_init_session` (default cookies-only mode) sends on **every** request, including the first creator-page GET: - `User-Agent: …Firefox/140.0` (not Chrome) - `Accept: */*` - `Accept-Language: en-US,en;q=0.5` - `Referer: https://subscribestar.adult/` (same-site) - **no `X-Requested-With`** anywhere (the "load more" endpoint is a plain GET parsed as JSON) Ours sent a Chrome UA, no `Referer`, no `Accept-Language`, and toggled `X-Requested-With` — unlike a browser, so SubscribeStar gated the adult-creator page. ## Changes - SubscribeStar session now uses Firefox UA + `Accept: */*` + `Accept-Language` (via `make_session` extra_headers). - Stamp `Referer: <base>/` per walk (mirrors gallery-dl's `root/`). - Drop the per-request XHR headers; both the feed and load-more use the shared profile (load-more is a plain GET parsed as JSON, exactly like gallery-dl). - Test updated to assert the gallery-dl-parity profile. If it still 302s after deploy, it's a genuine session/subscription issue (gallery-dl itself aborts on that redirect too). CI green on dev (run 1234). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
bvandeusen added 1 commit 2026-06-17 15:32:23 -04:00
fix(subscribestar): mirror gallery-dl's full request profile (verify_subscriber gate)
CI / lint (push) Successful in 2s
Details
CI / frontend-build (push) Successful in 21s
Details
CI / backend-lint-and-test (push) Successful in 32s
Details
CI / integration (push) Successful in 3m18s
Details
559d29fe1c 
After the delimiter fix, the live cheunart fetch 302'd to /cheunart/verify_
subscriber even with valid .adult cookies (confirmed present: _personalization_id
+ _subscribestar_session on .subscribestar.adult, logged in). Walking gallery-dl's
ENTIRE flow — including the base Extractor._init_session I'd not read — the
divergence is the HTTP request profile, not the cookies or parser.

gallery-dl's default (cookies-only) mode sends, on EVERY request including the
first creator-page GET: a Firefox UA, Accept: */*, Accept-Language, and a same-
site Referer (root/), with NO X-Requested-With anywhere (the load-more endpoint
is a plain GET parsed as JSON). Our Chrome UA + missing Referer + XHR toggling
looked unlike a browser → SubscribeStar gated the adult-creator page.

Make our SubscribeStar session identical: Firefox UA + Accept */* + Accept-
Language via make_session extra_headers; stamp Referer=<base>/ per walk; drop the
per-request XHR headers (both feed and load-more now use the shared profile).

Test updated to assert the gallery-dl-parity profile instead of the old
navigation-vs-XHR split.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bvandeusen merged commit a1d19bd96a into main 2026-06-17 15:32:28 -04:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
renovate-bot bvandeusen
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#120