fix(subscribestar): match gallery-dl's generic post delimiter (live-feed drift) #119

Merged
bvandeusen merged 2 commits from dev into main 2026-06-17 15:07:59 -04:00

2 Commits

Author SHA1 Message Date
bvandeusen 204d341a99 fix(subscribestar): match gallery-dl's generic post delimiter (live-feed drift)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 36s
CI / integration (push) Successful in 3m16s
The native client split the feed on `<div class="post is-shown`, but `is-shown`
is added by SubscribeStar's infinite-scroll JS when a post scrolls into view —
present in a browser-SAVED page (what the Step-0 characterization used) but
ABSENT from the raw server HTML we and gallery-dl actually fetch. So the live
feed (cheunart) parsed to zero posts and raised a false SubscribeStarDriftError.

Align with gallery-dl's proven `_pagination`: split on the generic
`<div class="post ` (trailing space rules out the hyphenated post-content/
post-date/post-body siblings). Also mirror gallery-dl's redirect-based gating
detection (/verify_subscriber, /age_confirmation_warning => auth, not drift).

Regression tests: raw server markup without is-shown now parses; an age-wall
redirect raises SubscribeStarAuthError.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 15:04:05 -04:00
bvandeusen 9201b7b539 diag(subscribestar): name the interstitial in the drift error (title + type)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m13s
The XHR fix worked (we now get a real 93KB HTML page, not JSON) but cheunart
still drifts — we're being served a full HTML page that isn't the feed. Add
_describe_page(): the drift error now reports the page <title> + which known
interstitial it resembles (cloudflare/bot-challenge, age-gate, login, captcha),
so the next run names the actual cause instead of "markup changed". Strong
suspicion: a Cloudflare challenge (python-requests has no JS; cf_clearance is
UA-locked and our hardcoded UA likely differs from the cookie-capturing browser).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 14:52:28 -04:00