fix(subscribestar): port gallery-dl content + preview-skip extraction (body bug) #121

2026-06-17T16:10:39-04:00

bvandeusen commented

2026-06-17 16:10:39 -04:00

What

Every imported SubscribeStar post rendered its body as a bogus "264 / 265" (a pagination counter). This ports gallery-dl's exact _data_from_post content rule and _media_from_post preview guard.

Root cause

Our balanced-</div> body regex either returned empty or over-captured into sibling upload divs and the View next posts (N / M) pagination counter. Replaced with gallery-dl's literal delimiters: content between <div class="post-content" data-role="post_content-text"> and </div><div class="post-uploads for-youtube", then strip the trix editor's <html><body>…</body></html> document wrapper to its inner. Verified against the live cheunart sample (clean per-post bodies; empty for genuinely text-less posts).

Also (gallery-dl parity)

Port _media_from_post's preview guard: skip gallery items whose URL is under /previews (locked/blurred teasers) — the SubscribeStar analog of the Patreon gated-preview bug (#874). A locked post now yields no downloadable media.

Tests

Body must not bleed into the pagination counter; trix html-document wrapper stripped; /previews items skipped. Fixture updated to include the youtube-uploads close marker present in real markup.

Note

Fixes new walks only — already-imported cheunart posts need a recapture to rewrite their bodies. Remaining gallery-dl parity gaps tracked separately: doc/audio attachments (uploads-docs/uploads-audios) and post tags.

CI green on dev unit/lint/frontend (run 1239); integration in progress (slow runner).

🤖 Generated with Claude Code

## What Every imported SubscribeStar post rendered its body as a bogus "264 / 265" (a pagination counter). This ports gallery-dl's exact `_data_from_post` content rule and `_media_from_post` preview guard. ## Root cause Our balanced-`</div>` body regex either returned empty or over-captured into sibling upload divs and the `View next posts (N / M)` pagination counter. Replaced with gallery-dl's literal delimiters: content between `<div class="post-content" data-role="post_content-text">` and `</div><div class="post-uploads for-youtube"`, then strip the trix editor's `<html><body>…</body></html>` document wrapper to its inner. Verified against the live cheunart sample (clean per-post bodies; empty for genuinely text-less posts). ## Also (gallery-dl parity) Port `_media_from_post`'s preview guard: skip gallery items whose URL is under `/previews` (locked/blurred teasers) — the SubscribeStar analog of the Patreon gated-preview bug (#874). A locked post now yields no downloadable media. ## Tests Body must not bleed into the pagination counter; trix html-document wrapper stripped; `/previews` items skipped. Fixture updated to include the youtube-uploads close marker present in real markup. ## Note Fixes new walks only — already-imported cheunart posts need a recapture to rewrite their bodies. Remaining gallery-dl parity gaps tracked separately: doc/audio attachments (`uploads-docs`/`uploads-audios`) and post tags. CI green on dev unit/lint/frontend (run 1239); integration in progress (slow runner). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 1 commit 2026-06-17 16:10:39 -04:00

fix(subscribestar): port gallery-dl's content + preview-skip extraction faithfully

CI / lint (push) Successful in 2s

Details

CI / frontend-build (push) Successful in 21s

Details

CI / backend-lint-and-test (push) Successful in 28s

Details

CI / integration (push) Successful in 3m20s

Details

8771364cee

Body rendered as a bogus '264 / 265' on every post: our balanced-</div> body
regex either returned empty or over-captured into sibling upload divs and the
'View next posts (N / M)' pagination counter. Replace it with gallery-dl's exact
_data_from_post rule — content between the post_content-text wrapper and the
youtube-uploads div (literal markers), then strip the trix editor's
<html><body>…</body></html> document wrapper to its inner. Verified against the
live cheunart sample: clean per-post bodies, empty for genuinely text-less posts.

Also port gallery-dl's _media_from_post preview guard: skip gallery items whose
URL is under /previews (locked/blurred teasers) — the SubscribeStar analog of
the Patreon gated-preview bug (#874); this is why a locked post yields no media.

Tests: body must not bleed into the pagination counter; trix html-document
wrapper stripped; /previews items skipped. Fixture now includes the youtube-
uploads close marker present in real markup.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

bvandeusen scheduled this pull request to auto merge when all checks succeed 2026-06-17 16:10:55 -04:00

bvandeusen merged commit 6f02806aec into main

2026-06-17 16:10:56 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#121