feat(subscribestar): port gallery-dl doc + audio attachment extraction
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m24s

Some SubscribeStar posts deliver content only through document/audio attachments,
which live OUTSIDE data-gallery. Port gallery-dl's _media_from_post for them:
- docs: scope uploads-docs..post-edit_form, split on doc_preview blocks, take the
  href URL + doc_preview-title + data-upload-id (kind=attachment).
- audio: scope uploads-audios..post-edit_form, split on audio_preview-data
  blocks, take the src URL + audio_preview-title + data-upload-id (kind=audio).

The existing downloader handles them unchanged (plain streaming GET; the file
validator only inspects image/video extensions via is_validatable, so PDFs/zips/
audio pass straight through, no quarantine).

Test covers doc + audio extraction (the cheunart sample has none, so this pins
gallery-dl's documented markup shape).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-17 16:18:10 -04:00
parent 8771364cee
commit 976f581aa2
2 changed files with 79 additions and 4 deletions
+27
View File
@@ -227,6 +227,33 @@ def test_extract_media_skips_locked_previews():
assert [m.media_id for m in items] == ["1"]
def test_extract_media_includes_doc_and_audio_attachments():
# gallery-dl _media_from_post: document (uploads-docs/doc_preview, href) and
# audio (uploads-audios/audio_preview-data, src) attachments are separate from
# data-gallery — some posts deliver content ONLY through these.
client = SubscribeStarClient(None)
chunk = (
'<div class="post " data-id="888">'
'<div class="uploads-docs">'
'<div class="doc_preview" data-upload-id="501">'
'<a href="/post_uploads/d/chapter.pdf">'
'<div class="doc_preview-title">chapter.pdf</div></a></div></div>'
'<div class="uploads-audios">'
'<div class="audio_preview-data" data-upload-id="502">'
'<audio src="/post_uploads/a/track.mp3"></audio>'
'<div class="audio_preview-title">track.mp3</div></div></div>'
'<div class="post-edit_form"></div></div>'
)
post = {"id": "888", "_html": chunk, "_base": "https://subscribestar.adult"}
by_kind = {m.kind: m for m in client.extract_media(post, {})}
assert by_kind["attachment"].url == "https://subscribestar.adult/post_uploads/d/chapter.pdf"
assert by_kind["attachment"].filename == "chapter.pdf"
assert by_kind["attachment"].media_id == "501"
assert by_kind["audio"].url == "https://subscribestar.adult/post_uploads/a/track.mp3"
assert by_kind["audio"].filename == "track.mp3"
assert by_kind["audio"].media_id == "502"
def test_text_post_has_no_media_but_keeps_body():
client = SubscribeStarClient(None)
[post] = client._parse_posts(_feed_page(_post_html("111", body="<p>text only</p>")))