fix(downloads): forward Patreon Referer/Origin to yt-dlp for Mux videos
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 13s
CI / frontend-build (push) Successful in 21s
CI / intimp (push) Successful in 3m58s
CI / intapi (push) Successful in 7m46s
CI / intcore (push) Successful in 8m19s

Operator-flagged 2026-06-01 (DaferQ patreon, event #38919). Video posts
hosted on Mux carry a JWT that encodes a playback restriction policy
(`playback_restriction_id` in the token). The token signature alone is
not sufficient — Mux's CDN ALSO checks Referer/Origin on every fetch.

gallery-dl's HEAD probe to `stream.mux.com/<id>.m3u8?token=...` returns
200 (token is valid, HEAD sends no Referer that Mux's policy rejects),
but when yt-dlp follows up with the actual manifest GET it sends its own
default Referer and Mux 403s. yt-dlp retries 4 times, gives up, the
single video fails — every other image in the same post still downloads.

Static `downloader.ytdl.raw-options.http_headers` block now pins Referer
and Origin to `https://www.patreon.com` so yt-dlp's manifest fetch
clears the policy check.

Headers-only restrictions are now handled. Mux IP-range restrictions
(if a creator opts into them) remain unfixable from our worker — those
would need a Patreon-region IP. Per-video PARTIAL classifier (shipped
in plan #544 last commit) already handles the residual case: a run that
grabs all images and fails 1 video classifies as status=ok rather than
flipping the whole event red.
This commit is contained in:
2026-06-01 19:00:50 -04:00
parent 19aece1fc4
commit 66ff671f09
2 changed files with 30 additions and 0 deletions
+19
View File
@@ -257,6 +257,25 @@ class GalleryDLService:
"part-directory": str(self._config_dir / "temp"),
"retries": 3,
"timeout": 120.0,
# Forward Patreon as Referer/Origin to yt-dlp when it
# fetches video manifests. Operator-flagged 2026-06-01
# (DaferQ patreon): video posts hosted on Mux carry a JWT
# playback restriction that checks Referer/Origin on every
# request — not just the token signature. gallery-dl's
# HEAD probe to stream.mux.com returns 200 (the token is
# valid), but yt-dlp's actual GET-with-Range to fetch the
# m3u8 manifest 403s because yt-dlp sends its own default
# Referer, which Mux's policy rejects. Forcing the right
# headers fixes the headers-only case; Mux IP-range
# restrictions are unfixable from here.
"ytdl": {
"raw-options": {
"http_headers": {
"Referer": "https://www.patreon.com/",
"Origin": "https://www.patreon.com",
},
},
},
},
"output": {"progress": True},
}
+11
View File
@@ -315,3 +315,14 @@ def test_categorize_tier_limited_wins_over_partial(gdl):
return_code=1, stdout=stdout, stderr=stderr,
)
assert etype == ErrorType.TIER_LIMITED
def test_default_config_forwards_patreon_referer_to_ytdl(gdl):
"""Operator-flagged 2026-06-01: Mux video playback restrictions reject
yt-dlp's manifest fetch when Referer/Origin don't match Patreon. The
fix is a static `downloader.ytdl.raw-options.http_headers` block, so
it's enough to assert the keys land in the default config."""
cfg = gdl._get_default_config()
headers = cfg["downloader"]["ytdl"]["raw-options"]["http_headers"]
assert headers["Referer"] == "https://www.patreon.com/"
assert headers["Origin"] == "https://www.patreon.com"