bvandeusen/GallerySubscriber
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

fix(downloader): don't misclassify tier-gated runs with per-item video 403s

Browse Source 
When a Patreon source loses tier access, gallery-dl emits "Not allowed to
view post N" warnings for every post AND yt-dlp tries to fetch HLS manifests
which also 403. The ytdl error text contained "HTTP Error 403: Forbidden",
which the classifier's ACCESS_DENIED_PATTERNS matched against the full
combined stdout+stderr — so the run was labeled ACCESS_DENIED instead of
TIER_LIMITED.

Two fixes in one edit, because they're interlocked:
- Compute per-item vs source-level error lines upfront. has_actual_error
  now reflects only source-level errors (previously the per-item exclusion
  was gated on skip evidence, which tier-limited runs don't produce since
  no content was accessed).
- Strip per-item error lines from `combined` before downstream pattern
  matching so noise from recovered per-item failures doesn't latch onto
  ACCESS_DENIED / HTTP_ERROR / NOT_FOUND classifiers.

Regression test: tier-gated Patreon run with yt-dlp HLS 403s → TIER_LIMITED,
not ACCESS_DENIED.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Bryan Van Deusen
2026-04-21 07:34:33 -04:00
parent 851df294f2
commit c2a2162c86
2 changed files with 75 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/tests/services/test_gallery_dl_categorize.py
+30
View File
@@ -163,6 +163,36 @@ def test_skip_activity_takes_precedence_over_tier_limited():
assert error_type == ErrorType.NO_NEW_CONTENT
def test_tier_limited_with_ytdl_403s_not_access_denied():
"""Real-world Patreon scenario: account lost tier access. Posts show up as
'[patreon][warning] Not allowed to view post N' (no downloads attempted)
AND yt-dlp tries to fetch video HLS manifests that are also tier-gated,
producing '[downloader.ytdl][error] HTTP Error 403: Forbidden' per item.
Before the fix, the 'Forbidden' text in per-item ytdl errors latched onto
ACCESS_DENIED_PATTERNS and misclassified the whole source as ACCESS_DENIED.
Correct behavior: TIER_LIMITED (the warnings are the dominant signal; the
per-item 403s are just tier gating manifesting at the HLS layer)."""
service = _make_service()
tier_warnings = "\n".join(
f"[patreon][warning] Not allowed to view post {100000 + i}"
for i in range(20)
)
ytdl_errors = "\n".join(
"[downloader.ytdl][error] ExtractorError: Failed to download m3u8 "
"information: HTTP Error 403: Forbidden "
f"(caused by <HTTPError 403: Forbidden>) ({i}/4)"
for i in range(1, 5)
) + "\n[download][error] Failed to download 02_video.mp4\n"
stderr = tier_warnings + "\n" + ytdl_errors
stdout = ""
error_type, message = service._categorize_error(1, stdout, stderr)
assert error_type == ErrorType.TIER_LIMITED, f"got {error_type}: {message}"
assert "20" in message
def test_ytdl_missing_with_skips_classifies_as_no_new_content():
"""Real-world scenario from a Patreon source: container is missing yt-dlp
so every video logs '[downloader.ytdl][error] Cannot import yt-dlp' plus