feat(patreon): scrape creator-page HTML as a campaign-id resolution fallback

Patreon's /api/campaigns?filter[vanity]= lookup returns empty data for creators that plainly exist (operator-flagged 2026-06-06 — Atole etc. erroring at the resolve step). gallery-dl never used that endpoint; it pulls the campaign id out of the creator page's bootstrap JSON. Add the same as a fallback: when the API misses, GET the creator page (bare + /c/ vanity paths) and scrape the first campaign id from any known embedding ("id":"…","type":"campaign" / "campaign":{"data":{"id" / /api/campaigns/<id> / "campaign_id"). API is still tried first (cheap, structured); the page scrape only runs on a miss. Tests: API-empty → page-scrape fallback resolves; _scrape_campaign_id pattern coverage. Existing API-path tests unchanged (happy paths short-circuit before the fallback; failure paths hit the guarded scrape and still return None). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 18:57:39 -04:00
parent 711fd2bb75
commit a559fabdd5
2 changed files with 101 additions and 0 deletions
@@ -91,6 +91,41 @@ async def test_loads_cookies_file_when_present(tmp_path):
    assert kwargs.get("cookies") is not None


+@pytest.mark.asyncio
+async def test_falls_back_to_creator_page_when_api_empty():
+    """When the campaigns API returns empty data (its known failure mode), the
+    resolver scrapes the campaign id out of the creator page (gallery-dl's
+    method)."""
+    api_resp = MagicMock()
+    api_resp.status_code = 200
+    api_resp.json.return_value = {"data": []}
+    page_resp = MagicMock()
+    page_resp.status_code = 200
+    page_resp.text = (
+        '<html><script>window.patreon={"bootstrap":{"campaign":'
+        '{"data":{"id":"7654321","type":"campaign"}}}}</script></html>'
+    )
+
+    def _get(url, **kwargs):
+        return api_resp if "/api/campaigns" in url else page_resp
+
+    with patch(
+        "backend.app.services.patreon_resolver.requests.get", side_effect=_get
+    ):
+        result = await resolve_campaign_id("alice", cookies_path=None)
+    assert result == "7654321"
+
+
+def test_scrape_campaign_id_patterns():
+    from backend.app.services.patreon_resolver import _scrape_campaign_id
+
+    assert _scrape_campaign_id('foo /api/campaigns/424242 bar') == "424242"
+    assert _scrape_campaign_id('"campaign_id":"99"') == "99"
+    assert _scrape_campaign_id('{"id":"5","type":"campaign"}') == "5"
+    assert _scrape_campaign_id("no id here") is None
+    assert _scrape_campaign_id(None) is None
+
+
 # -- resolve_campaign_id_for_source (shared by download + verify) ----------