feat: SubscribeStar joins the membership roster (387 D1)

The second platform through the seam note 3970 contracted, characterized first from a live capture of the account's /subscriptions page (note 3989). The capture lives in the gitignored captures dir; the committed fixture is hand-built with invented values and was verified tag-for-tag against it - card wrappers, both table heads, and every distinct row shape - before any code depended on it. What the page is, and the three decisions it forced: The table IS the status. SubscribeStar has no per-row status word: a creator is either in the active_subscriptions card or the cancelled_subscriptions one. The card's data-identifier is stored verbatim as Membership.status and mapped in MEMBERSHIP_STATUS, keyed on the identifier rather than the table class because the cancelled table's class names the same list differently (for-unsubscribed_users). The creator's numeric data-user-id is the key, not the slug. A slug re-keys when a creator renames; the old row stops appearing; and a disappearance is exactly what reconciliation reads as a lapse. Keyed on the slug, a rename would have told a paying subscriber they had cancelled. The slug rides as vanity, where the identity join already looks for a handle. Price is kept as text, never parsed into amount_cents. A bare $ names no currency and a page price is not proven to be the charge - 3970 finding 4. Tier names live behind a per-row modal and are not fetched. Refusals, because SubscribeStar offers nothing like Patreon's meta.pagination.total and every conclusion downstream is drawn from absence. The parser raises when: the active card is missing (auth error on a login/age wall, drift otherwise); a row lacks a numeric creator id or a creator link; anything renders after a card's table; or the page carries a page= link. Both cards are paginatable (app#embed_pagination) and the captured account was too small to show what pagination looks like, so possible pagination is a roster FC cannot prove complete. A loud error on a larger account beats a quiet half-list. A missing cancelled card is not drift, and a creator in both tables is reported once, as active. Fetched from subscribestar.adult, not the .art the capture came from: FC's requests never clear the .art age wall with the 18+ cookie (1259, 1284). Whether /subscriptions on .adult authenticates exactly as .art did in the browser is untested - if not, the sweep records a visible error and C6 shows its unavailable rung. The seam leak D1 found. Note 3970 promised a second platform would be one builders line plus the client method. The sweep instead called current_user_id() on every client, which only Patreon's has, so SubscribeStar would have raised AttributeError on the first sweep. roster_user_id probes it with getattr, the same way the sweep already probes iter_memberships. Two existing tests were passing for the wrong reason and now can fail: - "a platform that has never been characterised says nothing" named SubscribeStar, and stayed green only because active_patron is not a SubscribeStar word. Now uses hentaifoundry, with a positive SubscribeStar test beside it. - the freshness test gave SubscribeStar a Patreon word, so the vocabulary excluded it and deleting the freshness gate outright would have left it green. It now uses cancelled_subscriptions, making the gate the only thing that excludes it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
2026-09-13 10:53:01 -04:00
parent 529d4bff57
commit ef91fcfd26
7 changed files with 532 additions and 9 deletions
@@ -0,0 +1,3 @@
+<!DOCTYPE html><html><head><title>My Subscriptions | SubscribeStar.adult</title></head><body><div class="layout"><div class="section-body"><div class="warnings-wrapper"></div><div class="card for-table" data-identifier="active_subscriptions" data-view="app#embed_pagination" id="subscriptions_active"><h2 class="card-title">Active Subscriptions</h2><table class="details_table for-active_subscriptions"><thead><tr><th class="for-name">Profile</th><th class="for-updates">Updates</th><th class="for-date">Subscribed</th><th class="for-renewal">Renewed</th><th class="for-date">Paused</th><th class="for-earnings for-subscription_price">Price</th><th class="for-discord">Discord</th><th class="for-actions for-large_screen">Actions</th></tr></thead><tbody><tr><td class="for-name"><a href="/creator-alpha"><div class="inline_user_name"><img data-view="app#avatar" data-type="avatar" data-user-id="1001" alt="Creator Alpha" src="https://cdn.example.invalid/avatars/1001.jpg" />Creator Alpha</div> </a></td><td class="for-actions has-icons"><button class="details_table-link for-updates-unsubscribe" data-modal="/creator-alpha/unsubscribe" title="Unsubscribe from post updates">Mute updates</button></td><td class="for-date">Jan 2026</td><td class="for-renewal">Mar 2026</td><td class="for-date">-</td><td class="for-earnings for-subscription_price"><span class="subscription_cost">$5</span><span class="for-tiers-list" data-modal="/creator-alpha/tier_details" title="Show tiers"><svg class="icon"><use></use></svg>
+</span></td><td class="for-discord"><button class="details_table-link" data-modal="/creator-alpha/discord"><i class="md_icon is-fa_discord"><svg class="icon"><use></use></svg> </i><span>Joined</span></button></td><td class="for-actions is-multiple for-large_screen"><button class="details_table-link" data-modal="/creator-alpha/manage">Manage plan</button></td><tr class="for-actions"><td class="is-inline" colspan="100"><small>Manage options</small><button class="details_table-link" data-modal="/creator-alpha/manage">Manage plan</button></td></tr></tr><tr><td class="for-name"><a href="/creator-beta"><div class="inline_user_name"><img data-view="app#avatar" data-type="avatar" data-user-id="2002002" alt="Creator Beta Studio" src="https://cdn.example.invalid/avatars/2002002.jpg" />Creator Beta Studio</div> </a></td><td class="for-actions has-icons"><button class="details_table-link for-updates-unsubscribe" data-modal="/creator-beta/unsubscribe" title="Unsubscribe from post updates">Mute updates</button></td><td class="for-date">Feb 2025</td><td class="for-renewal">Mar 2026</td><td class="for-date">-</td><td class="for-earnings for-subscription_price"><span class="subscription_cost">$12</span><span class="for-tiers-list" data-modal="/creator-beta/tier_details" title="Show tiers"><svg class="icon"><use></use></svg>
+</span></td><td class="for-discord">&mdash;</td><td class="for-actions is-multiple for-large_screen"><button class="details_table-link" data-modal="/creator-beta/manage">Manage plan</button></td><tr class="for-actions"><td class="is-inline" colspan="100"><small>Manage options</small><button class="details_table-link" data-modal="/creator-beta/manage">Manage plan</button></td></tr></tr></tbody></table></div><div class="card for-table" data-identifier="cancelled_subscriptions" data-view="app#embed_pagination" id="subscriptions_cancelled"><h2 class="card-title">Cancelled subscriptions</h2><table class="details_table for-unsubscribed_users"><thead><tr><th class="for-name">Profile</th><th class="for-date">Unsubscribed</th><th class="for-earnings">Price</th><th class="for-actions">Actions</th></tr></thead><tbody><tr><td class="for-name"><a href="/creator-gamma"><div class="inline_user_name"><img data-view="app#avatar" data-type="avatar" data-user-id="3003003" alt="Creator Gamma" src="https://cdn.example.invalid/avatars/3003003.jpg" />Creator Gamma</div> </a></td><td class="for-date">Dec 2025</td><td class="for-earnings">$3</td><td class="for-actions"><button class="details_table-link" data-modal="/creator-gamma/resubscribe">Renew</button></td></tr><tr><td class="for-name"><a href="/creator-delta"><div class="inline_user_name"><img data-view="app#avatar" data-type="avatar" data-user-id="4004" alt="Creator Delta" src="https://cdn.example.invalid/avatars/4004.jpg" />Creator Delta</div> </a></td><td class="for-date">Aug 2025</td><td class="for-earnings">$10</td><td class="for-actions"><button class="details_table-link" data-modal="/creator-delta/resubscribe">Renew</button></td></tr></tbody></table></div></div></div></body></html>
@@ -74,8 +74,22 @@ def test_an_unrecognised_status_says_nothing(status):


 def test_a_platform_that_has_never_been_characterised_says_nothing():
-    """SubscribeStar and FANBOX (D1) inherit silence, not a Patreon guess."""
-    assert gated_reason("subscribestar", "active_patron") is None
+    """An uncharacterised platform inherits silence, not a Patreon guess.
+
+    This named SubscribeStar until D1 characterised it (note #3989). Left as it
+    was, it would have kept passing only because `active_patron` is not a
+    SubscribeStar word, testing nothing its name claims.
+    """
+    assert gated_reason("hentaifoundry", "active_patron") is None
+
+
+def test_subscribestar_explains_the_gate_in_its_own_words():
+    """SubscribeStar's status is which table the creator sits in, and those
+    identifiers reach the same three reasons as Patreon's words."""
+    assert gated_reason("subscribestar", "active_subscriptions") == GATED_TIER
+    assert gated_reason("subscribestar", "cancelled_subscriptions") == GATED_LAPSED
+    # Patreon's vocabulary does not leak across platforms.
+    assert gated_reason("subscribestar", "former_patron") is None


 # --- the join, against the database ----------------------------------------
@@ -178,8 +192,11 @@ async def test_one_platforms_fresh_sweep_does_not_vouch_for_anothers(db):
        url="https://subscribestar.adult/maewix",
    )
    await _membership(db, status="former_patron")
+    # SubscribeStar's REAL word, so that the freshness gate is the only thing
+    # excluding this membership. With a Patreon word (as this test first had),
+    # the vocabulary excluded it and removing the gate would have left it green.
    await _membership(db, platform="subscribestar", campaign="s1",
-                      status="former_patron",
+                      status="cancelled_subscriptions",
                      url="https://subscribestar.adult/maewix",
                      details={"campaign": {"vanity": "maewix"}})
    await _synced(db)  # patreon only
@@ -170,8 +170,12 @@ def test_the_status_map_contains_only_characterised_values():
    contain. Adding it because it "obviously" belongs is precisely the guess
    this test exists to stop.
    """
+    # subscribestar added with its own capture (note #3989, 2026-09-13). Its
+    # "words" are the two table identifiers, because the page carries no
+    # per-row status at all.
    assert MEMBERSHIP_STATUS == {
        "patreon": {"active_patron": True, "former_patron": False},
+        "subscribestar": {"active_subscriptions": True, "cancelled_subscriptions": False},
    }


@@ -0,0 +1,261 @@
+"""SubscribeStarClient.iter_memberships — parsing only, no network (#387 D1).
+
+The fixture mirrors a REAL capture of the operator's /subscriptions page
+(Scribe note #3989) with every value invented. Its structure was checked
+against the capture tag for tag before this file was written: the card
+wrappers, both table heads, and every distinct row shape — including the two
+forms the Discord cell takes, and the narrow-screen `<tr class="for-actions">`
+that active rows nest INSIDE their own row.
+
+Most of what follows pins refusals. SubscribeStar gives no completeness signal
+the way Patreon's `meta.pagination.total` does, and every conclusion drawn from
+this roster is drawn from absence, so the parser's job is to raise whenever it
+cannot vouch for the whole list.
+
+`_get` is stubbed rather than mocked at the socket: these tests are about what
+the client does with a page, and the HTTP path is covered by
+test_subscribestar_client.py.
+"""
+
+from pathlib import Path
+from types import SimpleNamespace
+
+import pytest
+
+from backend.app.services.membership_roster import has_paid_access, roster_user_id
+from backend.app.services.native_ingest_common import Membership
+from backend.app.services.subscribestar_client import (
+    SubscribeStarAuthError,
+    SubscribeStarClient,
+    SubscribeStarDriftError,
+    parse_subscriptions_page,
+)
+
+_FIXTURE = Path(__file__).parent / "fixtures" / "subscribestar_subscriptions_page1.html"
+
+
+@pytest.fixture
+def page():
+    return _FIXTURE.read_text()
+
+
+@pytest.fixture
+def client():
+    return SubscribeStarClient(cookies_path=None)
+
+
+def _serve(client, html):
+    calls = []
+
+    def fake(url, *, headers=None):
+        calls.append(url)
+        return SimpleNamespace(text=html)
+
+    client._get = fake
+    return calls
+
+
+def _by_id(memberships):
+    return {m.campaign_id: m for m in memberships}
+
+
+# --- the request -----------------------------------------------------------
+
+
+def test_the_roster_is_fetched_from_adult_not_art(client, page):
+    """The capture came from `.art`, but FC's requests never clear the `.art`
+    age wall with the 18+ cookie (#1259, #1284). Copying the browser's host is
+    the obvious move and would turn every sweep into an auth error."""
+    calls = _serve(client, page)
+    list(client.iter_memberships())
+    assert calls == ["https://subscribestar.adult/subscriptions"]
+
+
+def test_one_request_for_the_whole_roster(client, page):
+    calls = _serve(client, page)
+    list(client.iter_memberships(user_id="ignored"))
+    assert len(calls) == 1
+
+
+# --- what a membership contains --------------------------------------------
+
+
+def test_both_tables_are_read(page):
+    rows = parse_subscriptions_page(page)
+    assert all(isinstance(m, Membership) for m in rows)
+    assert len(rows) == 4
+    statuses = sorted(m.status for m in rows)
+    assert statuses == [
+        "active_subscriptions", "active_subscriptions",
+        "cancelled_subscriptions", "cancelled_subscriptions",
+    ]
+
+
+def test_the_creator_id_is_the_key_not_the_slug(page):
+    """A slug re-keys when a creator renames; the old row then stops appearing,
+    and reconciliation reads a disappearance as a lapse. The numeric id does not
+    change, so a rename can never surface as "you cancelled this creator"
+    (note #3989, CORRECTION 2)."""
+    rows = _by_id(parse_subscriptions_page(page))
+    assert set(rows) == {"1001", "2002002", "3003003", "4004"}
+    alpha = rows["1001"]
+    assert alpha.vanity == "creator-alpha"
+    # The handle is carried as a URL so `match_kind`'s URL-tail fallback reaches
+    # it — which is what a SubscribeStar Source.url ends in.
+    assert alpha.url == "https://subscribestar.adult/creator-alpha"
+    assert alpha.display_name == "Creator Alpha"
+
+
+def test_the_table_is_the_status(page):
+    """No per-row status word exists. The card's `data-identifier` is stored
+    verbatim — and is exactly what MEMBERSHIP_STATUS maps, so a renamed
+    identifier fails here rather than silently reading as unknown."""
+    rows = _by_id(parse_subscriptions_page(page))
+    assert rows["1001"].status == "active_subscriptions"
+    assert rows["3003003"].status == "cancelled_subscriptions"
+    assert has_paid_access("subscribestar", rows["1001"].status) is True
+    assert has_paid_access("subscribestar", rows["3003003"].status) is False
+
+
+def test_price_is_kept_as_text_and_never_parsed_into_money(page):
+    """A bare `$` names no currency and a page price is not proven to be the
+    charge. None keeps "unknown" distinct from zero (#3970 finding 4)."""
+    rows = _by_id(parse_subscriptions_page(page))
+    alpha = rows["1001"]
+    assert alpha.amount_cents is None
+    assert alpha.currency is None
+    assert alpha.details["columns"]["price"] == "$5"
+    assert alpha.tier_names == []
+    assert alpha.is_free_member is False
+
+
+def test_columns_are_paired_by_position_not_by_class(page):
+    """Two active columns share the `for-date` class, and the updates column's
+    <td> does not carry its <th>'s class — pairing by class would merge or drop
+    them. Controls and identity stay out of details."""
+    cols = _by_id(parse_subscriptions_page(page))["1001"].details["columns"]
+    assert cols == {
+        "subscribed": "Jan 2026",
+        "renewed": "Mar 2026",
+        "paused": "-",
+        "price": "$5",
+        "discord": "Joined",
+    }
+    cancelled = _by_id(parse_subscriptions_page(page))["3003003"].details["columns"]
+    assert cancelled == {"unsubscribed": "Dec 2025", "price": "$3"}
+
+
+def test_both_discord_cell_forms_read_as_text(page):
+    """The real page renders a linked Discord as a button with an icon, and an
+    unlinked one as the ENTITY `&mdash;`. Undecoded, the second would store the
+    literal string "&mdash;"."""
+    rows = _by_id(parse_subscriptions_page(page))
+    assert rows["1001"].details["columns"]["discord"] == "Joined"
+    assert rows["2002002"].details["columns"]["discord"] == "—"
+
+
+def test_the_nested_actions_row_is_not_read_as_columns(page):
+    """Active rows nest a narrow-screen `<tr class="for-actions">` inside their
+    own row. Its cells would otherwise zip onto the header as extra columns."""
+    cols = _by_id(parse_subscriptions_page(page))["2002002"].details["columns"]
+    assert "Manage options" not in cols.values()
+    assert set(cols) == {"subscribed", "renewed", "paused", "price", "discord"}
+
+
+# --- refusals --------------------------------------------------------------
+
+
+def test_a_creator_in_both_tables_is_reported_once_as_active(page):
+    """A current subscription is the fact that matters. Written the other way
+    round, the cancelled row would overwrite the active one in the upsert and
+    tell a paying subscriber they had lapsed."""
+    duplicated = page.replace('data-user-id="3003003"', 'data-user-id="1001"')
+    rows = parse_subscriptions_page(duplicated)
+    matches = [m for m in rows if m.campaign_id == "1001"]
+    assert len(matches) == 1
+    assert matches[0].status == "active_subscriptions"
+
+
+def test_an_account_that_never_cancelled_has_no_cancelled_card(page):
+    """Not drift — such an account plausibly renders no cancelled table."""
+    start = page.index('<div class="card for-table" data-identifier="cancelled_subscriptions"')
+    end = page.index("</table></div>", start) + len("</table></div>")
+    rows = parse_subscriptions_page(page[:start] + page[end:])
+    assert {m.status for m in rows} == {"active_subscriptions"}
+
+
+def test_a_login_wall_is_auth_not_drift():
+    """The fix for this is a fresh credential, not a scraper change — and the
+    distinction survives into membership_sync.last_error_type, where it is what
+    makes the UI's advice correct."""
+    wall = '<html><body><form action="/session/new" data-role="sign_in"></form></body></html>'
+    with pytest.raises(SubscribeStarAuthError):
+        parse_subscriptions_page(wall)
+
+
+def test_an_unrecognised_page_is_drift():
+    with pytest.raises(SubscribeStarDriftError):
+        parse_subscriptions_page("<html><title>Something else</title><body></body></html>")
+
+
+def test_a_row_without_a_creator_id_refuses_the_whole_roster(page):
+    """Identity is the point (#3970 §4). Skipping the row instead would return a
+    roster one creator short, which downstream reads as a cancellation."""
+    with pytest.raises(SubscribeStarDriftError, match="data-user-id"):
+        parse_subscriptions_page(page.replace(' data-user-id="2002002"', ""))
+
+
+def test_a_non_numeric_creator_id_is_drift(page):
+    with pytest.raises(SubscribeStarDriftError, match="data-user-id"):
+        parse_subscriptions_page(page.replace('data-user-id="2002002"', 'data-user-id="beta"'))
+
+
+def test_a_row_without_a_creator_link_is_drift(page):
+    with pytest.raises(SubscribeStarDriftError, match="creator link"):
+        parse_subscriptions_page(page.replace('<a href="/creator-gamma">', "<a>"))
+
+
+@pytest.mark.parametrize("card", ["active_subscriptions", "cancelled_subscriptions"])
+def test_content_after_a_table_is_treated_as_possible_pagination(page, card):
+    """Both cards are paginatable (`app#embed_pagination`), and the capture was
+    too short to show what pagination looks like. Anything rendered after a
+    table is therefore a roster FC cannot prove complete — raised, never
+    returned short."""
+    start = page.index(f'data-identifier="{card}"')
+    close = page.index("</table>", start) + len("</table>")
+    paginated = page[:close] + '<nav class="pagination"><a>2</a></nav>' + page[close:]
+    with pytest.raises(SubscribeStarDriftError, match="cannot be proven complete"):
+        parse_subscriptions_page(paginated)
+
+
+def test_a_page_link_anywhere_in_the_roster_is_drift(page):
+    """The same concern for a paginator rendered outside the card."""
+    paged = page.replace("</body>", '<a href="/subscriptions?page=2">Next</a></body>')
+    with pytest.raises(SubscribeStarDriftError, match="page="):
+        parse_subscriptions_page(paged)
+
+
+def test_a_drift_error_mid_page_yields_nothing(client, page):
+    """The whole page is parsed before anything is yielded, so a caller can never
+    be left holding the rows that came before the bad one."""
+    _serve(client, page.replace(' data-user-id="4004"', ""))
+    got = []
+    with pytest.raises(SubscribeStarDriftError):
+        for m in client.iter_memberships():
+            got.append(m)
+    assert got == []
+
+
+# --- the sweep's user-id probe ---------------------------------------------
+
+
+def test_a_client_without_current_user_id_needs_no_id():
+    """The seam leak D1 found: the sweep called `current_user_id()` on every
+    client, which exists only on Patreon's. Note #3970 promised a second platform
+    would be one `builders` line and the client method; without this probe it
+    would have raised AttributeError on the first sweep."""
+    assert roster_user_id(SubscribeStarClient(cookies_path=None)) is None
+
+
+def test_a_client_with_current_user_id_is_asked_for_it():
+    assert roster_user_id(SimpleNamespace(current_user_id=lambda: "248453")) == "248453"