feat: SubscribeStar joins the membership roster (387 D1)
CI / lint (push) Failing after 2s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
Build images / build-web (push) Successful in 55s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m41s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m12s

The second platform through the seam note 3970 contracted, characterized first from a live capture of the account's /subscriptions page (note 3989). The capture lives in the gitignored captures dir; the committed fixture is hand-built with invented values and was verified tag-for-tag against it - card wrappers, both table heads, and every distinct row shape - before any code depended on it.

What the page is, and the three decisions it forced:

The table IS the status. SubscribeStar has no per-row status word: a creator is either in the active_subscriptions card or the cancelled_subscriptions one. The card's data-identifier is stored verbatim as Membership.status and mapped in MEMBERSHIP_STATUS, keyed on the identifier rather than the table class because the cancelled table's class names the same list differently (for-unsubscribed_users).

The creator's numeric data-user-id is the key, not the slug. A slug re-keys when a creator renames; the old row stops appearing; and a disappearance is exactly what reconciliation reads as a lapse. Keyed on the slug, a rename would have told a paying subscriber they had cancelled. The slug rides as vanity, where the identity join already looks for a handle.

Price is kept as text, never parsed into amount_cents. A bare $ names no currency and a page price is not proven to be the charge - 3970 finding 4. Tier names live behind a per-row modal and are not fetched.

Refusals, because SubscribeStar offers nothing like Patreon's meta.pagination.total and every conclusion downstream is drawn from absence. The parser raises when: the active card is missing (auth error on a login/age wall, drift otherwise); a row lacks a numeric creator id or a creator link; anything renders after a card's table; or the page carries a page= link. Both cards are paginatable (app#embed_pagination) and the captured account was too small to show what pagination looks like, so possible pagination is a roster FC cannot prove complete. A loud error on a larger account beats a quiet half-list. A missing cancelled card is not drift, and a creator in both tables is reported once, as active.

Fetched from subscribestar.adult, not the .art the capture came from: FC's requests never clear the .art age wall with the 18+ cookie (1259, 1284). Whether /subscriptions on .adult authenticates exactly as .art did in the browser is untested - if not, the sweep records a visible error and C6 shows its unavailable rung.

The seam leak D1 found. Note 3970 promised a second platform would be one builders line plus the client method. The sweep instead called current_user_id() on every client, which only Patreon's has, so SubscribeStar would have raised AttributeError on the first sweep. roster_user_id probes it with getattr, the same way the sweep already probes iter_memberships.

Two existing tests were passing for the wrong reason and now can fail:
- "a platform that has never been characterised says nothing" named SubscribeStar, and stayed green only because active_patron is not a SubscribeStar word. Now uses hentaifoundry, with a positive SubscribeStar test beside it.
- the freshness test gave SubscribeStar a Patreon word, so the vocabulary excluded it and deleting the freshness gate outright would have left it green. It now uses cancelled_subscriptions, making the gate the only thing that excludes it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
This commit is contained in:
2026-09-13 10:53:01 -04:00
co-authored by Claude Opus 5
parent 529d4bff57
commit ef91fcfd26
7 changed files with 532 additions and 9 deletions
+28
View File
@@ -60,11 +60,21 @@ log = logging.getLogger(__name__)
# Unknown words are NOT an error: an unrecognised status means the roster # Unknown words are NOT an error: an unrecognised status means the roster
# records evidence it cannot yet interpret, which is a better state than # records evidence it cannot yet interpret, which is a better state than
# dropping the row or asserting a meaning for it. # dropping the row or asserting a meaning for it.
#
# subscribestar: from a live capture of the account's /subscriptions page,
# 2026-09-13 (Scribe note #3989). SubscribeStar gives NO per-row status word —
# a membership's state is which of two tables it sits in — so the "word" stored
# is the table card's own `data-identifier`, verbatim. Those two identifiers are
# the whole vocabulary; there is nothing further to characterise later.
MEMBERSHIP_STATUS: dict[str, dict[str, bool]] = { MEMBERSHIP_STATUS: dict[str, dict[str, bool]] = {
"patreon": { "patreon": {
"active_patron": True, "active_patron": True,
"former_patron": False, "former_patron": False,
}, },
"subscribestar": {
"active_subscriptions": True,
"cancelled_subscriptions": False,
},
} }
@@ -194,6 +204,24 @@ async def _record_sync(session: AsyncSession, platform: str, **values) -> None:
)) ))
def roster_user_id(client) -> str | None:
"""The account id a client's roster walk needs, if that client needs one.
Patreon's members endpoint filters on the account's own user id, so the
sweep has to resolve it first. SubscribeStar's /subscriptions page is simply
the logged-in account's, with nothing to resolve. Probed with `getattr`,
the same way the sweep probes `iter_memberships` itself (rule #169), rather
than called unconditionally.
Calling `current_user_id()` unconditionally was the one place the membership
seam was still Patreon-shaped: note #3970 promised a second platform would be
one `builders` line plus the client method, and D1 found the sweep would
instead have crashed on the first client without that method.
"""
resolve = getattr(client, "current_user_id", None)
return resolve() if resolve is not None else None
async def sync_platform( async def sync_platform(
session: AsyncSession, session: AsyncSession,
*, *,
@@ -43,6 +43,7 @@ import requests
from ..utils.paths import filehash_from_url from ..utils.paths import filehash_from_url
from .native_ingest_common import ( from .native_ingest_common import (
_MAX_429_RETRIES, _MAX_429_RETRIES,
Membership,
NativeAuthError, NativeAuthError,
NativeDriftError, NativeDriftError,
NativeIngestError, NativeIngestError,
@@ -297,6 +298,196 @@ def _extract_creator_name(html: str) -> str | None:
return name or None return name or None
# -- membership roster (#387 D1) ------------------------------------------
#
# Characterized from a live operator capture of the account's /subscriptions
# page, 2026-09-13 — Scribe note #3989. Read that note before changing any of
# this; each constant below is a finding from it, not a guess.
# The account page is fetched from `.adult`. The `.art` age wall never clears
# with the 18+ cookie for FC's requests (see _normalize_ss_host, issues #1259 /
# #1284). The capture itself came from `.art` only because a human had clicked
# through the gate in the browser.
_ROSTER_BASE = "https://subscribestar.adult"
_ROSTER_URL = f"{_ROSTER_BASE}/subscriptions"
# Two tables, and WHICH table a creator sits in is the only status the page
# gives — there is no per-row status word. Keyed on each card's
# `data-identifier`, the one vocabulary that names a state: the table class
# inside the cancelled card says `for-unsubscribed_users`, a different word for
# the same list (note #3989, CORRECTION 1). The identifier is stored verbatim as
# Membership.status and mapped in membership_roster.MEMBERSHIP_STATUS.
_ROSTER_ACTIVE = "active_subscriptions"
_ROSTER_CANCELLED = "cancelled_subscriptions"
_ROSTER_ROW_OPEN = '<td class="for-name">'
# Active rows nest a second `<tr class="for-actions">` INSIDE the row's own
# <tr> — a narrow-screen duplicate of the actions cell. Its <td>s are not
# columns, so every row is cut here before its cells are read.
_ROSTER_NESTED_ROW = '<tr class="for-actions"'
_ROSTER_HREF_RE = re.compile(r'<a href="/([^"/?#]+)"')
_ROSTER_USER_ID_RE = re.compile(r'data-user-id="([^"]*)"')
_ROSTER_NAME_RE = re.compile(r"<img [^>]*>([^<]*)</div>")
_ROSTER_HEAD_RE = re.compile(r'<th class="[^"]*"[^>]*>(.*?)</th>', re.DOTALL)
_ROSTER_CELL_RE = re.compile(r'<td class="[^"]*"[^>]*>(.*?)</td>', re.DOTALL)
_ROSTER_PAGE_LINK_RE = re.compile(r'href="[^"]*[?&]page=\d')
_TAG_RE = re.compile(r"<[^>]+>")
# Columns that hold identity or controls rather than facts about the
# subscription, so they stay out of `details`. Matched on the header's own text,
# lowercased — the page's words, not ours.
_ROSTER_SKIP_COLUMNS = frozenset({"profile", "updates", "actions"})
def _cell_text(fragment: str) -> str:
"""Visible text of a cell: tags dropped, entities decoded, whitespace folded.
Decoding matters here specifically: an active row with no Discord link
renders its cell as the entity `&mdash;`, not as an empty cell.
"""
return " ".join(unescape(_TAG_RE.sub(" ", fragment)).split())
def _roster_table(html: str, identifier: str) -> tuple[str, str] | None:
"""One roster card: (its table markup, whatever trails `</table>` inside it).
None when the card is absent. The trailing part is returned rather than
discarded because it is the pagination check: in the characterized page a
card closes the moment its table does.
"""
start = html.find(f'data-identifier="{identifier}"')
if start < 0:
return None
end = html.find("</table>", start)
if end < 0:
raise SubscribeStarDriftError(
f"SubscribeStar roster card {identifier!r} has no table"
)
close = html.find("</div>", end)
trailing = html[end + len("</table>"): close if close >= 0 else len(html)]
return html[start:end], trailing
def _roster_rows(table: str, identifier: str, base: str) -> list[Membership]:
labels = [_cell_text(h).lower() for h in _ROSTER_HEAD_RE.findall(table)]
body = table[table.find("<tbody>"):] if "<tbody>" in table else ""
starts = [m.start() for m in re.finditer(re.escape(_ROSTER_ROW_OPEN), body)]
rows = []
for n, start in enumerate(starts):
row = body[start: starts[n + 1] if n + 1 < len(starts) else len(body)]
row = row.split(_ROSTER_NESTED_ROW, 1)[0]
href = _ROSTER_HREF_RE.search(row)
if href is None:
raise SubscribeStarDriftError(
f"SubscribeStar roster row in {identifier!r} has no creator link"
)
# The creator's numeric id, NOT the slug, is the key (note #3989,
# CORRECTION 2). A slug re-keys when a creator renames; the old row then
# stops appearing, and a disappearance is exactly what reconciliation
# reads as a lapse. The id survives a rename.
user_id = _ROSTER_USER_ID_RE.search(row)
if user_id is None or not user_id.group(1).isdigit():
raise SubscribeStarDriftError(
f"SubscribeStar roster row in {identifier!r} has no numeric "
f"data-user-id — a membership that cannot be attributed to a "
f"creator is not usable"
)
name = _ROSTER_NAME_RE.search(row)
slug = unescape(href.group(1))
cells = _ROSTER_CELL_RE.findall(row)
rows.append(Membership(
campaign_id=user_id.group(1),
display_name=(_cell_text(name.group(1)) if name else "") or None,
url=f"{base}/{slug}",
vanity=slug,
status=identifier,
# No free-follow concept on this page (#3970 §2: False when a
# platform has none).
is_free_member=False,
# Tier names live behind a per-row modal, not inline. Fetching every
# modal would be N authenticated requests for a field nothing reads.
tier_names=[],
# Deliberately NOT parsed from the price cell: a bare `$` names no
# currency, and a page price is not proven to be the charge (#3970
# finding 4). None keeps "unknown" distinct from zero. The raw text
# is kept in `details`.
amount_cents=None,
currency=None,
details={
# Paired with the header text by POSITION: two columns share the
# `for-date` class, and the updates column's <td> does not carry
# its <th>'s class at all.
"columns": {
label: _cell_text(cell)
for label, cell in zip(labels, cells)
if label not in _ROSTER_SKIP_COLUMNS
},
},
))
return rows
def parse_subscriptions_page(html: str, *, base: str = _ROSTER_BASE) -> list[Membership]:
"""Every membership on the account's /subscriptions page.
Refuses rather than guessing, because every conclusion downstream is drawn
from ABSENCE — a roster that comes back short reads as "you cancelled
those". So this raises when:
* the active card is missing — as SubscribeStarAuthError if the page is a
login or age wall (the fix is credentials), otherwise as drift;
* a row has no creator link or no numeric creator id;
* anything renders after a card's table, or the page carries a `page=` link.
Both cards are paginatable (`data-view="app#embed_pagination"`), and the
characterized account was too small to show what pagination looks like —
so possible pagination is treated as a roster FC cannot prove complete.
A missing cancelled card is NOT drift: an account that has never cancelled
plausibly has no such table. A creator present in both tables is reported
once, as active — a current subscription is the fact that matters.
"""
active = _roster_table(html, _ROSTER_ACTIVE)
if active is None:
if any(marker in html for marker in _LOGIN_MARKERS):
raise SubscribeStarAuthError(
"SubscribeStar served a login/age wall instead of the "
"subscriptions page (cookies expired or age cookie missing)"
)
raise SubscribeStarDriftError(
f"SubscribeStar subscriptions page has no {_ROSTER_ACTIVE!r} card "
f"{_describe_page(html)}"
)
roster_region = html[html.find(f'data-identifier="{_ROSTER_ACTIVE}"'):]
if _ROSTER_PAGE_LINK_RE.search(roster_region):
raise SubscribeStarDriftError(
"SubscribeStar subscriptions page carries a page= link — the roster "
"may be paginated, and FC cannot prove it is complete (note #3989)"
)
memberships: list[Membership] = []
seen: set[str] = set()
for identifier, found in (
(_ROSTER_ACTIVE, active),
(_ROSTER_CANCELLED, _roster_table(html, _ROSTER_CANCELLED)),
):
if found is None:
continue
table, trailing = found
if trailing.strip():
raise SubscribeStarDriftError(
f"SubscribeStar roster card {identifier!r} renders content after "
f"its table — possibly pagination, so the roster cannot be "
f"proven complete (note #3989)"
)
for membership in _roster_rows(table, identifier, base):
if membership.campaign_id in seen:
continue
seen.add(membership.campaign_id)
memberships.append(membership)
return memberships
class SubscribeStarClient: class SubscribeStarClient:
"""Synchronous SubscribeStar HTML-scrape read client. Construct with a path """Synchronous SubscribeStar HTML-scrape read client. Construct with a path
to a Netscape cookies.txt (the same file CredentialService.get_cookies_path to a Netscape cookies.txt (the same file CredentialService.get_cookies_path
@@ -645,6 +836,23 @@ class SubscribeStarClient:
return None return None
return _extract_creator_name(html) return _extract_creator_name(html)
# -- membership roster (#387 D1) ----------------------------------------
def iter_memberships(self, user_id: str | None = None) -> Iterator[Membership]:
"""Yield every subscription the account holds (note #3989).
`user_id` exists for the seam's signature (note #3970) and is ignored:
the page is the logged-in account's own, so there is nothing to resolve.
The sweep only resolves an id for a client that exposes
`current_user_id`, which this one does not.
One request, and the whole page is parsed before anything is yielded, so
a drift error can never leave a caller holding part of a roster.
"""
self._session.headers["Referer"] = f"{_ROSTER_BASE}/"
resp = self._get(_ROSTER_URL)
yield from parse_subscriptions_page(resp.text or "", base=_ROSTER_BASE)
# -- verify ------------------------------------------------------------ # -- verify ------------------------------------------------------------
def verify_auth(self, campaign_id: str) -> tuple[bool | None, str]: def verify_auth(self, campaign_id: str) -> tuple[bool | None, str]:
+8 -6
View File
@@ -1247,17 +1247,20 @@ def sync_memberships() -> str:
from ..services.artist_membership_service import rescan as membership_rescan from ..services.artist_membership_service import rescan as membership_rescan
from ..services.credential_crypto import CredentialCrypto from ..services.credential_crypto import CredentialCrypto
from ..services.credential_service import CredentialService from ..services.credential_service import CredentialService
from ..services.membership_roster import sync_platform from ..services.membership_roster import roster_user_id, sync_platform
from ..services.patreon_client import PatreonClient from ..services.patreon_client import PatreonClient
from ..services.subscribestar_client import SubscribeStarClient
from ._async_session import async_session_factory from ._async_session import async_session_factory
key_path = IMAGES_ROOT / "secrets" / "credential_key.b64" key_path = IMAGES_ROOT / "secrets" / "credential_key.b64"
# platform -> how to build a client from a cookies path. A platform is in # platform -> how to build a client from a cookies path. A platform is in
# the sweep only if it is here AND its client exposes `iter_memberships` # the sweep only if it is here AND its client exposes `iter_memberships`
# AND a credential exists — three independent gates, each silent, so # AND a credential exists — three independent gates, each silent, so a
# adding SubscribeStar (D1) is one line here and nothing else. # platform is added with one line here and nothing else. SubscribeStar (D1)
builders = {"patreon": PatreonClient} # was the second; the only other change it needed was `roster_user_id`
# replacing an unconditional Patreon-only call below.
builders = {"patreon": PatreonClient, "subscribestar": SubscribeStarClient}
async def _run() -> dict: async def _run() -> dict:
async_factory, engine = async_session_factory() async_factory, engine = async_session_factory()
@@ -1287,8 +1290,7 @@ def sync_memberships() -> str:
# slow roster does not block the event loop, and bound the # slow roster does not block the event loop, and bound the
# whole walk rather than only its individual requests. # whole walk rather than only its individual requests.
def _walk(): def _walk():
user_id = _client.current_user_id() return list(_client.iter_memberships(roster_user_id(_client)))
return list(_client.iter_memberships(user_id))
return await asyncio.wait_for( return await asyncio.wait_for(
asyncio.to_thread(_walk), asyncio.to_thread(_walk),
+3
View File
@@ -0,0 +1,3 @@
<!DOCTYPE html><html><head><title>My Subscriptions | SubscribeStar.adult</title></head><body><div class="layout"><div class="section-body"><div class="warnings-wrapper"></div><div class="card for-table" data-identifier="active_subscriptions" data-view="app#embed_pagination" id="subscriptions_active"><h2 class="card-title">Active Subscriptions</h2><table class="details_table for-active_subscriptions"><thead><tr><th class="for-name">Profile</th><th class="for-updates">Updates</th><th class="for-date">Subscribed</th><th class="for-renewal">Renewed</th><th class="for-date">Paused</th><th class="for-earnings for-subscription_price">Price</th><th class="for-discord">Discord</th><th class="for-actions for-large_screen">Actions</th></tr></thead><tbody><tr><td class="for-name"><a href="/creator-alpha"><div class="inline_user_name"><img data-view="app#avatar" data-type="avatar" data-user-id="1001" alt="Creator Alpha" src="https://cdn.example.invalid/avatars/1001.jpg" />Creator Alpha</div> </a></td><td class="for-actions has-icons"><button class="details_table-link for-updates-unsubscribe" data-modal="/creator-alpha/unsubscribe" title="Unsubscribe from post updates">Mute updates</button></td><td class="for-date">Jan 2026</td><td class="for-renewal">Mar 2026</td><td class="for-date">-</td><td class="for-earnings for-subscription_price"><span class="subscription_cost">$5</span><span class="for-tiers-list" data-modal="/creator-alpha/tier_details" title="Show tiers"><svg class="icon"><use></use></svg>
</span></td><td class="for-discord"><button class="details_table-link" data-modal="/creator-alpha/discord"><i class="md_icon is-fa_discord"><svg class="icon"><use></use></svg> </i><span>Joined</span></button></td><td class="for-actions is-multiple for-large_screen"><button class="details_table-link" data-modal="/creator-alpha/manage">Manage plan</button></td><tr class="for-actions"><td class="is-inline" colspan="100"><small>Manage options</small><button class="details_table-link" data-modal="/creator-alpha/manage">Manage plan</button></td></tr></tr><tr><td class="for-name"><a href="/creator-beta"><div class="inline_user_name"><img data-view="app#avatar" data-type="avatar" data-user-id="2002002" alt="Creator Beta Studio" src="https://cdn.example.invalid/avatars/2002002.jpg" />Creator Beta Studio</div> </a></td><td class="for-actions has-icons"><button class="details_table-link for-updates-unsubscribe" data-modal="/creator-beta/unsubscribe" title="Unsubscribe from post updates">Mute updates</button></td><td class="for-date">Feb 2025</td><td class="for-renewal">Mar 2026</td><td class="for-date">-</td><td class="for-earnings for-subscription_price"><span class="subscription_cost">$12</span><span class="for-tiers-list" data-modal="/creator-beta/tier_details" title="Show tiers"><svg class="icon"><use></use></svg>
</span></td><td class="for-discord">&mdash;</td><td class="for-actions is-multiple for-large_screen"><button class="details_table-link" data-modal="/creator-beta/manage">Manage plan</button></td><tr class="for-actions"><td class="is-inline" colspan="100"><small>Manage options</small><button class="details_table-link" data-modal="/creator-beta/manage">Manage plan</button></td></tr></tr></tbody></table></div><div class="card for-table" data-identifier="cancelled_subscriptions" data-view="app#embed_pagination" id="subscriptions_cancelled"><h2 class="card-title">Cancelled subscriptions</h2><table class="details_table for-unsubscribed_users"><thead><tr><th class="for-name">Profile</th><th class="for-date">Unsubscribed</th><th class="for-earnings">Price</th><th class="for-actions">Actions</th></tr></thead><tbody><tr><td class="for-name"><a href="/creator-gamma"><div class="inline_user_name"><img data-view="app#avatar" data-type="avatar" data-user-id="3003003" alt="Creator Gamma" src="https://cdn.example.invalid/avatars/3003003.jpg" />Creator Gamma</div> </a></td><td class="for-date">Dec 2025</td><td class="for-earnings">$3</td><td class="for-actions"><button class="details_table-link" data-modal="/creator-gamma/resubscribe">Renew</button></td></tr><tr><td class="for-name"><a href="/creator-delta"><div class="inline_user_name"><img data-view="app#avatar" data-type="avatar" data-user-id="4004" alt="Creator Delta" src="https://cdn.example.invalid/avatars/4004.jpg" />Creator Delta</div> </a></td><td class="for-date">Aug 2025</td><td class="for-earnings">$10</td><td class="for-actions"><button class="details_table-link" data-modal="/creator-delta/resubscribe">Renew</button></td></tr></tbody></table></div></div></div></body></html>
+20 -3
View File
@@ -74,8 +74,22 @@ def test_an_unrecognised_status_says_nothing(status):
def test_a_platform_that_has_never_been_characterised_says_nothing(): def test_a_platform_that_has_never_been_characterised_says_nothing():
"""SubscribeStar and FANBOX (D1) inherit silence, not a Patreon guess.""" """An uncharacterised platform inherits silence, not a Patreon guess.
assert gated_reason("subscribestar", "active_patron") is None
This named SubscribeStar until D1 characterised it (note #3989). Left as it
was, it would have kept passing only because `active_patron` is not a
SubscribeStar word, testing nothing its name claims.
"""
assert gated_reason("hentaifoundry", "active_patron") is None
def test_subscribestar_explains_the_gate_in_its_own_words():
"""SubscribeStar's status is which table the creator sits in, and those
identifiers reach the same three reasons as Patreon's words."""
assert gated_reason("subscribestar", "active_subscriptions") == GATED_TIER
assert gated_reason("subscribestar", "cancelled_subscriptions") == GATED_LAPSED
# Patreon's vocabulary does not leak across platforms.
assert gated_reason("subscribestar", "former_patron") is None
# --- the join, against the database ---------------------------------------- # --- the join, against the database ----------------------------------------
@@ -178,8 +192,11 @@ async def test_one_platforms_fresh_sweep_does_not_vouch_for_anothers(db):
url="https://subscribestar.adult/maewix", url="https://subscribestar.adult/maewix",
) )
await _membership(db, status="former_patron") await _membership(db, status="former_patron")
# SubscribeStar's REAL word, so that the freshness gate is the only thing
# excluding this membership. With a Patreon word (as this test first had),
# the vocabulary excluded it and removing the gate would have left it green.
await _membership(db, platform="subscribestar", campaign="s1", await _membership(db, platform="subscribestar", campaign="s1",
status="former_patron", status="cancelled_subscriptions",
url="https://subscribestar.adult/maewix", url="https://subscribestar.adult/maewix",
details={"campaign": {"vanity": "maewix"}}) details={"campaign": {"vanity": "maewix"}})
await _synced(db) # patreon only await _synced(db) # patreon only
+4
View File
@@ -170,8 +170,12 @@ def test_the_status_map_contains_only_characterised_values():
contain. Adding it because it "obviously" belongs is precisely the guess contain. Adding it because it "obviously" belongs is precisely the guess
this test exists to stop. this test exists to stop.
""" """
# subscribestar added with its own capture (note #3989, 2026-09-13). Its
# "words" are the two table identifiers, because the page carries no
# per-row status at all.
assert MEMBERSHIP_STATUS == { assert MEMBERSHIP_STATUS == {
"patreon": {"active_patron": True, "former_patron": False}, "patreon": {"active_patron": True, "former_patron": False},
"subscribestar": {"active_subscriptions": True, "cancelled_subscriptions": False},
} }
+261
View File
@@ -0,0 +1,261 @@
"""SubscribeStarClient.iter_memberships — parsing only, no network (#387 D1).
The fixture mirrors a REAL capture of the operator's /subscriptions page
(Scribe note #3989) with every value invented. Its structure was checked
against the capture tag for tag before this file was written: the card
wrappers, both table heads, and every distinct row shape — including the two
forms the Discord cell takes, and the narrow-screen `<tr class="for-actions">`
that active rows nest INSIDE their own row.
Most of what follows pins refusals. SubscribeStar gives no completeness signal
the way Patreon's `meta.pagination.total` does, and every conclusion drawn from
this roster is drawn from absence, so the parser's job is to raise whenever it
cannot vouch for the whole list.
`_get` is stubbed rather than mocked at the socket: these tests are about what
the client does with a page, and the HTTP path is covered by
test_subscribestar_client.py.
"""
from pathlib import Path
from types import SimpleNamespace
import pytest
from backend.app.services.membership_roster import has_paid_access, roster_user_id
from backend.app.services.native_ingest_common import Membership
from backend.app.services.subscribestar_client import (
SubscribeStarAuthError,
SubscribeStarClient,
SubscribeStarDriftError,
parse_subscriptions_page,
)
_FIXTURE = Path(__file__).parent / "fixtures" / "subscribestar_subscriptions_page1.html"
@pytest.fixture
def page():
return _FIXTURE.read_text()
@pytest.fixture
def client():
return SubscribeStarClient(cookies_path=None)
def _serve(client, html):
calls = []
def fake(url, *, headers=None):
calls.append(url)
return SimpleNamespace(text=html)
client._get = fake
return calls
def _by_id(memberships):
return {m.campaign_id: m for m in memberships}
# --- the request -----------------------------------------------------------
def test_the_roster_is_fetched_from_adult_not_art(client, page):
"""The capture came from `.art`, but FC's requests never clear the `.art`
age wall with the 18+ cookie (#1259, #1284). Copying the browser's host is
the obvious move and would turn every sweep into an auth error."""
calls = _serve(client, page)
list(client.iter_memberships())
assert calls == ["https://subscribestar.adult/subscriptions"]
def test_one_request_for_the_whole_roster(client, page):
calls = _serve(client, page)
list(client.iter_memberships(user_id="ignored"))
assert len(calls) == 1
# --- what a membership contains --------------------------------------------
def test_both_tables_are_read(page):
rows = parse_subscriptions_page(page)
assert all(isinstance(m, Membership) for m in rows)
assert len(rows) == 4
statuses = sorted(m.status for m in rows)
assert statuses == [
"active_subscriptions", "active_subscriptions",
"cancelled_subscriptions", "cancelled_subscriptions",
]
def test_the_creator_id_is_the_key_not_the_slug(page):
"""A slug re-keys when a creator renames; the old row then stops appearing,
and reconciliation reads a disappearance as a lapse. The numeric id does not
change, so a rename can never surface as "you cancelled this creator"
(note #3989, CORRECTION 2)."""
rows = _by_id(parse_subscriptions_page(page))
assert set(rows) == {"1001", "2002002", "3003003", "4004"}
alpha = rows["1001"]
assert alpha.vanity == "creator-alpha"
# The handle is carried as a URL so `match_kind`'s URL-tail fallback reaches
# it — which is what a SubscribeStar Source.url ends in.
assert alpha.url == "https://subscribestar.adult/creator-alpha"
assert alpha.display_name == "Creator Alpha"
def test_the_table_is_the_status(page):
"""No per-row status word exists. The card's `data-identifier` is stored
verbatim — and is exactly what MEMBERSHIP_STATUS maps, so a renamed
identifier fails here rather than silently reading as unknown."""
rows = _by_id(parse_subscriptions_page(page))
assert rows["1001"].status == "active_subscriptions"
assert rows["3003003"].status == "cancelled_subscriptions"
assert has_paid_access("subscribestar", rows["1001"].status) is True
assert has_paid_access("subscribestar", rows["3003003"].status) is False
def test_price_is_kept_as_text_and_never_parsed_into_money(page):
"""A bare `$` names no currency and a page price is not proven to be the
charge. None keeps "unknown" distinct from zero (#3970 finding 4)."""
rows = _by_id(parse_subscriptions_page(page))
alpha = rows["1001"]
assert alpha.amount_cents is None
assert alpha.currency is None
assert alpha.details["columns"]["price"] == "$5"
assert alpha.tier_names == []
assert alpha.is_free_member is False
def test_columns_are_paired_by_position_not_by_class(page):
"""Two active columns share the `for-date` class, and the updates column's
<td> does not carry its <th>'s class — pairing by class would merge or drop
them. Controls and identity stay out of details."""
cols = _by_id(parse_subscriptions_page(page))["1001"].details["columns"]
assert cols == {
"subscribed": "Jan 2026",
"renewed": "Mar 2026",
"paused": "-",
"price": "$5",
"discord": "Joined",
}
cancelled = _by_id(parse_subscriptions_page(page))["3003003"].details["columns"]
assert cancelled == {"unsubscribed": "Dec 2025", "price": "$3"}
def test_both_discord_cell_forms_read_as_text(page):
"""The real page renders a linked Discord as a button with an icon, and an
unlinked one as the ENTITY `&mdash;`. Undecoded, the second would store the
literal string "&mdash;"."""
rows = _by_id(parse_subscriptions_page(page))
assert rows["1001"].details["columns"]["discord"] == "Joined"
assert rows["2002002"].details["columns"]["discord"] == ""
def test_the_nested_actions_row_is_not_read_as_columns(page):
"""Active rows nest a narrow-screen `<tr class="for-actions">` inside their
own row. Its cells would otherwise zip onto the header as extra columns."""
cols = _by_id(parse_subscriptions_page(page))["2002002"].details["columns"]
assert "Manage options" not in cols.values()
assert set(cols) == {"subscribed", "renewed", "paused", "price", "discord"}
# --- refusals --------------------------------------------------------------
def test_a_creator_in_both_tables_is_reported_once_as_active(page):
"""A current subscription is the fact that matters. Written the other way
round, the cancelled row would overwrite the active one in the upsert and
tell a paying subscriber they had lapsed."""
duplicated = page.replace('data-user-id="3003003"', 'data-user-id="1001"')
rows = parse_subscriptions_page(duplicated)
matches = [m for m in rows if m.campaign_id == "1001"]
assert len(matches) == 1
assert matches[0].status == "active_subscriptions"
def test_an_account_that_never_cancelled_has_no_cancelled_card(page):
"""Not drift — such an account plausibly renders no cancelled table."""
start = page.index('<div class="card for-table" data-identifier="cancelled_subscriptions"')
end = page.index("</table></div>", start) + len("</table></div>")
rows = parse_subscriptions_page(page[:start] + page[end:])
assert {m.status for m in rows} == {"active_subscriptions"}
def test_a_login_wall_is_auth_not_drift():
"""The fix for this is a fresh credential, not a scraper change — and the
distinction survives into membership_sync.last_error_type, where it is what
makes the UI's advice correct."""
wall = '<html><body><form action="/session/new" data-role="sign_in"></form></body></html>'
with pytest.raises(SubscribeStarAuthError):
parse_subscriptions_page(wall)
def test_an_unrecognised_page_is_drift():
with pytest.raises(SubscribeStarDriftError):
parse_subscriptions_page("<html><title>Something else</title><body></body></html>")
def test_a_row_without_a_creator_id_refuses_the_whole_roster(page):
"""Identity is the point (#3970 §4). Skipping the row instead would return a
roster one creator short, which downstream reads as a cancellation."""
with pytest.raises(SubscribeStarDriftError, match="data-user-id"):
parse_subscriptions_page(page.replace(' data-user-id="2002002"', ""))
def test_a_non_numeric_creator_id_is_drift(page):
with pytest.raises(SubscribeStarDriftError, match="data-user-id"):
parse_subscriptions_page(page.replace('data-user-id="2002002"', 'data-user-id="beta"'))
def test_a_row_without_a_creator_link_is_drift(page):
with pytest.raises(SubscribeStarDriftError, match="creator link"):
parse_subscriptions_page(page.replace('<a href="/creator-gamma">', "<a>"))
@pytest.mark.parametrize("card", ["active_subscriptions", "cancelled_subscriptions"])
def test_content_after_a_table_is_treated_as_possible_pagination(page, card):
"""Both cards are paginatable (`app#embed_pagination`), and the capture was
too short to show what pagination looks like. Anything rendered after a
table is therefore a roster FC cannot prove complete — raised, never
returned short."""
start = page.index(f'data-identifier="{card}"')
close = page.index("</table>", start) + len("</table>")
paginated = page[:close] + '<nav class="pagination"><a>2</a></nav>' + page[close:]
with pytest.raises(SubscribeStarDriftError, match="cannot be proven complete"):
parse_subscriptions_page(paginated)
def test_a_page_link_anywhere_in_the_roster_is_drift(page):
"""The same concern for a paginator rendered outside the card."""
paged = page.replace("</body>", '<a href="/subscriptions?page=2">Next</a></body>')
with pytest.raises(SubscribeStarDriftError, match="page="):
parse_subscriptions_page(paged)
def test_a_drift_error_mid_page_yields_nothing(client, page):
"""The whole page is parsed before anything is yielded, so a caller can never
be left holding the rows that came before the bad one."""
_serve(client, page.replace(' data-user-id="4004"', ""))
got = []
with pytest.raises(SubscribeStarDriftError):
for m in client.iter_memberships():
got.append(m)
assert got == []
# --- the sweep's user-id probe ---------------------------------------------
def test_a_client_without_current_user_id_needs_no_id():
"""The seam leak D1 found: the sweep called `current_user_id()` on every
client, which exists only on Patreon's. Note #3970 promised a second platform
would be one `builders` line and the client method; without this probe it
would have raised AttributeError on the first sweep."""
assert roster_user_id(SubscribeStarClient(cookies_path=None)) is None
def test_a_client_with_current_user_id_is_asked_for_it():
assert roster_user_id(SimpleNamespace(current_user_id=lambda: "248453")) == "248453"