feat: PatreonClient.iter_memberships — the roster seam (milestone 387 step C2)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 5s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 58s
Build images / build-web (push) Successful in 1m16s
Build images / smoke-web (push) Skipped
CI / integration (push) Successful in 2m25s
Build images / build-ml (push) Successful in 2m45s
Build images / promote (push) Skipped

Built on C0's real capture (Scribe note #3886), not on API docs — gallery-dl
has no membership extractor and Patreon's public v2 API is the CREATOR surface
behind OAuth, so the rule-130 reference had to be a characterized response.

**The request is deliberately minimal, and that is a privacy decision.** The
browser's own include set pulls `latest_pledge.card`, and those card resources
come back carrying the ACCOUNT HOLDER'S EMAIL in `merchant_name`; `address` is
in there too. Copying the query string wholesale is the obvious move and would
have FC fetching payment PII it has no use for and can only mishandle. We ask
for `include=campaign,reward` and nothing else, and a test asserts on the
params actually sent so nobody widens it back.

**We do not send `filter[membership_type]`.** The browser sends the six buckets
its settings page displays, which excludes lapsed memberships — and a
DISAPPEARANCE is precisely the signal the roster exists to read. Filtering here
would manufacture the event C4 acts on.

Two corrections the capture forced, both now in code:

* **The filter vocabulary is not the status vocabulary.** I had read the six
  filter words off a screenshot and was about to write them into
  MEMBERSHIP_STATUS as the enum. The body shows `patron_status` carrying
  `former_patron` — absent from that filter — on a row the filter selected as
  `free_member`. So the map is taught exactly the two OBSERVED values, and
  `declined_patron` stays out despite looking obviously right: believing the
  filter is the mistake that was just caught.
* **Free membership is a boolean, not a status.** `has_paid_access` gains an
  `is_free_member` axis, because `active_patron` alone would report a free
  follower as a paying patron and C4 would never offer to clean it up. Honest
  limit, stated in the docstring: the capture has no ACTIVE free member, so it
  shows the separation is possible, not that it occurs.

C1's tripwire test did its job — it was written to fail the moment anyone
populated the status map, and updating it here IS the confirmation step, done
with the capture rather than ahead of it.

`_fetch`'s retry/backoff/auth-vs-drift/Retry-After logic is extracted to a
shared `_request` so the roster rides the same path rather than growing a
second copy — two copies would drift, and the half that drifted would be the
one that only runs daily. Every error message and log line renders
byte-identically for the posts path, so the existing tests pin the refactor.

Pagination is driven by `page[offset]` against `meta.pagination.total`, never
by `links`: the response's own `links.first` is built WITHOUT the `/api/`
prefix the request uses, so following it would hit the web page. An empty page
is terminal regardless of what the total claims, so a server reporting more
rows than it hands over cannot spin the walk forever.

Drift is stricter here than on the posts path, on purpose: a missing
`meta.pagination.total` raises rather than returning a short list, because a
truncated roster reads downstream as "you cancelled those" — the worst wrong
answer this feature can give.

`current_user_id()` is marked INFERRED, not characterized: C0 captured
/api/members, not /api/current_user, so it relies only on the JSON:API envelope
this API demonstrably uses elsewhere, and raises drift rather than returning
something plausible if that is wrong.

The fixture is derived from the real capture with every piece of account data
replaced (the raw capture stays gitignored). Six members, each earning its
place: a former patron with a null pledge, an active patron with no tier, an
annual cadence, a previous_pledge whose included resource has no
`relationships` key at all, and a reward priced in CAD beside a USD charge —
the trap that makes reading `reward.amount_cents` report a number the operator
was never charged. A leak check caught a free-membership-subscription id and
six real campaign launch timestamps before any of it was staged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
This commit is contained in:
2026-09-10 22:26:01 -04:00
co-authored by Claude Opus 5
parent 533a1ce674
commit afcde8e457
5 changed files with 1346 additions and 35 deletions
+42 -10
View File
@@ -38,30 +38,62 @@ log = logging.getLogger(__name__)
# Platform word -> whether the account currently has paid access.
#
# EMPTY ON PURPOSE. Every entry here must come from a characterised response
# (step C0), not from what the API docs or a plausible guess suggest — that is
# the whole point of project rule 130, and inventing `active_patron` before
# seeing it in a real payload is exactly the failure it names. Populate per
# platform as each is characterised.
# Every entry here must come from a CHARACTERISED response, never from API docs
# or a plausible guess — project rule 130, and inventing a status before seeing
# it in a real payload is exactly the failure it names.
#
# patreon: from a live capture of the operator's own session, 2026-09-10
# (Scribe note #3886). Only two values were OBSERVED in `patron_status` and
# only those two are here.
#
# `declined_patron` is deliberately ABSENT even though it looks obviously
# right. It appears in the request's `filter[membership_type]`, and the capture
# proved that filter is NOT the same vocabulary as the attribute — a row
# selected by the filter as `free_member` came back with
# `patron_status: former_patron`, a word the filter does not contain. Reading
# the filter as an enum is the specific mistake the capture caught; adding
# `declined_patron` on the strength of it would be repeating that mistake one
# step later.
#
# Unknown words are NOT an error: an unrecognised status means the roster
# records evidence it cannot yet interpret, which is a better state than
# dropping the row or asserting a meaning for it.
MEMBERSHIP_STATUS: dict[str, dict[str, bool]] = {}
MEMBERSHIP_STATUS: dict[str, dict[str, bool]] = {
"patreon": {
"active_patron": True,
"former_patron": False,
},
}
def has_paid_access(platform: str, status: str | None) -> bool | None:
"""Does this status mean the account currently pays for access?
def has_paid_access(
platform: str, status: str | None, *, is_free_member: bool = False,
) -> bool | None:
"""Does this membership mean the account currently PAYS for access?
Returns None for a status this code has not been taught, which callers must
treat as "unknown" rather than as False. The difference matters: False says
the operator has lost access, and asserting that from an unrecognised word
would tell them to cancel a source they are still paying for.
`is_free_member` is a second axis, not a status, and that is Patreon's
design rather than ours: the capture shows a free follow expressed as a
boolean alongside `patron_status`, so a "current" membership can still be
one nobody is paying for. Taking status alone would report a free follower
as a paying patron, and C4 would then never offer to clean it up.
(Honest limit: the capture contains no ACTIVE free member, so it cannot
demonstrate the two axes coming apart. The separation is what the payload's
shape says; the sample only shows it is possible, not that it happens.)
"""
if status is None:
return None
entry = MEMBERSHIP_STATUS.get(platform, {})
return entry.get(status)
known = MEMBERSHIP_STATUS.get(platform, {}).get(status)
if known is None:
return None
if not known:
return False
return not is_free_member
async def touch_membership(
+257 -19
View File
@@ -14,6 +14,18 @@ the later step can drive it:
- extract_media(post, included_index) → list[MediaItem]
- parse_cursor_from_url(url) → cursor
Milestone 387 added a SECOND read path on the same session: the membership
roster — what the ACCOUNT subscribes to, as opposed to what one creator has
posted.
- iter_memberships(user_id) → Iterator[Membership]
- current_user_id() → str
It is an OPTIONAL seam by construction, probed with
`getattr(client, "iter_memberships", None)` exactly as `post_is_gated` already
is. A client that does not implement it (Discord, HentaiFoundry) makes the
whole feature invisible for that platform — no flag, no config row, no
"unsupported" branch to keep alive.
Drift detection is loud on purpose: Patreon ships JSON:API and the shapes we
depend on (top-level `data`, media resources carrying `file_name`/`url`) are
the contract. If a response comes back as an HTML login page or a media
@@ -52,8 +64,34 @@ from .native_ingest_common import (
log = logging.getLogger(__name__)
_POSTS_URL = "https://www.patreon.com/api/posts"
_MEMBERS_URL = "https://www.patreon.com/api/members"
_CURRENT_USER_URL = "https://www.patreon.com/api/current_user"
_TIMEOUT_SECONDS = 30.0
# --- membership roster contract (#387 C2) ---------------------------------
# Characterized from a real capture of the operator's own session — Scribe note
# #3886. NOT from Patreon's public v2 API, which is the CREATOR api behind
# OAuth scopes and a different surface entirely (project rule 130).
#
# DELIBERATELY MINIMAL, and that is a privacy decision rather than a
# performance one. The web app's own include set pulls `latest_pledge.card`
# and `address`; the card resources come back carrying the ACCOUNT HOLDER'S
# EMAIL in `merchant_name`. Copying the browser's query string wholesale — the
# obvious move — would have FC fetching payment PII it has no use for and can
# only mishandle. We ask for the creator and the tier, and nothing else.
_MEMBERS_INCLUDE = "campaign,reward"
_FIELDS_MEMBER = (
"patron_status,is_free_member,is_gifted,pledge_amount_cents,currency,"
"pledge_cadence,next_charge_date,access_expires_at"
)
_FIELDS_MEMBERS_CAMPAIGN = "name,url,vanity,is_active"
_FIELDS_REWARD = "title"
# The browser sends 1000. Whether a server-side ceiling applies below that is
# untested (note #3886, open question 4), so page conservatively: a wrong guess
# costs one extra request, and the paging loop is driven by meta.pagination
# rather than by this number.
_MEMBERS_PAGE_COUNT = 200
# JSON:API request contract (observed from real traffic — see module plan).
_INCLUDE = (
"campaign,access_rules,attachments,attachments_media,audio,images,media,"
@@ -125,6 +163,44 @@ class MediaItem:
post_id: str
@dataclass
class Membership:
"""One membership the ACCOUNT holds, as the roster needs it (#387 C2).
Deliberately not a raw JSON:API row: the sweep (C3) should not have to know
that a tier lives behind a `reward` relationship, and `platform_membership`
should not gain columns because Patreon shapes things a certain way.
`status` carries the PLATFORM's own word, verbatim and unmapped
(`active_patron`, `former_patron`, ...). Deciding what it means is the read
site's job — `membership_roster.has_paid_access` — precisely so an
unrecognised word records as evidence rather than as a decision.
`is_free_member` is SEPARATE from status and must stay that way. The
capture shows Patreon expressing a free follow as this boolean rather than
as a status value, so "does the account pay for this" is
`status == "active_patron" and not is_free_member` — a question the status
string alone cannot answer. NOTE: the capture contains no ACTIVE free
member, so the two fields are perfectly correlated in that sample; the
separation is what the schema says, not something the sample proves.
"""
campaign_id: str
display_name: str | None
url: str | None
vanity: str | None
status: str | None
is_free_member: bool
tier_names: list[str]
amount_cents: int | None
currency: str | None
# Everything the roster did not model, kept so a later question can be
# answered without another authenticated round-trip. Scoped to the member's
# own attributes plus the campaign's — never the raw page, which is where
# the card/address resources live.
details: dict
def _filehash(url: str) -> str | None:
# Delegate to the shared extractor (utils.paths) so capture-time persistence
# and render-time inline-image matching use the EXACT same identity.
@@ -182,20 +258,27 @@ class PatreonClient:
params["page[cursor]"] = cursor
return params
def _fetch(self, campaign_id: str, cursor: str | None) -> dict:
def _request(self, url: str, params: dict[str, str], *, what: str, scope: str) -> dict:
"""One paced, retried, error-classified GET returning parsed JSON.
Extracted from `_fetch` so the membership endpoint (#387 C2) rides the
SAME request path rather than growing a second copy of the 429 backoff,
the auth-vs-drift classification and the Retry-After plumbing. Two
copies of this would drift, and the half that drifted would be the one
that only runs once a day.
`what` / `scope` only shape the messages ("posts"/"campaign_id=123"),
so a failure still says which call failed and against what.
"""
if self._request_sleep > 0:
time.sleep(self._request_sleep) # pace the API endpoint
attempt = 0
while True:
try:
resp = self._session.get(
_POSTS_URL,
params=self._params(campaign_id, cursor),
timeout=_TIMEOUT_SECONDS,
)
resp = self._session.get(url, params=params, timeout=_TIMEOUT_SECONDS)
except requests.RequestException as exc:
raise PatreonAPIError(
f"Patreon posts request failed (campaign_id={campaign_id}): {exc}"
f"Patreon {what} request failed ({scope}): {exc}"
) from exc
# Transient rate-limit: back off and retry rather than failing the
@@ -205,8 +288,8 @@ class PatreonClient:
attempt += 1
delay = retry_after_seconds(resp, attempt)
log.warning(
"Patreon 429 (campaign_id=%s) — backing off %.1fs (retry %d/%d)",
campaign_id, delay, attempt, self._max_retries,
"Patreon 429 (%s) — backing off %.1fs (retry %d/%d)",
scope, delay, attempt, self._max_retries,
)
time.sleep(delay)
continue
@@ -216,9 +299,8 @@ class PatreonClient:
# Auth rejected — expired/missing cookies or an insufficient tier.
# Actionable as "rotate credentials", so it's auth, not drift/http.
raise PatreonAuthError(
f"Patreon posts API returned HTTP {resp.status_code} — auth "
f"rejected (cookies expired or tier insufficient; "
f"campaign_id={campaign_id})",
f"Patreon {what} API returned HTTP {resp.status_code} — auth "
f"rejected (cookies expired or tier insufficient; {scope})",
status_code=resp.status_code,
)
if resp.status_code != 200:
@@ -234,24 +316,27 @@ class PatreonClient:
except (TypeError, ValueError):
retry_after = None
raise PatreonAPIError(
f"Patreon posts API returned HTTP {resp.status_code} "
f"(campaign_id={campaign_id})",
f"Patreon {what} API returned HTTP {resp.status_code} ({scope})",
status_code=resp.status_code,
retry_after=retry_after,
)
try:
payload = resp.json()
return resp.json()
except ValueError as exc:
# A non-JSON body here is almost always the HTML login/challenge
# page served when cookies are missing/expired — that is an AUTH
# failure (rotate cookies), not API drift (update the ingester) and
# not a transient network error.
raise PatreonAuthError(
"Patreon posts API returned a non-JSON response (likely an "
f"HTML login/challenge page — session expired; "
f"campaign_id={campaign_id}): {exc}"
f"Patreon {what} API returned a non-JSON response (likely an "
f"HTML login/challenge page — session expired; {scope}): {exc}"
) from exc
return payload
def _fetch(self, campaign_id: str, cursor: str | None) -> dict:
return self._request(
_POSTS_URL, self._params(campaign_id, cursor),
what="posts", scope=f"campaign_id={campaign_id}",
)
# -- parsing -----------------------------------------------------------
@@ -510,6 +595,159 @@ class PatreonClient:
return
current_cursor = next_cursor
# -- membership roster (#387 C2) ---------------------------------------
def current_user_id(self) -> str:
"""The signed-in account's own numeric user id.
Needed because `/api/members` is filtered by `filter[user_id]` — the
endpoint answers "who are the members of X", and the account asking
about ITSELF still has to say so.
INFERRED, NOT CHARACTERIZED. C0 captured `/api/members`, not this; what
is relied on here is only the JSON:API envelope (`data.id`), which this
same API demonstrably uses everywhere else. If that inference is wrong
it raises drift rather than returning something plausible — which is
the right failure, because the alternative is a confidently empty
roster and an empty roster means "cancel everything" to C4.
"""
payload = self._request(
_CURRENT_USER_URL, {"json-api-version": "1.0"},
what="current_user", scope="self",
)
data = (payload or {}).get("data")
if not isinstance(data, dict) or not data.get("id"):
raise PatreonDriftError(
"Patreon current_user response had no data.id — cannot scope "
"the membership roster to this account"
)
return str(data["id"])
def _members_params(self, user_id: str | None, offset: int) -> dict[str, str]:
params = {
"include": _MEMBERS_INCLUDE,
"fields[member]": _FIELDS_MEMBER,
"fields[campaign]": _FIELDS_MEMBERS_CAMPAIGN,
"fields[reward]": _FIELDS_REWARD,
"page[offset]": str(offset),
"page[count]": str(_MEMBERS_PAGE_COUNT),
"json-api-version": "1.0",
"json-api-use-default-includes": "false",
}
if user_id:
params["filter[user_id]"] = user_id
# NOTE: `filter[membership_type]` is deliberately NOT sent. The browser
# sends the six values its settings page wants to show, and the capture
# proves that list is NOT the same vocabulary as the `patron_status`
# attribute — a row selected as `free_member` came back with
# `patron_status: former_patron`, a word absent from the filter. Sending
# no filter asks for everything the endpoint will give, which is what a
# roster wants: a membership that DISAPPEARS is the signal C4 reads, and
# a filter tuned for a UI that hides lapses would manufacture exactly
# that disappearance. (Note #3886, open question 1.)
return params
@staticmethod
def _validate_members_response(response: dict) -> None:
"""Drift checks specific to the roster.
Stricter than the posts path about pagination on purpose: `iter_posts`
can treat a missing `links.next` as "that was the last page", but here
a missing total is indistinguishable from a truncated page — and a
roster that silently stops half way reads downstream as "you cancelled
those", which is the worst wrong answer this feature can give.
"""
PatreonClient._validate_response(response)
meta = response.get("meta")
if not isinstance(meta, dict):
raise PatreonDriftError("Patreon members response missing 'meta'")
pagination = meta.get("pagination")
if not isinstance(pagination, dict) or "total" not in pagination:
raise PatreonDriftError(
"Patreon members response missing meta.pagination.total — "
"cannot tell a complete roster from a truncated one"
)
def _membership(self, member: dict, index: dict) -> Membership:
attrs = member.get("attributes") or {}
if "patron_status" not in attrs:
raise PatreonDriftError(
"Patreon member resource has no patron_status attribute"
)
campaign_ids = self._related_ids(member, "campaign")
if not campaign_ids:
raise PatreonDriftError(
"Patreon member resource has no campaign relationship — a "
"membership we cannot attribute to a creator is not usable"
)
campaign_id = campaign_ids[0]
campaign = index.get(("campaign", campaign_id)) or {}
# A member has at most one reward, and `reward.data` is legitimately
# null — an active patron with no tier. Absence is a fact about the
# membership, not a parse failure.
tier_names: list[str] = []
for reward_id in self._related_ids(member, "reward"):
title = (index.get(("reward", reward_id)) or {}).get("title")
if title:
tier_names.append(str(title))
return Membership(
campaign_id=campaign_id,
display_name=campaign.get("name"),
url=campaign.get("url"),
vanity=campaign.get("vanity"),
status=attrs.get("patron_status"),
# Default False, not None: the attribute is always present in the
# capture, and treating a missing one as "free" would understate
# access rather than overstate it.
is_free_member=bool(attrs.get("is_free_member")),
tier_names=tier_names,
# The MEMBER's amount, never the reward's. `reward.amount_cents` is
# the creator's list price in the CREATOR's currency (the capture
# has CAD, DKK and EUR rewards sitting on USD pledges), so reading
# it would report a number the operator has never been charged.
amount_cents=attrs.get("pledge_amount_cents"),
currency=attrs.get("currency"),
details={"member": attrs, "campaign": campaign},
)
def iter_memberships(self, user_id: str | None = None) -> Iterator[Membership]:
"""Yield every membership the account holds.
Pages on `page[offset]`/`page[count]` against `meta.pagination.total` —
NOT on `links`. The response's own `links.first` is built without the
`/api/` prefix the request uses, so following it verbatim would hit the
web page instead of the API (note #3886).
`user_id` omitted means the `filter[user_id]` parameter is omitted.
Whether the endpoint then defaults to self is UNTESTED — pass
`current_user_id()` unless you are deliberately probing that.
"""
user_id = user_id or None
offset = 0
seen = 0
while True:
response = self._request(
_MEMBERS_URL, self._members_params(user_id, offset),
what="members", scope="membership roster",
)
self._validate_members_response(response)
index = self._transform(response)
rows = [m for m in (response.get("data") or []) if isinstance(m, dict)]
for member in rows:
yield self._membership(member, index)
seen += len(rows)
total = int(response["meta"]["pagination"]["total"] or 0)
# An empty page terminates regardless of what `total` claims. Trusting
# the total alone would spin forever against a server that reports
# more rows than it will hand over.
if not rows or seen >= total:
return
offset += len(rows)
# -- detail (full body enrichment) -------------------------------------
def fetch_post_detail_content(self, post_id: str) -> str | None: