feat(patreon): capture full post body via adaptive detail-fetch
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m24s

The feed endpoint (/api/posts) returns `content` empty for many posts, so post
bodies — their formatting, inline <img>, and external <a href> links — were
never captured (the post showed "(no description)"). Enrich an empty feed body
from the per-post detail endpoint (/api/posts/{id}) before writing the importer
sidecar, memoized by mutating the shared post dict so a multi-image post fetches
detail exactly once and fully-seen posts (no fresh download) pay nothing.
Best-effort by design: a body we can't fetch returns None and never fails the
walk. No-doubling and no-clobber-of-populated-body already hold (post upsert is
keyed on external_post_id; an empty body parses to None and isn't applied).

First slice of milestone #64 (rich post capture + faithful rendering +
external-host downloads). Refs FC #830.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-14 12:29:06 -04:00
parent 7fcef53d5b
commit 2c67c27044
5 changed files with 161 additions and 1 deletions
+43
View File
@@ -562,6 +562,49 @@ class PatreonClient:
return
current_cursor = next_cursor
# -- detail (full body enrichment) -------------------------------------
def fetch_post_detail_content(self, post_id: str) -> str | None:
"""Best-effort fetch of a post's full HTML `content` from the per-post
DETAIL endpoint (`/api/posts/{id}`).
The feed/list endpoint (`/api/posts`) frequently returns `content` as
null even though we request it — the full body (its formatting, inline
`<img>`, and external `<a href>` links) only comes back from the
single-post detail resource. The downloader calls this to enrich a post
whose feed body was empty before writing the importer sidecar.
Best-effort BY DESIGN: the media download is the primary job, so a body
we can't fetch must never fail the walk. Every failure path (no id,
transport error, non-200, non-JSON, missing/empty content) returns None
rather than raising — distinct from the loud drift/auth raises on the
feed path, which gate real downloads.
"""
if not post_id:
return None
if self._request_sleep > 0:
time.sleep(self._request_sleep) # pace the API endpoint (plan #703)
url = f"{_POSTS_URL}/{post_id}"
params = {"fields[post]": "content", "json-api-version": "1.0"}
try:
resp = self._session.get(url, params=params, timeout=_TIMEOUT_SECONDS)
except requests.RequestException as exc:
log.warning("Patreon post-detail fetch failed (post %s): %s", post_id, exc)
return None
if resp.status_code != 200:
log.warning(
"Patreon post-detail fetch HTTP %s (post %s)", resp.status_code, post_id
)
return None
try:
payload = resp.json()
except ValueError:
return None
data = payload.get("data") if isinstance(payload, dict) else None
attrs = data.get("attributes") if isinstance(data, dict) else None
content = attrs.get("content") if isinstance(attrs, dict) else None
return content if isinstance(content, str) and content.strip() else None
# -- verify ------------------------------------------------------------
def verify_auth(self, campaign_id: str) -> tuple[bool | None, str]: