feat(ingest): per-post body-capture + recapture diagnostics logging
Operator-flagged: a recapture 'caught nothing' for a post and there were no logs explaining why. Three silent spots now log, so a recapture's per-post outcome is diagnosable (retention bounds the volume): - patreon_client.fetch_post_detail_content: the 200-OK-but-null-content branch was silent — now logs 'fetched N chars' on success AND 'empty/null content (tier-gated or no text)' on the empty case (the most common silent miss). - patreon_downloader.write_post_record: logs each post's FINAL body outcome (captured N chars / NO body) read off the memoized attrs after detail-fetch. - ingest_core summary: appends post-record + relinked counts to the run summary (surfaces on the event stdout the operator already reads). - download_service phase3: logs how many on-disk images got source_filehash relinked (N/total) per recapture. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -574,8 +574,20 @@ class PatreonDownloader:
|
||||
leading underscore keeps it from colliding with a media sidecar
|
||||
(`<NN>_<stem>.json`) and from being resolved as some media file's sidecar
|
||||
by find_sidecar. Returns the path, or None when the post has no id."""
|
||||
if not str(post.get("id") or ""):
|
||||
pid = str(post.get("id") or "")
|
||||
if not pid:
|
||||
return None
|
||||
post_dir = self.images_root / artist_slug / "patreon" / _post_dir_name(post)
|
||||
post_dir.mkdir(parents=True, exist_ok=True)
|
||||
return self._write_sidecar_data(post, post_dir / "_post.json")
|
||||
path = self._write_sidecar_data(post, post_dir / "_post.json")
|
||||
# Per-post body-capture diagnostic: _write_sidecar_data has, by now,
|
||||
# memoized any detail-fetched body onto post["attributes"]["content"], so
|
||||
# this reflects the FINAL body (feed or detail). Logs the success (N chars)
|
||||
# AND the skip (empty) so a recapture's per-post outcome is visible.
|
||||
body = (post.get("attributes") or {}).get("content")
|
||||
n = len(body) if isinstance(body, str) else 0
|
||||
if n:
|
||||
log.info("post-record: post %s body captured (%d chars)", pid, n)
|
||||
else:
|
||||
log.info("post-record: post %s has NO body (feed + detail both empty)", pid)
|
||||
return path
|
||||
|
||||
Reference in New Issue
Block a user