feat(patreon): capture media-less/text-only posts (post-only records)
Today the ingest core does `if not media: continue`, so a post with no downloadable media (a pure-text post — which often holds the ONLY copy of an external mega/gdrive/pixeldrain link) never upserts a Post. Now the native ingester emits a post-only sidecar (`_post.json`) for every media-less post, gated through the seen-ledger via a synthetic `post:<id>` key so the body is detail-fetched + recorded ONCE (not re-walked every tick); recovery bypasses the gate. Phase 3 imports these via Importer.upsert_post_record, keyed on external_post_id so it UPDATES the same Post a media import would create — never doubles, never clobbers a populated body with an empty one. - gallery_dl.py: DownloadResult.post_record_paths (default []; gallery-dl path unaffected — all constructions are keyword). - ingest_core.py: media-less branch (optional client/downloader seams via getattr; stub clients in tests skip it as before). - patreon_client.py: post_record_key(post). patreon_downloader.py: write_post_record + _write_sidecar_data refactor (shared serializer). - importer.py: upsert_post_record. download_service.py: phase-3 import loop. - tests: client/downloader/ingester (gate + recovery)/importer (no-double). Slice 0b of milestone #64. Refs FC #830. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -147,6 +147,12 @@ class DownloadResult:
|
||||
files_quarantined: int = 0
|
||||
quarantined_paths: list[str] = field(default_factory=list)
|
||||
written_paths: list[str] = field(default_factory=list)
|
||||
# Native ingester only: post-only sidecar paths for posts that have NO
|
||||
# downloadable media (pure-text posts), so the importer can still upsert the
|
||||
# Post + its body. Empty on the gallery-dl path. Phase 3 imports these via
|
||||
# Importer.upsert_post_record (keyed on external_post_id → updates, never
|
||||
# doubles, the same Post a media import would create).
|
||||
post_record_paths: list[str] = field(default_factory=list)
|
||||
stdout: str = ""
|
||||
stderr: str = ""
|
||||
return_code: int = 0
|
||||
|
||||
Reference in New Issue
Block a user