Nested-archive extraction (#718) + post-first ingest (#67) + post-body canary (#862) #109

Merged
bvandeusen merged 6 commits from dev into main 2026-06-15 21:37:39 -04:00
Showing only changes of commit 41aa8fe39e - Show all commits
+12 -3
View File
@@ -141,9 +141,18 @@ class Ingester:
# tick has no resumable backfill state.
checkpoint = mode in ("backfill", "recovery", "recapture")
ledger_key = self._ledger_key
# Optional seams (Patreon native ingester): capture pure-text posts that
# have NO downloadable media so the artist archive is complete. Absent on
# stub clients/downloaders (unit tests) → media-less posts skipped as before.
# POST-FIRST CONTRACT (milestone #67): these two optional seams make a
# platform "post-first" on the native core ingester — the post-record is
# the single authoritative writer of the post body/links/metadata, and the
# per-media sidecar carries image identity only (download_service flips
# importer.post_first via uses_native_ingester, so the import side follows
# automatically). A platform migrating off gallery-dl onto the native core
# adopts post-first by implementing BOTH:
# client.post_record_key(post) -> (ledger_key, post_id) | None (gate)
# downloader.write_post_record(post, artist_slug) -> PostRecordOutcome
# Absent on stub clients/downloaders (unit tests) and on not-yet-migrated
# platforms → media-less posts are skipped as before and the body still
# comes from the per-media sidecar (gallery-dl path). See [[post-first-ingest-contract]].
post_record_key = getattr(self.client, "post_record_key", None)
write_post_record = getattr(self.downloader, "write_post_record", None)
start = time.monotonic()