The dedup branches of _import_media linked the existing image to the new post via
_apply_sidecar(artist=None), relying on the SIDECAR to carry the artist. But an
archive member's artist comes from its path, and under post-first the per-media
sidecar is minimal (no artist) — so a re-packed / cross-posted archive image
deduped and was left UNLINKED from the new post, i.e. the post showed "no images".
Resolve the path-anchored artist (derive_top_level_artist) up-front in
_import_media and pass it to both enrich-on-duplicate branches (sha256 + phash
larger_exists) and the new-record path. Drop the now-dead _attach_artist helper
(its logic is inlined at the single new-record call site).
Surfaced by the new test_archive_all_deduped_is_benign_not_flagged (was asserting
2==4: the second post got no provenance links).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause (operator-confirmed via event metadata + lsar): a "High Resolution
files" pack often wraps a per-chapter .rar/.zip INSIDE one outer archive (incase).
_import_archive only extracted one level — a nested-archive member failed
is_supported and was skipped, so the real pages were silently dropped and the post
showed "archive but no images". The disk scan found this pattern recurring across
the attachment store.
- Recurse into nested archives via _collect_archive_members: a member that is
itself an archive is bomb-probed and extracted too, depth-capped at
_ARCHIVE_MAX_DEPTH=3. Nested members attribute to the OUTER archive's sidecar so
they link to the right Post. Each level is wrapped so one bad nested archive
can't abort the import. The shared path means external (mega/gdrive) archives
recurse too.
- Replace the catch-all "held no supported members" string with a per-outcome
tally (media/deduped/unsupported/failed/nested/nested_rejected). The all-deduped
case is now recognised as BENIGN — images already in the library, re-linked to
this post via enrich-on-duplicate — and returns attached WITHOUT error, so it no
longer false-flags in event metadata.unextracted_archives. Genuine failures
carry the precise breakdown.
Tests: nested zip-in-cbz imports both inner images + links them to the outer post;
all-deduped archive returns attached with error=None and links images to both posts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Milestone #67 step 3. Spell out, at the IngestCore.run seam resolution, that
post_record_key + write_post_record are the post-first contract a platform
implements when migrating onto the native core ingester — the post-record owns
the body/links, the per-media sidecar carries image identity only. The import
side is already self-enforcing via uses_native_ingester → importer.post_first.
Durable directive recorded as FC project rule #120.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The per-media sidecar no longer carries title/url/content (post-first, #856) —
update the assertion to expect image identity only (category/id/source_url).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Milestone #67 step 2. On the native core ingester the Post becomes the single
authoritative record for body/links/metadata, captured once per post by the
post-record; the per-media import only links image provenance + localization.
Before: every per-media sidecar carried the full post body, so a post with N
images wrote the body N+1 times (post-record + N media) — redundant on disk and
a divergence risk (#753). gallery-dl is unchanged (its sidecar is still the only
body source).
- patreon_downloader: the per-media sidecar is now minimal — {category, id,
source_url} only, no body. `_write_sidecar_data(minimal=True)` skips the body
resolution + detail-fetch (the post-record, written first in the walk, already
did it). Body no longer duplicated next to each image.
- importer: new per-instance `post_first` flag (Importer is per-task). When set,
`_apply_sidecar` still writes source_filehash + provenance + primary_post_id
but SKIPS `_apply_post_fields` (the post-record owns body/links/raw_metadata,
so applying a body-less sidecar would clobber raw_metadata + re-sync links off
empty data). Default False keeps gallery-dl writing post fields.
- download_service: `_phase3_persist` sets importer.post_first =
uses_native_ingester(platform) — the future-proof seam, so a platform migrating
onto the native core flips to post-first automatically (step 3). Media imports
before post-records but both unify on external_post_id, so the post ends with
its body either way.
Tests: per-media sidecar is minimal + never hits the detail fetcher; attach
post_first=True links provenance/localization but writes no post body/title;
post_first=False (gallery-dl) still applies them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
If Patreon renames/restructures the post body field again (as content →
content_json_string already did), every body silently comes back empty and we'd
archive empty posts without noticing. Surface that as a loud failure.
Research-grounded design (Patreon `content` is officially null|string, body has
no post_type gate, gallery-dl independently added the same content_json_string
fallback): empty bodies are LEGITIMATE for gallery/art posts, so a fraction
threshold would false-positive constantly. The robust, creator-independent break
signature is "a meaningful sample of posts, a body extracted from NONE of them."
- ingest_core counts posts_recorded / posts_with_body on the native post-record
path (gallery-dl never enters it, so the canary is native-only by construction).
- When posts_recorded >= _CANARY_MIN_SAMPLE (30) and posts_with_body == 0, return
ErrorType.API_DRIFT (maps to task_run status "error" — red; its semantics are
literally "fix the field-set/parser, not creds"). Placed after the timeout/stop
returns so it never masks a more specific failure.
- Run summary always appends "bodies X/Y" for sub-threshold observability (a
partial regression that still extracts some bodies shows in the Raw stdout).
Tests: zero bodies over the sample -> API_DRIFT; bodies present -> success;
below the sample floor -> success (tick safety).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bvandeusen