Files
FabledCurator/backend
bvandeusen 00607a309b
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m17s
feat(ingest): post-body schema-drift canary — fail a native walk red when zero bodies extracted (#862)
If Patreon renames/restructures the post body field again (as content →
content_json_string already did), every body silently comes back empty and we'd
archive empty posts without noticing. Surface that as a loud failure.

Research-grounded design (Patreon `content` is officially null|string, body has
no post_type gate, gallery-dl independently added the same content_json_string
fallback): empty bodies are LEGITIMATE for gallery/art posts, so a fraction
threshold would false-positive constantly. The robust, creator-independent break
signature is "a meaningful sample of posts, a body extracted from NONE of them."

- ingest_core counts posts_recorded / posts_with_body on the native post-record
  path (gallery-dl never enters it, so the canary is native-only by construction).
- When posts_recorded >= _CANARY_MIN_SAMPLE (30) and posts_with_body == 0, return
  ErrorType.API_DRIFT (maps to task_run status "error" — red; its semantics are
  literally "fix the field-set/parser, not creds"). Placed after the timeout/stop
  returns so it never masks a more specific failure.
- Run summary always appends "bodies X/Y" for sub-threshold observability (a
  partial regression that still extracts some bodies shows in the Raw stdout).

Tests: zero bodies over the sample -> API_DRIFT; bodies present -> success;
below the sample floor -> success (tick safety).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 08:56:11 -04:00
..