refactor(ingest): per-post handling into run stdout via a downloader outcome (#842)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m18s

Two corrections from operator review:
1. Reuse the existing 'Raw stdout' panel instead of a bespoke structured UI
   section — the native ingester now writes a per-post line into the run stdout
   (parity with gallery-dl's per-file stdout), so the per-post handling shows in
   the panel the operator already uses.
2. DRY: stop re-reading post['attributes'] inline in ingest_core. write_post_record
   now returns a PostRecordOutcome (path, post_type, title, body_chars) — mirroring
   the download_post -> MediaOutcome contract — and the downloader owns the read;
   ingest_core only formats the outcome into the log line.

Reverts the post_diagnostics metadata field + DownloadDetailModal 'Post capture'
section added earlier. Per-post line: 'post <id> [<post_type>] body: N chars' (+
' — EMPTY' when 0), so an empty body is self-explanatory by post_type.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-14 23:27:38 -04:00
parent bcc7266021
commit eb811e11f6
8 changed files with 71 additions and 162 deletions
+13 -9
View File
@@ -24,7 +24,7 @@ from backend.app.services.patreon_client import (
PatreonAuthError,
PatreonDriftError,
)
from backend.app.services.patreon_downloader import MediaOutcome
from backend.app.services.patreon_downloader import MediaOutcome, PostRecordOutcome
from backend.app.services.patreon_ingester import PatreonIngester, _ledger_key
pytestmark = pytest.mark.integration
@@ -137,7 +137,14 @@ class _FakeDownloader:
self.post_records += 1
p = self.tmp_path / f"{post.get('id')}__post.json"
p.write_text("{}")
return p
attrs = post.get("attributes") or {}
body = attrs.get("content")
return PostRecordOutcome(
path=p,
post_type=attrs.get("post_type"),
title=attrs.get("title"),
body_chars=len(body) if isinstance(body, str) else 0,
)
@pytest.fixture
@@ -654,13 +661,10 @@ async def test_backfill_skips_already_captured_post_but_recapture_forces_it(
# Diagnostic summary surfaces the post-record + relink counts (operator reads
# this off the event stdout to see what a recapture actually did).
assert "1 post-record(s), 1 relinked" in res_recap.stdout
# #842: per-post body diagnostic carries post_type + body length so the UI can
# show how each post was handled (and explain empties by post_type).
assert len(res_recap.post_diagnostics) == 1
diag = res_recap.post_diagnostics[0]
assert diag["post_id"] == "p1"
assert diag["post_type"] == "image_file"
assert diag["body_chars"] == len("<p>body p1</p>")
# #842: per-post handling is written into the SAME run stdout (reuses the
# existing "Raw stdout" panel) — post_type + body length, so an empty body is
# self-explanatory by its post_type.
assert f"post p1 [image_file] body: {len('<p>body p1</p>')} chars" in res_recap.stdout
@pytest.mark.asyncio