refactor(ingest): per-post handling into run stdout via a downloader outcome (#842)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m18s

Two corrections from operator review:
1. Reuse the existing 'Raw stdout' panel instead of a bespoke structured UI
   section — the native ingester now writes a per-post line into the run stdout
   (parity with gallery-dl's per-file stdout), so the per-post handling shows in
   the panel the operator already uses.
2. DRY: stop re-reading post['attributes'] inline in ingest_core. write_post_record
   now returns a PostRecordOutcome (path, post_type, title, body_chars) — mirroring
   the download_post -> MediaOutcome contract — and the downloader owns the read;
   ingest_core only formats the outcome into the log line.

Reverts the post_diagnostics metadata field + DownloadDetailModal 'Post capture'
section added earlier. Per-post line: 'post <id> [<post_type>] body: N chars' (+
' — EMPTY' when 0), so an empty body is self-explanatory by post_type.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-14 23:27:38 -04:00
parent bcc7266021
commit eb811e11f6
8 changed files with 71 additions and 162 deletions
+11 -5
View File
@@ -18,6 +18,7 @@ from backend.app.services.patreon_client import MediaItem
from backend.app.services.patreon_downloader import (
MediaOutcome,
PatreonDownloader,
PostRecordOutcome,
_sanitize,
)
from backend.app.utils.sidecar import find_sidecar
@@ -646,14 +647,17 @@ def test_write_post_record_writes_enriched_post_only_sidecar(tmp_path):
)
post = _post()
post["attributes"]["content"] = "" # media-less post, empty feed body
sc = dl.write_post_record(post, "artist-x")
rec = dl.write_post_record(post, "artist-x")
assert sc is not None
assert sc.name == "_post.json" # can't collide with a media `NN_*.json`
data = json.loads(sc.read_text())
assert isinstance(rec, PostRecordOutcome)
assert rec.path is not None
assert rec.path.name == "_post.json" # can't collide with a media `NN_*.json`
data = json.loads(rec.path.read_text())
assert data["id"] == "1001"
assert data["content"] == "<p>text post body</p>"
assert calls == ["1001"]
# Outcome reports the captured body's shape (detail-fetched here).
assert rec.body_chars == len("<p>text post body</p>")
def test_write_post_record_none_without_post_id(tmp_path):
@@ -661,4 +665,6 @@ def test_write_post_record_none_without_post_id(tmp_path):
images_root=tmp_path, cookies_path=None, validate=False,
session=_FakeSession(),
)
assert dl.write_post_record({"id": "", "attributes": {}}, "artist-x") is None
rec = dl.write_post_record({"id": "", "attributes": {}}, "artist-x")
assert rec.path is None
assert rec.body_chars == 0