fix: a post record is marked seen only after it is upserted, and an hourly sweep dates the posts killed walks left undated (#4436)
CI and images / lint (push) Successful in 3s
CI and images / extension-version (push) Successful in 3s
CI and images / extension-test (push) Successful in 17s
CI and images / frontend-build (push) Successful in 27s
CI and images / backend-lint-and-test (push) Successful in 33s
CI and images / integration (push) Failing after 2m25s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / build-agent (push) Skipped

ingest_core marked a post's record key seen when it wrote _post.json, but
the record reaches the database (and dates the post and its images) only
in phase 3. A walk killed before phase 3 left posts the ledger called
recorded and the database never dated; ticks early-out long before
reaching them again. Keys now join the media in mark_seen_after_import.

date_posts_from_records finds each undated native post's record under
its artist's folder and upserts it with the post's own source. Hourly on
maintenance_long; an empty query once everything is dated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
2026-09-25 11:36:17 -04:00
co-authored by Claude Opus 5.5
parent ca802bd885
commit 621c2b8315
7 changed files with 182 additions and 4 deletions
+5 -2
View File
@@ -943,8 +943,10 @@ async def test_a_run_that_dies_before_import_leaves_its_media_unmarked(
_FakeDownloader(tmp_path))
ing.run(source_id=source_id, campaign_id="c1", artist_slug="ingest",
url="https://patreon.com/ingest", mode="tick")
# Phase 3 never ran, so `mark_seen_after_import` never did: only the post key.
assert _count_ledger(sync_engine, source_id) == 1
# Phase 3 never ran, so `mark_seen_after_import` never did: neither the
# media nor the post record is marked (#4436 — the record, marked at write
# time, left the post undated for good once the run died before phase 3).
assert _count_ledger(sync_engine, source_id) == 0
# The next walk finds the file on disk with no record, and imports it.
ing2 = _ingester(sync_engine, tmp_path, _FakeClient([(None, [("p1", [m1])])]),
@@ -954,6 +956,7 @@ async def test_a_run_that_dies_before_import_leaves_its_media_unmarked(
assert result.written_paths == [str(tmp_path / "p1_1.jpg")]
assert result.files_downloaded == 0 # not fetched again
assert "on disk but never imported: p1_1.jpg" in result.stdout
assert len(result.post_record_paths) == 1 # the record is written again
result.mark_seen_after_import()
assert _count_ledger(sync_engine, source_id) == 2