fix: a post record is marked seen only after it is upserted, and an hourly sweep dates the posts killed walks left undated (#4436)
CI and images / lint (push) Successful in 3s
CI and images / extension-version (push) Successful in 3s
CI and images / extension-test (push) Successful in 17s
CI and images / frontend-build (push) Successful in 27s
CI and images / backend-lint-and-test (push) Successful in 33s
CI and images / integration (push) Failing after 2m25s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / build-agent (push) Skipped

ingest_core marked a post's record key seen when it wrote _post.json, but
the record reaches the database (and dates the post and its images) only
in phase 3. A walk killed before phase 3 left posts the ledger called
recorded and the database never dated; ticks early-out long before
reaching them again. Keys now join the media in mark_seen_after_import.

date_posts_from_records finds each undated native post's record under
its artist's folder and upserts it with the post's own source. Hourly on
maintenance_long; an empty query once everything is dated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
2026-09-25 11:36:17 -04:00
co-authored by Claude Opus 5.5
parent ca802bd885
commit 621c2b8315
7 changed files with 182 additions and 4 deletions
+9 -2
View File
@@ -289,6 +289,11 @@ class Ingester:
# Media handed to phase 3 for import. Marked seen by phase 3 once the
# import has run (`mark_seen_after_import`), not here — see there.
fetched: list[tuple[str, str]] = []
# Post-record keys written this walk. Marked with the media, after
# phase 3 has upserted the records — marking them at write time left a
# walk killed before phase 3 with posts the ledger calls recorded that
# the database never dated, and no later tick walks back to them (#4436).
recorded: list[tuple[str, str]] = []
downloaded = 0
errors = 0
quarantined = 0
@@ -342,7 +347,9 @@ class Ingester:
written_paths=written,
post_record_paths=list(post_records),
relink_source_paths=list(relink),
mark_seen_after_import=lambda: self._mark_seen(source_id, fetched),
mark_seen_after_import=lambda: self._mark_seen(
source_id, fetched + recorded,
),
stdout="\n".join(log_lines),
stderr="",
return_code=return_code,
@@ -511,7 +518,7 @@ class Ingester:
posts_with_body += 1
if rec.path is not None:
post_records.append(str(rec.path))
self._mark_seen(source_id, [(pkey, ppid)])
recorded.append((pkey, ppid))
# Per-post handling line in the run stdout (the existing
# "Raw stdout" panel) — the downloader already read the
# post; we only format its outcome here. post_type beside