fix: a download is marked seen only after it is imported
CI and images / lint (push) Successful in 3s
CI and images / extension-version (push) Successful in 3s
CI and images / frontend-build (push) Successful in 23s
CI and images / backend-lint-and-test (push) Successful in 31s
CI and images / integration (push) Failing after 2m17s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / build-agent (push) Skipped

A run killed between download and import left its files on disk and in the
seen-ledger but never in the library, and every later walk trusted the
ledger. TamadaHeijun's 12PCG post lost 7 of 13 images this way (stranded
run 90402), which read as the duplicates filter.

The ingester now hands phase 3 a mark_seen_after_import hook, called after
the import loop. A file on disk with no ImageRecord at its path is fed to
import, not reconciled into the ledger. Recapture reaches files already
orphaned, because it looks past the ledger to the disk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
2026-09-24 14:13:12 -04:00
co-authored by Claude Opus 5.5
parent 49d18c1757
commit da2091a875
6 changed files with 141 additions and 6 deletions
+11 -2
View File
@@ -9,9 +9,9 @@ from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock
import pytest
from sqlalchemy import select
from sqlalchemy import func, select
from backend.app.models import Artist, DownloadEvent, ImportSettings, Source
from backend.app.models import Artist, DownloadEvent, ImageRecord, ImportSettings, Source
from backend.app.services.credential_crypto import CredentialCrypto
from backend.app.services.credential_service import CredentialService
from backend.app.services.thumbnailer import Thumbnailer
@@ -152,6 +152,14 @@ async def test_download_source_attaches_written_files(
files_downloaded=2,
stdout=f"{f1}\n{f2}\n",
)
# The ledger is marked only once the files are in: a run killed before
# import must leave them unmarked so the next walk imports them.
imported_when_marked = []
result.mark_seen_after_import = lambda: imported_when_marked.append(
db_sync.execute(
select(func.count(ImageRecord.id)).where(ImageRecord.path.in_([str(f1), str(f2)]))
).scalar_one()
)
fake_gdl = _fake_gdl_with_result(result)
sync_settings = db_sync.execute(
@@ -182,6 +190,7 @@ async def test_download_source_attaches_written_files(
assert ev.files_count == 2
assert ev.metadata_["import_summary"]["attached"] == 2
assert ev.metadata_["run_stats"]["downloaded_count"] == 2
assert imported_when_marked == [2]
@pytest.mark.asyncio