fix: a download is marked seen only after it is imported
CI and images / lint (push) Successful in 3s
CI and images / extension-version (push) Successful in 3s
CI and images / frontend-build (push) Successful in 23s
CI and images / backend-lint-and-test (push) Successful in 31s
CI and images / integration (push) Failing after 2m17s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / build-agent (push) Skipped
CI and images / lint (push) Successful in 3s
CI and images / extension-version (push) Successful in 3s
CI and images / frontend-build (push) Successful in 23s
CI and images / backend-lint-and-test (push) Successful in 31s
CI and images / integration (push) Failing after 2m17s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / build-agent (push) Skipped
A run killed between download and import left its files on disk and in the seen-ledger but never in the library, and every later walk trusted the ledger. TamadaHeijun's 12PCG post lost 7 of 13 images this way (stranded run 90402), which read as the duplicates filter. The ingester now hands phase 3 a mark_seen_after_import hook, called after the import loop. A file on disk with no ImageRecord at its path is fed to import, not reconciled into the ledger. Recapture reaches files already orphaned, because it looks past the ledger to the disk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
@@ -9,9 +9,9 @@ from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from backend.app.models import Artist, DownloadEvent, ImportSettings, Source
|
||||
from backend.app.models import Artist, DownloadEvent, ImageRecord, ImportSettings, Source
|
||||
from backend.app.services.credential_crypto import CredentialCrypto
|
||||
from backend.app.services.credential_service import CredentialService
|
||||
from backend.app.services.thumbnailer import Thumbnailer
|
||||
@@ -152,6 +152,14 @@ async def test_download_source_attaches_written_files(
|
||||
files_downloaded=2,
|
||||
stdout=f"{f1}\n{f2}\n",
|
||||
)
|
||||
# The ledger is marked only once the files are in: a run killed before
|
||||
# import must leave them unmarked so the next walk imports them.
|
||||
imported_when_marked = []
|
||||
result.mark_seen_after_import = lambda: imported_when_marked.append(
|
||||
db_sync.execute(
|
||||
select(func.count(ImageRecord.id)).where(ImageRecord.path.in_([str(f1), str(f2)]))
|
||||
).scalar_one()
|
||||
)
|
||||
fake_gdl = _fake_gdl_with_result(result)
|
||||
|
||||
sync_settings = db_sync.execute(
|
||||
@@ -182,6 +190,7 @@ async def test_download_source_attaches_written_files(
|
||||
assert ev.files_count == 2
|
||||
assert ev.metadata_["import_summary"]["attached"] == 2
|
||||
assert ev.metadata_["run_stats"]["downloaded_count"] == 2
|
||||
assert imported_when_marked == [2]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
|
||||
Reference in New Issue
Block a user