Commit Graph

6 Commits

Author SHA1 Message Date
bvandeusen 68cffce322 fix(importer): catch PIL OSError during transparency + phash blocks, skip as invalid_image instead of letting Celery autoretry loop forever
Operator hit a corrupt JPEG in the IR set 2026-05-25: PIL.verify() only
validates header structure but doesn't catch truncated/broken pixel
data. The error surfaces later in _transparency_pct (via getchannel
'A' -> load) or compute_phash (load) — both blow up with OSError
'broken data stream when reading image file'. Celery's autoretry_for
then bounces the same file forever instead of marking it skipped.

Wrap both PIL.load-triggering call sites with try/except OSError ->
ImportResult(status=skipped, skip_reason=invalid_image).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 02:34:36 -04:00
bvandeusen f97551e2f6 feat(attachments): importer dispatch — archive extract + non-media capture
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 11:13:36 -04:00
bvandeusen 80701dabf7 fix(test): update FC-2b/2a tests that pinned pre-vii-c artist behavior
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 21:39:54 -04:00
bvandeusen 531ab6243e fix(fc2a): move pytestmark below imports in test_importer.py (E402)
Misplaced the integration marker between import groups in the previous
commit — I only read the top 12 lines so I missed the 3 more imports
below. ruff flagged it correctly as E402.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 15:34:46 -04:00
bvandeusen 22bc24b6b6 fix(fc2a): align CI with FabledRulebook — lint + short unit tests only
The CI failure resolving 'postgres' hostname was the symptom; the cause is
that the workflow violated FabledRulebook/forgejo.md's "CI philosophy —
lint + short unit tests only" rule. Integration tests against a real
Postgres are supposed to run locally via docker-compose, not in CI.

Changes:
- Marked 8 DB-dependent test files with @pytest.mark.integration:
  test_tag_service, test_importer, test_gallery_service, test_api_gallery,
  test_api_tags, test_api_settings, test_api_import_admin, test_maintenance.
- CI workflow drops the postgres/redis service containers and the alembic
  upgrade smoke step entirely.
- Pytest invocation in CI changes to `pytest -v -m "not integration"`.
- Added pytest marker registration to pyproject.toml.
- DB_PASSWORD and SECRET_KEY env vars retained because config.py reads
  them at import time even though unit tests don't actually use them
  (set to placeholder values).

What CI now runs:
- ruff check
- pytest on the 6 unit test files: test_slug, test_paths,
  test_migration_0002, test_thumbnailer, test_celery_smoke,
  test_tasks_register.
- npm install + npm run build

What CI no longer runs:
- alembic upgrade (no live DB)
- the 8 integration test files (these run locally via docker-compose)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 15:23:07 -04:00
bvandeusen bb9c183ff7 feat(fc2a): add Importer service
Single-file pipeline: validate as image, apply filter rules (min dimensions,
transparency), SHA256 hash dedup, atomic copy to /images/{subdir}/, create
ImageRecord with origin='imported_filesystem' and integrity_status='unknown',
auto-derive top-level folder name as Artist + artist tag.

Importer is intentionally sync (consumed by a Celery worker process); the
async Quart side uses the same ORM through its own async session. The
db_sync fixture in conftest.py was added in Task 3 to support these tests.

Thumbnail generation is NOT inlined; the calling Celery task enqueues a
separate thumbnail task so the import queue keeps moving on big batches.
pHash dedup is FC-2d, not here.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 12:07:01 -04:00