Commit Graph

2 Commits

Author SHA1 Message Date
bvandeusen e77afe8295 feat(import-resilience L1): poison-pill circuit breaker — cap stuck-task re-queues
Layer 1 of the import-task resilience work (operator-requested
2026-05-28). The recover_interrupted_tasks sweep re-queues rows stuck
in 'processing' — correct for a worker crash, but without a cap a row
that RELIABLY hard-crashes the worker (OOM/segfault/SIGKILL on a
corrupt or oversized input) loops forever: re-queue → crash → re-queue,
burning a worker slot every 5 min. A caught exception flips to terminal
'failed' and never enters this loop; only process-killing inputs do.

- alembic 0026: import_task.recovery_count (int, default 0) +
  import_task.refetched (bool, default false — backs Layer 2).
- recover_interrupted_tasks now runs a poison-pill UPDATE FIRST: stuck
  rows whose recovery_count has already reached MAX_RECOVERY_ATTEMPTS-1
  are marked 'failed' with a diagnostic ("crashed or stalled the worker
  N times … likely a corrupt or oversized input … inspect/replace the
  file, then retry via /api/import/retry-failed") instead of re-queued.
  The re-queue pass then handles the remaining stuck rows and bumps
  recovery_count. Shared stuck_predicate (and_/or_) keeps the
  media-5min / archive-40min split.
- MAX_RECOVERY_ATTEMPTS=3 (two recoveries then give up).

The failed poison pill surfaces in the existing import-failures view
with its file path, directly answering "help me identify them."

Test test_recover_interrupted_poison_pill_caps_at_max pins both
branches: a row at the cap is failed (not re-enqueued, diagnostic
present), a row one short is re-queued + incremented.
2026-05-27 23:54:35 -04:00
bvandeusen 9eb1614fbf feat(fc2a): schema migration 0002 — tag kinds + fandom hierarchy + import lifecycle
Adds Tag.kind enum (artist/character/fandom/general/series/archive/post/meta/rating),
Tag.fandom_id FK with CHECK constraint (only valid for kind='character'), and a
kind-aware uniqueness index so the same name can exist across kinds and the same
character name can exist in different fandoms.

Adds ImportBatch + ImportTask state-machine tables for scan tracking, plus a
single-row ImportSettings table (CHECK id=1) holding the importer's filter knobs.

Adds image_record.integrity_status column (defaults to 'unknown'); FC-2e
populates this via the integrity verifier.

Drops the unused tag.namespace column from FC-1 — superseded by kind.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 12:03:41 -04:00