Commit Graph

5 Commits

Author SHA1 Message Date
bvandeusen bb48845268 fix(fc2b): add ML beat schedule entries (missed in Task 9 commit)
The celery_app.py beat-schedule edit failed with a stale-read error in
the Task 9 commit, so the ML daily jobs weren't registered. Adds
ml-backfill-daily, recompute-centroids-daily, apply-allowlist-sweep-daily.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:44:12 -04:00
bvandeusen ac7e0d13bc feat(fc2b): add tag_and_embed + backfill Celery tasks
tag_and_embed: Camie + SigLIP on one image (video → 10-frame sample,
max-pool tags, mean-pool embeddings), stores predictions/embedding with
model versions, then enqueues per-image allowlist apply. backfill:
keyset-paginated discovery of images missing predictions/embeddings for
the current model versions (restart-safe). apply_allowlist_tags stub
included so .delay() resolves between commits (filled in Task 9).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:40:27 -04:00
bvandeusen 509c19ce86 feat(fc2a): add maintenance tasks (recovery + cleanup) with beat schedule
recover_interrupted_tasks runs every 5 minutes, finds ImportTask rows
stuck in 'processing' for >30 minutes (well above any legitimate import
duration), and re-queues them. cleanup_old_tasks runs daily and deletes
finished tasks older than 7 days so the task table stays an operational
view rather than an archive.

Both thresholds match ImageRepo's precedent. The 30-min stuck threshold
is documented inline so a future reader can adjust it intentionally
rather than mistaking it for a 'magic number'.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 12:09:09 -04:00
bvandeusen d3bb8c509a feat(fc2a): add Celery tasks — scan_directory, import_media_file, generate_thumbnail
scan_directory walks ImportSettings.import_scan_path, creates an
ImportBatch, enumerates supported files into ImportTasks, and enqueues
import_media_file per task. import_media_file moves the task through
its state machine (pending → queued → processing → complete/skipped/failed),
updates ImportBatch counters atomically (UPDATE ... SET col = col + 1),
enqueues a thumbnail task on success, and marks the batch complete when
the last task drains.

generate_thumbnail runs on its own queue (thumbnail) so big imports
don't starve thumbnail throughput; failure here is logged and does not
fail the import.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 12:08:37 -04:00
bvandeusen 8773f2aae6 feat: configure Celery with per-feature queue lanes and a smoke task
Routes are pre-declared for FC-2/FC-3 task modules (import, ml, thumbnail,
download, scan, maintenance). Queue lanes match the ImageRepo pattern where
beat+maintenance run on a separate worker so long imports don't starve
periodic tasks. Smoke ping task confirms the wiring in eager mode for CI.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 07:34:47 -04:00