bvandeusen/FabledCurator
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases 35 Wiki Activity
1,094 Commits 3 Branches 35 Tags
306de50f615e7254156dddd64ab5dd2bca021027
Commit Graph
7 Commits
Author SHA1 Message Date
bvandeusen 9cbdb70e13 fix(thumbnails): surface backfill results + tighten validity check
CI / lint (push) Successful in 2s
Details
CI / backend-lint-and-test (push) Successful in 16s
Details
CI / frontend-build (push) Successful in 24s
Details
CI / intimp (push) Successful in 3m48s
Details
CI / intapi (push) Successful in 7m54s
Details
CI / intcore (push) Failing after 8m18s
Details 
Two coupled problems, operator-flagged 2026-06-01: "missing thumbnails
but triggering backfill found nothing."

1. **Backfill UI was a black box.** `POST /api/thumbnails/backfill`
   returned just `{celery_task_id}` and the admin card said
   "Enqueued." with no counts. There was no way to tell whether the
   scan found 0 candidates, 5000 candidates, or whether the worker
   even picked up the task. "Found nothing" was indistinguishable
   from a broken queue.

   Fix: refactor the scan into a sync helper (`_run_backfill_scan`)
   shared by the Celery task and the API endpoint. The API now runs
   the scan in an executor and returns `{scanned, enqueued, ok,
   regenerated}`. The actual thumbnail generation work still goes
   to the thumbnail Celery queue per row via
   `generate_thumbnail.delay()` — the scan itself is fast
   (SELECT id+thumbnail_path + a file.stat() per row).

2. **`_thumb_is_valid` accepted header-only corrupt files.** The
   magic-byte check passed for any 8-byte file starting with a JPEG
   or PNG header, including empty/truncated/zero-pad files that
   browsers render as broken. Backfill counted these as `ok` and
   never regenerated.

   Fix: also require file size ≥ MIN_THUMB_BYTES (256). Real
   thumbnails are minimum ~2KB even on solid-color sources; header-
   only corrupt files top out around 12 bytes. 256 is well above
   the corrupt floor and well below any legitimate thumbnail.

Plus the admin card now shows the per-run counts instead of
"Enqueued.":
  Scanned 5,432 · enqueued 3 (2 regenerated) · 5,429 ok
 2026-06-01 21:57:29 -04:00
bvandeusenandClaude Opus 4.7 a41eddae3f feat(thumb-backfill): backfill_thumbnails planner task — keyset-paginates ImageRecord, NULLs bad thumb paths, enqueues generate_thumbnail for NULL/missing/corrupt 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 10:36:21 -04:00
bvandeusenandClaude Opus 4.7 7aa7f5a3d6 feat(thumb-backfill): _thumb_is_valid helper — JPEG/PNG magic-byte check on the on-disk thumbnail file 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 10:35:06 -04:00
bvandeusenandClaude Opus 4.7 be0f472894 fix(workers): worker-level recovery — autoretry on transient errors + tightened sweep (5min) + import_media_file body-wrap so no path leaves rows stuck in 'processing' 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 11:46:36 -04:00
bvandeusenandClaude Opus 4.7 7d8b9c3d90 fix(tasks): share one sync engine per worker process to stop connection-pool leak 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-23 10:44:16 -04:00
bvandeusenandClaude Opus 4.7 80a5690740 fix(fc2a): apply ruff autofix + skip vue-tsc check (no tsconfig) 
Ruff:
The remaining I001 errors came from ruff treating `alembic` as a first-
party module (because the alembic/ directory exists in the repo root)
rather than third-party. Ran `ruff check --fix` locally — auto-sorted
import groupings to put alembic/sqlalchemy alongside backend.* as first-
party, and trimmed redundant blank lines after a few import blocks.

Frontend:
`npm run check` (vue-tsc --noEmit) was failing because vue-tsc has no
tsconfig.json to read against, and the frontend is pure JS without
JSDoc annotations — vue-tsc had nothing to do. Skipping the step until
we add a tsconfig + convert to TS or add JSDoc annotations.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-14 14:24:25 -04:00
bvandeusenandClaude Opus 4.7 d3bb8c509a feat(fc2a): add Celery tasks — scan_directory, import_media_file, generate_thumbnail 
scan_directory walks ImportSettings.import_scan_path, creates an
ImportBatch, enumerates supported files into ImportTasks, and enqueues
import_media_file per task. import_media_file moves the task through
its state machine (pending → queued → processing → complete/skipped/failed),
updates ImportBatch counters atomically (UPDATE ... SET col = col + 1),
enqueues a thumbnail task on success, and marks the batch complete when
the last task drains.

generate_thumbnail runs on its own queue (thumbnail) so big imports
don't starve thumbnail throughput; failure here is logged and does not
fail the import.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-14 12:08:37 -04:00