fix(thumbnails): surface backfill counts + tighten validity check #47

Merged

bvandeusen merged 2 commits from dev into main 2026-06-01 22:34:38 -04:00

Conversation 0 Commits 2 Files Changed 6

+153 -67

bvandeusen commented 2026-06-01 22:34:31 -04:00

Owner

Copy Link

Copy Source

Summary

Two coupled fixes for "missing thumbnails but triggering backfill found
nothing" (operator-flagged 2026-06-01):

1. Backfill UI was a black box

POST /api/thumbnails/backfill returned only {celery_task_id} and the
admin card said "Enqueued." with no counts. Impossible to tell whether
the scan found 0 candidates, 5000 candidates, or whether the worker was
even picking up the task. "Found nothing" was indistinguishable from a
broken queue.

• Scan logic factored into _run_backfill_scan (shared helper).
• API runs it synchronously in an executor and returns
  {scanned, enqueued, ok, regenerated}. The actual thumbnail
  generation still goes to the thumbnail Celery queue per row.
• Admin card now shows: Scanned 5,432 · enqueued 3 (2 regenerated) · 5,429 ok.

2. _thumb_is_valid accepted header-only corrupt files

The magic-byte check passed for any 8+ byte file starting with a JPEG or
PNG header, including empty/truncated/zero-pad files that browsers
render as broken. Backfill counted these as ok and never regenerated.

• Also require file size ≥ MIN_THUMB_BYTES (256). Real thumbnails are
  minimum ~2KB even on solid-color sources; header-only corrupt files
  top out around 12 bytes. 256 sits well above corrupt and well below
  any legitimate thumbnail.

Test plan

• CI green on 43b778a
• Operator triggers backfill and reports the surfaced counts; that'll
  tell us which scenario the broken-thumbnail tiles are actually in.

🤖 Generated with Claude Code

## Summary Two coupled fixes for "missing thumbnails but triggering backfill found nothing" (operator-flagged 2026-06-01): ### 1. Backfill UI was a black box `POST /api/thumbnails/backfill` returned only `{celery_task_id}` and the admin card said "Enqueued." with no counts. Impossible to tell whether the scan found 0 candidates, 5000 candidates, or whether the worker was even picking up the task. "Found nothing" was indistinguishable from a broken queue. - Scan logic factored into `_run_backfill_scan` (shared helper). - API runs it synchronously in an executor and returns `{scanned, enqueued, ok, regenerated}`. The actual thumbnail generation still goes to the thumbnail Celery queue per row. - Admin card now shows: `Scanned 5,432 · enqueued 3 (2 regenerated) · 5,429 ok`. ### 2. `_thumb_is_valid` accepted header-only corrupt files The magic-byte check passed for any 8+ byte file starting with a JPEG or PNG header, including empty/truncated/zero-pad files that browsers render as broken. Backfill counted these as `ok` and never regenerated. - Also require file size ≥ `MIN_THUMB_BYTES` (256). Real thumbnails are minimum ~2KB even on solid-color sources; header-only corrupt files top out around 12 bytes. 256 sits well above corrupt and well below any legitimate thumbnail. ## Test plan - [x] CI green on `43b778a` - [ ] Operator triggers backfill and reports the surfaced counts; that'll tell us which scenario the broken-thumbnail tiles are actually in. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 2 commits 2026-06-01 22:34:31 -04:00

fix(thumbnails): surface backfill results + tighten validity check ... 9cbdb70e13

Two coupled problems, operator-flagged 2026-06-01: "missing thumbnails
but triggering backfill found nothing."

1. **Backfill UI was a black box.** `POST /api/thumbnails/backfill`
   returned just `{celery_task_id}` and the admin card said
   "Enqueued." with no counts. There was no way to tell whether the
   scan found 0 candidates, 5000 candidates, or whether the worker
   even picked up the task. "Found nothing" was indistinguishable
   from a broken queue.

   Fix: refactor the scan into a sync helper (`_run_backfill_scan`)
   shared by the Celery task and the API endpoint. The API now runs
   the scan in an executor and returns `{scanned, enqueued, ok,
   regenerated}`. The actual thumbnail generation work still goes
   to the thumbnail Celery queue per row via
   `generate_thumbnail.delay()` — the scan itself is fast
   (SELECT id+thumbnail_path + a file.stat() per row).

2. **`_thumb_is_valid` accepted header-only corrupt files.** The
   magic-byte check passed for any 8-byte file starting with a JPEG
   or PNG header, including empty/truncated/zero-pad files that
   browsers render as broken. Backfill counted these as `ok` and
   never regenerated.

   Fix: also require file size ≥ MIN_THUMB_BYTES (256). Real
   thumbnails are minimum ~2KB even on solid-color sources; header-
   only corrupt files top out around 12 bytes. 256 is well above
   the corrupt floor and well below any legitimate thumbnail.

Plus the admin card now shows the per-run counts instead of
"Enqueued.":
  Scanned 5,432 · enqueued 3 (2 regenerated) · 5,429 ok

fix(test): include 'scanned' in all backfill result assertions ... 43b778aa04

Same 'change shared shape, miss pinned tests' trap as cfa4fb4 — the
prior commit updated only test_backfill_mixed_aggregate; three sibling
tests in the same file also pinned the old {enqueued,ok,regenerated}
shape without 'scanned' and CI bounced.

bvandeusen merged commit 9d0c0b7da8 into main 2026-06-01 22:34:38 -04:00

bvandeusen referenced this issue from a commit 2026-06-03 15:27:17 -04:00

feat(artist): "new since last visit" badge + banner

bvandeusen referenced this pull request 2026-06-03 16:20:47 -04:00

feat(artist): "new since last visit" badge + banner #58

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#47