feat(ml): drop image_record.tagger_predictions — image_prediction is sole store (#768 step 3) #96

2026-06-11T19:32:00-04:00

bvandeusen commented

2026-06-11 19:32:00 -04:00

Final step of #768. Read cutover verified in prod (suggestions + allowlist read image_prediction; backfill complete at 908k rows / 51k images).

Change

Migration 0046 DROPs image_record.tagger_predictions. The model field is removed; tagger_model_version stays as the "tagged / current?" signal.
tag_and_embed no longer dual-writes the JSON — image_prediction is the only write path. The backfill needs-tagging check switched to tagger_model_version IS NULL.
Importer re-import reset and all tests updated to image_prediction.
Retired the one-time #768 backfill task + #764 prune task, both admin endpoints, and both Maintenance cards.

Deploy notes

Quiesce the ml-worker (docker service scale curator_ml-worker=0) during the migration — 0046's DROP COLUMN takes a brief ACCESS EXCLUSIVE on image_record, and an old ml-worker still writing the JSON column would error against the new schema.
Force a real :latest re-pull and roll all services; web runs 0046 on boot.
Backups (#739) are fixed by the DROP alone (pg_dump is logical). Optional, off-hours: VACUUM (FULL, ANALYZE) image_record to return the ~100 GB physical disk to the OS — it takes an ACCESS EXCLUSIVE lock for the duration (likely minutes; only live data is copied), so run it quiesced.

Dev CI green (run 895: lint, backend, integration incl. the 0046 migration, frontend-build).

🤖 Generated with Claude Code

Final step of #768. Read cutover verified in prod (suggestions + allowlist read `image_prediction`; backfill complete at 908k rows / 51k images). ## Change - **Migration 0046 DROPs `image_record.tagger_predictions`.** The model field is removed; `tagger_model_version` stays as the "tagged / current?" signal. - `tag_and_embed` no longer dual-writes the JSON — `image_prediction` is the only write path. The backfill needs-tagging check switched to `tagger_model_version IS NULL`. - Importer re-import reset and all tests updated to `image_prediction`. - Retired the one-time #768 backfill task + #764 prune task, both admin endpoints, and both Maintenance cards. ## Deploy notes 1. **Quiesce the ml-worker** (`docker service scale curator_ml-worker=0`) during the migration — `0046`'s `DROP COLUMN` takes a brief `ACCESS EXCLUSIVE` on `image_record`, and an old ml-worker still writing the JSON column would error against the new schema. 2. Force a real `:latest` re-pull and roll **all** services; web runs `0046` on boot. 3. Backups (#739) are fixed by the DROP alone (`pg_dump` is logical). Optional, off-hours: `VACUUM (FULL, ANALYZE) image_record` to return the ~100 GB physical disk to the OS — it takes an ACCESS EXCLUSIVE lock for the duration (likely minutes; only live data is copied), so run it quiesced. Dev CI green (run 895: lint, backend, integration incl. the 0046 migration, frontend-build). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 1 commit 2026-06-11 19:32:00 -04:00

feat(ml): drop image_record.tagger_predictions — image_prediction is sole store (#768 step 3)

CI / lint (push) Successful in 3s

Details

CI / frontend-build (push) Successful in 23s

Details

CI / backend-lint-and-test (push) Successful in 33s

Details

CI / integration (push) Successful in 3m14s

Details

3610ba495f

Read cutover verified in prod (suggestions + allowlist read image_prediction;
backfill complete at 908k rows / 51k images). Removes the old JSON column and
everything that fed it:

- ImageRecord.tagger_predictions column removed; migration 0046 DROPs it.
  tagger_model_version kept as the "tagged / current?" signal the backfill
  sweep reads (needs-tagging check switched to tagger_model_version IS NULL).
- tag_and_embed no longer dual-writes the JSON — image_prediction is the only
  write path.
- importer re-import reset drops the JSON line (image_prediction rows are
  already deleted on re-import).
- Retired the one-time #768 backfill task + the #764 prune task, their admin
  endpoints, and their Maintenance cards (Backfill/PrunePredictionsCard).
- Tests seed/assert via image_prediction; stale column refs removed.

Disk reclaim is NOT automatic: DROP COLUMN is a catalog change. Run
`VACUUM FULL image_record` off-hours afterward to return the ~100 GB to the OS
so DB backups go small (#739). image_prediction (~90 MB) stays in pg_dump — it's
the source of truth now.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

bvandeusen merged commit 618f7cdc36 into main

2026-06-11 19:32:05 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#96