feat(ml): drop image_record.tagger_predictions — image_prediction is sole store (#768 step 3)
Read cutover verified in prod (suggestions + allowlist read image_prediction; backfill complete at 908k rows / 51k images). Removes the old JSON column and everything that fed it: - ImageRecord.tagger_predictions column removed; migration 0046 DROPs it. tagger_model_version kept as the "tagged / current?" signal the backfill sweep reads (needs-tagging check switched to tagger_model_version IS NULL). - tag_and_embed no longer dual-writes the JSON — image_prediction is the only write path. - importer re-import reset drops the JSON line (image_prediction rows are already deleted on re-import). - Retired the one-time #768 backfill task + the #764 prune task, their admin endpoints, and their Maintenance cards (Backfill/PrunePredictionsCard). - Tests seed/assert via image_prediction; stale column refs removed. Disk reclaim is NOT automatic: DROP COLUMN is a catalog change. Run `VACUUM FULL image_record` off-hours afterward to return the ~100 GB to the OS so DB backups go small (#739). image_prediction (~90 MB) stays in pg_dump — it's the source of truth now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -251,7 +251,7 @@ async def tags_reset_content():
|
||||
"""Tier-A: delete ALL general + character tags (the Camie-suggestable
|
||||
content vocabulary) so the operator can re-tag from scratch via
|
||||
auto-suggest. fandom + series tags + series_page ordering are preserved,
|
||||
and image tagger_predictions are untouched so suggestions repopulate.
|
||||
and image_prediction rows are untouched so suggestions repopulate.
|
||||
dry-run preview returns per-kind counts + applications + a sample so the
|
||||
UI shows exactly what'll go before the operator confirms (dry_run=false).
|
||||
Irreversible except via DB backup restore."""
|
||||
@@ -348,28 +348,3 @@ async def trigger_reextract_archives():
|
||||
|
||||
async_result = reextract_archive_attachments_task.delay()
|
||||
return jsonify({"task_id": async_result.id, "status": "queued"}), 202
|
||||
|
||||
|
||||
@admin_bp.route("/maintenance/prune-predictions", methods=["POST"])
|
||||
async def trigger_prune_predictions():
|
||||
"""Operator-triggered #764 backfill: drop stored tagger predictions below
|
||||
the current ml_settings.tagger_store_floor and clamp allowlist thresholds
|
||||
up to it. Shrinks image_record's TOAST (~100 GB of sub-0.70 scores).
|
||||
Idempotent + self-resuming; runs on the maintenance_long lane."""
|
||||
from ..tasks.admin import prune_low_confidence_predictions_task
|
||||
|
||||
async_result = prune_low_confidence_predictions_task.delay()
|
||||
return jsonify({"task_id": async_result.id, "status": "queued"}), 202
|
||||
|
||||
|
||||
@admin_bp.route("/maintenance/backfill-predictions", methods=["POST"])
|
||||
async def trigger_backfill_predictions():
|
||||
"""Operator-triggered #768 backfill: copy stored tagger predictions from the
|
||||
image_record.tagger_predictions JSON into the normalized image_prediction
|
||||
table. Batched + resumable + idempotent; runs on the maintenance_long lane.
|
||||
Run this once after deploying migration 0045 (which creates the empty table)
|
||||
to populate predictions for the existing library."""
|
||||
from ..tasks.admin import backfill_image_predictions_task
|
||||
|
||||
async_result = backfill_image_predictions_task.delay()
|
||||
return jsonify({"task_id": async_result.id, "status": "queued"}), 202
|
||||
|
||||
Reference in New Issue
Block a user