feat(ml): drop image_record.tagger_predictions — image_prediction is sole store (#768 step 3)
Read cutover verified in prod (suggestions + allowlist read image_prediction; backfill complete at 908k rows / 51k images). Removes the old JSON column and everything that fed it: - ImageRecord.tagger_predictions column removed; migration 0046 DROPs it. tagger_model_version kept as the "tagged / current?" signal the backfill sweep reads (needs-tagging check switched to tagger_model_version IS NULL). - tag_and_embed no longer dual-writes the JSON — image_prediction is the only write path. - importer re-import reset drops the JSON line (image_prediction rows are already deleted on re-import). - Retired the one-time #768 backfill task + the #764 prune task, their admin endpoints, and their Maintenance cards (Backfill/PrunePredictionsCard). - Tests seed/assert via image_prediction; stale column refs removed. Disk reclaim is NOT automatic: DROP COLUMN is a catalog change. Run `VACUUM FULL image_record` off-hours afterward to return the ~100 GB to the OS so DB backups go small (#739). image_prediction (~90 MB) stays in pg_dump — it's the source of truth now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,43 @@
|
||||
"""drop image_record.tagger_predictions (predictions normalized to image_prediction)
|
||||
|
||||
Final step of #768. The per-tag predictions now live in the image_prediction
|
||||
table (backfilled from the JSON, read by suggestions + allowlist, written by
|
||||
tag_and_embed). The old JSON column is dead weight — and it's the ~100 GB of
|
||||
sub-0.70 score tail that bloated image_record's TOAST and broke DB backups
|
||||
(#739). Dropping it is a fast catalog change; it does NOT reclaim the disk on
|
||||
its own — run `VACUUM FULL image_record` (or pg_repack) afterward, off-hours,
|
||||
to return the space to the OS so backups go small.
|
||||
|
||||
DROP COLUMN needs a brief ACCESS EXCLUSIVE lock on image_record; env.py's
|
||||
lock_timeout guards it, so quiesce the ml-worker if a tagging run is in flight
|
||||
(see the migration-lock reference). tagger_model_version is kept — it's the
|
||||
"has this been tagged / is it current?" signal the backfill sweep reads.
|
||||
|
||||
Revision ID: 0046
|
||||
Revises: 0045
|
||||
Create Date: 2026-06-11
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "0046"
|
||||
down_revision: Union[str, None] = "0045"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.drop_column("image_record", "tagger_predictions")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Re-add the column empty. The JSON data is not restored (it lived only in
|
||||
# this column); a downgrade would re-tag or backfill from image_prediction
|
||||
# separately if ever needed.
|
||||
op.add_column(
|
||||
"image_record",
|
||||
sa.Column("tagger_predictions", sa.JSON(), nullable=True),
|
||||
)
|
||||
Reference in New Issue
Block a user