feat(ml): image_prediction table + backfill + dual-write (#768 step 1)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 36s
CI / integration (push) Successful in 3m22s

Normalize tagger predictions out of the image_record.tagger_predictions JSON
blob into a queryable per-prediction table. Step 1 of the cutover (expand):
additive + low-risk — reads still use the JSON, this just adds the table and
keeps it populated.

- ImagePrediction(image_record_id, raw_name, category, score) — stores the
  RAW tagger vocab name (not tag_id) so read-time alias→canonical resolution
  is unchanged. Indexed for per-image reads + by (raw_name, score).
- Migration 0045: create table + set-based backfill from the JSON via
  json_each (fast post-#764-prune). The old column stays (vestigial) and is
  dropped in a later follow-up — DROP needs an ACCESS EXCLUSIVE lock on the
  hot image_record table, so it waits for a quiesced-worker window.
- tag_and_embed dual-writes the rows (delete-then-insert, idempotent);
  tagger_store_floor already applied in infer().

Next: switch suggestion + allowlist reads to the table, then drop the JSON
write. Plan-task #768.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 15:55:32 -04:00
parent 7a40a50fe9
commit 79089b50b0
5 changed files with 188 additions and 2 deletions
+19 -2
View File
@@ -10,11 +10,11 @@ import logging
from pathlib import Path
from celery.exceptions import SoftTimeLimitExceeded
from sqlalchemy import select
from sqlalchemy import delete, select
from sqlalchemy.exc import DBAPIError, OperationalError
from ..celery_app import celery
from ..models import ImageRecord, MLSettings
from ..models import ImagePrediction, ImageRecord, MLSettings
from ._sync_engine import sync_session_factory as _sync_session_factory
log = logging.getLogger(__name__)
@@ -162,6 +162,23 @@ def tag_and_embed(self, image_id: int) -> dict:
record.siglip_embedding = embedding.tolist()
record.siglip_model_version = settings.embedder_model_version
session.add(record)
# Write the normalized image_prediction rows (#768). Delete-then-
# insert keeps a re-tag idempotent. tagger_store_floor was already
# applied in tagger.infer, so preds is the >=floor set. (Transitional
# dual-write alongside the JSON column until the read cutover lands.)
session.execute(
delete(ImagePrediction).where(
ImagePrediction.image_record_id == image_id
)
)
session.add_all([
ImagePrediction(
image_record_id=image_id, raw_name=name,
category=p.get("category", "general"),
score=float(p.get("confidence", 0.0)),
)
for name, p in preds.items()
])
session.commit()
except SoftTimeLimitExceeded:
log.error(