bvandeusen
79089b50b0
Normalize tagger predictions out of the image_record.tagger_predictions JSON blob into a queryable per-prediction table. Step 1 of the cutover (expand): additive + low-risk — reads still use the JSON, this just adds the table and keeps it populated. - ImagePrediction(image_record_id, raw_name, category, score) — stores the RAW tagger vocab name (not tag_id) so read-time alias→canonical resolution is unchanged. Indexed for per-image reads + by (raw_name, score). - Migration 0045: create table + set-based backfill from the JSON via json_each (fast post-#764-prune). The old column stays (vestigial) and is dropped in a later follow-up — DROP needs an ACCESS EXCLUSIVE lock on the hot image_record table, so it waits for a quiesced-worker window. - tag_and_embed dual-writes the rows (delete-then-insert, idempotent); tagger_store_floor already applied in infer(). Next: switch suggestion + allowlist reads to the table, then drop the JSON write. Plan-task #768. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
38 lines
1.6 KiB
Python
38 lines
1.6 KiB
Python
"""ImagePrediction — one row per (image, tagger vocab prediction).
|
|
|
|
Replaces the image_record.tagger_predictions JSON blob (#768). Storing the
|
|
raw Camie/booru vocab name (not a tag_id) preserves the suggestion read
|
|
path's semantics: raw_name → canonical Tag resolution happens at read time
|
|
via the alias map, and accepting a prediction can CREATE the Tag. The store
|
|
floor (ml_settings.tagger_store_floor) is applied at WRITE time, so only
|
|
predictions >= the floor land here.
|
|
"""
|
|
|
|
from sqlalchemy import Float, ForeignKey, Index, String, UniqueConstraint
|
|
from sqlalchemy.orm import Mapped, mapped_column
|
|
|
|
from .base import Base
|
|
|
|
|
|
class ImagePrediction(Base):
|
|
__tablename__ = "image_prediction"
|
|
__table_args__ = (
|
|
UniqueConstraint(
|
|
"image_record_id", "raw_name", name="image_raw_name",
|
|
),
|
|
# Per-image read (suggestion build) and the "images with tag X above
|
|
# Y" query the JSON blob never allowed.
|
|
Index("ix_image_prediction_image", "image_record_id"),
|
|
Index("ix_image_prediction_name_score", "raw_name", "score"),
|
|
)
|
|
|
|
id: Mapped[int] = mapped_column(primary_key=True)
|
|
image_record_id: Mapped[int] = mapped_column(
|
|
ForeignKey("image_record.id", ondelete="CASCADE"), nullable=False,
|
|
)
|
|
# The raw tagger vocab key (booru form) — NOT a tag_id. Resolved to a
|
|
# canonical Tag at read time, exactly as the old JSON keys were.
|
|
raw_name: Mapped[str] = mapped_column(String(255), nullable=False)
|
|
category: Mapped[str] = mapped_column(String(64), nullable=False)
|
|
score: Mapped[float] = mapped_column(Float, nullable=False)
|