bvandeusenandClaude Opus 5
573228b9da
CI / lint (push) Failing after 3s
Build images / sign-extension (push) Successful in 4s
CI / extension-version (push) Successful in 3s
Build images / build-agent (push) Successful in 9s
CI / frontend-build (push) Successful in 34s
Build images / build-ml (push) Successful in 53s
Build images / build-web (push) Successful in 44s
CI / backend-lint-and-test (push) Successful in 1m6s
CI / integration (push) Successful in 4m5s
Closes the residue the first reconciliation pass left, and corrects a
factual error I put into the record.
sha256 was NOT missing a uniqueness guarantee. I read
`op.create_index("ix_image_record_sha256", ...)` at 0001 line 151 and
concluded duplicates were possible, without reading line 149 two lines
above it:
sa.UniqueConstraint("sha256", name="uq_image_record_sha256"),
Uniqueness has held since the initial schema. The database expresses it
as a CONSTRAINT plus a separate non-unique lookup index; the model said
`unique=True, index=True`, which is one UNIQUE index under a different
name. Same guarantee, different objects — which is exactly why the two
schemas did not line up. The model now declares both objects. No DDL.
0088's docstring, which repeated the claim, is corrected in place.
Two real divergences, both the MODEL over-claiming:
* source: uq_source_artist_platform_url (alembic 0010) was declared
nowhere in the models — source.py had no __table_args__ at all — so
autogenerate would have proposed DROPPING it.
* head_metrics_snapshot.tag_id: model said NOT NULL, 0060 created it
nullable. Left nullable; the FK already cascades.
Seven constraints renamed to what the chain actually created, rather than
what base.py's naming convention renders: uq_series_page_image,
uq_series_chapter_anchor_page, fk_series_chapter_anchor_page,
fk_image_record_artist_id, fk_image_provenance_from_attachment, and the
two hand-shortened fk_tsr_* names from 0003.
Float server_defaults now mirror their own migration, per column. The
chain is MIXED: a plain string renders DEFAULT '0.90'::double precision,
sa.text() renders DEFAULT 0.90, and the migrations used both. Seven
columns take text(); the rest stay strings. Two literals also disagreed
outright — process_{auto_apply,conflict}_threshold said 0.9/0.5 against
the migration's 0.90/0.50.
baseline.yml gains two things. A repair for a SECOND generator defect in
the same class as the missing pgvector import: base.py's ck convention
contains %(constraint_name)s, so it applies even to a NAMED
CheckConstraint — autogenerate writes the already-rendered name into the
migration and running it applies the convention again, yielding
ck_ml_settings_ck_ml_settings_singleton. That is round-tripping damage,
not a claim the models make, so it is undone rather than counted.
And the diff now runs twice. Column ORDER differs permanently between a
schema built by 87 ADD COLUMNs and one built in a single shot — the
operator's database keeps chain order forever, a fresh install gets model
order — so a check that failed on it could never pass. The second pass
SORTS column lines within each CREATE TABLE instead of deleting them,
which cannot hide a column present on one side only, or one whose type,
nullability or default differs. Ordered diff is reported as information;
the order-insensitive one is the verdict.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw
152 lines
7.1 KiB
Python
152 lines
7.1 KiB
Python
"""ImageRecord — the gallery's primary entity, ported from ImageRepo.
|
||
|
||
ML fields and thumbnails are declared now (in FC-1) so FC-2 can populate them
|
||
without a schema migration. The SigLIP embedding column uses pgvector's Vector
|
||
type — pgvector extension is enabled in the initial migration.
|
||
"""
|
||
|
||
from datetime import datetime
|
||
|
||
from pgvector.sqlalchemy import Vector
|
||
from sqlalchemy import (
|
||
BigInteger,
|
||
DateTime,
|
||
Enum,
|
||
Float,
|
||
ForeignKey,
|
||
Index,
|
||
Integer,
|
||
String,
|
||
Text,
|
||
UniqueConstraint,
|
||
func,
|
||
text,
|
||
)
|
||
from sqlalchemy.orm import Mapped, mapped_column
|
||
|
||
from .base import Base
|
||
|
||
ORIGIN_CHOICES = ("downloaded", "imported_filesystem", "uploaded")
|
||
|
||
|
||
class ImageRecord(Base):
|
||
__tablename__ = "image_record"
|
||
|
||
|
||
__table_args__ = (
|
||
# alembic 0001. The database enforces sha256 uniqueness with a
|
||
# CONSTRAINT and carries a SEPARATE non-unique btree index; the model
|
||
# said `unique=True, index=True`, which collapses both into a single
|
||
# UNIQUE index under a different name. Same guarantee either way, but
|
||
# not the same objects, so autogenerate saw a drop and an add (#3275).
|
||
UniqueConstraint("sha256", name="uq_image_record_sha256"),
|
||
# alembic 0036, and the last thing in this schema that lived only in a
|
||
# migration. SQLAlchemy CAN express an hnsw index with an operator
|
||
# class, so there is no reason for it to be invisible to the models —
|
||
# and its absence was the quietest failure of the lot: everything
|
||
# works, similarity search just silently stops using an index.
|
||
Index(
|
||
"ix_image_record_siglip_hnsw",
|
||
"siglip_embedding",
|
||
postgresql_using="hnsw",
|
||
postgresql_ops={"siglip_embedding": "vector_cosine_ops"},
|
||
),
|
||
# alembic 0035/0071: the date-ordered browse indexes (#3275).
|
||
Index("ix_image_record_effective_date", text("effective_date DESC"), text("id DESC")),
|
||
Index("ix_image_record_earliest_post_date", text("earliest_post_date DESC"), text("id DESC")),
|
||
)
|
||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||
|
||
# On-disk identity
|
||
path: Mapped[str] = mapped_column(Text, nullable=False, unique=True)
|
||
# index=True only: the UNIQUE half is the named constraint in
|
||
# __table_args__ above, matching what 0001 actually created.
|
||
sha256: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
|
||
phash: Mapped[str | None] = mapped_column(String(32), nullable=True, index=True)
|
||
size_bytes: Mapped[int] = mapped_column(BigInteger, nullable=False)
|
||
mime: Mapped[str] = mapped_column(String(64), nullable=False)
|
||
width: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||
height: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||
# Video container duration (seconds); NULL for images. The Tier-1 video
|
||
# near-dup key (#871): two videos of the same artist with matching duration
|
||
# (+ aspect) are the same content across re-encodes — dedup like image pHash.
|
||
duration_seconds: Mapped[float | None] = mapped_column(Float, nullable=True)
|
||
|
||
# Integrity verification status. FC-2e populates this; FC-2a leaves rows at 'unknown'.
|
||
# Values: 'unknown' (default), 'ok', 'corrupt', 'failed_verification'.
|
||
integrity_status: Mapped[str] = mapped_column(
|
||
String(24), nullable=False, default="unknown", index=True,
|
||
server_default="unknown",
|
||
)
|
||
|
||
# Thumbnail (populated by FC-2)
|
||
thumbnail_path: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||
|
||
# Source provenance for downloaded media (#830 Phase 2). `source_url` is the
|
||
# CDN/origin URL the file was fetched from (debugging + future re-fetch).
|
||
# `source_filehash` is the URL's 32-hex CDN identity segment
|
||
# (utils.paths.filehash_from_url) — the JOIN KEY that maps a post body's
|
||
# inline `<img src=CDN>` back to this local copy so the rendered body serves
|
||
# our stored image instead of hotlinking the public source. Indexed for the
|
||
# render-time lookup. NULL for filesystem-imported / pre-Phase-2 rows.
|
||
source_url: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||
source_filehash: Mapped[str | None] = mapped_column(
|
||
String(32), nullable=True, index=True
|
||
)
|
||
|
||
# Origin / provenance pointers
|
||
origin: Mapped[str] = mapped_column(Enum(*ORIGIN_CHOICES, name="origin_enum"), nullable=False)
|
||
primary_post_id: Mapped[int | None] = mapped_column(
|
||
ForeignKey("post.id", ondelete="SET NULL"), nullable=True, index=True
|
||
)
|
||
# FC-2d-vii-c: canonical per-image artist (the single source of truth
|
||
# for attribution; provenance posts remain lineage detail).
|
||
# FK named explicitly: the naming convention renders this
|
||
# `fk_image_record_artist_id_artist`, but alembic 0008 created it as
|
||
# `fk_image_record_artist_id` (#3275).
|
||
artist_id: Mapped[int | None] = mapped_column(
|
||
ForeignKey(
|
||
"artist.id", ondelete="SET NULL", name="fk_image_record_artist_id"
|
||
),
|
||
nullable=True,
|
||
index=True,
|
||
)
|
||
|
||
# ML fields (populated by the ml-worker / GPU agent). 1152 = SigLIP-so400m
|
||
# embedding dim; siglip_model_version stamps which model produced it (so an
|
||
# operator model swap, #1190, can re-embed the stale rows). A different-dim
|
||
# model would need a column-width migration.
|
||
siglip_embedding: Mapped[list[float] | None] = mapped_column(Vector(1152), nullable=True)
|
||
siglip_model_version: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
||
|
||
created_at: Mapped[datetime] = mapped_column(
|
||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||
)
|
||
# Denormalized gallery sort key = COALESCE(primary post's post_date,
|
||
# created_at) (alembic 0035). The gallery used to compute this as a
|
||
# COALESCE across the Post outer join on every /scroll, which can't use
|
||
# an index and re-sorted a large slice of the library per page (×10 with
|
||
# the old serial batching). Materializing it lets the cursor scroll read
|
||
# ix_image_record_effective_date directly. Maintained by the importer
|
||
# (services/importer.py _apply_sidecar) when a primary post with a date
|
||
# is linked; plain inserts keep the created_at-equivalent server default.
|
||
effective_date: Mapped[datetime] = mapped_column(
|
||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||
)
|
||
# Denormalized ORIGINAL-publish sort key (alembic 0071) = MIN(post_date)
|
||
# across ALL of the image's provenance posts, else created_at. effective_date
|
||
# above keys off the PRIMARY post (often the repost/download the file came
|
||
# from); this keys off the earliest publish across EVERY post the image
|
||
# appears in, so the gallery can sort by when content was first posted rather
|
||
# than when it was downloaded (operator-flagged 2026-07-01). Maintained by
|
||
# services/importer.py, recomputed whenever a dated post is linked.
|
||
earliest_post_date: Mapped[datetime] = mapped_column(
|
||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||
)
|
||
updated_at: Mapped[datetime] = mapped_column(
|
||
DateTime(timezone=True),
|
||
nullable=False,
|
||
server_default=func.now(),
|
||
onupdate=func.now(),
|
||
)
|