db: reconcile the models with the deployed schema (#3275)
Build images / sign-extension (push) Successful in 4s
CI / lint (push) Failing after 2s
CI / extension-version (push) Successful in 2s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 27s
Build images / build-ml (push) Successful in 48s
CI / backend-lint-and-test (push) Successful in 1m7s
Build images / build-web (push) Successful in 40s
CI / integration (push) Successful in 4m1s

Milestone 328's acceptance test compared a database built by the real
0001..0087 chain against one built from the models, and found ~130
places where they disagree. This closes them.

Almost all were the MODEL being wrong, so almost all of this is model
edits with no DDL — the database already had these things, nothing in it
changes, and no deploy is needed for this part:

* 92 columns gained server_default. The models carried Python-side
  `default=` only, so the ORM filled the value and the column had no
  database default. Anything inserting outside the ORM behaved
  differently from production.

* Eleven indexes that existed only in migrations are now declared:
  the three backup_run reporting indexes, the two date-ordered
  image_record browse indexes, import_task and presentation_review,
  and the three task_run history indexes. All use text() for their DESC
  ordering and postgresql_where for the partial one.

* Two UNIQUE indexes that autogenerate silently proposed DROPPING,
  because neither is expressible as a UniqueConstraint:
    uq_tag_name_kind_fandom  — an EXPRESSION index over
                               (name, kind, COALESCE(fandom_id, 0))
    uq_post_artist_external_id_null_source — PARTIAL, WHERE source_id
                               IS NULL
  post.py already had a comment describing the second one. The comment
  was right; nothing declared it.

* The two external_link enum CHECKs (host, status) — rule 36 territory,
  and absent from the model entirely.

* Two indexes were named explicitly. A bare index=True generated
  ix_tag_alias_canonical_tag_id where the database has
  ix_tag_alias_canonical, so autogenerate proposed a drop+create of an
  index that was already there under another name. Same for
  tag_suggestion_rejection.

Only ONE thing needed DDL, as 0088: tag.fandom_id is declared
index=True but no migration ever created that index.

Deliberately NOT here: image_record.sha256. The model says unique=True;
0001 created a plain index. Duplicates are possible today and the ORM
believes otherwise. The fix depends on whether duplicates already exist
— if they do, that is a dedupe decision, not a constraint — so it waits
on an answer about live data.

The real severity of #3275 is not the squash. It is that --autogenerate
has been unsafe on this project: run against the old models it would
have proposed dropping eleven indexes and two uniqueness guarantees.
This commit is contained in:
2026-08-30 14:42:30 -04:00
parent 98b56330d0
commit 5e1996e77f
26 changed files with 259 additions and 90 deletions
+8 -5
View File
@@ -8,7 +8,7 @@ reads it and routes through cleanup_service.delete_images.
from datetime import datetime
from typing import Any
from sqlalchemy import DateTime, Integer, String, Text, func
from sqlalchemy import DateTime, Integer, String, Text, func, text
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
@@ -23,6 +23,7 @@ class LibraryAuditRun(Base):
params: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
status: Mapped[str] = mapped_column(
String(16), nullable=False, default="running", index=True,
server_default="running",
)
# running | ready | applied | cancelled | error
started_at: Mapped[datetime] = mapped_column(
@@ -31,14 +32,16 @@ class LibraryAuditRun(Base):
finished_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True,
)
scanned_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
matched_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
matched_ids: Mapped[list[int]] = mapped_column(JSONB, nullable=False, default=list)
scanned_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
matched_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
matched_ids: Mapped[list[int]] = mapped_column(
JSONB, nullable=False, default=list, server_default=text("'[]'::jsonb")
)
error: Mapped[str | None] = mapped_column(Text, nullable=True)
# Chunked-scan state (alembic 0039): keyset cursor the next chunk resumes
# from, and the last time a chunk made progress (so the recovery sweep can
# tell a progressing multi-chunk audit from a stuck one).
resume_after_id: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
resume_after_id: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
last_progress_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True,
)