Files
FabledCurator/alembic/versions/0089_index_hygiene.py
T
bvandeusenandClaude Opus 5 08418d54a3
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 11s
Build images / build-ml (push) Successful in 32s
Build images / build-web (push) Successful in 26s
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m44s
extension / lint (pull_request) Successful in 24s
db: index the seven unindexed FKs, drop the seven redundant ones (#3300, #3301)
A structural sweep of the deployed schema, run AFTER 0088 got the models
and the chain to exact agreement. That agreement is what 0088 achieved,
and it is worth naming what it does not prove: a models-vs-chain diff
shows the two describe the same schema, not that the schema is right.
Everything here was wrong in BOTH.

The one that matters: image_tag has PRIMARY KEY (image_record_id, tag_id)
and no other index, so tag_id is unindexed. That is the gallery's tag
filter (tag_query.py builds `image_tag.c.tag_id == tid`) and the ON
DELETE CASCADE from tag, both scanning the largest table in the schema.
Six more FKs were unindexed on smaller tables; presentation_review.tag_id
also CASCADEs.

Dropped, on the other side: ix_image_record_sha256 was an exact duplicate
of the index uq_image_record_sha256 already builds — two btrees on the
same column of the highest-insert-rate table. The other six are
single-column indexes a later composite superseded without the narrow one
being retired; a btree on (a,b) already serves lookups on a.

0088 deliberately taught the models to declare BOTH sha256 indexes so
they would describe reality. This changes the reality instead, and the
models change with it — otherwise the next baseline.yml run reintroduces
exactly the drift 0088 removed.

CONCURRENTLY throughout, so building the image_tag index does not hold an
ACCESS EXCLUSIVE lock over every write for the duration. The cost is that
the migration cannot run in a transaction and so is not atomic: every
statement is IF NOT EXISTS / IF EXISTS, making a re-run after a partial
failure safe. The docstring carries the query for finding an INVALID
index left by an interrupted CONCURRENTLY build.

What the sweep found clean, for the record: all 43 tables have a primary
key; all 51 FKs declare an explicit ON DELETE, so none silently blocks a
delete; the three enum CHECKs match the code that writes them (rule 36).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw
2026-08-31 08:25:41 -04:00

121 lines
5.1 KiB
Python

"""Index the seven unindexed FKs; drop the seven redundant indexes (#3300, #3301).
Found by a structural sweep of the deployed schema done AFTER 0088 brought the
models and the migration chain into exact agreement. That agreement is what
0088 achieved, and it is worth being precise about what it does NOT prove: a
models-vs-chain diff shows the two describe the same schema. It says nothing
about whether that schema is right. Everything here was wrong in BOTH, which is
exactly the class of problem the reconciliation could not see.
## Added: seven FK indexes
`image_tag.tag_id` is the one that matters. The table's only index is
PRIMARY KEY (image_record_id, tag_id), which leads with the wrong column for
the two hottest things done with it:
* the gallery's tag filter — services/tag_query.py builds
`image_tag.c.tag_id == tid` (and `.in_(tids)`) on every tag-scoped browse;
* ON DELETE CASCADE from `tag` — deleting or merging a tag makes Postgres
find that tag's rows before it can remove them.
Both had to scan the largest table in the schema. The other six are the same
shape on much smaller tables; `presentation_review.tag_id` is the notable one,
since it also CASCADEs.
## Dropped: seven redundant indexes
`ix_image_record_sha256` was an exact duplicate. A UNIQUE constraint builds its
own index, so `uq_image_record_sha256` already covered the column and
`image_record` carried two btrees on `sha256` — on the highest-insert-rate
table in the system.
The other six are single-column indexes that a later composite superseded
without the narrow one being retired. A btree on (a, b) already serves lookups
on `a`, so each was pure write amplification. `task_run` and `backup_run` are
append-heavy operational logs, which is where that cost lands hardest.
Note for anyone reading 0088 next to this: 0088 deliberately taught the models
to declare BOTH sha256 indexes, so they would describe reality. That was right.
This migration changes the reality instead, and the models change with it.
## CONCURRENTLY, and why this migration has no transaction
`CREATE INDEX` takes an ACCESS EXCLUSIVE lock for the whole build, which on
`image_tag` means stalling every write for as long as it takes. CONCURRENTLY
builds without blocking writers, at the cost of two table passes and an
inability to run inside a transaction — hence `autocommit_block()`.
The consequence to know about: this migration is NOT atomic. If it fails
partway, the work already done stays done. Every statement is therefore written
IF NOT EXISTS / IF EXISTS so that re-running it after a failure is safe rather
than an error.
A failed CONCURRENTLY build also leaves an INVALID index behind — it is not
used by the planner and not repaired automatically. Find them with:
SELECT c.relname FROM pg_index i
JOIN pg_class c ON c.oid = i.indexrelid
WHERE NOT i.indisvalid;
Drop what that returns and re-run; nothing else is needed.
Revision ID: 0089
Revises: 0088
Create Date: 2026-08-31
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0089"
down_revision: Union[str, None] = "0088"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
# (index name, table, column) — names match what the models render under
# base.py's naming convention, so autogenerate stays quiet after this.
MISSING_FK_INDEXES = (
("ix_image_tag_tag_id", "image_tag", "tag_id"),
("ix_presentation_review_tag_id", "presentation_review", "tag_id"),
("ix_presentation_review_conflict_tag_id", "presentation_review", "conflict_tag_id"),
("ix_import_task_result_image_id", "import_task", "result_image_id"),
("ix_external_link_attachment_id", "external_link", "attachment_id"),
("ix_character_prototype_region_id", "character_prototype", "region_id"),
("ix_backup_run_restored_from_id", "backup_run", "restored_from_id"),
)
# (index name, table, column) — redundant; the second element of each pair in
# the docstring is what still covers the column after the drop.
REDUNDANT_INDEXES = (
("ix_image_record_sha256", "image_record", "sha256"),
("ix_backup_run_kind", "backup_run", "kind"),
("ix_backup_run_status", "backup_run", "status"),
("ix_task_run_queue", "task_run", "queue"),
("ix_task_run_status", "task_run", "status"),
("ix_task_run_task_name", "task_run", "task_name"),
("ix_external_link_post_id", "external_link", "post_id"),
)
def upgrade() -> None:
with op.get_context().autocommit_block():
for name, table, column in MISSING_FK_INDEXES:
op.execute(
f"CREATE INDEX CONCURRENTLY IF NOT EXISTS {name} "
f"ON {table} ({column})"
)
for name, _table, _column in REDUNDANT_INDEXES:
op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {name}")
def downgrade() -> None:
with op.get_context().autocommit_block():
for name, table, column in REDUNDANT_INDEXES:
op.execute(
f"CREATE INDEX CONCURRENTLY IF NOT EXISTS {name} "
f"ON {table} ({column})"
)
for name, _table, _column in MISSING_FK_INDEXES:
op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {name}")