perf(gallery): materialize indexed effective_date sort key
The gallery cursored on COALESCE(post.post_date, image_record.created_at) across the Post outer join — an expression spanning two tables that no index can serve, so every /scroll sorted a large slice of the library (and the old frontend fired ten serially). Materialize it: - image_record.effective_date column + ix_image_record_effective_date (effective_date DESC, id DESC); alembic 0035 backfills COALESCE(primary post's post_date, created_at) for existing rows. - gallery_service._effective_date_col() now returns the column, so scroll / timeline / jump / neighbors all order off the index instead of re-deriving the COALESCE. _neighbors reads record.effective_date directly (drops an extra Post lookup). - importer._apply_sidecar maintains it: when a primary post with a date is linked, effective_date = post.post_date; plain inserts keep the created_at-equivalent server default. Tests: sidecar import asserts effective_date == post.post_date; gallery ordering/timeline/jump test seeds set effective_date alongside created_at. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,70 @@
|
||||
"""image_record.effective_date: materialized gallery sort key + index
|
||||
|
||||
Revision ID: 0035
|
||||
Revises: 0034
|
||||
Create Date: 2026-06-04
|
||||
|
||||
The gallery ordered/cursored on COALESCE(post.post_date,
|
||||
image_record.created_at) across the Post outer join. That expression spans
|
||||
two tables, so no index can serve it — every /scroll sorted a large slice
|
||||
of the library, and the frontend fired ten of them serially per initial
|
||||
load. Materialize the value into image_record.effective_date and index
|
||||
(effective_date DESC, id DESC) so the cursor scroll is an index range scan.
|
||||
|
||||
Backfill = COALESCE(primary post's post_date, created_at) so existing rows
|
||||
keep their exact ordering. New rows get the created_at-equivalent server
|
||||
default; services/importer.py overrides it with the post's date when a
|
||||
primary post with a date is linked.
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "0035"
|
||||
down_revision: Union[str, None] = "0034"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# Add nullable first so the backfill can populate before NOT NULL.
|
||||
op.add_column(
|
||||
"image_record",
|
||||
sa.Column("effective_date", sa.DateTime(timezone=True), nullable=True),
|
||||
)
|
||||
# Pure set-based UPDATEs (no per-row params) — immune to the 65535
|
||||
# bind-parameter ceiling regardless of library size.
|
||||
op.execute(
|
||||
"""
|
||||
UPDATE image_record AS ir
|
||||
SET effective_date = COALESCE(p.post_date, ir.created_at)
|
||||
FROM post AS p
|
||||
WHERE ir.primary_post_id = p.id
|
||||
"""
|
||||
)
|
||||
op.execute(
|
||||
"""
|
||||
UPDATE image_record
|
||||
SET effective_date = created_at
|
||||
WHERE effective_date IS NULL
|
||||
"""
|
||||
)
|
||||
op.alter_column(
|
||||
"image_record",
|
||||
"effective_date",
|
||||
nullable=False,
|
||||
server_default=sa.text("now()"),
|
||||
)
|
||||
# DESC/DESC matches the gallery's ORDER BY effective_date DESC, id DESC
|
||||
# so the scroll is a forward index scan; raw SQL because alembic's
|
||||
# column list doesn't express per-column DESC cleanly.
|
||||
op.execute(
|
||||
"CREATE INDEX ix_image_record_effective_date "
|
||||
"ON image_record (effective_date DESC, id DESC)"
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("ix_image_record_effective_date", table_name="image_record")
|
||||
op.drop_column("image_record", "effective_date")
|
||||
Reference in New Issue
Block a user