This reverts 2529b51. Not a retreat — a reordering, on the operator's
call, and the better sequence.
The squash's acceptance test (run 4971) found ~130 places where the ORM
models do not describe the deployed schema (#3275), including a
unique=True the database never had and two UNIQUE indexes that exist
only in migrations. Collapsing now would have baked all of that into the
one file a public installer starts from.
So: fix the drift first as ordinary migrations on the intact chain, let
the operator deploy so their database moves to the corrected head, and
only then collapse. The baseline is then generated from reconciled
models and reproduces a schema worth reproducing.
Nothing is lost by reverting. The baseline was never deployed, and
regenerating it after the fixes is strictly better than patching this
copy — it will come out of autogenerate correct rather than needing the
same hand-finishing twice.
87 revisions narrating this project's build-out become one file that
creates the schema in a single step. They cost nothing at runtime — all
86 upgrade steps ran in 0.2s (note #3260) — so this is a presentation
change, not a performance one: a new installer should not inherit our
development history to stand up a database.
Deleted: 87 revisions (6,052 lines), the 10 tests/test_migration_*.py
files (483 lines) that asserted intermediate states and backfills which
no longer exist, and backend/app/utils/artist_backfill.py — the only
live module a migration imported, with no other consumer anywhere. That
last one satisfies the operator's separate request to inline it into
0008 and delete the module; the squash removes both outright.
THE REVISION ID IS "0087", NOT "0001", ON PURPOSE. It is the id of the
last revision collapsed, so an existing database is already at head and
`alembic upgrade head` does nothing. The alternative is `alembic stamp`
against live data, and stamp validates NOTHING — it writes a version
string whether or not the schema matches, so a wrong baseline surfaces
later, via the next real migration, with no clean way back. This removes
that operation rather than making it safe. Future revisions run from
0088.
Four things are hand-written because SQLAlchemy metadata does not carry
them, and none fail at generation time:
1. CREATE EXTENSION vector — the VECTOR columns cannot be
created without it, so it is ordered first in upgrade().
2. CREATE EXTENSION tsm_system_rows — surfaces only when the random
sample query runs.
3. the HNSW index on image_record.siglip_embedding, raw SQL because
create_index cannot express USING hnsw (... vector_cosine_ops).
The quietest of the four: everything works, similarity search just
stops using an index.
4. import pgvector.sqlalchemy.vector — autogenerate EMITS
pgvector.sqlalchemy.vector.VECTOR references without importing it,
so the generated file dies with NameError on first run.
The candidate came out of CI (run 4967) as checksummed base64 rather
than a plain cat, because run 4964's cat was truncated mid-line inside a
column definition with the step still green — 29 tables instead of 42,
and it looked entirely plausible. Verified here: 56,582 bytes,
sha256 471acfca69c0…, 42 tables, 66 indexes, 42 drops.
NOT YET PROVEN against the old chain. baseline.yml does that, and it is
step 2's gate; this commit does not claim the schemas match.
GalleryService.similar() ranks images by pgvector cosine distance to a source
image's precomputed SigLIP embedding — no query-time ML inference. Composes
with the Phase-1/2 scope filters (AND) but replaces the date sort (always
nearest-first, bounded top-N, no cursor). Returns None for a missing source
(→404), [] for a source with no embedding (video / pending ML); excludes self
and NULL-embedding rows. New GET /api/gallery/similar?similar_to=<id>&limit=N.
Image-detail payload gains has_embedding so the UI can hide the surface.
Alembic 0036 adds an HNSW vector_cosine_ops index on siglip_embedding (1152<2000
dims) so the search is sub-50ms ANN instead of a full scan; one-time ~30-60s
build over existing embeddings on deploy. Shared _gallery_images/_image_json
helpers de-dup the scroll/similar builders.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bvandeusen
