3 Commits

Include renames

Author	SHA1	Message	Date
bvandeusen	6959e1220c	Revert "db: collapse alembic 0001..0087 into one baseline" ... This reverts 2529b51. Not a retreat — a reordering, on the operator's call, and the better sequence. The squash's acceptance test (run 4971) found ~130 places where the ORM models do not describe the deployed schema (#3275), including a unique=True the database never had and two UNIQUE indexes that exist only in migrations. Collapsing now would have baked all of that into the one file a public installer starts from. So: fix the drift first as ordinary migrations on the intact chain, let the operator deploy so their database moves to the corrected head, and only then collapse. The baseline is then generated from reconciled models and reproduces a schema worth reproducing. Nothing is lost by reverting. The baseline was never deployed, and regenerating it after the fixes is strictly better than patching this copy — it will come out of autogenerate correct rather than needing the same hand-finishing twice.	2026-08-30 14:34:28 -04:00
bvandeusen	2529b516e6	db: collapse alembic 0001..0087 into one baseline (milestone 328 step 1) ... 87 revisions narrating this project's build-out become one file that creates the schema in a single step. They cost nothing at runtime — all 86 upgrade steps ran in 0.2s (note #3260) — so this is a presentation change, not a performance one: a new installer should not inherit our development history to stand up a database. Deleted: 87 revisions (6,052 lines), the 10 tests/test_migration_*.py files (483 lines) that asserted intermediate states and backfills which no longer exist, and backend/app/utils/artist_backfill.py — the only live module a migration imported, with no other consumer anywhere. That last one satisfies the operator's separate request to inline it into 0008 and delete the module; the squash removes both outright. THE REVISION ID IS "0087", NOT "0001", ON PURPOSE. It is the id of the last revision collapsed, so an existing database is already at head and `alembic upgrade head` does nothing. The alternative is `alembic stamp` against live data, and stamp validates NOTHING — it writes a version string whether or not the schema matches, so a wrong baseline surfaces later, via the next real migration, with no clean way back. This removes that operation rather than making it safe. Future revisions run from 0088. Four things are hand-written because SQLAlchemy metadata does not carry them, and none fail at generation time: 1. CREATE EXTENSION vector — the VECTOR columns cannot be created without it, so it is ordered first in upgrade(). 2. CREATE EXTENSION tsm_system_rows — surfaces only when the random sample query runs. 3. the HNSW index on image_record.siglip_embedding, raw SQL because create_index cannot express USING hnsw (... vector_cosine_ops). The quietest of the four: everything works, similarity search just stops using an index. 4. import pgvector.sqlalchemy.vector — autogenerate EMITS pgvector.sqlalchemy.vector.VECTOR references without importing it, so the generated file dies with NameError on first run. The candidate came out of CI (run 4967) as checksummed base64 rather than a plain cat, because run 4964's cat was truncated mid-line inside a column definition with the step still green — 29 tables instead of 42, and it looked entirely plausible. Verified here: 56,582 bytes, sha256 471acfca69c0…, 42 tables, 66 indexes, 42 drops. NOT YET PROVEN against the old chain. baseline.yml does that, and it is step 2's gate; this commit does not claim the schemas match.	2026-08-30 13:45:35 -04:00
bvandeusenandClaude Opus 4.8	4daa3f2790	feat(ml): operator model swap — GPU re-embed + embedder as a setting (#1190) ... Make the SigLIP embedder an operator choice (drop-in to SigLIP 2: google/siglip2-so400m-patch16-512 is a verified 1152-d model at 512px → no schema change, better small-cue fidelity). A swap = set model + re-embed + retrain, all operator-driven; the GPU agent does the re-embed so it's fast. - settings: embedder_model_name is now a setting (migration 0065) alongside the existing embedder_model_version; both editable + validated (non-empty) in the ml admin API. The server embedder loads by HF name (AutoImageProcessor/Model, model-agnostic), preferring the pre-downloaded local dir for the default so existing deploys don't re-download; rebuilds on a name change. - agent: new 'embed' job = whole-image SigLIP embedding (mean-pool video frames) under the lease-announced model → POST /jobs/submit_embedding writes image_record.siglip_embedding + siglip_model_version. The lease now announces the model FROM THE SETTING (not a constant). - re-embed routing: enqueue_gpu_backfill('embed') selects unembedded + stale- version images; 'siglip' now re-embeds concept crops whose version != current (so a swap re-triggers crops, not just the never-embedded back-catalogue). The CPU ml-worker backfill no longer re-embeds on a version mismatch (it can't churn the library at 512px) — the GPU agent owns version re-embeds. Daily 'embed' + 'siglip' beats self-heal. - scoring: score_image only bags embeddings in the CURRENT model's space (whole- image gated by siglip_model_version, concept regions by embedding_version) so a mid-swap stale vector isn't scored by new-space heads; legacy NULL = current. - UI: GpuAgentCard "Embedding model (advanced)" — edit name/version, Save, and "Re-embed library (GPU)" (queues embed + siglip); points at SigLIP 2. Tests: lease announces model + submit_embedding round-trip; enqueue 'embed' selects stale/unembedded; stale-version excluded from scoring; embedder model settable + empty rejected; siglip gate updated to current-version concept. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa	2026-06-30 10:24:30 -04:00