Commit Graph
3 Commits
Author SHA1 Message Date
bvandeusen 6959e1220c Revert "db: collapse alembic 0001..0087 into one baseline"
This reverts 2529b51. Not a retreat — a reordering, on the operator's
call, and the better sequence.

The squash's acceptance test (run 4971) found ~130 places where the ORM
models do not describe the deployed schema (#3275), including a
unique=True the database never had and two UNIQUE indexes that exist
only in migrations. Collapsing now would have baked all of that into the
one file a public installer starts from.

So: fix the drift first as ordinary migrations on the intact chain, let
the operator deploy so their database moves to the corrected head, and
only then collapse. The baseline is then generated from reconciled
models and reproduces a schema worth reproducing.

Nothing is lost by reverting. The baseline was never deployed, and
regenerating it after the fixes is strictly better than patching this
copy — it will come out of autogenerate correct rather than needing the
same hand-finishing twice.
2026-08-30 14:34:28 -04:00
bvandeusen 2529b516e6 db: collapse alembic 0001..0087 into one baseline (milestone 328 step 1)
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 9s
CI / lint (push) Successful in 4s
CI / extension-version (push) Successful in 5s
CI / frontend-build (push) Successful in 24s
Build images / build-ml (push) Successful in 42s
CI / backend-lint-and-test (push) Successful in 53s
Build images / build-web (push) Successful in 33s
CI / integration (push) Failing after 3m47s
87 revisions narrating this project's build-out become one file that
creates the schema in a single step. They cost nothing at runtime — all
86 upgrade steps ran in 0.2s (note #3260) — so this is a presentation
change, not a performance one: a new installer should not inherit our
development history to stand up a database.

Deleted: 87 revisions (6,052 lines), the 10 tests/test_migration_*.py
files (483 lines) that asserted intermediate states and backfills which
no longer exist, and backend/app/utils/artist_backfill.py — the only
live module a migration imported, with no other consumer anywhere. That
last one satisfies the operator's separate request to inline it into
0008 and delete the module; the squash removes both outright.

THE REVISION ID IS "0087", NOT "0001", ON PURPOSE. It is the id of the
last revision collapsed, so an existing database is already at head and
`alembic upgrade head` does nothing. The alternative is `alembic stamp`
against live data, and stamp validates NOTHING — it writes a version
string whether or not the schema matches, so a wrong baseline surfaces
later, via the next real migration, with no clean way back. This removes
that operation rather than making it safe. Future revisions run from
0088.

Four things are hand-written because SQLAlchemy metadata does not carry
them, and none fail at generation time:

  1. CREATE EXTENSION vector          — the VECTOR columns cannot be
     created without it, so it is ordered first in upgrade().
  2. CREATE EXTENSION tsm_system_rows — surfaces only when the random
     sample query runs.
  3. the HNSW index on image_record.siglip_embedding, raw SQL because
     create_index cannot express USING hnsw (... vector_cosine_ops).
     The quietest of the four: everything works, similarity search just
     stops using an index.
  4. import pgvector.sqlalchemy.vector — autogenerate EMITS
     pgvector.sqlalchemy.vector.VECTOR references without importing it,
     so the generated file dies with NameError on first run.

The candidate came out of CI (run 4967) as checksummed base64 rather
than a plain cat, because run 4964's cat was truncated mid-line inside a
column definition with the step still green — 29 tables instead of 42,
and it looked entirely plausible. Verified here: 56,582 bytes,
sha256 471acfca69c0…, 42 tables, 66 indexes, 42 drops.

NOT YET PROVEN against the old chain. baseline.yml does that, and it is
step 2's gate; this commit does not claim the schemas match.
2026-08-30 13:45:35 -04:00
bvandeusenandClaude Opus 4.8 a8f624a0f1 fix(posts): link duplicate items to every post + prune bare shells
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m20s
The native Patreon backfill flooded the feed with bare 'Post <id>' shells
(1589 for Anduo). Root cause: PostAttachment.sha256 was GLOBALLY unique, so a
non-art file reused across posts only ever linked to the first one, and
_capture_attachment created the Post before that dedup check — leaving later
posts with no image and no attachment. Duplicate IMAGES had the mirror gap:
attach_in_place returned duplicate_hash/duplicate_phash before _apply_sidecar,
so the second post got no provenance row, and the feed only rendered via
primary_post_id (one post per image).

Operator requirement: a duplicate item must show on EVERY post it appears in.
Unify the fix as link-not-suppress:

- importer: on duplicate_hash / duplicate_phash(larger_exists), append an
  image_provenance row for the new post (keep primary on the first). Both the
  download path (attach_in_place) and the filesystem path (_import_media).
- post_feed_service: render thumbnails by image_provenance UNION primary_post_id,
  so a cross-posted image shows on every post (and legacy primary-only images
  still show).
- PostAttachment: per-post uniqueness — drop UNIQUE(sha256), add partial
  UNIQUE(post_id, sha256) + partial UNIQUE(sha256) WHERE post_id IS NULL
  (migration 0043); _capture_attachment dedups per-(post,sha) over the shared
  sha-addressed blob, so no post is left bare.
- cleanup: new prune-bare-posts maintenance action (cleanup_service
  _bare_post_conditions shared by preview/count/delete per preview/apply parity;
  admin endpoint; PostMaintenanceCard). Deletes posts with zero image links
  (primary or provenance) AND zero attachments. Run after the feed fix so a
  hidden provenance link spares the post instead of deleting it.

Tests: dup image shows on both posts; dup attachment shows on both posts; feed
renders provenance-linked duplicates; prune-bare delete-path == preview.

Operator redeploys (migration 0043) then runs the prune to clear the shells.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 19:28:33 -04:00