bvandeusen/FabledCurator
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 12 Wiki Activity
Files
2529b516e60d07689d8460339da93b51332256af
FabledCurator/alembic/versions/0087_baseline.py
T
bvandeusen 2529b516e6
Build images / sign-extension (push) Successful in 4s
Details
CI / lint (push) Successful in 4s
Details
CI / extension-version (push) Successful in 5s
Details
Build images / build-agent (push) Successful in 9s
Details
CI / frontend-build (push) Successful in 24s
Details
Build images / build-ml (push) Successful in 42s
Details
CI / backend-lint-and-test (push) Successful in 53s
Details
Build images / build-web (push) Successful in 33s
Details
CI / integration (push) Failing after 3m47s
Details
db: collapse alembic 0001..0087 into one baseline (milestone 328 step 1) 
87 revisions narrating this project's build-out become one file that
creates the schema in a single step. They cost nothing at runtime — all
86 upgrade steps ran in 0.2s (note #3260) — so this is a presentation
change, not a performance one: a new installer should not inherit our
development history to stand up a database.

Deleted: 87 revisions (6,052 lines), the 10 tests/test_migration_*.py
files (483 lines) that asserted intermediate states and backfills which
no longer exist, and backend/app/utils/artist_backfill.py — the only
live module a migration imported, with no other consumer anywhere. That
last one satisfies the operator's separate request to inline it into
0008 and delete the module; the squash removes both outright.

THE REVISION ID IS "0087", NOT "0001", ON PURPOSE. It is the id of the
last revision collapsed, so an existing database is already at head and
`alembic upgrade head` does nothing. The alternative is `alembic stamp`
against live data, and stamp validates NOTHING — it writes a version
string whether or not the schema matches, so a wrong baseline surfaces
later, via the next real migration, with no clean way back. This removes
that operation rather than making it safe. Future revisions run from
0088.

Four things are hand-written because SQLAlchemy metadata does not carry
them, and none fail at generation time:

  1. CREATE EXTENSION vector          — the VECTOR columns cannot be
     created without it, so it is ordered first in upgrade().
  2. CREATE EXTENSION tsm_system_rows — surfaces only when the random
     sample query runs.
  3. the HNSW index on image_record.siglip_embedding, raw SQL because
     create_index cannot express USING hnsw (... vector_cosine_ops).
     The quietest of the four: everything works, similarity search just
     stops using an index.
  4. import pgvector.sqlalchemy.vector — autogenerate EMITS
     pgvector.sqlalchemy.vector.VECTOR references without importing it,
     so the generated file dies with NameError on first run.

The candidate came out of CI (run 4967) as checksummed base64 rather
than a plain cat, because run 4964's cat was truncated mid-line inside a
column definition with the step still green — 29 tables instead of 42,
and it looked entirely plausible. Verified here: 56,582 bytes,
sha256 471acfca69c0…, 42 tables, 66 indexes, 42 drops.

NOT YET PROVEN against the old chain. baseline.yml does that, and it is
step 2's gate; this commit does not claim the schemas match.
2026-08-30 13:45:35 -04:00

873 lines
58 KiB
Python
Raw Blame History

"""Collapsed baseline — the whole schema in one revision.
Replaces revisions 0001..0087, which narrated the build-out of this project
and were deleted in milestone 328 step 1. A new install creates the schema in
one step instead of replaying that history.
WHY THE REVISION ID IS "0087" AND NOT "0001"
--------------------------------------------
It is deliberately the id of the LAST revision this baseline collapses, so an
existing database needs no intervention at all:
* a fresh install finds current=none, head=0087, runs this file once, and
ends stamped at 0087.
* an existing install is ALREADY at 0087, so `alembic upgrade head` finds
current == head and does nothing.
The alternative — numbering this 0001 and stamping every existing database —
means running `alembic stamp` against live data, and stamp VALIDATES NOTHING.
It writes a version string whether or not the schema actually matches, so a
wrong baseline would be discovered later, by the next real migration, with no
clean way back. Keeping the id removes that operation instead of making it
safe. Future revisions continue at 0088.
The one case this makes worse, and it fails LOUDLY rather than silently: a
database still sitting between 0001 and 0086 (i.e. never upgraded to head)
cannot be located in this chain and errors out. Upgrade to 0087 on a
pre-squash build first, then take this one.
WHAT IS HAND-WRITTEN HERE
-------------------------
Most of this file is `alembic revision --autogenerate` output, but four
things are NOT in SQLAlchemy metadata and the generator cannot produce them.
Each fails differently, and none of them fail at generation time:
1. CREATE EXTENSION vector (was 0001) — without it the VECTOR
columns below cannot be created at all.
2. CREATE EXTENSION tsm_system_rows (was 0004) — used by the random-sample
query path; its absence surfaces only when that query runs.
3. The HNSW index on image_record.siglip_embedding (was 0036). Raw SQL
because alembic's create_index cannot express `USING hnsw (...
vector_cosine_ops)`. Its absence is the quietest failure of the four:
everything works, similarity search just stops using an index.
4. `import pgvector.sqlalchemy.vector`. Autogenerate EMITS references to
pgvector.sqlalchemy.vector.VECTOR but does not add the import, so the
generated file dies with NameError on first run.
The acceptance test for this file is not that it reads correctly — it is
`.forgejo/workflows/baseline.yml`, which builds a database from the old
0001..0087 chain (read out of git) and one from this file, and diffs
pg_dump --schema-only output. That is what proves nothing was missed.
Revision ID: 0087
Revises:
Create Date: 2026-08-30
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
# Autogenerate references pgvector.sqlalchemy.vector.VECTOR without importing
# it. Item 4 above.
import pgvector.sqlalchemy.vector
revision: str = "0087"
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Extensions FIRST: the VECTOR columns below cannot be created without
# `vector`, so ordering here is load-bearing, not tidiness.
op.execute("CREATE EXTENSION IF NOT EXISTS vector")
op.execute("CREATE EXTENSION IF NOT EXISTS tsm_system_rows")
op.create_table('app_setting',
sa.Column('key', sa.String(length=64), nullable=False),
sa.Column('value', sa.Text(), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.PrimaryKeyConstraint('key', name=op.f('pk_app_setting'))
)
op.create_table('artist',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('slug', sa.String(length=255), nullable=False),
sa.Column('notes', sa.Text(), nullable=True),
sa.Column('is_subscription', sa.Boolean(), nullable=False),
sa.Column('auto_check', sa.Boolean(), nullable=False),
sa.Column('check_interval_seconds', sa.Integer(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_artist')),
sa.UniqueConstraint('slug', name=op.f('uq_artist_slug'))
)
op.create_table('backup_run',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('kind', sa.String(length=16), nullable=False),
sa.Column('status', sa.String(length=16), nullable=False),
sa.Column('tag', sa.String(length=64), nullable=True),
sa.Column('triggered_by', sa.String(length=32), nullable=False),
sa.Column('started_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('finished_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('sql_path', sa.Text(), nullable=True),
sa.Column('tar_path', sa.Text(), nullable=True),
sa.Column('size_bytes', sa.BigInteger(), nullable=True),
sa.Column('error', sa.Text(), nullable=True),
sa.Column('manifest', sa.JSON(), server_default='{}', nullable=False),
sa.Column('restored_from_id', sa.Integer(), nullable=True),
sa.ForeignKeyConstraint(['restored_from_id'], ['backup_run.id'], name=op.f('fk_backup_run_restored_from_id_backup_run'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_backup_run'))
)
op.create_index(op.f('ix_backup_run_finished_at'), 'backup_run', ['finished_at'], unique=False)
op.create_index(op.f('ix_backup_run_kind'), 'backup_run', ['kind'], unique=False)
op.create_index(op.f('ix_backup_run_started_at'), 'backup_run', ['started_at'], unique=False)
op.create_index(op.f('ix_backup_run_status'), 'backup_run', ['status'], unique=False)
op.create_index(op.f('ix_backup_run_tag'), 'backup_run', ['tag'], unique=False)
op.create_table('credential',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('platform', sa.String(length=64), nullable=False),
sa.Column('credential_type', sa.String(length=32), nullable=False),
sa.Column('encrypted_blob', sa.LargeBinary(), nullable=False),
sa.Column('captured_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('expires_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('last_verified', sa.DateTime(timezone=True), nullable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_credential')),
sa.UniqueConstraint('platform', name=op.f('uq_credential_platform'))
)
op.create_table('head_auto_apply_run',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('dry_run', sa.Boolean(), nullable=False),
sa.Column('params', postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column('status', sa.String(length=16), nullable=False),
sa.Column('started_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('finished_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('n_applied', sa.Integer(), nullable=True),
sa.Column('report', postgresql.JSONB(astext_type=sa.Text()), nullable=True),
sa.Column('error', sa.Text(), nullable=True),
sa.Column('last_progress_at', sa.DateTime(timezone=True), nullable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_head_auto_apply_run'))
)
op.create_index(op.f('ix_head_auto_apply_run_status'), 'head_auto_apply_run', ['status'], unique=False)
op.create_table('head_training_run',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('params', postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column('status', sa.String(length=16), nullable=False),
sa.Column('started_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('finished_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('n_trained', sa.Integer(), nullable=True),
sa.Column('n_skipped', sa.Integer(), nullable=True),
sa.Column('error', sa.Text(), nullable=True),
sa.Column('last_progress_at', sa.DateTime(timezone=True), nullable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_head_training_run'))
)
op.create_index(op.f('ix_head_training_run_status'), 'head_training_run', ['status'], unique=False)
op.create_table('import_batch',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('triggered_by', sa.String(length=32), nullable=False),
sa.Column('source_path', sa.Text(), nullable=False),
sa.Column('scan_mode', sa.String(length=16), nullable=False),
sa.Column('started_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('finished_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('total_files', sa.Integer(), nullable=False),
sa.Column('imported', sa.Integer(), nullable=False),
sa.Column('skipped', sa.Integer(), nullable=False),
sa.Column('failed', sa.Integer(), nullable=False),
sa.Column('attachments', sa.Integer(), nullable=False),
sa.Column('refreshed', sa.Integer(), nullable=False),
sa.Column('status', sa.String(length=16), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_import_batch'))
)
op.create_index(op.f('ix_import_batch_status'), 'import_batch', ['status'], unique=False)
op.create_table('import_settings',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('import_scan_path', sa.Text(), nullable=False),
sa.Column('min_width', sa.Integer(), nullable=False),
sa.Column('min_height', sa.Integer(), nullable=False),
sa.Column('skip_transparent', sa.Boolean(), nullable=False),
sa.Column('transparency_threshold', sa.Float(), nullable=False),
sa.Column('skip_single_color', sa.Boolean(), nullable=False),
sa.Column('single_color_threshold', sa.Float(), nullable=False),
sa.Column('single_color_tolerance', sa.Integer(), nullable=False),
sa.Column('phash_threshold', sa.Integer(), nullable=False),
sa.Column('download_rate_limit_seconds', sa.Float(), nullable=False),
sa.Column('download_validate_files', sa.Boolean(), nullable=False),
sa.Column('download_schedule_default_seconds', sa.Integer(), nullable=False),
sa.Column('download_event_retention_days', sa.Integer(), nullable=False),
sa.Column('download_failure_warning_threshold', sa.Integer(), nullable=False),
sa.Column('backup_db_nightly_enabled', sa.Boolean(), nullable=False),
sa.Column('backup_db_nightly_hour_utc', sa.Integer(), nullable=False),
sa.Column('backup_db_keep_last_n', sa.Integer(), nullable=False),
sa.Column('backup_images_keep_last_n', sa.Integer(), nullable=False),
sa.Column('series_suggest_enabled', sa.Boolean(), nullable=False),
sa.Column('series_suggest_threshold', sa.Float(), nullable=False),
sa.Column('extdl_mega_enabled', sa.Boolean(), server_default='true', nullable=False),
sa.Column('extdl_gdrive_enabled', sa.Boolean(), server_default='true', nullable=False),
sa.Column('extdl_mediafire_enabled', sa.Boolean(), server_default='true', nullable=False),
sa.Column('extdl_dropbox_enabled', sa.Boolean(), server_default='true', nullable=False),
sa.Column('extdl_pixeldrain_enabled', sa.Boolean(), server_default='true', nullable=False),
sa.Column('translation_enabled', sa.Boolean(), server_default='false', nullable=False),
sa.Column('interpreter_base_url', sa.Text(), server_default='', nullable=False),
sa.Column('translation_target_lang', sa.Text(), server_default='en', nullable=False),
sa.Column('translation_min_confidence', sa.Float(), server_default='0.9', nullable=False),
sa.Column('wip_title_tagging_enabled', sa.Boolean(), server_default='true', nullable=False),
sa.Column('wip_soft_title_tagging_enabled', sa.Boolean(), server_default='false', nullable=False),
sa.CheckConstraint('id = 1', name=op.f('ck_import_settings_singleton')),
sa.PrimaryKeyConstraint('id', name=op.f('pk_import_settings'))
)
op.create_table('library_audit_run',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('rule', sa.String(length=32), nullable=False),
sa.Column('params', postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column('status', sa.String(length=16), nullable=False),
sa.Column('started_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('finished_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('scanned_count', sa.Integer(), nullable=False),
sa.Column('matched_count', sa.Integer(), nullable=False),
sa.Column('matched_ids', postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column('error', sa.Text(), nullable=True),
sa.Column('resume_after_id', sa.Integer(), nullable=False),
sa.Column('last_progress_at', sa.DateTime(timezone=True), nullable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_library_audit_run'))
)
op.create_index(op.f('ix_library_audit_run_rule'), 'library_audit_run', ['rule'], unique=False)
op.create_index(op.f('ix_library_audit_run_status'), 'library_audit_run', ['status'], unique=False)
op.create_table('ml_settings',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('cpu_embed_enabled', sa.Boolean(), nullable=False),
sa.Column('video_frame_interval_seconds', sa.Float(), nullable=False),
sa.Column('video_max_frames', sa.Integer(), nullable=False),
sa.Column('head_min_positives', sa.Integer(), nullable=False),
sa.Column('head_auto_apply_precision', sa.Float(), nullable=False),
sa.Column('head_auto_apply_enabled', sa.Boolean(), nullable=False),
sa.Column('head_auto_apply_min_positives', sa.Integer(), nullable=False),
sa.Column('ccip_match_threshold', sa.Float(), nullable=False),
sa.Column('ccip_auto_apply_enabled', sa.Boolean(), nullable=False),
sa.Column('ccip_auto_apply_threshold', sa.Float(), nullable=False),
sa.Column('presentation_auto_apply_enabled', sa.Boolean(), nullable=False),
sa.Column('presentation_auto_apply_threshold', sa.Float(), nullable=False),
sa.Column('presentation_conflict_threshold', sa.Float(), nullable=False),
sa.Column('process_auto_apply_enabled', sa.Boolean(), nullable=False),
sa.Column('process_auto_apply_threshold', sa.Float(), nullable=False),
sa.Column('process_conflict_threshold', sa.Float(), nullable=False),
sa.Column('embedder_model_version', sa.String(length=128), nullable=False),
sa.Column('embedder_model_name', sa.String(length=128), nullable=False),
sa.Column('detector_person_enabled', sa.Boolean(), nullable=False),
sa.Column('detector_person_weights', sa.String(length=512), nullable=False),
sa.Column('detector_person_conf', sa.Float(), nullable=False),
sa.Column('detector_anatomy_enabled', sa.Boolean(), nullable=False),
sa.Column('detector_anatomy_weights', sa.String(length=512), nullable=False),
sa.Column('detector_anatomy_conf', sa.Float(), nullable=False),
sa.Column('detector_panel_enabled', sa.Boolean(), nullable=False),
sa.Column('detector_panel_weights', sa.String(length=512), nullable=False),
sa.Column('detector_panel_conf', sa.Float(), nullable=False),
sa.Column('detector_max_figures', sa.Integer(), nullable=False),
sa.Column('detector_max_components', sa.Integer(), nullable=False),
sa.Column('detector_max_panels', sa.Integer(), nullable=False),
sa.Column('detector_max_regions', sa.Integer(), nullable=False),
sa.Column('detector_dedupe_iou', sa.Float(), nullable=False),
sa.Column('ccip_ref_signature', sa.String(length=128), nullable=True),
sa.Column('ccip_prototype_cap', sa.Integer(), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.CheckConstraint('id = 1', name=op.f('ck_ml_settings_singleton')),
sa.PrimaryKeyConstraint('id', name=op.f('pk_ml_settings'))
)
op.create_table('tag',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('kind', sa.Enum('artist', 'character', 'fandom', 'general', 'series', 'archive', 'post', name='tag_kind'), nullable=False),
sa.Column('fandom_id', sa.Integer(), nullable=True),
sa.Column('is_system', sa.Boolean(), server_default=sa.text('false'), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.CheckConstraint("(fandom_id IS NULL) OR (kind = 'character')", name=op.f('ck_tag_ck_tag_fandom_requires_character')),
sa.ForeignKeyConstraint(['fandom_id'], ['tag.id'], name=op.f('fk_tag_fandom_id_tag'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_tag'))
)
op.create_index(op.f('ix_tag_fandom_id'), 'tag', ['fandom_id'], unique=False)
op.create_table('task_run',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('celery_task_id', sa.String(length=64), nullable=False),
sa.Column('queue', sa.String(length=32), nullable=False),
sa.Column('task_name', sa.String(length=128), nullable=False),
sa.Column('target_id', sa.Integer(), nullable=True),
sa.Column('started_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('finished_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('duration_ms', sa.Integer(), nullable=True),
sa.Column('status', sa.String(length=16), nullable=False),
sa.Column('error_type', sa.String(length=128), nullable=True),
sa.Column('error_message', sa.Text(), nullable=True),
sa.Column('retry_count', sa.Integer(), nullable=True),
sa.Column('worker_hostname', sa.String(length=128), nullable=True),
sa.Column('args_summary', sa.String(length=255), nullable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_task_run'))
)
op.create_index(op.f('ix_task_run_celery_task_id'), 'task_run', ['celery_task_id'], unique=False)
op.create_index(op.f('ix_task_run_finished_at'), 'task_run', ['finished_at'], unique=False)
op.create_index(op.f('ix_task_run_queue'), 'task_run', ['queue'], unique=False)
op.create_index(op.f('ix_task_run_started_at'), 'task_run', ['started_at'], unique=False)
op.create_index(op.f('ix_task_run_status'), 'task_run', ['status'], unique=False)
op.create_index(op.f('ix_task_run_task_name'), 'task_run', ['task_name'], unique=False)
op.create_table('artist_visit',
sa.Column('artist_id', sa.Integer(), nullable=False),
sa.Column('last_viewed_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['artist_id'], ['artist.id'], name=op.f('fk_artist_visit_artist_id_artist'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('artist_id', name=op.f('pk_artist_visit'))
)
op.create_table('ccip_prototype_state',
sa.Column('tag_id', sa.Integer(), nullable=False),
sa.Column('fingerprint', sa.String(length=64), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['tag_id'], ['tag.id'], name=op.f('fk_ccip_prototype_state_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('tag_id', name=op.f('pk_ccip_prototype_state'))
)
op.create_table('head_metric',
sa.Column('tag_id', sa.Integer(), nullable=False),
sa.Column('n_misfires', sa.Integer(), nullable=False),
sa.Column('n_underfires', sa.Integer(), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['tag_id'], ['tag.id'], name=op.f('fk_head_metric_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('tag_id', name=op.f('pk_head_metric'))
)
op.create_table('head_metrics_snapshot',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('tag_id', sa.Integer(), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('snapshot_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('n_auto_applied', sa.Integer(), nullable=False),
sa.Column('n_misfires', sa.Integer(), nullable=False),
sa.Column('n_underfires', sa.Integer(), nullable=False),
sa.Column('ap', sa.Float(), nullable=True),
sa.Column('precision_cv', sa.Float(), nullable=True),
sa.Column('recall', sa.Float(), nullable=True),
sa.Column('n_pos', sa.Integer(), nullable=True),
sa.ForeignKeyConstraint(['tag_id'], ['tag.id'], name=op.f('fk_head_metrics_snapshot_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_head_metrics_snapshot'))
)
op.create_index(op.f('ix_head_metrics_snapshot_snapshot_at'), 'head_metrics_snapshot', ['snapshot_at'], unique=False)
op.create_index(op.f('ix_head_metrics_snapshot_tag_id'), 'head_metrics_snapshot', ['tag_id'], unique=False)
op.create_table('source',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('artist_id', sa.Integer(), nullable=False),
sa.Column('platform', sa.String(length=64), nullable=False),
sa.Column('url', sa.Text(), nullable=False),
sa.Column('enabled', sa.Boolean(), nullable=False),
sa.Column('config_overrides', sa.JSON(), nullable=True),
sa.Column('last_checked_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('last_error', sa.Text(), nullable=True),
sa.Column('error_type', sa.String(length=32), nullable=True),
sa.Column('check_interval_override', sa.Integer(), nullable=True),
sa.Column('consecutive_failures', sa.Integer(), nullable=False),
sa.Column('backfill_runs_remaining', sa.Integer(), server_default='0', nullable=False),
sa.ForeignKeyConstraint(['artist_id'], ['artist.id'], name=op.f('fk_source_artist_id_artist'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_source'))
)
op.create_index(op.f('ix_source_artist_id'), 'source', ['artist_id'], unique=False)
op.create_index(op.f('ix_source_error_type'), 'source', ['error_type'], unique=False)
op.create_table('tag_alias',
sa.Column('alias_string', sa.String(length=255), nullable=False),
sa.Column('alias_category', sa.String(length=32), nullable=False),
sa.Column('canonical_tag_id', sa.Integer(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['canonical_tag_id'], ['tag.id'], name=op.f('fk_tag_alias_canonical_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('alias_string', 'alias_category', name=op.f('pk_tag_alias'))
)
op.create_index(op.f('ix_tag_alias_canonical_tag_id'), 'tag_alias', ['canonical_tag_id'], unique=False)
op.create_table('tag_head',
sa.Column('tag_id', sa.Integer(), nullable=False),
sa.Column('embedding_version', sa.String(length=128), nullable=False),
sa.Column('weights', pgvector.sqlalchemy.vector.VECTOR(dim=1152), nullable=False),
sa.Column('bias', sa.Float(), nullable=False),
sa.Column('suggest_threshold', sa.Float(), nullable=False),
sa.Column('auto_apply_threshold', sa.Float(), nullable=True),
sa.Column('n_pos', sa.Integer(), nullable=False),
sa.Column('n_neg', sa.Integer(), nullable=False),
sa.Column('ap', sa.Float(), nullable=False),
sa.Column('precision_cv', sa.Float(), nullable=False),
sa.Column('recall', sa.Float(), nullable=False),
sa.Column('trained_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('train_fingerprint', sa.String(length=128), nullable=True),
sa.Column('metrics', postgresql.JSONB(astext_type=sa.Text()), nullable=True),
sa.ForeignKeyConstraint(['tag_id'], ['tag.id'], name=op.f('fk_tag_head_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('tag_id', name=op.f('pk_tag_head'))
)
op.create_table('patreon_failed_media',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('source_id', sa.Integer(), nullable=False),
sa.Column('filehash', sa.String(length=128), nullable=False),
sa.Column('attempts', sa.Integer(), nullable=False),
sa.Column('last_error', sa.Text(), nullable=True),
sa.Column('first_failed_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('last_failed_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['source_id'], ['source.id'], name=op.f('fk_patreon_failed_media_source_id_source'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_patreon_failed_media')),
sa.UniqueConstraint('source_id', 'filehash', name='uq_patreon_failed_media_source_id')
)
op.create_index(op.f('ix_patreon_failed_media_source_id'), 'patreon_failed_media', ['source_id'], unique=False)
op.create_table('patreon_seen_media',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('source_id', sa.Integer(), nullable=False),
sa.Column('filehash', sa.String(length=128), nullable=False),
sa.Column('post_id', sa.String(length=64), nullable=True),
sa.Column('seen_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['source_id'], ['source.id'], name=op.f('fk_patreon_seen_media_source_id_source'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_patreon_seen_media')),
sa.UniqueConstraint('source_id', 'filehash', name='uq_patreon_seen_media_source_id')
)
op.create_index(op.f('ix_patreon_seen_media_source_id'), 'patreon_seen_media', ['source_id'], unique=False)
op.create_table('pixiv_failed_media',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('source_id', sa.Integer(), nullable=False),
sa.Column('filehash', sa.String(length=128), nullable=False),
sa.Column('attempts', sa.Integer(), nullable=False),
sa.Column('last_error', sa.Text(), nullable=True),
sa.Column('first_failed_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('last_failed_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['source_id'], ['source.id'], name=op.f('fk_pixiv_failed_media_source_id_source'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_pixiv_failed_media')),
sa.UniqueConstraint('source_id', 'filehash', name='uq_pixiv_failed_media_source_id')
)
op.create_index(op.f('ix_pixiv_failed_media_source_id'), 'pixiv_failed_media', ['source_id'], unique=False)
op.create_table('pixiv_seen_media',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('source_id', sa.Integer(), nullable=False),
sa.Column('filehash', sa.String(length=128), nullable=False),
sa.Column('post_id', sa.String(length=64), nullable=True),
sa.Column('seen_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['source_id'], ['source.id'], name=op.f('fk_pixiv_seen_media_source_id_source'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_pixiv_seen_media')),
sa.UniqueConstraint('source_id', 'filehash', name='uq_pixiv_seen_media_source_id')
)
op.create_index(op.f('ix_pixiv_seen_media_source_id'), 'pixiv_seen_media', ['source_id'], unique=False)
op.create_table('post',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('source_id', sa.Integer(), nullable=True),
sa.Column('artist_id', sa.Integer(), nullable=False),
sa.Column('external_post_id', sa.String(length=128), nullable=False),
sa.Column('post_url', sa.Text(), nullable=True),
sa.Column('post_title', sa.Text(), nullable=True),
sa.Column('post_date', sa.DateTime(timezone=True), nullable=True),
sa.Column('raw_metadata', sa.JSON(), nullable=True),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('attachment_count', sa.Integer(), nullable=True),
sa.Column('post_title_translated', sa.Text(), nullable=True),
sa.Column('description_translated', sa.Text(), nullable=True),
sa.Column('translated_source_lang', sa.String(length=8), nullable=True),
sa.Column('translation_engine_version', sa.String(length=128), nullable=True),
sa.Column('translated_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('translation_override', sa.String(length=16), server_default='auto', nullable=False),
sa.Column('downloaded_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.CheckConstraint("translation_override IN ('auto', 'force', 'original')", name=op.f('ck_post_ck_post_translation_override')),
sa.ForeignKeyConstraint(['artist_id'], ['artist.id'], name=op.f('fk_post_artist_id_artist'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['source_id'], ['source.id'], name=op.f('fk_post_source_id_source'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_post')),
sa.UniqueConstraint('source_id', 'external_post_id', name='uq_post_source_external_id')
)
op.create_index(op.f('ix_post_artist_id'), 'post', ['artist_id'], unique=False)
op.create_index(op.f('ix_post_source_id'), 'post', ['source_id'], unique=False)
op.create_table('subscribestar_failed_media',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('source_id', sa.Integer(), nullable=False),
sa.Column('filehash', sa.String(length=128), nullable=False),
sa.Column('attempts', sa.Integer(), nullable=False),
sa.Column('last_error', sa.Text(), nullable=True),
sa.Column('first_failed_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('last_failed_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['source_id'], ['source.id'], name=op.f('fk_subscribestar_failed_media_source_id_source'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_subscribestar_failed_media')),
sa.UniqueConstraint('source_id', 'filehash', name='uq_subscribestar_failed_media_source_id')
)
op.create_index(op.f('ix_subscribestar_failed_media_source_id'), 'subscribestar_failed_media', ['source_id'], unique=False)
op.create_table('subscribestar_seen_media',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('source_id', sa.Integer(), nullable=False),
sa.Column('filehash', sa.String(length=128), nullable=False),
sa.Column('post_id', sa.String(length=64), nullable=True),
sa.Column('seen_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['source_id'], ['source.id'], name=op.f('fk_subscribestar_seen_media_source_id_source'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_subscribestar_seen_media')),
sa.UniqueConstraint('source_id', 'filehash', name='uq_subscribestar_seen_media_source_id')
)
op.create_index(op.f('ix_subscribestar_seen_media_source_id'), 'subscribestar_seen_media', ['source_id'], unique=False)
op.create_table('download_event',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('source_id', sa.Integer(), nullable=False),
sa.Column('post_id', sa.Integer(), nullable=True),
sa.Column('status', sa.String(length=32), nullable=False),
sa.Column('started_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('finished_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('bytes_downloaded', sa.BigInteger(), nullable=False),
sa.Column('files_count', sa.Integer(), nullable=False),
sa.Column('error', sa.Text(), nullable=True),
sa.Column('metadata', postgresql.JSONB(astext_type=sa.Text()), server_default=sa.text("'{}'::jsonb"), nullable=False),
sa.ForeignKeyConstraint(['post_id'], ['post.id'], name=op.f('fk_download_event_post_id_post'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['source_id'], ['source.id'], name=op.f('fk_download_event_source_id_source'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_download_event'))
)
op.create_index(op.f('ix_download_event_post_id'), 'download_event', ['post_id'], unique=False)
op.create_index(op.f('ix_download_event_source_id'), 'download_event', ['source_id'], unique=False)
op.create_table('image_record',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('path', sa.Text(), nullable=False),
sa.Column('sha256', sa.String(length=64), nullable=False),
sa.Column('phash', sa.String(length=32), nullable=True),
sa.Column('size_bytes', sa.BigInteger(), nullable=False),
sa.Column('mime', sa.String(length=64), nullable=False),
sa.Column('width', sa.Integer(), nullable=True),
sa.Column('height', sa.Integer(), nullable=True),
sa.Column('duration_seconds', sa.Float(), nullable=True),
sa.Column('integrity_status', sa.String(length=24), nullable=False),
sa.Column('thumbnail_path', sa.Text(), nullable=True),
sa.Column('source_url', sa.Text(), nullable=True),
sa.Column('source_filehash', sa.String(length=32), nullable=True),
sa.Column('origin', sa.Enum('downloaded', 'imported_filesystem', 'uploaded', name='origin_enum'), nullable=False),
sa.Column('primary_post_id', sa.Integer(), nullable=True),
sa.Column('artist_id', sa.Integer(), nullable=True),
sa.Column('siglip_embedding', pgvector.sqlalchemy.vector.VECTOR(dim=1152), nullable=True),
sa.Column('siglip_model_version', sa.String(length=128), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('effective_date', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('earliest_post_date', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['artist_id'], ['artist.id'], name=op.f('fk_image_record_artist_id_artist'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['primary_post_id'], ['post.id'], name=op.f('fk_image_record_primary_post_id_post'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_image_record')),
sa.UniqueConstraint('path', name=op.f('uq_image_record_path'))
)
op.create_index(op.f('ix_image_record_artist_id'), 'image_record', ['artist_id'], unique=False)
op.create_index(op.f('ix_image_record_integrity_status'), 'image_record', ['integrity_status'], unique=False)
op.create_index(op.f('ix_image_record_phash'), 'image_record', ['phash'], unique=False)
op.create_index(op.f('ix_image_record_primary_post_id'), 'image_record', ['primary_post_id'], unique=False)
op.create_index(op.f('ix_image_record_sha256'), 'image_record', ['sha256'], unique=True)
op.create_index(op.f('ix_image_record_source_filehash'), 'image_record', ['source_filehash'], unique=False)
op.create_table('post_attachment',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('post_id', sa.Integer(), nullable=True),
sa.Column('artist_id', sa.Integer(), nullable=True),
sa.Column('sha256', sa.String(length=64), nullable=False),
sa.Column('path', sa.Text(), nullable=False),
sa.Column('original_filename', sa.Text(), nullable=False),
sa.Column('ext', sa.String(length=32), nullable=False),
sa.Column('mime', sa.String(length=128), nullable=True),
sa.Column('size_bytes', sa.BigInteger(), nullable=False),
sa.Column('captured_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['artist_id'], ['artist.id'], name=op.f('fk_post_attachment_artist_id_artist'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['post_id'], ['post.id'], name=op.f('fk_post_attachment_post_id_post'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_post_attachment'))
)
op.create_index(op.f('ix_post_attachment_artist_id'), 'post_attachment', ['artist_id'], unique=False)
op.create_index(op.f('ix_post_attachment_post_id'), 'post_attachment', ['post_id'], unique=False)
op.create_index(op.f('ix_post_attachment_sha256'), 'post_attachment', ['sha256'], unique=False)
op.create_index('uq_post_attachment_null_post_sha', 'post_attachment', ['sha256'], unique=True, postgresql_where=sa.text('post_id IS NULL'))
op.create_index('uq_post_attachment_post_sha', 'post_attachment', ['post_id', 'sha256'], unique=True, postgresql_where=sa.text('post_id IS NOT NULL'))
op.create_table('series_suggestion',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('post_id', sa.Integer(), nullable=False),
sa.Column('series_tag_id', sa.Integer(), nullable=False),
sa.Column('score', sa.Float(), nullable=False),
sa.Column('signals', sa.JSON(), nullable=True),
sa.Column('status', sa.String(length=16), server_default='pending', nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['post_id'], ['post.id'], name=op.f('fk_series_suggestion_post_id_post'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['series_tag_id'], ['tag.id'], name=op.f('fk_series_suggestion_series_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_series_suggestion')),
sa.UniqueConstraint('post_id', 'series_tag_id', name='uq_series_suggestion_post_series')
)
op.create_index(op.f('ix_series_suggestion_post_id'), 'series_suggestion', ['post_id'], unique=False)
op.create_index(op.f('ix_series_suggestion_series_tag_id'), 'series_suggestion', ['series_tag_id'], unique=False)
op.create_index(op.f('ix_series_suggestion_status'), 'series_suggestion', ['status'], unique=False)
op.create_table('external_link',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('post_id', sa.Integer(), nullable=False),
sa.Column('artist_id', sa.Integer(), nullable=True),
sa.Column('host', sa.String(length=16), nullable=False),
sa.Column('url', sa.Text(), nullable=False),
sa.Column('label', sa.Text(), nullable=True),
sa.Column('status', sa.String(length=16), server_default='pending', nullable=False),
sa.Column('attempts', sa.Integer(), server_default=sa.text('0'), nullable=False),
sa.Column('last_error', sa.Text(), nullable=True),
sa.Column('attachment_id', sa.Integer(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('completed_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('duration_seconds', sa.Float(), nullable=True),
sa.ForeignKeyConstraint(['artist_id'], ['artist.id'], name=op.f('fk_external_link_artist_id_artist'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['attachment_id'], ['post_attachment.id'], name=op.f('fk_external_link_attachment_id_post_attachment'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['post_id'], ['post.id'], name=op.f('fk_external_link_post_id_post'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_external_link'))
)
op.create_index(op.f('ix_external_link_artist_id'), 'external_link', ['artist_id'], unique=False)
op.create_index(op.f('ix_external_link_post_id'), 'external_link', ['post_id'], unique=False)
op.create_index('ix_external_link_status', 'external_link', ['status'], unique=False)
op.create_index('uq_external_link_post_url', 'external_link', ['post_id', 'url'], unique=True)
op.create_table('gpu_job',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('image_record_id', sa.Integer(), nullable=False),
sa.Column('task', sa.String(length=32), nullable=False),
sa.Column('status', sa.String(length=16), nullable=False),
sa.Column('lease_token', sa.String(length=64), nullable=True),
sa.Column('leased_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('lease_expires_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('attempts', sa.Integer(), nullable=False),
sa.Column('error', sa.Text(), nullable=True),
sa.Column('triage_status', sa.String(length=16), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['image_record_id'], ['image_record.id'], name=op.f('fk_gpu_job_image_record_id_image_record'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_gpu_job'))
)
op.create_index(op.f('ix_gpu_job_image_record_id'), 'gpu_job', ['image_record_id'], unique=False)
op.create_index('ix_gpu_job_leased_expires', 'gpu_job', ['lease_expires_at'], unique=False, postgresql_where=sa.text("status = 'leased'"))
op.create_index('ix_gpu_job_pending', 'gpu_job', ['id'], unique=False, postgresql_where=sa.text("status = 'pending'"))
op.create_index(op.f('ix_gpu_job_status'), 'gpu_job', ['status'], unique=False)
op.create_table('image_provenance',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('image_record_id', sa.Integer(), nullable=False),
sa.Column('post_id', sa.Integer(), nullable=False),
sa.Column('source_id', sa.Integer(), nullable=True),
sa.Column('from_attachment_id', sa.Integer(), nullable=True),
sa.Column('captured_metadata', sa.JSON(), nullable=True),
sa.Column('captured_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['from_attachment_id'], ['post_attachment.id'], name=op.f('fk_image_provenance_from_attachment_id_post_attachment'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['image_record_id'], ['image_record.id'], name=op.f('fk_image_provenance_image_record_id_image_record'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['post_id'], ['post.id'], name=op.f('fk_image_provenance_post_id_post'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['source_id'], ['source.id'], name=op.f('fk_image_provenance_source_id_source'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_image_provenance')),
sa.UniqueConstraint('image_record_id', 'post_id', name='uq_image_provenance_image_post')
)
op.create_index(op.f('ix_image_provenance_from_attachment_id'), 'image_provenance', ['from_attachment_id'], unique=False)
op.create_index(op.f('ix_image_provenance_image_record_id'), 'image_provenance', ['image_record_id'], unique=False)
op.create_index(op.f('ix_image_provenance_post_id'), 'image_provenance', ['post_id'], unique=False)
op.create_index(op.f('ix_image_provenance_source_id'), 'image_provenance', ['source_id'], unique=False)
op.create_table('image_region',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('image_record_id', sa.Integer(), nullable=False),
sa.Column('kind', sa.String(length=16), nullable=False),
sa.Column('frame_time', sa.Float(), nullable=True),
sa.Column('rx', sa.Float(), nullable=False),
sa.Column('ry', sa.Float(), nullable=False),
sa.Column('rw', sa.Float(), nullable=False),
sa.Column('rh', sa.Float(), nullable=False),
sa.Column('score', sa.Float(), nullable=True),
sa.Column('detector_version', sa.String(length=64), nullable=True),
sa.Column('crop_version', sa.String(length=64), nullable=True),
sa.Column('embedding_version', sa.String(length=128), nullable=True),
sa.Column('ccip_embedding', pgvector.sqlalchemy.vector.VECTOR(dim=768), nullable=True),
sa.Column('siglip_embedding', pgvector.sqlalchemy.vector.VECTOR(dim=1152), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['image_record_id'], ['image_record.id'], name=op.f('fk_image_region_image_record_id_image_record'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_image_region'))
)
op.create_index(op.f('ix_image_region_image_record_id'), 'image_region', ['image_record_id'], unique=False)
op.create_table('image_tag',
sa.Column('image_record_id', sa.Integer(), nullable=False),
sa.Column('tag_id', sa.Integer(), nullable=False),
sa.Column('source', sa.String(length=32), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['image_record_id'], ['image_record.id'], name=op.f('fk_image_tag_image_record_id_image_record'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tag_id'], ['tag.id'], name=op.f('fk_image_tag_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('image_record_id', 'tag_id', name=op.f('pk_image_tag'))
)
op.create_table('import_task',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('batch_id', sa.Integer(), nullable=False),
sa.Column('source_path', sa.Text(), nullable=False),
sa.Column('task_type', sa.String(length=16), nullable=False),
sa.Column('status', sa.String(length=16), nullable=False),
sa.Column('recovery_count', sa.Integer(), nullable=False),
sa.Column('refetched', sa.Boolean(), nullable=False),
sa.Column('result_image_id', sa.Integer(), nullable=True),
sa.Column('error', sa.Text(), nullable=True),
sa.Column('size_bytes', sa.BigInteger(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('started_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('finished_at', sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(['batch_id'], ['import_batch.id'], name=op.f('fk_import_task_batch_id_import_batch'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['result_image_id'], ['image_record.id'], name=op.f('fk_import_task_result_image_id_image_record'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_import_task'))
)
op.create_index(op.f('ix_import_task_batch_id'), 'import_task', ['batch_id'], unique=False)
op.create_index(op.f('ix_import_task_status'), 'import_task', ['status'], unique=False)
op.create_table('presentation_review',
sa.Column('image_record_id', sa.Integer(), nullable=False),
sa.Column('tag_id', sa.Integer(), nullable=False),
sa.Column('conflict_tag_id', sa.Integer(), nullable=True),
sa.Column('conflict_score', sa.Float(), nullable=False),
sa.Column('mode', sa.String(length=16), server_default='chrome', nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('resolved_at', sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(['conflict_tag_id'], ['tag.id'], name=op.f('fk_presentation_review_conflict_tag_id_tag'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['image_record_id'], ['image_record.id'], name=op.f('fk_presentation_review_image_record_id_image_record'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tag_id'], ['tag.id'], name=op.f('fk_presentation_review_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('image_record_id', 'tag_id', name=op.f('pk_presentation_review'))
)
op.create_table('series_page',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('series_tag_id', sa.Integer(), nullable=False),
sa.Column('image_id', sa.Integer(), nullable=False),
sa.Column('status', sa.String(length=16), server_default='placed', nullable=False),
sa.Column('page_number', sa.Integer(), nullable=True),
sa.Column('stated_page', sa.Integer(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['image_id'], ['image_record.id'], name=op.f('fk_series_page_image_id_image_record'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['series_tag_id'], ['tag.id'], name=op.f('fk_series_page_series_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_series_page')),
sa.UniqueConstraint('image_id', name=op.f('uq_series_page_image_id'))
)
op.create_index(op.f('ix_series_page_series_tag_id'), 'series_page', ['series_tag_id'], unique=False)
op.create_table('tag_positive_confirmation',
sa.Column('image_record_id', sa.Integer(), nullable=False),
sa.Column('tag_id', sa.Integer(), nullable=False),
sa.Column('confirmed_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['image_record_id'], ['image_record.id'], name=op.f('fk_tag_positive_confirmation_image_record_id_image_record'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tag_id'], ['tag.id'], name=op.f('fk_tag_positive_confirmation_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('image_record_id', 'tag_id', name=op.f('pk_tag_positive_confirmation'))
)
op.create_index(op.f('ix_tag_positive_confirmation_tag_id'), 'tag_positive_confirmation', ['tag_id'], unique=False)
op.create_table('tag_suggestion_rejection',
sa.Column('image_record_id', sa.Integer(), nullable=False),
sa.Column('tag_id', sa.Integer(), nullable=False),
sa.Column('rejected_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['image_record_id'], ['image_record.id'], name=op.f('fk_tag_suggestion_rejection_image_record_id_image_record'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tag_id'], ['tag.id'], name=op.f('fk_tag_suggestion_rejection_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('image_record_id', 'tag_id', name=op.f('pk_tag_suggestion_rejection'))
)
op.create_index(op.f('ix_tag_suggestion_rejection_tag_id'), 'tag_suggestion_rejection', ['tag_id'], unique=False)
op.create_table('character_prototype',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('tag_id', sa.Integer(), nullable=False),
sa.Column('ccip_embedding', pgvector.sqlalchemy.vector.VECTOR(dim=768), nullable=False),
sa.Column('region_id', sa.Integer(), nullable=True),
sa.ForeignKeyConstraint(['region_id'], ['image_region.id'], name=op.f('fk_character_prototype_region_id_image_region'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['tag_id'], ['tag.id'], name=op.f('fk_character_prototype_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_character_prototype'))
)
op.create_index(op.f('ix_character_prototype_tag_id'), 'character_prototype', ['tag_id'], unique=False)
op.create_table('series_chapter',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('series_tag_id', sa.Integer(), nullable=False),
sa.Column('anchor_page_id', sa.Integer(), nullable=False),
sa.Column('title', sa.Text(), nullable=True),
sa.Column('stated_part', sa.Integer(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['anchor_page_id'], ['series_page.id'], name=op.f('fk_series_chapter_anchor_page_id_series_page'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['series_tag_id'], ['tag.id'], name=op.f('fk_series_chapter_series_tag_id_tag'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_series_chapter')),
sa.UniqueConstraint('anchor_page_id', name=op.f('uq_series_chapter_anchor_page_id'))
)
op.create_index(op.f('ix_series_chapter_series_tag_id'), 'series_chapter', ['series_tag_id'], unique=False)
# The HNSW index, item 3 above. Must match the query's cosine-distance
# operator class or the planner will not use it.
op.execute(
"CREATE INDEX ix_image_record_siglip_hnsw "
"ON image_record USING hnsw (siglip_embedding vector_cosine_ops)"
)
def downgrade() -> None:
# Dropping image_record takes its indexes with it, so the HNSW index needs
# no separate drop. The extensions are deliberately left in place: they are
# database-scoped and something else may be using them.
op.drop_index(op.f('ix_series_chapter_series_tag_id'), table_name='series_chapter')
op.drop_table('series_chapter')
op.drop_index(op.f('ix_character_prototype_tag_id'), table_name='character_prototype')
op.drop_table('character_prototype')
op.drop_index(op.f('ix_tag_suggestion_rejection_tag_id'), table_name='tag_suggestion_rejection')
op.drop_table('tag_suggestion_rejection')
op.drop_index(op.f('ix_tag_positive_confirmation_tag_id'), table_name='tag_positive_confirmation')
op.drop_table('tag_positive_confirmation')
op.drop_index(op.f('ix_series_page_series_tag_id'), table_name='series_page')
op.drop_table('series_page')
op.drop_table('presentation_review')
op.drop_index(op.f('ix_import_task_status'), table_name='import_task')
op.drop_index(op.f('ix_import_task_batch_id'), table_name='import_task')
op.drop_table('import_task')
op.drop_table('image_tag')
op.drop_index(op.f('ix_image_region_image_record_id'), table_name='image_region')
op.drop_table('image_region')
op.drop_index(op.f('ix_image_provenance_source_id'), table_name='image_provenance')
op.drop_index(op.f('ix_image_provenance_post_id'), table_name='image_provenance')
op.drop_index(op.f('ix_image_provenance_image_record_id'), table_name='image_provenance')
op.drop_index(op.f('ix_image_provenance_from_attachment_id'), table_name='image_provenance')
op.drop_table('image_provenance')
op.drop_index(op.f('ix_gpu_job_status'), table_name='gpu_job')
op.drop_index('ix_gpu_job_pending', table_name='gpu_job', postgresql_where=sa.text("status = 'pending'"))
op.drop_index('ix_gpu_job_leased_expires', table_name='gpu_job', postgresql_where=sa.text("status = 'leased'"))
op.drop_index(op.f('ix_gpu_job_image_record_id'), table_name='gpu_job')
op.drop_table('gpu_job')
op.drop_index('uq_external_link_post_url', table_name='external_link')
op.drop_index('ix_external_link_status', table_name='external_link')
op.drop_index(op.f('ix_external_link_post_id'), table_name='external_link')
op.drop_index(op.f('ix_external_link_artist_id'), table_name='external_link')
op.drop_table('external_link')
op.drop_index(op.f('ix_series_suggestion_status'), table_name='series_suggestion')
op.drop_index(op.f('ix_series_suggestion_series_tag_id'), table_name='series_suggestion')
op.drop_index(op.f('ix_series_suggestion_post_id'), table_name='series_suggestion')
op.drop_table('series_suggestion')
op.drop_index('uq_post_attachment_post_sha', table_name='post_attachment', postgresql_where=sa.text('post_id IS NOT NULL'))
op.drop_index('uq_post_attachment_null_post_sha', table_name='post_attachment', postgresql_where=sa.text('post_id IS NULL'))
op.drop_index(op.f('ix_post_attachment_sha256'), table_name='post_attachment')
op.drop_index(op.f('ix_post_attachment_post_id'), table_name='post_attachment')
op.drop_index(op.f('ix_post_attachment_artist_id'), table_name='post_attachment')
op.drop_table('post_attachment')
op.drop_index(op.f('ix_image_record_source_filehash'), table_name='image_record')
op.drop_index(op.f('ix_image_record_sha256'), table_name='image_record')
op.drop_index(op.f('ix_image_record_primary_post_id'), table_name='image_record')
op.drop_index(op.f('ix_image_record_phash'), table_name='image_record')
op.drop_index(op.f('ix_image_record_integrity_status'), table_name='image_record')
op.drop_index(op.f('ix_image_record_artist_id'), table_name='image_record')
op.drop_table('image_record')
op.drop_index(op.f('ix_download_event_source_id'), table_name='download_event')
op.drop_index(op.f('ix_download_event_post_id'), table_name='download_event')
op.drop_table('download_event')
op.drop_index(op.f('ix_subscribestar_seen_media_source_id'), table_name='subscribestar_seen_media')
op.drop_table('subscribestar_seen_media')
op.drop_index(op.f('ix_subscribestar_failed_media_source_id'), table_name='subscribestar_failed_media')
op.drop_table('subscribestar_failed_media')
op.drop_index(op.f('ix_post_source_id'), table_name='post')
op.drop_index(op.f('ix_post_artist_id'), table_name='post')
op.drop_table('post')
op.drop_index(op.f('ix_pixiv_seen_media_source_id'), table_name='pixiv_seen_media')
op.drop_table('pixiv_seen_media')
op.drop_index(op.f('ix_pixiv_failed_media_source_id'), table_name='pixiv_failed_media')
op.drop_table('pixiv_failed_media')
op.drop_index(op.f('ix_patreon_seen_media_source_id'), table_name='patreon_seen_media')
op.drop_table('patreon_seen_media')
op.drop_index(op.f('ix_patreon_failed_media_source_id'), table_name='patreon_failed_media')
op.drop_table('patreon_failed_media')
op.drop_table('tag_head')
op.drop_index(op.f('ix_tag_alias_canonical_tag_id'), table_name='tag_alias')
op.drop_table('tag_alias')
op.drop_index(op.f('ix_source_error_type'), table_name='source')
op.drop_index(op.f('ix_source_artist_id'), table_name='source')
op.drop_table('source')
op.drop_index(op.f('ix_head_metrics_snapshot_tag_id'), table_name='head_metrics_snapshot')
op.drop_index(op.f('ix_head_metrics_snapshot_snapshot_at'), table_name='head_metrics_snapshot')
op.drop_table('head_metrics_snapshot')
op.drop_table('head_metric')
op.drop_table('ccip_prototype_state')
op.drop_table('artist_visit')
op.drop_index(op.f('ix_task_run_task_name'), table_name='task_run')
op.drop_index(op.f('ix_task_run_status'), table_name='task_run')
op.drop_index(op.f('ix_task_run_started_at'), table_name='task_run')
op.drop_index(op.f('ix_task_run_queue'), table_name='task_run')
op.drop_index(op.f('ix_task_run_finished_at'), table_name='task_run')
op.drop_index(op.f('ix_task_run_celery_task_id'), table_name='task_run')
op.drop_table('task_run')
op.drop_index(op.f('ix_tag_fandom_id'), table_name='tag')
op.drop_table('tag')
op.drop_table('ml_settings')
op.drop_index(op.f('ix_library_audit_run_status'), table_name='library_audit_run')
op.drop_index(op.f('ix_library_audit_run_rule'), table_name='library_audit_run')
op.drop_table('library_audit_run')
op.drop_table('import_settings')
op.drop_index(op.f('ix_import_batch_status'), table_name='import_batch')
op.drop_table('import_batch')
op.drop_index(op.f('ix_head_training_run_status'), table_name='head_training_run')
op.drop_table('head_training_run')
op.drop_index(op.f('ix_head_auto_apply_run_status'), table_name='head_auto_apply_run')
op.drop_table('head_auto_apply_run')
op.drop_table('credential')
op.drop_index(op.f('ix_backup_run_tag'), table_name='backup_run')
op.drop_index(op.f('ix_backup_run_status'), table_name='backup_run')
op.drop_index(op.f('ix_backup_run_started_at'), table_name='backup_run')
op.drop_index(op.f('ix_backup_run_kind'), table_name='backup_run')
op.drop_index(op.f('ix_backup_run_finished_at'), table_name='backup_run')
op.drop_table('backup_run')
op.drop_table('artist')
op.drop_table('app_setting')
Reference in New Issue View Git Blame Copy Permalink