bvandeusenandClaude Opus 5
08418d54a3
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 11s
Build images / build-ml (push) Successful in 32s
Build images / build-web (push) Successful in 26s
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m44s
extension / lint (pull_request) Successful in 24s
A structural sweep of the deployed schema, run AFTER 0088 got the models and the chain to exact agreement. That agreement is what 0088 achieved, and it is worth naming what it does not prove: a models-vs-chain diff shows the two describe the same schema, not that the schema is right. Everything here was wrong in BOTH. The one that matters: image_tag has PRIMARY KEY (image_record_id, tag_id) and no other index, so tag_id is unindexed. That is the gallery's tag filter (tag_query.py builds `image_tag.c.tag_id == tid`) and the ON DELETE CASCADE from tag, both scanning the largest table in the schema. Six more FKs were unindexed on smaller tables; presentation_review.tag_id also CASCADEs. Dropped, on the other side: ix_image_record_sha256 was an exact duplicate of the index uq_image_record_sha256 already builds — two btrees on the same column of the highest-insert-rate table. The other six are single-column indexes a later composite superseded without the narrow one being retired; a btree on (a,b) already serves lookups on a. 0088 deliberately taught the models to declare BOTH sha256 indexes so they would describe reality. This changes the reality instead, and the models change with it — otherwise the next baseline.yml run reintroduces exactly the drift 0088 removed. CONCURRENTLY throughout, so building the image_tag index does not hold an ACCESS EXCLUSIVE lock over every write for the duration. The cost is that the migration cannot run in a transaction and so is not atomic: every statement is IF NOT EXISTS / IF EXISTS, making a re-run after a partial failure safe. The docstring carries the query for finding an INVALID index left by an interrupted CONCURRENTLY build. What the sweep found clean, for the record: all 43 tables have a primary key; all 51 FKs declare an explicit ON DELETE, so none silently blocks a delete; the three enum CHECKs match the code that writes them (rule 36). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw
93 lines
3.7 KiB
Python
93 lines
3.7 KiB
Python
"""ExternalLink — an off-platform file-host link found in a post body.
|
|
|
|
Creators host the actual files (films, packs) on mega.nz / Google Drive /
|
|
MediaFire / Dropbox / Pixeldrain and drop the link in the post text. This row
|
|
is the record that the link existed (so nothing is silently dropped), the
|
|
dedup + dead-letter ledger for fetching it, and the driver the download worker
|
|
walks. `url` keeps the FULL link including the `#fragment` (mega's decryption
|
|
key) — truncating it makes the file undownloadable.
|
|
|
|
status lifecycle: pending → downloading → downloaded | failed | dead
|
|
(too many attempts) | skipped (host disabled). `attachment_id` links the
|
|
captured file once a download lands (SET NULL so deleting the attachment
|
|
doesn't delete the link record).
|
|
"""
|
|
|
|
from datetime import datetime
|
|
|
|
from sqlalchemy import (
|
|
CheckConstraint,
|
|
DateTime,
|
|
Float,
|
|
ForeignKey,
|
|
Index,
|
|
Integer,
|
|
String,
|
|
Text,
|
|
func,
|
|
text,
|
|
)
|
|
from sqlalchemy.orm import Mapped, mapped_column
|
|
|
|
from .base import Base
|
|
|
|
# Kept in sync with link_extract.SUPPORTED_HOSTS and the CHECK in migration 0049.
|
|
HOSTS = ("mega", "gdrive", "mediafire", "dropbox", "pixeldrain")
|
|
STATUSES = ("pending", "downloading", "downloaded", "failed", "skipped", "dead")
|
|
|
|
|
|
class ExternalLink(Base):
|
|
__tablename__ = "external_link"
|
|
__table_args__ = (
|
|
# alembic 0028 enum CHECKs. Rule 36 territory: a new host or status value
|
|
# needs its constraint swapped in the same migration (#3275).
|
|
CheckConstraint(
|
|
"host IN ('mega', 'gdrive', 'mediafire', 'dropbox', 'pixeldrain')",
|
|
# Bare name: Base.metadata's naming convention prepends
|
|
# ck_<table>_. Pre-prefixing it here doubles the prefix — see
|
|
# alembic 0088, which renames the four constraints that shipped
|
|
# that way (#3275).
|
|
name="host",
|
|
),
|
|
CheckConstraint(
|
|
"status IN ('pending', 'downloading', 'downloaded', 'failed', 'skipped', 'dead')",
|
|
name="status",
|
|
),
|
|
# One row per (post, url). The full url (incl. #fragment) is the identity
|
|
# — the same file linked twice in a post collapses to one row.
|
|
Index("uq_external_link_post_url", "post_id", "url", unique=True),
|
|
Index("ix_external_link_status", "status"),
|
|
# Unindexed FK (#3300).
|
|
Index("ix_external_link_attachment_id", "attachment_id"),
|
|
)
|
|
|
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
|
# No index=True: uq_external_link_post_url (post_id, url) already leads
|
|
# with post_id (#3301).
|
|
post_id: Mapped[int] = mapped_column(
|
|
ForeignKey("post.id", ondelete="CASCADE"), nullable=False
|
|
)
|
|
artist_id: Mapped[int | None] = mapped_column(
|
|
ForeignKey("artist.id", ondelete="SET NULL"), nullable=True, index=True
|
|
)
|
|
host: Mapped[str] = mapped_column(String(16), nullable=False)
|
|
url: Mapped[str] = mapped_column(Text, nullable=False)
|
|
label: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
status: Mapped[str] = mapped_column(
|
|
String(16), nullable=False, server_default="pending"
|
|
)
|
|
attempts: Mapped[int] = mapped_column(
|
|
Integer, nullable=False, server_default=text("0")
|
|
)
|
|
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
attachment_id: Mapped[int | None] = mapped_column(
|
|
ForeignKey("post_attachment.id", ondelete="SET NULL"), nullable=True
|
|
)
|
|
created_at: Mapped[datetime] = mapped_column(
|
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
|
)
|
|
completed_at: Mapped[datetime | None] = mapped_column(
|
|
DateTime(timezone=True), nullable=True
|
|
)
|
|
duration_seconds: Mapped[float | None] = mapped_column(Float, nullable=True)
|