db: reconcile the models with the deployed schema (#3275)
Build images / sign-extension (push) Successful in 4s
CI / lint (push) Failing after 2s
CI / extension-version (push) Successful in 2s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 27s
Build images / build-ml (push) Successful in 48s
CI / backend-lint-and-test (push) Successful in 1m7s
Build images / build-web (push) Successful in 40s
CI / integration (push) Successful in 4m1s
Build images / sign-extension (push) Successful in 4s
CI / lint (push) Failing after 2s
CI / extension-version (push) Successful in 2s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 27s
Build images / build-ml (push) Successful in 48s
CI / backend-lint-and-test (push) Successful in 1m7s
Build images / build-web (push) Successful in 40s
CI / integration (push) Successful in 4m1s
Milestone 328's acceptance test compared a database built by the real
0001..0087 chain against one built from the models, and found ~130
places where they disagree. This closes them.
Almost all were the MODEL being wrong, so almost all of this is model
edits with no DDL — the database already had these things, nothing in it
changes, and no deploy is needed for this part:
* 92 columns gained server_default. The models carried Python-side
`default=` only, so the ORM filled the value and the column had no
database default. Anything inserting outside the ORM behaved
differently from production.
* Eleven indexes that existed only in migrations are now declared:
the three backup_run reporting indexes, the two date-ordered
image_record browse indexes, import_task and presentation_review,
and the three task_run history indexes. All use text() for their DESC
ordering and postgresql_where for the partial one.
* Two UNIQUE indexes that autogenerate silently proposed DROPPING,
because neither is expressible as a UniqueConstraint:
uq_tag_name_kind_fandom — an EXPRESSION index over
(name, kind, COALESCE(fandom_id, 0))
uq_post_artist_external_id_null_source — PARTIAL, WHERE source_id
IS NULL
post.py already had a comment describing the second one. The comment
was right; nothing declared it.
* The two external_link enum CHECKs (host, status) — rule 36 territory,
and absent from the model entirely.
* Two indexes were named explicitly. A bare index=True generated
ix_tag_alias_canonical_tag_id where the database has
ix_tag_alias_canonical, so autogenerate proposed a drop+create of an
index that was already there under another name. Same for
tag_suggestion_rejection.
Only ONE thing needed DDL, as 0088: tag.fandom_id is declared
index=True but no migration ever created that index.
Deliberately NOT here: image_record.sha256. The model says unique=True;
0001 created a plain index. Duplicates are possible today and the ORM
believes otherwise. The fix depends on whether duplicates already exist
— if they do, that is a dedupe decision, not a constraint — so it waits
on an answer about live data.
The real severity of #3275 is not the squash. It is that --autogenerate
has been unsafe on this project: run against the old models it would
have proposed dropping eleven indexes and two uniqueness guarantees.
This commit is contained in:
@@ -0,0 +1,48 @@
|
|||||||
|
"""Reconcile the database with what the models have always claimed (#3275).
|
||||||
|
|
||||||
|
Milestone 328 discovered ~130 places where the ORM models and the deployed
|
||||||
|
schema disagreed. Almost all of them were the MODEL being wrong — missing
|
||||||
|
`server_default`s, indexes and CHECK constraints that only ever existed in a
|
||||||
|
migration — and those are fixed in the model files with no DDL at all, because
|
||||||
|
the database already had them.
|
||||||
|
|
||||||
|
This migration carries the remainder: the one case where the MODEL was right
|
||||||
|
and the database was missing something.
|
||||||
|
|
||||||
|
`tag.fandom_id` is declared `index=True` on the model, but no migration ever
|
||||||
|
created that index. Every autogenerate run since would have proposed adding
|
||||||
|
it; nobody ran one, so the model and the database simply drifted apart and
|
||||||
|
stayed that way.
|
||||||
|
|
||||||
|
Deliberately NOT in this migration: making `image_record.sha256` unique. The
|
||||||
|
model says `unique=True` and `0001` created a plain, non-unique index, so
|
||||||
|
duplicates are possible today and the ORM believes they are not. Adding the
|
||||||
|
constraint is a real change that FAILS if duplicates already exist, and if
|
||||||
|
they do exist the right response is a dedupe decision rather than a constraint
|
||||||
|
— so it needs an answer about live data before it is written, not after.
|
||||||
|
Tracked in #3275.
|
||||||
|
|
||||||
|
Revision ID: 0088
|
||||||
|
Revises: 0087
|
||||||
|
Create Date: 2026-08-30
|
||||||
|
|
||||||
|
"""
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
revision: str = "0088"
|
||||||
|
down_revision: Union[str, None] = "0087"
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
# IF NOT EXISTS because the index is what the model already asks for: any
|
||||||
|
# database built from metadata rather than from this chain will have it,
|
||||||
|
# and this migration must be a no-op there rather than an error.
|
||||||
|
op.execute("CREATE INDEX IF NOT EXISTS ix_tag_fandom_id ON tag (fandom_id)")
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.execute("DROP INDEX IF EXISTS ix_tag_fandom_id")
|
||||||
@@ -27,10 +27,10 @@ class Artist(Base):
|
|||||||
notes: Mapped[str | None] = mapped_column(Text, nullable=True)
|
notes: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||||
|
|
||||||
# True once a Source is attached; flips false if all sources removed.
|
# True once a Source is attached; flips false if all sources removed.
|
||||||
is_subscription: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
is_subscription: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
|
||||||
|
|
||||||
# Per-artist scheduling overrides; null means "use global default".
|
# Per-artist scheduling overrides; null means "use global default".
|
||||||
auto_check: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
|
auto_check: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
|
||||||
check_interval_seconds: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
check_interval_seconds: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||||
|
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ feedback_check_existing_enums):
|
|||||||
|
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import JSON, BigInteger, DateTime, ForeignKey, Integer, String, Text
|
from sqlalchemy import BigInteger, DateTime, ForeignKey, Index, Integer, JSON, String, Text, text
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from .base import Base
|
from .base import Base
|
||||||
@@ -29,10 +29,18 @@ from .base import Base
|
|||||||
class BackupRun(Base):
|
class BackupRun(Base):
|
||||||
__tablename__ = "backup_run"
|
__tablename__ = "backup_run"
|
||||||
|
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
# alembic 0017: reporting indexes, never declared on the model (#3275).
|
||||||
|
Index("ix_backup_run_kind_started", "kind", text("started_at DESC")),
|
||||||
|
Index("ix_backup_run_status_finished", "status", text("finished_at DESC")),
|
||||||
|
Index("ix_backup_run_tag_partial", "tag", postgresql_where=text("tag IS NOT NULL")),
|
||||||
|
)
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||||
kind: Mapped[str] = mapped_column(String(16), nullable=False, index=True)
|
kind: Mapped[str] = mapped_column(String(16), nullable=False, index=True)
|
||||||
status: Mapped[str] = mapped_column(
|
status: Mapped[str] = mapped_column(
|
||||||
String(16), nullable=False, default="pending", index=True,
|
String(16), nullable=False, default="pending", index=True,
|
||||||
|
server_default="pending",
|
||||||
)
|
)
|
||||||
tag: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
|
tag: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
|
||||||
triggered_by: Mapped[str] = mapped_column(String(32), nullable=False)
|
triggered_by: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
|||||||
@@ -25,8 +25,8 @@ class DownloadEvent(Base):
|
|||||||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
)
|
)
|
||||||
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||||
bytes_downloaded: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0)
|
bytes_downloaded: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0, server_default="0")
|
||||||
files_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
files_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||||
metadata_: Mapped[dict] = mapped_column(
|
metadata_: Mapped[dict] = mapped_column(
|
||||||
"metadata", JSONB, nullable=False, default=dict,
|
"metadata", JSONB, nullable=False, default=dict,
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ doesn't delete the link record).
|
|||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import (
|
from sqlalchemy import (
|
||||||
|
CheckConstraint,
|
||||||
DateTime,
|
DateTime,
|
||||||
Float,
|
Float,
|
||||||
ForeignKey,
|
ForeignKey,
|
||||||
@@ -38,6 +39,16 @@ STATUSES = ("pending", "downloading", "downloaded", "failed", "skipped", "dead")
|
|||||||
class ExternalLink(Base):
|
class ExternalLink(Base):
|
||||||
__tablename__ = "external_link"
|
__tablename__ = "external_link"
|
||||||
__table_args__ = (
|
__table_args__ = (
|
||||||
|
# alembic 0028 enum CHECKs. Rule 36 territory: a new host or status value
|
||||||
|
# needs its constraint swapped in the same migration (#3275).
|
||||||
|
CheckConstraint(
|
||||||
|
"host IN ('mega', 'gdrive', 'mediafire', 'dropbox', 'pixeldrain')",
|
||||||
|
name="ck_external_link_host",
|
||||||
|
),
|
||||||
|
CheckConstraint(
|
||||||
|
"status IN ('pending', 'downloading', 'downloaded', 'failed', 'skipped', 'dead')",
|
||||||
|
name="ck_external_link_status",
|
||||||
|
),
|
||||||
# One row per (post, url). The full url (incl. #fragment) is the identity
|
# One row per (post, url). The full url (incl. #fragment) is the identity
|
||||||
# — the same file linked twice in a post collapses to one row.
|
# — the same file linked twice in a post collapses to one row.
|
||||||
Index("uq_external_link_post_url", "post_id", "url", unique=True),
|
Index("uq_external_link_post_url", "post_id", "url", unique=True),
|
||||||
|
|||||||
@@ -50,7 +50,8 @@ class GpuJob(Base):
|
|||||||
# What to compute, e.g. 'ccip' (detect figures + CCIP-embed) or 'siglip_region'.
|
# What to compute, e.g. 'ccip' (detect figures + CCIP-embed) or 'siglip_region'.
|
||||||
task: Mapped[str] = mapped_column(String(32), nullable=False)
|
task: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
status: Mapped[str] = mapped_column(
|
status: Mapped[str] = mapped_column(
|
||||||
String(16), nullable=False, default="pending", index=True
|
String(16), nullable=False, default="pending", index=True,
|
||||||
|
server_default="pending",
|
||||||
)
|
)
|
||||||
# pending | leased | done | error
|
# pending | leased | done | error
|
||||||
lease_token: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
lease_token: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||||
@@ -60,7 +61,7 @@ class GpuJob(Base):
|
|||||||
lease_expires_at: Mapped[datetime | None] = mapped_column(
|
lease_expires_at: Mapped[datetime | None] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=True
|
DateTime(timezone=True), nullable=True
|
||||||
)
|
)
|
||||||
attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||||
# Triage verdict for an ERRORED job (#125): NULL = not yet probed;
|
# Triage verdict for an ERRORED job (#125): NULL = not yet probed;
|
||||||
# 'defect' = the integrity probe says the FILE itself is bad (surfaced for
|
# 'defect' = the integrity probe says the FILE itself is bad (surfaced for
|
||||||
|
|||||||
@@ -24,10 +24,11 @@ class HeadAutoApplyRun(Base):
|
|||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||||
# dry_run=True is a PREVIEW: scores + counts what WOULD apply, writes nothing
|
# dry_run=True is a PREVIEW: scores + counts what WOULD apply, writes nothing
|
||||||
# (preview/apply parity, rule 93).
|
# (preview/apply parity, rule 93).
|
||||||
dry_run: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
dry_run: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
|
||||||
params: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
|
params: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
|
||||||
status: Mapped[str] = mapped_column(
|
status: Mapped[str] = mapped_column(
|
||||||
String(16), nullable=False, default="running", index=True
|
String(16), nullable=False, default="running", index=True,
|
||||||
|
server_default="running",
|
||||||
)
|
)
|
||||||
# running | ready | error
|
# running | ready | error
|
||||||
started_at: Mapped[datetime] = mapped_column(
|
started_at: Mapped[datetime] = mapped_column(
|
||||||
|
|||||||
@@ -24,9 +24,9 @@ class HeadMetric(Base):
|
|||||||
ForeignKey("tag.id", ondelete="CASCADE"), primary_key=True
|
ForeignKey("tag.id", ondelete="CASCADE"), primary_key=True
|
||||||
)
|
)
|
||||||
# An auto-applied (source='head_auto') tag the operator later REMOVED.
|
# An auto-applied (source='head_auto') tag the operator later REMOVED.
|
||||||
n_misfires: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
n_misfires: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
# A tag with a head that the operator added by HAND (the head missed it).
|
# A tag with a head that the operator added by HAND (the head missed it).
|
||||||
n_underfires: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
n_underfires: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
updated_at: Mapped[datetime] = mapped_column(
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -28,9 +28,9 @@ class HeadMetricsSnapshot(Base):
|
|||||||
DateTime(timezone=True), nullable=False, server_default=func.now(), index=True
|
DateTime(timezone=True), nullable=False, server_default=func.now(), index=True
|
||||||
)
|
)
|
||||||
# Current count of source='head_auto' applications still standing.
|
# Current count of source='head_auto' applications still standing.
|
||||||
n_auto_applied: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
n_auto_applied: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
n_misfires: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
n_misfires: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
n_underfires: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
n_underfires: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
# The head's measured quality at snapshot time (null if no head exists).
|
# The head's measured quality at snapshot time (null if no head exists).
|
||||||
ap: Mapped[float | None] = mapped_column(Float, nullable=True)
|
ap: Mapped[float | None] = mapped_column(Float, nullable=True)
|
||||||
precision_cv: Mapped[float | None] = mapped_column(Float, nullable=True)
|
precision_cv: Mapped[float | None] = mapped_column(Float, nullable=True)
|
||||||
|
|||||||
@@ -24,7 +24,8 @@ class HeadTrainingRun(Base):
|
|||||||
# Training parameters: {min_positives, neg_ratio, precision_target, ...}.
|
# Training parameters: {min_positives, neg_ratio, precision_target, ...}.
|
||||||
params: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
|
params: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
|
||||||
status: Mapped[str] = mapped_column(
|
status: Mapped[str] = mapped_column(
|
||||||
String(16), nullable=False, default="running", index=True
|
String(16), nullable=False, default="running", index=True,
|
||||||
|
server_default="running",
|
||||||
)
|
)
|
||||||
# running | ready | error
|
# running | ready | error
|
||||||
started_at: Mapped[datetime] = mapped_column(
|
started_at: Mapped[datetime] = mapped_column(
|
||||||
|
|||||||
@@ -14,10 +14,12 @@ from sqlalchemy import (
|
|||||||
Enum,
|
Enum,
|
||||||
Float,
|
Float,
|
||||||
ForeignKey,
|
ForeignKey,
|
||||||
|
Index,
|
||||||
Integer,
|
Integer,
|
||||||
String,
|
String,
|
||||||
Text,
|
Text,
|
||||||
func,
|
func,
|
||||||
|
text,
|
||||||
)
|
)
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
@@ -29,6 +31,12 @@ ORIGIN_CHOICES = ("downloaded", "imported_filesystem", "uploaded")
|
|||||||
class ImageRecord(Base):
|
class ImageRecord(Base):
|
||||||
__tablename__ = "image_record"
|
__tablename__ = "image_record"
|
||||||
|
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
# alembic 0035/0071: the date-ordered browse indexes (#3275).
|
||||||
|
Index("ix_image_record_effective_date", text("effective_date DESC"), text("id DESC")),
|
||||||
|
Index("ix_image_record_earliest_post_date", text("earliest_post_date DESC"), text("id DESC")),
|
||||||
|
)
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||||
|
|
||||||
# On-disk identity
|
# On-disk identity
|
||||||
@@ -47,7 +55,8 @@ class ImageRecord(Base):
|
|||||||
# Integrity verification status. FC-2e populates this; FC-2a leaves rows at 'unknown'.
|
# Integrity verification status. FC-2e populates this; FC-2a leaves rows at 'unknown'.
|
||||||
# Values: 'unknown' (default), 'ok', 'corrupt', 'failed_verification'.
|
# Values: 'unknown' (default), 'ok', 'corrupt', 'failed_verification'.
|
||||||
integrity_status: Mapped[str] = mapped_column(
|
integrity_status: Mapped[str] = mapped_column(
|
||||||
String(24), nullable=False, default="unknown", index=True
|
String(24), nullable=False, default="unknown", index=True,
|
||||||
|
server_default="unknown",
|
||||||
)
|
)
|
||||||
|
|
||||||
# Thumbnail (populated by FC-2)
|
# Thumbnail (populated by FC-2)
|
||||||
|
|||||||
@@ -21,17 +21,17 @@ class ImportBatch(Base):
|
|||||||
)
|
)
|
||||||
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||||
|
|
||||||
total_files: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
total_files: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
imported: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
imported: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
skipped: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
skipped: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
failed: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
failed: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
attachments: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
attachments: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
# Deep-scan only: count of already-imported files whose sidecar metadata
|
# Deep-scan only: count of already-imported files whose sidecar metadata
|
||||||
# got re-applied this run (post/source/provenance upsert). Stays 0 on
|
# got re-applied this run (post/source/provenance upsert). Stays 0 on
|
||||||
# quick-scan batches. See `Importer.import_one(deep_scan=True)`.
|
# quick-scan batches. See `Importer.import_one(deep_scan=True)`.
|
||||||
refreshed: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
refreshed: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
|
|
||||||
status: Mapped[str] = mapped_column(String(16), nullable=False, default="running", index=True)
|
status: Mapped[str] = mapped_column(String(16), nullable=False, default="running", index=True, server_default="running")
|
||||||
# running | complete | cancelled
|
# running | complete | cancelled
|
||||||
|
|
||||||
tasks = relationship("ImportTask", back_populates="batch", cascade="all, delete-orphan")
|
tasks = relationship("ImportTask", back_populates="batch", cascade="all, delete-orphan")
|
||||||
|
|||||||
@@ -17,60 +17,71 @@ class ImportSettings(Base):
|
|||||||
__table_args__ = (CheckConstraint("id = 1", name="singleton"),)
|
__table_args__ = (CheckConstraint("id = 1", name="singleton"),)
|
||||||
|
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||||
import_scan_path: Mapped[str] = mapped_column(Text, nullable=False, default="/import")
|
import_scan_path: Mapped[str] = mapped_column(Text, nullable=False, default="/import", server_default="/import")
|
||||||
|
|
||||||
min_width: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
min_width: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
min_height: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
min_height: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
|
|
||||||
skip_transparent: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
skip_transparent: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
|
||||||
transparency_threshold: Mapped[float] = mapped_column(Float, nullable=False, default=0.9)
|
transparency_threshold: Mapped[float] = mapped_column(Float, nullable=False, default=0.9, server_default="0.9")
|
||||||
|
|
||||||
skip_single_color: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
skip_single_color: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
|
||||||
single_color_threshold: Mapped[float] = mapped_column(Float, nullable=False, default=0.95)
|
single_color_threshold: Mapped[float] = mapped_column(Float, nullable=False, default=0.95, server_default="0.95")
|
||||||
single_color_tolerance: Mapped[int] = mapped_column(Integer, nullable=False, default=30)
|
single_color_tolerance: Mapped[int] = mapped_column(Integer, nullable=False, default=30, server_default="30")
|
||||||
|
|
||||||
phash_threshold: Mapped[int] = mapped_column(Integer, nullable=False, default=10)
|
phash_threshold: Mapped[int] = mapped_column(Integer, nullable=False, default=10, server_default="10")
|
||||||
|
|
||||||
# FC-3c downloader knobs
|
# FC-3c downloader knobs
|
||||||
download_rate_limit_seconds: Mapped[float] = mapped_column(
|
download_rate_limit_seconds: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=3.0
|
Float, nullable=False, default=3.0,
|
||||||
|
server_default="3",
|
||||||
)
|
)
|
||||||
download_validate_files: Mapped[bool] = mapped_column(
|
download_validate_files: Mapped[bool] = mapped_column(
|
||||||
Boolean, nullable=False, default=True
|
Boolean, nullable=False, default=True,
|
||||||
|
server_default="true",
|
||||||
)
|
)
|
||||||
|
|
||||||
# FC-3d scheduling knobs
|
# FC-3d scheduling knobs
|
||||||
download_schedule_default_seconds: Mapped[int] = mapped_column(
|
download_schedule_default_seconds: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=28800
|
Integer, nullable=False, default=28800,
|
||||||
|
server_default="28800",
|
||||||
)
|
)
|
||||||
download_event_retention_days: Mapped[int] = mapped_column(
|
download_event_retention_days: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=90
|
Integer, nullable=False, default=90,
|
||||||
|
server_default="90",
|
||||||
)
|
)
|
||||||
download_failure_warning_threshold: Mapped[int] = mapped_column(
|
download_failure_warning_threshold: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=5
|
Integer, nullable=False, default=5,
|
||||||
|
server_default="5",
|
||||||
)
|
)
|
||||||
|
|
||||||
# FC-3h backup knobs.
|
# FC-3h backup knobs.
|
||||||
backup_db_nightly_enabled: Mapped[bool] = mapped_column(
|
backup_db_nightly_enabled: Mapped[bool] = mapped_column(
|
||||||
Boolean, nullable=False, default=False,
|
Boolean, nullable=False, default=False,
|
||||||
|
server_default="false",
|
||||||
)
|
)
|
||||||
backup_db_nightly_hour_utc: Mapped[int] = mapped_column(
|
backup_db_nightly_hour_utc: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=3,
|
Integer, nullable=False, default=3,
|
||||||
|
server_default="3",
|
||||||
)
|
)
|
||||||
backup_db_keep_last_n: Mapped[int] = mapped_column(
|
backup_db_keep_last_n: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=14,
|
Integer, nullable=False, default=14,
|
||||||
|
server_default="14",
|
||||||
)
|
)
|
||||||
backup_images_keep_last_n: Mapped[int] = mapped_column(
|
backup_images_keep_last_n: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=3,
|
Integer, nullable=False, default=3,
|
||||||
|
server_default="3",
|
||||||
)
|
)
|
||||||
|
|
||||||
# FC-6.3 series continuation matcher. enabled gates the rescan; threshold is
|
# FC-6.3 series continuation matcher. enabled gates the rescan; threshold is
|
||||||
# the weighted-score cut-off (0..1) above which a pending suggestion is made.
|
# the weighted-score cut-off (0..1) above which a pending suggestion is made.
|
||||||
series_suggest_enabled: Mapped[bool] = mapped_column(
|
series_suggest_enabled: Mapped[bool] = mapped_column(
|
||||||
Boolean, nullable=False, default=True,
|
Boolean, nullable=False, default=True,
|
||||||
|
server_default="true",
|
||||||
)
|
)
|
||||||
series_suggest_threshold: Mapped[float] = mapped_column(
|
series_suggest_threshold: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=0.5,
|
Float, nullable=False, default=0.5,
|
||||||
|
server_default="0.5",
|
||||||
)
|
)
|
||||||
|
|
||||||
# #830 off-platform file-host downloads — per-host enable lever (default on,
|
# #830 off-platform file-host downloads — per-host enable lever (default on,
|
||||||
|
|||||||
@@ -13,10 +13,12 @@ from sqlalchemy import (
|
|||||||
Boolean,
|
Boolean,
|
||||||
DateTime,
|
DateTime,
|
||||||
ForeignKey,
|
ForeignKey,
|
||||||
|
Index,
|
||||||
Integer,
|
Integer,
|
||||||
String,
|
String,
|
||||||
Text,
|
Text,
|
||||||
func,
|
func,
|
||||||
|
text,
|
||||||
)
|
)
|
||||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||||
|
|
||||||
@@ -26,6 +28,10 @@ from .base import Base
|
|||||||
class ImportTask(Base):
|
class ImportTask(Base):
|
||||||
__tablename__ = "import_task"
|
__tablename__ = "import_task"
|
||||||
|
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
Index("ix_import_task_created_at_desc", text("created_at DESC")),
|
||||||
|
)
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||||
batch_id: Mapped[int] = mapped_column(
|
batch_id: Mapped[int] = mapped_column(
|
||||||
ForeignKey("import_batch.id", ondelete="CASCADE"), nullable=False, index=True
|
ForeignKey("import_batch.id", ondelete="CASCADE"), nullable=False, index=True
|
||||||
@@ -33,14 +39,14 @@ class ImportTask(Base):
|
|||||||
|
|
||||||
source_path: Mapped[str] = mapped_column(Text, nullable=False)
|
source_path: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
task_type: Mapped[str] = mapped_column(String(16), nullable=False) # media|archive
|
task_type: Mapped[str] = mapped_column(String(16), nullable=False) # media|archive
|
||||||
status: Mapped[str] = mapped_column(String(16), nullable=False, default="pending", index=True)
|
status: Mapped[str] = mapped_column(String(16), nullable=False, default="pending", index=True, server_default="pending")
|
||||||
|
|
||||||
# Poison-pill circuit breaker (alembic 0026). recovery_count tracks
|
# Poison-pill circuit breaker (alembic 0026). recovery_count tracks
|
||||||
# how many times the stuck-task sweep has re-queued this row; after
|
# how many times the stuck-task sweep has re-queued this row; after
|
||||||
# the cap it's failed with a diagnostic instead of looping. refetched
|
# the cap it's failed with a diagnostic instead of looping. refetched
|
||||||
# bounds the one-shot re-download remediation to a single attempt.
|
# bounds the one-shot re-download remediation to a single attempt.
|
||||||
recovery_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
recovery_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
refetched: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
refetched: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
|
||||||
|
|
||||||
result_image_id: Mapped[int | None] = mapped_column(
|
result_image_id: Mapped[int | None] = mapped_column(
|
||||||
ForeignKey("image_record.id", ondelete="SET NULL"), nullable=True
|
ForeignKey("image_record.id", ondelete="SET NULL"), nullable=True
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ reads it and routes through cleanup_service.delete_images.
|
|||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy import DateTime, Integer, String, Text, func
|
from sqlalchemy import DateTime, Integer, String, Text, func, text
|
||||||
from sqlalchemy.dialects.postgresql import JSONB
|
from sqlalchemy.dialects.postgresql import JSONB
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
@@ -23,6 +23,7 @@ class LibraryAuditRun(Base):
|
|||||||
params: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
|
params: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
|
||||||
status: Mapped[str] = mapped_column(
|
status: Mapped[str] = mapped_column(
|
||||||
String(16), nullable=False, default="running", index=True,
|
String(16), nullable=False, default="running", index=True,
|
||||||
|
server_default="running",
|
||||||
)
|
)
|
||||||
# running | ready | applied | cancelled | error
|
# running | ready | applied | cancelled | error
|
||||||
started_at: Mapped[datetime] = mapped_column(
|
started_at: Mapped[datetime] = mapped_column(
|
||||||
@@ -31,14 +32,16 @@ class LibraryAuditRun(Base):
|
|||||||
finished_at: Mapped[datetime | None] = mapped_column(
|
finished_at: Mapped[datetime | None] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=True,
|
DateTime(timezone=True), nullable=True,
|
||||||
)
|
)
|
||||||
scanned_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
scanned_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
matched_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
matched_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
matched_ids: Mapped[list[int]] = mapped_column(JSONB, nullable=False, default=list)
|
matched_ids: Mapped[list[int]] = mapped_column(
|
||||||
|
JSONB, nullable=False, default=list, server_default=text("'[]'::jsonb")
|
||||||
|
)
|
||||||
error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||||
# Chunked-scan state (alembic 0039): keyset cursor the next chunk resumes
|
# Chunked-scan state (alembic 0039): keyset cursor the next chunk resumes
|
||||||
# from, and the last time a chunk made progress (so the recovery sweep can
|
# from, and the last time a chunk made progress (so the recovery sweep can
|
||||||
# tell a progressing multi-chunk audit from a stuck one).
|
# tell a progressing multi-chunk audit from a stuck one).
|
||||||
resume_after_id: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
resume_after_id: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
last_progress_at: Mapped[datetime | None] = mapped_column(
|
last_progress_at: Mapped[datetime | None] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=True,
|
DateTime(timezone=True), nullable=True,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -31,17 +31,20 @@ class MLSettings(Base):
|
|||||||
# queueing embed work nothing will consume (the daily GPU 'embed' backfill
|
# queueing embed work nothing will consume (the daily GPU 'embed' backfill
|
||||||
# covers those images instead).
|
# covers those images instead).
|
||||||
cpu_embed_enabled: Mapped[bool] = mapped_column(
|
cpu_embed_enabled: Mapped[bool] = mapped_column(
|
||||||
Boolean, nullable=False, default=True
|
Boolean, nullable=False, default=True,
|
||||||
|
server_default="true",
|
||||||
)
|
)
|
||||||
# Video embedding (#747). Sample one frame every N seconds (fixed CADENCE, not
|
# Video embedding (#747). Sample one frame every N seconds (fixed CADENCE, not
|
||||||
# a fixed count) so coverage reflects real screen time regardless of length;
|
# a fixed count) so coverage reflects real screen time regardless of length;
|
||||||
# cap the total so a long video can't explode into hundreds of embeds. The
|
# cap the total so a long video can't explode into hundreds of embeds. The
|
||||||
# per-frame SigLIP embeddings are mean-pooled. Operator-tunable.
|
# per-frame SigLIP embeddings are mean-pooled. Operator-tunable.
|
||||||
video_frame_interval_seconds: Mapped[float] = mapped_column(
|
video_frame_interval_seconds: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=4.0
|
Float, nullable=False, default=4.0,
|
||||||
|
server_default="4",
|
||||||
)
|
)
|
||||||
video_max_frames: Mapped[int] = mapped_column(
|
video_max_frames: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=64
|
Integer, nullable=False, default=64,
|
||||||
|
server_default="64",
|
||||||
)
|
)
|
||||||
# Tagging-v2 head training (#114). The head is the suggestion source that
|
# Tagging-v2 head training (#114). The head is the suggestion source that
|
||||||
# LEARNS from the operator's tags (replacing Camie + centroid). A concept
|
# LEARNS from the operator's tags (replacing Camie + centroid). A concept
|
||||||
@@ -49,10 +52,12 @@ class MLSettings(Base):
|
|||||||
# head_auto_apply_precision is the precision bar a head must clear (at some
|
# head_auto_apply_precision is the precision bar a head must clear (at some
|
||||||
# operating point) to "graduate" into earned auto-apply. Operator-tunable.
|
# operating point) to "graduate" into earned auto-apply. Operator-tunable.
|
||||||
head_min_positives: Mapped[int] = mapped_column(
|
head_min_positives: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=8
|
Integer, nullable=False, default=8,
|
||||||
|
server_default="8",
|
||||||
)
|
)
|
||||||
head_auto_apply_precision: Mapped[float] = mapped_column(
|
head_auto_apply_precision: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=0.97
|
Float, nullable=False, default=0.97,
|
||||||
|
server_default="0.97",
|
||||||
)
|
)
|
||||||
# Earned auto-apply (#114). A graduated head fires (tags images without a
|
# Earned auto-apply (#114). A graduated head fires (tags images without a
|
||||||
# human) when this master switch is on AND the head has at least
|
# human) when this master switch is on AND the head has at least
|
||||||
@@ -61,29 +66,34 @@ class MLSettings(Base):
|
|||||||
# default (operator-asked 2026-06-29: opt-OUT, not opt-in); the support +
|
# default (operator-asked 2026-06-29: opt-OUT, not opt-in); the support +
|
||||||
# measured-precision gates keep it safe, and every auto-tag is reversible.
|
# measured-precision gates keep it safe, and every auto-tag is reversible.
|
||||||
head_auto_apply_enabled: Mapped[bool] = mapped_column(
|
head_auto_apply_enabled: Mapped[bool] = mapped_column(
|
||||||
Boolean, nullable=False, default=True
|
Boolean, nullable=False, default=True,
|
||||||
|
server_default="true",
|
||||||
)
|
)
|
||||||
head_auto_apply_min_positives: Mapped[int] = mapped_column(
|
head_auto_apply_min_positives: Mapped[int] = mapped_column(
|
||||||
# Support floor raised 30→50 (operator-asked 2026-07-06): a head needs
|
# Support floor raised 30→50 (operator-asked 2026-07-06): a head needs
|
||||||
# more human labels before it may fire without a human.
|
# more human labels before it may fire without a human.
|
||||||
Integer, nullable=False, default=50
|
Integer, nullable=False, default=50,
|
||||||
|
server_default="30",
|
||||||
)
|
)
|
||||||
# CCIP character-match cosine cut (#114). 0.85 default — the v1 flat 0.75
|
# CCIP character-match cosine cut (#114). 0.85 default — the v1 flat 0.75
|
||||||
# over-fired (high-reference characters matched a scatter of images); 0.85
|
# over-fired (high-reference characters matched a scatter of images); 0.85
|
||||||
# keeps the confident single-character matches. Tunable from the agent card.
|
# keeps the confident single-character matches. Tunable from the agent card.
|
||||||
ccip_match_threshold: Mapped[float] = mapped_column(
|
ccip_match_threshold: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=0.85
|
Float, nullable=False, default=0.85,
|
||||||
|
server_default="0.85",
|
||||||
)
|
)
|
||||||
# CCIP auto-apply (#114). Confident matches (>= ccip_auto_apply_threshold,
|
# CCIP auto-apply (#114). Confident matches (>= ccip_auto_apply_threshold,
|
||||||
# above the suggest cut) auto-tag on a daily sweep. ON by default (opt-out);
|
# above the suggest cut) auto-tag on a daily sweep. ON by default (opt-out);
|
||||||
# single-character references + the high bar keep it safe, every tag reversible.
|
# single-character references + the high bar keep it safe, every tag reversible.
|
||||||
ccip_auto_apply_enabled: Mapped[bool] = mapped_column(
|
ccip_auto_apply_enabled: Mapped[bool] = mapped_column(
|
||||||
Boolean, nullable=False, default=True
|
Boolean, nullable=False, default=True,
|
||||||
|
server_default="true",
|
||||||
)
|
)
|
||||||
ccip_auto_apply_threshold: Mapped[float] = mapped_column(
|
ccip_auto_apply_threshold: Mapped[float] = mapped_column(
|
||||||
# Raised 0.92→0.95 (operator-asked 2026-07-06) so only very confident
|
# Raised 0.92→0.95 (operator-asked 2026-07-06) so only very confident
|
||||||
# character matches auto-tag.
|
# character matches auto-tag.
|
||||||
Float, nullable=False, default=0.95
|
Float, nullable=False, default=0.95,
|
||||||
|
server_default="0.92",
|
||||||
)
|
)
|
||||||
# -- Presentation chrome auto-hide (#141) -------------------------------
|
# -- Presentation chrome auto-hide (#141) -------------------------------
|
||||||
# `banner` (chrome — clusters on UI, not content) auto-applies on the sweep
|
# `banner` (chrome — clusters on UI, not content) auto-applies on the sweep
|
||||||
@@ -95,13 +105,16 @@ class MLSettings(Base):
|
|||||||
# (opt-out); every auto-tag is reversible. NOTE (#1464): `wip` + `editor
|
# (opt-out); every auto-tag is reversible. NOTE (#1464): `wip` + `editor
|
||||||
# screenshot` are no longer chrome — they went to the PROCESS path below.
|
# screenshot` are no longer chrome — they went to the PROCESS path below.
|
||||||
presentation_auto_apply_enabled: Mapped[bool] = mapped_column(
|
presentation_auto_apply_enabled: Mapped[bool] = mapped_column(
|
||||||
Boolean, nullable=False, default=True
|
Boolean, nullable=False, default=True,
|
||||||
|
server_default="true",
|
||||||
)
|
)
|
||||||
presentation_auto_apply_threshold: Mapped[float] = mapped_column(
|
presentation_auto_apply_threshold: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=0.90
|
Float, nullable=False, default=0.90,
|
||||||
|
server_default="0.90",
|
||||||
)
|
)
|
||||||
presentation_conflict_threshold: Mapped[float] = mapped_column(
|
presentation_conflict_threshold: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=0.50
|
Float, nullable=False, default=0.50,
|
||||||
|
server_default="0.50",
|
||||||
)
|
)
|
||||||
# -- Process auto-apply (#1464) ----------------------------------------
|
# -- Process auto-apply (#1464) ----------------------------------------
|
||||||
# `wip` / `editor screenshot` are PROCESS art — unfinished pieces + program
|
# `wip` / `editor screenshot` are PROCESS art — unfinished pieces + program
|
||||||
@@ -115,24 +128,29 @@ class MLSettings(Base):
|
|||||||
# (PresentationReview, mode='process') rather than silently marked. OFF by
|
# (PresentationReview, mode='process') rather than silently marked. OFF by
|
||||||
# default — a new whole-library auto-tagger is opt-in; every auto-tag reversible.
|
# default — a new whole-library auto-tagger is opt-in; every auto-tag reversible.
|
||||||
process_auto_apply_enabled: Mapped[bool] = mapped_column(
|
process_auto_apply_enabled: Mapped[bool] = mapped_column(
|
||||||
Boolean, nullable=False, default=False
|
Boolean, nullable=False, default=False,
|
||||||
|
server_default="false",
|
||||||
)
|
)
|
||||||
process_auto_apply_threshold: Mapped[float] = mapped_column(
|
process_auto_apply_threshold: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=0.90
|
Float, nullable=False, default=0.90,
|
||||||
|
server_default="0.9",
|
||||||
)
|
)
|
||||||
process_conflict_threshold: Mapped[float] = mapped_column(
|
process_conflict_threshold: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=0.50
|
Float, nullable=False, default=0.50,
|
||||||
|
server_default="0.5",
|
||||||
)
|
)
|
||||||
# Default = SigLIP 2 (so400m, 512px) for new installs (migration 0069);
|
# Default = SigLIP 2 (so400m, 512px) for new installs (migration 0069);
|
||||||
# existing libraries keep their stored value until the operator re-embeds.
|
# existing libraries keep their stored value until the operator re-embeds.
|
||||||
embedder_model_version: Mapped[str] = mapped_column(
|
embedder_model_version: Mapped[str] = mapped_column(
|
||||||
String(128), nullable=False, default="siglip2-so400m-patch16-512"
|
String(128), nullable=False, default="siglip2-so400m-patch16-512",
|
||||||
|
server_default="siglip2-so400m-patch16-512",
|
||||||
)
|
)
|
||||||
# The HF model NAME the embedder loads (server CPU embed + announced to the
|
# The HF model NAME the embedder loads (server CPU embed + announced to the
|
||||||
# GPU agent in the lease). Operator-settable so the embedder is a choice, not
|
# GPU agent in the lease). Operator-settable so the embedder is a choice, not
|
||||||
# a hardcode (#1190): set name + version together, then re-embed + retrain.
|
# a hardcode (#1190): set name + version together, then re-embed + retrain.
|
||||||
embedder_model_name: Mapped[str] = mapped_column(
|
embedder_model_name: Mapped[str] = mapped_column(
|
||||||
String(128), nullable=False, default="google/siglip2-so400m-patch16-512"
|
String(128), nullable=False, default="google/siglip2-so400m-patch16-512",
|
||||||
|
server_default="google/siglip2-so400m-patch16-512",
|
||||||
)
|
)
|
||||||
# -- Crop proposers / detectors (#1202, #134) --------------------------
|
# -- Crop proposers / detectors (#1202, #134) --------------------------
|
||||||
# WHERE-to-crop YOLO detectors feeding the crop→SigLIP bag + CCIP. Config
|
# WHERE-to-crop YOLO detectors feeding the crop→SigLIP bag + CCIP. Config
|
||||||
@@ -145,20 +163,24 @@ class MLSettings(Base):
|
|||||||
# person: general COCO figure detector for Western/realistic art the anime
|
# person: general COCO figure detector for Western/realistic art the anime
|
||||||
# person-detector misses → NMS-merged with imgutils → CCIP + concept.
|
# person-detector misses → NMS-merged with imgutils → CCIP + concept.
|
||||||
detector_person_enabled: Mapped[bool] = mapped_column(
|
detector_person_enabled: Mapped[bool] = mapped_column(
|
||||||
Boolean, nullable=False, default=True
|
Boolean, nullable=False, default=True,
|
||||||
|
server_default="true",
|
||||||
)
|
)
|
||||||
detector_person_weights: Mapped[str] = mapped_column(
|
detector_person_weights: Mapped[str] = mapped_column(
|
||||||
String(512), nullable=False, default="yolo11n.pt"
|
String(512), nullable=False, default="yolo11n.pt",
|
||||||
|
server_default="yolo11n.pt",
|
||||||
)
|
)
|
||||||
detector_person_conf: Mapped[float] = mapped_column(
|
detector_person_conf: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=0.35
|
Float, nullable=False, default=0.35,
|
||||||
|
server_default="0.35",
|
||||||
)
|
)
|
||||||
# anatomy: booru_yolo anime/furry/NSFW torso components → concept crops.
|
# anatomy: booru_yolo anime/furry/NSFW torso components → concept crops.
|
||||||
# Default = yolov11m_aa22 (26 classes, best mAP50-95 0.96), committed in the
|
# Default = yolov11m_aa22 (26 classes, best mAP50-95 0.96), committed in the
|
||||||
# upstream repo so the URL resolves. License UNSTATED — fine for a private
|
# upstream repo so the URL resolves. License UNSTATED — fine for a private
|
||||||
# homelab (operator accepted #1202).
|
# homelab (operator accepted #1202).
|
||||||
detector_anatomy_enabled: Mapped[bool] = mapped_column(
|
detector_anatomy_enabled: Mapped[bool] = mapped_column(
|
||||||
Boolean, nullable=False, default=True
|
Boolean, nullable=False, default=True,
|
||||||
|
server_default="true",
|
||||||
)
|
)
|
||||||
detector_anatomy_weights: Mapped[str] = mapped_column(
|
detector_anatomy_weights: Mapped[str] = mapped_column(
|
||||||
String(512), nullable=False,
|
String(512), nullable=False,
|
||||||
@@ -166,37 +188,47 @@ class MLSettings(Base):
|
|||||||
"https://github.com/aperveyev/booru_yolo/raw/main/models/"
|
"https://github.com/aperveyev/booru_yolo/raw/main/models/"
|
||||||
"yolov11m_aa22.pt"
|
"yolov11m_aa22.pt"
|
||||||
),
|
),
|
||||||
|
server_default="https://github.com/aperveyev/booru_yolo/raw/main/models/yolov11m_aa22.pt",
|
||||||
)
|
)
|
||||||
detector_anatomy_conf: Mapped[float] = mapped_column(
|
detector_anatomy_conf: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=0.30
|
Float, nullable=False, default=0.30,
|
||||||
|
server_default="0.30",
|
||||||
)
|
)
|
||||||
# panel: comic page → panel regions → concept crops (Apache-2.0, YOLOv12x).
|
# panel: comic page → panel regions → concept crops (Apache-2.0, YOLOv12x).
|
||||||
detector_panel_enabled: Mapped[bool] = mapped_column(
|
detector_panel_enabled: Mapped[bool] = mapped_column(
|
||||||
Boolean, nullable=False, default=True
|
Boolean, nullable=False, default=True,
|
||||||
|
server_default="true",
|
||||||
)
|
)
|
||||||
detector_panel_weights: Mapped[str] = mapped_column(
|
detector_panel_weights: Mapped[str] = mapped_column(
|
||||||
String(512), nullable=False,
|
String(512), nullable=False,
|
||||||
default="mosesb/best-comic-panel-detection::best.pt",
|
default="mosesb/best-comic-panel-detection::best.pt",
|
||||||
|
server_default="mosesb/best-comic-panel-detection::best.pt",
|
||||||
)
|
)
|
||||||
detector_panel_conf: Mapped[float] = mapped_column(
|
detector_panel_conf: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=0.30
|
Float, nullable=False, default=0.30,
|
||||||
|
server_default="0.30",
|
||||||
)
|
)
|
||||||
# Per-frame caps bound the crop→embed explosion; max_regions is the hard
|
# Per-frame caps bound the crop→embed explosion; max_regions is the hard
|
||||||
# per-job backstop; dedupe_iou drops near-duplicate crops before the embed.
|
# per-job backstop; dedupe_iou drops near-duplicate crops before the embed.
|
||||||
detector_max_figures: Mapped[int] = mapped_column(
|
detector_max_figures: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=8
|
Integer, nullable=False, default=8,
|
||||||
|
server_default="8",
|
||||||
)
|
)
|
||||||
detector_max_components: Mapped[int] = mapped_column(
|
detector_max_components: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=8
|
Integer, nullable=False, default=8,
|
||||||
|
server_default="8",
|
||||||
)
|
)
|
||||||
detector_max_panels: Mapped[int] = mapped_column(
|
detector_max_panels: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=8
|
Integer, nullable=False, default=8,
|
||||||
|
server_default="8",
|
||||||
)
|
)
|
||||||
detector_max_regions: Mapped[int] = mapped_column(
|
detector_max_regions: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=128
|
Integer, nullable=False, default=128,
|
||||||
|
server_default="128",
|
||||||
)
|
)
|
||||||
detector_dedupe_iou: Mapped[float] = mapped_column(
|
detector_dedupe_iou: Mapped[float] = mapped_column(
|
||||||
Float, nullable=False, default=0.85
|
Float, nullable=False, default=0.85,
|
||||||
|
server_default="0.85",
|
||||||
)
|
)
|
||||||
# -- CCIP character prototypes (#1317) ---------------------------------
|
# -- CCIP character prototypes (#1317) ---------------------------------
|
||||||
# The per-character reference set is precomputed + refreshed INCREMENTALLY
|
# The per-character reference set is precomputed + refreshed INCREMENTALLY
|
||||||
@@ -208,7 +240,8 @@ class MLSettings(Base):
|
|||||||
String(128), nullable=True
|
String(128), nullable=True
|
||||||
)
|
)
|
||||||
ccip_prototype_cap: Mapped[int] = mapped_column(
|
ccip_prototype_cap: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=64
|
Integer, nullable=False, default=64,
|
||||||
|
server_default="64",
|
||||||
)
|
)
|
||||||
updated_at: Mapped[datetime] = mapped_column(
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
|||||||
@@ -35,7 +35,7 @@ class PatreonFailedMedia(Base):
|
|||||||
ForeignKey("source.id", ondelete="CASCADE"), nullable=False, index=True
|
ForeignKey("source.id", ondelete="CASCADE"), nullable=False, index=True
|
||||||
)
|
)
|
||||||
filehash: Mapped[str] = mapped_column(String(128), nullable=False)
|
filehash: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
|
attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=1, server_default="1")
|
||||||
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||||
first_failed_at: Mapped[datetime] = mapped_column(
|
first_failed_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
|||||||
@@ -35,7 +35,7 @@ class PixivFailedMedia(Base):
|
|||||||
ForeignKey("source.id", ondelete="CASCADE"), nullable=False, index=True
|
ForeignKey("source.id", ondelete="CASCADE"), nullable=False, index=True
|
||||||
)
|
)
|
||||||
filehash: Mapped[str] = mapped_column(String(128), nullable=False)
|
filehash: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
|
attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=1, server_default="1")
|
||||||
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||||
first_failed_at: Mapped[datetime] = mapped_column(
|
first_failed_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
|||||||
@@ -9,15 +9,17 @@ artist-filter queries don't depend on the Source detour).
|
|||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import (
|
from sqlalchemy import (
|
||||||
JSON,
|
|
||||||
CheckConstraint,
|
CheckConstraint,
|
||||||
DateTime,
|
DateTime,
|
||||||
ForeignKey,
|
ForeignKey,
|
||||||
|
Index,
|
||||||
Integer,
|
Integer,
|
||||||
|
JSON,
|
||||||
String,
|
String,
|
||||||
Text,
|
Text,
|
||||||
UniqueConstraint,
|
UniqueConstraint,
|
||||||
func,
|
func,
|
||||||
|
text,
|
||||||
)
|
)
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
@@ -27,6 +29,10 @@ from .base import Base
|
|||||||
class Post(Base):
|
class Post(Base):
|
||||||
__tablename__ = "post"
|
__tablename__ = "post"
|
||||||
__table_args__ = (
|
__table_args__ = (
|
||||||
|
# alembic 0030. The comment above described this index; nothing declared
|
||||||
|
# it, so autogenerate proposed dropping it (#3275).
|
||||||
|
Index("uq_post_artist_external_id_null_source", "artist_id", "external_post_id",
|
||||||
|
unique=True, postgresql_where=text("source_id IS NULL")),
|
||||||
# Source-bound dedup. Postgres treats NULL != NULL so rows
|
# Source-bound dedup. Postgres treats NULL != NULL so rows
|
||||||
# with source_id IS NULL aren't deduped by this constraint;
|
# with source_id IS NULL aren't deduped by this constraint;
|
||||||
# the partial unique index `uq_post_artist_external_id_null_source`
|
# the partial unique index `uq_post_artist_external_id_null_source`
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ are pruned by retention.
|
|||||||
|
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import DateTime, Float, ForeignKey, String, func
|
from sqlalchemy import DateTime, Float, ForeignKey, Index, String, func
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from .base import Base
|
from .base import Base
|
||||||
@@ -20,6 +20,10 @@ from .base import Base
|
|||||||
class PresentationReview(Base):
|
class PresentationReview(Base):
|
||||||
__tablename__ = "presentation_review"
|
__tablename__ = "presentation_review"
|
||||||
|
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
Index("ix_presentation_review_resolved_at", "resolved_at"),
|
||||||
|
)
|
||||||
image_record_id: Mapped[int] = mapped_column(
|
image_record_id: Mapped[int] = mapped_column(
|
||||||
ForeignKey("image_record.id", ondelete="CASCADE"), primary_key=True
|
ForeignKey("image_record.id", ondelete="CASCADE"), primary_key=True
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ Multiple sources per artist support creators with cross-platform presence.
|
|||||||
|
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import JSON, Boolean, DateTime, ForeignKey, Integer, String, Text
|
from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, JSON, String, Text
|
||||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||||
|
|
||||||
from .base import Base
|
from .base import Base
|
||||||
@@ -20,7 +20,7 @@ class Source(Base):
|
|||||||
)
|
)
|
||||||
platform: Mapped[str] = mapped_column(String(64), nullable=False)
|
platform: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
url: Mapped[str] = mapped_column(Text, nullable=False)
|
url: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
|
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
|
||||||
|
|
||||||
config_overrides: Mapped[dict | None] = mapped_column(JSON, nullable=True)
|
config_overrides: Mapped[dict | None] = mapped_column(JSON, nullable=True)
|
||||||
|
|
||||||
@@ -32,7 +32,7 @@ class Source(Base):
|
|||||||
# by _update_source_health alongside last_error; cleared on 'ok'.
|
# by _update_source_health alongside last_error; cleared on 'ok'.
|
||||||
error_type: Mapped[str | None] = mapped_column(String(32), nullable=True, index=True)
|
error_type: Mapped[str | None] = mapped_column(String(32), nullable=True, index=True)
|
||||||
check_interval_override: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
check_interval_override: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||||
consecutive_failures: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
consecutive_failures: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
|
|
||||||
# alembic 0031: sticky deep-scan budget. When > 0, the next N download
|
# alembic 0031: sticky deep-scan budget. When > 0, the next N download
|
||||||
# runs use gallery-dl's full-walk config (skip: True + 1800s timeout);
|
# runs use gallery-dl's full-walk config (skip: True + 1800s timeout);
|
||||||
|
|||||||
@@ -34,7 +34,7 @@ class SubscribeStarFailedMedia(Base):
|
|||||||
ForeignKey("source.id", ondelete="CASCADE"), nullable=False, index=True
|
ForeignKey("source.id", ondelete="CASCADE"), nullable=False, index=True
|
||||||
)
|
)
|
||||||
filehash: Mapped[str] = mapped_column(String(128), nullable=False)
|
filehash: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
|
attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=1, server_default="1")
|
||||||
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||||
first_failed_at: Mapped[datetime] = mapped_column(
|
first_failed_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
|||||||
@@ -15,11 +15,13 @@ from sqlalchemy import (
|
|||||||
Column,
|
Column,
|
||||||
DateTime,
|
DateTime,
|
||||||
ForeignKey,
|
ForeignKey,
|
||||||
|
Index,
|
||||||
Integer,
|
Integer,
|
||||||
String,
|
String,
|
||||||
Table,
|
Table,
|
||||||
false,
|
false,
|
||||||
func,
|
func,
|
||||||
|
text,
|
||||||
)
|
)
|
||||||
from sqlalchemy import (
|
from sqlalchemy import (
|
||||||
Enum as SQLEnum,
|
Enum as SQLEnum,
|
||||||
@@ -67,7 +69,7 @@ image_tag = Table(
|
|||||||
primary_key=True,
|
primary_key=True,
|
||||||
),
|
),
|
||||||
Column("tag_id", ForeignKey("tag.id", ondelete="CASCADE"), primary_key=True),
|
Column("tag_id", ForeignKey("tag.id", ondelete="CASCADE"), primary_key=True),
|
||||||
Column("source", String(32), nullable=False, default="manual"),
|
Column("source", String(32), nullable=False, default="manual", server_default="manual"),
|
||||||
Column("created_at", DateTime(timezone=True), nullable=False, server_default=func.now()),
|
Column("created_at", DateTime(timezone=True), nullable=False, server_default=func.now()),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -75,6 +77,10 @@ image_tag = Table(
|
|||||||
class Tag(Base):
|
class Tag(Base):
|
||||||
__tablename__ = "tag"
|
__tablename__ = "tag"
|
||||||
__table_args__ = (
|
__table_args__ = (
|
||||||
|
# alembic 0002. An EXPRESSION index — COALESCE cannot be expressed as a
|
||||||
|
# UniqueConstraint, which is why it only ever existed in a migration (#3275).
|
||||||
|
Index("uq_tag_name_kind_fandom", "name", "kind", text("COALESCE(fandom_id, 0)"),
|
||||||
|
unique=True),
|
||||||
CheckConstraint(
|
CheckConstraint(
|
||||||
"(fandom_id IS NULL) OR (kind = 'character')",
|
"(fandom_id IS NULL) OR (kind = 'character')",
|
||||||
name="ck_tag_fandom_requires_character",
|
name="ck_tag_fandom_requires_character",
|
||||||
@@ -87,6 +93,7 @@ class Tag(Base):
|
|||||||
SQLEnum(TagKind, name="tag_kind", values_callable=lambda e: [m.value for m in e]),
|
SQLEnum(TagKind, name="tag_kind", values_callable=lambda e: [m.value for m in e]),
|
||||||
nullable=False,
|
nullable=False,
|
||||||
default=TagKind.general,
|
default=TagKind.general,
|
||||||
|
server_default="general",
|
||||||
)
|
)
|
||||||
fandom_id: Mapped[int | None] = mapped_column(
|
fandom_id: Mapped[int | None] = mapped_column(
|
||||||
ForeignKey("tag.id", ondelete="SET NULL"), nullable=True, index=True
|
ForeignKey("tag.id", ondelete="SET NULL"), nullable=True, index=True
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ in image_prediction stay unmolested.
|
|||||||
|
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import DateTime, ForeignKey, String, func
|
from sqlalchemy import DateTime, ForeignKey, Index, String, func
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from .base import Base
|
from .base import Base
|
||||||
@@ -14,10 +14,17 @@ from .base import Base
|
|||||||
class TagAlias(Base):
|
class TagAlias(Base):
|
||||||
__tablename__ = "tag_alias"
|
__tablename__ = "tag_alias"
|
||||||
|
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
# Named explicitly: the database calls this ix_tag_alias_canonical, while
|
||||||
|
# a bare index=True on the column would generate ix_tag_alias_canonical_tag_id
|
||||||
|
# and silently propose a drop+create on the next autogenerate (#3275).
|
||||||
|
Index("ix_tag_alias_canonical", "canonical_tag_id"),
|
||||||
|
)
|
||||||
alias_string: Mapped[str] = mapped_column(String(255), primary_key=True)
|
alias_string: Mapped[str] = mapped_column(String(255), primary_key=True)
|
||||||
alias_category: Mapped[str] = mapped_column(String(32), primary_key=True)
|
alias_category: Mapped[str] = mapped_column(String(32), primary_key=True)
|
||||||
canonical_tag_id: Mapped[int] = mapped_column(
|
canonical_tag_id: Mapped[int] = mapped_column(
|
||||||
ForeignKey("tag.id", ondelete="CASCADE"), nullable=False, index=True
|
ForeignKey("tag.id", ondelete="CASCADE"), nullable=False
|
||||||
)
|
)
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ Prevents re-suggestion AND prevents allowlist auto-apply on that image.
|
|||||||
|
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import DateTime, ForeignKey, func
|
from sqlalchemy import DateTime, ForeignKey, Index, func
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from .base import Base
|
from .base import Base
|
||||||
@@ -14,11 +14,16 @@ from .base import Base
|
|||||||
class TagSuggestionRejection(Base):
|
class TagSuggestionRejection(Base):
|
||||||
__tablename__ = "tag_suggestion_rejection"
|
__tablename__ = "tag_suggestion_rejection"
|
||||||
|
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
# Named explicitly; see tag_alias for why (#3275).
|
||||||
|
Index("ix_tag_suggestion_rejection_tag", "tag_id"),
|
||||||
|
)
|
||||||
image_record_id: Mapped[int] = mapped_column(
|
image_record_id: Mapped[int] = mapped_column(
|
||||||
ForeignKey("image_record.id", ondelete="CASCADE"), primary_key=True
|
ForeignKey("image_record.id", ondelete="CASCADE"), primary_key=True
|
||||||
)
|
)
|
||||||
tag_id: Mapped[int] = mapped_column(
|
tag_id: Mapped[int] = mapped_column(
|
||||||
ForeignKey("tag.id", ondelete="CASCADE"), primary_key=True, index=True
|
ForeignKey("tag.id", ondelete="CASCADE"), primary_key=True
|
||||||
)
|
)
|
||||||
rejected_at: Mapped[datetime] = mapped_column(
|
rejected_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ backend.app.tasks.maintenance.recover_stalled_task_runs (Beat 5 min).
|
|||||||
|
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import DateTime, Integer, String, Text
|
from sqlalchemy import DateTime, Index, Integer, String, Text, text
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from .base import Base
|
from .base import Base
|
||||||
@@ -24,6 +24,13 @@ from .base import Base
|
|||||||
class TaskRun(Base):
|
class TaskRun(Base):
|
||||||
__tablename__ = "task_run"
|
__tablename__ = "task_run"
|
||||||
|
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
# alembic 0016: the three task-history indexes (#3275).
|
||||||
|
Index("ix_task_run_name_started", "task_name", text("started_at DESC")),
|
||||||
|
Index("ix_task_run_queue_started", "queue", text("started_at DESC")),
|
||||||
|
Index("ix_task_run_status_started", "status", text("started_at DESC")),
|
||||||
|
)
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||||
celery_task_id: Mapped[str] = mapped_column(
|
celery_task_id: Mapped[str] = mapped_column(
|
||||||
String(64), nullable=False, index=True,
|
String(64), nullable=False, index=True,
|
||||||
@@ -40,6 +47,7 @@ class TaskRun(Base):
|
|||||||
duration_ms: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
duration_ms: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||||
status: Mapped[str] = mapped_column(
|
status: Mapped[str] = mapped_column(
|
||||||
String(16), nullable=False, default="running", index=True,
|
String(16), nullable=False, default="running", index=True,
|
||||||
|
server_default="running",
|
||||||
)
|
)
|
||||||
error_type: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
error_type: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
||||||
error_message: Mapped[str | None] = mapped_column(Text, nullable=True)
|
error_message: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||||
|
|||||||
Reference in New Issue
Block a user