Files
FabledCurator/backend/app/models/import_settings.py
T
bvandeusenandClaude Opus 5 573228b9da
CI / lint (push) Failing after 3s
Build images / sign-extension (push) Successful in 4s
CI / extension-version (push) Successful in 3s
Build images / build-agent (push) Successful in 9s
CI / frontend-build (push) Successful in 34s
Build images / build-ml (push) Successful in 53s
Build images / build-web (push) Successful in 44s
CI / backend-lint-and-test (push) Successful in 1m6s
CI / integration (push) Successful in 4m5s
db: finish reconciling the models with the deployed schema (#3275)
Closes the residue the first reconciliation pass left, and corrects a
factual error I put into the record.

sha256 was NOT missing a uniqueness guarantee. I read
`op.create_index("ix_image_record_sha256", ...)` at 0001 line 151 and
concluded duplicates were possible, without reading line 149 two lines
above it:

    sa.UniqueConstraint("sha256", name="uq_image_record_sha256"),

Uniqueness has held since the initial schema. The database expresses it
as a CONSTRAINT plus a separate non-unique lookup index; the model said
`unique=True, index=True`, which is one UNIQUE index under a different
name. Same guarantee, different objects — which is exactly why the two
schemas did not line up. The model now declares both objects. No DDL.
0088's docstring, which repeated the claim, is corrected in place.

Two real divergences, both the MODEL over-claiming:

  * source: uq_source_artist_platform_url (alembic 0010) was declared
    nowhere in the models — source.py had no __table_args__ at all — so
    autogenerate would have proposed DROPPING it.
  * head_metrics_snapshot.tag_id: model said NOT NULL, 0060 created it
    nullable. Left nullable; the FK already cascades.

Seven constraints renamed to what the chain actually created, rather than
what base.py's naming convention renders: uq_series_page_image,
uq_series_chapter_anchor_page, fk_series_chapter_anchor_page,
fk_image_record_artist_id, fk_image_provenance_from_attachment, and the
two hand-shortened fk_tsr_* names from 0003.

Float server_defaults now mirror their own migration, per column. The
chain is MIXED: a plain string renders DEFAULT '0.90'::double precision,
sa.text() renders DEFAULT 0.90, and the migrations used both. Seven
columns take text(); the rest stay strings. Two literals also disagreed
outright — process_{auto_apply,conflict}_threshold said 0.9/0.5 against
the migration's 0.90/0.50.

baseline.yml gains two things. A repair for a SECOND generator defect in
the same class as the missing pgvector import: base.py's ck convention
contains %(constraint_name)s, so it applies even to a NAMED
CheckConstraint — autogenerate writes the already-rendered name into the
migration and running it applies the convention again, yielding
ck_ml_settings_ck_ml_settings_singleton. That is round-tripping damage,
not a claim the models make, so it is undone rather than counted.

And the diff now runs twice. Column ORDER differs permanently between a
schema built by 87 ADD COLUMNs and one built in a single shot — the
operator's database keeps chain order forever, a fresh install gets model
order — so a check that failed on it could never pass. The second pass
SORTS column lines within each CREATE TABLE instead of deleting them,
which cannot hide a column present on one side only, or one whose type,
nullability or default differs. Ordered diff is reported as information;
the order-insensitive one is the verdict.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw
2026-08-31 00:24:00 -04:00

167 lines
7.4 KiB
Python

"""ImportSettings — single-row table holding the importer's tunable knobs.
Enforced as a single row via a CHECK (id = 1) constraint. The application
always SELECTs id=1 and never inserts/deletes after the initial migration.
"""
from sqlalchemy import (
Boolean,
CheckConstraint,
Float,
Integer,
Text,
select,
text,
)
from sqlalchemy.orm import Mapped, mapped_column
from .base import Base
class ImportSettings(Base):
__tablename__ = "import_settings"
# Bare constraint name — Base.metadata's naming convention applies the
# ck_<table>_<name> prefix, producing the final ck_import_settings_singleton.
__table_args__ = (CheckConstraint("id = 1", name="singleton"),)
id: Mapped[int] = mapped_column(Integer, primary_key=True)
import_scan_path: Mapped[str] = mapped_column(Text, nullable=False, default="/import", server_default="/import")
min_width: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
min_height: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
skip_transparent: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
transparency_threshold: Mapped[float] = mapped_column(Float, nullable=False, default=0.9, server_default="0.9")
skip_single_color: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
single_color_threshold: Mapped[float] = mapped_column(Float, nullable=False, default=0.95, server_default="0.95")
single_color_tolerance: Mapped[int] = mapped_column(Integer, nullable=False, default=30, server_default="30")
phash_threshold: Mapped[int] = mapped_column(Integer, nullable=False, default=10, server_default="10")
# FC-3c downloader knobs
download_rate_limit_seconds: Mapped[float] = mapped_column(
Float, nullable=False, default=3.0,
server_default="3",
)
download_validate_files: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True,
server_default="true",
)
# FC-3d scheduling knobs
download_schedule_default_seconds: Mapped[int] = mapped_column(
Integer, nullable=False, default=28800,
server_default="28800",
)
download_event_retention_days: Mapped[int] = mapped_column(
Integer, nullable=False, default=90,
server_default="90",
)
download_failure_warning_threshold: Mapped[int] = mapped_column(
Integer, nullable=False, default=5,
server_default="5",
)
# FC-3h backup knobs.
backup_db_nightly_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False,
server_default="false",
)
backup_db_nightly_hour_utc: Mapped[int] = mapped_column(
Integer, nullable=False, default=3,
server_default="3",
)
backup_db_keep_last_n: Mapped[int] = mapped_column(
Integer, nullable=False, default=14,
server_default="14",
)
backup_images_keep_last_n: Mapped[int] = mapped_column(
Integer, nullable=False, default=3,
server_default="3",
)
# FC-6.3 series continuation matcher. enabled gates the rescan; threshold is
# the weighted-score cut-off (0..1) above which a pending suggestion is made.
series_suggest_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True,
server_default="true",
)
series_suggest_threshold: Mapped[float] = mapped_column(
Float, nullable=False, default=0.5,
server_default="0.5",
)
# #830 off-platform file-host downloads — per-host enable lever (default on,
# rule #26). Column names are extdl_<host>_enabled so the worker reads them
# via getattr(settings, f"extdl_{host}_enabled", True).
extdl_mega_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True, server_default="true",
)
extdl_gdrive_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True, server_default="true",
)
extdl_mediafire_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True, server_default="true",
)
extdl_dropbox_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True, server_default="true",
)
extdl_pixeldrain_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True, server_default="true",
)
# -- Post-text translation via the Interpreter LAN service (milestone 143).
# Off by default with NO default host — it needs a reachable Interpreter
# service (the operator's, behind a reverse proxy), which not every install
# has; the operator sets the URL and flips it on. Empty base_url OR disabled
# → the translate sweep no-ops.
translation_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False, server_default="false",
)
interpreter_base_url: Mapped[str] = mapped_column(
Text, nullable=False, default="", server_default="",
)
translation_target_lang: Mapped[str] = mapped_column(
Text, nullable=False, default="en", server_default="en",
)
# The latin-script acceptance floor for the translation gate: a translation
# whose Interpreter-reported confidence is below this is kept as the original
# (operator-tunable, milestone 155). Default 0.9 — stricter than the old
# hardcoded 0.8, because Interpreter confidently mis-detects short ASCII
# English (e.g. "… WIP Part 1") as a European language at ~0.86. CJK stays
# trusted regardless (script-detected). Per-post overrides handle the misses.
translation_min_confidence: Mapped[float] = mapped_column(
# text() because alembic 0084 used sa.text(); see ml_settings for why
# the form matters and why it is per-column (#3275).
Float, nullable=False, default=0.9, server_default=text("0.9"),
)
# Title-based WIP auto-tagging (task #1458). When a freshly-imported post's
# TITLE explicitly declares work-in-progress ("WIP" / "work in progress"),
# the importer applies the `wip` system tag to its images — the artist's own
# label, used to keep unfinished pieces out of the Explore/gallery browse. ON
# by default (rule 26 — the feature works out of the box). Gates only the
# LIVE import hook; the existing catalogue is caught by the operator-triggered
# "Scan existing posts" backfill (which runs regardless of this flag).
wip_title_tagging_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True, server_default="true",
)
# Soft WIP title tier (#1474): also tag sketch/doodle/scribble titles, but with
# a PROVISIONAL source (`wip_title_soft`) that never trains the head, since these
# are lower-precision (a finished "sketch" isn't WIP). OFF by default — a lower-
# precision tier is opt-in (the ring-loud audit surfaces false positives).
wip_soft_title_tagging_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False, server_default="false",
)
@classmethod
async def load(cls, session) -> ImportSettings:
"""The singleton settings row (id=1), via an async session."""
return (await session.execute(select(cls).where(cls.id == 1))).scalar_one()
@classmethod
def load_sync(cls, session) -> ImportSettings:
"""The singleton settings row (id=1), via a sync session."""
return session.execute(select(cls).where(cls.id == 1)).scalar_one()