bvandeusen
22c3b54746
The eval (#1130) proved the frozen-embedding + trained-head spine; this lands its production form (the first of three slices that make heads the suggestion source, replacing Camie + centroid). - tag_head: one logistic-regression head per general/character concept with enough labelled positives. Weights (pgvector), honest CV-derived suggest threshold + earned-auto-apply point, and per-concept quality metrics. - head_training_run: persisted batch lifecycle (mirrors tag_eval_run) so the admin card shows live + historical status across navigation. - services/ml/heads.py: TRAIN (sync, ml worker, reuses tag_eval's proven data loaders + metric math so production heads match measured eval numbers) and SCORE (async, API worker — numpy via pgvector, no scikit-learn): score one image's embedding against all heads → the rail's suggestions, cached on (count, max trained_at) so a retrain invalidates without per-request loads. - tasks.ml.train_heads (ml queue, commits per head so a kill leaves progress) + recover_stalled_head_training_runs sweep + retention(20) + 5-min beat (rule 89). - api/heads.py: POST /api/heads/train (one run at a time, 409 guard) + GET /api/heads (count, graduated, last-trained, running, per-concept table, recent runs). - ml_settings: head_min_positives + head_auto_apply_precision, tunable via /api/ml/settings. Scoring isn't wired into the rail yet (slice C) and the admin UI is slice B — this slice makes training + scoring exist and CI-verifiable. 'precision' column stored as precision_cv (SQL reserved word). Migration 0058. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
78 lines
3.6 KiB
Python
78 lines
3.6 KiB
Python
"""MLSettings — single-row table holding ML pipeline tunables."""
|
||
|
||
from datetime import datetime
|
||
|
||
from sqlalchemy import CheckConstraint, DateTime, Float, Integer, String, func
|
||
from sqlalchemy.orm import Mapped, mapped_column
|
||
|
||
from .base import Base
|
||
|
||
|
||
class MLSettings(Base):
|
||
__tablename__ = "ml_settings"
|
||
# Bare name — Base.metadata's naming convention prepends ck_<table>_,
|
||
# producing the final ck_ml_settings_singleton (matches migration 0003).
|
||
__table_args__ = (CheckConstraint("id = 1", name="singleton"),)
|
||
|
||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||
suggestion_threshold_character: Mapped[float] = mapped_column(
|
||
Float, nullable=False, default=0.70
|
||
)
|
||
# Default raised 0.50 → 0.70 on 2026-06-02 — operator-flagged 0.50
|
||
# surfaced too many low-confidence picks; 0.70 keeps the rail
|
||
# signal-rich while still surfacing more than the original 0.95
|
||
# which hid almost everything. Operator-tunable via Settings → ML.
|
||
suggestion_threshold_general: Mapped[float] = mapped_column(
|
||
Float, nullable=False, default=0.70
|
||
)
|
||
centroid_similarity_threshold: Mapped[float] = mapped_column(
|
||
Float, nullable=False, default=0.55
|
||
)
|
||
# Ingest floor: tagger predictions below this confidence are not stored
|
||
# (tagger.Tagger.infer). Default 0.70 — the suggestion path already
|
||
# filters at 0.70 and the centroid/learned path covers low-confidence
|
||
# preferred tags, so the sub-0.70 tail is redundant weight (it had
|
||
# bloated image_record's TOAST to ~100 GB; plan-task #764). Operator-
|
||
# tunable via Settings → ML; must stay ≤ the suggestion thresholds.
|
||
tagger_store_floor: Mapped[float] = mapped_column(
|
||
Float, nullable=False, default=0.70
|
||
)
|
||
min_reference_images: Mapped[int] = mapped_column(
|
||
Integer, nullable=False, default=5
|
||
)
|
||
# Video tagging (#747). Sample one frame every N seconds (fixed CADENCE, not a
|
||
# fixed count) so a tag's frame-presence reflects real screen time regardless
|
||
# of video length; cap the total so a long video can't explode into hundreds
|
||
# of inferences (the cadence stretches past the cap). A tag is kept only if it
|
||
# appears in >= video_min_tag_frames sampled frames (≈ that many × interval
|
||
# seconds on screen) — duration-independent noise rejection. Operator-tunable.
|
||
video_frame_interval_seconds: Mapped[float] = mapped_column(
|
||
Float, nullable=False, default=4.0
|
||
)
|
||
video_max_frames: Mapped[int] = mapped_column(
|
||
Integer, nullable=False, default=64
|
||
)
|
||
video_min_tag_frames: Mapped[int] = mapped_column(
|
||
Integer, nullable=False, default=3
|
||
)
|
||
# Tagging-v2 head training (#114). The head is the suggestion source that
|
||
# LEARNS from the operator's tags (replacing Camie + centroid). A concept
|
||
# needs >= head_min_positives labelled images before a head is trained;
|
||
# head_auto_apply_precision is the precision bar a head must clear (at some
|
||
# operating point) to "graduate" into earned auto-apply. Operator-tunable.
|
||
head_min_positives: Mapped[int] = mapped_column(
|
||
Integer, nullable=False, default=8
|
||
)
|
||
head_auto_apply_precision: Mapped[float] = mapped_column(
|
||
Float, nullable=False, default=0.97
|
||
)
|
||
tagger_model_version: Mapped[str] = mapped_column(
|
||
String(128), nullable=False, default="camie-tagger-v2"
|
||
)
|
||
embedder_model_version: Mapped[str] = mapped_column(
|
||
String(128), nullable=False, default="siglip-so400m-patch14-384"
|
||
)
|
||
updated_at: Mapped[datetime] = mapped_column(
|
||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||
)
|