feat(ml): tag-eval backend — head-vs-centroid learning-curve eval (persisted)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m23s

Slice 1 of milestone #114 (tagging v2). Proves the frozen-embedding + trained-
head spine on the operator's own data, reusing the SigLIP embeddings already
stored on image_record — no re-embedding, no GPU.

Per concept: train a logistic-regression HEAD (positives + negatives = explicit
rejections + sampled unlabeled) vs the old single-CENTROID baseline; report
cross-validated precision/recall/AP for both, a LEARNING CURVE (AP/F1 as tagged
positives grow 10→30→100→300), and example image ids (head-would-suggest /
head-doubts-positive) to eyeball.

Persisted so the report SURVIVES navigation (operator-flagged): the run + full
report live in a new tag_eval_run row (mirrors library_audit_run); the admin
card will rehydrate from GET on mount, not transient state.

- models.TagEvalRun + migration 0056; runs on the ml queue (only worker with
  numpy/sklearn) — numpy/sklearn lazy-imported so the API can still enqueue.
- services/ml/tag_eval (compute + start helper, one-running guard), tasks.ml
  .tag_eval_run, api/tag-eval (POST create, GET history light / detail w/ report).
- recover_stalled_tag_eval_runs sweep + retention (keep last 20) + 5-min beat
  (rule 89). scikit-learn added to requirements-ml.
- tests: param normalization + the rehydrate read-path + create/conflict.

Frontend admin card (trigger + render persisted report) follows next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-27 22:49:10 -04:00
parent 958378312c
commit 6e3c5f697f
11 changed files with 655 additions and 0 deletions
+43
View File
@@ -0,0 +1,43 @@
"""tag_eval_run: persisted head-vs-centroid tagging eval runs (#1130)
Milestone #114 slice 1. A long ml-queue eval whose full report must SURVIVE
navigation, so the run + report live in a row the admin card rehydrates from
(mirrors library_audit_run). running -> ready / error.
Revision ID: 0056
Revises: 0055
Create Date: 2026-06-28
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects.postgresql import JSONB
revision: str = "0056"
down_revision: Union[str, None] = "0055"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"tag_eval_run",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("params", JSONB(), nullable=False),
sa.Column("status", sa.String(length=16), nullable=False, server_default="running"),
sa.Column(
"started_at", sa.DateTime(timezone=True), nullable=False,
server_default=sa.func.now(),
),
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("report", JSONB(), nullable=True),
sa.Column("error", sa.Text(), nullable=True),
sa.Column("last_progress_at", sa.DateTime(timezone=True), nullable=True),
)
op.create_index("ix_tag_eval_run_status", "tag_eval_run", ["status"])
def downgrade() -> None:
op.drop_index("ix_tag_eval_run_status", table_name="tag_eval_run")
op.drop_table("tag_eval_run")