feat(ml): tag-eval backend — head-vs-centroid learning-curve eval (persisted)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m23s

Slice 1 of milestone #114 (tagging v2). Proves the frozen-embedding + trained-
head spine on the operator's own data, reusing the SigLIP embeddings already
stored on image_record — no re-embedding, no GPU.

Per concept: train a logistic-regression HEAD (positives + negatives = explicit
rejections + sampled unlabeled) vs the old single-CENTROID baseline; report
cross-validated precision/recall/AP for both, a LEARNING CURVE (AP/F1 as tagged
positives grow 10→30→100→300), and example image ids (head-would-suggest /
head-doubts-positive) to eyeball.

Persisted so the report SURVIVES navigation (operator-flagged): the run + full
report live in a new tag_eval_run row (mirrors library_audit_run); the admin
card will rehydrate from GET on mount, not transient state.

- models.TagEvalRun + migration 0056; runs on the ml queue (only worker with
  numpy/sklearn) — numpy/sklearn lazy-imported so the API can still enqueue.
- services/ml/tag_eval (compute + start helper, one-running guard), tasks.ml
  .tag_eval_run, api/tag-eval (POST create, GET history light / detail w/ report).
- recover_stalled_tag_eval_runs sweep + retention (keep last 20) + 5-min beat
  (rule 89). scikit-learn added to requirements-ml.
- tests: param normalization + the rehydrate read-path + create/conflict.

Frontend admin card (trigger + render persisted report) follows next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-27 22:49:10 -04:00
parent 958378312c
commit 6e3c5f697f
11 changed files with 655 additions and 0 deletions
+45
View File
@@ -538,3 +538,48 @@ def recompute_centroids(self) -> int:
for tid in drifted:
recompute_centroid.delay(tid)
return len(drifted)
@celery.task(
name="backend.app.tasks.ml.tag_eval_run",
bind=True,
# The head-vs-centroid eval (#1130) loads embeddings + fits sklearn heads
# for several concepts — minutes, not seconds. Runs on the ml queue because
# only that worker has numpy/scikit-learn.
soft_time_limit=1800, time_limit=2100,
)
def tag_eval_run(self, run_id: int) -> str:
"""Compute the eval report into the persisted TagEvalRun row so it survives
navigation (the admin card rehydrates from the row, not transient state)."""
from datetime import UTC, datetime
from ..models import TagEvalRun
from ..services.ml.tag_eval import run_eval
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
run = session.get(TagEvalRun, run_id)
if run is None:
return "missing"
run.last_progress_at = datetime.now(UTC)
session.commit()
try:
report = run_eval(session, run.params)
except SoftTimeLimitExceeded:
run.status = "error"
run.error = "timed out"
run.finished_at = datetime.now(UTC)
session.commit()
raise
except Exception as exc:
log.exception("tag_eval_run %d failed", run_id)
run.status = "error"
run.error = str(exc)
run.finished_at = datetime.now(UTC)
session.commit()
return "error"
run.report = report
run.status = "ready"
run.finished_at = datetime.now(UTC)
session.commit()
return "ready"