feat(ml): tag-eval backend — head-vs-centroid learning-curve eval (persisted)
Slice 1 of milestone #114 (tagging v2). Proves the frozen-embedding + trained- head spine on the operator's own data, reusing the SigLIP embeddings already stored on image_record — no re-embedding, no GPU. Per concept: train a logistic-regression HEAD (positives + negatives = explicit rejections + sampled unlabeled) vs the old single-CENTROID baseline; report cross-validated precision/recall/AP for both, a LEARNING CURVE (AP/F1 as tagged positives grow 10→30→100→300), and example image ids (head-would-suggest / head-doubts-positive) to eyeball. Persisted so the report SURVIVES navigation (operator-flagged): the run + full report live in a new tag_eval_run row (mirrors library_audit_run); the admin card will rehydrate from GET on mount, not transient state. - models.TagEvalRun + migration 0056; runs on the ml queue (only worker with numpy/sklearn) — numpy/sklearn lazy-imported so the API can still enqueue. - services/ml/tag_eval (compute + start helper, one-running guard), tasks.ml .tag_eval_run, api/tag-eval (POST create, GET history light / detail w/ report). - recover_stalled_tag_eval_runs sweep + retention (keep last 20) + 5-min beat (rule 89). scikit-learn added to requirements-ml. - tests: param normalization + the rehydrate read-path + create/conflict. Frontend admin card (trigger + render persisted report) follows next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -538,3 +538,48 @@ def recompute_centroids(self) -> int:
|
||||
for tid in drifted:
|
||||
recompute_centroid.delay(tid)
|
||||
return len(drifted)
|
||||
|
||||
|
||||
@celery.task(
|
||||
name="backend.app.tasks.ml.tag_eval_run",
|
||||
bind=True,
|
||||
# The head-vs-centroid eval (#1130) loads embeddings + fits sklearn heads
|
||||
# for several concepts — minutes, not seconds. Runs on the ml queue because
|
||||
# only that worker has numpy/scikit-learn.
|
||||
soft_time_limit=1800, time_limit=2100,
|
||||
)
|
||||
def tag_eval_run(self, run_id: int) -> str:
|
||||
"""Compute the eval report into the persisted TagEvalRun row so it survives
|
||||
navigation (the admin card rehydrates from the row, not transient state)."""
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from ..models import TagEvalRun
|
||||
from ..services.ml.tag_eval import run_eval
|
||||
|
||||
SessionLocal = _sync_session_factory()
|
||||
with SessionLocal() as session:
|
||||
run = session.get(TagEvalRun, run_id)
|
||||
if run is None:
|
||||
return "missing"
|
||||
run.last_progress_at = datetime.now(UTC)
|
||||
session.commit()
|
||||
try:
|
||||
report = run_eval(session, run.params)
|
||||
except SoftTimeLimitExceeded:
|
||||
run.status = "error"
|
||||
run.error = "timed out"
|
||||
run.finished_at = datetime.now(UTC)
|
||||
session.commit()
|
||||
raise
|
||||
except Exception as exc:
|
||||
log.exception("tag_eval_run %d failed", run_id)
|
||||
run.status = "error"
|
||||
run.error = str(exc)
|
||||
run.finished_at = datetime.now(UTC)
|
||||
session.commit()
|
||||
return "error"
|
||||
run.report = report
|
||||
run.status = "ready"
|
||||
run.finished_at = datetime.now(UTC)
|
||||
session.commit()
|
||||
return "ready"
|
||||
|
||||
Reference in New Issue
Block a user