refactor(ml): remove the dead per-tag centroid subsystem (#1189)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m32s

The v2 pivot replaced per-tag SigLIP centroids with learned heads + CCIP.
Centroids were still recomputed (on every tag merge + a daily beat) but NOTHING
read them — suggestions come from heads+CCIP and apply_allowlist_tags applies
via Camie predictions, not centroids. Pure dead wiring; remove it.

Removed: CentroidService, recompute_centroid/recompute_centroids tasks, the
daily beat, POST /api/ml/recompute-centroids, the recompute-on-merge trigger,
the tag_reference_embedding table + model, the centroid_similarity_threshold +
min_reference_images settings (migration 0066), the CentroidRecomputeCard +
its store action + MaintenancePanel tile, and the centroid slider in
MLThresholdSliders. _keep_as_alias drops its vestigial has-centroid branch (the
allowlist branch already covers "could re-emit"); tag merge no longer clears a
table that no longer exists.

NOT touched (still live, parallel to heads): the Camie tagger, ImagePrediction,
and the allowlist bulk-apply — accepting a suggestion still allowlists + applies
it across the library. The tag-eval "centroid" baseline metric is unrelated
(in-memory) and stays. (image_record.centroid_scores JSON column also remains —
separate legacy field, its own micro-cleanup.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
2026-06-30 11:48:09 -04:00
parent 4daa3f2790
commit 3d77a38a25
19 changed files with 78 additions and 508 deletions
+5 -58
View File
@@ -1,9 +1,9 @@
"""ML Celery tasks: per-image inference, backfill discovery, centroid
recompute, allowlist auto-apply, model self-heal.
"""ML Celery tasks: per-image inference, backfill discovery, head training,
allowlist auto-apply, model self-heal.
All run on the ml-worker (queue 'ml') except recompute_centroids and
apply_allowlist_tags sweeps which are 'maintenance' lane. Sync sessions
(Celery workers are sync processes), same pattern as FC-2a tasks.
All run on the ml-worker (queue 'ml') except apply_allowlist_tags sweeps which
are 'maintenance' lane. Sync sessions (Celery workers are sync processes), same
pattern as FC-2a tasks.
"""
import logging
@@ -487,59 +487,6 @@ def _confidence_for_tag(session, tag, preds: dict) -> float | None:
return best
@celery.task(name="backend.app.tasks.ml.recompute_centroid", bind=True)
def recompute_centroid(self, tag_id: int) -> bool:
import asyncio
from ..services.ml.centroids import CentroidService
from ._async_session import async_session_factory
async def _run() -> bool:
# Per-task NullPool engine bound to THIS asyncio.run loop — the shared
# process-wide engine reuses connections across loops and raises
# "Future attached to a different loop" on every call after the first.
async_factory, async_engine = async_session_factory()
try:
async with async_factory() as session:
svc = CentroidService(session)
result = await svc.recompute_for_tag(tag_id)
await session.commit()
return result
finally:
await async_engine.dispose()
return asyncio.run(_run())
@celery.task(
name="backend.app.tasks.ml.recompute_centroids",
bind=True,
# Audit 2026-06-02 — drifted-centroid rebuild over potentially
# hundreds of tags.
soft_time_limit=1800, time_limit=2100,
)
def recompute_centroids(self) -> int:
"""Daily: find drifted centroids, enqueue recompute_centroid for each."""
import asyncio
from ..services.ml.centroids import CentroidService
from ._async_session import async_session_factory
async def _list() -> list[int]:
# Per-task NullPool engine bound to this loop (see recompute_centroid).
async_factory, async_engine = async_session_factory()
try:
async with async_factory() as session:
return await CentroidService(session).list_drifted()
finally:
await async_engine.dispose()
drifted = asyncio.run(_list())
for tid in drifted:
recompute_centroid.delay(tid)
return len(drifted)
@celery.task(
name="backend.app.tasks.ml.tag_eval_run",
bind=True,