feat(heads): earned auto-apply — sweep mechanism, off by default (#114 auto-apply A)

Graduated heads can now apply their tag without a human — gated so it's safe:
- FIRING GATE: a head fires only when the master switch (head_auto_apply_enabled,
  default OFF) is on AND it has >= head_auto_apply_min_positives (default 30)
  clean labels. A precise-looking but under-supported low-N head can't spray tags.
- auto_apply_sweep (heads.py): streams every embedded image in chunks, scores
  against the eligible heads (numpy, no sklearn), applies each head's tag where
  score >= its auto_apply_threshold and the tag isn't already applied/rejected,
  with source='head_auto' (distinguishable + reversible). dry_run counts only.
- HeadAutoApplyRun (migration 0059) tracks each sweep / preview; apply_head_tags
  task (ml queue) + scheduled_apply_head_tags daily beat (no-op unless enabled)
  + recovery sweep + retention(20).
- API: POST /api/heads/auto-apply {dry_run} (202 / 409 running / 400 disabled),
  GET /api/heads/auto-apply (recent runs + per-concept report). Settings
  head_auto_apply_enabled + min_positives via /api/ml/settings.

Tests: sweep applies above threshold, dry-run writes nothing, skips under-
supported + ungraduated heads; API disabled/dry-run/conflict guards.

NEXT (slice 2): the observability the operator asked for — per-concept misfire
(auto-applied-then-removed) + under-fire tracking, time-series snapshots, and a
reporting API to tune. Slice 3: the UI (enable, preview, trends).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa

backend/app/tasks/maintenance.py

@@ -13,6 +13,7 @@ from ..celery_app import celery
 from ..models import (
     BackupRun,
     DownloadEvent,
+    HeadAutoApplyRun,
     HeadTrainingRun,
     ImageRecord,
     ImportBatch,
@@ -101,6 +102,9 @@ TAG_EVAL_KEEP_RUNS = 20
 # head training (#114) has a 60-min soft limit; flag no-progress past 75.
 HEAD_TRAINING_STALL_THRESHOLD_MINUTES = 75
 HEAD_TRAINING_KEEP_RUNS = 20
+# head auto-apply (#114) shares the 60-min soft limit; flag past 75.
+HEAD_AUTO_APPLY_STALL_THRESHOLD_MINUTES = 75
+HEAD_AUTO_APPLY_KEEP_RUNS = 20
 # Import batches finalize only after every child ImportTask hits a
 # terminal state. The recovery sweep targets the case where every
 # task is done but the batch never got its closing UPDATE
@@ -800,6 +804,48 @@ def recover_stalled_head_training_runs() -> int:
         return recovered
 
 
+@celery.task(name="backend.app.tasks.maintenance.recover_stalled_head_auto_apply_runs")
+def recover_stalled_head_auto_apply_runs() -> int:
+    """Flip stalled HeadAutoApplyRun 'running' rows to 'error' + prune to the
+    last HEAD_AUTO_APPLY_KEEP_RUNS (retention, rule 89). 5-min maintenance lane."""
+    SessionLocal = _sync_session_factory()
+    now = datetime.now(UTC)
+    cutoff = now - timedelta(minutes=HEAD_AUTO_APPLY_STALL_THRESHOLD_MINUTES)
+    with SessionLocal() as session:
+        result = session.execute(
+            update(HeadAutoApplyRun)
+            .where(HeadAutoApplyRun.status == "running")
+            .where(
+                func.coalesce(
+                    HeadAutoApplyRun.last_progress_at, HeadAutoApplyRun.started_at
+                )
+                < cutoff
+            )
+            .values(
+                status="error", finished_at=now,
+                error=(
+                    f"stranded by recovery sweep (no progress for "
+                    f"{HEAD_AUTO_APPLY_STALL_THRESHOLD_MINUTES} min)"
+                ),
+            )
+        )
+        keep = session.execute(
+            select(HeadAutoApplyRun.id).order_by(HeadAutoApplyRun.id.desc())
+            .limit(HEAD_AUTO_APPLY_KEEP_RUNS)
+        ).scalars().all()
+        if keep:
+            session.execute(
+                delete(HeadAutoApplyRun).where(HeadAutoApplyRun.id.not_in(keep))
+            )
+        session.commit()
+        recovered = result.rowcount or 0
+        if recovered:
+            log.info(
+                "recover_stalled_head_auto_apply_runs: recovered %d rows", recovered
+            )
+        return recovered
+
+
 @celery.task(name="backend.app.tasks.maintenance.recover_stalled_import_batches")
 def recover_stalled_import_batches() -> int:
     """Finalize ImportBatch rows stuck in running past the hard limit