fix(ml): video branch needs longer time limits; recovery sweep is now per-queue

Operator-flagged 2026-05-28: tag_and_embed on image 6288 (an mp4) was
marked failed by recover_stalled_task_runs at the 5-min sweep tick
while still legitimately running. The error_type='RecoverySweep' /
"no completion signal received within 5 min" message was misleading
— the worker was busy, not stuck.

Root cause is two interacting limits, both undersized for video work:

  tag_and_embed: soft_time_limit=300, time_limit=420
                 (sized for the image branch, ≈2 GPU ops)
  recovery sweep: STUCK_THRESHOLD_MINUTES = 5 across all queues

The video branch samples 10 frames via ffmpeg, then runs tagger +
embedder on EACH frame — ~20 GPU ops vs 2 for an image. A loaded
ml-worker can take 5-10 min on a long video, which trips both
limits well before the task naturally finishes.

**Two-part fix**

1. `tag_and_embed` time limits bumped to soft=900 (15 min) / time=1200
   (20 min). Sized for the video path's worst case; image runs return
   in seconds and don't care.

2. New `QUEUE_STUCK_THRESHOLD_MINUTES` override dict in maintenance.py.
   Queues with legitimately-long-running tasks (currently just `ml` at
   25 min — 5-min buffer past the new hard kill) get their own
   threshold; queues not in the dict use the default 5 min. The sweep
   now issues one UPDATE per distinct threshold value, with
   `queue.notin_(override_queues)` on the default pass so each row is
   touched at most once.

Tests:
- _make_task_run helper accepts `queue=` (defaults to "default") so
  existing tests use the default-threshold path.
- New test `test_recover_stalled_task_runs_ml_queue_uses_longer_threshold`
  pins both directions: a 10-min-old ml row survives (fresh by 25-min
  override), a 30-min-old ml row gets flagged.

After deploy, operator's mp4 ML jobs run to completion without
spurious RecoverySweep failures.

backend/app/tasks/ml.py

@@ -31,8 +31,15 @@ def _is_video(path: Path) -> bool:
     retry_backoff_max=60,
     retry_jitter=True,
     max_retries=3,
-    soft_time_limit=300,
-    time_limit=420,
+    # Sized for the video branch: sample 10 frames, run tagger +
+    # embedder on each (≈20 GPU ops vs 2 for an image). A loaded
+    # ml-worker can take 5-10 min on a long video; bumped from
+    # 5min/7min on 2026-05-28 after operator-flagged image 6288 (a
+    # .mp4) hit the recovery sweep at 5 min while still legitimately
+    # processing. Image runs return in seconds; the bump doesn't
+    # affect their UX.
+    soft_time_limit=900,   # 15 min
+    time_limit=1200,       # 20 min hard
 )
 def tag_and_embed(self, image_id: int) -> dict:
     """Run Camie + SigLIP on one image; store predictions + embedding;