Video tag quality: cadence sampling + min-frame aggregation + ML thread cap (#747) #111

Merged

bvandeusen merged 3 commits from dev into main 2026-06-16 14:08:40 -04:00

Conversation 0 Commits 3 Files Changed 9

+269 -43

bvandeusen commented 2026-06-16 14:08:30 -04:00

Owner

Copy Link

Copy Source

Merges #747 (CI-green, tip 60a9c9e) to main / :latest.

• Video tag quality — sample frames at a fixed cadence (video_frame_interval_seconds, default 4) capped at video_max_frames (64); keep a tag only if it appears in ≥ video_min_tag_frames (3) sampled frames, scored by the mean over those frames (not max — that was the noise source). Duration-independent: works across short and long multi-scene videos. All three knobs are tunable in Settings → ML (migration 0053; replaces the VIDEO_ML_FRAMES env var).
• ML thread cap — bound ONNX + torch inference threads to 4 by default so each ml-worker is a predictable core consumer; scale throughput via worker replicas on the shared node (each --concurrency=1 + its own cgroup limit), not by cranking concurrency in one container.
• GPU enablement (#872) was explored and cancelled (Pascal-specific build not worth it); the net effect here is just the thread cap.

One new migration (0053). No new runtime deps.

After merge: force a re-pull of :latest. Then operators can tune the video knobs in Settings → ML, and (optional) scale ml-worker replicas per the recipe in the #747 notes.

🤖 Generated with Claude Code

Merges #747 (CI-green, tip `60a9c9e`) to main / `:latest`. - **Video tag quality** — sample frames at a fixed cadence (`video_frame_interval_seconds`, default 4) capped at `video_max_frames` (64); keep a tag only if it appears in ≥ `video_min_tag_frames` (3) sampled frames, scored by the **mean** over those frames (not max — that was the noise source). Duration-independent: works across short and long multi-scene videos. All three knobs are tunable in Settings → ML (migration 0053; replaces the `VIDEO_ML_FRAMES` env var). - **ML thread cap** — bound ONNX + torch inference threads to 4 by default so each ml-worker is a predictable core consumer; scale throughput via worker **replicas** on the shared node (each `--concurrency=1` + its own cgroup limit), not by cranking concurrency in one container. - GPU enablement (#872) was explored and **cancelled** (Pascal-specific build not worth it); the net effect here is just the thread cap. One new migration (0053). No new runtime deps. **After merge:** force a re-pull of `:latest`. Then operators can tune the video knobs in Settings → ML, and (optional) scale `ml-worker` replicas per the recipe in the #747 notes. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 3 commits 2026-06-16 14:08:30 -04:00

feat(ml): cadence-based video frame sampling + min-frame tag aggregation (#747) ... 369e3de684

Video tag noise root cause: frames were a FIXED count (6) max-pooled — a tag
firing on one frame survived at peak confidence, and a fixed count under-samples
long multi-scene videos so real scene-local tags looked like noise.

Redesign (operator-steered):
- Sample at a fixed CADENCE — one frame every `video_frame_interval_seconds`
  (default 4) across the 5–95% window — so a tag's frame-presence reflects real
  screen time independent of video length. Capped at `video_max_frames` (default
  64): a long video stretches the spacing instead of exploding into hundreds of
  inferences, bounding per-video cost on the single ml-worker (per-frame ffmpeg
  timeout also cut 60s→30s).
- Aggregate with `_aggregate_video_predictions`: keep a tag only if it appears in
  >= `video_min_tag_frames` sampled frames (≈ that many × interval seconds on
  screen — duration-independent noise rejection), with confidence = MEAN over the
  frames it appears in (not max). Clamps the threshold to the sample count so a
  1–2-frame short video still tags.
- All three knobs are DB-backed ml_settings (migration 0053), patchable via
  /api/ml/settings + sliders in the ML settings card — replaces the
  VIDEO_ML_FRAMES env var (product-not-project).

Tests: aggregation drops one-frame noise + means corroborated tags + clamps on
short videos; settings round-trip + min>max validation. Replaced the
_maxpool_predictions unit test.

NOTE: this is the QUALITY half of #747. The perf half — the ml-worker runs
CPU-only — is GPU enablement, tracked separately in #872.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(ml): GPU-capable tagger + embedder with CPU fallback (#872) ... db7e1f2b59

Step 1 of GPU enablement (code only — CPU-safe, CI-green; the CUDA image is a
separate step pending the host driver version).

- New services/ml/device.py: FC_ML_DEVICE (auto|cuda|cpu) intent + VRAM knobs
  (FC_ML_ONNX_GPU_MEM_GB, FC_ML_TORCH_MEM_FRACTION). Per-worker-host bootstrap →
  env, not a DB setting (the GPU host runs CUDA, others CPU).
- tagger: use CUDAExecutionProvider (with gpu_mem_limit) when requested AND the
  provider is actually present (onnxruntime-gpu), else CPUExecutionProvider. Logs
  the active providers.
- embedder: move model + inputs to cuda when requested AND torch.cuda is
  available; cap torch's VRAM share; .detach().cpu() before numpy. fp32 kept so
  GPU embeddings stay in the same space as existing CPU ones.

Both AND the env intent with the framework's real availability, so on CPU
(CI / CPU onnxruntime / no GPU) they fall back cleanly — behavior unchanged.
The 8GB P4 is shared by both frameworks, hence the conservative default caps.

Tests: device env parsing. (tagger/embedder GPU paths are operator-verified on
the GPU host — models aren't in CI.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

refactor(ml): drop GPU code, cap inference threads by default (#747/#872) ... 60a9c9e6ef

GPU enablement (#872) cancelled — not worth the Pascal-specific build for a
modest CPU→GPU win on an old P4. Remove the dead GPU code (device.py, the CUDA
provider branch in tagger, the .to('cuda') path in embedder) so nothing carries
it forward.

Instead, bound CPU inference threads by default so the ml-worker is a predictable
core consumer on a SHARED node — the intended scaling model is multiple worker
replicas (each --concurrency=1, each its own cgroup limit), not one big
container. ONNX Runtime and torch otherwise size their thread pools to ALL host
cores, so each replica would grab every core and oversubscribe / starve the
co-located DB+web. Cap both to _INTRA_OP_THREADS=4 (matches the prior per-worker
cpus:4 unit): run N replicas where N×4 stays within the cores allotted to ML.

- tagger: ort.SessionOptions().intra_op_num_threads = 4 (CPUExecutionProvider).
- embedder: torch.set_num_threads(4).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

bvandeusen merged commit 49bee77cdc into main 2026-06-16 14:08:40 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#111