feat(ml): cadence-based video frame sampling + min-frame tag aggregation (#747)
Video tag noise root cause: frames were a FIXED count (6) max-pooled — a tag firing on one frame survived at peak confidence, and a fixed count under-samples long multi-scene videos so real scene-local tags looked like noise. Redesign (operator-steered): - Sample at a fixed CADENCE — one frame every `video_frame_interval_seconds` (default 4) across the 5–95% window — so a tag's frame-presence reflects real screen time independent of video length. Capped at `video_max_frames` (default 64): a long video stretches the spacing instead of exploding into hundreds of inferences, bounding per-video cost on the single ml-worker (per-frame ffmpeg timeout also cut 60s→30s). - Aggregate with `_aggregate_video_predictions`: keep a tag only if it appears in >= `video_min_tag_frames` sampled frames (≈ that many × interval seconds on screen — duration-independent noise rejection), with confidence = MEAN over the frames it appears in (not max). Clamps the threshold to the sample count so a 1–2-frame short video still tags. - All three knobs are DB-backed ml_settings (migration 0053), patchable via /api/ml/settings + sliders in the ML settings card — replaces the VIDEO_ML_FRAMES env var (product-not-project). Tests: aggregation drops one-frame noise + means corroborated tags + clamps on short videos; settings round-trip + min>max validation. Replaced the _maxpool_predictions unit test. NOTE: this is the QUALITY half of #747. The perf half — the ml-worker runs CPU-only — is GPU enablement, tracked separately in #872. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,49 @@
|
||||
"""ml_settings: video tagging knobs (cadence sampling + noise floor)
|
||||
|
||||
#747. Video tag quality/perf: sample frames at a fixed cadence (interval) so a
|
||||
tag's frame-presence reflects real screen time, cap total frames so long videos
|
||||
stay bounded, and keep a tag only if it appears in >= min_tag_frames sampled
|
||||
frames. Operator-tunable via Settings → ML (replaces the VIDEO_ML_FRAMES env var).
|
||||
|
||||
Revision ID: 0053
|
||||
Revises: 0052
|
||||
Create Date: 2026-06-16
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "0053"
|
||||
down_revision: Union[str, None] = "0052"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"ml_settings",
|
||||
sa.Column(
|
||||
"video_frame_interval_seconds", sa.Float(), nullable=False,
|
||||
server_default="4.0",
|
||||
),
|
||||
)
|
||||
op.add_column(
|
||||
"ml_settings",
|
||||
sa.Column(
|
||||
"video_max_frames", sa.Integer(), nullable=False, server_default="64",
|
||||
),
|
||||
)
|
||||
op.add_column(
|
||||
"ml_settings",
|
||||
sa.Column(
|
||||
"video_min_tag_frames", sa.Integer(), nullable=False,
|
||||
server_default="3",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("ml_settings", "video_min_tag_frames")
|
||||
op.drop_column("ml_settings", "video_max_frames")
|
||||
op.drop_column("ml_settings", "video_frame_interval_seconds")
|
||||
Reference in New Issue
Block a user