feat(ml): cadence-based video frame sampling + min-frame tag aggregation (#747)
Video tag noise root cause: frames were a FIXED count (6) max-pooled — a tag firing on one frame survived at peak confidence, and a fixed count under-samples long multi-scene videos so real scene-local tags looked like noise. Redesign (operator-steered): - Sample at a fixed CADENCE — one frame every `video_frame_interval_seconds` (default 4) across the 5–95% window — so a tag's frame-presence reflects real screen time independent of video length. Capped at `video_max_frames` (default 64): a long video stretches the spacing instead of exploding into hundreds of inferences, bounding per-video cost on the single ml-worker (per-frame ffmpeg timeout also cut 60s→30s). - Aggregate with `_aggregate_video_predictions`: keep a tag only if it appears in >= `video_min_tag_frames` sampled frames (≈ that many × interval seconds on screen — duration-independent noise rejection), with confidence = MEAN over the frames it appears in (not max). Clamps the threshold to the sample count so a 1–2-frame short video still tags. - All three knobs are DB-backed ml_settings (migration 0053), patchable via /api/ml/settings + sliders in the ML settings card — replaces the VIDEO_ML_FRAMES env var (product-not-project). Tests: aggregation drops one-frame noise + means corroborated tags + clamps on short videos; settings round-trip + min>max validation. Replaced the _maxpool_predictions unit test. NOTE: this is the QUALITY half of #747. The perf half — the ml-worker runs CPU-only — is GPU enablement, tracked separately in #872. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -30,6 +30,39 @@
|
||||
</div>
|
||||
</v-col>
|
||||
</v-row>
|
||||
|
||||
<v-divider class="my-4" />
|
||||
|
||||
<div class="text-subtitle-2 mb-1">Video tagging</div>
|
||||
<div class="text-caption fc-muted mb-3">
|
||||
Videos are tagged by sampling frames at a fixed cadence. A tag is kept
|
||||
only if it shows up in enough frames (≈ that many × the interval in
|
||||
seconds of screen time), which filters one-frame noise without losing
|
||||
tags that only appear in part of a longer video.
|
||||
</div>
|
||||
<v-row>
|
||||
<v-col cols="12" sm="4">
|
||||
<v-text-field
|
||||
v-model.number="local.video_frame_interval_seconds"
|
||||
label="Frame interval (s)" type="number" min="0.5" step="0.5"
|
||||
density="comfortable" hide-details @change="save"
|
||||
/>
|
||||
</v-col>
|
||||
<v-col cols="12" sm="4">
|
||||
<v-text-field
|
||||
v-model.number="local.video_max_frames"
|
||||
label="Max frames" type="number" min="1" step="1"
|
||||
density="comfortable" hide-details @change="save"
|
||||
/>
|
||||
</v-col>
|
||||
<v-col cols="12" sm="4">
|
||||
<v-text-field
|
||||
v-model.number="local.video_min_tag_frames"
|
||||
label="Min frames per tag" type="number" min="1" step="1"
|
||||
density="comfortable" hide-details @change="save"
|
||||
/>
|
||||
</v-col>
|
||||
</v-row>
|
||||
</v-card-text>
|
||||
<v-card-text v-else><v-skeleton-loader type="paragraph" /></v-card-text>
|
||||
</v-card>
|
||||
@@ -59,9 +92,14 @@ async function save() {
|
||||
const floor = local.tagger_store_floor
|
||||
local.suggestion_threshold_character = Math.max(local.suggestion_threshold_character, floor)
|
||||
local.suggestion_threshold_general = Math.max(local.suggestion_threshold_general, floor)
|
||||
// Mirror the server invariant: a tag can't require more frames than are sampled.
|
||||
local.video_min_tag_frames = Math.min(local.video_min_tag_frames, local.video_max_frames)
|
||||
const patch = {}
|
||||
for (const f of fields) patch[f.key] = local[f.key]
|
||||
patch.tagger_store_floor = local.tagger_store_floor
|
||||
patch.video_frame_interval_seconds = local.video_frame_interval_seconds
|
||||
patch.video_max_frames = local.video_max_frames
|
||||
patch.video_min_tag_frames = local.video_min_tag_frames
|
||||
try { await store.patchSettings(patch) }
|
||||
catch (e) { toast({ text: e.message, type: 'error' }) }
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user