bvandeusen
f154603811
Videos deduped on sha256 only (pHash is images-only), so a different encode/remux of the same clip imported as a distinct record — the "same video from multiple sources" clutter surfaced by #859. Tier-1 metadata fingerprint: identity = container duration (±1.0s) + matching aspect ratio, scoped to the same artist; quality axis = pixel dimensions (mirrors image pHash: larger_exists→skip+link, smaller_exists→supersede). Codec/bitrate are deliberately NOT part of identity (the point is matching across re-encodes). Tight tolerances because a wrong video merge is destructive. - image_record.duration_seconds (Float, nullable; migration 0052). NULL for images. - safe_probe.probe_video also reads format=duration (one extra ffprobe field on the call that already runs); ProbeResult.duration. - _find_similar_video(duration,w,h,artist) shared by both import pipelines. - _import_media (filesystem/archive path): captures duration, video near-dup branch, persists duration. - attach_in_place (download path — handles #859's videos, previously didn't probe video at all): best-effort probe for dims+duration (LENIENT — never newly rejects a downloaded video on probe failure), video near-dup branch, persists duration. - _supersede carries duration onto the kept row. Reuses SkipReason.duplicate_phash so the existing download/external dup-cleanup (path-safe unlink, #859) applies unchanged. Tests: skip-smaller, supersede-larger (+ duration adopted), and distinct-durations-not-merged (false-merge guard). Follow-up (Phase 2, #871): a backfill to re-probe NULL-duration existing videos so the current library participates in dedup; retroactive merge of existing dups is a separate destructive maintenance action. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
33 lines
1.0 KiB
Python
33 lines
1.0 KiB
Python
"""image_record: duration_seconds (Tier-1 video near-dup key)
|
|
|
|
#871. Videos previously deduped on sha256 only (pHash is images-only), so a
|
|
different encode/remux of the same video imported as a distinct record. Persist
|
|
the container duration so the importer can treat same-artist videos with matching
|
|
duration (+ aspect ratio) as the same content and dedup/supersede like images.
|
|
NULL for images and for video rows imported before this column existed (a
|
|
backfill re-probes those so they participate in dedup).
|
|
|
|
Revision ID: 0052
|
|
Revises: 0051
|
|
Create Date: 2026-06-16
|
|
"""
|
|
from typing import Sequence, Union
|
|
|
|
import sqlalchemy as sa
|
|
from alembic import op
|
|
|
|
revision: str = "0052"
|
|
down_revision: Union[str, None] = "0051"
|
|
branch_labels: Union[str, Sequence[str], None] = None
|
|
depends_on: Union[str, Sequence[str], None] = None
|
|
|
|
|
|
def upgrade() -> None:
|
|
op.add_column(
|
|
"image_record", sa.Column("duration_seconds", sa.Float(), nullable=True)
|
|
)
|
|
|
|
|
|
def downgrade() -> None:
|
|
op.drop_column("image_record", "duration_seconds")
|