M400: acoustic duplicate detection, history-preserving merge, and fingerprinting settings #134
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: bvandeusen/minstrel#134
Reference in New Issue
Block a user
Eight commits. CI green on
516413f4(validators, web, and the build with the container image pushed).What this adds
Duplicate detection (#3909, #3910, #3912) — a matcher comparing stored chromaprints by offset voting, a streaming sweep that proposes groups in duration order without holding the library in memory, and an admin Duplicates report. Two tiers: identical audio (hash equality, no threshold) and same recording (acoustic, above the match threshold). A group is a proposal; nothing is removed without being asked.
Merge (#3911) — keeps one copy and moves every play, skip, like, playlist entry, tag and similarity edge onto it before deleting the losers' files. Refuses to proceed if a file can't be removed, so history is never dropped for a copy that survives on disk.
Move detection by audio (#3914) — a moved or retagged file is recognised by its audio stream hash instead of the old size-and-duration guess, which could both miss re-encodes and match unrelated files.
Fingerprinting settings (#3913) — on/off, seconds of audio, match threshold, files at once, and hours between sweeps, in a card on the Duplicates page. Each fingerprint records the length it was taken at, and only fingerprints at the current length are compared, so changing the length can't leave the library holding prints that silently never match.
Two re-acquisition fixes (#3936, #3937) — settings saved in the admin card now reach the running sweeper without a restart, and a refused save shows the field and range the server named instead of a generic failure.
Deploy notes
Three migrations run on deploy:
0059duplicate groups and sweeps,0060aplay_eventsindex ontrack_id(the merge needs it),0061fingerprint settings plus the per-row fingerprint length.The fingerprint backfill and duplicate sweep are background workers, both operator-tunable. The sweep only runs when fingerprints have changed since the last one.
🤖 Generated with Claude Code
https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
Merge keeps one copy of a duplicate group and removes the rest. Every table that references tracks does so ON DELETE CASCADE, so deleting a duplicate's row outright would silently destroy its likes, plays, playlist entries and tags. The merge moves all of that onto the kept copy first, then deletes the empty row. In one transaction, holding a lock on the group: - repoints play_events, skip_events, contextual_likes, playback_errors, lidarr_requests.matched_track_id and playlist_tracks. The last is keyed by position, so every entry stays where it was. - merges general_likes one per user, dated to the earlier like - takes the union of track_tags, keeping the kept copy's own weight on a shared tag - rewrites track_similarity onto the kept copy, dropping edges that would point a track at itself and keeping the kept copy's existing edge on a collision - lets the kept copy take a recording MBID only the removed copy had - deletes the removed copies' rows, tidies emptied albums and artists, marks the group merged - logs sync changes: track deletes, and like and playlist-track delete/upsert pairs The removed copies' files are deleted first, before any row changes, through the same helper as DeleteTrackFile (now shared, along with the album tidy-up). A merge that left the file behind would be undone by the next scan re-importing it. An unwritable library answers 409 library_not_writable and nothing changes. tracks.Service.MergeDuplicates wraps it with the opt-in Lidarr unmonitor from RemoveTrack, skipped when the removed copy is a second file of the kept copy's own album track: unmonitoring that would stop Lidarr managing the kept file. It writes a duplicate_merge audit row after commit, per the audit package's best-effort contract, naming both paths. POST /api/admin/library/duplicates/{id}/merge takes an optional survivor_track_id (the report's proposal otherwise) and unmonitor. On the report page: - each copy gets a Keep choice, defaulting to the proposed one - Merge needs a second click, on a button that says how many files it removes, with the consequence stated beside an opt-in Lidarr checkbox Integration tests cover: - every piece of history landing on the kept copy exactly: likes deduped at the earlier time, plays and skips counted, playlist position unchanged, tags unioned, similarity rewritten with no duplicate or self-edge, MBID inherited - the removed file gone, and a second merge refused - an unwritable file leaving likes, plays, row and group untouched - a survivor outside the group refused Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH