feat: duplicates settle themselves: retitled copies across releases link, alike versions on one album are dismissed, and a file in another song's slot is removed so the album is fetched again (M498 #6159-#6162)
release / govulncheck (push) Successful in 14s
release / web (push) Failing after 1m26s
release / go (push) Successful in 1m35s
release / integration (push) Successful in 5m8s
release / android (push) Successful in 5m12s
release / Build signed APK (releases and dev) (push) Successful in 5m26s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
release / govulncheck (push) Successful in 14s
release / web (push) Failing after 1m26s
release / go (push) Successful in 1m35s
release / integration (push) Successful in 5m8s
release / android (push) Successful in 5m12s
release / Build signed APK (releases and dev) (push) Successful in 5m26s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
The operator's library had 536 groups waiting for review, 2026-10-19. A sample
showed most were one recording under two titles on two releases.
- The headline and the Duplicates count leave out cross-release groups, which
the resolver keeps and links: the headline read 3,479 when 536 needed review.
- Any group spanning albums is cross_release and is linked as one song,
whatever its titles say. Nothing is deleted.
- One album, two versions named as such ("Lovesick (Instrumental)") and not
byte-identical: a new class, distinct, dismissed by the resolver.
- One album, one recording under different song titles (mismatch): the
album's cached MusicBrainz track list decides. When exactly one slot's song
is as long as the file and every other slot's song is clearly not, the other
copies are removed and their tracks marked missing, so the reacquisition
sweeper wants the album back. Otherwise the group stays for review.
- The duplicates_found notice moves from the sweep to the resolver: it names
only groups a pass first left for review.
- Resolved automatically lists dismissals and wrong-slot removals beside
merges.
Migration 0094 adds 'distinct' to the class CHECK (rule 36).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -6,21 +6,26 @@ import (
|
||||
"unicode"
|
||||
)
|
||||
|
||||
// Duplicate classification (M498 #5435).
|
||||
// Duplicate classification (M498 #5435; settled further 2026-10-19).
|
||||
//
|
||||
// A duplicate group is one recording found more than once. What should happen
|
||||
// to it depends on where the copies sit, and the measured library (milestone
|
||||
// 498's body) splits into four shapes:
|
||||
// to it depends on where the copies sit:
|
||||
//
|
||||
// - same_release: every copy is on one album. An album holds a song once, so
|
||||
// the extra copies are the ones to fold away.
|
||||
// - cross_release: the copies are on different albums under one title. A
|
||||
// single and the album it came from, a deluxe edition and the standard one:
|
||||
// each copy belongs to its own release, so none is deleted (unless it is a
|
||||
// video rip; the resolver decides).
|
||||
// - mismatch: byte-identical audio on different albums under different
|
||||
// titles. One file carries another song's tags — a wrong-file import.
|
||||
// - review: anything else. The operator decides.
|
||||
// - same_release: every copy is on one album under one title. An album holds
|
||||
// a song once, so the extra copies are the ones to fold away.
|
||||
// - cross_release: the copies are on different albums. A single and the
|
||||
// album it came from, a deluxe edition and the standard one, or one
|
||||
// recording a later release retitled ("The Ragtime Cat", "F.O.D. (live)",
|
||||
// a romanized title): each copy belongs to its own release, so none is
|
||||
// deleted (unless it is a video rip; the resolver decides). The measured
|
||||
// library (2026-10-19) had 451 such groups waiting on the operator under
|
||||
// different titles, and they were retitles, not different songs.
|
||||
// - distinct: one album, two versions of a song that sound alike, named as
|
||||
// such ("Lovesick", "Lovesick (Instrumental)"). Not duplicates.
|
||||
// - mismatch: one album holds the same audio under different song titles:
|
||||
// one file filling several track slots, a broken download. Which slot is
|
||||
// right is the resolver's to decide from MusicBrainz's track lengths.
|
||||
// - review: what none of these fits. The operator decides.
|
||||
|
||||
// DuplicateClass is a group's shape. The values are duplicate_groups.class.
|
||||
type DuplicateClass string
|
||||
@@ -28,6 +33,7 @@ type DuplicateClass string
|
||||
const (
|
||||
ClassSameRelease DuplicateClass = "same_release"
|
||||
ClassCrossRelease DuplicateClass = "cross_release"
|
||||
ClassDistinct DuplicateClass = "distinct"
|
||||
ClassMismatch DuplicateClass = "mismatch"
|
||||
ClassReview DuplicateClass = "review"
|
||||
)
|
||||
@@ -44,17 +50,17 @@ type ClassifyMember struct {
|
||||
//
|
||||
// Titles are compared by MatchTitleKey, which drops what a release or an upload
|
||||
// adds to a title (featuring credits, remaster notes, video markers) and keeps
|
||||
// what names a different recording (live, demo, remix, instrumental). So one
|
||||
// album holding "WWW" and "WWW (instrumental)" as an acoustic match is review,
|
||||
// not a copy to fold away: the #3885 case, where they were in fact different
|
||||
// recordings mislabelled. Byte-identical copies on one album are the same file
|
||||
// twice whatever their tags say.
|
||||
// what names a different recording (live, demo, remix, instrumental). On one
|
||||
// album, titles that differ only by such a word are two versions (distinct)
|
||||
// unless the bytes are identical, which is one file twice: the #3885 case of
|
||||
// "WWW" and "WWW (instrumental)" holding the same file is a wrong slot.
|
||||
func ClassifyDuplicateGroup(tier string, members []ClassifyMember) DuplicateClass {
|
||||
if len(members) < 2 {
|
||||
return ClassReview
|
||||
}
|
||||
oneAlbum, oneTitle := true, true
|
||||
oneAlbum, oneTitle, oneSong := true, true, true
|
||||
key := MatchTitleKey(members[0].Title, members[0].ArtistName)
|
||||
song := songKey(members[0].Title, members[0].ArtistName)
|
||||
for _, m := range members[1:] {
|
||||
if m.AlbumID != members[0].AlbumID {
|
||||
oneAlbum = false
|
||||
@@ -62,22 +68,33 @@ func ClassifyDuplicateGroup(tier string, members []ClassifyMember) DuplicateClas
|
||||
if MatchTitleKey(m.Title, m.ArtistName) != key {
|
||||
oneTitle = false
|
||||
}
|
||||
if songKey(m.Title, m.ArtistName) != song {
|
||||
oneSong = false
|
||||
}
|
||||
}
|
||||
exact := tier == string(tierExact)
|
||||
switch {
|
||||
case oneAlbum && (oneTitle || exact):
|
||||
return ClassSameRelease
|
||||
case oneAlbum:
|
||||
return ClassReview
|
||||
case oneTitle:
|
||||
case !oneAlbum:
|
||||
return ClassCrossRelease
|
||||
case exact:
|
||||
return ClassMismatch
|
||||
case oneTitle:
|
||||
return ClassSameRelease
|
||||
case oneSong && !exact:
|
||||
return ClassDistinct
|
||||
default:
|
||||
return ClassReview
|
||||
return ClassMismatch
|
||||
}
|
||||
}
|
||||
|
||||
// songKey is MatchTitleKey with every bracketed segment dropped: the song a
|
||||
// title names, whatever version of it. "Lovesick (Instrumental)" and
|
||||
// "Lovesick" share one.
|
||||
func songKey(title, artist string) string {
|
||||
if key := MatchTitleKey(bracketed.ReplaceAllString(title, " "), artist); key != "" {
|
||||
return key
|
||||
}
|
||||
return MatchTitleKey(title, artist)
|
||||
}
|
||||
|
||||
var (
|
||||
// A bracketed credit: "(feat. Panda Bear)", "[ft. X]", "(featuring Y)".
|
||||
featBracket = regexp.MustCompile(`(?i)[(\[][^)\]]*\b(feat\.?|ft\.?|featuring)\s[^)\]]*[)\]]`)
|
||||
|
||||
Reference in New Issue
Block a user