feat: duplicates resolve themselves without Lidarr (M502 #5517)
release / govulncheck (push) Successful in 37s
release / web (push) Successful in 1m13s
release / go (push) Successful in 1m29s
release / integration (push) Successful in 4m46s
release / android (push) Successful in 5m36s
release / Build signed APK (releases and dev) (push) Successful in 5m54s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 6m48s
release / Verify release artifacts (tag releases only) (push) Skipped

Lidarr is off, and with it the resolver had stopped merging: it only
merged when Lidarr had said which copies it mapped. Minstrel is now the
only program writing the library, so the rule is about the copies:

- A song held twice on one album: the extra copies merge into the one
  whose tags fit best (then lossless, larger, older).
- Across releases or under another title: when every copy but one is a
  video rip (held back, or named like one and not marked fine), the rips
  merge into the one that is not.
- Clean copies on different releases are linked as one song, as before.
- Anything else waits for the operator.

Gone with Lidarr: release switches and their settle window, the
unmapped-file read, the rip search, the merge guard and its 409/503,
lidarr.Client's release methods, and duplicate_group_members.lidarr_state
(migration 0078). The duplicates page drops the per-copy Lidarr badge
and the blocked-merge notice; past release changes stay in its history.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 20:12:49 -04:00
co-authored by Claude Opus 5.5
parent 8bd48b4302
commit 5306a261c1
27 changed files with 301 additions and 1743 deletions
+93 -556
View File
@@ -2,12 +2,9 @@ package library
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"sort"
"strings"
"time"
"github.com/jackc/pgx/v5"
@@ -16,100 +13,46 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/audit"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/lidarr"
"git.fabledsword.com/bvandeusen/minstrel/internal/notifications"
syncpkg "git.fabledsword.com/bvandeusen/minstrel/internal/sync"
)
// Duplicate resolver (M498 #5435, #5437).
// Duplicate resolver (M498 #5435, #5437; without Lidarr since M502 #5517).
//
// The sweep proposes duplicate groups; the resolver decides what each one is
// and settles the ones that are safe to settle without asking. Safety is
// Lidarr's to define: Lidarr maps one file to each track of the release it
// monitors, and re-downloads any mapped file that disappears. So:
// and settles the ones that are safe to settle without asking. Minstrel is the
// only program writing the library, so no copy is kept because something else
// depends on it. What decides is what the copies are:
//
// - A copy Lidarr maps is never removed.
// - On one album, copies Lidarr does not map are merged into the copy it does
// (or, when it maps none, into the best by ProposeSurvivor).
// - When Lidarr maps every copy on one album, the release it monitors lists
// the song twice: the Humanz box set, 14 vinyl sides then the digital
// medium. The resolver moves Lidarr to a release of that album that lists
// each song once. Lidarr then rescans the folder, maps one copy and leaves
// the rest unmapped, and the next pass merges them.
//
// The release change has a fixed point (lesson #4183): it happens only while
// the monitored release repeats a title, and the release it chooses repeats
// none, so the next pass finds nothing to change.
// LidarrLibrary is the part of Lidarr's API the resolver uses. *lidarr.Client
// satisfies it.
type LidarrLibrary interface {
ListUnmappedTrackFiles(ctx context.Context) ([]lidarr.TrackFile, error)
LookupAlbumByMBID(ctx context.Context, mbid string) (lidarr.LidarrAlbum, error)
ListAlbumTracks(ctx context.Context, albumID int) ([]lidarr.ReleaseTrack, error)
GetAlbumReleases(ctx context.Context, albumID int) ([]lidarr.AlbumRelease, error)
ListReleaseTracks(ctx context.Context, releaseID int) ([]lidarr.ReleaseTrack, error)
SetMonitoredRelease(ctx context.Context, albumID, releaseID int) error
SearchAlbums(ctx context.Context, albumIDs []int) error
}
// - On one album, a song is held once: the extra copies are merged into the
// best by ProposeSurvivor.
// - Across releases, each copy belongs to its own release (a single and its
// album): nothing is removed, and the copies are linked as one song.
// - Except that a video rip is no release's copy. When every copy but one is
// a rip, across releases or under another title, the rips are merged into
// the one that is not.
// - Anything else waits for the operator.
const (
// resolveMergeCap bounds the merges one pass makes. The first pass after
// deploy meets the whole backlog (433 groups measured on 2026-10-09); a
// cap spreads it over a few hours, each pass short.
// deploy meets the whole backlog; a cap spreads it over a few hours, each
// pass short.
resolveMergeCap = 200
// resolveReleaseCap bounds the Lidarr release changes one pass makes. Each
// makes Lidarr rescan an artist folder.
resolveReleaseCap = 10
// releaseChangeSettle is how long after a release change the resolver
// leaves that album's groups alone. Lidarr unmaps every file of the album
// and rescans; until the rescan lands, every copy reads as unmapped, and a
// merge then could remove the very file Lidarr is about to map.
releaseChangeSettle = 24 * time.Hour
// duplicateResolveTick is how often the worker runs a pass.
duplicateResolveTick = time.Hour
)
const (
lidarrStateTracked = "tracked"
lidarrStateUnmapped = "unmapped"
)
// ReleaseChange is one Lidarr release the resolver switched.
type ReleaseChange struct {
AlbumTitle string
ArtistName string
From, To lidarr.AlbumRelease
}
// DuplicateResolveResult tallies one pass.
type DuplicateResolveResult struct {
Groups int
Classes map[DuplicateClass]int
LidarrConsulted bool
Merged int
MergeFailed int
ReleaseChanges []ReleaseChange
Groups int
Classes map[DuplicateClass]int
Merged int
MergeFailed int
// SongsLinked counts cross-release groups newly linked as one song.
SongsLinked int
// HeldBack counts tracks newly held back from radio and the mixes as video
// rips; Released counts held-back tracks whose name no longer says so.
HeldBack, Released int64
// RipSearches counts albums Lidarr was asked to search for a better copy
// of a rip that has no clean copy (#5447).
RipSearches int
}
// LidarrPathKey is the part of a path Minstrel and Lidarr agree on: the last
// three components (artist folder, album folder, file). Both see the library
// under their own mount; on the operator's deploy both happen to say /music,
// and the key keeps the match from depending on that.
func LidarrPathKey(p string) string {
parts := strings.Split(strings.Trim(p, "/"), "/")
if len(parts) > 3 {
parts = parts[len(parts)-3:]
}
return strings.Join(parts, "/")
}
// resolveGroup is one pending group and its present copies.
@@ -119,53 +62,52 @@ type resolveGroup struct {
members []dbq.ListDuplicateGroupsForResolveRow
class DuplicateClass
note string
states []string // per member: tracked, unmapped, or "" when Lidarr was not asked
}
// autoMergeable says whether the resolver may merge the group on its own: when
// every copy it would remove is one Lidarr does not map (D-a rule 1). On one
// album that is any group with at most one mapped copy. Across releases each
// mapped copy fulfils its own release (D-b), so only a group with exactly one
// mapped copy qualifies: the others fulfil nothing — a rip beside the clean
// copy on another release, or a wrong-file import Lidarr holds unmapped
// (#5439) — and the mapped copy is the clear one to keep.
func autoMergeable(g *resolveGroup) bool {
// isRip says whether a copy is a video rip: held back as one, or named like
// one and not kept by the operator. The name is read as well as the verdict
// because a pass reads the groups before it flags new rips.
func isRip(m dbq.ListDuplicateGroupsForResolveRow) bool {
if m.SourceVerdict != nil {
return *m.SourceVerdict == "suspect"
}
return len(SourceMarkersFor(m.FilePath)) > 0
}
// lone returns the index of the group's only copy that is not a rip, or -1
// when there is none or more than one.
func (g *resolveGroup) lone() int {
at := -1
for i, m := range g.members {
if isRip(m) {
continue
}
if at >= 0 {
return -1
}
at = i
}
return at
}
// mergePlan says whether the resolver may merge the group on its own, and into
// which copy (an index into members; -1 means ProposeSurvivor decides).
func mergePlan(g *resolveGroup) (ok bool, survivor int) {
switch g.class {
case ClassSameRelease:
return g.tracked() <= 1
return true, -1
case ClassCrossRelease, ClassMismatch:
return g.tracked() == 1
default:
return false
}
}
// settling says whether any of the group's albums had its Lidarr release
// changed recently, so Lidarr may still be remapping its files.
func (g *resolveGroup) settling(albums map[string]bool) bool {
for _, m := range g.members {
if albums[syncpkg.FormatUUID(m.AlbumID)] {
return true
if i := g.lone(); i >= 0 {
return true, i
}
}
return false
return false, -1
}
func (g *resolveGroup) tracked() int {
n := 0
for _, s := range g.states {
if s == lidarrStateTracked {
n++
}
}
return n
}
// ResolveDuplicates runs one pass. lid is nil when Lidarr is disabled; act is
// the operator's auto-resolve setting. Without Lidarr, or with act off, the
// pass only classifies: nothing is removed when Lidarr cannot say what is safe.
// ResolveDuplicates runs one pass. act is the operator's auto-resolve setting;
// with it off, the pass only classifies.
func ResolveDuplicates(
ctx context.Context, pool *pgxpool.Pool, logger *slog.Logger, dataDir string, lid LidarrLibrary, act bool,
ctx context.Context, pool *pgxpool.Pool, logger *slog.Logger, dataDir string, act bool,
) (DuplicateResolveResult, error) {
if logger == nil {
logger = slog.Default()
@@ -180,32 +122,10 @@ func ResolveDuplicates(
groups := foldResolveGroups(rows)
res.Groups = len(groups)
var unmapped map[string]bool
if lid != nil {
files, err := lid.ListUnmappedTrackFiles(ctx)
if err != nil {
logger.Warn("duplicate resolve: Lidarr's unmapped files unavailable; classifying only", "err", err)
} else {
unmapped = make(map[string]bool, len(files))
for _, f := range files {
unmapped[LidarrPathKey(f.Path)] = true
}
res.LidarrConsulted = true
}
}
for _, g := range groups {
cm := make([]ClassifyMember, len(g.members))
g.states = make([]string, len(g.members))
for i, m := range g.members {
cm[i] = ClassifyMember{AlbumID: syncpkg.FormatUUID(m.AlbumID), Title: m.Title, ArtistName: m.ArtistName}
if res.LidarrConsulted {
if unmapped[LidarrPathKey(m.FilePath)] {
g.states[i] = lidarrStateUnmapped
} else {
g.states[i] = lidarrStateTracked
}
}
}
g.class = ClassifyDuplicateGroup(g.tier, cm)
g.note = resolveNote(g)
@@ -219,31 +139,15 @@ func ResolveDuplicates(
return res, nil
}
// Holding rips back and linking songs remove nothing and need nothing from
// Lidarr.
if res.HeldBack, err = q.FlagSuspectSourceTracks(ctx, SuspectSourcePattern); err != nil {
return res, fmt.Errorf("hold back video rips: %w", err)
}
if res.Released, err = q.ClearStaleSuspectSourceTracks(ctx, SuspectSourcePattern); err != nil {
return res, fmt.Errorf("release renamed tracks: %w", err)
}
linked, err := linkCrossRelease(ctx, pool, logger, groups)
res.SongsLinked = linked
if err != nil {
return res, err
}
if !res.LidarrConsulted {
return res, nil
}
settling, err := recentlyChangedAlbums(ctx, q, time.Now())
if err != nil {
return res, err
}
// Merges first, on the state Lidarr reported at the start of the pass.
removable := func(p string) bool { return unmapped[LidarrPathKey(p)] }
// Merges before links, so a rip merged away is not linked first.
merged := map[pgtype.UUID]bool{}
for _, g := range groups {
if res.Merged >= resolveMergeCap {
break
@@ -251,35 +155,29 @@ func ResolveDuplicates(
if ctx.Err() != nil {
return res, ctx.Err()
}
if !autoMergeable(g) || g.settling(settling) {
ok, survivor := mergePlan(g)
if !ok {
continue
}
if err := autoMerge(ctx, pool, logger, dataDir, g, removable); err != nil {
if err := autoMerge(ctx, pool, logger, dataDir, g, survivor); err != nil {
res.MergeFailed++
logger.Warn("duplicate resolve: merge failed", "group_id", syncpkg.FormatUUID(g.id), "err", err)
continue
}
merged[g.id] = true
res.Merged++
}
// Then release changes, for albums where Lidarr maps every copy.
changes, err := changeRepeatingReleases(ctx, q, pool, logger, lid, groups, settling)
res.ReleaseChanges = changes
linked, err := linkCrossRelease(ctx, pool, logger, groups, merged)
res.SongsLinked = linked
if err != nil {
return res, err
}
// Last, rips with no clean copy anywhere: ask Lidarr for something better.
searched, err := searchSoleCopyRips(ctx, q, pool, logger, lid)
res.RipSearches = searched
if err != nil {
logger.Warn("duplicate resolve: rip search failed", "err", err)
}
if res.Merged > 0 || len(changes) > 0 {
if res.Merged > 0 {
notifyAdmins(ctx, notifications.KindDuplicatesResolved, notifications.Payload{
Count: int64(res.Merged + len(changes)),
Detail: resolvedDetail(res.Merged, changes),
Count: int64(res.Merged),
Detail: fmt.Sprintf("Merged %d duplicate %s.", res.Merged, pluralWord(res.Merged, "copy", "copies")),
})
}
return res, nil
@@ -287,13 +185,15 @@ func ResolveDuplicates(
// linkCrossRelease links each cross-release group's copies as one song
// (#5438): a like on one is a like on all, and the Liked list, shuffle and the
// mixes count the song once. Linking an already-linked group changes nothing,
// mixes count the song once. A group the pass merged has nothing left to link. Linking an already-linked group changes nothing,
// so this runs every pass; likes are shared only when a link is new, since a
// like made after it reaches every copy as it is made.
func linkCrossRelease(ctx context.Context, pool *pgxpool.Pool, logger *slog.Logger, groups []*resolveGroup) (int, error) {
func linkCrossRelease(
ctx context.Context, pool *pgxpool.Pool, logger *slog.Logger, groups []*resolveGroup, merged map[pgtype.UUID]bool,
) (int, error) {
linked := 0
for _, g := range groups {
if g.class != ClassCrossRelease {
if g.class != ClassCrossRelease || merged[g.id] {
continue
}
if ctx.Err() != nil {
@@ -345,61 +245,6 @@ func linkSong(ctx context.Context, pool *pgxpool.Pool, ids []pgtype.UUID) (bool,
return true, tx.Commit(ctx)
}
// ripSearchCap bounds the albums one pass asks Lidarr to search, so the first
// pass after deploy does not queue the whole backlog of rips at once.
const ripSearchCap = 10
// searchSoleCopyRips asks Lidarr to search each album holding a video rip that
// is the only copy of its song (#5447, operator's choice: option 2). While
// Lidarr maps the rip it counts the track fulfilled, so nothing else asks for
// better. An album search grabs only when the quality profile allows an
// upgrade, so this removes nothing and may change nothing: the rip stays held
// back either way. An album is searched at most once a week (the query reads
// the audit log), and an album Lidarr does not hold is skipped.
func searchSoleCopyRips(
ctx context.Context, q *dbq.Queries, pool *pgxpool.Pool, logger *slog.Logger, lid LidarrLibrary,
) (int, error) {
albums, err := q.ListAlbumsWithSoleCopyRips(ctx, ripSearchCap)
if err != nil {
return 0, fmt.Errorf("list albums with sole-copy rips: %w", err)
}
type found struct {
row dbq.ListAlbumsWithSoleCopyRipsRow
lidarrID int
}
var targets []found
for _, a := range albums {
la, err := lid.LookupAlbumByMBID(ctx, a.ReleaseGroupMbid)
if errors.Is(err, lidarr.ErrNotFound) {
continue
}
if err != nil {
return 0, fmt.Errorf("look up %s: %w", a.ReleaseGroupMbid, err)
}
targets = append(targets, found{row: a, lidarrID: la.ID})
}
if len(targets) == 0 {
return 0, nil
}
ids := make([]int, len(targets))
for i, t := range targets {
ids[i] = t.lidarrID
}
if err := lid.SearchAlbums(ctx, ids); err != nil {
return 0, fmt.Errorf("search albums: %w", err)
}
for _, t := range targets {
audit.WriteOrLog(ctx, pool, logger, pgtype.UUID{}, pgtype.UUID{}, audit.ActionLidarrRipSearch, map[string]any{
"album_id": syncpkg.FormatUUID(t.row.AlbumID),
"album_title": t.row.AlbumTitle,
"artist_name": t.row.ArtistName,
"lidarr_album_id": t.lidarrID,
"rips": t.row.Rips,
})
}
return len(targets), nil
}
func foldResolveGroups(rows []dbq.ListDuplicateGroupsForResolveRow) []*resolveGroup {
var groups []*resolveGroup
for _, r := range rows {
@@ -419,16 +264,14 @@ func foldResolveGroups(rows []dbq.ListDuplicateGroupsForResolveRow) []*resolveGr
return out
}
// resolveNote is what the report says beside a group the resolver leaves for
// the operator, when there is something to say.
// resolveNote is what the report says beside a group, when there is something
// to say.
func resolveNote(g *resolveGroup) string {
switch g.class {
case ClassSameRelease:
if g.tracked() > 1 {
return "Lidarr tracks every copy as its own track, so removing one would make Lidarr download it again."
}
case ClassCrossRelease:
return "Each copy belongs to its own release, and Lidarr keeps each one. Nothing is removed; they count as one song."
if g.lone() < 0 {
return "Each copy belongs to its own release, so none is removed; they count as one song."
}
case ClassMismatch:
return "Identical audio filed under different titles: one file carries another song's tags."
}
@@ -437,16 +280,10 @@ func resolveNote(g *resolveGroup) string {
func writeResolveVerdicts(ctx context.Context, q *dbq.Queries, groups []*resolveGroup) error {
classes := dbq.SetDuplicateGroupClassesParams{}
states := dbq.SetDuplicateMemberLidarrStatesParams{}
for _, g := range groups {
classes.GroupIds = append(classes.GroupIds, g.id)
classes.Classes = append(classes.Classes, string(g.class))
classes.Notes = append(classes.Notes, g.note)
for i, m := range g.members {
states.GroupIds = append(states.GroupIds, g.id)
states.TrackIds = append(states.TrackIds, m.TrackID)
states.States = append(states.States, g.states[i])
}
}
if len(classes.GroupIds) == 0 {
return nil
@@ -454,9 +291,6 @@ func writeResolveVerdicts(ctx context.Context, q *dbq.Queries, groups []*resolve
if err := q.SetDuplicateGroupClasses(ctx, classes); err != nil {
return fmt.Errorf("write duplicate classes: %w", err)
}
if err := q.SetDuplicateMemberLidarrStates(ctx, states); err != nil {
return fmt.Errorf("write lidarr states: %w", err)
}
return nil
}
@@ -465,29 +299,31 @@ func survivorCandidates(g *resolveGroup) []SurvivorCandidate {
cands := make([]SurvivorCandidate, len(g.members))
for i, m := range g.members {
cands[i] = SurvivorCandidate{
TrackID: syncpkg.FormatUUID(m.TrackID),
FileFormat: m.FileFormat,
FileSize: m.FileSize,
AddedAt: m.AddedAt.Time,
LidarrTracked: g.states[i] == lidarrStateTracked,
TagFit: TagFitScore(m.TrackNumber != nil, m.PositionClash, m.FilePath, m.HasMbid),
TrackID: syncpkg.FormatUUID(m.TrackID),
FileFormat: m.FileFormat,
FileSize: m.FileSize,
AddedAt: m.AddedAt.Time,
TagFit: TagFitScore(m.TrackNumber != nil, m.PositionClash, m.FilePath, m.HasMbid),
}
}
return cands
}
// autoMerge merges one group into the copy ProposeSurvivor picks, which is the
// copy Lidarr tracks when there is one. removable is the guard: a copy Lidarr
// maps can never be among those removed, whatever the survivor rule said.
// autoMerge merges one group into the copy at index survivor, or the copy
// ProposeSurvivor picks when survivor is -1, and records the merge as the
// resolver's.
func autoMerge(
ctx context.Context, pool *pgxpool.Pool, logger *slog.Logger, dataDir string, g *resolveGroup, removable RemovableFunc,
ctx context.Context, pool *pgxpool.Pool, logger *slog.Logger, dataDir string, g *resolveGroup, survivor int,
) error {
survivorKey, reason := ProposeSurvivor(survivorCandidates(g))
if survivor >= 0 {
survivorKey, reason = syncpkg.FormatUUID(g.members[survivor].TrackID), "the copy that is not a video rip"
}
var survivorID pgtype.UUID
if err := survivorID.Scan(survivorKey); err != nil {
return fmt.Errorf("parse survivor id: %w", err)
}
res, err := MergeDuplicateGroupGuarded(ctx, pool, logger, dataDir, g.id, survivorID, removable)
res, err := MergeDuplicateGroup(ctx, pool, logger, dataDir, g.id, survivorID)
if err != nil {
return err
}
@@ -515,294 +351,6 @@ func autoMerge(
return nil
}
// recentlyChangedAlbums is the Minstrel albums whose Lidarr release the
// resolver changed within releaseChangeSettle, read from the audit log so a
// restart does not forget them.
func recentlyChangedAlbums(ctx context.Context, q *dbq.Queries, now time.Time) (map[string]bool, error) {
rows, err := q.ListAuditLogByActions(ctx, dbq.ListAuditLogByActionsParams{
Actions: []string{string(audit.ActionLidarrReleaseChange)}, SystemOnly: true,
PageLimit: 200, PageOffset: 0,
})
if err != nil {
return nil, fmt.Errorf("read recent release changes: %w", err)
}
out := map[string]bool{}
for _, r := range rows {
if now.Sub(r.CreatedAt.Time) > releaseChangeSettle {
break // newest first
}
var meta struct {
AlbumID string `json:"album_id"`
}
if json.Unmarshal(r.Metadata, &meta) == nil && meta.AlbumID != "" {
out[meta.AlbumID] = true
}
}
return out, nil
}
// repeatingAlbum is a Minstrel album where Lidarr maps more than one copy of a
// song, and the groups that showed it.
type repeatingAlbum struct {
albumID pgtype.UUID
releaseGroupMbid string
title, artist string
groups []*resolveGroup
}
func changeRepeatingReleases(
ctx context.Context, q *dbq.Queries, pool *pgxpool.Pool, logger *slog.Logger,
lid LidarrLibrary, groups []*resolveGroup, settling map[string]bool,
) ([]ReleaseChange, error) {
albums := map[string]*repeatingAlbum{}
for _, g := range groups {
if g.class != ClassSameRelease || g.tracked() < 2 {
continue
}
m := g.members[0]
key := syncpkg.FormatUUID(m.AlbumID)
if settling[key] || m.ReleaseGroupMbid == nil || *m.ReleaseGroupMbid == "" {
continue
}
a := albums[key]
if a == nil {
a = &repeatingAlbum{albumID: m.AlbumID, releaseGroupMbid: *m.ReleaseGroupMbid, title: m.AlbumTitle, artist: m.ArtistName}
albums[key] = a
}
a.groups = append(a.groups, g)
}
keys := make([]string, 0, len(albums))
for k := range albums {
keys = append(keys, k)
}
sort.Strings(keys)
var changes []ReleaseChange
for _, k := range keys {
if len(changes) >= resolveReleaseCap {
break
}
if ctx.Err() != nil {
return changes, ctx.Err()
}
a := albums[k]
change, note, err := changeRelease(ctx, q, lid, a)
if err != nil {
logger.Warn("duplicate resolve: Lidarr release check failed", "album", a.title, "err", err)
continue
}
if change != nil {
changes = append(changes, *change)
audit.WriteOrLog(ctx, pool, logger, pgtype.UUID{}, pgtype.UUID{}, audit.ActionLidarrReleaseChange, map[string]any{
"automatic": true,
"album_id": k,
"album_title": a.title,
"artist_name": a.artist,
"from_release": releaseLabel(change.From),
"to_release": releaseLabel(change.To),
})
}
if note != "" {
if err := setGroupNotes(ctx, q, a.groups, note); err != nil {
return changes, err
}
}
}
return changes, nil
}
// changeRelease checks one album's monitored release and, when it lists a
// song twice, moves Lidarr to the release that lists each song once and best
// covers what is on disk. It returns the change made (nil when none) and the
// note for the album's groups.
func changeRelease(
ctx context.Context, q *dbq.Queries, lid LidarrLibrary, a *repeatingAlbum,
) (*ReleaseChange, string, error) {
la, err := lid.LookupAlbumByMBID(ctx, a.releaseGroupMbid)
if errors.Is(err, lidarr.ErrNotFound) {
return nil, "Lidarr does not hold this album, so its copies are left for you.", nil
}
if err != nil {
return nil, "", err
}
current, err := lid.ListAlbumTracks(ctx, la.ID)
if err != nil {
return nil, "", err
}
if !releaseRepeats(current) {
return nil, "Lidarr maps each copy to a different track of this album, so none can go without a download.", nil
}
releases, err := lid.GetAlbumReleases(ctx, la.ID)
if err != nil {
return nil, "", err
}
titles, err := q.ListAlbumPresentTrackTitles(ctx, a.albumID)
if err != nil {
return nil, "", fmt.Errorf("list album titles: %w", err)
}
onDisk := map[string]bool{}
for _, t := range titles {
if k := ReleaseTitleKey(t); k != "" {
onDisk[k] = true
}
}
var from lidarr.AlbumRelease
var candidates []releaseCandidate
for _, r := range releases {
if r.Monitored {
from = r
continue
}
tracks, err := lid.ListReleaseTracks(ctx, r.ID)
if err != nil {
return nil, "", err
}
if len(tracks) == 0 || releaseRepeats(tracks) {
continue
}
candidates = append(candidates, releaseCandidate{release: r, coverage: coverage(tracks, onDisk)})
}
best, ok := pickRelease(candidates, coverage(current, onDisk))
if !ok {
return nil, "Lidarr's release of this album lists songs twice, and no other release lists each once while keeping what is on disk.", nil
}
if err := lid.SetMonitoredRelease(ctx, la.ID, best.ID); err != nil {
// Lidarr's album update unlinks the album's files and queues the rescan
// before it answers, and on a large album that outlasts the client's
// timeout while the change still lands (Cracker Island on the first
// deploy pass). An unanswered PUT is read back: when the release did
// change, it is recorded like any other, so the album gets its settle
// window and the report shows it.
if !errors.Is(err, lidarr.ErrUnreachable) || !releaseMonitored(ctx, lid, la.ID, best.ID) {
return nil, "", err
}
}
note := fmt.Sprintf("Lidarr now monitors the %s release, which lists each song once. The extra copies are merged once Lidarr has rescanned.", releaseLabel(best))
return &ReleaseChange{AlbumTitle: a.title, ArtistName: a.artist, From: from, To: best}, note, nil
}
// releaseMonitored reads whether the album's monitored release is now
// releaseID. A failed read answers false: the change is not assumed.
func releaseMonitored(ctx context.Context, lid LidarrLibrary, albumID, releaseID int) bool {
releases, err := lid.GetAlbumReleases(ctx, albumID)
if err != nil {
return false
}
for _, r := range releases {
if r.ID == releaseID {
return r.Monitored
}
}
return false
}
type releaseCandidate struct {
release lidarr.AlbumRelease
coverage int
}
// pickRelease chooses among releases that list each song once: the one
// covering most of the titles on disk, then the fewest tracks (the plainest
// edition that holds them), then a digital one, then the lowest id so the
// choice is stable. It refuses a release covering fewer titles on disk than
// the current one does: the change must not drop a song Lidarr now keeps.
func pickRelease(cands []releaseCandidate, currentCoverage int) (lidarr.AlbumRelease, bool) {
if len(cands) == 0 {
return lidarr.AlbumRelease{}, false
}
sort.SliceStable(cands, func(i, j int) bool {
a, b := cands[i], cands[j]
if a.coverage != b.coverage {
return a.coverage > b.coverage
}
if a.release.TrackCount != b.release.TrackCount {
return a.release.TrackCount < b.release.TrackCount
}
if da, db := isDigital(a.release), isDigital(b.release); da != db {
return da
}
return a.release.ID < b.release.ID
})
if cands[0].coverage < currentCoverage {
return lidarr.AlbumRelease{}, false
}
return cands[0].release, true
}
func isDigital(r lidarr.AlbumRelease) bool {
return strings.Contains(strings.ToLower(r.Format), "digital")
}
// releaseRepeats reports whether a release lists one title more than once.
func releaseRepeats(tracks []lidarr.ReleaseTrack) bool {
seen := map[string]bool{}
for _, t := range tracks {
k := ReleaseTitleKey(t.Title)
if k == "" {
continue
}
if seen[k] {
return true
}
seen[k] = true
}
return false
}
// coverage counts the distinct titles on disk that the release lists.
func coverage(tracks []lidarr.ReleaseTrack, onDisk map[string]bool) int {
hit := map[string]bool{}
for _, t := range tracks {
if k := ReleaseTitleKey(t.Title); onDisk[k] {
hit[k] = true
}
}
return len(hit)
}
func releaseLabel(r lidarr.AlbumRelease) string {
label := r.Format
if label == "" {
label = r.Title
}
if r.Disambiguation != "" {
label = r.Disambiguation + ", " + label
}
return fmt.Sprintf("%s (%d tracks)", label, r.TrackCount)
}
func setGroupNotes(ctx context.Context, q *dbq.Queries, groups []*resolveGroup, note string) error {
p := dbq.SetDuplicateGroupClassesParams{}
for _, g := range groups {
g.note = note
p.GroupIds = append(p.GroupIds, g.id)
p.Classes = append(p.Classes, string(g.class))
p.Notes = append(p.Notes, note)
}
if err := q.SetDuplicateGroupClasses(ctx, p); err != nil {
return fmt.Errorf("write duplicate notes: %w", err)
}
return nil
}
func resolvedDetail(merged int, changes []ReleaseChange) string {
var parts []string
if merged > 0 {
parts = append(parts, fmt.Sprintf("Merged %d %s Lidarr does not need.", merged, pluralWord(merged, "copy", "copies")))
}
switch len(changes) {
case 0:
case 1:
c := changes[0]
parts = append(parts, fmt.Sprintf("Lidarr now monitors the %s release of %s by %s, which lists each song once.",
releaseLabel(c.To), c.AlbumTitle, c.ArtistName))
default:
parts = append(parts, fmt.Sprintf("Lidarr now monitors a release that lists each song once for %d albums.", len(changes)))
}
return strings.Join(parts, " ")
}
func pluralWord(n int, one, many string) string {
if n == 1 {
return one
@@ -816,20 +364,14 @@ type DuplicateResolveWorker struct {
logger *slog.Logger
dataDir string
settings *FingerprintSettingsService
lidarr func() LidarrLibrary
tick time.Duration
}
// NewDuplicateResolveWorker builds a worker with the production cadence.
// lidarrFn is asked each pass, so a Lidarr setting saved in admin takes effect
// without a restart; it returns nil while Lidarr is disabled.
func NewDuplicateResolveWorker(
pool *pgxpool.Pool, logger *slog.Logger, dataDir string,
settings *FingerprintSettingsService, lidarrFn func() LidarrLibrary,
pool *pgxpool.Pool, logger *slog.Logger, dataDir string, settings *FingerprintSettingsService,
) *DuplicateResolveWorker {
return &DuplicateResolveWorker{
pool: pool, logger: logger, dataDir: dataDir, settings: settings, lidarr: lidarrFn, tick: duplicateResolveTick,
}
return &DuplicateResolveWorker{pool: pool, logger: logger, dataDir: dataDir, settings: settings, tick: duplicateResolveTick}
}
// Run blocks until ctx is cancelled, running a pass at start and then each tick.
@@ -865,16 +407,11 @@ func (w *DuplicateResolveWorker) tickOnce(ctx context.Context) {
}
return
}
var lid LidarrLibrary
if w.lidarr != nil {
lid = w.lidarr()
}
res, err := ResolveDuplicates(ctx, w.pool, w.logger, w.dataDir, lid, w.settings.Get().AutoResolve)
res, err := ResolveDuplicates(ctx, w.pool, w.logger, w.dataDir, w.settings.Get().AutoResolve)
if err != nil && ctx.Err() == nil {
w.logger.Warn("duplicate resolve: pass failed", "err", err)
}
w.logger.Info("duplicate resolve complete",
"groups", res.Groups, "lidarr", res.LidarrConsulted, "merged", res.Merged,
"merge_failed", res.MergeFailed, "release_changes", len(res.ReleaseChanges), "songs_linked", res.SongsLinked,
"held_back", res.HeldBack, "released", res.Released, "rip_searches", res.RipSearches)
"groups", res.Groups, "merged", res.Merged, "merge_failed", res.MergeFailed,
"songs_linked", res.SongsLinked, "held_back", res.HeldBack, "released", res.Released)
}