feat: duplicates resolve themselves where Lidarr says it is safe (M498)
release / govulncheck (push) Successful in 45s
release / web (push) Successful in 1m23s
release / go (push) Successful in 1m39s
release / integration (push) Successful in 4m25s
release / android (push) Successful in 6m17s
release / Build signed APK (releases and dev) (push) Successful in 5m57s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m54s
release / Verify release artifacts (tag releases only) (push) Skipped

The duplicate sweep proposed 4,197 groups and every one waited for the
operator. Most are safe to settle, and Lidarr defines what safe means: it
maps one file to each track of the release it monitors and downloads any
mapped file that disappears. Deleting a mapped copy opens exactly the hole
the operator saw Lidarr fill.

Classify (#5435)
- Migration 0075: duplicate_groups.class (same_release, cross_release,
  mismatch, review), resolve_note, resolved_automatically;
  duplicate_group_members.lidarr_state (tracked, unmapped);
  fingerprint_settings.auto_resolve; notification kind
  duplicates_resolved with both kind CHECKs swapped (rule 36).
- library.ClassifyDuplicateGroup, with MatchTitleKey dropping featuring
  credits, remaster notes and video-rip markers, and keeping live, demo,
  remix and instrumental. The rip markers move from api to library.

Choose the copy to keep (#5436)
- ProposeSurvivor ranks the copy Lidarr maps first, then tag fit (a
  clash-free track number, no rip marker in the name, an MBID), then the
  quality rules. File size picked the wrong Humanz copy in 6 of 21 groups.

Act (#5437)
- An hourly resolver pass reads Lidarr's unmapped files, matched by the
  last three path components, and records each copy's state.
- Same album, with at most one copy mapped: merged into the mapped copy.
  The merge is guarded, so a mapped copy can never be removed
  (MergeDuplicateGroupGuarded, ErrCopyTrackedByLidarr).
- Same album, every copy mapped: the monitored release lists the song
  twice (Humanz's 14x12" box set). The pass moves Lidarr to the release
  that lists each song once and best covers what is on disk. It never
  picks one covering less, and is capped at 10 albums per pass.
  - Fixed point (lesson #4183): the chosen release no longer repeats.
  - The album is left alone for 24h while Lidarr rescans, so "every copy
    unmapped" mid-rescan is never read as licence to merge.
- Both actions are audited with no actor and summarised to admins. The
  operator can switch them off in the Fingerprinting card (rule 25).
- Manual merges use the same guard: 409 copy_tracked_by_lidarr, or 503
  lidarr_unavailable when Lidarr cannot say.

Web
- Duplicates gets tabs: Needs review, Across releases, Resolved
  automatically. Each loads as you scroll (rule 172), replacing the
  pager.
- Each copy says whether Lidarr uses it.
- The resolver's note shows on each group.
- The merge confirm blocks, before sending, a merge that would remove
  the copy Lidarr uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-08 21:49:37 -04:00
co-authored by Claude Opus 5.5
parent 01e2294471
commit 4ecff52f19
48 changed files with 3068 additions and 254 deletions
+16
View File
@@ -6,3 +6,19 @@ VALUES ($1, $2, $3, $4);
SELECT * FROM audit_log
ORDER BY created_at DESC
LIMIT $1 OFFSET $2;
-- name: CountAuditLogByActions :one
-- System actions carry no actor; system_only narrows to them.
SELECT count(*)::bigint FROM audit_log
WHERE action = ANY(sqlc.arg(actions)::text[])
AND (NOT sqlc.arg(system_only)::boolean OR actor_id IS NULL);
-- name: ListAuditLogByActions :many
-- One page of the given actions, newest first: the duplicates report's
-- "resolved automatically" activity (M498) reads the resolver's merges and
-- Lidarr release changes from here.
SELECT * FROM audit_log
WHERE action = ANY(sqlc.arg(actions)::text[])
AND (NOT sqlc.arg(system_only)::boolean OR actor_id IS NULL)
ORDER BY created_at DESC, id
LIMIT sqlc.arg(page_limit) OFFSET sqlc.arg(page_offset);
+104 -5
View File
@@ -112,16 +112,36 @@ SELECT count(*)::bigint
WHERE g.status = 'pending'
AND (SELECT count(*) FROM duplicate_group_members m WHERE m.group_id = g.id) >= 2;
-- name: CountPendingDuplicateGroupsInView :one
-- CountPendingDuplicateGroups narrowed to one view of the report (M498):
-- review what the operator still has to decide: every class but
-- cross_release, and groups not yet classified
-- cross_release the same song on different releases, which is kept, not merged
SELECT count(*)::bigint
FROM duplicate_groups g
WHERE g.status = 'pending'
AND (SELECT count(*) FROM duplicate_group_members m WHERE m.group_id = g.id) >= 2
AND CASE sqlc.arg(view)::text
WHEN 'cross_release' THEN g.class = 'cross_release'
ELSE g.class IS DISTINCT FROM 'cross_release'
END;
-- name: ListPendingDuplicateGroupMembers :many
-- One page of proposals, newest first, flattened to one row per member so the
-- handler folds them without a query per group. What each copy carries — likes
-- and plays from every user — is here because it is what the operator weighs
-- when deciding which copy to keep.
-- One page of proposals in one view (see CountPendingDuplicateGroupsInView),
-- newest first, flattened to one row per member so the handler folds them
-- without a query per group. What each copy carries — likes and plays from
-- every user — is here because it is what the operator weighs when deciding
-- which copy to keep; Lidarr's view of it, because a copy Lidarr tracks cannot
-- be removed without Lidarr downloading it again.
WITH page AS (
SELECT g.id, g.tier, g.worst_bit_error_rate, g.detected_at
SELECT g.id, g.tier, g.worst_bit_error_rate, g.detected_at, g.class, g.resolve_note
FROM duplicate_groups g
WHERE g.status = 'pending'
AND (SELECT count(*) FROM duplicate_group_members m WHERE m.group_id = g.id) >= 2
AND CASE sqlc.arg(view)::text
WHEN 'cross_release' THEN g.class = 'cross_release'
ELSE g.class IS DISTINCT FROM 'cross_release'
END
ORDER BY g.detected_at DESC, g.id
LIMIT sqlc.arg(page_limit) OFFSET sqlc.arg(page_offset)
)
@@ -129,6 +149,17 @@ SELECT p.id AS group_id,
p.tier,
p.worst_bit_error_rate,
p.detected_at,
p.class,
p.resolve_note,
m.lidarr_state,
t.disc_number,
t.track_number,
(t.mbid IS NOT NULL)::boolean AS has_mbid,
EXISTS (SELECT 1 FROM tracks o
WHERE o.album_id = t.album_id AND o.id <> t.id AND o.missing_since IS NULL
AND t.track_number IS NOT NULL
AND o.disc_number IS NOT DISTINCT FROM t.disc_number
AND o.track_number = t.track_number)::boolean AS position_clash,
t.id AS track_id,
t.title,
artists.name AS artist_name,
@@ -164,3 +195,71 @@ SELECT count(*)::bigint
FROM duplicate_groups
WHERE status = 'pending'
AND detected_at >= sqlc.arg(since);
-- name: ListDuplicateGroupsForResolve :many
-- Every pending group's present copies, for the resolver (M498 #5435): what it
-- needs to classify the group and to choose the copy to keep. One row per copy,
-- grouped by group. position_clash is true when another present track on the
-- same album claims the same disc and track number: tags that do not fit the
-- album, which the survivor rule weighs against a copy.
SELECT g.id AS group_id,
g.tier,
t.id AS track_id,
t.album_id,
t.title,
artists.name AS artist_name,
t.file_path,
t.file_format,
t.file_size,
t.added_at,
t.disc_number,
t.track_number,
(t.mbid IS NOT NULL)::boolean AS has_mbid,
albums.release_group_mbid,
albums.title AS album_title,
EXISTS (SELECT 1 FROM tracks o
WHERE o.album_id = t.album_id AND o.id <> t.id AND o.missing_since IS NULL
AND t.track_number IS NOT NULL
AND o.disc_number IS NOT DISTINCT FROM t.disc_number
AND o.track_number = t.track_number)::boolean AS position_clash
FROM duplicate_groups g
JOIN duplicate_group_members m ON m.group_id = g.id
JOIN tracks t ON t.id = m.track_id
JOIN albums ON albums.id = t.album_id
JOIN artists ON artists.id = t.artist_id
WHERE g.status = 'pending'
AND t.missing_since IS NULL
ORDER BY g.id, t.id;
-- name: SetDuplicateGroupClasses :exec
-- The resolver's verdicts, written in one statement. Only rows whose class or
-- note actually changed are touched, so an hourly pass over thousands of
-- unchanged groups writes nothing. An empty note clears it.
UPDATE duplicate_groups g
SET class = v.class, resolve_note = NULLIF(v.note, '')
FROM (SELECT unnest(sqlc.arg(group_ids)::uuid[]) AS id,
unnest(sqlc.arg(classes)::text[]) AS class,
unnest(sqlc.arg(notes)::text[]) AS note) v
WHERE g.id = v.id
AND g.status = 'pending'
AND (g.class IS DISTINCT FROM v.class OR g.resolve_note IS DISTINCT FROM NULLIF(v.note, ''));
-- name: SetDuplicateMemberLidarrStates :exec
-- Lidarr's view of each copy as of this pass; an empty state clears it (Lidarr
-- not consulted). Unchanged rows are left alone.
UPDATE duplicate_group_members m
SET lidarr_state = NULLIF(v.state, '')
FROM (SELECT unnest(sqlc.arg(group_ids)::uuid[]) AS group_id,
unnest(sqlc.arg(track_ids)::uuid[]) AS track_id,
unnest(sqlc.arg(states)::text[]) AS state) v
WHERE m.group_id = v.group_id
AND m.track_id = v.track_id
AND m.lidarr_state IS DISTINCT FROM NULLIF(v.state, '');
-- name: MarkDuplicateGroupResolvedAutomatically :exec
UPDATE duplicate_groups SET resolved_automatically = true WHERE id = sqlc.arg(id);
-- name: ListAlbumPresentTrackTitles :many
-- The titles an album has on disk, for choosing the Lidarr release that covers
-- them best (M498 #5437).
SELECT title FROM tracks WHERE album_id = sqlc.arg(album_id) AND missing_since IS NULL;
@@ -10,6 +10,7 @@ UPDATE fingerprint_settings
acoustic_max_bit_error_rate = sqlc.arg(acoustic_max_bit_error_rate),
backfill_concurrency = sqlc.arg(backfill_concurrency),
sweep_interval_hours = sqlc.arg(sweep_interval_hours),
auto_resolve = sqlc.arg(auto_resolve),
updated_at = now()
WHERE id = true
RETURNING *;