feat(library): the duplicate sweep — propose duplicate groups from fingerprints (M400 #3910)

Reads fingerprints, runs them through the matcher, and records
proposals in duplicate_groups (migration 0059). Nothing is merged or
deleted: a group is a proposal for the admin report (#3912).

Streaming. The whole library's fingerprints are hundreds of megabytes,
but tracks are only compared within 3s of each other in duration. So
candidates stream in (duration_ms, id) order, keyset-paged on a new
tracks(duration_ms, id) index. The grouper holds only the tracks within
3s of the oldest one not yet settled. A seed is settled once a track
arrives beyond its window, which gives the same result as grouping the
whole sorted list. groupDuplicates is rebuilt on the same streamGrouper,
so there is one grouping rule and the #3909 tests still cover it. Each
fingerprint's alignment index and variety check are computed once
instead of for every pair.

Exact duplicates are grouped library-wide in SQL. The first member the
stream meets stands in for the whole group in the acoustic pass. An
exact group caught in an oversize acoustic cluster is still proposed:
the acoustic evidence is discarded, identical bytes are not.

Re-sweeping:
- a group is identified by its sorted member ids, so finding it again
  refreshes the row in place
- a proposal whose members all sat in one dismissed group is not
  proposed again (a subset repeats the verdict; a superset is new
  evidence)
- a pending proposal no sweep has found again is retired, but only
  after a complete sweep, and only if an earlier sweep last confirmed
  it, so two overlapping sweeps cannot delete each other's findings
- dismissals are kept

DuplicateSweepWorker checks hourly and sweeps only when a fingerprint
was written after the last sweep started. TryStartDuplicateSweep guards
against two sweeps at once and reaps one stuck in flight for 2h. The
sweep row is closed on a detached context with a deadline, so a sweep
cancelled at shutdown still records that it ended.

The integration test pages one row at a time and checks:
- an acoustic pair and an exact pair are found
- a track with no fingerprint, a missing track and a near-duration
  unrelated song are left out
- a dismissed group is suppressed while the pending one refreshes
  without duplicating
- a proposal that stops holding is retired and the dismissal survives

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH

internal/db/queries/duplicates.sql

@@ -0,0 +1,100 @@
+-- name: StartDuplicateSweep :one
+INSERT INTO duplicate_sweeps DEFAULT VALUES RETURNING id, started_at;
+
+-- name: FinishDuplicateSweep :exec
+UPDATE duplicate_sweeps
+   SET finished_at       = now(),
+       candidates        = sqlc.arg(candidates),
+       groups_found      = sqlc.arg(groups_found),
+       oversize_clusters = sqlc.arg(oversize_clusters),
+       error_message     = NULLIF(sqlc.arg(error_message)::text, '')
+ WHERE id = sqlc.arg(id);
+
+-- name: GetInFlightDuplicateSweep :one
+-- The guard against two sweeps at once: "in flight" is finished_at IS NULL.
+SELECT id, started_at
+  FROM duplicate_sweeps
+ WHERE finished_at IS NULL
+ ORDER BY started_at DESC
+ LIMIT 1;
+
+-- name: GetLatestDuplicateSweep :one
+SELECT id, started_at, finished_at, candidates, groups_found, oversize_clusters, error_message
+  FROM duplicate_sweeps
+ ORDER BY started_at DESC
+ LIMIT 1;
+
+-- name: GetLatestFingerprintComputedAt :one
+-- Whether a sweep has anything new to look at: fingerprints written since the
+-- last sweep started.
+SELECT max(computed_at)::timestamptz AS latest FROM track_fingerprints;
+
+-- name: ListExactDuplicateHashes :many
+-- The exact tier, library-wide in one pass: identical encoded audio shared by
+-- more than one present track.
+SELECT f.audio_stream_sha256,
+       array_agg(t.id ORDER BY t.id)::uuid[] AS track_ids
+  FROM track_fingerprints f
+  JOIN tracks t ON t.id = f.track_id
+ WHERE t.missing_since IS NULL
+   AND f.fingerprint_version >= sqlc.arg(current_version)
+   AND f.audio_stream_sha256 IS NOT NULL
+ GROUP BY f.audio_stream_sha256
+HAVING count(*) > 1;
+
+-- name: ListDuplicateCandidates :many
+-- The acoustic tier's input, one page at a time in (duration_ms, id) order so the
+-- sweep holds only a sliding window of durations. Tracks without a chromaprint
+-- cannot be compared acoustically and are left out; any exact duplicates among
+-- them come from ListExactDuplicateHashes.
+SELECT t.id, t.duration_ms, f.audio_stream_sha256, f.chromaprint
+  FROM tracks t
+  JOIN track_fingerprints f ON f.track_id = t.id
+ WHERE t.missing_since IS NULL
+   AND f.fingerprint_version >= sqlc.arg(current_version)
+   AND f.chromaprint IS NOT NULL
+   AND (t.duration_ms, t.id) > (sqlc.arg(after_duration_ms)::integer, sqlc.arg(after_id)::uuid)
+ ORDER BY t.duration_ms, t.id
+ LIMIT sqlc.arg(page_limit);
+
+-- name: ListDismissedDuplicateMemberSets :many
+-- What the operator has already said are not duplicates. A new proposal whose
+-- every member sat together in one of these is not proposed again.
+SELECT g.id, array_agg(m.track_id ORDER BY m.track_id)::uuid[] AS track_ids
+  FROM duplicate_groups g
+  JOIN duplicate_group_members m ON m.group_id = g.id
+ WHERE g.status = 'dismissed'
+ GROUP BY g.id;
+
+-- name: UpsertDuplicateGroup :one
+-- Proposes a group, or refreshes one already pending. A group already dismissed
+-- or merged is left exactly as it is: the WHERE on the update makes the conflict
+-- a no-op, and the caller sees no row.
+INSERT INTO duplicate_groups (member_key, tier, worst_bit_error_rate, last_seen_sweep_id)
+VALUES (sqlc.arg(member_key), sqlc.arg(tier), sqlc.narg(worst_bit_error_rate), sqlc.arg(sweep_id))
+ON CONFLICT (member_key) DO UPDATE
+   SET tier                 = EXCLUDED.tier,
+       worst_bit_error_rate = EXCLUDED.worst_bit_error_rate,
+       last_seen_sweep_id   = EXCLUDED.last_seen_sweep_id
+ WHERE duplicate_groups.status = 'pending'
+RETURNING id;
+
+-- name: AddDuplicateGroupMember :exec
+INSERT INTO duplicate_group_members (group_id, track_id)
+VALUES (sqlc.arg(group_id), sqlc.arg(track_id))
+ON CONFLICT DO NOTHING;
+
+-- name: DeleteStalePendingDuplicateGroups :execrows
+-- A pending proposal this sweep did not find again no longer describes the
+-- library: a member was re-fingerprinted, merged away or went missing. Dismissed
+-- groups are kept regardless — they are the memory of a decision.
+--
+-- Only proposals last confirmed by an EARLIER sweep go. Should two sweeps ever
+-- overlap (a manual trigger racing the worker), neither may delete what the other
+-- has just found.
+DELETE FROM duplicate_groups g
+ WHERE g.status = 'pending'
+   AND g.last_seen_sweep_id IS DISTINCT FROM sqlc.arg(sweep_id)
+   AND (g.last_seen_sweep_id IS NULL
+        OR (SELECT s.started_at FROM duplicate_sweeps s WHERE s.id = g.last_seen_sweep_id)
+           < (SELECT s.started_at FROM duplicate_sweeps s WHERE s.id = sqlc.arg(sweep_id)));