release / govulncheck (push) Successful in 45s
release / web (push) Successful in 1m23s
release / go (push) Successful in 1m39s
release / integration (push) Successful in 4m25s
release / android (push) Successful in 6m17s
release / Build signed APK (releases and dev) (push) Successful in 5m57s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m54s
release / Verify release artifacts (tag releases only) (push) Skipped
The duplicate sweep proposed 4,197 groups and every one waited for the operator. Most are safe to settle, and Lidarr defines what safe means: it maps one file to each track of the release it monitors and downloads any mapped file that disappears. Deleting a mapped copy opens exactly the hole the operator saw Lidarr fill. Classify (#5435) - Migration 0075: duplicate_groups.class (same_release, cross_release, mismatch, review), resolve_note, resolved_automatically; duplicate_group_members.lidarr_state (tracked, unmapped); fingerprint_settings.auto_resolve; notification kind duplicates_resolved with both kind CHECKs swapped (rule 36). - library.ClassifyDuplicateGroup, with MatchTitleKey dropping featuring credits, remaster notes and video-rip markers, and keeping live, demo, remix and instrumental. The rip markers move from api to library. Choose the copy to keep (#5436) - ProposeSurvivor ranks the copy Lidarr maps first, then tag fit (a clash-free track number, no rip marker in the name, an MBID), then the quality rules. File size picked the wrong Humanz copy in 6 of 21 groups. Act (#5437) - An hourly resolver pass reads Lidarr's unmapped files, matched by the last three path components, and records each copy's state. - Same album, with at most one copy mapped: merged into the mapped copy. The merge is guarded, so a mapped copy can never be removed (MergeDuplicateGroupGuarded, ErrCopyTrackedByLidarr). - Same album, every copy mapped: the monitored release lists the song twice (Humanz's 14x12" box set). The pass moves Lidarr to the release that lists each song once and best covers what is on disk. It never picks one covering less, and is capped at 10 albums per pass. - Fixed point (lesson #4183): the chosen release no longer repeats. - The album is left alone for 24h while Lidarr rescans, so "every copy unmapped" mid-rescan is never read as licence to merge. - Both actions are audited with no actor and summarised to admins. The operator can switch them off in the Fingerprinting card (rule 25). - Manual merges use the same guard: 409 copy_tracked_by_lidarr, or 503 lidarr_unavailable when Lidarr cannot say. Web - Duplicates gets tabs: Needs review, Across releases, Resolved automatically. Each loads as you scroll (rule 172), replacing the pager. - Each copy says whether Lidarr uses it. - The resolver's note shows on each group. - The merge confirm blocks, before sending, a merge that would remove the copy Lidarr uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
171 lines
6.9 KiB
Go
171 lines
6.9 KiB
Go
package library
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"fmt"
|
|
"sync"
|
|
"time"
|
|
|
|
"github.com/jackc/pgx/v5/pgxpool"
|
|
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
|
|
)
|
|
|
|
// Fingerprinting settings (M400 #3913).
|
|
//
|
|
// Rule 25: what an operator might tune lives in the database and changes without
|
|
// a restart. One service instance is shared by the scanner, the backfill, the
|
|
// duplicate sweep and the admin API, so a save reaches all of them at once.
|
|
//
|
|
// The length is the dangerous knob: chromaprints taken at two lengths cannot be
|
|
// compared. Rather than bumping fingerprintVersion when it changes, every row
|
|
// records the length it was taken at (migration 0061) and every reader filters
|
|
// on the current one. A row at another length is stale to the backfill, pending
|
|
// to the gauge and invisible to the sweep, which is the effect a version bump
|
|
// would have, with one difference worth having: setting the length back makes
|
|
// rows not yet re-derived current again, instead of redoing the library twice.
|
|
|
|
// Bounds for each setting, mirrored by migration 0061's CHECKs.
|
|
const (
|
|
// Below 30s a print has too few items to align at the matcher's larger
|
|
// offsets and still overlap by minOverlapItems. Above 600s each print passes
|
|
// 20 KB and a candidate page of them grows past what one sweep should hold.
|
|
minChromaprintLengthSec = 30
|
|
maxChromaprintLengthSec = 600
|
|
// Unrelated audio sits near 0.5; a ceiling of 0.35 keeps a loosened
|
|
// threshold well clear of proposing noise.
|
|
minAcousticMaxBitErrorRate = 0.01
|
|
maxAcousticMaxBitErrorRate = 0.35
|
|
minBackfillConcurrency = 1
|
|
maxBackfillConcurrency = 8
|
|
minSweepIntervalHours = 1
|
|
maxSweepIntervalHours = 168
|
|
|
|
defaultSweepIntervalHours = 1
|
|
)
|
|
|
|
// FingerprintSettings mirrors the fingerprint_settings row.
|
|
type FingerprintSettings struct {
|
|
// Enabled off stops every decode: the scan takes only the stream hash (a
|
|
// demux, and what recognises a moved file) and stores nothing, and the
|
|
// backfill idles.
|
|
Enabled bool
|
|
ChromaprintLengthSec int32
|
|
AcousticMaxBitErrorRate float64
|
|
BackfillConcurrency int32
|
|
SweepIntervalHours int32
|
|
// AutoResolve lets the duplicate resolver (M498) act on its own: merge
|
|
// copies Lidarr does not need and change a Lidarr release that lists songs
|
|
// twice. Off, it only classifies, and every group waits for the operator.
|
|
AutoResolve bool
|
|
// UpdatedAt is when the settings were last saved. Set by the database;
|
|
// ignored by Set.
|
|
UpdatedAt time.Time
|
|
}
|
|
|
|
// DefaultFingerprintSettings mirrors migration 0061's column defaults, so a
|
|
// database that cannot be read still fingerprints the way a fresh install does.
|
|
var DefaultFingerprintSettings = FingerprintSettings{
|
|
Enabled: true,
|
|
ChromaprintLengthSec: defaultChromaprintLengthSec,
|
|
AcousticMaxBitErrorRate: defaultAcousticMaxBitErrorRate,
|
|
BackfillConcurrency: fingerprintBackfillConcurrency,
|
|
SweepIntervalHours: defaultSweepIntervalHours,
|
|
AutoResolve: true,
|
|
}
|
|
|
|
// ErrFingerprintSettingOutOfRange is returned by Set for a value migration
|
|
// 0061's CHECKs would reject, so the API answers 400 naming the field rather
|
|
// than surfacing a constraint violation.
|
|
var ErrFingerprintSettingOutOfRange = errors.New("fingerprint setting out of range")
|
|
|
|
// FingerprintSettingsService caches the settings and owns their persistence.
|
|
// Cached because the scanner reads them for every file it fingerprints.
|
|
type FingerprintSettingsService struct {
|
|
pool *pgxpool.Pool
|
|
|
|
mu sync.RWMutex
|
|
cur FingerprintSettings
|
|
}
|
|
|
|
// NewFingerprintSettingsService loads once and caches. It always returns a
|
|
// usable service, holding the defaults when the load fails; the error says so.
|
|
func NewFingerprintSettingsService(ctx context.Context, pool *pgxpool.Pool) (*FingerprintSettingsService, error) {
|
|
s := &FingerprintSettingsService{pool: pool, cur: DefaultFingerprintSettings}
|
|
row, err := dbq.New(pool).GetFingerprintSettings(ctx)
|
|
if err != nil {
|
|
return s, fmt.Errorf("fingerprint settings: load: %w", err)
|
|
}
|
|
s.cur = fingerprintSettingsFromRow(row)
|
|
return s, nil
|
|
}
|
|
|
|
// Get returns the cached settings. A nil service answers with the defaults, so
|
|
// a Scanner or worker built without one fingerprints as a fresh install would.
|
|
func (s *FingerprintSettingsService) Get() FingerprintSettings {
|
|
if s == nil {
|
|
return DefaultFingerprintSettings
|
|
}
|
|
s.mu.RLock()
|
|
defer s.mu.RUnlock()
|
|
return s.cur
|
|
}
|
|
|
|
// Set validates, persists and re-caches.
|
|
func (s *FingerprintSettingsService) Set(ctx context.Context, in FingerprintSettings) (FingerprintSettings, error) {
|
|
if err := validateFingerprintSettings(in); err != nil {
|
|
return FingerprintSettings{}, err
|
|
}
|
|
if s == nil {
|
|
return FingerprintSettings{}, errors.New("fingerprint settings: no settings service")
|
|
}
|
|
row, err := dbq.New(s.pool).UpdateFingerprintSettings(ctx, dbq.UpdateFingerprintSettingsParams{
|
|
Enabled: in.Enabled,
|
|
ChromaprintLengthSec: in.ChromaprintLengthSec,
|
|
AcousticMaxBitErrorRate: in.AcousticMaxBitErrorRate,
|
|
BackfillConcurrency: in.BackfillConcurrency,
|
|
SweepIntervalHours: in.SweepIntervalHours,
|
|
AutoResolve: in.AutoResolve,
|
|
})
|
|
if err != nil {
|
|
return FingerprintSettings{}, fmt.Errorf("fingerprint settings: save: %w", err)
|
|
}
|
|
out := fingerprintSettingsFromRow(row)
|
|
s.mu.Lock()
|
|
s.cur = out
|
|
s.mu.Unlock()
|
|
return out, nil
|
|
}
|
|
|
|
func validateFingerprintSettings(in FingerprintSettings) error {
|
|
switch {
|
|
case in.ChromaprintLengthSec < minChromaprintLengthSec || in.ChromaprintLengthSec > maxChromaprintLengthSec:
|
|
return fmt.Errorf("%w: chromaprint_length_sec must be %d-%d",
|
|
ErrFingerprintSettingOutOfRange, minChromaprintLengthSec, maxChromaprintLengthSec)
|
|
// Written as a negated range so NaN, which fails every comparison, is refused.
|
|
case !(in.AcousticMaxBitErrorRate >= minAcousticMaxBitErrorRate && in.AcousticMaxBitErrorRate <= maxAcousticMaxBitErrorRate):
|
|
return fmt.Errorf("%w: acoustic_max_bit_error_rate must be %.2f-%.2f",
|
|
ErrFingerprintSettingOutOfRange, minAcousticMaxBitErrorRate, maxAcousticMaxBitErrorRate)
|
|
case in.BackfillConcurrency < minBackfillConcurrency || in.BackfillConcurrency > maxBackfillConcurrency:
|
|
return fmt.Errorf("%w: backfill_concurrency must be %d-%d",
|
|
ErrFingerprintSettingOutOfRange, minBackfillConcurrency, maxBackfillConcurrency)
|
|
case in.SweepIntervalHours < minSweepIntervalHours || in.SweepIntervalHours > maxSweepIntervalHours:
|
|
return fmt.Errorf("%w: sweep_interval_hours must be %d-%d",
|
|
ErrFingerprintSettingOutOfRange, minSweepIntervalHours, maxSweepIntervalHours)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func fingerprintSettingsFromRow(row dbq.FingerprintSetting) FingerprintSettings {
|
|
return FingerprintSettings{
|
|
Enabled: row.Enabled,
|
|
ChromaprintLengthSec: row.ChromaprintLengthSec,
|
|
AcousticMaxBitErrorRate: row.AcousticMaxBitErrorRate,
|
|
BackfillConcurrency: row.BackfillConcurrency,
|
|
SweepIntervalHours: row.SweepIntervalHours,
|
|
AutoResolve: row.AutoResolve,
|
|
UpdatedAt: row.UpdatedAt.Time,
|
|
}
|
|
}
|