docs: library comments say what the scanner and resolver do, not how they came to be (#6277)
release / govulncheck (push) Successful in 20s
release / web (push) Successful in 1m20s
release / go (push) Successful in 1m37s
release / integration (push) Successful in 5m15s
release / android (push) Successful in 5m26s
release / Build signed APK (releases and dev) (push) Successful in 5m45s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 16s
release / Verify release artifacts (tag releases only) (push) Skipped

Rewrites the internal/library comment blocks to rule 32. Task, milestone,
decision and rule ids are gone; the incidents behind them are restated as
the constraint they left (why a file is deleted before its row, why the
missing mark is a mark, why duplicates block on duration). Counts and
album names from one library are dropped from production comments; test
comments keep the names their fixtures use. The tagReadVersion log keeps
each version's before and after, without the measurements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-23 12:27:34 -04:00
co-authored by Claude Opus 5.5
parent 08a2c06b85
commit 7e6228c13f
66 changed files with 578 additions and 564 deletions
+33 -33
View File
@@ -6,16 +6,16 @@ import (
"sort"
)
// Duplicate matching (M400 #3909).
// Duplicate matching.
//
// Pure functions over fingerprints: no database, no files. This is the part that
// decides whether two tracks in the operator's library are proposed as one
// recording, so every rule in it has to be falsifiable in a unit test.
// Pure functions over fingerprints: no database, no files. This is the part
// that decides whether two tracks in the library are proposed as one recording,
// so every rule in it has to be falsifiable in a unit test.
//
// Two tiers, answering different questions:
//
// exact equal audio_stream_sha256 — the same encoded audio bytes. No score,
// no threshold, no false positives (the #3885 pair).
// no threshold, no false positives.
// acoustic chromaprint fingerprints that agree closely once aligned — the
// same recording at another bitrate or in another codec.
//
@@ -52,23 +52,22 @@ const minDistinctFraction = 0.3
// fingerprints may show and still be proposed as one recording. Unrelated audio
// sits near 0.5; the same recording re-encoded lands well under 0.1.
//
// Deliberately conservative. The operator's stated worry is the opposite of a
// missed duplicate: "the same song can appear in different albums, usually it's
// a different recording", and an instrumental shares its vocal version's
// harmony, which chroma features capture. A false merge is the failure that
// matters, and the report is reviewed anyway. This is an unmeasured default:
// calibrate it against real pairs once the backfill (#3908) has populated the
// library, then expose it in Settings (#3913).
// Deliberately conservative. A false merge is the failure that matters, worse
// than a missed duplicate: the same song on different albums is usually a
// different recording, and an instrumental shares its vocal version's harmony,
// which chroma features capture. An admin can change it in Settings
// (FingerprintSettings).
const defaultAcousticMaxBitErrorRate = 0.15
// durationToleranceMs is how far apart two tracks' durations may be and still be
// compared. Encoders pad and trim a little; different edits differ by more.
// durationToleranceMs is how far apart two tracks' durations may be and still
// be compared. Encoders pad and trim a little; different edits differ by more.
const durationToleranceMs = 3000
// maxAcousticGroupSize caps an acoustic group. A cluster bigger than this is far
// more likely a shared jingle, a skit or a low-information pattern than eight
// copies of one recording, and proposing it would bury the real duplicates.
// Exact-tier groups are not capped: identical bytes are identical however many.
// maxAcousticGroupSize caps an acoustic group. A cluster bigger than this is
// far more likely a shared jingle, a skit or a low-information pattern than
// eight copies of one recording, and proposing it would bury the real
// duplicates. Exact-tier groups are not capped: identical bytes are identical
// however many.
const maxAcousticGroupSize = 8
// acousticScore is the result of comparing two fingerprints.
@@ -84,8 +83,8 @@ type acousticScore struct {
// preparedPrint is a fingerprint with the parts every comparison needs worked
// out once. The sweep compares each track with every other track within a few
// seconds of its duration, so rebuilding the alignment index for each pair would
// dominate its cost.
// seconds of its duration, so rebuilding the alignment index for each pair
// would dominate its cost.
type preparedPrint struct {
items []int32
index map[uint32][]int
@@ -208,13 +207,13 @@ type duplicateGroup struct {
type groupingResult struct {
Groups []duplicateGroup
// OversizeClusters counts acoustic clusters discarded for exceeding
// maxAcousticGroupSize. Reported rather than silent: a sudden rise means the
// cap or the information floor needs attention.
// maxAcousticGroupSize. Reported rather than silent: a sudden rise means
// the cap or the information floor needs attention.
OversizeClusters int
}
// groupUnit is one thing the acoustic pass compares: a single track, or an exact
// group standing in for all its byte-identical copies.
// groupUnit is one thing the acoustic pass compares: a single track, or an
// exact group standing in for all its byte-identical copies.
type groupUnit struct {
ids []string // every member, sorted
durationMs int32
@@ -224,11 +223,11 @@ type groupUnit struct {
assigned bool
}
// streamGrouper is the acoustic pass over units arriving in (durationMs, sortKey)
// order. It holds only the units within durationToleranceMs of the oldest one
// not yet settled, so memory is bounded by the densest few seconds of the
// library rather than by its size — the whole library's fingerprints would be
// hundreds of megabytes.
// streamGrouper is the acoustic pass over units arriving in (durationMs,
// sortKey) order. It holds only the units within durationToleranceMs of the
// oldest one not yet settled, so memory is bounded by the densest few seconds
// of the library rather than by its size — the whole library's fingerprints
// would be hundreds of megabytes.
//
// A seed can be settled as soon as a unit arrives beyond its window: everything
// it could group with has already arrived, and no later seed can reach back to
@@ -237,9 +236,9 @@ type groupUnit struct {
//
// Grouping is COMPLETE-LINKAGE: a unit joins a group only if it matches every
// unit already in it, within the duration tolerance and the bit-error limit.
// Single-linkage would let a chain of near-misses — A close to B, B close to C —
// drag A and C, which are not close, into one proposed merge. Complete linkage
// also means any member can be chosen as the survivor (#3911).
// Single-linkage would let a chain of near-misses — A close to B, B close to C
// — drag A and C, which are not close, into one proposed merge. Complete
// linkage also means any member can be chosen as the survivor.
//
// When an acoustic group absorbs an exact group, the result is tier acoustic: a
// group is only as certain as its weakest link.
@@ -320,7 +319,8 @@ func (g *streamGrouper) settleOldest() {
for _, member := range group {
member.assigned = true
// The acoustic evidence is untrustworthy; identical bytes are not.
// An exact group caught inside an oversize cluster is still proposed.
// An exact group caught inside an oversize cluster is still
// proposed.
if member.exact {
g.res.Groups = append(g.res.Groups, duplicateGroup{Tier: tierExact, Members: member.ids})
}