feat(library): fingerprint every new or changed file — M400 #3905-#3907
test-go / test (push) Successful in 1m9s
test-go / integration (push) Successful in 3m28s
release / Build signed APK (releases and dev) (push) Successful in 4m38s
release / Build + push container image (push) Successful in 1m26s
release / Verify release artifacts (tag releases only) (push) Skipped
test-go / test (push) Successful in 1m9s
test-go / integration (push) Successful in 3m28s
release / Build signed APK (releases and dev) (push) Successful in 4m38s
release / Build + push container image (push) Successful in 1m26s
release / Verify release artifacts (tag releases only) (push) Skipped
Two identities per track, because they answer different questions: - audio_stream_sha256: SHA-256 of the ENCODED audio packets (ffmpeg -map 0:a -c:a copy -f hash). Equal means identical audio whatever the tags say. Measured against the #3885 pair: the two WWW files hash identically here and differently as whole files. Packets rather than decoded samples, so an ffmpeg upgrade cannot silently change every stored hash, and nothing is decoded. - chromaprint: fpcalc -raw -signed. The same recording at another bitrate or codec, for the acoustic tier. fpcalc ships in the image (libchromaprint-tools); shelled out because CGO_ENABLED=0 rules out bindings. Stored in a track_fingerprints table rather than on tracks: eight queries read tracks with SELECT *, including album pages, search and the Subsonic surface, and a ~4 KB array there would be de-TOASTed on every one of them. The scan fingerprints only bytes it has not seen (a new path, or mtime past the row's). A tag-repair pass leaves fingerprints alone, and unchanged files with no fingerprint are the backfill's job (#3908). Folding that into the skip check would re-decode the whole library on the first scan after upgrade and push a sync change per track. A failure that says nothing about the file (timeout, cancelled scan, tool not installed) is never stored, and on changed bytes it removes the old row. A tool that rejects the file stores NULL at the current version, so the backfill does not retry it every boot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
This commit is contained in:
@@ -75,10 +75,15 @@ type Scanner struct {
|
||||
pool *pgxpool.Pool
|
||||
logger *slog.Logger
|
||||
paths []string
|
||||
// fingerprint derives a file's acoustic identity (M400). A field so an
|
||||
// integration test can substitute a deterministic one: CI has no real audio
|
||||
// to fingerprint, and what the test pins is WHEN the scan fingerprints, not
|
||||
// what the tools print. Call it through fingerprintFile.
|
||||
fingerprint func(ctx context.Context, path string) fingerprintResult
|
||||
}
|
||||
|
||||
func New(pool *pgxpool.Pool, logger *slog.Logger, paths []string) *Scanner {
|
||||
return &Scanner{pool: pool, logger: logger, paths: paths}
|
||||
return &Scanner{pool: pool, logger: logger, paths: paths, fingerprint: computeFingerprint}
|
||||
}
|
||||
|
||||
// Scan walks every configured root and upserts any audio file whose mtime is
|
||||
@@ -295,6 +300,25 @@ func (s *Scanner) scanFile(
|
||||
durationMs = probed
|
||||
}
|
||||
|
||||
// Fingerprint only bytes this row has not seen: a new path, or a file whose
|
||||
// mtime moved past the row's. An unchanged file re-read for a tag repair
|
||||
// keeps its stored fingerprint, for the same reason it keeps its duration
|
||||
// above — a tagReadVersion bump must stay bound by tag reads, not become a
|
||||
// decode of the whole library.
|
||||
//
|
||||
// An unchanged file with NO fingerprint yet is the backfill's job (#3908),
|
||||
// deliberately not the scan's. Folding it into the skip check would make
|
||||
// the first scan after an upgrade re-decode every track and push a sync
|
||||
// change to every client for each one.
|
||||
//
|
||||
// Computed before move adoption so adoption can match on the audio hash
|
||||
// (#3914); stored after the upsert, once the row id is known.
|
||||
var fp fingerprintResult
|
||||
fingerprinted := !unchanged
|
||||
if fingerprinted {
|
||||
fp = s.fingerprintFile(ctx, path)
|
||||
}
|
||||
|
||||
// A path we've never seen might not be a new track — it might be one that
|
||||
// moved or was renamed (#2528). Adopting re-points the existing row at this
|
||||
// path and clears its missing mark, so the UpsertTrack below conflicts on
|
||||
@@ -359,6 +383,9 @@ func (s *Scanner) scanFile(
|
||||
// touches this track will re-emit the change.
|
||||
s.logger.Warn("library scan: LogChange track upsert failed", "track_id", track.ID, "err", err)
|
||||
}
|
||||
if fingerprinted {
|
||||
s.storeFingerprint(ctx, q, track.ID, path, fp)
|
||||
}
|
||||
|
||||
if knownTrack {
|
||||
stats.Updated++
|
||||
|
||||
Reference in New Issue
Block a user