docs: library comments say what the scanner and resolver do, not how they came to be (#6277)
release / govulncheck (push) Successful in 20s
release / web (push) Successful in 1m20s
release / go (push) Successful in 1m37s
release / integration (push) Successful in 5m15s
release / android (push) Successful in 5m26s
release / Build signed APK (releases and dev) (push) Successful in 5m45s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 16s
release / Verify release artifacts (tag releases only) (push) Skipped
release / govulncheck (push) Successful in 20s
release / web (push) Successful in 1m20s
release / go (push) Successful in 1m37s
release / integration (push) Successful in 5m15s
release / android (push) Successful in 5m26s
release / Build signed APK (releases and dev) (push) Successful in 5m45s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 16s
release / Verify release artifacts (tag releases only) (push) Skipped
Rewrites the internal/library comment blocks to rule 32. Task, milestone, decision and rule ids are gone; the incidents behind them are restated as the constraint they left (why a file is deleted before its row, why the missing mark is a mark, why duplicates block on duration). Counts and album names from one library are dropped from production comments; test comments keep the names their fixtures use. The tagReadVersion log keeps each version's before and after, without the measurements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -13,11 +13,11 @@ import (
|
||||
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
|
||||
)
|
||||
|
||||
// Fingerprint backfill (M400 #3908).
|
||||
// Fingerprint backfill.
|
||||
//
|
||||
// The scan fingerprints only bytes it has not seen (see scanFile), so every track
|
||||
// imported before fingerprinting existed — and every row derived by an older
|
||||
// fingerprintVersion — needs a pass of its own. That pass is this worker.
|
||||
// The scan fingerprints only bytes it has not seen (see scanFile), so every
|
||||
// track imported before fingerprinting existed — and every row derived by an
|
||||
// older fingerprintVersion — needs a pass of its own. That pass is this worker.
|
||||
//
|
||||
// Its own worker rather than a stage in RunScan, for two reasons, both about
|
||||
// time:
|
||||
@@ -28,20 +28,23 @@ import (
|
||||
// - A long stage holds the scan run in flight, and a manual rescan answers 409
|
||||
// for as long as it runs.
|
||||
//
|
||||
// Progress is read live (FingerprintCoverage, the admin gauge) rather than from a
|
||||
// scan_runs tally: the work spans many passes with no single run to attach to.
|
||||
// Progress is read live (FingerprintCoverage, the admin gauge) rather than from
|
||||
// a scan_runs tally: the work spans many passes with no single run to attach
|
||||
// to.
|
||||
|
||||
// fingerprintBackfillTick is how often the worker looks for work. Once the
|
||||
// library has caught up, a tick is one indexed query; mostly the hour bounds how
|
||||
// long a file that timed out on a slow mount waits before it is tried again.
|
||||
// library has caught up, a tick is one indexed query; mostly the hour bounds
|
||||
// how long a file that timed out on a slow mount waits before it is tried
|
||||
// again.
|
||||
const fingerprintBackfillTick = time.Hour
|
||||
|
||||
// fingerprintBackfillBatch is how many tracks one query hands the worker. Small,
|
||||
// so tracks the scan adds mid-pass are not stuck behind one enormous page.
|
||||
// fingerprintBackfillBatch is how many tracks one query hands the worker.
|
||||
// Small, so tracks the scan adds mid-pass are not stuck behind one enormous
|
||||
// page.
|
||||
const fingerprintBackfillBatch = 50
|
||||
|
||||
// fingerprintBackfillConcurrency is the shipped value of the concurrency setting
|
||||
// (#3913): how many files are decoded at once. Two is deliberately low: fpcalc
|
||||
// fingerprintBackfillConcurrency is the shipped value of the concurrency
|
||||
// setting: how many files are decoded at once. Two is deliberately low: fpcalc
|
||||
// and the stream hash compete with playback transcoding for CPU and with
|
||||
// streaming for the mount, and a backfill that makes playback stutter is worse
|
||||
// than one that takes longer.
|
||||
@@ -75,7 +78,8 @@ type FingerprintBackfillWorker struct {
|
||||
tick time.Duration
|
||||
batch int32
|
||||
// fingerprint is a field for the same reason as Scanner.fingerprint: an
|
||||
// integration test pins which tracks a pass touches, not what the tools print.
|
||||
// integration test pins which tracks a pass touches, not what the tools
|
||||
// print.
|
||||
fingerprint func(ctx context.Context, path string, opts fingerprintOptions) fingerprintResult
|
||||
}
|
||||
|
||||
@@ -110,8 +114,8 @@ func (w *FingerprintBackfillWorker) Run(ctx context.Context) {
|
||||
}
|
||||
}
|
||||
|
||||
// runOnce contains a pass so that nothing it does — an error, a panic — can stop
|
||||
// the next tick from firing (rule 157).
|
||||
// runOnce contains a pass so that nothing it does — an error, a panic — can
|
||||
// stop the next tick from firing.
|
||||
func (w *FingerprintBackfillWorker) runOnce(ctx context.Context) {
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
@@ -135,10 +139,10 @@ func (w *FingerprintBackfillWorker) runOnce(ctx context.Context) {
|
||||
// retried forever within the pass.
|
||||
//
|
||||
// Settings are read before every batch, so a save takes effect within a batch
|
||||
// rather than an hour (#3913): switching fingerprinting off ends the pass, a new
|
||||
// concurrency applies to the next batch, and a new length restarts the walk from
|
||||
// the top at that length, because every row written at the old one went stale
|
||||
// the moment it changed.
|
||||
// rather than an hour: switching fingerprinting off ends the pass, a new
|
||||
// concurrency applies to the next batch, and a new length restarts the walk
|
||||
// from the top at that length, because every row written at the old one went
|
||||
// stale the moment it changed.
|
||||
func (w *FingerprintBackfillWorker) pass(ctx context.Context) (BackfillFingerprintsResult, error) {
|
||||
q := dbq.New(w.pool)
|
||||
var (
|
||||
@@ -146,7 +150,8 @@ func (w *FingerprintBackfillWorker) pass(ctx context.Context) (BackfillFingerpri
|
||||
mu sync.Mutex
|
||||
)
|
||||
// The all-zero uuid sorts before every real id. Valid must be true: a NULL
|
||||
// cursor would make "id > NULL" match nothing and every pass a silent no-op.
|
||||
// cursor would make "id > NULL" match nothing and every pass a silent
|
||||
// no-op.
|
||||
start := pgtype.UUID{Valid: true}
|
||||
after := start
|
||||
lengthSec := w.settings.Get().ChromaprintLengthSec
|
||||
|
||||
Reference in New Issue
Block a user