feat(library): measure every track's loudness in the background (M464 #4995)
release / go (push) Failing after 1m18s
release / web (push) Successful in 1m27s
release / govulncheck (push) Successful in 35s
release / integration (push) Failing after 4m13s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / android (push) Canceled after 5m47s
release / Build signed APK (releases and dev) (push) Canceled after 4m21s

The first step of loudness normalization: the server measures each track
with ffmpeg's EBU R128 filter (true peak, mono as dual mono) and stores the
integrated loudness, true peak and loudness range in track_loudness
(migration 0065).

It also keeps a histogram of the 400 ms gating blocks at 0.1 LU, so album
loudness can be computed exactly later with no second decode (#4996). The
histogram reproduces ffmpeg's own figure (-10.68 against -10.7 on the
captured fixture), and the analyzer logs a warning if the two ever drift.

- A background worker, cloned from the fingerprint backfill, measures every
  track, new ones included. Measuring inline in the scan was dropped: the
  analysis decodes the whole file, and a large import could pass the scan's
  one-hour stuck threshold. The scan only deletes a changed file's
  measurement; the worker ticks every 10 minutes.
- Timeouts, the cancel/missing-binary split and settled verdicts follow the
  fingerprint runner. Silence and undecodable files are stored as verdicts;
  stalls are retried. The deadline scales with track length.
- loudness_settings (enabled, files at once) and an admin card with the
  coverage gauge, under GET/PUT /api/admin/library/loudness-settings and
  GET /api/admin/library/loudness.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-06 13:01:01 -04:00
co-authored by Claude Opus 5.5
parent edd9a3a6db
commit f3196b3443
22 changed files with 2139 additions and 3 deletions
+77
View File
@@ -0,0 +1,77 @@
-- name: UpsertTrackLoudness :exec
-- Written when the backfill measures a track (#4995). Replaces the row
-- wholesale: a measurement of the old bytes has no standing once the file has
-- changed.
INSERT INTO track_loudness (
track_id, integrated_lufs, true_peak_dbtp, loudness_range_lu,
block_hist_start, block_hist, unreadable, analysis_version
) VALUES (
sqlc.arg(track_id), sqlc.narg(integrated_lufs), sqlc.narg(true_peak_dbtp),
sqlc.narg(loudness_range_lu), sqlc.narg(block_hist_start), sqlc.narg(block_hist),
sqlc.arg(unreadable), sqlc.arg(analysis_version)
)
ON CONFLICT (track_id) DO UPDATE SET
integrated_lufs = EXCLUDED.integrated_lufs,
true_peak_dbtp = EXCLUDED.true_peak_dbtp,
loudness_range_lu = EXCLUDED.loudness_range_lu,
block_hist_start = EXCLUDED.block_hist_start,
block_hist = EXCLUDED.block_hist,
unreadable = EXCLUDED.unreadable,
analysis_version = EXCLUDED.analysis_version,
analyzed_at = now();
-- name: DeleteTrackLoudness :exec
-- The scan saw new bytes at this path. The stored measurement describes the old
-- ones, so it goes, and the backfill measures the file again.
DELETE FROM track_loudness WHERE track_id = $1;
-- name: ListTracksNeedingLoudness :many
-- The backfill's work queue: tracks with no measurement, or one taken by an
-- older method. Keyset-paged on id so a pass visits each track at most once;
-- an inconclusive attempt writes no row, and without the cursor a file that
-- keeps timing out would be listed again straight away. Missing tracks are
-- skipped: there is no file to read.
SELECT t.id, t.file_path, t.duration_ms
FROM tracks t
LEFT JOIN track_loudness l ON l.track_id = t.id
WHERE t.missing_since IS NULL
AND (l.track_id IS NULL OR l.analysis_version < sqlc.arg(current_version))
AND t.id > sqlc.arg(after_id)
ORDER BY t.id
LIMIT sqlc.arg(batch_limit);
-- name: GetLoudnessCoverage :one
-- The admin gauge. measured + silent + unreadable + pending = total. Missing
-- tracks are excluded, or the gauge could never reach the end.
SELECT count(*)::bigint AS total,
count(*) FILTER (
WHERE l.analysis_version >= sqlc.arg(current_version)
AND l.integrated_lufs IS NOT NULL
)::bigint AS measured,
count(*) FILTER (
WHERE l.analysis_version >= sqlc.arg(current_version)
AND l.integrated_lufs IS NULL AND NOT l.unreadable
)::bigint AS silent,
count(*) FILTER (
WHERE l.analysis_version >= sqlc.arg(current_version)
AND l.unreadable
)::bigint AS unreadable,
count(*) FILTER (
WHERE l.track_id IS NULL OR l.analysis_version < sqlc.arg(current_version)
)::bigint AS pending
FROM tracks t
LEFT JOIN track_loudness l ON l.track_id = t.id
WHERE t.missing_since IS NULL;
-- name: GetLoudnessSettings :one
SELECT * FROM loudness_settings WHERE id = true;
-- name: UpdateLoudnessSettings :one
-- Whole-row write from the admin card; migration 0065's CHECK is the backstop
-- behind the service's own validation.
UPDATE loudness_settings
SET enabled = sqlc.arg(enabled),
backfill_concurrency = sqlc.arg(backfill_concurrency),
updated_at = now()
WHERE id = true
RETURNING *;