Compare commits

...
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 5c19a916ba feat(library): store each album's MusicBrainz release-group id (M483 #5242)
release / govulncheck (push) Successful in 22s
release / web (push) Successful in 1m14s
release / go (push) Successful in 1m33s
release / integration (push) Successful in 4m36s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / android (push) Canceled after 5m24s
release / Build signed APK (releases and dev) (push) Canceled after 5m29s
albums.mbid is the release id (Picard's musicbrainz_albumid, one edition).
Lidarr names albums by release group, so re-acquisition and request
completion need that id too (#5241).

Migration 0070 adds albums.release_group_mbid (nullable, non-unique index:
several releases share a group). The scanner reads musicbrainz_releasegroupid
through extractReleaseGroupMBID, writes it on insert and heals it onto
existing rows when NULL. tagReadVersion goes to 3 so the next scan fills
it for the library already indexed, bound by tag reads (no ffprobe, no
decode).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:33:26 -04:00
bvandeusenandClaude Opus 5.5 bf6364b709 fix(lidarr): send the artist add's monitor choice in addOptions (#5239)
release / govulncheck (push) Successful in 29s
release / web (push) Successful in 1m58s
release / go (push) Successful in 2m14s
release / integration (push) Successful in 5m17s
release / android (push) Successful in 6m21s
release / Build signed APK (releases and dev) (push) Successful in 6m38s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 2m1s
release / Verify release artifacts (tag releases only) (push) Skipped
ArtistResource has no top-level `monitor`. Lidarr reads the choice from
AddOptions (AddArtistOptions, a MonitoringOptions), so the "all"/"future"
we sent there was dropped on deserialisation. AddOptions.Monitor stayed
Unknown, and AlbumMonitoredService.SetAlbumMonitoredStatus returns early
on Unknown. The request's monitoring was never applied.

Send monitor and monitored inside addOptions with searchForMissingAlbums,
the shape Lidarr's getNewArtist.js posts, and set monitorNewItems "all"
explicitly: both choices mean new releases are watched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:58:55 -04:00
bvandeusenandClaude Opus 5.5 670b30c954 fix(lidarr): add an album as the looked-up resource with its artist nested (#5234)
release / web (push) Successful in 1m42s
release / go (push) Successful in 2m7s
release / govulncheck (push) Successful in 37s
release / integration (push) Successful in 5m31s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / android (push) Canceled after 5m14s
release / Build signed APK (releases and dev) (push) Canceled after 4m42s
Lidarr's POST /api/v1/album validates `artist` as a nested resource
(AlbumController: RuleFor(s => s.Artist).NotNull()), so the flat payload
we sent was refused with "'Artist' must not be empty" every time. The
album add has never worked against a real Lidarr; approved album and
track requests sat in the reconciler retrying every 5 minutes.

AddAlbum now does what Lidarr's own add-album UI does (getNewAlbum /
getNewArtist): look the album up by MBID (album/lookup?term=lidarr:<mbid>),
then POST that resource back with monitored + searchForNewAlbum. When
Lidarr doesn't have the artist yet, the nested artist gets the request's
quality/metadata profile and root folder, monitors this album only
(monitor "none" + albumsToMonitor, which AlbumMonitoredService prefers)
and no future releases. An artist Lidarr already has is left as it is.
An MBID Lidarr's metadata doesn't know is ErrNotFound with no POST.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:55:41 -04:00
bvandeusenandClaude Opus 5.5 e8eee55325 fix(web): AcoustID card's loading line names itself (M401 #3922)
release / integration (push) Successful in 6m4s
release / android (push) Successful in 8m1s
release / Build signed APK (releases and dev) (push) Successful in 8m22s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m31s
release / Verify release artifacts (tag releases only) (push) Skipped
release / govulncheck (push) Successful in 28s
release / go (push) Successful in 1m41s
release / web (push) Successful in 1m21s
Its bare "Loading…" made the Integrations page's cover-providers test find
two matches for /loading…/i (Vitest, run 8487). "Loading AcoustID
settings…" also says which card is loading.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 23:47:14 -04:00
bvandeusenandClaude Opus 5.5 03e07e7c66 fix(web): send the empty body api.post requires for AcoustID run-now (M401 #3922)
release / govulncheck (push) Successful in 25s
release / web (push) Failing after 1m18s
release / go (push) Successful in 1m43s
release / Build + push container image (push) Canceled after 0s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / integration (push) Canceled after 4m51s
release / android (push) Canceled after 4m51s
release / Build signed APK (releases and dev) (push) Canceled after 4m6s
svelte-check on 0a7f7883: api.post takes a body. Every other body-less POST
in the client passes {}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 23:42:25 -04:00
bvandeusenandClaude Opus 5.5 0a7f788390 feat(web): AcoustID card on Integrations — key, threshold, coverage by source (M401 #3922)
release / go (push) Successful in 2m25s
release / web (push) Failing after 26s
release / govulncheck (push) Successful in 21s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / integration (push) Canceled after 3m34s
release / android (push) Canceled after 1m3s
release / Build signed APK (releases and dev) (push) Canceled after 1m3s
The card takes the slot of the unimplemented "MusicBrainz overrides"
placeholder. Rows:
- the on switch
- a write-only key field (the stored key is never sent back), with a
  link to register an application
- the minimum score (0.5 to 1)

Below them, recording-id coverage reads as a column: from tags, looked
up, none, and of the none how many are waiting, no match, ambiguous or
failed. There is a "Look up now" button and a folded list of the tracks
the lookup could not settle.

Off, keyless and stopped-short passes are each a visible state with the
reason (rule 164).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 23:29:34 -04:00
bvandeusenandClaude Opus 5.5 3c575b137c feat(library): AcoustID lookup worker fills the MBIDs tags leave empty (M401 #3920 #3921)
release / web (push) Successful in 1m44s
release / go (push) Successful in 2m1s
release / govulncheck (push) Successful in 17s
release / integration (push) Successful in 5m22s
release / android (push) Successful in 5m48s
release / Build signed APK (releases and dev) (push) Successful in 5m53s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 2m6s
release / Verify release artifacts (tag releases only) (push) Skipped
Migration 0069 adds tracks.mbid_source (tag | acoustid), a lookup state per
track (matched | ambiguous | no_match | failed) and the acoustid_settings
row (off, no key, min score 0.85).

The file's tag outranks a lookup (D4). UpsertTrack keeps a looked-up id
through a re-read that finds no tag id and replaces it as soon as one
appears. SetTrackMbidFromAcoustID refuses to write over a tag id.

The worker fingerprints each untagged track with fpcalc's compressed
print, looks it up and writes an id only when D5 settles it: one
recording at or above the threshold, or one left after matching title and
length. Ambiguous and no-match results write nothing. A key AcoustID
refuses, or the service being unreachable, stops the pass and is reported
in the worker's status. It never counts as a verdict on a track.

A changed file drops its lookup in the scan. The re-lookup takes back an
id that no longer matches.

Admin API: GET /api/admin/library/acoustid (settings, status, coverage by
source), PUT …/acoustid-settings (write-only key), POST …/acoustid/run,
GET …/acoustid/unsettled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 23:26:03 -04:00
bvandeusenandClaude Opus 5.5 f13da62797 feat(library): AcoustID lookup client and the compressed fpcalc print (M401 #3919)
internal/acoustid posts AcoustID's v2 lookup as a gzip form with
meta=recordings, one request per 400ms (their limit is 3/s) and a 20s
deadline. It returns every linked recording with its best score; choosing
among them is the worker's job (D5). The server's error codes map to an
invalid key (stop and say so), a rejected fingerprint (a verdict on the
track) or unavailable (try again later). A cancelled caller stays a
cancellation.

fpcalcLookupArgs and parseFpcalcCompressed read fpcalc's default output,
the compressed string the lookup takes (D1), always over the first 120s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 23:16:52 -04:00
bvandeusenandClaude Opus 5.5 b28cbe0600 feat(android): refuse plain http:// to a public server address (#5111)
release / go (push) Successful in 1m47s
release / govulncheck (push) Successful in 18s
release / web (push) Successful in 1m15s
release / integration (push) Successful in 4m44s
release / android (push) Successful in 5m37s
release / Build signed APK (releases and dev) (push) Successful in 5m43s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m13s
release / Verify release artifacts (tag releases only) (push) Skipped
Cleartext stays permitted app-wide for LAN servers and UPnP (#2439),
but a password or session cookie sent over plain HTTP to a public
address can be read by anyone on the path. A network interceptor now
refuses a cleartext request to the Minstrel server when the connection
lands on a public address, before any request byte is written.

Checked per connection, on the address actually reached, rather than
when the URL is typed: a name that resolved to the home network at
entry resolves to a public address once the phone leaves home.
Allowed: loopback, 10/8, 172.16/12, 192.168/16, link-local, 100.64/10
(Tailscale and other overlay VPNs) and fc00::/7. Only requests
BaseUrlInterceptor tagged as server-bound are checked; external
fetches and UPnP are untouched. The refusal has its own message.

Family baseline #5105, practice 13.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 23:02:07 -04:00
bvandeusenandClaude Opus 5.5 40dd5bb52c ci(android): pin the release certificate in the signer check (#5116)
release / govulncheck (push) Successful in 17s
release / web (push) Successful in 1m20s
release / go (push) Successful in 1m36s
release / integration (push) Successful in 5m25s
release / android (push) Successful in 6m25s
release / Build signed APK (releases and dev) (push) Successful in 7m0s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 1m3s
The check failed only on a debug signer, so an APK signed by any other
wrong key (a regenerated keystore, a swapped secret) would publish and
then reach no installed phone: Android updates in place only when the
signer matches. The step now requires exactly one signer whose
SHA-256 digest is the release certificate's (CN=Minstrel,
O=FabledSword, read from run 8446), and names a debug key or the
digest it got when it fails. Rotating the key on purpose changes the
digest in the same commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 22:51:33 -04:00
bvandeusenandClaude Opus 5.5 a1e9de2c84 ci(android): never ship a debug-signed APK; check the signer (#5116)
release / integration (push) Successful in 5m19s
release / android (push) Successful in 5m57s
release / Build signed APK (releases and dev) (push) Successful in 5m54s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m26s
release / Verify release artifacts (tag releases only) (push) Skipped
release / govulncheck (push) Successful in 26s
release / go (push) Successful in 2m5s
release / web (push) Successful in 1m24s
Adopts the rest of family idea #5103 (distributing your own APK):

- Practice 2: build.gradle.kts no longer falls back to the debug key
  when ANDROID_KEYSTORE_PATH is unset; the release build is signed with
  the release key or left unsigned. Main no longer builds and uploads a
  debug-signed app-debug.apk, which no install could ever update.
- Practice 3: android-release runs apksigner on the built APK, prints
  the signer's DN and SHA-256 digest, and fails on a debug signer.
  An unsigned build fails the same step, since there is no
  app-release.apk to verify.
- Practice 9: debug builds offer no server update. The banner does not
  poll and the About card says updates come from Android Studio, since
  the release-signed APK cannot install over a debug-signed app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 21:33:22 -04:00
bvandeusenandClaude Opus 5.5 13a7a3629a fix(library): MBID backfills skip tracks whose files are missing (#5139)
release / govulncheck (push) Successful in 37s
release / web (push) Successful in 1m13s
release / go (push) Successful in 1m34s
release / integration (push) Successful in 4m55s
release / Build signed APK (releases and dev) (push) Successful in 6m39s
release / android (push) Successful in 6m22s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m53s
release / Verify release artifacts (tag releases only) (push) Skipped
The track backfill listed every track with a NULL mbid, missing or
not, so each scan tried to open every missing file and logged an
"open failed" warning per track. Nothing ever healed. The album
backfill could pick a missing track as the one to read, and since
that pass is capped per scan, albums stuck that way were retried
ahead of the rest every time.

Both now read only tracks still on disk; an album with none left is
skipped until a scan finds its files again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 21:22:05 -04:00
bvandeusenandClaude Opus 5.5 2e36e70268 feat(android): Sonos/UPnP queue plays leveled URLs, rendered a track ahead (M464 #5002)
release / go (push) Successful in 2m49s
release / web (push) Successful in 2m21s
release / govulncheck (push) Successful in 25s
release / integration (push) Successful in 5m54s
release / android (push) Successful in 8m10s
release / Build signed APK (releases and dev) (push) Successful in 8m35s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m42s
release / Verify release artifacts (tag releases only) (push) Skipped
Every URL the Sonos queue loader sends is minted with level=true and
the track's album-play verdict from its neighbours in the queue; the
server returns the plain stream when leveling is off or changes
nothing. The playing track and the one after it are rendered ahead,
and each time the renderer moves on, the next is.

Server: a mint no longer prerenders on its own. A queue load mints
every track, which would have started an ffmpeg render per track at
once. The request now carries prerender, and at most two prerenders
run at a time; past that they are dropped, since a fetch renders on
demand anyway.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:38:21 -04:00
bvandeusenandClaude Opus 5.5 f34423a0e0 feat: leveled FLAC stream for Sonos/UPnP speakers (M464 #5001)
release / go (push) Successful in 2m17s
release / govulncheck (push) Successful in 26s
release / web (push) Successful in 2m4s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / integration (push) Canceled after 4m39s
release / android (push) Canceled after 2m53s
release / Build signed APK (releases and dev) (push) Canceled after 2m24s
Speakers fetch their own audio, so the phone cannot level it. A cast
token minted with level=true (and the client's asAlbum, which only the
queue holder knows) now returns GET /api/tracks/{id}/leveled.flac: the
track rendered by ffmpeg at the user's gain (volume=XdB, plus
alimiter at -1 dBFS for a limiter-mode boost), metadata stripped, FLAC
at 16 or 24 bits and at most 48 kHz. The gain is computed server-side
from the user's preference and the stored loudness, carried as
?g=<centi-dB>&lim=0|1 and signed into the token, so an edited URL does
not verify. Unity gains get the plain stream.

Renders are written beside the cache file and renamed in, keyed by the
source's size and mtime, coalesced per file (singleflight, detached
from the requesting speaker so a retry finds the render running),
started at mint time so the fetch finds them ready, and evicted least
recently used past leveled_cache_mb, a new admin setting (migration
0068, Loudness analysis card).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:31:07 -04:00
bvandeusenandClaude Opus 5.5 92c3f9bdb8 fix(android): look gains up by key, not value (M464 #5000)
release / govulncheck (push) Successful in 16s
release / web (push) Successful in 1m28s
release / go (push) Successful in 1m43s
release / integration (push) Successful in 4m38s
release / android (push) Successful in 5m19s
release / Build signed APK (releases and dev) (push) Successful in 5m30s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m13s
release / Verify release artifacts (tag releases only) (push) Skipped
`id in map` on a ConcurrentHashMap resolves to its legacy contains(),
which tests values (KT-18053); the compiler refuses it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:18:15 -04:00
bvandeusenandClaude Opus 5.5 1013c283da feat(android): level playback with a gain processor in the audio sink (M464 #5000)
release / go (push) Successful in 1m43s
release / web (push) Successful in 1m27s
release / govulncheck (push) Successful in 35s
release / integration (push) Successful in 5m16s
release / android (push) Failing after 3m52s
release / Build signed APK (releases and dev) (push) Failing after 3m20s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
Media3 1.10.1 -> 1.11.0, the version Renovate proposes; 1.11 flushes the
sink's audio processors at every item boundary with the playlist timeline
and the new item's period. GainAudioProcessor uses that to find the track
and its play-order neighbours (auto mode's album rule) and applies the
gain from the first sample, gapless transitions included, with a -1 dBFS
peak limiter in limiter mode and a full-scale clamp otherwise.

Gains come from the library cache first (sync now carries track and album
ReplayGain values; Room v10 adds the columns and rewinds the sync cursor
so an existing cache re-pulls them), then GET /api/tracks/replay-gain,
then none. The player service refreshes the leveling preference at start.

Web: a same-album neighbour without a track number no longer counts as
in-order album play, matching Android.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:10:35 -04:00
bvandeusenandClaude Opus 5.5 38290bf8f9 feat(web): level playback by the user's normalization preference (M464 #4999)
release / integration (push) Successful in 4m26s
release / android (push) Successful in 5m20s
release / Build signed APK (releases and dev) (push) Successful in 5m31s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 32s
release / Verify release artifacts (tag releases only) (push) Skipped
release / govulncheck (push) Successful in 18s
release / web (push) Successful in 1m13s
release / go (push) Successful in 1m29s
Cuts go through element.volume. Boosts route the element through a Web
Audio GainNode and a DynamicsCompressor (a -1 dBFS limiter in limiter
mode, a pass-through otherwise), built only when a track wants a boost
and only once an AudioContext is confirmed running; iOS never gets the
graph. Auto mode takes album gain when a queue neighbour is from the
same album in track order. Gains are fetched for the next 50 tracks as
the queue moves, with a 10s deadline. The prefetch element is untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 18:21:41 -04:00
bvandeusenandClaude Opus 5.5 2a7eb3dd19 ci(integration): give the suite a 20m package timeout
release / govulncheck (push) Successful in 37s
release / go (push) Successful in 1m32s
release / web (push) Successful in 1m12s
release / android (push) Successful in 6m26s
release / Build signed APK (releases and dev) (push) Successful in 6m49s
release / integration (push) Successful in 19m9s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m18s
release / Verify release artifacts (tag releases only) (push) Skipped
internal/api takes ~6.5 min under -race on an idle runner; with a second
run on the same runner it crossed go test's default 10m (run 8368: FAIL
at 600.016s with the running test 2s old, so load, not a hang).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 17:52:01 -04:00
bvandeusenandClaude Opus 5.5 d5dfcf5b7c fix: boost control is a switch; MutationQueue keeps one enqueue per kind (M464 #4998)
release / go (push) Successful in 2m15s
release / govulncheck (push) Successful in 29s
release / web (push) Successful in 1m42s
release / android (push) Successful in 5m57s
release / Build signed APK (releases and dev) (push) Successful in 5m49s
release / integration (push) Failing after 20m33s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
The ListenBrainz settings test finds the page's one checkbox, and the
boost control is a toggle anyway. detekt counts MutationQueue's enqueue
functions; suppressed as the replayer's dispatchers already are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 14:34:28 -04:00
bvandeusenandClaude Opus 5.5 af36b2f24a feat: per-user volume leveling preference, synced across devices (M464 #4998)
release / web (push) Failing after 1m5s
release / govulncheck (push) Successful in 22s
release / go (push) Successful in 1m17s
release / android (push) Failing after 1m51s
release / integration (push) Canceled after 4m12s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / Build signed APK (releases and dev) (push) Canceled after 3m21s
Mode (off, auto, track, album), target (-18, -16, -14 LUFS) and boost
(within headroom, or fully with a limiter), stored per user on the server
so the web player, the Android app and casts apply the same one.

- Server: user_normalization_prefs (migration 0067), GET/PUT
  /api/me/normalization; a whole-body PUT, validated, last write wins.
- Web: Settings > Playback > Volume leveling. Saves at once, restores the
  old choice if the save fails, and caches the value for the player.
- Android: Settings card. The device keeps a copy for offline playback
  (Room v9 with an explicit migration, so the upgrade wipes nothing).
  Writes are offline-first: shown at once, PUT best effort, queued on
  failure (NORMALIZATION_SET, collapsed to the newest). A refresh never
  overwrites a change still queued.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 14:29:40 -04:00
bvandeusenandClaude Opus 5.5 e3aa8629d3 feat(api): deliver loudness gains to every client (M464 #4997)
release / web (push) Successful in 1m44s
release / go (push) Successful in 2m12s
release / govulncheck (push) Successful in 40s
release / android (push) Successful in 5m28s
release / Build signed APK (releases and dev) (push) Successful in 4m42s
release / integration (push) Successful in 15m33s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m21s
release / Verify release artifacts (tag releases only) (push) Skipped
ReplayGain 2.0 values (gain to -18 LUFS, linear peak) derived from the
stored track and album loudness:

- Web: GET /api/tracks/replay-gain?ids=... (up to 200), a lookup the
  player calls for its queue, rather than a field on every TrackRef
  surface.
- Android: track_gain/track_peak and album_gain/album_peak on the sync
  views, so cached tracks level offline. Storing a measurement logs a
  track change, and an album's values moving logs an album change, both
  before the write (#2704), so caches pick the gains up.
- OpenSubsonic: replayGain on every song (album, getSong, search3,
  starred), as a JSON object and an XML element.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 13:54:05 -04:00
bvandeusenandClaude Opus 5.5 c2f81bf8df feat(library): album loudness from the tracks' summed block histograms (M464 #4996)
release / go (push) Successful in 1m47s
release / govulncheck (push) Successful in 27s
release / web (push) Successful in 1m27s
release / integration (push) Successful in 4m51s
release / android (push) Successful in 6m23s
release / Build signed APK (releases and dev) (push) Successful in 6m25s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m30s
release / Verify release artifacts (tag releases only) (push) Skipped
Album-mode normalization plays a whole album at one gain. That gain comes
from album_loudness (migration 0066): BS.1770's gated loudness over every
block on the album, computed by summing the tracks' stored histograms and
gating the sum. No audio is decoded again. Album true peak is the loudest
track's.

- Recomputed by the loudness worker each tick, after the track pass and
  whether or not analysis is switched on. ListAlbumsNeedingLoudness lists
  albums whose md5 over (present track id, measurement version and time) no
  longer matches the stored digest. One comparison covers every way
  membership changes (scan retag, duplicate merge, delete, missing and
  restored) without hooking each.
- No album value until every present track has a settled measurement, so
  an album's gain doesn't shift mid-listen as the rest is measured. Silent
  and unreadable tracks count as settled.
- Rows for albums with no present track left are dropped.
- The parser and the merge share trimBins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 13:27:02 -04:00
bvandeusenandClaude Opus 5.5 aee4b50bd4 test(api): pass the loudness settings to Mount in the library test router (M464 #4995)
release / go (push) Successful in 2m21s
release / web (push) Successful in 2m7s
release / govulncheck (push) Successful in 19s
release / integration (push) Successful in 5m15s
release / android (push) Successful in 6m24s
release / Build signed APK (releases and dev) (push) Successful in 6m37s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 2m24s
release / Verify release artifacts (tag releases only) (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 13:06:44 -04:00
bvandeusenandClaude Opus 5.5 f3196b3443 feat(library): measure every track's loudness in the background (M464 #4995)
release / go (push) Failing after 1m18s
release / govulncheck (push) Successful in 35s
release / web (push) Successful in 1m27s
release / android (push) Canceled after 5m47s
release / Build signed APK (releases and dev) (push) Canceled after 4m21s
release / integration (push) Failing after 4m13s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
The first step of loudness normalization: the server measures each track
with ffmpeg's EBU R128 filter (true peak, mono as dual mono) and stores the
integrated loudness, true peak and loudness range in track_loudness
(migration 0065).

It also keeps a histogram of the 400 ms gating blocks at 0.1 LU, so album
loudness can be computed exactly later with no second decode (#4996). The
histogram reproduces ffmpeg's own figure (-10.68 against -10.7 on the
captured fixture), and the analyzer logs a warning if the two ever drift.

- A background worker, cloned from the fingerprint backfill, measures every
  track, new ones included. Measuring inline in the scan was dropped: the
  analysis decodes the whole file, and a large import could pass the scan's
  one-hour stuck threshold. The scan only deletes a changed file's
  measurement; the worker ticks every 10 minutes.
- Timeouts, the cancel/missing-binary split and settled verdicts follow the
  fingerprint runner. Silence and undecodable files are stored as verdicts;
  stalls are retried. The deadline scales with track length.
- loudness_settings (enabled, files at once) and an admin card with the
  coverage gauge, under GET/PUT /api/admin/library/loudness-settings and
  GET /api/admin/library/loudness.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 13:01:01 -04:00
bvandeusenandClaude Opus 5.5 edd9a3a6db fix(auth): the Subsonic password is generated, never the login password (M462 #5026)
release / govulncheck (push) Successful in 39s
release / web (push) Successful in 1m8s
release / go (push) Successful in 1m30s
release / integration (push) Successful in 4m37s
release / android (push) Successful in 5m56s
release / Build signed APK (releases and dev) (push) Successful in 5m55s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m23s
release / Verify release artifacts (tag releases only) (push) Skipped
`minstrel admin reset-password` copied the new login password into
subsonic_password, which is stored in plain text because Subsonic t/s
sign-in needs it. Every account recovered through the CLI had its login
password readable in the database, and changing the password later left
the copy behind.

- reset-password now changes only password_hash.
- Migration 0064 clears every subsonic_password, removing the copies.
- Settings gets a Subsonic password card: the server generates a random
  password, shows it once, and it can be regenerated or turned off
  (GET/POST/DELETE /api/me/subsonic-password, audited). Generated rather
  than user-chosen so it can never be a reused password.
- docs/security.md describes the separate password instead of the known
  issue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:54:30 -04:00
bvandeusenandClaude Opus 5.5 522503e011 docs: hosting guide and security notes; README setup and HTTPS guidance (M462 #4986)
- docs/hosting.md: LAN vs internet; binding 4533 to 127.0.0.1 behind an
  HTTPS proxy (Caddy example, no buffering, long read timeouts for SSE and
  streams); the Client IP detection hop count (default 1, so 0 with no
  proxy or clients can forge X-Forwarded-For); the public address that
  password-reset links need; finding the setup token.
- docs/security.md: sessions, API keys, rate limits, headers and CSP; why
  CSRF rests on SameSite=Strict plus JSON-only cookie writes; the Subsonic
  password column, including the known issue that admin reset-password
  writes the login password there (#5026); why Android allows plain HTTP;
  the CI publish gate.
- README: keeps the LAN-first port mapping with a pointer for internet
  hosts, scopes "plain http:// is fine" to trusted networks, explains the
  setup token in first-run step 1, and links both docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:37:32 -04:00
bvandeusenandClaude Opus 5.5 6de8d4136d feat(android): keep the session cookie in Keystore-encrypted storage (M462 #4985)
release / govulncheck (push) Successful in 18s
release / web (push) Successful in 1m18s
release / go (push) Successful in 1m40s
release / integration (push) Successful in 4m29s
release / android (push) Successful in 5m17s
release / Build signed APK (releases and dev) (push) Successful in 5m20s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 16s
release / Verify release artifacts (tag releases only) (push) Skipped
The session cookie is a bearer credential, and it sat in plain text in the
Room auth_session row. It now lives in a SessionVault: AES-256-GCM under a
key held in the Android Keystore, with only the ciphertext in a private
prefs file. A copy of the app's files no longer yields a usable session.
Platform APIs only, no new dependency (androidx.security-crypto is
deprecated).

Nobody is signed out by the upgrade. On first launch AuthStore moves a
cookie still in the row into the vault and clears the column. If the
Keystore can't be used on a device, the cookie stays in the row as before
rather than being lost. A sign-in or 401 that lands during the move wins
over the value it read, and the move never throws. The auth gate now
waits for this before choosing Login or Home, with a 10s deadline so a
wedged Keystore can't leave the start screen spinning.

Tests: AuthStoreSessionVaultTest (upgrade move, vault-only load, Keystore
fallback, sign-in/out, hydration race) and SealedBoxTest (round trip,
fresh IV, tamper and wrong-key rejection). The real Keystore path needs
a device; the first launch after updating is that check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:29:32 -04:00
bvandeusenandClaude Opus 5.5 60c87da38e ci(govulncheck): check out with plain git; the golang image has no node (M462 #4984)
release / go (push) Successful in 2m13s
release / web (push) Successful in 1m40s
release / govulncheck (push) Successful in 41s
release / integration (push) Successful in 4m54s
release / android (push) Successful in 5m32s
release / Build signed APK (releases and dev) (push) Successful in 5m11s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m15s
release / Verify release artifacts (tag releases only) (push) Skipped
actions/checkout runs on node, which golang:1.26-bookworm does not carry,
so the lane died at checkout (run 8272, exit 127) before scanning
anything. That run was also the gate's first red: every other lane
passed, and image-release and release-assets both skipped, leaving :dev
on the previous build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:00:24 -04:00
bvandeusenandClaude Opus 5.5 b36125fa67 ci: one workflow graph, so nothing publishes on red; add govulncheck and npm audit (M462 #4984)
release / govulncheck (push) Failing after 2s
release / go (push) Successful in 1m49s
release / web (push) Successful in 1m8s
release / integration (push) Successful in 4m39s
release / android (push) Successful in 5m45s
release / Build signed APK (releases and dev) (push) Successful in 5m58s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
test-go, test-web and android were separate workflows on the same push as
release.yml, so the image build could not see their verdict: :dev meant
"it built", never "it passed". All lanes now live in release.yml, and
both publishing jobs (image-release and the new release-assets) need
every lane and require `result == 'success'` from each by name, so a
skipped lane blocks the publish just as a failed one does (rule 177).

- New lanes: govulncheck (in golang:1.26-bookworm, the builder's image,
  so it checks the stdlib that ships) and `npm audit --omit=dev` in web.
- Attaching the APK to a Release moved out of android-release into
  release-assets, behind the gate; the APK still builds in parallel.
- `docker buildx build --pull`, so floating base tags can't serve a
  stale Go patch release from the runner's cache.
- Integration wait uses `pg_isready` via docker exec: the old /dev/tcp
  probe never connects under dash (rule 81) and burned two minutes a run.
- workflow_dispatch input force_red fails the go lane on purpose, to
  watch the gate refuse.
- release_gate_test.go pins the gate: every job must be classified, and
  every publisher must need and require success from every lane.

Lanes have no path filters any more; a web-only push runs the Go suite
too, because "not run" must never read as "passed".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:51:37 -04:00
bvandeusenandClaude Opus 5.5 3217e10168 fix(deps): clear known vulnerabilities in what ships; build on the Go line CI tests (M462 #4984)
- golang.org/x/text v0.37.0 -> v0.39.0 (GO-2026-5970, infinite loop on
  invalid input, reachable from pgxpool). x/sync follows to v0.21.0.
- web lockfile: in-range updates from `npm audit fix` for devalue (high)
  and svelte (moderate), both of which ship in the browser bundle.
  package.json is unchanged.
- Dockerfile builder golang:1.25 -> golang:1.26. CI has tested on 1.26
  since the ci-go migration while the image was still compiled with 1.25,
  left over from the April skeleton; the shipped binary now uses the
  toolchain the tests ran on. govulncheck under golang:1.26-bookworm
  (go1.26.8) reports 0 vulnerabilities reachable from our code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:51:37 -04:00
bvandeusenandClaude Opus 5.5 327d49428f feat(auth): store Subsonic API keys hashed; a new key is shown once (M462 #4983)
test-web / test (push) Successful in 2m4s
test-go / test (push) Successful in 2m23s
test-go / integration (push) Successful in 5m28s
release / Build signed APK (releases and dev) (push) Successful in 6m29s
release / Build + push container image (push) Successful in 29s
release / Verify release artifacts (tag releases only) (push) Skipped
users.api_token held each user's apiKey in plaintext and was looked up by
equality, so a leaked row or backup handed out working keys. Migration
0063 replaces it with api_token_hash (sha256, hex), computed in place
from the existing keys so every Subsonic client keeps working.

The key can no longer be read back: GET /api/me/api-token is gone, and
POST returns the new key once. Settings shows it right after Regenerate
with a copy button and a "won't be shown again" note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:42:35 -04:00
bvandeusenandClaude Opus 5.5 2f3fbccab6 feat(auth): first account on a new server needs the setup token from the server log (M462 #4982)
test-go / test (push) Successful in 2m3s
test-web / test (push) Successful in 1m14s
test-go / integration (push) Successful in 4m38s
release / Build signed APK (releases and dev) (push) Successful in 5m36s
release / Build + push container image (push) Successful in 1m24s
release / Verify release artifacts (tag releases only) (push) Skipped
While no accounts exist, the server mints a random setup token at boot and
logs it. Registering the first account (which becomes admin) must carry it,
so whoever reaches a freshly exposed instance first cannot claim it. The
register page asks GET /api/auth/setup-status and shows a "Setup token"
field in place of the invite field while setup is pending.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:35:52 -04:00
bvandeusenandClaude Opus 5.5 46194a609d fix(auth): build password-reset links from an operator-set public address, never the Host header (M462 #4981)
test-go / test (push) Successful in 1m29s
test-web / test (push) Successful in 1m37s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
test-go / integration (push) Canceled after 2m45s
release / Build signed APK (releases and dev) (push) Canceled after 3m40s
buildResetURL used r.Host and r.TLS, so a forgot-password request with a
forged Host emailed the victim a real reset token on a link to the
attacker's server. Links now come only from network_settings.public_url
(migration 0062), and no reset email is sent while it is empty; the response
stays the same opaque 200 and the log says why.

The address is set on a new "Public address" card under Admin → Integrations,
which offers the page's own origin and warns while unset. PUT
/api/admin/network-settings takes either field alone, so the proxy card and
this one can't overwrite each other.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:32:10 -04:00
bvandeusenandClaude Opus 5.5 d411693bb2 feat(server): security headers and a hash-based CSP for the web app (M462 #4980)
test-web / test (push) Successful in 55s
test-go / test (push) Successful in 1m14s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / Build signed APK (releases and dev) (push) Canceled after 2m50s
test-go / integration (push) Canceled after 2m50s
There were no security headers at all. Now:
- every response: nosniff, Referrer-Policy strict-origin-when-cross-origin,
  Permissions-Policy (no camera/mic/geolocation), X-Frame-Options DENY;
  each set only if the handler hasn't.
- HSTS only when the trusted proxy reports HTTPS (rule 94); never a redirect.
- index.html carries a Content-Security-Policy whose script-src is 'self'
  plus the sha256 of each inline script in the page as served, computed
  after the branding template runs. No 'unsafe-inline' or 'unsafe-eval'
  for scripts. img-src admits remote https/http because Lidarr suggestion
  art is a remote poster URL.

Hashing in Go rather than via SvelteKit's kit.csp covers the inline scripts
SvelteKit doesn't know about (app.html's theme bootstrap and the branding
global injected at build) and stays correct whatever the app name is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:29:26 -04:00
bvandeusenandClaude Opus 5.5 24b767c23b feat(server): cap request bodies, bound body reads, private cache headers, JSON-only cookie writes (M462 #4979)
test-go / test (push) Successful in 1m48s
test-web / test (push) Successful in 2m0s
android / Build + lint + test (push) Successful in 6m27s
release / Build signed APK (releases and dev) (push) Successful in 6m42s
test-go / integration (push) Successful in 4m55s
release / Build + push container image (push) Successful in 1m20s
release / Verify release artifacts (tag releases only) (push) Skipped
- Every request body is capped at 4 MiB and must arrive within 30s. The
  deadline is set per request and cleared at end of body rather than via
  http.Server.ReadTimeout, which would cancel audio streams and the SSE
  stream once the background read hit it.
- IdleTimeout 120s closes idle keep-alive connections. Still no global
  WriteTimeout, for the same streaming reason.
- Streams, album covers and playlist covers are Cache-Control: private, so
  a shared cache never keeps an authenticated response for others.
- A cookie-authenticated write to /api must be application/json (415
  otherwise). SameSite=Strict can't see a sibling app on the same
  registrable domain; forms and no-preflight fetches can't send JSON.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:34:26 -04:00
bvandeusenandClaude Opus 5.5 755f997b0d feat(auth): sessions expire server-side; password change and reset end other sessions (M462 #4978)
Sessions had no server-side expiry: only the web cookie's 30-day Max-Age
limited them, and a bearer token (Android) lived until revoked by hand.
GetSessionByTokenHash and ListSessionsForUser now ignore sessions idle for
30 days or older than a year, and the GC worker deletes them hourly.

A password change was a plain UPDATE, so a session opened with the old
password survived it. Now:
- self-service change signs out every other device and keeps this one;
- reset by email ends every session the account has;
- an admin reset ends the target's sessions (keeping the admin's own when
  they reset themselves).

The success copy on web and Android says the other devices were signed out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:32:39 -04:00
bvandeusenandClaude Opus 5.5 719dc62b0d fix(auth): set the session cookie's Secure flag behind a TLS-terminating proxy (M462 #4977)
The cookie set Secure from r.TLS, which is nil whenever TLS terminates at
Traefik or Cloudflare, so a public deployment's session cookie went out
without Secure. auth.IsHTTPS now decides it from X-Forwarded-Proto, believed
only through the trusted-hop count (rule 94) and read positionally like
X-Forwarded-For. Plain-HTTP and LAN logins are unchanged; nothing redirects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:30:46 -04:00
bvandeusenandClaude Opus 5.5 3bfddd0862 feat(auth): throttle login, register, password reset and Subsonic auth failures (M462 #4976)
test-go / test (push) Successful in 1m54s
test-web / test (push) Successful in 1m34s
test-go / integration (push) Successful in 4m56s
android / Build + lint + test (push) Successful in 5m41s
release / Build signed APK (releases and dev) (push) Successful in 5m52s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
Every password-shaped check was mounted bare, so guessing was limited only
by bcrypt cost. A shared in-memory AttemptLimiter now sits in front of them:

- login: 10 failures per account and 50 per address per 15 min, checked
  before the user lookup and bcrypt; 429 with Retry-After. A success clears
  the account's count but not the address's.
- unknown usernames run a dummy bcrypt compare, so timing no longer says
  which accounts exist.
- register: 10 per address per hour; forgot-password: 5 per address and 3
  per email per hour (applied whether or not the email matches); reset: 20
  failed tokens per address per 15 min.
- Subsonic /rest: same limits as login, counting only wrong credentials,
  since clients authenticate on every request.

Web login, register, reset and forgot-password screens say how long to
wait; web and Android carry copy for the rate_limited code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:28:29 -04:00
bvandeusenandClaude Opus 5 516413f4ca fix(admin): re-acquisition settings take effect without a restart, and say why a save was refused (#3936, #3937)
test-web / test (push) Successful in 1m9s
test-go / test (push) Successful in 1m28s
test-go / integration (push) Successful in 3m57s
release / Build signed APK (releases and dev) (push) Successful in 5m20s
release / Build + push container image (push) Successful in 1m23s
release / Verify release artifacts (tag releases only) (push) Skipped
#3936: Router() built a reacquisition.SettingsService of its own, so a save
from the admin card refreshed that instance's cache while the sweeper in
main.go kept serving what it loaded at boot. The card showed the new
policy, the feature ran the old one, and only a restart reconciled them.
main.go now hands its instance to the server (srv.ReacqSettings), as it
already did for RecSettings, TagSettings and FingerprintSettings, and
Router() constructs one only when that field is nil. The regression test
saves through the router and reads the sweeper's instance.

#3937: the card's catch tested `e instanceof Error`, but api.put throws a
plain {code, message, status} object, so every reason the server gave was
discarded in favour of "Couldn't save settings." It now uses errMessage,
which appends the server's message for invalid_setting. Its test rejected
with an Error no code path produces, so it passed throughout; it now
rejects with what the client actually throws.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 20:15:35 -04:00
bvandeusenandClaude Opus 5 37d4906033 test(api): pass fingerprint settings to Mount in the route-registration test (M400 #3913)
test-go / test (push) Successful in 1m0s
test-go / integration (push) Successful in 3m25s
release / Build signed APK (releases and dev) (push) Successful in 4m35s
release / Build + push container image (push) Successful in 25s
release / Verify release artifacts (tag releases only) (push) Skipped
The unprefixed Mount call in library_test.go was missed when #3913 added
the parameter, failing go vet. Also pins the fingerprint coverage,
fingerprint settings and duplicates routes as admin-gated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 17:58:05 -04:00
bvandeusenandClaude Opus 5 077ae61235 feat(admin): fingerprinting settings — on/off, length, match threshold, concurrency, sweep interval (M400 #3913)
test-go / test (push) Failing after 44s
test-web / test (push) Successful in 49s
test-go / integration (push) Failing after 2m42s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / Build signed APK (releases and dev) (push) Canceled after 4m8s
Rule 25: the fingerprinting knobs move out of source into a DB-backed
singleton (migration 0061), edited from a card on the Duplicates page and
shared live with the scanner, the backfill and the duplicate sweep through
one service instance, so a save needs no restart.

The length is the knob that can silently break the library: prints taken
at two lengths never match. Each track_fingerprints row now records the
length it was taken at, and every reader filters on the current one — the
backfill treats another length as stale, the gauge counts it pending, the
sweep never streams it. Equivalent to a version bump, except that setting
the length back makes rows not yet redone current again. The card warns
before a length change re-fingerprints the library.

Off stops every decode: the scan takes only the stream hash (a demux, and
what recognises a moved file) and stores nothing, dropping a changed file's
stale row; the backfill idles. A save also makes a sweep due, since a new
threshold or length changes what the same prints group into, and the sweep
interval gains slack so an hourly interval on an hourly tick doesn't skip
every other tick.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 17:53:55 -04:00
bvandeusenandClaude Opus 5 c8bf9dc929 refactor(library): move detection matches on the audio hash, not size and duration (M400 #3914)
test-go / test (push) Successful in 1m9s
test-go / integration (push) Successful in 3m45s
release / Build signed APK (releases and dev) (push) Successful in 4m52s
release / Build + push container image (push) Successful in 14s
release / Verify release artifacts (tag releases only) (push) Skipped
A file that comes back renamed or moved keeps its track row, and with
it its likes and play history, by being matched to the missing row it
replaces (#2528). Untagged files were matched on (file_size,
duration_ms), which was never a fingerprint. It could pair two
unrelated files that happened to share a byte count and a duration,
and it missed a file retagged in place, whose size changes. The only
defence was requiring a unique match and otherwise giving up.

Now there is a real identity. FindMissingTrackByAudioHash matches a
missing track by the SHA-256 of its encoded audio (track_fingerprints,
#3906). That survives a rename, a move and a retag, and only an
identical recording can match it. adoptMovedTrack takes the new file's
hash, which the scan already computes before adoption. The size and
duration query and fallback are removed outright, with no second path
(rule 22).

Unchanged:
- MBID first: it identifies the recording and survives a re-encode
  that even the hash does not
- a unique match is still required
- an absent hash is never looked up, so unhashable files cannot pair
  with each other

The test fake answers the hash lookup only for the hash it holds, so
the tests can tell adoption by identity apart from adoption by
coincidence. That includes the case the old pair got wrong: different
audio of equal size and duration is not adopted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 17:31:42 -04:00
bvandeusenandClaude Opus 5 11ef044ef6 feat(library): merge duplicates without losing history (M400 #3911)
test-web / test (push) Successful in 57s
test-go / test (push) Successful in 1m16s
test-go / integration (push) Successful in 3m39s
release / Build signed APK (releases and dev) (push) Successful in 4m46s
release / Build + push container image (push) Successful in 26s
release / Verify release artifacts (tag releases only) (push) Skipped
Merge keeps one copy of a duplicate group and removes the rest. Every
table that references tracks does so ON DELETE CASCADE, so deleting a
duplicate's row outright would silently destroy its likes, plays,
playlist entries and tags. The merge moves all of that onto the kept
copy first, then deletes the empty row.

In one transaction, holding a lock on the group:
- repoints play_events, skip_events, contextual_likes, playback_errors,
  lidarr_requests.matched_track_id and playlist_tracks. The last is
  keyed by position, so every entry stays where it was.
- merges general_likes one per user, dated to the earlier like
- takes the union of track_tags, keeping the kept copy's own weight on
  a shared tag
- rewrites track_similarity onto the kept copy, dropping edges that
  would point a track at itself and keeping the kept copy's existing
  edge on a collision
- lets the kept copy take a recording MBID only the removed copy had
- deletes the removed copies' rows, tidies emptied albums and artists,
  marks the group merged
- logs sync changes: track deletes, and like and playlist-track
  delete/upsert pairs

The removed copies' files are deleted first, before any row changes,
through the same helper as DeleteTrackFile (now shared, along with the
album tidy-up). A merge that left the file behind would be undone by
the next scan re-importing it. An unwritable library answers 409
library_not_writable and nothing changes.

tracks.Service.MergeDuplicates wraps it with the opt-in Lidarr unmonitor
from RemoveTrack, skipped when the removed copy is a second file of the
kept copy's own album track: unmonitoring that would stop Lidarr
managing the kept file. It writes a duplicate_merge audit row after
commit, per the audit package's best-effort contract, naming both
paths.

POST /api/admin/library/duplicates/{id}/merge takes an optional
survivor_track_id (the report's proposal otherwise) and unmonitor.

On the report page:
- each copy gets a Keep choice, defaulting to the proposed one
- Merge needs a second click, on a button that says how many files it
  removes, with the consequence stated beside an opt-in Lidarr checkbox

Integration tests cover:
- every piece of history landing on the kept copy exactly: likes
  deduped at the earlier time, plays and skips counted, playlist
  position unchanged, tags unioned, similarity rewritten with no
  duplicate or self-edge, MBID inherited
- the removed file gone, and a second merge refused
- an unwritable file leaving likes, plays, row and group untouched
- a survivor outside the group refused

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 17:25:06 -04:00
bvandeusenandClaude Opus 5 ff493a8c7d feat(admin): the duplicates report — review proposed duplicate groups (M400 #3912)
test-web / test (push) Successful in 52s
test-go / test (push) Successful in 1m9s
test-go / integration (push) Successful in 3m31s
release / Build signed APK (releases and dev) (push) Successful in 4m32s
release / Build + push container image (push) Successful in 24s
release / Verify release artifacts (tag releases only) (push) Skipped
A new admin tab, Duplicates, beside Missing files: the proposals from
the duplicate sweep, with a Sweep now trigger and a Not duplicates
dismissal. Nothing on it merges or deletes; the merge is #3911.

Each group shows:
- whether it is identical audio or the same recording, with a match
  percentage from the weakest link between members
- every copy's format, size, duration, path, and the likes and plays
  it carries (every user's; this is admin-only, and it is what decides
  which copy to keep)
- the copy proposed to keep, and the rule that chose it

The survivor rule is library.ProposeSurvivor, a pure function the
merge will reuse: lossless over lossy, then the larger file, then the
copy in the library longest, then lowest id. Bitrate is not in it
because the scanner never fills tracks.bitrate, and for one recording
at one duration a larger file is the higher bitrate. m4a is not counted
as lossless: it may be AAC. The reason names the rule that separated
first place from second, not every rule the winner passed.

An empty report has three causes, and the page says which: still
fingerprinting, the sweep has never run, or it ran and found nothing.
The sweep's state and the backfill's progress come back with the groups
for that reason. Groups left with fewer than two members since the
sweep are not shown.

GET /api/admin/library/duplicates, POST .../sweep (202, or 409
sweep_in_progress), POST .../{id}/dismiss (404
duplicate_group_not_pending when already resolved).

Migration 0060 indexes play_events by track_id. Its only indexes led
with user_id, so each copy's play count, and the merge's repointing of
play history, would scan the whole table.

Web only, like Missing files: Android has no library-health admin
screens.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 17:11:11 -04:00
bvandeusenandClaude Opus 5 6379b6c31d feat(library): the duplicate sweep — propose duplicate groups from fingerprints (M400 #3910)
test-go / test (push) Successful in 1m0s
test-go / integration (push) Successful in 3m17s
release / Build signed APK (releases and dev) (push) Successful in 4m51s
release / Build + push container image (push) Successful in 14s
release / Verify release artifacts (tag releases only) (push) Skipped
Reads fingerprints, runs them through the matcher, and records
proposals in duplicate_groups (migration 0059). Nothing is merged or
deleted: a group is a proposal for the admin report (#3912).

Streaming. The whole library's fingerprints are hundreds of megabytes,
but tracks are only compared within 3s of each other in duration. So
candidates stream in (duration_ms, id) order, keyset-paged on a new
tracks(duration_ms, id) index. The grouper holds only the tracks within
3s of the oldest one not yet settled. A seed is settled once a track
arrives beyond its window, which gives the same result as grouping the
whole sorted list. groupDuplicates is rebuilt on the same streamGrouper,
so there is one grouping rule and the #3909 tests still cover it. Each
fingerprint's alignment index and variety check are computed once
instead of for every pair.

Exact duplicates are grouped library-wide in SQL. The first member the
stream meets stands in for the whole group in the acoustic pass. An
exact group caught in an oversize acoustic cluster is still proposed:
the acoustic evidence is discarded, identical bytes are not.

Re-sweeping:
- a group is identified by its sorted member ids, so finding it again
  refreshes the row in place
- a proposal whose members all sat in one dismissed group is not
  proposed again (a subset repeats the verdict; a superset is new
  evidence)
- a pending proposal no sweep has found again is retired, but only
  after a complete sweep, and only if an earlier sweep last confirmed
  it, so two overlapping sweeps cannot delete each other's findings
- dismissals are kept

DuplicateSweepWorker checks hourly and sweeps only when a fingerprint
was written after the last sweep started. TryStartDuplicateSweep guards
against two sweeps at once and reaps one stuck in flight for 2h. The
sweep row is closed on a detached context with a deadline, so a sweep
cancelled at shutdown still records that it ended.

The integration test pages one row at a time and checks:
- an acoustic pair and an exact pair are found
- a track with no fingerprint, a missing track and a near-duration
  unrelated song are left out
- a dismissed group is suppressed while the pending one refreshes
  without duplicating
- a proposal that stops holding is retired and the dismissal survives

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 17:00:07 -04:00
bvandeusenandClaude Opus 5 c06af48cd6 feat(library): the duplicate matcher — a pure comparison over fingerprints (M400 #3909)
test-go / test (push) Successful in 1m2s
test-go / integration (push) Successful in 3m24s
release / Build signed APK (releases and dev) (push) Successful in 5m8s
release / Build + push container image (push) Successful in 1m15s
release / Verify release artifacts (tag releases only) (push) Skipped
Decides whether tracks are proposed as one recording. No database, no
files, so every rule is falsifiable in a unit test.

Two tiers:
- exact: equal audio_stream_sha256 (identical encoded audio bytes). No
  threshold and no false positives.
- acoustic: chromaprint fingerprints that agree once aligned. Two
  fingerprints can start at slightly different points in the audio
  (padding trimmed differently), so offsets within ±120 items (~15s)
  are voted on using items that share their high 14 bits. Bit-error
  rate is then measured over the overlap at the winning offset. The
  approach and both constants follow AcoustID's pg_acoustid; it was
  reimplemented from that description and no code was copied.

No verdict below ~10s of overlap, or for low-information fingerprints
(silence, a sustained tone). Two such tracks agree without being one
recording.

Grouping uses complete linkage: a track joins a group only if it
matches every member. Otherwise A close to B and B close to C would
merge A and C, which are not close, and it means any member can be the
survivor. Other rules:
- durations must be within 3s
- acoustic groups are capped at 8, and larger clusters are reported
  and discarded as a likely shared jingle
- an exact group absorbed into an acoustic one takes the acoustic tier
- output does not depend on input order

The acoustic threshold is 0.15 bit-error rate: deliberately
conservative, since the operator's concern is different recordings of
one song being merged, and an instrumental shares its vocal's harmony.
It is unmeasured, and needs calibrating against real pairs once the
backfill has populated fingerprints (#3913 exposes it).

Nothing calls this yet; the sweep (#3910) does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 16:48:06 -04:00
bvandeusenandClaude Opus 5 21c698a616 test(web): give the admin page mock the fingerprint coverage query
test-web / test (push) Successful in 35s
release / Build signed APK (releases and dev) (push) Successful in 4m25s
release / Build + push container image (push) Successful in 34s
release / Verify release artifacts (tag releases only) (push) Skipped
b8855b48 made the admin overview page create a fingerprint coverage
query, but admin.test.ts mocks $lib/api/admin with an explicit factory
that only returned the cover coverage query. Every test that renders
the page threw on the missing export (run 6512, 11 failures). The mock
now returns it in the same empty-store shape, so the gauge stays hidden
in these tests the way the cover gauge does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 15:00:38 -04:00
bvandeusenandClaude Opus 5 b8855b480f feat(library): backfill fingerprints for the existing library — M400 #3908
test-web / test (push) Failing after 50s
test-go / test (push) Successful in 1m7s
test-go / integration (push) Successful in 3m27s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / Build signed APK (releases and dev) (push) Canceled after 4m23s
The scan fingerprints only bytes it has not seen, so everything imported
before fingerprinting existed, and any row derived by an older
fingerprintVersion, needs a pass of its own.

That pass is a background worker, not a stage in RunScan. RunScan runs
at boot and then every 12h, and an in-flight scan older than an hour is
reaped and a second started beside it. A stage would have to stop inside
the hour: a few hundred decodes a run, so about a month for a 50k-track
library. It would also hold the run in flight and answer manual
rescans with 409 while it worked.

FingerprintBackfillWorker runs once at start, then hourly. Nothing a
pass does (error or panic) can stop the next tick. A pass walks tracks
with no fingerprint or a stale version, skipping missing tracks,
keyset-paged on id. The cursor is what lets a pass end: an inconclusive
attempt writes no row, so a file that keeps timing out would otherwise
be re-listed and retried forever. Two decodes at a time, deliberately:
they compete with transcoding for CPU and with streaming for the mount.

storeFingerprint is now one package function shared by the scan and
the worker, and reports whether the attempt was fingerprinted,
rejected, inconclusive or failed to store.

Progress is a live gauge on the Admin scan card, served by
GET /api/admin/library/fingerprints: fingerprinted / rejected / pending
of total, with missing tracks excluded so it can reach the end.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 14:56:18 -04:00
bvandeusenandClaude Opus 5 71d4335584 docs(readme): the music mount is writable — Minstrel deletes when asked
test-go / test (push) Successful in 1m8s
test-go / integration (push) Successful in 4m10s
release / Build signed APK (releases and dev) (push) Successful in 5m23s
release / Build + push container image (push) Successful in 1m16s
release / Verify release artifacts (tag releases only) (push) Skipped
The quickstart mounted the library :ro and promised "Minstrel never
writes to your library". That stopped being true long before #3918:
quarantine's Delete file removes files, and under :ro it failed. The
operator has accepted delete ownership (Scribe note #3926).

The quickstart now mounts it writable and says exactly what Minstrel
writes: it deletes a file when an admin asks, and never moves, renames
or retags. It notes that uid 1000 needs write access, and that :ro
still works, with deletes refusing and explaining why.

Reorganising and tag writes stay out, pending whether Minstrel absorbs
Lidarr's role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 14:34:28 -04:00
bvandeusenandClaude Opus 5 702b48ce36 fix(lidarrquarantine): pass dataDir at the four stub-client test constructors
d7a8e5f3 added a dataDir parameter to NewService and updated the 13
call sites spelled NewService(pool, lidarrconfig.New(pool), nil). Four
more build their client from a Lidarr stub, NewService(pool, cfg,
clientFn), and were missed, so the package's tests did not compile and
run 6495 failed both go vet and the integration build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 14:34:28 -04:00
bvandeusenandClaude Opus 5 d7a8e5f300 fix(library): a track delete that cannot remove its file deletes nothing — #3918
test-go / test (push) Failing after 55s
test-web / test (push) Successful in 56s
test-go / integration (push) Failing after 4m50s
android / Build + lint + test (push) Successful in 5m52s
release / Build signed APK (releases and dev) (push) Successful in 6m5s
release / Build + push container image (push) Successful in 1m14s
release / Verify release artifacts (tag releases only) (push) Skipped
Two delete paths had opposite failure policies. tracks.RemoveTrack
logged a failed os.Remove and deleted the row anyway, which CASCADEs
likes, plays, playlist memberships and tags, while the file survived
for the next scan to re-import as a stranger. library.DeleteTrackFile
stopped correctly but reported it as a bare 500 nobody could read.

One path now: library.DeleteTrackFile removes the file first and, on
anything but ErrNotExist, returns *FileRemoveError with nothing
deleted. Only then does it delete the row and tidy an emptied album
and artist in one transaction, log the sync change and clear orphaned
artist art. RemoveTrack calls it, which also fixes RemoveTrack never
logging a sync change. Quarantine Delete file now tidies emptied
albums and artists too.

Both endpoints answer an unwritable library (EROFS, EACCES, EPERM) with
409 library_not_writable. The message names the directory (removal
writes to the parent), the uid:gid the server runs as, and that
nothing was deleted. Other remove errors are 500 file_delete_failed
with the path.

The reachable surface is quarantine Delete file, which failed
silently: no copy for the code on either client, and Android swallowed
the exception so the row just reappeared. Web and Android now have
copy for both codes and append the server message for exactly those
two. Android's quarantine screen shows it in a snackbar.

DELETE /api/admin/tracks/{id} has had no client since f7278f24, which
kept it on purpose for a safer admin surface, so its history loss was
latent. Fixed rather than removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 14:23:01 -04:00
bvandeusenandClaude Opus 5 cba77a5187 feat(library): fingerprint every new or changed file — M400 #3905-#3907
test-go / test (push) Successful in 1m9s
test-go / integration (push) Successful in 3m28s
release / Build signed APK (releases and dev) (push) Successful in 4m38s
release / Build + push container image (push) Successful in 1m26s
release / Verify release artifacts (tag releases only) (push) Skipped
Two identities per track, because they answer different questions:

- audio_stream_sha256: SHA-256 of the ENCODED audio packets
  (ffmpeg -map 0:a -c:a copy -f hash). Equal means identical audio
  whatever the tags say. Measured against the #3885 pair: the two WWW
  files hash identically here and differently as whole files. Packets
  rather than decoded samples, so an ffmpeg upgrade cannot silently
  change every stored hash, and nothing is decoded.
- chromaprint: fpcalc -raw -signed. The same recording at another
  bitrate or codec, for the acoustic tier.

fpcalc ships in the image (libchromaprint-tools); shelled out because
CGO_ENABLED=0 rules out bindings.

Stored in a track_fingerprints table rather than on tracks: eight
queries read tracks with SELECT *, including album pages, search and
the Subsonic surface, and a ~4 KB array there would be de-TOASTed on
every one of them.

The scan fingerprints only bytes it has not seen (a new path, or mtime
past the row's). A tag-repair pass leaves fingerprints alone, and
unchanged files with no fingerprint are the backfill's job (#3908).
Folding that into the skip check would re-decode the whole library on
the first scan after upgrade and push a sync change per track.

A failure that says nothing about the file (timeout, cancelled scan,
tool not installed) is never stored, and on changed bytes it removes
the old row. A tool that rejects the file stores NULL at the current
version, so the backfill does not retry it every boot.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 13:21:15 -04:00
bvandeusenandClaude Opus 5 eff3d88931 fix(recommendation): make the candidate draw reproducible, not accidentally so
test-go / test (push) Successful in 1m18s
test-go / integration (push) Successful in 4m52s
release / Build signed APK (releases and dev) (push) Successful in 6m9s
release / Build + push container image (push) Successful in 2m5s
release / Verify release artifacts (tag releases only) (push) Skipped
Four arms of the candidate query ended in a bare `ORDER BY random()` with no
seed: similar_artists, likes_overlap, coplay_artists and random_fill.

Such an arm returns a STABLE set only while its LIMIT exceeds the rows
eligible for it — at that point it returns all of them and the order stops
mattering, because scoreAndSortCandidates sorts by track id before drawing
jitter. Below that threshold it returns a random SUBSET, and two builds on
the same day draw different ones.

So daily determinism held BY ACCIDENT, and only for libraries smaller than
the limits. Any real library is larger, which means same-day rebuilds have
been producing different mixes since those arms were written — invisible,
because a mix that changes after a refresh looks like a feature rather than
a broken promise.

Found by breaking it: cutting RandomFill to 10 while tuning Songs-like
turned TestBuildSystemPlaylists_DailyNonceDeterminism red. That test seeds
~20 tracks against a default RandomFill of 30, so its determinism came from
the limit exceeding the library, not from the code being right. It is now a
real guard.

The arms order by md5(id || $12) instead. The CALLER decides what that
means, which is the point: system mixes pass a per-(user, day) seed and get
the determinism they promise, radio passes a fresh value per request and
keeps varying, which is what a radio should do. Same shape the browse
queries in this file already use (`md5(id::text || current_date::text)`) —
existing idiom, not a new one.

This also unblocks the trim that #3881 wanted and could not have. Shrinking
a randomly-ordered arm was what broke membership; a seeded one takes a
smaller but REPRODUCIBLE slice. Songs-like's seed-independent share drops
from 29% to 12%, which was the original intent before determinism forced it
back to 20%.

TestSongsLikeLimits_DoNotShrinkTheUnseededRandomArms is DELETED rather than
kept passing. It existed to stop anyone trimming those arms while the
ordering was broken; the ordering is fixed, so the constraint is gone and a
guard enforcing it would now forbid correct code.

Was filed as blocked on tooling. It was not: `make generate-go` runs sqlc as
a pinned Go tool and is the same path CI takes.

One thing worth knowing for next time: three files in internal/db/dbq are
owned by root, left by `make generate` running sqlc in Docker. sqlc errored
on the first it could not write. They are untouched by this change and the
regeneration of recommendation.sql.go completed, but `make generate` will
keep failing until they are chowned.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-11 08:44:33 -04:00
bvandeusenandClaude Opus 5 4ce47397a9 fix(recommendation): a nil LibrarySize must degrade, not panic
test-go / test (push) Successful in 1m4s
test-go / integration (push) Successful in 3m41s
release / Build signed APK (releases and dev) (push) Successful in 4m37s
release / Build + push container image (push) Successful in 1m58s
release / Verify release artifacts (tag releases only) (push) Skipped
Fixes the integration failure from 72115484: a SIGSEGV inside handleRadio
took down TestHandleRadio_ColdStart_OnlySeedReturned.

    recommendation.(*LibrarySize).Get(0x0, ...)
      library_scale.go:146
    api.(*handlers).handleRadio(...)
      radio.go:95

internal/api builds its handlers struct directly in a dozen tests, none of
which know about every field, so librarySize arrives nil there. Get took
l.mu.Lock() straight off the nil receiver.

The shape of the bug is what matters more than the nil check. This value's
entire contract is that it degrades — an errored count keeps the last known
number, a never-counted cache returns 0, and 0 scales to the base limits,
i.e. today's behaviour. A pool-sizing HINT then turned a request into a
crash, which is the precise opposite of that.

A nil receiver is now VALID and means "no cache": the count still runs, it
is just not memoised. Correct-but-uncached rather than zero, so a wiring
miss in production would cost a query per request, not silently unscale
every pool — a performance bug is findable, a quietly-wrong pool is not.

Patching the test constructors was the alternative and is worse: a dozen
call sites, and the next test to build a handlers literal reintroduces it.

Guarded with the nil path exercised directly, including that it counts
again rather than memoising, and still returns 0 on a failed count. The old
shape fails it by panicking.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 23:29:35 -04:00
bvandeusenandClaude Opus 5 721154847e fix(recommendation): size the candidate pool to the library
test-go / test (push) Successful in 1m5s
test-go / integration (push) Failing after 3m39s
release / Build signed APK (releases and dev) (push) Successful in 4m55s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 1m38s
Operator, 2026-09-10: "is the pool that we draw from somehow scaled to the
amount of music in the library... my earlier understanding of the tuning and
work may have been skewed by what was in my library."

It was not. DefaultCandidateSourceLimits returns what its own comment calls
"the v1 hardcoded constants per spec" — ~170 candidates for a 500-track
library and a 100,000-track one alike. The pool therefore samples a
shrinking FRACTION of a growing collection: 17% of 1,000 tracks, 1.7% of
10,000, 0.17% of 100,000. RandomFill, whose whole job is exploration,
becomes a thinner and noisier slice at exactly the moment a library gets
more diverse — which is the "starting to feel weird" being reported.

    1,000 tracks -> pool 170     (unchanged)
    5,000        -> pool 170     (unchanged)
   20,000        -> pool 280
   80,000        -> pool 500     (ceiling)

THE SCALING IS PER-ARM, and that is the substance rather than a refinement.
A limit only matters if there are rows for it to cut off, so what an arm is
BOUNDED BY decides whether library size can help it. LBSimilar,
SimilarArtist, TagOverlap and RandomFill grow: they are bounded by
similarity/tag data and by the library itself. LikesOverlap, UserCoplay and
TasteOverlap do not: they are bounded by the user's likes, the instance's
co-play graph and the taste profile, none of which grow when the library
does. Raising those would sample more of a set that did not change — churn,
not reach. It also keeps this from inflating the sim_score-0 share, since
TasteOverlap is one of the two zero-similarity arms.

sqrt, not linear: linear would put a 100,000-track library at a
3,400-candidate pool, long past where more candidates improve the answer.
A 4x ceiling bounds it at ~500.

Never shrinks an arm. The base limits are a floor, and #3889 makes that
load-bearing rather than tidy — shrinking an arm ordered by unseeded
random() changes pool membership between same-day rebuilds.

Library size comes from a TTL-cached count reusing CountTracksMatching with
an empty pattern (rule 28 — a new query would need sqlc regeneration, which
is blocked). The ILIKE defeats every index, so it is a full scan and must
not run per request. It degrades rather than fails: an error keeps the last
known value, a never-counted cache returns 0, and 0 scales to the base
limits — today's behaviour exactly. Nothing about sizing a pool justifies
failing the request it is sizing. Bounded by a 3s deadline (rule 156), and
a failed refresh does not stamp the clock, so a blip cannot pin a stale
value for the whole TTL.

THE REFERENCE IS ASSUMED, NOT MEASURED. libraryScaleReference = 5000 is
where growth starts, and the size the v1 constants were really tuned against
is unrecorded. #3879 should replace it; until then that constant is the one
thing to change. Deliberately conservative: below it nothing scales at all,
so no existing install changes behaviour.

Falsification caught a weak guard: the sqrt-vs-linear assertion was written
at SIXTEEN times the reference, where linear has already been clamped by the
ceiling and both curves land on 4x. It proved nothing. Moved to four times
the reference, below the ceiling for both, where sqrt gives 2x and linear
would give 4x.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 23:23:04 -04:00
bvandeusenandClaude Opus 5 633d4f591f fix(radio): cap any one artist's share of a radio session
test-go / test (push) Successful in 1m13s
test-go / integration (push) Successful in 3m53s
release / Build signed APK (releases and dev) (push) Successful in 5m9s
release / Build + push container image (push) Successful in 1m52s
release / Verify release artifacts (tag releases only) (push) Skipped
Operator, 2026-09-10: started radio from a song and "literally all of the
songs in the playlist after that were from a single artist which was not
expected."

There was no per-artist cap anywhere in the radio path. radio.go built the
pool and handed it straight to Shuffle, which scores, sorts and takes the
top N — nothing between those steps bounded any artist's share, so a pool
dominated by one artist produced an output dominated by it. The asymmetry
was the tell: discover.go, you_might_like.go and home.go all cap; radio
never got one.

With the fixture that reproduces it — 20 liked tracks by one artist plus 10
by ten others — the old path returns 10 tracks from 1 artist. It now returns
10 from 8.

TWO PASSES, and that is the whole design. A hard cap was the easy mistake:
radio asks for 50 tracks by default and 200 at most, so capping at three per
artist over a concentrated pool would hand back a six-track "radio". Pass
one takes candidates that fit under the caps; pass two fills any remaining
slots from those it skipped, still in score order. The result always holds
min(limit, len(candidates)) — the caps change WHICH tracks are picked, never
HOW MANY. Rule 131's principle past the system mixes it was written for.

The caps SCALE with the requested length rather than being a constant.
Three-per-artist is a sensible 12% of a 25-track mix and an absurd 1.5% of a
200-track radio, where every selection would sit in the relaxation path and
the cap would be decorative. RadioDiversityCaps holds the system mixes'
proportion at any length: 3/2 at 25, 6/4 at 50, 24/16 at 200, with floors so
a very short radio is not capped down to one track per artist.

A BOUND, NOT AN EXCLUSION — the operator asked for the opposite of removal:
"again it should be able to add songs from the same artist." The dominant
artist still appears, just not exclusively. Guarded, because the tempting
wrong fix is the filter songs-like used to carry.

Shuffle grew the parameter rather than gaining a capped twin: radio is its
only production caller, so a second function would have left the original
dead (rule 22).

Falsified against each named regression: uncapped gives 10/10 to one artist;
a hard cap returns 3 of 10 on a single-artist pool; a cap-as-exclusion drops
the artist entirely; a fixed cap stays 3 where the scaled one reaches 24.

Caught while writing the guards: the artist-key constant was hand-written
hex and wrong — the fixture's artist UUID carries 0001 in its fourth group,
so the lookup missed and the assertion measured nothing. Derived from the
same construction the fixture uses now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 22:14:42 -04:00
bvandeusenandClaude Opus 5 f5dd4462de test(playlists): the same-artist guard needed a fixture that has same artists
test-go / test (push) Successful in 1m25s
test-go / integration (push) Successful in 4m46s
release / Build signed APK (releases and dev) (push) Successful in 5m45s
release / Build + push container image (push) Successful in 17s
release / Verify release artifacts (tag releases only) (push) Skipped
Fixes the integration failure from 31190657. The test was wrong, not the
code: it could not have passed whatever produceSeedMixes did.

seedActiveLibrary builds its tracks through seedTrack, whose own comment
says "artist and album are not deduplicated across calls (mbid-less
upsert)". So every track gets a fresh artist row despite sharing a name —
4 artists x 5 tracks is really 20 artists with one track each. A seed
artist's only track IS the seed, which is excluded from its own mix, so
"does this mix contain a track by its seed artist" was structurally
answerable only as no.

That is the failure mode worth naming: the assertion was measuring the
fixture, not the behaviour, and it reported the behaviour as broken.

seedSharedArtistLibrary upserts each artist ONCE and reuses the id across
its tracks, so a seed artist genuinely owns five others. Albums are still
not deduplicated, which suits this test — the per-album cap never binds, so
the per-artist cap (3) is unambiguously what is under test. Noted in the
fixture, because "tidying" the album titles into something shared would
silently change which cap the assertion measures.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 21:25:49 -04:00
bvandeusenandClaude Opus 5 31190657d8 feat(recommendation): Songs-like can include the seed artist's own music
test-go / test (push) Successful in 1m16s
test-go / integration (push) Failing after 3m55s
release / Build signed APK (releases and dev) (push) Successful in 5m11s
release / Build + push container image (push) Successful in 16s
release / Verify release artifacts (tag releases only) (push) Skipped
Operator, 2026-09-10: "it should also be able to include music from the same
artist." Completes #3881 — the weights and pool landed in f367eeaa; this is
the eligibility half.

produceSeedMixes filtered the seed artist out entirely:

    // "Songs like X" excludes X's own songs.
    if !pgtypeUUIDEqual(c.Track.ArtistID, artistID) { ... }

That reads as obviously right and is not. The seed is a TRACK — the artist's
top-played one — and the tracks most likely to sound like it are usually the
rest of that artist's catalogue. The filter threw away the seed's nearest
neighbours, then reached FURTHER OUT to replace them. On the one surface
whose job is staying in a neighbourhood, that is backwards, and it worked
against the coherence tuning rather than with it.

Domination is bounded by the cap instead of by exclusion, which is the
distinction that makes this safe rather than a new problem:
capCandidatesByAlbumAndArtist already allows at most 3 tracks per artist in
a 25-track mix, so the seed artist gets 12% at most — a presence, not a
takeover. Without that bound this would just be the radio failure (#3882)
arriving on a different surface. The seed track itself still cannot appear;
it is passed to LoadCandidatesFromSimilarity as an exclusion.

Guarded end-to-end rather than by reading the source, for two reasons: the
check has to survive the filter returning in a different shape, and an
absence check would now match the comment that explains why the filter is
gone — rule 167's prose trap exactly. The test asserts both directions, that
at least one mix contains its seed artist and that none exceeds the cap.

Its falsification is by construction rather than by execution: under the
previous code every mix's own-artist count was necessarily zero, so the
assertion could not have passed. Running it needs Postgres, which is the
integration lane's job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 21:13:32 -04:00
bvandeusenandClaude Opus 5 ecfa056d4d fix(recommendation): don't shrink a candidate arm ordered by unseeded random()
test-go / test (push) Successful in 1m10s
test-go / integration (push) Successful in 3m35s
release / Build signed APK (releases and dev) (push) Successful in 5m6s
release / Build + push container image (push) Successful in 16s
release / Verify release artifacts (tag releases only) (push) Skipped
Fixes the integration failure from f367eeaa:
"same-day rebuild produced different track lists".

Cutting RandomFill 30→10 for Songs-like broke
TestBuildSystemPlaylists_DailyNonceDeterminism, and the reason is worth
stating because the number is not the bug.

`likes_overlap` and `random_fill` end in a bare `ORDER BY random()` with no
daily seed (recommendation.sql:118, :161). Such an arm returns a STABLE set
only while its LIMIT exceeds the rows eligible for it — then it returns all
of them, and the random order stops mattering because scoreAndSortCandidates
sorts by track id before drawing jitter. Below that threshold the arm
returns a random SUBSET, and two builds on the same day draw different ones.

So the test was green by accident. It seeds ~20 tracks against a default
RandomFill of 30; the limit exceeded the library, so the arm returned
everything. Determinism held for a reason unrelated to the code being right.

Which means it does NOT hold in production. Any real library is larger than
30, so same-day rebuilds have been drawing different mixes since that arm
was written — invisible, because a mix changing after a refresh looks like a
feature. Filed as #3889; the fix is a seeded ordering per (user, day), which
needs a .sql change and sqlc regeneration and so cannot land from here.

The correction: grow an arm freely, never shrink one whose ordering is
unseeded random. LikesOverlap and RandomFill go back to the defaults;
TasteOverlap stays halved because it sorts by `tpa.weight DESC, t.id` and is
genuinely deterministic. Guarded by a test that names the reasoning, so the
next person to trim these has to read why first — and it should be DELETED
once #3889 lands rather than worked around.

The cost is honest: the seed-independent share of the Songs-like pool falls
from 29% to 20% instead of the intended cut. That matters less than it
sounds. The pool only biases the draw; the songs_like WEIGHTS are what
actually demote sim_score-0 candidates, and they are untouched here — a
perfect match still scores 5.00 against an unrelated favourite's 2.00.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 20:59:39 -04:00
bvandeusenandClaude Opus 5 f367eeaa9d fix(recommendation): Songs-like gets its own profile so it stops wandering
test-web / test (push) Successful in 1m7s
test-go / test (push) Successful in 1m31s
test-go / integration (push) Failing after 4m21s
release / Build signed APK (releases and dev) (push) Successful in 5m11s
release / Build + push container image (push) Successful in 1m52s
release / Verify release artifacts (tag releases only) (push) Skipped
Operator, 2026-09-10: "when I play it I'm expecting to get a consistent
sound and style from the experience... I was getting a seeming wide variety
of music from each one when I was hoping to stay in a certain neighborhood."

Songs-like shared the `daily_mix` weight profile with For-You, and that
sharing WAS the bug. The two surfaces want opposite things: For-You answers
"what will they enjoy today" and is supposed to roam; Songs-like answers
"what sounds like THIS". Under one profile the broad answer wins.

The arithmetic, from the shared weights:

    unrelated track, liked, not played recently → 1.0 + 2.0 + 1.0 = 4.0
    PERFECT similarity match, not liked         → 1.0 + 1.5       = 2.5

Liking something outranked sounding like the seed, because LikeBoost (2.0)
exceeded SimilarityWeight's whole range (1.5) and TasteWeight (1.5, and
seed-INDEPENDENT) matched it outright. Under the new profile the same pair
scores 5.00 vs 2.00.

Two levers, because either alone leaves the other's failure intact:

POOL. Songs-like now takes its own CandidateSourceLimits. The default gave
~29% of candidates a sim_score of literally zero — `taste_overlap` and
`random_fill` are both `0.0::float8` in recommendation.sql, seed-independent
by construction. Same total pool size; composition shifts to arms that
measure distance from the seed, LBSimilar doubled.

WEIGHTS. A third profile beside radio and daily_mix, DB-backed and live per
rule 25, with the property that similarity's range exceeds the combined
range of every seed-independent differentiator — so a closer match cannot
be beaten on likes, freshness and taste alone, while tracks within ~0.39
similarity of each other still get ordered by what the user likes.

Rule 131 changed the pool design mid-way and for the better. Zeroing the
two seed-independent arms was the first instinct and is exactly the
vanish-or-nothing shape that rule forbids: a seed with thin ListenBrainz
coverage would yield a short mix or none. They are the tier-3 FLOOR — cut
hard, never removed — and the weights keep them at the bottom of the
ranking rather than out of the pool. "A few tracks further from the seed
than we'd like" beats "no playlist".

Caught while wiring it: switching only pickTopN's final Score would have
been nearly INERT. scoreAndSortCandidates does the selection sort, and the
caller caps and truncates in that order — so the playlist would still have
been chosen by daily_mix and merely relabelled with songs_like numbers. It
now takes the profile as a parameter, and each surface passes its own.

Also corrects the daily_mix card's blurb, which claimed Songs-like as one
of its surfaces and no longer is.

Guards pin behaviour rather than the numbers, since numbers get retuned:
that similarity beats an unrelated liked track, that daily_mix still
DOESN'T (or the split buys nothing), that the tier-3 floor is non-zero,
and that the UI card shows its own values rather than falling back. Each
falsified against its named regression first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 20:51:15 -04:00
bvandeusenandClaude Opus 5 270ad7a71b fix(ci): tests do not ship, so they must not re-version an artifact
test-go / test (push) Successful in 1m1s
test-go / integration (push) Successful in 3m17s
release / Build signed APK (releases and dev) (push) Successful in 4m41s
release / Build + push container image (push) Successful in 16s
release / Verify release artifacts (tag releases only) (push) Skipped
Completes the pathspec. 17212e9e excluded CI, docs and tooling but left
tests in the shipped set, so its own commit re-versioned the image on the
strength of a _test.go file. `go build` drops *_test.go outright and the
Vite build never imports a .test.ts — neither reaches an image or an APK.

Globs over files rather than a directory exclusion, because this repo has
no tests/ tree to exclude: Go tests sit inline beside the code they cover
(158 files) and the web suite beside its modules (115). Patterns match what
exists and nothing speculative — there are no .spec.* files, no __tests__/
directories and no androidTest/ tree. If any appear they re-version until
named, which is the harmless direction and the point of a denylist.

The guard that matters is not "a test-only commit is inert" — it is that a
commit touching a test AND its source still moves the version. `':!internal'`
would satisfy every inertness assertion while silently excluding the entire
server, which is the stale-version-on-changed-artifact failure this whole
derivation exists to prevent.

Falsified: drop the Go exclusion and a _test.go commit moves the version;
drop the web one and a .test.ts does; replace the globs with `':!internal'`
and the source-alongside-test case breaks.

That last check failed first time, on a bug in the FIXTURE rather than the
derivation, and it is worth recording because it makes a test pass for the
wrong reason. Both commit helpers wrote the constant "x\n", so re-writing a
file with identical bytes recorded NOTHING — the "source and test together"
commit actually contained only the test, and the assertion was quietly
checking the case it was meant to contrast against. Content is now derived
from the commit's epoch, and the test asserts HEAD really contains both
paths before drawing any conclusion from it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 18:21:12 -04:00
bvandeusenandClaude Opus 5 17212e9eb4 fix(ci): version derives from the shipped set; untrack an 18MB binary
test-go / test (push) Successful in 1m5s
test-go / integration (push) Successful in 3m30s
release / Build signed APK (releases and dev) (push) Successful in 5m10s
release / Build + push container image (push) Successful in 1m31s
release / Verify release artifacts (tag releases only) (push) Skipped
Three build-hygiene fixes that turned up while explaining the pathspec.

**version.sh derives from what SHIPPED.** It read bare HEAD, so any commit
moved the version — including one touching only CI or a README. Rules 148
and 149 both specify the pathspec form. Now a denylist, and the direction
is the point: as an allowlist the list must be updated by whoever adds a
directory and nothing fails if they don't, so the failure mode is a changed
artifact keeping its old version silently on a green run. Inverted, new
content counts by default.

android/ is deliberately NOT excluded, and that is the subtle part. This
repo ships TWO artifacts from ONE derivation: android/ is in no server
image, but it is the APK's entire source, and excluding it would stop an
Android-only commit from moving the APK's own version — the silent
downgrade the versioning rework exists to prevent. So the list is the
union: exclude only what ships in neither, and accept that an Android
commit also nudges the server's reported version. Over-inclusion across the
two, which is the harmless direction. roundtable/roundtable-android each
keep tighter lists because they are one-artifact repos; don't copy theirs.

**.dockerignore excluded the wrong CI directory.** It named .forgejo/ and
.github/, neither of which this repo has. Gitea Actions reads .gitea/, so
the one directory that exists was the one not excluded. The "Flutter mobile
client" block had also lost its PATTERN when flutter_client/ was deleted,
leaving a comment describing an exclusion that was not happening — android/
never took its place, so 4.1MB of Gradle project entered the context and
busted the `COPY . .` layer on every Android-only change. bin/ excluded too.

**bin/minstrel was tracked** — an 18MB binary last refreshed by a commit
about web test mocks, and re-dirtied by every `make build` since. Untracked
and ignored; the file stays on disk.

Guards are behavioural rather than textual: they build throwaway repos with
pinned commit timestamps and run version.sh against them, so they break when
the derivation changes rather than when the wording does. Falsified — drop
the .gitea exclusion and the CI-only commit moves the version; add an
android exclusion and an Android commit stops moving it; exclude everything
and a source commit refuses.

One honest note on the refusal test: the script already refused an empty
result via the downstream date check, so the new explicit check improves the
diagnostic ("no commit touches the shipped file set — shallow clone?") and
not the safety. The test pins the property, which is defended in depth.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 17:55:36 -04:00
bvandeusenandClaude Opus 5 8f4b76a638 fix(ci): artifacts move to stock upload-artifact@v7 / download-artifact@v8
test-go / test (push) Successful in 1m4s
test-go / integration (push) Successful in 3m54s
android / Build + lint + test (push) Successful in 4m58s
release / Build signed APK (releases and dev) (push) Successful in 5m11s
release / Build + push container image (push) Successful in 1m52s
release / Verify release artifacts (tag releases only) (push) Skipped
android.yml's debug upload and release.yml's minstrel-apk pair went
through the bvandeusen fork mirrors, with comments saying stock actions
refuse this hostname, that the pair had to be matched on the bundled
@actions/artifact major, and that download v7 was off-limits for node24.
None of that holds on gitea/runner 3.x: the runner edits the GHES refusal
out of the action bundles, every download major v4-v8 reads every upload
major v4-v7 (Scribe spike #3843, CI-runner run 6312), and every CI image
carries Node 24. The mirror pair itself was last verified at tag run 6286.

Same artifact names, paths and if-no-files-found. ci-requirements.md
drops the pairing table and keeps what is still true: @v3 is invisible.

Scribe snippet #2271, milestone 395.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DwoKYuw3qJmUUYsJeNherB
2026-09-10 17:25:10 -04:00
bvandeusenandClaude Opus 5 aeb8781c4e fix(release): drop version image tags, mint the rollback unit on main
test-go / test (push) Successful in 1m43s
test-web / test (push) Successful in 1m13s
test-go / integration (push) Successful in 4m12s
release / Build signed APK (releases and dev) (push) Successful in 5m11s
release / Build + push container image (push) Successful in 38s
release / Verify release artifacts (tag releases only) (push) Skipped
The image tag map was the inverse of family rules 145 and 147 on every
count: it published :vYYYY.MM.DD.HHMM that nobody pinned, published :main
that rule 147 says should not exist, and published no commit-addressable
image at all — so the rollback unit the rule names did not exist in this
repo. A bad main push had nothing to roll back to but the previous
release tag, which may be many commits back.

The whole map is now:

  dev  → :dev
  main → :latest + :<sha>
  tag  → :latest

A release refreshes the channel and mints nothing else. The tag build
rebuilds the SAME SOURCE as main's build minutes earlier, differing only
in which APK is baked in, so rule 145's immutability clause applies
directly: move the channel tag, never re-push a commit-addressable one.
:latest has to move here rather than waiting for the next main push, or
the channel would carry the previous release's APK indefinitely — a
channel that cannot refresh itself (rule 146).

Two consequences that are not optional:

The verify job asserted the :<version> image existed. With version tags
gone that would fail every release for a tag nothing mints. Re-pointed at
the :<sha> image rather than deleted — deleting it is the tempting way to
make a failing guard go green, and it earns its keep twice now: it still
catches an image push that silently did not happen, and it additionally
proves the ordering, since a tag cut on a commit whose main build never
completed has no rollback target.

The server's self-reported version was the literal string "main" or
"dev". That was survivable while :vYYYY.MM.DD.HHMM existed to identify a
build; with version tags gone it is the ONLY thing that says which build
is running, and two dev images months apart were indistinguishable. It
now carries the derived name from ci/version.sh on every lane, with the
channel as a sibling field (rule 149) rather than folded into the string.
Surfaced at /healthz and beside the version in Settings.

Guards added for each arm of the policy, and every one was falsified
against the specific regression it names before committing. That caught
two real bugs in the guards themselves: stepBody cut at the next
`- name:`, which returns an EMPTY body for the last step in a job and
made the assertions pass vacuously, and its replacement cut at any blank
line followed by indentation, which truncated a step mid-run-block. The
helper now refuses an empty body outright.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 15:10:15 -04:00
342 changed files with 25596 additions and 1522 deletions
+19 -5
View File
@@ -6,9 +6,20 @@
**/build
web/build
# Flutter mobile client — built separately on developer machines / Flutter CI.
# Including it in the Go build context wastes ~70 files and invalidates the
# `COPY . .` layer cache on every Flutter-only change.
# The Android client — built by its own job, never from this context. The APK
# reaches the image through client/, downloaded as a CI artifact, so nothing
# here reads android/ sources.
#
# This block named `flutter_client/` until 2026-09-10 and lost its PATTERN when
# that tree was deleted, leaving a comment describing an exclusion that was no
# longer happening. android/ never took its place, so 4.1 MB of Gradle project
# has been entering the context and busting the `COPY . .` layer on every
# Android-only change.
android/
# Local `make build` output — an 18 MB binary the image never uses, since the
# builder stage compiles its own.
bin/
# Docs and IDE noise
docs/
@@ -26,5 +37,8 @@ docs/
!.env.example
# CI workflow files don't need to ship in the image.
.forgejo/
.github/
#
# This said `.forgejo/` and `.github/` — neither of which this repo has. Gitea
# Actions reads `.gitea/`, so the one directory that actually exists was the
# one not excluded, and every workflow edit invalidated the context.
.gitea/
-95
View File
@@ -1,95 +0,0 @@
name: android
# Native Android (Kotlin/Compose/Media3) — M8 rewrite, now the only client.
# This workflow is testing only — lint + detekt + unit tests on every push
# to dev/main, plus a debug APK artifact for main. The signed-release
# build + asset attach + image-bundling lives in release.yml under a
# `needs:` chain so the docker image cannot ship without the APK.
on:
push:
branches: [main, dev]
paths:
- 'android/**'
- '.gitea/workflows/android.yml'
# pull_request trigger intentionally omitted — see test-web.yml for
# the rationale (single-author repo, push covers PR-merge equivalent).
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
# Silences the JDK 22+ "restricted method in java.lang.System has been
# called" warning that Gradle 9.1's bundled native-platform jar trips
# at launch (System.load for native primitives). Affects the LAUNCHER
# JVM, not the daemon — that's why org.gradle.jvmargs in
# gradle.properties isn't enough. Future-compat: required opt-in once
# JDK 25 promotes the warning to an error.
JAVA_TOOL_OPTIONS: "--enable-native-access=ALL-UNNAMED"
jobs:
build:
name: Build + lint + test
# Using flutter-ci runner label because it's the only proven-working
# label with docker that can pull our container.image. Switch to
# android-ci once the operator registers that runner label.
runs-on: flutter-ci
container:
image: git.fabledsword.com/bvandeusen/ci-android:36
defaults:
run:
working-directory: android
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Cache Gradle dirs
# Resolved deps + Gradle distribution + Kotlin daemon caches.
# Saves ~3 min per CI run after the first warm-up.
uses: actions/cache@v4
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
~/.kotlin
key: gradle-${{ runner.os }}-${{ hashFiles('android/gradle/wrapper/gradle-wrapper.properties', 'android/gradle/libs.versions.toml', 'android/**/*.gradle.kts') }}
restore-keys: |
gradle-${{ runner.os }}-
- name: Make gradlew executable
run: chmod +x ./gradlew
- name: Gradle wrapper validation
run: ./gradlew --version
- name: ktlint
run: ./gradlew ktlintCheck
- name: detekt
run: ./gradlew detekt
- name: Unit tests
run: ./gradlew testDebugUnitTest
- name: Assemble debug
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
run: ./gradlew assembleDebug
- name: Upload debug APK
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
# Mirrored action, never actions/upload-artifact. @v4+ throws
# GHESNotSupportedError client-side on the hostname (no server setting
# reaches that check), and @v3 is worse — it reports success while Gitea
# serves artifacts back only through the v4 API, so the upload is stored
# and invisible to every retrieval path. @v3 is what left 72 unreachable
# artifacts on this repo. Pinned by SHA because the mirror auto-syncs;
# full URL because DEFAULT_ACTIONS_URL sends bare owner/repo to github.com.
# See Scribe issues 2255 / 2270.
uses: https://git.fabledsword.com/bvandeusen/upload-artifact@cb8afe72b42edc798abfb8fcb556cf660d894245
with:
name: minstrel-android-debug-${{ github.sha }}
path: android/app/build/outputs/apk/debug/app-debug.apk
if-no-files-found: error
+500 -67
View File
@@ -2,11 +2,31 @@ name: release
# Builds and pushes the minstrel container image to the Gitea registry.
#
# push to dev → :dev (freshly-built dev APK bundled)
# push to main → :main and :latest (latest-release APK bundled)
# push tag vYYYY.MM.DD.HHMM → :vYYYY.MM.DD.HHMM and :latest (fresh APK bundled)
# push to dev → :dev (freshly-built dev APK bundled)
# push to main → :latest + :<sha> (latest-release APK bundled)
# push tag vYYYY.MM.DD.HHMM → :latest (fresh APK bundled)
# workflow_dispatch → manual trigger (same rules based on the ref)
#
# That is the whole tag map, and it is family rule 145 + 147 as written.
#
# :<sha> on main is the ROLLBACK UNIT — every production commit addressable
# without a release ceremony. It is minted only on main, where rollback is
# actually worth having: merges are gated (rule 2) so they number in the dozens
# per year, while on dev they would be one per push, forever, for a channel
# whose entire contract is that it moves.
#
# There are NO :<version> image tags. This repo published :vYYYY.MM.DD.HHMM
# until 2026-09-10 and it was the inverse of the rule on both counts — minting
# a version tag nobody pinned while the rollback unit the rule names did not
# exist here at all. Git and the build's own self-reported version answer
# "which build is this"; a third name for the same thing is upkeep for a model
# we do not run. Operator, 2026-09-10: "only things like the APK need that kind
# of versioning for their update process."
#
# There is no :main either. :latest tracks main's tip with no gate between them
# (rule 147), so a second name for the same image sends readers looking for a
# distinction that does not exist.
#
# The dev channel exists so testing a build does not require shipping one.
# Before it, the only way to get an APK onto a phone was to cut a release,
# which made `main` the staging area by default. `:dev` carries its own
@@ -52,9 +72,8 @@ name: release
# Android APK and uploads it as a workflow artifact. The image-release
# job declares `needs: android-release`, so the docker image cannot
# start building until the APK is guaranteed-ready — no polling, no
# race, no silent-failure mode. Asset attachment to the gitea Release
# happens in the same android-release job, so the Release-page download
# link and the in-image bundled APK are both populated atomically.
# race, no silent-failure mode. Attaching the APK to the gitea Release is
# its own job (release-assets), behind the test gate below.
#
# :latest always carries an APK. Because every main push also moves
# :latest (not just tags), a main build with no APK would silently strip
@@ -64,8 +83,33 @@ name: release
# recomputed ones — so no rebuild is needed, just a rebundle. Tag builds
# keep bundling their own freshly-built APK.
#
# Android testing (lint + detekt + unit tests, debug APK upload on main)
# lives in android.yml and runs independently on every push.
# THE GATE (rule 177, M462 #4984). Every verifying lane lives in this file —
# Go (vet, lint, short tests), the Postgres integration suite, the web app
# (npm audit, svelte-check, vitest), Android (ktlint, detekt, unit tests) and
# govulncheck — and every job that publishes something names each of them in
# `needs:` and requires `success` from each, by name. Nothing publishes on red.
#
# They used to be three separate workflows (test-go, test-web, android) on the
# same push trigger as this one. Separate workflows cannot see each other's
# verdict, so :dev meant "it built", never "it passed": a red test run and a
# fresh :dev could carry the same timestamp. One graph is the only place the
# edge can be written.
#
# A skipped lane is NOT a pass. The publishing conditions check
# `result == 'success'` per lane rather than `!failure()`, so a lane that
# never started blocks the publish exactly as a red one does. Lanes carry no
# path filters for the same reason: a web-only push still runs the Go suite,
# because "not run" must never read as "passed".
#
# What publishes, and is therefore gated: the image tags (image-release) and
# the APK + version sidecar attached to a tag's Release (release-assets).
# android-release only BUILDS the signed APK into a workflow artifact, which
# nobody outside this run can pull, so it runs in parallel with the lanes
# instead of after them; attaching it to the Release is the publishing half,
# and that half waits for the gate.
#
# To watch the gate refuse: dispatch this workflow with force_red=true. The go
# lane fails on purpose, and both publishing jobs must report skipped.
on:
push:
@@ -75,6 +119,11 @@ on:
- 'docs/**'
- '**/*.md'
workflow_dispatch:
inputs:
force_red:
description: Fail the go lane on purpose, to check that nothing publishes on red
type: boolean
default: false
# A rapid re-push to main should supersede the in-flight build — the
# operator explicitly wants the later commit to win. Tags no longer enter
@@ -85,6 +134,269 @@ concurrency:
cancel-in-progress: true
jobs:
# ---------------------------------------------------------------- lanes --
# Verifying jobs. Each one is named in the `needs:` of every publishing job
# below; add a lane here and it must be added there in the same commit.
go:
runs-on: go-ci
container:
image: git.fabledsword.com/bvandeusen/ci-go:1.26
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Forced failure (gate check)
if: github.event.inputs.force_red == 'true'
run: |
echo "::error::force_red dispatch: failing on purpose so the publishing jobs must skip"
exit 1
- name: Toolchain versions
run: |
go version
golangci-lint --version
- name: Generated code matches queries (sqlc)
run: make verify-generate
- name: go vet
run: go vet ./...
- name: golangci-lint
run: golangci-lint run ./...
- name: go test (short, race)
run: go test -short -race ./...
# Full `go test -race` against an ephemeral Postgres.
#
# DB wiring follows the act_runner shared-daemon pattern: the runner's Docker
# daemon also runs the operator's dev compose stack, so service containers
# get NO published ports (collision) and no service-name DNS. We discover the
# service container through the mounted docker socket and reach it by bridge
# IP. The exactly-one assertion is a hard guard — pointing tests at the dev
# Postgres would truncate it (the disaster Fable #339 exists to prevent).
#
# The key stays `integration` with no `name:` (rule 80): act_runner derives
# the service container's name from the job's display name.
#
# `web/build/` has a committed placeholder index.html so go:embed succeeds
# without the SPA being built first.
integration:
runs-on: go-ci
container:
image: git.fabledsword.com/bvandeusen/ci-go:1.26
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: minstrel
POSTGRES_PASSWORD: minstrel
POSTGRES_DB: minstrel_test
# No `ports:` — the runner shares the operator's dev compose
# Docker daemon; publishing a fixed host port collides.
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Integration suite (discover service by bridge IP, migrate, test)
run: |
set -eux
# Discover THIS job's Postgres service container via the
# mounted docker socket. act_runner attaches the job
# container and its service container(s) to a shared per-job
# network, so scope discovery to a postgres that sits on a
# network THIS job container is also on. The old
# `--filter name=integration` matched EVERY concurrent
# integration run's postgres (a dev push + the main-merge run
# overlap → 2 candidates → false "expected exactly 1" abort).
# The operator's dev compose `minstrel-postgres-*` is never on
# this job's network; skip it explicitly as belt-and-suspenders
# (a wrong target would truncate real data).
SELF=$(cat /etc/hostname)
SELF_NETS=$(docker inspect -f '{{range $k,$v := .NetworkSettings.Networks}}{{$k}} {{end}}' "$SELF")
test -n "$SELF_NETS"
echo "self ($SELF) networks: $SELF_NETS"
PG_ID=""
PG_NAME=""
for cid in $(docker ps --filter "ancestor=postgres:16-alpine" -q); do
nm=$(docker inspect -f '{{.Name}}' "$cid" | sed 's#^/##')
case "$nm" in *minstrel-postgres*|*_postgres_*) continue ;; esac
for net in $(docker inspect -f '{{range $k,$v := .NetworkSettings.Networks}}{{$k}} {{end}}' "$cid"); do
case " $SELF_NETS " in *" $net "*) PG_ID="$cid"; PG_NAME="$nm"; break 2 ;; esac
done
done
test -n "$PG_ID" || { echo "FATAL: no postgres service container on this job's network (self nets: $SELF_NETS)"; exit 1; }
echo "selected postgres: $PG_ID $PG_NAME"
PG_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$PG_ID")
test -n "$PG_IP"
export MINSTREL_TEST_DATABASE_URL="postgres://minstrel:minstrel@${PG_IP}:5432/minstrel_test?sslmode=disable"
# Wait for Postgres to accept connections. Asked of the service
# container itself: the run: shell is dash (rule 81), where the
# old `/dev/tcp` probe never connects and the loop silently
# burned its full two minutes on every run.
ready=""
for i in $(seq 1 60); do
if docker exec "$PG_ID" pg_isready -U minstrel -d minstrel_test -q; then ready=1; break; fi
sleep 2
done
test -n "$ready" || { echo "FATAL: postgres never became ready"; exit 1; }
# Relax durability on the throwaway CI Postgres. Our test pattern
# is dbtest.ResetDB → TRUNCATE … RESTART IDENTITY CASCADE before
# every test, and the per-TRUNCATE commit fsync is the dominant
# cost of the integration suite. The CI DB is rebuilt every run so
# fsync / full_page_writes / synchronous_commit buy nothing. Apply
# via docker exec because:
# - The act_runner `services:` block can't override the container
# command, so `postgres -c fsync=off` at boot isn't an option.
# - ALTER SYSTEM cannot run inside a transaction; psql -c
# auto-commits each statement, which is what we need.
# - fsync / full_page_writes are sighup GUCs and
# synchronous_commit is user-context, so pg_reload_conf() picks
# all three up with no restart.
# Non-fatal: a perms surprise degrades to "slower", never red CI.
docker exec "$PG_ID" psql -U minstrel -d minstrel_test \
-c "ALTER SYSTEM SET fsync = off" \
-c "ALTER SYSTEM SET synchronous_commit = off" \
-c "ALTER SYSTEM SET full_page_writes = off" \
-c "SELECT pg_reload_conf()" \
|| echo "WARN: durability relax failed; continuing"
# Apply embedded migrations to the fresh test DB, then run the
# full suite (no -short → integration tests execute). -p 1:
# every integration package TRUNCATEs the one shared test DB;
# concurrent package binaries → TRUNCATE deadlocks. Serialize
# package execution (the documented local invocation too).
# -timeout 20m: internal/api alone takes ~6.5 min under -race on
# an idle runner, and a dev and a main run sharing the runner
# pushed it past go test's default 10m (run 8368, 600.016s, the
# running test 2s old — nothing hung).
MINSTREL_DATABASE_URL="$MINSTREL_TEST_DATABASE_URL" go run ./cmd/minstrel migrate
go test -p 1 -race -timeout 20m ./...
web:
runs-on: go-ci
container:
image: git.fabledsword.com/bvandeusen/ci-go:1.26
defaults:
run:
working-directory: web
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Install deps
run: npm ci
# What ships to browsers: `dependencies` and the runtime they pull in
# (svelte, devalue). Build and test tooling (vite, vitest, tailwind,
# kit's dev server) is left out because none of it reaches a user, and
# its open advisories need major-version upgrades tracked separately.
- name: npm audit (shipped dependencies)
run: npm audit --omit=dev --audit-level=moderate
- name: Type-check + svelte-check
run: npm run check
- name: Vitest
run: npm test
android:
# Using flutter-ci runner label because it's the only proven-working
# label with docker that can pull our container.image.
runs-on: flutter-ci
container:
image: git.fabledsword.com/bvandeusen/ci-android:36
defaults:
run:
working-directory: android
env:
# Silences the JDK 22+ "restricted method in java.lang.System has been
# called" warning that Gradle's bundled native-platform jar trips at
# launch. Affects the LAUNCHER JVM, not the daemon — that's why
# org.gradle.jvmargs in gradle.properties isn't enough.
JAVA_TOOL_OPTIONS: "--enable-native-access=ALL-UNNAMED"
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Cache Gradle dirs
uses: actions/cache@v4
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
~/.kotlin
key: gradle-${{ runner.os }}-${{ hashFiles('android/gradle/wrapper/gradle-wrapper.properties', 'android/gradle/libs.versions.toml', 'android/**/*.gradle.kts') }}
restore-keys: |
gradle-${{ runner.os }}-
- name: Make gradlew executable
run: chmod +x ./gradlew
- name: Gradle wrapper validation
run: ./gradlew --version
- name: ktlint
run: ./gradlew ktlintCheck
- name: detekt
run: ./gradlew detekt
- name: Unit tests
run: ./gradlew testDebugUnitTest
# No debug APK is built or uploaded here. Main used to upload a
# debug-signed app-debug.apk: a build signed by a key regenerated in
# every container, which no install can update (family idea #5103,
# practice 2). Phones get builds from android-release, signed with
# the one release key, on dev and on tags.
# Known vulnerabilities in the Go code and the standard library it is built
# with. Runs in the SAME image the Dockerfile's builder stage uses, so the
# standard library it checks is the one that ends up in the shipped binary;
# the ci-go image carries its own Go and would be checking a different
# toolchain. Keep this image and the Dockerfile's builder in step.
#
# govulncheck is fetched at CI time, unpinned (rule 154): the vulnerability
# database and the tool that reads it should both be current.
govulncheck:
runs-on: go-ci
container:
image: golang:1.26-bookworm
steps:
# Plain git, not actions/checkout: that action runs on node, which the
# golang image does not carry. This step is dash (rule 81).
- name: Checkout
env:
TOKEN: ${{ github.token }}
run: |
set -eu
auth=$(printf 'x-access-token:%s' "$TOKEN" | base64 -w0)
git init -q .
git remote add origin "${{ github.server_url }}/${{ github.repository }}.git"
git -c http.extraHeader="Authorization: Basic ${auth}" fetch -q --depth 1 origin "${{ github.sha }}"
git checkout -q FETCH_HEAD
git log -1 --format='%H %s'
- name: govulncheck
run: |
go version
go run golang.org/x/vuln/cmd/govulncheck@latest ./...
# ------------------------------------------------------------ artifacts --
android-release:
name: Build signed APK (releases and dev)
# Also builds on `dev`, which is what makes a test channel possible at
@@ -211,14 +523,49 @@ jobs:
-PMINSTREL_VERSION_NAME=${{ steps.ver.outputs.name }} \
-PMINSTREL_VERSION_CODE=${{ steps.ver.outputs.code }}
# The APK every phone updates from must carry THE release key: Android
# updates an app in place only when the signer matches, so an APK
# signed by any other key (debug, a regenerated keystore, a swapped
# secret) reaches no installed phone. The certificate's digest is
# pinned below; it is public, not a secret. Gradle signs with the
# release key or leaves the APK unsigned, and an unsigned build fails
# here too, as there is no app-release.apk to verify (family idea
# #5103, practice 3). apksigner, not keytool: keytool prints nothing
# for a v2-only APK.
#
# Rotating the key on purpose means every install must be removed and
# reinstalled; change the digest here in the same commit.
- name: The APK carries the release key
shell: bash
env:
# CN=Minstrel, O=FabledSword. Read from run 8446 (#5116).
RELEASE_CERT_SHA256: 43d183307bc46b821789d90444a960b137f78f2166ff431efb2406d0fceaf612
run: |
set -euo pipefail
sdk="${ANDROID_HOME:-${ANDROID_SDK_ROOT:-}}"
signer="$(ls "$sdk"/build-tools/*/apksigner 2>/dev/null | sort -V | tail -1 || true)"
test -n "$signer" || { echo "::error::no apksigner under '$sdk/build-tools'"; exit 1; }
certs="$("$signer" verify --print-certs app/build/outputs/apk/release/app-release.apk)"
printf '%s\n' "$certs" | grep -E '^Signer #[0-9]+ certificate (DN|SHA-256 digest)'
# One signer, and it is ours. A second signer would be a lineage or
# a mistake; either way not something to ship unexamined.
digests="$(printf '%s\n' "$certs" | sed -n 's/^Signer #[0-9]* certificate SHA-256 digest: //p')"
if [ "$digests" != "$RELEASE_CERT_SHA256" ]; then
if printf '%s' "$certs" | grep -q 'CN=Android Debug'; then
echo "::error::the release APK is signed with a debug key"
else
echo "::error::the release APK is not signed by the release key: got '${digests//$'\n'/ }', want ${RELEASE_CERT_SHA256}"
fi
exit 1
fi
- name: Upload APK as workflow artifact
# Mirrored action, never actions/upload-artifact — @v4+ refuses on the
# hostname, @v3 uploads something Gitea will never serve back. This is
# the producing half of a pair: image-release downloads `minstrel-apk`
# below with the matching download-artifact mirror. Both must stay on
# the v4 protocol — mixing a v3 upload with a v4 download (or the
# reverse) yields an empty listing, not an error. See Scribe 2255 / 2270.
uses: https://git.fabledsword.com/bvandeusen/upload-artifact@cb8afe72b42edc798abfb8fcb556cf660d894245
# Stock action (snippet #2271) — never @v3, which uploads something Gitea
# will never serve back. This is the producing half of a pair:
# image-release downloads `minstrel-apk` below. Any upload v4+ pairs with
# any download v4+ on this forge (every combination tested 2026-09-10,
# Scribe spike #3843), so the two pins need not move together.
uses: actions/upload-artifact@v7
with:
name: minstrel-apk
path: android/app/build/outputs/apk/release/app-release.apk
@@ -226,21 +573,47 @@ jobs:
# artifact existing, so an empty upload must fail here, not there.
if-no-files-found: error
# Publishes the signed APK and its version sidecar on the tag's Release.
# Split out of android-release so the APK can be BUILT in parallel with the
# lanes while being PUBLISHED only once they have all passed.
release-assets:
name: Attach APK to the Release (tag releases only)
needs: [go, integration, web, android, govulncheck, android-release]
if: >-
${{
!cancelled()
&& needs.go.result == 'success'
&& needs.integration.result == 'success'
&& needs.web.result == 'success'
&& needs.android.result == 'success'
&& needs.govulncheck.result == 'success'
&& needs.android-release.result == 'success'
&& startsWith(github.ref, 'refs/tags/v') }}
runs-on: go-ci
container:
image: git.fabledsword.com/bvandeusen/ci-go:1.26
steps:
- name: Download signed APK artifact
uses: actions/download-artifact@v8
with:
name: minstrel-apk
path: release-apk/
- name: Attach APK to gitea Release
# Tag releases only. A dev build has no Release to hang assets on and
# does not need one — the :dev image bundles the APK, and the server
# serves it from /api/client/apk like any other.
if: startsWith(github.ref, 'refs/tags/v')
shell: bash
env:
CI_TOKEN: ${{ secrets.CI_TOKEN }}
VERSION_NAME: ${{ steps.ver.outputs.name }}
VERSION_CODE: ${{ steps.ver.outputs.code }}
VERSION_NAME: ${{ needs.android-release.outputs.version_name }}
VERSION_CODE: ${{ needs.android-release.outputs.version_code }}
run: |
set -euxo pipefail
TAG="${GITHUB_REF#refs/tags/}"
REPO="${GITHUB_REPOSITORY}"
APK_PATH="app/build/outputs/apk/release/app-release.apk"
APK_PATH="release-apk/app-release.apk"
ls -lh "${APK_PATH}"
# Publish the version sidecar as a release asset next to the APK.
@@ -294,13 +667,21 @@ jobs:
image-release:
name: Build + push container image
# `needs:` waits for android-release. For tag pushes android-release
# runs and must succeed before this job starts — guaranteeing the
# APK artifact is present. For main pushes android-release is
# skipped; the `if: ...` below lets this job run anyway and the
# download/copy steps gate themselves on the tag context.
needs: [android-release]
if: ${{ !failure() && !cancelled() }}
# Every lane must have SUCCEEDED, each named here (rule 177). Then the
# APK: tag and dev pushes build one and it must have succeeded; main
# pushes skip android-release and bundle the latest release's APK
# instead, so for main alone a skipped android-release is expected.
needs: [go, integration, web, android, govulncheck, android-release]
if: >-
${{
!cancelled()
&& needs.go.result == 'success'
&& needs.integration.result == 'success'
&& needs.web.result == 'success'
&& needs.android.result == 'success'
&& needs.govulncheck.result == 'success'
&& (needs.android-release.result == 'success'
|| (needs.android-release.result == 'skipped' && github.ref == 'refs/heads/main')) }}
runs-on: go-ci
container:
image: git.fabledsword.com/bvandeusen/ci-go:1.26
@@ -312,11 +693,16 @@ jobs:
- name: Checkout
uses: actions/checkout@v4
with:
# Shallow is fine here. This job used to need full history + tags to
# re-derive the bundled APK's version from the tagged commit; it now
# downloads the sidecar the release recorded, and touches git for
# nothing. MINSTREL_VERSION comes from GITHUB_REF, not from git.
fetch-depth: 1
# Full history, and rule 149 names this specifically: any job that
# DERIVES the version name needs it, because a shallow clone changes
# what git-derived values resolve to WITHOUT failing — a too-low
# value, silently, with every lane green.
#
# This job was depth-1 while it took the version from GITHUB_REF. It
# now runs ci/version.sh itself, because with :<version> image tags
# gone the server's self-reported version is the only thing that says
# which build an image is.
fetch-depth: 0
- name: Detect buildable project
id: guard
@@ -334,30 +720,68 @@ jobs:
if: steps.guard.outputs.ready == 'true'
shell: bash
run: |
set -euo pipefail
# THE VERSION, and it is derived the same way on every ref — the
# branch decides the CHANNEL, never the version (family rule 149).
#
# This used to be three different things: the literal string "main"
# on main, "dev" on dev, and the tag name on a tag. None of them
# ordered, and the first two were the same string forever — two dev
# images eight weeks apart were indistinguishable in the UI. That
# mattered little while :vYYYY.MM.DD.HHMM existed to identify a
# build; with version image tags gone, this IS how an operator tells
# which build a container is running.
#
# `sed -n s///p` rather than `grep`: it exits 0 when nothing matches,
# so the empty check below is actually reachable. A grep here would
# kill the step at the assignment under the runner's pipefail — the
# exact bug that took down the first main build after the version
# rework.
VERSION="$(ci/version.sh HEAD | sed -n 's/^name=//p')"
if [ -z "${VERSION}" ]; then
echo "::error::could not derive a build version from ci/version.sh"
exit 1
fi
if [[ "${GITHUB_REF}" == refs/tags/v* ]]; then
VERSION="${GITHUB_REF#refs/tags/}"
echo "args=-t ${IMAGE}:${VERSION} -t ${IMAGE}:latest" >> "$GITHUB_OUTPUT"
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
echo "::notice::Release build: ${VERSION} + latest"
# A release refreshes the CHANNEL and mints nothing else.
#
# The tag build exists to produce the signed APK and attach it to
# the release; the image it rebuilds is the SAME SOURCE as the main
# build minutes earlier, differing only in which APK is baked in.
# Rule 145 is explicit about that case: when the same source is
# rebuilt with different contents, publish the moving channel tag
# and never a commit-addressable one.
#
# :latest must move here rather than waiting for the next main
# push, or the channel would carry the PREVIOUS release's APK
# indefinitely — a channel that cannot refresh itself (rule 146).
CHANNEL=stable
echo "args=-t ${IMAGE}:latest" >> "$GITHUB_OUTPUT"
echo "::notice::Release build ${VERSION}: refreshing :latest around the new APK"
elif [[ "${GITHUB_REF}" == "refs/heads/dev" ]]; then
# The rolling test channel, and :dev ALONE — deliberately no
# per-commit tag. A rolling channel is rolling by definition, so a
# commit-addressable image here would be a rollback target nobody
# has ever pulled, accumulating in the registry forever. Recovery
# on dev is to fix forward.
CHANNEL=dev
echo "args=-t ${IMAGE}:dev" >> "$GITHUB_OUTPUT"
echo "version=dev" >> "$GITHUB_OUTPUT"
echo "::notice::Dev-branch build: :dev"
echo "::notice::Dev-branch build ${VERSION}: :dev"
else
# Main is the protected, post-PR-merge branch. Treat it as the
# rolling stable channel — every main push moves :latest.
# Pinned consumers can target :vYYYY.MM.DD.HHMM, which never
# moves; everyone else gets the newest main.
echo "args=-t ${IMAGE}:main -t ${IMAGE}:latest" >> "$GITHUB_OUTPUT"
echo "version=main" >> "$GITHUB_OUTPUT"
echo "::notice::Main-branch build: :main + :latest"
# The production line: :latest tracks main's tip (rule 147) and
# :<sha> is the rollback unit (rule 145). Full 40-char SHA, matching
# the family's other repos, so a rollback target is addressable
# straight from the commit anyone is reading.
CHANNEL=stable
echo "args=-t ${IMAGE}:latest -t ${IMAGE}:${GITHUB_SHA}" >> "$GITHUB_OUTPUT"
echo "::notice::Main-branch build ${VERSION}: :latest + :${GITHUB_SHA}"
fi
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
echo "channel=${CHANNEL}" >> "$GITHUB_OUTPUT"
- name: Registry login
if: steps.guard.outputs.ready == 'true'
shell: bash
@@ -372,20 +796,11 @@ jobs:
if: >-
steps.guard.outputs.ready == 'true' &&
(startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/dev')
# Consuming half of the pair — never actions/download-artifact. Same fork,
# same reason: upstream's client-side GHES check rejects this hostname
# before it connects. bvandeusen/download-artifact mirrors
# code.forgejo.org/forgejo/download-artifact.
#
# SHA below is that fork's `v6` tag. Match on @actions/artifact, NOT on
# the action's own version number — the two actions release on unrelated
# cadences, and download v5 would pair a ^2.3.2 client with this file's
# ^4.0.0 uploader. v6 is the tag whose bundled library major (^4.0.0) is
# the same one proven against this instance by the upload side.
# Deliberately NOT v7: it moves to node24 and upstream requires runner
# >= 2.327.1 for it, which act_runner does not claim to satisfy.
# Pinned, not tagged — the mirror auto-syncs every 8h.
uses: https://git.fabledsword.com/bvandeusen/download-artifact@8d4e9521a5f7e5f8b6351f341f719f9f45a92a3a
# Consuming half of the pair: stock download-artifact, which works here for
# the same reason as the upload (gitea/runner 3.x edits the GHES refusal
# out of the bundle; snippet #2271). v8 runs on node24, which every
# CI-runner image carries — the runner uses the image's own node.
uses: actions/download-artifact@v8
with:
name: minstrel-apk
path: client/
@@ -475,9 +890,15 @@ jobs:
- name: Build and push
if: steps.guard.outputs.ready == 'true'
# --pull: the Dockerfile's base images are floating tags (golang:1.26,
# debian:bookworm-slim). Without it the runner's daemon reuses
# whatever it cached, and the shipped binary can sit on a Go patch
# release govulncheck already flagged while the lane, which pulls
# fresh, reports clean.
run: |
docker buildx build \
docker buildx build --pull \
--build-arg MINSTREL_VERSION="${{ steps.tags.outputs.version }}" \
--build-arg MINSTREL_CHANNEL="${{ steps.tags.outputs.channel }}" \
--push ${{ steps.tags.outputs.args }} .
# Verifies a tag release actually ended up complete, and names the specific
@@ -488,8 +909,8 @@ jobs:
# `failure` with none executed and image-release showed `skipped`. The run was
# red, but the *release page rendered fine*, and `main`'s own push build had
# already moved `:latest`, so the code was deployable and nothing looked
# obviously wrong. The release was simply missing its APK and its immutable
# `:vYYYY.MM.DD` image, which is easy to skim past.
# obviously wrong. The release was simply missing its APK and its image,
# which is easy to skim past.
#
# This job cannot prevent that (the cause was a runner failing to launch, not
# anything in this file). What it does is turn an incomplete release into an
@@ -500,7 +921,7 @@ jobs:
# above did NOT succeed.
verify-release:
name: Verify release artifacts (tag releases only)
needs: [android-release, image-release]
needs: [android-release, release-assets, image-release]
if: ${{ always() && startsWith(github.ref, 'refs/tags/v') }}
runs-on: go-ci
container:
@@ -540,18 +961,30 @@ jobs:
# missing when v2026.08.07 had to be re-cut. `always()` on this job means
# it runs even when image-release failed, so without this the guard would
# cheerfully verify an incomplete release.
- name: Immutable image tag must exist
#
# This asserted `:${TAG}` — the :vYYYY.MM.DD.HHMM image — until
# 2026-09-10. Version image tags are no longer published (rule 145), so
# that assertion would now fail every release for a tag nothing mints.
# The rollback target it was really protecting is the :<sha> image, which
# main's own build published for this same commit before the tag was cut.
#
# Checking it here earns its keep twice over: it still catches an image
# push that silently did not happen, and it additionally proves the
# ORDERING — a tag cut on a commit whose main build never completed has
# no rollback target, and that is worth failing on rather than
# discovering during an incident.
- name: Rollback image must exist for the tagged commit
shell: bash
run: |
set -euo pipefail
TAG="${GITHUB_REF#refs/tags/}"
IMAGE="git.fabledsword.com/bvandeusen/minstrel"
echo "${{ secrets.CI_TOKEN }}" \
| docker login git.fabledsword.com -u "${{ github.actor }}" --password-stdin
if ! docker manifest inspect "${IMAGE}:${TAG}" > /dev/null 2>&1; then
echo "::error::image ${IMAGE}:${TAG} was never pushed — the release tag has no immutable image, so there is nothing to pin or roll back to. Re-run this workflow run."
if ! docker manifest inspect "${IMAGE}:${GITHUB_SHA}" > /dev/null 2>&1; then
echo "::error::image ${IMAGE}:${GITHUB_SHA} does not exist — this commit has no rollback target."
echo "::error::That image is published by the MAIN build of this commit, not by the tag build. If main's build never ran or failed, fix that first; a release whose commit cannot be rolled back to is the thing this check exists to refuse."
exit 1
fi
echo "::notice::image verified: ${IMAGE}:${TAG}"
echo "::notice::rollback target verified: ${IMAGE}:${GITHUB_SHA}"
-156
View File
@@ -1,156 +0,0 @@
name: test-go
# Go server: vet + golangci-lint + short race tests. Runs on push to
# dev/main and PRs to main, scoped to Go-side files only — web-only or
# Flutter-only diffs don't trigger this workflow.
#
# Two jobs: `test` (fast — vet + lint + `go test -short -race`, no DB) and
# `integration` (full `go test -race` against an ephemeral Postgres).
#
# Integration-job DB wiring follows the act_runner shared-daemon pattern:
# the runner's Docker daemon also runs the operator's dev compose stack,
# so service containers get NO published ports (collision) and no
# service-name DNS. We discover the service container by the job-scoped
# name filter via the mounted docker socket and reach it by bridge IP.
# The exactly-one assertion is a hard guard — pointing tests at the dev
# Postgres would truncate it (the disaster Fable #339 exists to prevent).
#
# `web/build/` has a committed placeholder index.html so go:embed succeeds
# without needing the SPA to be freshly built. Real builds happen in
# release.yml (container) and locally during dev.
on:
push:
branches: [dev, main]
paths:
- '**/*.go'
- 'go.mod'
- 'go.sum'
- 'sqlc.yaml'
- 'Makefile'
- 'internal/**'
- 'cmd/**'
- '.golangci.yml'
- '.gitea/workflows/test-go.yml'
# The release lane's own trigger is `main` + tags, so nothing it
# contains is exercised until a release is already running. These two
# entries are what let internal/server/release_version_test.go guard
# the version derivation on ordinary dev pushes instead.
- 'ci/**'
- '.gitea/workflows/release.yml'
# pull_request trigger intentionally omitted — see test-web.yml for
# the rationale (single-author repo, push covers PR-merge equivalent).
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
runs-on: go-ci
container:
image: git.fabledsword.com/bvandeusen/ci-go:1.26
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Toolchain versions
run: |
go version
golangci-lint --version
- name: Generated code matches queries (sqlc)
run: make verify-generate
- name: go vet
run: go vet ./...
- name: golangci-lint
run: golangci-lint run ./...
- name: go test (short, race)
run: go test -short -race ./...
integration:
runs-on: go-ci
container:
image: git.fabledsword.com/bvandeusen/ci-go:1.26
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: minstrel
POSTGRES_PASSWORD: minstrel
POSTGRES_DB: minstrel_test
# No `ports:` — the runner shares the operator's dev compose
# Docker daemon; publishing a fixed host port collides.
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Integration suite (discover service by bridge IP, migrate, test)
run: |
set -eux
# Discover THIS job's Postgres service container via the
# mounted docker socket. act_runner attaches the job
# container and its service container(s) to a shared per-job
# network, so scope discovery to a postgres that sits on a
# network THIS job container is also on. The old
# `--filter name=integration` matched EVERY concurrent
# integration run's postgres (a dev push + the main-merge run
# overlap → 2 candidates → false "expected exactly 1" abort).
# The operator's dev compose `minstrel-postgres-*` is never on
# this job's network; skip it explicitly as belt-and-suspenders
# (a wrong target would truncate real data).
SELF=$(cat /etc/hostname)
SELF_NETS=$(docker inspect -f '{{range $k,$v := .NetworkSettings.Networks}}{{$k}} {{end}}' "$SELF")
test -n "$SELF_NETS"
echo "self ($SELF) networks: $SELF_NETS"
PG_ID=""
PG_NAME=""
for cid in $(docker ps --filter "ancestor=postgres:16-alpine" -q); do
nm=$(docker inspect -f '{{.Name}}' "$cid" | sed 's#^/##')
case "$nm" in *minstrel-postgres*|*_postgres_*) continue ;; esac
for net in $(docker inspect -f '{{range $k,$v := .NetworkSettings.Networks}}{{$k}} {{end}}' "$cid"); do
case " $SELF_NETS " in *" $net "*) PG_ID="$cid"; PG_NAME="$nm"; break 2 ;; esac
done
done
test -n "$PG_ID" || { echo "FATAL: no postgres service container on this job's network (self nets: $SELF_NETS)"; exit 1; }
echo "selected postgres: $PG_ID $PG_NAME"
PG_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$PG_ID")
test -n "$PG_IP"
export MINSTREL_TEST_DATABASE_URL="postgres://minstrel:minstrel@${PG_IP}:5432/minstrel_test?sslmode=disable"
# Wait for Postgres to accept TCP (no health-check dependency).
for i in $(seq 1 60); do (echo > "/dev/tcp/${PG_IP}/5432") 2>/dev/null && break; sleep 2; done
# Relax durability on the throwaway CI Postgres. Our test pattern
# is dbtest.ResetDB → TRUNCATE … RESTART IDENTITY CASCADE before
# every test, and the per-TRUNCATE commit fsync is the dominant
# cost of the integration suite. The CI DB is rebuilt every run so
# fsync / full_page_writes / synchronous_commit buy nothing. Apply
# via docker exec because:
# - The act_runner `services:` block can't override the container
# command, so `postgres -c fsync=off` at boot isn't an option.
# - ALTER SYSTEM cannot run inside a transaction; psql -c
# auto-commits each statement, which is what we need.
# - fsync / full_page_writes are sighup GUCs and
# synchronous_commit is user-context, so pg_reload_conf() picks
# all three up with no restart.
# Non-fatal: a perms surprise degrades to "slower", never red CI.
docker exec "$PG_ID" psql -U minstrel -d minstrel_test \
-c "ALTER SYSTEM SET fsync = off" \
-c "ALTER SYSTEM SET synchronous_commit = off" \
-c "ALTER SYSTEM SET full_page_writes = off" \
-c "SELECT pg_reload_conf()" \
|| echo "WARN: durability relax failed; continuing"
# Apply embedded migrations to the fresh test DB, then run the
# full suite (no -short → integration tests execute). -p 1:
# every integration package TRUNCATEs the one shared test DB;
# concurrent package binaries → TRUNCATE deadlocks. Serialize
# package execution (the documented local invocation too).
MINSTREL_DATABASE_URL="$MINSTREL_TEST_DATABASE_URL" go run ./cmd/minstrel migrate
go test -p 1 -race ./...
-44
View File
@@ -1,44 +0,0 @@
name: test-web
# Web SPA: vitest + svelte-check. Runs on push to dev/main only —
# the `pull_request` trigger is intentionally omitted because every
# branch on this repo is local-only (no fork PRs), so the dev push
# fully covers what a PR run would re-execute. Keeping both events
# doubled CI cost on every commit.
on:
push:
branches: [dev, main]
paths:
- 'web/**'
- '.gitea/workflows/test-web.yml'
# Cancel an earlier in-flight run for the same ref when a newer
# commit arrives. With cancel-in-progress, rapid re-pushes don't
# pile up zombie runs.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
runs-on: go-ci
container:
image: git.fabledsword.com/bvandeusen/ci-go:1.26
defaults:
run:
working-directory: web
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Install deps
run: npm ci
- name: Type-check + svelte-check
run: npm run check
- name: Vitest
run: npm test
+5
View File
@@ -12,6 +12,11 @@
# Test binary, built with `go test -c`
*.test
# `make build` output. bin/minstrel was tracked until 2026-09-10 — an 18 MB
# binary committed by accident, last refreshed by a commit about web test
# mocks, and re-dirtied by every local build since.
bin/
# Bundled Android APK + version sidecar (#397). Populated by CI for
# tag releases; never committed. README in client/ explains the flow.
client/minstrel.apk
+21 -6
View File
@@ -7,7 +7,7 @@ RUN npm ci
COPY web/ ./
RUN npm run build
FROM golang:1.25-bookworm AS builder
FROM golang:1.26-bookworm AS builder
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
@@ -15,17 +15,32 @@ COPY . .
# Overwrite the committed placeholder with the freshly-built SPA assets.
COPY --from=web /web/build ./web/build
ENV CGO_ENABLED=0
# Version stamping: release.yml passes the git tag via MINSTREL_VERSION
# build-arg; local `docker build` falls back to "dev". Surfaced at
# /healthz for operator-side image-version verification.
# Version stamping. release.yml passes the DERIVED version name
# (YYYY.MM.DD.HHMM) and the lane's channel; a local `docker build` falls back
# to "dev"/"local". Both are surfaced at /healthz.
#
# These are two values on purpose (family rule 149): the same commit built on
# dev and on main reports the same NAME and differs only in CHANNEL. Folding
# the channel into the version string is what the rule forbids — the version
# used to BE the channel word here ("main"/"dev"), which meant two dev images
# eight weeks apart were indistinguishable.
ARG MINSTREL_VERSION=dev
ARG MINSTREL_CHANNEL=local
RUN go build -trimpath \
-ldflags="-s -w -X 'git.fabledsword.com/bvandeusen/minstrel/internal/server.ServerVersion=${MINSTREL_VERSION}'" \
-ldflags="-s -w \
-X 'git.fabledsword.com/bvandeusen/minstrel/internal/server.ServerVersion=${MINSTREL_VERSION}' \
-X 'git.fabledsword.com/bvandeusen/minstrel/internal/server.ServerChannel=${MINSTREL_CHANNEL}'" \
-o /out/minstrel ./cmd/minstrel
FROM debian:bookworm-slim
# ffmpeg: duration probes and the exact-tier audio hash (a SHA-256 of the
# encoded audio packets, so no decode). libchromaprint-tools: fpcalc, the
# acoustic fingerprint that tells the same recording at two bitrates apart
# from two different recordings (M400). Both are baked in at build time so a
# deployed instance never fetches either (rule 164); fpcalc is shelled out
# rather than bound because CGO_ENABLED=0 above rules out cgo.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates ffmpeg \
&& apt-get install -y --no-install-recommends ca-certificates ffmpeg libchromaprint-tools \
&& rm -rf /var/lib/apt/lists/*
RUN groupadd --system --gid 1000 minstrel \
+26 -13
View File
@@ -34,11 +34,18 @@ Minstrel is not affiliated with or endorsed by Lidarr, ListenBrainz, MusicBrainz
services:
minstrel:
image: git.fabledsword.com/bvandeusen/minstrel:latest
# Reachable from your LAN at http://<host>:4533. If this host faces the
# internet, bind it to 127.0.0.1 and put an HTTPS proxy in front instead:
# see docs/hosting.md.
ports: ['4533:4533']
volumes:
# Your music library. Point ./music at wherever your audio files
# live. Mounted read-only — Minstrel never writes to your library.
- ./music:/music:ro
# live. Writable, because Minstrel deletes a file when an admin asks
# it to (for example, quarantine's "Delete file"). It never moves,
# renames or retags anything. The container runs as uid 1000, so that
# user needs write access to the folders. Mount it :ro to forbid even
# deletes: those actions then refuse, say why, and delete nothing.
- ./music:/music
# Generated data: playlist cover collages, artist art, caches.
# The path must match MINSTREL_STORAGE_DATA_DIR, which the image
# sets to /app/data — keep this mount on /app/data or your cache
@@ -47,7 +54,7 @@ services:
environment:
MINSTREL_DATABASE_URL: postgres://minstrel:minstrel@db:5432/minstrel?sslmode=disable
# Colon-separated library roots to scan; must match the container
# path of the read-only music mount above (/music here).
# path of the music mount above (/music here).
MINSTREL_LIBRARY_SCAN_PATHS: /music
depends_on: [db]
@@ -72,9 +79,9 @@ docker compose up -d
## First run
With the stack up, a handful of in-app steps get you to a working library. Use your own host in place of `localhost` if you're reaching the server over a LAN/VPN address (plain `http://` is fine — no TLS required).
With the stack up, a handful of in-app steps get you to a working library. Use your own host in place of `localhost` if you're reaching the server over a LAN/VPN address. Plain `http://` is fine on a network you trust; a server reachable from the internet belongs behind HTTPS, which [docs/hosting.md](docs/hosting.md) walks through.
**1. Create your admin account.** Visit `http://localhost:4533/register`. The first account on a fresh instance is automatically the administrator; later users join through the same form or an invite token (step 5).
**1. Create your admin account.** Visit `http://localhost:4533/register`. The first account on a fresh instance becomes the administrator, and creating it asks for the **setup token** the server prints in its log (`docker compose logs minstrel | grep setup_token`), so nobody else can claim a newly exposed server first. Later users join through the same form or an invite token (step 5).
<a href="docs/screenshots/register.png"><img src="docs/screenshots/register.png" width="320" alt="Creating the first (admin) account on a fresh instance"></a>
@@ -96,6 +103,8 @@ With the stack up, a handful of in-app steps get you to a working library. Use y
For the full configuration surface, see [`config.example.yaml`](./config.example.yaml).
Hosting Minstrel on the internet: see [docs/hosting.md](docs/hosting.md). What Minstrel does to protect accounts, and why: [docs/security.md](docs/security.md).
## Configuration
Most operators only need the env vars in the quickstart above. A few extras worth knowing:
@@ -112,18 +121,21 @@ Most operational keys have a `MINSTREL_<SECTION>_<FIELD>` env override. Recommen
Image tags (`git.fabledsword.com/bvandeusen/minstrel:<tag>`):
- `:latest` — the newest blessed image. Moves on every `main` push **and** every release. Recommended for most operators.
- `:vYYYY.MM.DD.HHMM` — immutable release tags, never moved or deleted. Pin one for a deployment you don't want changing under you. The tag is the build's own version name with a `v` in front, derived from the tagged commit's UTC timestamp, so two releases can never collide and a re-cut is simply a new tag.
- `:main` — the rolling post-merge tip. Same image as `:latest` at push time; choose it if you want to track `main` explicitly rather than the release line.
- `:dev` — the rolling test channel, rebuilt on every push to `dev` and carrying its own freshly-built Android APK. Run this when you want to try something before it ships. It moves constantly, has no per-commit tag to pin, and its only recovery path is forward — if a `:dev` image is broken, the fix is the next push, not a rollback.
- `:latest` — production. Tracks `main`'s tip and moves on every `main` push and every release. What most operators should run.
- `:<commit-sha>` — the rollback unit. Every `main` push publishes one, so any production commit is addressable without a release ceremony. Immutable: a given SHA tag is never re-pushed. Pin one if you need a deployment that cannot change under you, and use it to roll back.
- `:dev` — the rolling test channel, rebuilt on every push to `dev` and carrying its own freshly-built Android APK. Run this to try something before it ships. It moves constantly, has no per-commit tag, and its only recovery path is forward — if a `:dev` image is broken, the fix is the next push, not a rollback.
Every `:latest`, `:vYYYY.MM.DD.HHMM` and `:dev` bundles a signed Android APK, so the in-app update channel is always live. All of them are signed with the same key, so a phone can move between the stable and dev channels without uninstalling — point it at a `:dev` server and the in-app updater offers that channel's build.
That is the whole tag map. **There are no version-numbered image tags**, and no `:main`. Git and the build's own self-reported version answer "which build is this" — the Settings page shows it, and so does `/healthz`. Release *tags* in git are still `vYYYY.MM.DD.HHMM`; they name a changelog entry and the APK attached to it, not an image.
Rolling back to `:<commit-sha>` pins the **server code** at that commit — not the server-and-app pair. The Android APK is baked in at image build time, so a SHA image carries whichever app was current when that commit was built, which may be older than what `:latest` bundles now. If both halves matter, check what the image bundles rather than trusting the tag's name.
Every `:latest`, `:<commit-sha>` and `:dev` bundles a signed Android APK, so the in-app update channel is always live. All are signed with the same key, so a phone can move between the stable and dev channels without uninstalling — point it at a `:dev` server and the in-app updater offers that channel's build.
The app reports which channel it is on alongside its version, and decides whether an update is available using the build's ordering key rather than its displayed name — the same value Android installs by, so an offer it makes is one the platform will accept.
Database migrations run automatically at startup; rollbacks require restoring a Postgres dump.
Releases before 2026-09-10 use the older per-day `:vYYYY.MM.DD` shape. Those tags still exist and still work — they are simply not extended.
Releases up to 2026-09-10 also published a `:vYYYY.MM.DD[.HHMM]` image tag. Those images still exist and still work — they are simply not extended.
## Specs
@@ -147,7 +159,8 @@ Two concurrent dev processes:
truncates your dev `minstrel` data (admin user, library, likes). It
brings up the compose Postgres and creates the test DB if missing.
- CI runs both: a fast `go test -short -race` gate plus an integration
job with its own ephemeral Postgres (`.gitea/workflows/test-go.yml`).
job with its own ephemeral Postgres (the `integration` lane in
`.gitea/workflows/release.yml`, which also gates every image publish).
### Production build
@@ -157,7 +170,7 @@ Two concurrent dev processes:
- Day-to-day work happens on `dev` (or feature branches merged into `dev`).
- `main` is **protected** — changes land via PR from `dev`.
- Releases are cut by tagging `v*` off `main`; the release workflow builds and pushes the container image to the Gitea registry.
- Releases are cut by tagging `v*` off `main`; the release workflow builds the signed APK, attaches it to the release, and refreshes `:latest` around it.
Task and milestone tracking: Fable (`Minstrel` project, id 12).
+5 -1
View File
@@ -72,9 +72,13 @@ android {
getDefaultProguardFile("proguard-android-optimize.txt"),
"proguard-rules.pro",
)
// Signed with the release key or not at all. Falling back to the
// debug key made a missing secret into a published APK that no
// install could ever update (family idea #5103, practice 2). An
// unsigned build installs nowhere, so the gap shows at once.
signingConfig =
if (System.getenv("ANDROID_KEYSTORE_PATH").isNullOrEmpty()) {
signingConfigs.getByName("debug")
null
} else {
signingConfigs.getByName("release")
}
@@ -15,10 +15,14 @@ import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.Scaffold
import androidx.compose.material3.SnackbarHost
import androidx.compose.material3.SnackbarHostState
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.remember
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.style.TextOverflow
@@ -42,6 +46,12 @@ fun AdminQuarantineScreen(
viewModel: AdminQuarantineViewModel = hiltViewModel(),
) {
val state by viewModel.uiState.collectAsStateWithLifecycle()
val snackbarHostState = remember { SnackbarHostState() }
LaunchedEffect(Unit) {
viewModel.transientMessages.collect { msg ->
snackbarHostState.showSnackbar(msg)
}
}
Scaffold(
contentWindowInsets = ShellContentWindowInsets,
modifier = Modifier.fillMaxSize(),
@@ -53,6 +63,7 @@ fun AdminQuarantineScreen(
onBack = { navController.popBackStack() },
)
},
snackbarHost = { SnackbarHost(snackbarHostState) },
) { inner ->
PullToRefreshScaffold(
onRefresh = { viewModel.refresh().join() },
@@ -10,10 +10,13 @@ import com.fabledsword.minstrel.events.EventsStream
import com.fabledsword.minstrel.models.AdminQuarantineItemRef
import dagger.hilt.android.lifecycle.HiltViewModel
import kotlinx.coroutines.Job
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.flow.filter
import kotlinx.coroutines.flow.receiveAsFlow
import kotlinx.coroutines.launch
import javax.inject.Inject
@@ -34,6 +37,15 @@ class AdminQuarantineViewModel @Inject constructor(
private val internal = MutableStateFlow<AdminQuarantineUiState>(AdminQuarantineUiState.Loading)
val uiState: StateFlow<AdminQuarantineUiState> = internal.asStateFlow()
/**
* One-shot messages for the screen's snackbar. A failed action has to say
* why: the row quietly reappearing reads as a glitch, and for a Delete
* file refused by a read-only library it hides the one thing the
* operator can fix (#3918).
*/
private val transientMessagesChannel = Channel<String>(Channel.BUFFERED)
val transientMessages: Flow<String> = transientMessagesChannel.receiveAsFlow()
init {
refresh()
viewModelScope.launch {
@@ -86,8 +98,9 @@ class AdminQuarantineViewModel @Inject constructor(
try {
action(trackId)
} catch (
@Suppress("TooGenericExceptionCaught", "SwallowedException") e: Throwable,
@Suppress("TooGenericExceptionCaught") e: Throwable,
) {
transientMessagesChannel.trySend(ErrorCopy.fromThrowable(e))
refresh()
}
}
@@ -50,7 +50,14 @@ class BaseUrlInterceptor @Inject constructor(
.port(baseUrl.port)
.build()
} ?: original.url
return chain.proceed(original.newBuilder().url(rewritten).build())
return chain.proceed(
original.newBuilder()
.url(rewritten)
// Lets CleartextGuardInterceptor tell server requests from
// external fetches once the placeholder host is gone.
.tag(MinstrelServerRequest::class.java, MinstrelServerRequest)
.build(),
)
}
companion object {
@@ -0,0 +1,84 @@
package com.fabledsword.minstrel.api
import okhttp3.Interceptor
import okhttp3.Response
import java.io.IOException
import java.net.Inet4Address
import java.net.Inet6Address
import java.net.InetAddress
/**
* Marks a request as bound for the Minstrel server, set by
* [BaseUrlInterceptor] when it retargets the placeholder host. Those are the
* requests that carry the session cookie and the password.
*/
object MinstrelServerRequest
/**
* Plain `http://` to the Minstrel server is allowed only when the connection
* actually lands on a private address (family security baseline #5105,
* practice 13).
*
* Cleartext stays permitted app-wide for LAN servers and UPnP
* (network_security_config.xml, #2439), but a password or session cookie sent
* over plain HTTP to a public address can be read by anyone on the path.
*
* Checked per connection, on the address the socket really reached, not on
* the URL when it was typed: a name that resolved to the home network when it
* was entered resolves to a public address once the phone leaves home, and
* that is exactly when the password would go out in the clear. A network
* interceptor runs after the connection is made and before any request byte
* is written, so nothing is sent.
*/
class CleartextGuardInterceptor(
// The policy is a parameter so a test can refuse loopback, the only
// address a test server can listen on.
private val allows: (InetAddress) -> Boolean = CleartextPolicy::allows,
) : Interceptor {
override fun intercept(chain: Interceptor.Chain): Response {
val request = chain.request()
if (request.isHttps || request.tag(MinstrelServerRequest::class.java) == null) {
return chain.proceed(request)
}
val address = chain.connection()?.route()?.socketAddress?.address
if (address != null && !allows(address)) {
throw CleartextToPublicHostException(request.url.host)
}
return chain.proceed(request)
}
}
/** The server was reached over plain HTTP at a public address, and refused. */
class CleartextToPublicHostException(host: String) :
IOException("refusing plain http:// to $host: it is a public address")
/** Which addresses plain HTTP may reach: the home network, never the internet. */
object CleartextPolicy {
private const val CGNAT_FIRST_OCTET = 100
private const val CGNAT_SECOND_MASK = 0xC0
private const val CGNAT_SECOND_PREFIX = 64
private const val ULA_MASK = 0xFE
private const val ULA_PREFIX = 0xFC
private const val BYTE = 0xFF
fun allows(address: InetAddress): Boolean =
address.isLoopbackAddress ||
address.isSiteLocalAddress || // 10/8, 172.16/12, 192.168/16
address.isLinkLocalAddress || // 169.254/16, fe80::/10
address.isAnyLocalAddress ||
isCarrierGradeNat(address) ||
isUniqueLocal(address)
// 100.64/10. Tailscale and other overlay VPNs hand these out; the overlay
// encrypts the traffic itself.
private fun isCarrierGradeNat(address: InetAddress): Boolean {
if (address !is Inet4Address) return false
val b = address.address
return (b[0].toInt() and BYTE) == CGNAT_FIRST_OCTET &&
(b[1].toInt() and CGNAT_SECOND_MASK) == CGNAT_SECOND_PREFIX
}
// fc00::/7, IPv6's private range.
private fun isUniqueLocal(address: InetAddress): Boolean =
address is Inet6Address && (address.address[0].toInt() and ULA_MASK) == ULA_PREFIX
}
@@ -37,18 +37,36 @@ object ErrorCopy {
* as connection failures.
*/
fun fromThrowable(t: Throwable): String = when (t) {
is HttpException -> messageFor(codeFromHttp(t))
is HttpException -> fromHttp(t)
is CleartextToPublicHostException -> messageFor("cleartext_public")
is IOException -> messageFor("connection_refused")
else -> TABLE.getValue("unknown")
}
private fun codeFromHttp(e: HttpException): String {
/**
* Codes whose server message carries specifics the operator needs in
* order to act — which directory, which uid — that fixed copy cannot say.
* For these the message follows the copy (#3918). Kept to a named set on
* purpose: most server messages are internal detail. Mirrors web's
* errors.ts.
*/
private val DETAIL_CODES = setOf("library_not_writable", "file_delete_failed")
private fun fromHttp(e: HttpException): String {
val body = bodyFromHttp(e)
val copy = messageFor(body.code.ifEmpty { "unknown" })
return if (body.code in DETAIL_CODES && body.message.isNotBlank()) {
"$copy ${body.message}"
} else {
copy
}
}
private fun bodyFromHttp(e: HttpException): Body {
val raw = runCatching { e.response()?.errorBody()?.string() }.getOrNull()
?: return "unknown"
val code = runCatching { json.decodeFromString<Envelope>(raw).error?.code }
.getOrNull()
.orEmpty()
return code.ifEmpty { "unknown" }
?: return Body()
return runCatching { json.decodeFromString<Envelope>(raw).error }
.getOrNull() ?: Body()
}
private val TABLE: Map<String, String> = mapOf(
@@ -58,6 +76,7 @@ object ErrorCopy {
"forbidden" to "You don't have permission to do that.",
"not_authorized" to "You don't have permission to do that.",
"invalid_credentials" to "Wrong username or password.",
"rate_limited" to "Too many attempts. Wait a few minutes and try again.",
"wrong_password" to "Current password is incorrect.",
"password_too_short" to "Password must be at least 8 characters.",
"username_invalid" to "That username isn't valid.",
@@ -83,6 +102,9 @@ object ErrorCopy {
"mbid_required" to "An MBID is required for this lookup.",
"system_playlist_readonly" to "System playlists can't be edited directly.",
"connection_refused" to "Couldn't reach the server. Check the URL and try again.",
"cleartext_public" to
"This server is on the internet, so its URL must start with https://. " +
"Plain http:// only works on your home network.",
"lidarr_unreachable" to
"Lidarr is unreachable right now. Try again, or check Admin → Integrations.",
"lidarr_disabled" to "Lidarr integration is not enabled.",
@@ -99,6 +121,8 @@ object ErrorCopy {
"request_not_pending" to "This request is no longer pending.",
"request_not_found" to "That request no longer exists.",
"track_not_found" to "That track no longer exists.",
"library_not_writable" to "The music library isn't writable by the server.",
"file_delete_failed" to "The file couldn't be deleted.",
"album_not_found" to "That album no longer exists.",
"artist_not_found" to "That artist no longer exists.",
"playlist_not_found" to "That playlist no longer exists.",
@@ -70,6 +70,9 @@ object NetworkModule {
.addInterceptor(auth)
.addInterceptor(baseUrl)
.addInterceptor(logging)
// A network interceptor, so it sees the address the connection
// really reached and runs before any request byte is written.
.addNetworkInterceptor(CleartextGuardInterceptor())
.connectTimeout(CONNECT_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.readTimeout(READ_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.build()
@@ -29,11 +29,20 @@ interface CastApi {
* Request body. [expSeconds] is clamped server-side to [60, 86400];
* the 21_600 default (6h) is long enough to play through any typical
* track without re-minting mid-playback.
*
* [level] asks for the leveled stream (M464 #5001): the track rendered at
* the user's loudness gain, which the server works out from their setting.
* [asAlbum] says the track plays among its album in order, which picks
* album gain in auto mode. [prerender] says the speaker will fetch it
* soon, so the server renders it ahead.
*/
@Serializable
data class StreamTokenRequest(
val trackId: String,
val expSeconds: Int = 21_600,
val level: Boolean = false,
val asAlbum: Boolean = false,
val prerender: Boolean = false,
)
/**
@@ -53,4 +62,6 @@ data class StreamTokenResponse(
val url: String,
val mime: String = "audio/mpeg",
val title: String = "",
/** [url] is the leveled stream; false when leveling is off or changes nothing. */
val leveled: Boolean = false,
)
@@ -3,6 +3,7 @@ package com.fabledsword.minstrel.api.endpoints
import com.fabledsword.minstrel.models.wire.ListenBrainzStatusWire
import com.fabledsword.minstrel.models.wire.MyProfileWire
import com.fabledsword.minstrel.models.wire.SystemPlaylistsStatusWire
import com.fabledsword.minstrel.settings.data.NormalizationPrefs
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import retrofit2.http.Body
@@ -59,6 +60,14 @@ interface MeApi {
*/
@PUT("api/me/listenbrainz")
suspend fun setListenBrainz(@Body body: ListenBrainzPutBody): ListenBrainzStatusWire
/** The caller's loudness-normalization preference, or the defaults if never set. */
@GET("api/me/normalization")
suspend fun getNormalization(): NormalizationPrefs
/** Replaces the whole preference; returns what the server stored. */
@PUT("api/me/normalization")
suspend fun putNormalization(@Body body: NormalizationPrefs): NormalizationPrefs
}
/**
@@ -0,0 +1,15 @@
package com.fabledsword.minstrel.api.endpoints
import com.fabledsword.minstrel.models.wire.ReplayGainResponseWire
import retrofit2.http.GET
import retrofit2.http.Query
/** The player's loudness lookup (#4997), kept apart from the browse surface in [LibraryApi]. */
interface ReplayGainApi {
/**
* ReplayGain values for up to 200 comma-separated track ids. An id
* missing from `items` has not been measured yet.
*/
@GET("api/tracks/replay-gain")
suspend fun getReplayGain(@Query("ids") ids: String): ReplayGainResponseWire
}
@@ -4,12 +4,19 @@ import com.fabledsword.minstrel.cache.audiocache.CacheSettings
import com.fabledsword.minstrel.cache.db.dao.AuthSessionDao
import com.fabledsword.minstrel.cache.db.entities.AuthSessionEntity
import com.fabledsword.minstrel.di.ApplicationScope
import com.fabledsword.minstrel.settings.data.NormalizationPrefs
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Deferred
import kotlinx.coroutines.async
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withTimeoutOrNull
import kotlinx.serialization.json.Json
import timber.log.Timber
import javax.inject.Inject
import javax.inject.Singleton
@@ -27,6 +34,14 @@ import javax.inject.Singleton
* in-memory state changes synchronously so the next interceptor read
* sees the new value immediately; the DAO write coroutine catches up
* shortly after.
*
* **The session cookie is the exception** (M462 #4985): it is persisted
* through [SessionVault] (Keystore-encrypted), not the Room row. On the
* first launch after the upgrade, a cookie still in the row is moved into
* the vault and the column cleared, so nobody is signed out by the change.
* If the Keystore cannot be used on a device, the cookie stays in the row
* as before rather than being lost. [awaitSessionHydrated] lets a caller
* that needs a definitive answer (the auth gate) wait for this.
*/
// AuthStore is the single-row facade over auth_session (de-facto
// app_preferences — see entity comment). It legitimately owns one
@@ -39,6 +54,7 @@ import javax.inject.Singleton
@Singleton
class AuthStore @Inject constructor(
private val dao: AuthSessionDao,
private val vault: SessionVault,
@ApplicationScope private val scope: CoroutineScope,
) {
private val sessionCookieState = MutableStateFlow<String?>(null)
@@ -62,18 +78,32 @@ class AuthStore @Inject constructor(
private val diagnosticsOptOutState = MutableStateFlow(false)
val diagnosticsOptOut: StateFlow<Boolean> = diagnosticsOptOutState.asStateFlow()
private val normalizationState = MutableStateFlow(NormalizationPrefs.DEFAULT)
val normalization: StateFlow<NormalizationPrefs> = normalizationState.asStateFlow()
private val json = Json { ignoreUnknownKeys = true }
// Serialises every cookie persist with the one-time hydration, so a
// sign-in or a 401 that lands while hydration runs is never overwritten
// by the stale value hydration read.
private val cookieLock = Mutex()
// Set by setSessionCookie. Once something has written the cookie this
// process, that value wins over whatever hydration finds on disk.
@Volatile private var cookieTouched = false
private val cookieHydration: Deferred<Unit> = scope.async { hydrateSessionCookie() }
init {
scope.launch {
dao.observe().collect { row ->
sessionCookieState.value = row?.sessionCookie
baseUrlState.value = row?.baseUrl ?: DEFAULT_BASE_URL
userJsonState.value = row?.userJson
themeModeState.value = row?.themeMode
clientIdState.value = row?.clientId
cacheSettingsState.value = decodeCacheSettings(row?.cacheSettingsJson)
diagnosticsOptOutState.value = row?.diagnosticsOptOut ?: false
normalizationState.value = decodeNormalization(row?.normalizationJson)
}
}
}
@@ -85,9 +115,57 @@ class AuthStore @Inject constructor(
}.getOrDefault(CacheSettings.DEFAULT)
}
private fun decodeNormalization(raw: String?): NormalizationPrefs {
if (raw.isNullOrEmpty()) return NormalizationPrefs.DEFAULT
return runCatching {
json.decodeFromString(NormalizationPrefs.serializer(), raw)
}.getOrDefault(NormalizationPrefs.DEFAULT)
}
/**
* Suspends until the stored session cookie has been loaded into
* [sessionCookie], or [HYDRATION_DEADLINE_MS] passes (rule 156: a wedged
* Keystore must not leave the start screen spinning). Returns false on
* the deadline; the caller then decides from whatever has loaded, and a
* late hydration still lands in [sessionCookie].
*/
suspend fun awaitSessionHydrated(): Boolean {
val done = withTimeoutOrNull(HYDRATION_DEADLINE_MS) { cookieHydration.await() } != null
if (!done) Timber.w("auth store: session hydration passed its deadline; deciding without it")
return done
}
fun setSessionCookie(value: String?) {
cookieTouched = true
sessionCookieState.value = value
scope.launch { persistCookie(value) }
scope.launch { cookieLock.withLock { storeCookie(value) } }
}
private suspend fun hydrateSessionCookie() = cookieLock.withLock {
val legacy = runCatching { dao.get()?.sessionCookie }.getOrNull()
if (cookieTouched) return@withLock
// A cookie in the row is the newer one when both exist: the row is
// only written when the vault failed, and an install upgrading from
// before the vault has nothing in the vault yet.
val cookie = legacy ?: vault.read()
// Best-effort: if moving it fails, the session still loads this time
// and the move is retried on the next launch. Hydration must never
// throw, or awaitSessionHydrated would leave the auth gate stuck.
if (legacy != null) {
runCatching { storeCookie(legacy) }
.onFailure { Timber.w(it, "auth store: could not move the session cookie into the vault") }
}
sessionCookieState.value = cookie
}
// Vault first; the Room row only when the Keystore is unusable, so a
// broken Keystore degrades to the old storage rather than a sign-out.
private suspend fun storeCookie(value: String?) {
if (vault.write(value)) {
if (dao.get()?.sessionCookie != null) dao.setSessionCookie(null)
} else {
persistLegacyCookie(value)
}
}
fun setBaseUrl(value: String) {
@@ -121,7 +199,13 @@ class AuthStore @Inject constructor(
scope.launch { persistDiagnosticsOptOut(value) }
}
private suspend fun persistCookie(value: String?) {
fun setNormalization(value: NormalizationPrefs) {
normalizationState.value = value
val encoded = json.encodeToString(NormalizationPrefs.serializer(), value)
scope.launch { persistNormalization(encoded) }
}
private suspend fun persistLegacyCookie(value: String?) {
if (dao.get() == null) {
dao.upsert(currentEntity().copy(sessionCookie = value))
} else {
@@ -177,9 +261,19 @@ class AuthStore @Inject constructor(
}
}
private suspend fun persistNormalization(json: String) {
if (dao.get() == null) {
dao.upsert(currentEntity().copy(normalizationJson = json))
} else {
dao.setNormalizationJson(json)
}
}
private fun currentEntity(): AuthSessionEntity = AuthSessionEntity(
id = ROW_ID,
sessionCookie = sessionCookieState.value,
// Never copied into the row: the cookie lives in the vault, and
// persistLegacyCookie sets it explicitly on the fallback path.
sessionCookie = null,
baseUrl = baseUrlState.value,
userJson = userJsonState.value,
themeMode = themeModeState.value,
@@ -189,10 +283,18 @@ class AuthStore @Inject constructor(
cacheSettingsState.value,
),
diagnosticsOptOut = diagnosticsOptOutState.value,
normalizationJson = json.encodeToString(
NormalizationPrefs.serializer(),
normalizationState.value,
),
)
companion object {
const val DEFAULT_BASE_URL: String = "http://localhost:8080"
// Generous on purpose: hydration is one local row read and one
// Keystore decrypt, normally milliseconds. This only bounds "never".
const val HYDRATION_DEADLINE_MS: Long = 10_000
private const val ROW_ID = 0
}
}
@@ -0,0 +1,147 @@
package com.fabledsword.minstrel.auth
import android.content.Context
import android.security.keystore.KeyGenParameterSpec
import android.security.keystore.KeyProperties
import dagger.Binds
import dagger.Module
import dagger.hilt.InstallIn
import dagger.hilt.android.qualifiers.ApplicationContext
import dagger.hilt.components.SingletonComponent
import timber.log.Timber
import java.security.KeyStore
import java.util.Base64
import javax.crypto.Cipher
import javax.crypto.KeyGenerator
import javax.crypto.SecretKey
import javax.crypto.spec.GCMParameterSpec
import javax.inject.Inject
import javax.inject.Singleton
/**
* Where the session cookie lives at rest (M462 #4985).
*
* The cookie is a bearer credential: anyone holding it is signed in as the
* user until the server expires or revokes it. It used to sit in plain text
* in the Room `auth_session` row, readable from any copy of the app's data
* directory (a rooted device, an adb backup of a debuggable build, a
* forensic image). Now only ciphertext is stored, under an AES key that
* lives in the Android Keystore and never leaves it, so a copy of the
* files alone yields nothing usable.
*/
interface SessionVault {
/** The stored cookie, or null when none is stored or it can't be decrypted. */
fun read(): String?
/**
* Stores [value], or clears the stored cookie when null. Returns false
* when the Keystore could not be used, so the caller can fall back
* rather than lose the session.
*/
fun write(value: String?): Boolean
}
/**
* AES-GCM sealing of a short string, framed as base64(iv || ciphertext+tag).
* Kept apart from the Keystore so the framing can be unit-tested on the JVM
* with an ordinary key; the Android Keystore has no JVM implementation.
*/
internal object SealedBox {
private const val TRANSFORMATION = "AES/GCM/NoPadding"
private const val TAG_BITS = 128
private const val IV_BYTES = 12
// Binds a sealed value to its purpose: a blob sealed for something else
// under the same key will not open as a session cookie.
private val AAD = "minstrel-session-cookie-v1".toByteArray(Charsets.UTF_8)
fun seal(key: SecretKey, plaintext: String): String {
val cipher = Cipher.getInstance(TRANSFORMATION)
// No IV passed: the provider generates a fresh random one. Keystore
// keys refuse a caller-chosen IV for encryption by default.
cipher.init(Cipher.ENCRYPT_MODE, key)
cipher.updateAAD(AAD)
val sealed = cipher.iv + cipher.doFinal(plaintext.toByteArray(Charsets.UTF_8))
return Base64.getEncoder().encodeToString(sealed)
}
fun open(key: SecretKey, sealed: String): String {
val bytes = Base64.getDecoder().decode(sealed)
require(bytes.size > IV_BYTES) { "sealed value too short" }
val cipher = Cipher.getInstance(TRANSFORMATION)
cipher.init(Cipher.DECRYPT_MODE, key, GCMParameterSpec(TAG_BITS, bytes, 0, IV_BYTES))
cipher.updateAAD(AAD)
return String(cipher.doFinal(bytes, IV_BYTES, bytes.size - IV_BYTES), Charsets.UTF_8)
}
}
/**
* [SessionVault] backed by a Keystore AES key and a private prefs file that
* holds only the sealed value.
*
* A value that will not open (the key was wiped by a factory-reset of the
* Keystore, or the file was restored onto another device; app backup is off,
* but a vendor transfer tool may still copy files) is discarded and reported
* as absent. The user signs in again, which is the right outcome for a
* credential that no longer verifies.
*/
@Singleton
class KeystoreSessionVault @Inject constructor(
@ApplicationContext context: Context,
) : SessionVault {
private val prefs = context.getSharedPreferences(PREFS_NAME, Context.MODE_PRIVATE)
override fun read(): String? {
val sealed = prefs.getString(KEY_COOKIE, null) ?: return null
return runCatching { SealedBox.open(key(), sealed) }
.onFailure {
Timber.w(it, "session vault: stored cookie would not decrypt; discarding it")
prefs.edit().remove(KEY_COOKIE).commit()
}
.getOrNull()
}
override fun write(value: String?): Boolean = runCatching {
val editor = prefs.edit()
if (value == null) {
editor.remove(KEY_COOKIE)
} else {
editor.putString(KEY_COOKIE, SealedBox.seal(key(), value))
}
editor.commit()
}.onFailure {
Timber.w(it, "session vault: Keystore unavailable; cookie not stored in the vault")
}.getOrDefault(false)
private fun key(): SecretKey {
val keyStore = KeyStore.getInstance(ANDROID_KEYSTORE).apply { load(null) }
(keyStore.getKey(KEY_ALIAS, null) as? SecretKey)?.let { return it }
val generator = KeyGenerator.getInstance(KeyProperties.KEY_ALGORITHM_AES, ANDROID_KEYSTORE)
generator.init(
KeyGenParameterSpec.Builder(
KEY_ALIAS,
KeyProperties.PURPOSE_ENCRYPT or KeyProperties.PURPOSE_DECRYPT,
)
.setBlockModes(KeyProperties.BLOCK_MODE_GCM)
.setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_NONE)
.setKeySize(KEY_BITS)
.build(),
)
return generator.generateKey()
}
private companion object {
const val ANDROID_KEYSTORE = "AndroidKeyStore"
const val KEY_ALIAS = "minstrel_session_cookie"
const val KEY_BITS = 256
const val PREFS_NAME = "session_vault"
const val KEY_COOKIE = "sealed_cookie"
}
}
@Module
@InstallIn(SingletonComponent::class)
abstract class SessionVaultModule {
@Binds
abstract fun bindSessionVault(impl: KeystoreSessionVault): SessionVault
}
@@ -16,10 +16,11 @@ import javax.inject.Inject
/**
* Computes the initial startDestination for the root NavHost based on
* persisted auth state. Sits on top of AuthSessionDao directly rather
* than AuthStore's StateFlow because the StateFlow defaults to null
* until Room's first async emission — we need a definitive answer
* before drawing any nav graph.
* persisted auth state. Reads the row from AuthSessionDao directly, and
* the session cookie only after [AuthStore.awaitSessionHydrated]: both
* StateFlows default to null until their async load lands, and we need a
* definitive answer before drawing any nav graph. The cookie is no longer
* in the row (it lives in the Keystore-backed SessionVault, #4985).
*
* - no row at all → ServerUrl (first launch)
* - row with baseUrl, no cookie → Login (URL configured, not yet signed in)
@@ -31,6 +32,7 @@ import javax.inject.Inject
@HiltViewModel
class AuthGateViewModel @Inject constructor(
private val dao: AuthSessionDao,
private val authStore: AuthStore,
) : ViewModel() {
private val internal = MutableStateFlow<Any?>(null)
@@ -38,12 +40,13 @@ class AuthGateViewModel @Inject constructor(
init {
viewModelScope.launch {
authStore.awaitSessionHydrated()
val signedIn = !authStore.sessionCookie.value.isNullOrEmpty()
val row = dao.get()
internal.value = when {
row == null -> ServerUrl
row.baseUrl == AuthStore.DEFAULT_BASE_URL && row.sessionCookie.isNullOrEmpty() ->
ServerUrl
row.sessionCookie.isNullOrEmpty() -> Login
row.baseUrl == AuthStore.DEFAULT_BASE_URL && !signedIn -> ServerUrl
!signedIn -> Login
else -> Home
}
}
@@ -3,6 +3,8 @@ package com.fabledsword.minstrel.cache.db
import androidx.room.Database
import androidx.room.RoomDatabase
import androidx.room.TypeConverters
import androidx.room.migration.Migration
import androidx.sqlite.db.SupportSQLiteDatabase
import com.fabledsword.minstrel.cache.db.dao.AudioCacheIndexDao
import com.fabledsword.minstrel.cache.db.dao.AuthSessionDao
import com.fabledsword.minstrel.cache.db.dao.CachedAlbumDao
@@ -65,13 +67,22 @@ import com.fabledsword.minstrel.cache.db.entities.SyncMetadataEntity
AuthSessionEntity::class,
DiagnosticEventEntity::class,
],
// v10: + cached_tracks.trackGain/trackPeak and cached_albums.albumGain/
// albumPeak, the ReplayGain values the player levels by (M464 #5000).
// MIGRATION_9_10 also rewinds the sync cursor, so the next sync re-sends
// every row and an existing cache gains its values.
// v9: + auth_session.normalizationJson, the loudness-normalization
// preference (M464 #4998). The first schema step with an explicit
// Migration (MIGRATION_8_9): a destructive rebuild would also wipe this
// row — the server address and theme — and the queued offline writes,
// which is too much to lose for one added column.
// v8: + cached_tracks.missing, the server's missing-file mark (#2704),
// so cache-first surfaces stop offering files that cannot stream.
// v7: + diagnostic_events table (M9) and the diagnosticsOptOut column
// on auth_session. Pre-v1 destructive fallback rebuilds on mismatch —
// which is exactly right here: the next sync refills every row with the
// new column populated, so there is nothing to migrate by hand.
version = 8,
version = 10,
exportSchema = true,
)
@TypeConverters(MinstrelTypeConverters::class)
@@ -92,3 +103,25 @@ abstract class AppDatabase : RoomDatabase() {
abstract fun authSessionDao(): AuthSessionDao
abstract fun diagnosticEventDao(): DiagnosticEventDao
}
/** v8 → v9: add the nullable normalization preference column (#4998). */
val MIGRATION_8_9: Migration = object : Migration(8, 9) {
override fun migrate(db: SupportSQLiteDatabase) {
db.execSQL("ALTER TABLE auth_session ADD COLUMN normalizationJson TEXT")
}
}
/**
* v9 → v10: the gain columns (#5000). Rows synced before this carry no gains,
* and the sync is incremental, so it would never re-send them: the cursor goes
* back to 0 and the next sync is a full one, upserting every row in place.
*/
val MIGRATION_9_10: Migration = object : Migration(9, 10) {
override fun migrate(db: SupportSQLiteDatabase) {
db.execSQL("ALTER TABLE cached_tracks ADD COLUMN trackGain REAL")
db.execSQL("ALTER TABLE cached_tracks ADD COLUMN trackPeak REAL")
db.execSQL("ALTER TABLE cached_albums ADD COLUMN albumGain REAL")
db.execSQL("ALTER TABLE cached_albums ADD COLUMN albumPeak REAL")
db.execSQL("UPDATE sync_metadata SET cursor = 0")
}
}
@@ -37,6 +37,7 @@ object DatabaseModule {
// launch, so users lose only the unsynced mutation queue
// (acceptable while we're iterating). Replace with explicit
// Migration entries before the first tagged release.
.addMigrations(MIGRATION_8_9, MIGRATION_9_10)
.fallbackToDestructiveMigration(dropAllTables = true)
.build()
@@ -46,4 +46,8 @@ interface AuthSessionDao {
/** Partial update: change only the per-device diagnostics opt-out. */
@Query("UPDATE auth_session SET diagnosticsOptOut = :optOut WHERE id = 0")
suspend fun setDiagnosticsOptOut(optOut: Boolean)
/** Partial update: change only the serialized normalization preference. */
@Query("UPDATE auth_session SET normalizationJson = :json WHERE id = 0")
suspend fun setNormalizationJson(json: String?)
}
@@ -31,4 +31,8 @@ interface CachedMutationDao {
@Query("DELETE FROM cached_mutations")
suspend fun clear()
/** Whether a write of [kind] is still waiting to be replayed. */
@Query("SELECT EXISTS(SELECT 1 FROM cached_mutations WHERE kind = :kind)")
suspend fun hasPending(kind: String): Boolean
}
@@ -38,6 +38,27 @@ interface CachedTrackDao {
@Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun upsertAll(rows: List<CachedTrackEntity>)
/**
* ReplayGain values for [ids] (M464 #5000): the track's own from its row,
* the album's from its album row. A track not in the cache has no row.
*/
@Query(
"SELECT t.id AS id, t.trackGain AS trackGain, t.trackPeak AS trackPeak, " +
"a.albumGain AS albumGain, a.albumPeak AS albumPeak " +
"FROM cached_tracks t LEFT JOIN cached_albums a ON a.id = t.albumId " +
"WHERE t.id IN (:ids)",
)
suspend fun replayGains(ids: List<String>): List<CachedReplayGain>
@Query("DELETE FROM cached_tracks WHERE id IN (:ids)")
suspend fun deleteByIds(ids: List<String>)
}
/** One row of [CachedTrackDao.replayGains]. */
data class CachedReplayGain(
val id: String,
val trackGain: Float?,
val trackPeak: Float?,
val albumGain: Float?,
val albumPeak: Float?,
)
@@ -43,4 +43,10 @@ data class AuthSessionEntity(
* choice lives here. Default false = honor the account flag.
*/
val diagnosticsOptOut: Boolean = false,
/**
* JSON-encoded NormalizationPrefs (settings/data), the last value seen
* from the server or set here (M464 #4998). Null = never fetched; the
* defaults apply. Kept so offline playback still levels.
*/
val normalizationJson: String? = null,
)
@@ -18,5 +18,8 @@ data class CachedAlbumEntity(
val releaseDate: String? = null,
val coverPath: String? = null,
val mbid: String? = null,
// ReplayGain 2.0 album values (M464); null until every track is measured.
val albumGain: Float? = null,
val albumPeak: Float? = null,
val fetchedAt: Instant = Clock.System.now(),
)
@@ -26,5 +26,9 @@ data class CachedTrackEntity(
val fileFormat: String? = null,
val genre: String? = null,
val missing: Boolean = false,
// ReplayGain 2.0 track values (M464), kept so cached audio levels
// offline. Null until the server has measured the track.
val trackGain: Float? = null,
val trackPeak: Float? = null,
val fetchedAt: Instant = Clock.System.now(),
)
@@ -2,6 +2,7 @@ package com.fabledsword.minstrel.cache.mutations
import com.fabledsword.minstrel.cache.db.dao.CachedMutationDao
import com.fabledsword.minstrel.cache.db.entities.CachedMutationEntity
import com.fabledsword.minstrel.settings.data.NormalizationPrefs
import kotlinx.coroutines.channels.BufferOverflow
import kotlinx.coroutines.flow.MutableSharedFlow
import kotlinx.coroutines.flow.SharedFlow
@@ -41,6 +42,11 @@ object MutationKind {
// an undo collapses to the latest intent instead of replaying as two
// opposed calls whose order decides the outcome.
const val SUGGESTION_SNOOZE_TOGGLE: String = "suggestion_snooze_toggle"
// M464 #4998 loudness-normalization preference. The payload is the whole
// preference, a target state like the toggles above, so queued changes
// collapse to the last one and an older one can never be replayed last.
const val NORMALIZATION_SET: String = "normalization_set"
}
/**
@@ -85,6 +91,7 @@ data class RequestCreatePayload(
* This matches `feedback_offline_first_for_server_writes` — writes
* never go fire-and-forget.
*/
@Suppress("TooManyFunctions") // one enqueue per mutation kind, like the replayer's dispatchers
@Singleton
class MutationQueue @Inject constructor(
private val dao: CachedMutationDao,
@@ -177,6 +184,12 @@ class MutationQueue @Inject constructor(
),
)
/** Queues the user's whole normalization preference for replay. */
suspend fun enqueueNormalizationSet(prefs: NormalizationPrefs): Long = insertUserDriven(
MutationKind.NORMALIZATION_SET,
json.encodeToString(NormalizationPrefs.serializer(), prefs),
)
suspend fun enqueueRequestCancel(requestId: String): Long = insertUserDriven(
MutationKind.REQUEST_CANCEL,
json.encodeToString(
@@ -7,6 +7,7 @@ import com.fabledsword.minstrel.api.endpoints.DiscoverApi
import com.fabledsword.minstrel.api.endpoints.EventsApi
import com.fabledsword.minstrel.api.endpoints.FlagRequest
import com.fabledsword.minstrel.api.endpoints.LikesApi
import com.fabledsword.minstrel.api.endpoints.MeApi
import com.fabledsword.minstrel.api.endpoints.PlaybackErrorReportRequest
import com.fabledsword.minstrel.api.endpoints.PlaybackErrorsApi
import com.fabledsword.minstrel.api.endpoints.PlaylistsApi
@@ -22,6 +23,7 @@ import com.fabledsword.minstrel.cache.db.dao.CachedMutationDao
import com.fabledsword.minstrel.cache.db.entities.CachedMutationEntity
import com.fabledsword.minstrel.di.ApplicationScope
import com.fabledsword.minstrel.likes.data.LikesRepository
import com.fabledsword.minstrel.settings.data.NormalizationPrefs
import com.fabledsword.minstrel.models.wire.CreateRequestBody
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.flow.distinctUntilChanged
@@ -79,6 +81,7 @@ class MutationReplayer @Inject constructor(
private val eventsApi: EventsApi = retrofit.create()
private val requestsApi: RequestsApi = retrofit.create()
private val playbackErrorsApi: PlaybackErrorsApi = retrofit.create()
private val meApi: MeApi = retrofit.create()
private val mutex = Mutex()
@@ -166,6 +169,7 @@ class MutationReplayer @Inject constructor(
MutationKind.REQUEST_CANCEL -> dispatchRequestCancel(row.payload)
MutationKind.PLAYBACK_ERROR_REPORT -> dispatchPlaybackErrorReport(row.payload)
MutationKind.SUGGESTION_SNOOZE_TOGGLE -> dispatchSuggestionSnoozeToggle(row.payload)
MutationKind.NORMALIZATION_SET -> dispatchNormalizationSet(row.payload)
// Unknown kind — drop so a stale schema entry can't wedge the queue.
else -> Outcome.DROP
}
@@ -279,6 +283,16 @@ class MutationReplayer @Inject constructor(
return Outcome.SENT
}
/**
* Sends the queued normalization preference. The device already shows
* it, so the server's echo is not written back: a change made since the
* row was queued would be a newer row, and the collapse keeps only that.
*/
private suspend fun dispatchNormalizationSet(payload: String): Outcome {
meApi.putNormalization(json.decodeFromString(NormalizationPrefs.serializer(), payload))
return Outcome.SENT
}
private suspend fun dispatchPlaybackErrorReport(payload: String): Outcome {
val decoded = json.decodeFromString(PlaybackErrorReportPayload.serializer(), payload)
playbackErrorsApi.report(
@@ -303,7 +317,8 @@ class MutationReplayer @Inject constructor(
/**
* Row ids of desired-state toggles superseded by a later toggle for the same
* entity. Applies to every kind whose payload encodes a TARGET state rather
* than an action — like-toggles and suggestion snoozes (#2374) — because
* than an action — like-toggles, suggestion snoozes (#2374) and the
* normalization preference (#4998) — because
* replaying a stale one last would invert the final state.
*
* Top-level and pure so it can be unit-tested without standing up a Retrofit
@@ -340,5 +355,10 @@ private fun toggleKeyOf(row: CachedMutationEntity, json: Json): String? = when (
json.decodeFromString(SuggestionSnoozeTogglePayload.serializer(), row.payload)
}.getOrNull()?.let { "${row.kind}:${it.mbid}" }
// One preference per user, so every normalization row shares one key.
MutationKind.NORMALIZATION_SET -> runCatching {
json.decodeFromString(NormalizationPrefs.serializer(), row.payload)
}.getOrNull()?.let { row.kind }
else -> null
}
@@ -206,6 +206,8 @@ private fun SyncAlbumWire.toEntity(): CachedAlbumEntity = CachedAlbumEntity(
releaseDate = releaseDate,
coverPath = coverArtPath,
mbid = mbid,
albumGain = albumGain,
albumPeak = albumPeak,
)
private fun SyncTrackWire.toEntity(): CachedTrackEntity = CachedTrackEntity(
@@ -220,4 +222,6 @@ private fun SyncTrackWire.toEntity(): CachedTrackEntity = CachedTrackEntity(
fileFormat = fileFormat,
genre = genre,
missing = missing,
trackGain = trackGain,
trackPeak = trackPeak,
)
@@ -0,0 +1,18 @@
package com.fabledsword.minstrel.models.wire
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
/** One track's entry in `GET /api/tracks/replay-gain` (#4997). Decode-only. */
@Serializable
data class ReplayGainWire(
@SerialName("track_gain") val trackGain: Float? = null,
@SerialName("track_peak") val trackPeak: Float? = null,
@SerialName("album_gain") val albumGain: Float? = null,
@SerialName("album_peak") val albumPeak: Float? = null,
)
@Serializable
data class ReplayGainResponseWire(
val items: Map<String, ReplayGainWire> = emptyMap(),
)
@@ -28,6 +28,10 @@ data class SyncAlbumWire(
@SerialName("release_date") val releaseDate: String? = null,
@SerialName("cover_art_path") val coverArtPath: String? = null,
val mbid: String? = null,
// The album's ReplayGain 2.0 values (#4997): dB to -18 LUFS and a linear
// peak. Null until every track on the album is measured.
@SerialName("album_gain") val albumGain: Float? = null,
@SerialName("album_peak") val albumPeak: Float? = null,
)
@Serializable
@@ -51,6 +55,9 @@ data class SyncTrackWire(
// its tracks stay playable, which is the correct reading of "this server
// has nothing to say about missing files".
val missing: Boolean = false,
// The track's ReplayGain 2.0 values (#4997); null until it is measured.
@SerialName("track_gain") val trackGain: Float? = null,
@SerialName("track_peak") val trackPeak: Float? = null,
)
/**
@@ -93,6 +93,8 @@ class MinstrelForwardingPlayer(
* one. Diagnostics-only; see [TransportObservation].
*/
val onTransport: (TransportObservation) -> Unit = {},
/** The renderer's 1-based queue position, every poll. */
val onRendererTrack: (trackNumber: Int) -> Unit = {},
)
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
@@ -622,6 +624,7 @@ class MinstrelForwardingPlayer(
trackNumber = info.track,
)
syncLocalCursorToRemote(sonosTrack = info.track, trackUri = info.trackUri)
events.onRendererTrack(info.track)
val transport = active.avTransport.getTransportInfo()
when (transport.state) {
TransportState.PLAYING -> {
@@ -12,8 +12,10 @@ import androidx.media3.session.SessionCommand
import com.fabledsword.minstrel.MainActivity
import com.fabledsword.minstrel.likes.data.LikesRepository
import com.fabledsword.minstrel.likes.data.LikesRepository.Companion.ENTITY_TRACK
import com.fabledsword.minstrel.settings.data.NormalizationRepository
import com.google.common.collect.ImmutableList
import dagger.hilt.android.AndroidEntryPoint
import kotlinx.coroutines.CancellationException
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.ExperimentalCoroutinesApi
@@ -25,6 +27,7 @@ import kotlinx.coroutines.flow.flatMapLatest
import kotlinx.coroutines.flow.flowOf
import kotlinx.coroutines.flow.onStart
import kotlinx.coroutines.launch
import timber.log.Timber
import javax.inject.Inject
/**
@@ -66,6 +69,8 @@ class MinstrelPlayerService : MediaSessionService() {
@Inject lateinit var likesRepository: LikesRepository
@Inject lateinit var normalizationRepository: NormalizationRepository
private val serviceScope = CoroutineScope(SupervisorJob() + Dispatchers.Main.immediate)
private var mediaSession: MediaSession? = null
@@ -82,6 +87,21 @@ class MinstrelPlayerService : MediaSessionService() {
.build()
mediaSession = session
serviceScope.launch { observeLikeState(session, player) }
serviceScope.launch { refreshNormalization() }
}
// Takes up a leveling preference changed on another device (M464 #5000).
// Offline, the device's copy stands, which is the one playback reads.
private suspend fun refreshNormalization() {
try {
normalizationRepository.refresh()
} catch (e: CancellationException) {
throw e
} catch (
@Suppress("TooGenericExceptionCaught") e: Throwable,
) {
Timber.d(e, "normalization refresh skipped")
}
}
/**
@@ -14,6 +14,8 @@ import androidx.media3.session.MediaController
import androidx.media3.session.SessionToken
import com.fabledsword.minstrel.di.ApplicationScope
import com.fabledsword.minstrel.models.TrackRef
import com.fabledsword.minstrel.player.gain.GainAudioProcessor
import com.fabledsword.minstrel.player.gain.ReplayGainStore
import com.fabledsword.minstrel.playlists.data.PlaylistsRepository
import com.fabledsword.minstrel.playlists.data.toPlayableTrackRefs
import com.fabledsword.minstrel.shared.resolveServerUrl
@@ -69,6 +71,7 @@ class PlayerController @Inject constructor(
private val playerFactory: PlayerFactory,
private val activeUpnpHolder: com.fabledsword.minstrel.player.output.ActiveUpnpHolder,
private val remoteState: RemotePlayerState,
private val replayGains: ReplayGainStore,
) {
/**
@@ -277,6 +280,11 @@ class PlayerController @Inject constructor(
queueRefs = playable.tracks
val items = playable.tracks.map { it.toMediaItem(source) }
val startIndex = playable.initialIndex
// Gains for the first tracks, so the first one levels from its first
// sample rather than ramping in once the lookup lands.
replayGains.request(
playable.tracks.drop(startIndex).take(GAIN_PREFETCH).map { it.id },
)
// Drift #562 cold-boot resume calls this from a non-Main suspend
// context after awaitReady() unblocks (ResumeController launches
// on Dispatchers.Default by the time it reaches us). MediaController
@@ -792,7 +800,13 @@ class PlayerController @Inject constructor(
// scrubber a real total even when the wrapped ExoPlayer is
// paused under UPnP (it never probes a duration in that state).
if (durationSec > 0) setDurationMs(durationSec.toLong() * MS_PER_SECOND)
if (source != null) setExtras(sourceExtras(source))
// Album and track position let the gain processor tell an
// album played in order from a mix (M464 #5000).
trackNumber?.let { setTrackNumber(it) }
discNumber?.let { setDiscNumber(it) }
val extras = GainAudioProcessor.albumExtras(albumId)
if (source != null) extras.putAll(sourceExtras(source))
setExtras(extras)
// Point the notification / lock-screen art at the SAME album
// cover the in-app surfaces use (TrackRef.coverUrl ->
// /api/albums/{id}/cover). Without this, Media3 falls back to
@@ -860,6 +874,7 @@ class PlayerController @Inject constructor(
const val MINSTREL_SOURCE_KEY: String = "minstrel_source"
private const val MS_PER_SECOND = 1_000L
private const val MAX_INTERPOLATION_DRIFT_MS = 5_000L
private const val GAIN_PREFETCH = 20
}
}
@@ -4,6 +4,7 @@ import android.content.Context
import androidx.media3.common.AudioAttributes
import androidx.media3.common.C
import androidx.media3.common.Player
import androidx.media3.common.audio.AudioProcessor
import androidx.media3.common.util.BitmapLoader
import androidx.media3.database.StandaloneDatabaseProvider
import androidx.media3.datasource.DataSourceBitmapLoader
@@ -12,11 +13,18 @@ import androidx.media3.datasource.cache.CacheDataSource
import androidx.media3.datasource.cache.LeastRecentlyUsedCacheEvictor
import androidx.media3.datasource.cache.SimpleCache
import androidx.media3.datasource.okhttp.OkHttpDataSource
import androidx.media3.exoplayer.DefaultRenderersFactory
import androidx.media3.exoplayer.ExoPlayer
import androidx.media3.exoplayer.audio.AudioSink
import androidx.media3.exoplayer.audio.DefaultAudioSink
import androidx.media3.exoplayer.source.DefaultMediaSourceFactory
import androidx.media3.session.CacheBitmapLoader
import com.fabledsword.minstrel.auth.AuthStore
import com.fabledsword.minstrel.cache.audiocache.CacheConfig
import com.fabledsword.minstrel.player.gain.GainAudioProcessor
import com.fabledsword.minstrel.player.gain.ReplayGainStore
import com.fabledsword.minstrel.player.output.ActiveUpnpHolder
import com.fabledsword.minstrel.player.output.SonosQueueLoader
import dagger.hilt.android.qualifiers.ApplicationContext
import kotlinx.coroutines.channels.BufferOverflow
import kotlinx.coroutines.flow.MutableSharedFlow
@@ -56,6 +64,9 @@ class PlayerFactory @Inject constructor(
private val activeUpnpHolder: ActiveUpnpHolder,
private val remoteState: RemotePlayerState,
private val serverHealth: com.fabledsword.minstrel.connectivity.NetworkStatusController,
private val authStore: AuthStore,
private val replayGains: ReplayGainStore,
private val sonosQueue: SonosQueueLoader,
) {
private val cacheDir: File = File(context.cacheDir, "audio_cache").apply { mkdirs() }
@@ -120,6 +131,7 @@ class PlayerFactory @Inject constructor(
onStalled = { trackId -> stallEventsInternal.tryEmit(trackId) },
onQueueTruncated = { queueRepairInternal.tryEmit(Unit) },
onTransport = { transportInternal.tryEmit(it) },
onRendererTrack = { sonosQueue.onRendererTrack(it) },
),
)
}
@@ -142,7 +154,28 @@ class PlayerFactory @Inject constructor(
val mediaSourceFactory = DefaultMediaSourceFactory(context)
.setDataSourceFactory(cacheDataSource)
return ExoPlayer.Builder(context)
// Loudness normalization (M464 #5000) runs inside the audio sink so
// each track's gain starts on its first sample. Audio offload would
// bypass the sink's processors; ExoPlayer leaves it off unless asked
// (TrackSelectionParameters.audioOffloadPreferences), and nothing here
// asks, so leveling always applies.
val gainProcessor = GainAudioProcessor(
prefs = { authStore.normalization.value },
store = replayGains,
)
val renderersFactory = object : DefaultRenderersFactory(context) {
override fun buildAudioSink(
context: Context,
enableFloatOutput: Boolean,
enableAudioOutputPlaybackParams: Boolean,
): AudioSink = DefaultAudioSink.Builder(context)
.setEnableFloatOutput(enableFloatOutput)
.setEnableAudioOutputPlaybackParameters(enableAudioOutputPlaybackParams)
.setAudioProcessors(arrayOf<AudioProcessor>(gainProcessor))
.build()
}
val exo = ExoPlayer.Builder(context, renderersFactory)
.setMediaSourceFactory(mediaSourceFactory)
.setAudioAttributes(
AudioAttributes.Builder()
@@ -153,6 +186,17 @@ class PlayerFactory @Inject constructor(
)
.setHandleAudioBecomingNoisy(true)
.build()
// The processor finds a track's neighbours in play order, which
// depends on shuffle mode.
gainProcessor.shuffleEnabled = exo.shuffleModeEnabled
exo.addListener(
object : Player.Listener {
override fun onShuffleModeEnabledChanged(shuffleModeEnabled: Boolean) {
gainProcessor.shuffleEnabled = shuffleModeEnabled
}
},
)
return exo
}
/**
@@ -19,6 +19,13 @@ import javax.inject.Singleton
class StreamTokenProvider @Inject constructor(retrofit: Retrofit) {
private val api: CastApi = retrofit.create()
suspend fun mint(trackId: String): StreamTokenResponse =
api.streamToken(StreamTokenRequest(trackId = trackId))
/**
* Mints a URL for a speaker, leveled when the user's setting calls for
* it (M464 #5002). The server answers with the plain stream when it does
* not, so every speaker URL asks.
*/
suspend fun mint(trackId: String, asAlbum: Boolean = false, prerender: Boolean = false): StreamTokenResponse =
api.streamToken(
StreamTokenRequest(trackId = trackId, level = true, asAlbum = asAlbum, prerender = prerender),
)
}
@@ -0,0 +1,139 @@
package com.fabledsword.minstrel.player.gain
import android.os.Bundle
import androidx.media3.common.C
import androidx.media3.common.MediaItem
import androidx.media3.common.Player
import androidx.media3.common.Timeline
import androidx.media3.common.audio.AudioProcessor
import androidx.media3.common.audio.BaseAudioProcessor
import com.fabledsword.minstrel.settings.data.NormalizationBoost
import com.fabledsword.minstrel.settings.data.NormalizationMode
import com.fabledsword.minstrel.settings.data.NormalizationPrefs
import java.nio.ByteBuffer
/**
* Applies each track's loudness gain inside ExoPlayer's audio sink (M464
* #5000), so the level changes on the exact sample a track starts, gapless
* transitions included. `Player.setVolume` could do neither: it stops at 1,
* and set from a transition callback it lands about two seconds late
* because the next track is already buffered (androidx/media#418).
*
* Media3 1.11 flushes the sink's processors at every item boundary with the
* playlist [Timeline] and the new item's period, which is how this knows
* which track it is processing and who its neighbours are.
*
* Boosts above unity are held under -1 dBFS by a peak limiter when the user
* chose one. In headroom mode the gain already stops short of the track's
* true peak, and the final clamp only guards against a bad measurement.
*
* Runs on the playback thread. [prefs] and [store] are read per buffer, so a
* changed preference or a gain that arrives mid-track applies at once,
* through a short ramp rather than a step.
*/
class GainAudioProcessor(
private val prefs: () -> NormalizationPrefs,
private val store: ReplayGainStore,
) : BaseAudioProcessor() {
/** Mirrors the player's shuffle mode, so neighbours are found in play order. */
@Volatile var shuffleEnabled: Boolean = false
private var currentId: String? = null
private var asAlbum = false
private val stage = GainStage()
override fun onConfigure(inputAudioFormat: AudioProcessor.AudioFormat): AudioProcessor.AudioFormat =
when (inputAudioFormat.encoding) {
C.ENCODING_PCM_16BIT, C.ENCODING_PCM_FLOAT -> inputAudioFormat
else -> AudioProcessor.AudioFormat.NOT_SET
}
override fun onFlush(streamMetadata: AudioProcessor.StreamMetadata) {
identify(streamMetadata)
// A new stream starts at its own level: ramping in from the previous
// track's gain would swell or dip its first moments.
stage.reset(inputAudioFormat.sampleRate, inputAudioFormat.channelCount, targetGain())
}
override fun onReset() {
currentId = null
asAlbum = false
}
override fun queueInput(inputBuffer: ByteBuffer) {
val size = inputBuffer.remaining()
if (size == 0) return
val out = replaceOutputBuffer(size)
val p = prefs()
val target = targetGain(p)
val limiting = p.mode != NormalizationMode.OFF && p.boost == NormalizationBoost.LIMITER
if (stage.isUnity(target, limiting)) {
out.put(inputBuffer)
} else {
stage.process(
input = inputBuffer,
out = out,
channels = inputAudioFormat.channelCount,
isFloat = inputAudioFormat.encoding == C.ENCODING_PCM_FLOAT,
target = target,
limiting = limiting,
)
}
out.flip()
}
private fun targetGain(p: NormalizationPrefs = prefs()): Float {
val id = currentId ?: return 1f
return GainMath.dbToLinear(GainMath.gainDb(p, store.get(id), asAlbum))
}
// Which track this stream is, and whether it is playing as part of its
// album. Also asks the store for the gains of the tracks coming up, so
// each is known before it starts.
private fun identify(meta: AudioProcessor.StreamMetadata) {
currentId = null
asAlbum = false
val uid = meta.periodUid
if (uid == null || meta.timeline.isEmpty) return
val timeline = meta.timeline
val index = timeline.getPeriodByUid(uid, Timeline.Period()).windowIndex
val window = Timeline.Window()
fun itemAt(i: Int): MediaItem? =
if (i == C.INDEX_UNSET) null else timeline.getWindow(i, window).mediaItem
fun nextOf(i: Int) = timeline.getNextWindowIndex(i, Player.REPEAT_MODE_OFF, shuffleEnabled)
val cur = itemAt(index)
val prev = itemAt(timeline.getPreviousWindowIndex(index, Player.REPEAT_MODE_OFF, shuffleEnabled))
val next = itemAt(nextOf(index))
if (cur != null) {
currentId = cur.mediaId
asAlbum = GainMath.playingAsAlbum(prev?.albumPosition(), cur.albumPosition(), next?.albumPosition())
}
val upcoming = mutableListOf<String>()
var i = index
while (i != C.INDEX_UNSET && upcoming.size < LOOKAHEAD) {
itemAt(i)?.let { upcoming += it.mediaId }
i = nextOf(i)
}
store.request(upcoming)
}
companion object {
/** MediaMetadata extras key holding the track's album id. */
const val EXTRA_ALBUM_ID = "minstrel.album_id"
private const val LOOKAHEAD = 20
/** Puts what [albumPosition] reads into a MediaItem's metadata extras. */
fun albumExtras(albumId: String, into: Bundle = Bundle()): Bundle =
into.apply { putString(EXTRA_ALBUM_ID, albumId) }
}
}
private fun MediaItem.albumPosition(): AlbumPosition = AlbumPosition(
albumId = mediaMetadata.extras?.getString(GainAudioProcessor.EXTRA_ALBUM_ID),
discNumber = mediaMetadata.discNumber,
trackNumber = mediaMetadata.trackNumber,
)
@@ -0,0 +1,96 @@
package com.fabledsword.minstrel.player.gain
import com.fabledsword.minstrel.settings.data.NormalizationBoost
import com.fabledsword.minstrel.settings.data.NormalizationMode
import com.fabledsword.minstrel.settings.data.NormalizationPrefs
import kotlin.math.log10
import kotlin.math.min
import kotlin.math.pow
/**
* One track's ReplayGain 2.0 values from the server (#4997): dB to the
* -18 LUFS reference, and linear true peaks. A null field has not been
* measured yet.
*/
data class ReplayGain(
val trackGain: Float?,
val trackPeak: Float?,
val albumGain: Float?,
val albumPeak: Float?,
) {
companion object {
val NONE = ReplayGain(null, null, null, null)
}
}
/** Where a queue item sits in its album, for the auto-mode album rule. */
data class AlbumPosition(val albumId: String?, val discNumber: Int?, val trackNumber: Int?)
/**
* Loudness-normalization math (M464 #5000). The same rules as the web
* player's `web/src/lib/player/gain.ts`, so a track levels the same on
* every device.
*/
object GainMath {
private const val REFERENCE_LUFS = -18
/** Headroom mode raises a quiet track only until its true peak reaches this. */
const val PEAK_CEILING_DBTP = -1f
/**
* No track is raised more than this, whatever its measurement says: a
* near-silent track would otherwise come out as amplified noise.
*/
const val MAX_BOOST_DB = 12f
private const val DISC_STRIDE = 1000
// Amplitude decibels: 20 dB per factor of ten.
private const val DB_PER_DECADE = 20f
/**
* The gain to apply, in dB. 0 when leveling is off or the track has not
* been measured: an unmeasured track plays as mastered.
*/
fun gainDb(prefs: NormalizationPrefs, g: ReplayGain?, asAlbum: Boolean): Float {
if (prefs.mode == NormalizationMode.OFF || g == null) return 0f
val wantAlbum = prefs.mode == NormalizationMode.ALBUM ||
(prefs.mode == NormalizationMode.AUTO && asAlbum)
// Album gain falls back to track gain while the album is still being
// measured; track gain never falls back to album gain.
val useAlbum = wantAlbum && g.albumGain != null
val gain = if (useAlbum) g.albumGain else g.trackGain
val peak = if (useAlbum) g.albumPeak else g.trackPeak
return gain?.let { leveled(prefs, it, peak) } ?: 0f
}
private fun leveled(prefs: NormalizationPrefs, gain: Float, peak: Float?): Float {
var db = gain + (prefs.targetLufs - REFERENCE_LUFS)
if (prefs.boost == NormalizationBoost.HEADROOM && peak != null && peak > 0f) {
db = min(db, PEAK_CEILING_DBTP - DB_PER_DECADE * log10(peak))
}
return min(db, MAX_BOOST_DB)
}
/**
* Whether the current item is being played as part of its album, in
* order: a neighbour in play order is from the same album and sits on
* the right side of it. That is when album gain keeps the album's own
* dynamics (a quiet intro stays quiet); anywhere else track gain levels
* the mix.
*/
fun playingAsAlbum(prev: AlbumPosition?, cur: AlbumPosition, next: AlbumPosition?): Boolean {
val curOrder = order(cur)
if (cur.albumId == null || curOrder == null) return false
val prevOrder = prev?.takeIf { it.albumId == cur.albumId }?.let { order(it) }
val nextOrder = next?.takeIf { it.albumId == cur.albumId }?.let { order(it) }
return (prevOrder != null && prevOrder < curOrder) || (nextOrder != null && nextOrder > curOrder)
}
// Disc-major track order. A track with no number has no place in the
// order and is never evidence of album play.
private fun order(p: AlbumPosition): Int? =
p.trackNumber?.let { (p.discNumber ?: 1) * DISC_STRIDE + it }
fun dbToLinear(db: Float): Float = 10f.pow(db / DB_PER_DECADE)
}
@@ -0,0 +1,82 @@
package com.fabledsword.minstrel.player.gain
import java.nio.ByteBuffer
import kotlin.math.abs
import kotlin.math.exp
import kotlin.math.max
import kotlin.math.roundToInt
/**
* The sample arithmetic of [GainAudioProcessor], apart from Media3 so it can
* be tested on the JVM: a gain that ramps toward its target, then an
* optional peak limiter, then a clamp to full scale. Interleaved 16-bit or
* float PCM in, the same format out.
*/
internal class GainStage {
private var gain = 1f
private var envelope = 0f
private var rampCoeff = 1f
private var releaseCoeff = 0f
private var frame = FloatArray(2)
/** Sets the sample rate and channel count, and starts the gain at [startGain]. */
fun reset(sampleRate: Int, channels: Int, startGain: Float) {
val rate = sampleRate.coerceAtLeast(1).toFloat()
rampCoeff = 1f - exp(-1f / (RAMP_SECONDS * rate))
releaseCoeff = exp(-1f / (RELEASE_SECONDS * rate))
if (frame.size < channels) frame = FloatArray(channels)
gain = startGain
envelope = 0f
}
/** True when processing would copy the input unchanged. */
fun isUnity(target: Float, limiting: Boolean): Boolean = gain == 1f && target == 1f && !limiting
/**
* Processes every whole frame of [input] into [out]. A trailing partial
* frame is dropped, as Media3's own processors do.
*/
@Suppress("LongParameterList") // the PCM layout is four facts; a holder type would only rename them
fun process(
input: ByteBuffer,
out: ByteBuffer,
channels: Int,
isFloat: Boolean,
target: Float,
limiting: Boolean,
) {
val bytesPerFrame = channels * if (isFloat) FLOAT_BYTES else PCM16_BYTES
val frames = input.remaining() / bytesPerFrame
repeat(frames) {
gain += (target - gain) * rampCoeff
var peak = 0f
for (c in 0 until channels) {
val s = (if (isFloat) input.float else input.short / PCM16_SCALE) * gain
frame[c] = s
peak = max(peak, abs(s))
}
var reduce = 1f
if (limiting) {
envelope = max(peak, envelope * releaseCoeff)
if (envelope > LIMIT_CEILING) reduce = LIMIT_CEILING / envelope
}
for (c in 0 until channels) {
val v = (frame[c] * reduce).coerceIn(-1f, 1f)
if (isFloat) out.putFloat(v) else out.putShort((v * PCM16_MAX).roundToInt().toShort())
}
}
input.position(input.limit())
}
companion object {
/** -1 dBFS, the ceiling the web player's limiter holds too. */
const val LIMIT_CEILING = 0.8913f
private const val RAMP_SECONDS = 0.05f
private const val RELEASE_SECONDS = 0.25f
private const val PCM16_SCALE = 32768f
private const val PCM16_MAX = 32767f
private const val PCM16_BYTES = 2
private const val FLOAT_BYTES = 4
}
}
@@ -0,0 +1,140 @@
package com.fabledsword.minstrel.player.gain
import com.fabledsword.minstrel.api.endpoints.ReplayGainApi
import com.fabledsword.minstrel.cache.db.dao.CachedTrackDao
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.ServerHealth
import com.fabledsword.minstrel.di.ApplicationScope
import kotlinx.coroutines.CancellationException
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.launch
import kotlinx.coroutines.withTimeoutOrNull
import retrofit2.Retrofit
import retrofit2.create
import timber.log.Timber
import java.util.concurrent.ConcurrentHashMap
import javax.inject.Inject
import javax.inject.Singleton
/**
* The gains the player levels by (M464 #5000), looked up per track and
* held for the life of the process. The audio thread reads them with [get];
* [request] fills them in the background.
*
* Sources, most to least preferred:
* 1. the library cache, synced with the gains, so cached audio levels
* offline;
* 2. the server's replay-gain lookup, for a track not cached yet or
* measured since the last sync;
* 3. none: the track plays as mastered, and the player picks the gain up
* the moment it lands.
*/
@Singleton
class ReplayGainStore internal constructor(
private val scope: CoroutineScope,
private val trackDao: CachedTrackDao,
private val api: ReplayGainApi,
private val serverHealth: () -> ServerHealth,
private val clock: () -> Long,
) {
@Inject constructor(
@ApplicationScope scope: CoroutineScope,
trackDao: CachedTrackDao,
retrofit: Retrofit,
network: NetworkStatusController,
) : this(
scope = scope,
trackDao = trackDao,
api = retrofit.create(),
serverHealth = { network.state.value },
clock = System::currentTimeMillis,
)
private val gains = ConcurrentHashMap<String, ReplayGain>()
private val inflight: MutableSet<String> = ConcurrentHashMap.newKeySet()
// When the server last said it had nothing for a track. Not cached for
// good: the backfill may measure it minutes later.
private val missedAt = ConcurrentHashMap<String, Long>()
/** The track's gains, or null while they are unknown. Safe from any thread. */
fun get(trackId: String): ReplayGain? = gains[trackId]
/** Starts loading gains for any of [trackIds] not already known or loading. */
fun request(trackIds: Collection<String>) {
val now = clock()
val wanted = trackIds.filter { id ->
!gains.containsKey(id) &&
(missedAt[id]?.let { now - it > MISS_RETRY_MS } ?: true) &&
inflight.add(id)
}
if (wanted.isEmpty()) return
scope.launch {
try {
load(wanted)
} finally {
inflight.removeAll(wanted.toSet())
}
}
}
internal suspend fun load(ids: List<String>) {
fromCache(ids)
val rest = ids.filter { !gains.containsKey(it) }
val health = serverHealth()
if (rest.isEmpty() || health == ServerHealth.Offline || health == ServerHealth.ServerDown) return
for (batch in rest.chunked(MAX_IDS_PER_REQUEST)) {
// A failed lookup leaves the batch unknown; the next request asks again.
if (!fromServer(batch)) return
}
}
private suspend fun fromCache(ids: List<String>) {
val rows = try {
trackDao.replayGains(ids)
} catch (e: CancellationException) {
throw e
} catch (
@Suppress("TooGenericExceptionCaught") e: Throwable,
) {
Timber.w(e, "replay gain: cache read failed")
emptyList()
}
for (row in rows) {
// A cached row without a track gain is not an answer: the server
// may have measured the track since this device last synced.
if (row.trackGain == null) continue
gains[row.id] = ReplayGain(row.trackGain, row.trackPeak, row.albumGain, row.albumPeak)
}
}
/** Asks the server about [batch]; false when it could not be asked. */
private suspend fun fromServer(batch: List<String>): Boolean {
val res = try {
withTimeoutOrNull(REQUEST_TIMEOUT_MS) { api.getReplayGain(batch.joinToString(",")) }
} catch (e: CancellationException) {
throw e
} catch (
@Suppress("TooGenericExceptionCaught") e: Throwable,
) {
Timber.w(e, "replay gain: lookup failed")
null
} ?: return false
val now = clock()
for (id in batch) {
val w = res.items[id]
if (w == null) {
missedAt[id] = now
} else {
gains[id] = ReplayGain(w.trackGain, w.trackPeak, w.albumGain, w.albumPeak)
}
}
return true
}
private companion object {
const val MAX_IDS_PER_REQUEST = 200 // the endpoint's limit
const val REQUEST_TIMEOUT_MS = 10_000L
const val MISS_RETRY_MS = 5 * 60_000L
}
}
@@ -4,6 +4,8 @@ import com.fabledsword.minstrel.di.ApplicationScope
import com.fabledsword.minstrel.models.TrackRef
import com.fabledsword.minstrel.player.RemotePlayerState
import com.fabledsword.minstrel.player.StreamTokenProvider
import com.fabledsword.minstrel.player.gain.AlbumPosition
import com.fabledsword.minstrel.player.gain.GainMath
import com.fabledsword.minstrel.player.output.upnp.AVTransportClient
import com.fabledsword.minstrel.player.output.upnp.SoapFaultException
import com.fabledsword.minstrel.player.output.upnp.bareUdn
@@ -24,6 +26,11 @@ import javax.inject.Singleton
* with its own failure modes, and it had grown large enough to hide one:
* every write here is a SOAP call that can fail individually, and until
* [verifyQueueLength] nothing ever read the result back.
*
* Every URL sent is a leveled one (M464 #5002): the server renders the track
* at the user's loudness gain, or hands back the plain stream when leveling
* is off. Renders are slow enough to matter, so the track playing and the
* one after it are rendered ahead; the rest render when the speaker asks.
*/
@Singleton
class SonosQueueLoader @Inject constructor(
@@ -32,6 +39,13 @@ class SonosQueueLoader @Inject constructor(
private val activeUpnpHolder: ActiveUpnpHolder,
private val remoteState: RemotePlayerState,
) {
// The queue as last sent to the renderer, for looking up the track after
// the one it is playing.
@Volatile private var sent: List<TrackRef> = emptyList()
// The renderer track the next one was last prerendered for.
@Volatile private var prerenderedAfter = 0
suspend fun load(
transport: AVTransportClient,
route: OutputRoute,
@@ -46,8 +60,10 @@ class SonosQueueLoader @Inject constructor(
"UPnP select: add %d initial tracks (currentIndex=%d, totalQueue=%d)",
initialBatch.size, currentIndex, queue.size,
)
initialBatch.forEachIndexed { idx, ref ->
val token = streamTokens.mint(ref.id)
sent = queue
prerenderedAfter = currentIndex + 1
initialBatch.indices.forEach { idx ->
val token = mint(queue, idx, prerender = idx == currentIndex)
transport.addURIToQueue(
uri = token.url,
mime = token.mime,
@@ -64,12 +80,12 @@ class SonosQueueLoader @Inject constructor(
Timber.w("UPnP select: Play")
transport.play()
Timber.w("UPnP select: initial done; backgrounding remainder")
val remaining = queue.drop(initialEnd)
// Verify even when there is no tail to append: the initial batch is
// sent the same way and can be dropped the same way.
scope.launch {
if (remaining.isNotEmpty()) {
extendQueueOnSonos(transport, route, remaining, initialEnd)
prerender(queue, currentIndex + 1)
if (initialEnd < queue.size) {
extendQueueOnSonos(transport, route, queue, initialEnd)
}
verifyQueueLength(transport, route, queue)
}
@@ -84,15 +100,16 @@ class SonosQueueLoader @Inject constructor(
private suspend fun extendQueueOnSonos(
transport: AVTransportClient,
route: OutputRoute,
tracks: List<TrackRef>,
queue: List<TrackRef>,
startPosition: Int,
) {
val count = queue.size - startPosition
Timber.w(
"UPnP extend: appending %d tracks starting at position %d",
tracks.size, startPosition + 1,
count, startPosition + 1,
)
val succeeded = appendTracksToQueue(transport, route, tracks, startPosition)
Timber.w("UPnP extend: done (%d / %d appended)", succeeded, tracks.size)
val succeeded = appendTracksToQueue(transport, route, queue, startPosition)
Timber.w("UPnP extend: done (%d / %d appended)", succeeded, count)
}
/**
@@ -136,18 +153,17 @@ class SonosQueueLoader @Inject constructor(
Timber.w("UPnP verify: renderer holds %d tracks, queue intact", nrTracks)
return
}
val missing = fullQueue.drop(nrTracks)
Timber.w(
"UPnP verify: %s holds %d of %d tracks; appending %d missing (round %d)",
route.name, nrTracks, fullQueue.size, missing.size, round + 1,
route.name, nrTracks, fullQueue.size, fullQueue.size - nrTracks, round + 1,
)
appendTracksToQueue(transport, route, missing, nrTracks)
appendTracksToQueue(transport, route, fullQueue, nrTracks)
}
Timber.w("UPnP verify: gave up repairing queue length on %s", route.name)
}
/**
* Append [tracks] at [startPosition] (0-based), returning how many landed.
* Append [queue] from [startPosition] (0-based) on, returning how many landed.
* Tolerates individual AddURIToQueue failures — log and continue so some
* tracks loaded is better than zero tracks loaded — and stops early after
* [EXTEND_ABORT_AFTER_FAILURES] consecutive ones.
@@ -155,20 +171,20 @@ class SonosQueueLoader @Inject constructor(
private suspend fun appendTracksToQueue(
transport: AVTransportClient,
route: OutputRoute,
tracks: List<TrackRef>,
queue: List<TrackRef>,
startPosition: Int,
): Int {
var consecutiveFailures = 0
var succeeded = 0
var aborted = false
for ((i, ref) in tracks.withIndex()) {
for (i in 0 until queue.size - startPosition) {
if (aborted) break
if (activeUpnpHolder.active.value?.routeId != route.id) {
Timber.w("UPnP extend: cancelled at offset %d (route changed)", i)
aborted = true
} else {
val outcome = runCatching {
val token = streamTokens.mint(ref.id)
val token = mint(queue, startPosition + i)
transport.addURIToQueue(
uri = token.url,
mime = token.mime,
@@ -220,6 +236,7 @@ class SonosQueueLoader @Inject constructor(
newQueue: List<TrackRef>,
): Boolean {
val newIds = newQueue.map { it.id }
sent = newQueue
if (oldIds == newIds) return true
val prefixLen = commonPrefixLength(oldIds, newIds)
val suffixLen = commonSuffixLength(
@@ -271,8 +288,7 @@ class SonosQueueLoader @Inject constructor(
prefixLen + 1,
)
for (i in 0 until addedCount) {
val ref = newQueue[prefixLen + i]
val token = streamTokens.mint(ref.id)
val token = mint(newQueue, prefixLen + i)
transport.addURIToQueue(
uri = token.url,
mime = token.mime,
@@ -283,6 +299,27 @@ class SonosQueueLoader @Inject constructor(
}
}
/**
* Called on every poll with the renderer's 1-based track number. When it
* moves, the track after it is rendered ahead, so the speaker's fetch of
* it finds the render ready.
*/
fun onRendererTrack(trackNumber: Int) {
if (trackNumber <= 0 || trackNumber == prerenderedAfter) return
prerenderedAfter = trackNumber
val queue = sent
scope.launch { prerender(queue, trackNumber) }
}
private suspend fun prerender(queue: List<TrackRef>, index: Int) {
if (index !in queue.indices) return
runCatching { mint(queue, index, prerender = true) }
.onFailure { Timber.w(it, "UPnP prerender failed for %s", queue[index].id) }
}
private suspend fun mint(queue: List<TrackRef>, index: Int, prerender: Boolean = false) =
streamTokens.mint(queue[index].id, asAlbum = playingAsAlbum(queue, index), prerender = prerender)
private fun commonPrefixLength(a: List<String>, b: List<String>): Int {
val limit = minOf(a.size, b.size)
for (i in 0 until limit) {
@@ -299,18 +336,33 @@ class SonosQueueLoader @Inject constructor(
return limit
}
private companion object {
companion object {
/**
* Whether [queue]'s track at [index] plays among its album in order,
* judged by its neighbours in the renderer's queue, which plays
* straight through.
*/
internal fun playingAsAlbum(queue: List<TrackRef>, index: Int): Boolean =
GainMath.playingAsAlbum(
prev = queue.getOrNull(index - 1)?.let(::position),
cur = position(queue[index]),
next = queue.getOrNull(index + 1)?.let(::position),
)
private fun position(t: TrackRef) =
AlbumPosition(t.albumId.ifEmpty { null }, t.discNumber, t.trackNumber)
// Abort the append loop after this many consecutive AddURIToQueue
// failures; Sonos rate-limits burst adds and a wall of failures means
// it has stopped accepting, not that the next one might land.
const val EXTEND_ABORT_AFTER_FAILURES = 3
const val EXTEND_THROTTLE_MS = 50L
private const val EXTEND_ABORT_AFTER_FAILURES = 3
private const val EXTEND_THROTTLE_MS = 50L
// Verify/repair passes after a queue load. Two: one to catch the
// common case (a rate-limit burst dropped a chunk), one to catch a
// repair that itself got rate-limited. Beyond that the renderer is
// refusing for a reason retrying won't fix, and the stall watchdog
// becomes the backstop.
const val VERIFY_ROUNDS = 2
private const val VERIFY_ROUNDS = 2
}
}
@@ -0,0 +1,55 @@
package com.fabledsword.minstrel.settings.data
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
/**
* The user's loudness-normalization preference (M464 #4998). The server
* holds it, so this device, the web player and casts all level the same
* way; a copy is kept on the device so offline playback still applies it.
* Field names and values match `GET/PUT /api/me/normalization`.
*
* No constructor defaults, on purpose: the app's Json leaves out any field
* equal to its default (encodeDefaults is off), and the server refuses a PUT
* missing a field. Without defaults every field is always sent.
*/
@Serializable
data class NormalizationPrefs(
val mode: NormalizationMode,
@SerialName("target_lufs") val targetLufs: Int,
val boost: NormalizationBoost,
) {
companion object {
/** The targets the server accepts, quietest first. */
val TARGETS: List<Int> = listOf(-18, -16, -14)
/** Must match library.DefaultNormalizationPrefs on the server. */
val DEFAULT: NormalizationPrefs = NormalizationPrefs(
mode = NormalizationMode.AUTO,
targetLufs = -18,
boost = NormalizationBoost.HEADROOM,
)
}
}
@Serializable
enum class NormalizationMode {
/** Tracks play at their mastered volume. */
@SerialName("off") OFF,
/** Album gain while an album plays in order, track gain otherwise. */
@SerialName("auto") AUTO,
@SerialName("track") TRACK,
@SerialName("album") ALBUM,
}
@Serializable
enum class NormalizationBoost {
/** Quiet tracks are raised only as far as their true peak allows. */
@SerialName("headroom") HEADROOM,
/** Quiet tracks are raised all the way and a limiter holds the peaks. */
@SerialName("limiter") LIMITER,
}
@@ -0,0 +1,67 @@
package com.fabledsword.minstrel.settings.data
import com.fabledsword.minstrel.api.endpoints.MeApi
import com.fabledsword.minstrel.auth.AuthStore
import com.fabledsword.minstrel.cache.db.dao.CachedMutationDao
import com.fabledsword.minstrel.cache.mutations.MutationKind
import com.fabledsword.minstrel.cache.mutations.MutationQueue
import kotlinx.coroutines.CancellationException
import kotlinx.coroutines.flow.StateFlow
import retrofit2.Retrofit
import retrofit2.create
import timber.log.Timber
import javax.inject.Inject
import javax.inject.Singleton
/**
* The user's loudness-normalization preference (M464 #4998). The device's
* copy lives in [AuthStore] so playback reads it offline; the server's copy
* is what other devices and casts use.
*
* Writes follow rule 100: the device shows the change at once, the PUT is
* best effort, and a failed PUT is queued for the MutationReplayer.
*/
@Singleton
class NormalizationRepository @Inject constructor(
retrofit: Retrofit,
private val authStore: AuthStore,
private val mutationQueue: MutationQueue,
private val mutationDao: CachedMutationDao,
) {
private val api: MeApi = retrofit.create()
val prefs: StateFlow<NormalizationPrefs> = authStore.normalization
/**
* Takes the server's value, unless a change made here is still queued:
* the server has not seen it yet, and taking its older value would undo
* the change on screen until the replay landed. Throws on a network
* failure; the device's copy then stands.
*/
suspend fun refresh() {
val server = api.getNormalization()
if (mutationDao.hasPending(MutationKind.NORMALIZATION_SET)) return
authStore.setNormalization(server)
}
suspend fun set(next: NormalizationPrefs) {
authStore.setNormalization(next)
// An earlier change still queued would be replayed after this PUT
// and undo it. Queue behind it instead: the replayer sends only the
// newest queued preference.
if (mutationDao.hasPending(MutationKind.NORMALIZATION_SET)) {
mutationQueue.enqueueNormalizationSet(next)
return
}
try {
api.putNormalization(next)
} catch (e: CancellationException) {
throw e
} catch (
@Suppress("TooGenericExceptionCaught") e: Throwable,
) {
Timber.i(e, "normalization: PUT failed; queued for replay")
mutationQueue.enqueueNormalizationSet(next)
}
}
}
@@ -41,6 +41,10 @@ data class AboutUiState(
// check must decide on. Held in state rather than read inline so a test
// can drive the comparison without a BuildConfig.
val installedCode: Long = BuildConfig.VERSION_CODE.toLong(),
// A debug build is signed with this machine's debug key, so the server's
// release-signed APK can never install over it (family idea #5103,
// practice 9). It updates from Android Studio instead.
val selfUpdates: Boolean = !BuildConfig.DEBUG,
val isChecking: Boolean = false,
val installStage: InstallStage = InstallStage.IDLE,
val installMessage: String? = null,
@@ -67,7 +71,7 @@ class AboutCardViewModel @Inject constructor(
val state: StateFlow<AboutUiState> = internal.asStateFlow()
fun checkForUpdates() {
if (internal.value.isChecking) return
if (internal.value.isChecking || !internal.value.selfUpdates) return
viewModelScope.launch {
internal.update { it.copy(isChecking = true, installMessage = null) }
val installed = internal.value.installedVersion
@@ -0,0 +1,143 @@
package com.fabledsword.minstrel.settings.ui
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.material3.ElevatedCard
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.SegmentedButton
import androidx.compose.material3.SegmentedButtonDefaults
import androidx.compose.material3.SingleChoiceSegmentedButtonRow
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import androidx.hilt.navigation.compose.hiltViewModel
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import com.fabledsword.minstrel.settings.data.NormalizationBoost
import com.fabledsword.minstrel.settings.data.NormalizationMode
import com.fabledsword.minstrel.settings.data.NormalizationPrefs
/**
* Volume leveling (M464 #4998). The choice is stored on the server, so the
* web player and casts follow it too.
*/
@Composable
fun NormalizationCard(viewModel: NormalizationViewModel = hiltViewModel()) {
val prefs by viewModel.prefs.collectAsStateWithLifecycle()
NormalizationCardContent(prefs = prefs, onChange = viewModel::update)
}
@OptIn(ExperimentalMaterial3Api::class)
@Composable
internal fun NormalizationCardContent(
prefs: NormalizationPrefs,
onChange: (NormalizationPrefs) -> Unit,
) {
ElevatedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(
text = "Volume leveling",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
)
Hint(modeHint(prefs.mode))
val modes = NormalizationMode.entries
SingleChoiceSegmentedButtonRow(modifier = Modifier.fillMaxWidth()) {
modes.forEachIndexed { index, mode ->
SegmentedButton(
selected = mode == prefs.mode,
onClick = { onChange(prefs.copy(mode = mode)) },
shape = SegmentedButtonDefaults.itemShape(
index = index,
count = modes.size,
),
) { Text(modeLabel(mode)) }
}
}
if (prefs.mode != NormalizationMode.OFF) {
TargetRow(prefs = prefs, onChange = onChange)
BoostRow(prefs = prefs, onChange = onChange)
}
}
}
}
@OptIn(ExperimentalMaterial3Api::class)
@Composable
private fun TargetRow(prefs: NormalizationPrefs, onChange: (NormalizationPrefs) -> Unit) {
Text(
text = "Target loudness",
style = MaterialTheme.typography.bodyLarge,
color = MaterialTheme.colorScheme.onSurface,
)
val targets = NormalizationPrefs.TARGETS
SingleChoiceSegmentedButtonRow(modifier = Modifier.fillMaxWidth()) {
targets.forEachIndexed { index, lufs ->
SegmentedButton(
selected = lufs == prefs.targetLufs,
onClick = { onChange(prefs.copy(targetLufs = lufs)) },
shape = SegmentedButtonDefaults.itemShape(
index = index,
count = targets.size,
),
) { Text("$lufs LUFS") }
}
}
}
@Composable
private fun BoostRow(prefs: NormalizationPrefs, onChange: (NormalizationPrefs) -> Unit) {
Row(verticalAlignment = Alignment.CenterVertically) {
Column(modifier = Modifier.weight(1f)) {
Text(
text = "Boost quiet tracks fully",
style = MaterialTheme.typography.bodyLarge,
color = MaterialTheme.colorScheme.onSurface,
)
Hint("A limiter catches the peaks.")
}
Spacer(Modifier.size(12.dp))
Switch(
checked = prefs.boost == NormalizationBoost.LIMITER,
onCheckedChange = { on ->
val boost = if (on) NormalizationBoost.LIMITER else NormalizationBoost.HEADROOM
onChange(prefs.copy(boost = boost))
},
)
}
}
@Composable
private fun Hint(text: String) {
Text(
text = text,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
private fun modeLabel(mode: NormalizationMode): String = when (mode) {
NormalizationMode.OFF -> "Off"
NormalizationMode.AUTO -> "Auto"
NormalizationMode.TRACK -> "Track"
NormalizationMode.ALBUM -> "Album"
}
private fun modeHint(mode: NormalizationMode): String = when (mode) {
NormalizationMode.OFF -> "Tracks play at their mastered volume."
NormalizationMode.AUTO -> "Album gain for whole albums, track gain otherwise."
NormalizationMode.TRACK -> "Every track at the same loudness."
NormalizationMode.ALBUM -> "Albums keep their own quiet and loud tracks."
}
@@ -0,0 +1,44 @@
package com.fabledsword.minstrel.settings.ui
import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.settings.data.NormalizationPrefs
import com.fabledsword.minstrel.settings.data.NormalizationRepository
import dagger.hilt.android.lifecycle.HiltViewModel
import kotlinx.coroutines.CancellationException
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.launch
import timber.log.Timber
import javax.inject.Inject
/**
* Backs the Volume leveling card. Shows the device's copy straight away and
* refreshes it from the server on open, so a change made on the web shows
* here too.
*/
@HiltViewModel
class NormalizationViewModel @Inject constructor(
private val repository: NormalizationRepository,
) : ViewModel() {
val prefs: StateFlow<NormalizationPrefs> = repository.prefs
init {
viewModelScope.launch {
try {
repository.refresh()
} catch (e: CancellationException) {
throw e
} catch (
@Suppress("TooGenericExceptionCaught") e: Throwable,
) {
// Offline: the device's copy stands.
Timber.i(e, "normalization: refresh failed")
}
}
}
fun update(next: NormalizationPrefs) {
viewModelScope.launch { repository.set(next) }
}
}
@@ -57,7 +57,7 @@ class PasswordViewModel @Inject constructor(
try {
repository.changePassword(current = s.current, next = s.next)
internal.update {
PasswordUiState(message = "Password changed.")
PasswordUiState(message = "Password changed. Your other devices have been signed out.")
}
} catch (
@Suppress("TooGenericExceptionCaught") e: Throwable,
@@ -161,6 +161,7 @@ private fun SettingsList(
onToggle = onToggleDiagnostics,
)
}
NormalizationCard()
AppearanceCard(themeMode = themeMode, onPick = onPickTheme)
StorageCard()
AboutCard()
@@ -382,6 +383,14 @@ private fun AboutCard(viewModel: AboutCardViewModel = hiltViewModel()) {
@Composable
private fun UpdateControls(state: AboutUiState, viewModel: AboutCardViewModel) {
if (!state.selfUpdates) {
Text(
text = "Debug build: updates install from Android Studio.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
return
}
UpdateCheckLine(result = state.result)
Button(
onClick = viewModel::checkForUpdates,
@@ -29,6 +29,9 @@ private const val POLL_INTERVAL_MS = 24 * 60 * 60 * 1000L
* restart re-shows it, which is acceptable nudging for v1 (matches
* Flutter). Server 404 / network errors stay silent. Constructed at
* launch via the construct-the-singleton trick in `MinstrelApplication`.
*
* A debug build never polls: it is signed with a local debug key, so the
* server's release-signed APK could never install over it (#5103).
*/
@Singleton
class UpdateBannerController @Inject constructor(
@@ -45,6 +48,10 @@ class UpdateBannerController @Inject constructor(
}.stateIn(scope, SharingStarted.Eagerly, null)
init {
if (!BuildConfig.DEBUG) startPolling()
}
private fun startPolling() {
scope.launch {
while (true) {
runOnce()
@@ -17,8 +17,12 @@
hostnames rather than CIDR ranges, and both sets of hosts above are unknowable
until runtime. So a permissive base-config is an honest description of our
situation — the gain over the manifest attribute is that the reasoning now
lives somewhere, and there is one place to tighten if a future settings screen
can distinguish a LAN server from a WAN one.
lives somewhere.
The LAN/WAN line this file cannot draw is drawn in code instead:
api/CleartextGuard.kt refuses plain http:// to the Minstrel server whenever
the connection lands on a public address (family baseline #5105, practice
13). LAN servers and UPnP speakers are unaffected.
Worth stating because it looks worse than it is: this is NOT a tamper risk for
the in-app updater. An APK altered in transit and re-signed is rejected by the
@@ -1,6 +1,7 @@
package com.fabledsword.minstrel.api
import com.fabledsword.minstrel.auth.AuthStore
import com.fabledsword.minstrel.auth.FakeSessionVault
import com.fabledsword.minstrel.cache.db.dao.AuthSessionDao
import io.mockk.coEvery
import io.mockk.every
@@ -43,7 +44,7 @@ class AuthCookieInterceptorTest {
coEvery { setSessionCookie(any()) } returns Unit
coEvery { setBaseUrl(any()) } returns Unit
}
authStore = AuthStore(dao, TestScope(UnconfinedTestDispatcher()))
authStore = AuthStore(dao, FakeSessionVault(), TestScope(UnconfinedTestDispatcher()))
// BaseUrlInterceptor rewrites placeholder.invalid → mock server.
// AuthCookieInterceptor scopes its attach + clear behavior to
@@ -1,6 +1,7 @@
package com.fabledsword.minstrel.api
import com.fabledsword.minstrel.auth.AuthStore
import com.fabledsword.minstrel.auth.FakeSessionVault
import com.fabledsword.minstrel.cache.db.dao.AuthSessionDao
import io.mockk.coEvery
import io.mockk.every
@@ -38,7 +39,7 @@ class BaseUrlInterceptorTest {
coEvery { setSessionCookie(any()) } returns Unit
coEvery { setBaseUrl(any()) } returns Unit
}
authStore = AuthStore(dao, TestScope(UnconfinedTestDispatcher()))
authStore = AuthStore(dao, FakeSessionVault(), TestScope(UnconfinedTestDispatcher()))
}
@AfterEach
@@ -0,0 +1,89 @@
package com.fabledsword.minstrel.api
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.mockwebserver.MockResponse
import okhttp3.mockwebserver.MockWebServer
import org.junit.jupiter.api.AfterEach
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.assertThrows
import java.net.InetAddress
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertTrue
/** Plain http:// to the Minstrel server only on a private address (#5105, practice 13). */
class CleartextGuardTest {
private lateinit var server: MockWebServer
@BeforeEach
fun setup() {
server = MockWebServer().apply { start() }
}
@AfterEach
fun teardown() {
server.shutdown()
}
private fun ip(s: String) = InetAddress.getByName(s)
@Test
fun `home network and overlay addresses may use plain http`() {
for (a in listOf(
"127.0.0.1", "10.1.2.3", "172.16.0.9", "172.31.255.1", "192.168.1.20",
"169.254.3.4", "100.64.0.1", "100.127.255.254", "::1", "fd12:3456::1", "fe80::1",
)) {
assertTrue(CleartextPolicy.allows(ip(a)), "$a should be allowed")
}
}
@Test
fun `public addresses may not`() {
for (a in listOf(
"8.8.8.8", "172.32.0.1", "100.128.0.1", "100.63.255.255", "203.0.113.5",
"2001:db8::1", "2606:4700::1111",
)) {
assertFalse(CleartextPolicy.allows(ip(a)), "$a should be refused")
}
}
private fun client(allows: Boolean) = OkHttpClient.Builder()
.addNetworkInterceptor(CleartextGuardInterceptor { allows })
.build()
private fun serverRequest() = Request.Builder()
.url(server.url("/api/auth/login"))
.tag(MinstrelServerRequest::class.java, MinstrelServerRequest)
.build()
@Test
fun `a refused server request sends nothing`() {
server.enqueue(MockResponse().setResponseCode(200))
assertThrows<CleartextToPublicHostException> {
client(allows = false).newCall(serverRequest()).execute()
}
assertEquals(0, server.requestCount, "the request reached the server")
}
@Test
fun `an allowed server request goes through`() {
server.enqueue(MockResponse().setResponseCode(200))
client(allows = true).newCall(serverRequest()).execute().use { assertEquals(200, it.code) }
assertEquals(1, server.requestCount)
}
@Test
fun `requests not bound for the Minstrel server are left alone`() {
server.enqueue(MockResponse().setResponseCode(200))
client(allows = false).newCall(Request.Builder().url(server.url("/art.jpg")).build())
.execute().use { assertEquals(200, it.code) }
}
@Test
fun `the refusal has its own message`() {
val msg = ErrorCopy.fromThrowable(CleartextToPublicHostException("music.example.com"))
assertTrue(msg.contains("https://"), msg)
}
}
@@ -0,0 +1,60 @@
package com.fabledsword.minstrel.api
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.ResponseBody.Companion.toResponseBody
import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Test
import retrofit2.HttpException
import retrofit2.Response
import java.io.IOException
class ErrorCopyTest {
private fun httpError(status: Int, body: String): HttpException =
HttpException(
Response.error<Unit>(status, body.toResponseBody("application/json".toMediaType())),
)
@Test
fun libraryNotWritableAppendsTheServerDetail() {
val detail = "Minstrel runs as uid 1000, gid 1000 and cannot delete from /music/A " +
"(read-only file system). The library mount must be writable by that user. " +
"Nothing was deleted."
val e = httpError(409, """{"error":{"code":"library_not_writable","message":"$detail"}}""")
assertEquals(
"${ErrorCopy.messageFor("library_not_writable")} $detail",
ErrorCopy.fromThrowable(e),
)
}
@Test
fun detailCodeWithoutAMessageShowsTheCopyAlone() {
val e = httpError(409, """{"error":{"code":"library_not_writable","message":""}}""")
assertEquals(ErrorCopy.messageFor("library_not_writable"), ErrorCopy.fromThrowable(e))
}
// Server messages are usually internal detail; appending them for every
// code would leak driver errors into snackbars. This pins the scope.
@Test
fun otherCodesNeverCarryTheServerMessage() {
val e = httpError(404, """{"error":{"code":"track_not_found","message":"pgx: no rows"}}""")
assertEquals(ErrorCopy.messageFor("track_not_found"), ErrorCopy.fromThrowable(e))
}
@Test
fun anUnparseableBodyFallsBackToUnknown() {
val e = httpError(500, "not json")
assertEquals(ErrorCopy.messageFor("unknown"), ErrorCopy.fromThrowable(e))
}
@Test
fun transportFailureMapsToConnectionRefused() {
assertEquals(
ErrorCopy.messageFor("connection_refused"),
ErrorCopy.fromThrowable(IOException("refused")),
)
}
}
@@ -0,0 +1,111 @@
package com.fabledsword.minstrel.auth
import com.fabledsword.minstrel.cache.db.dao.AuthSessionDao
import com.fabledsword.minstrel.cache.db.entities.AuthSessionEntity
import io.mockk.coEvery
import io.mockk.coVerify
import io.mockk.every
import io.mockk.mockk
import kotlinx.coroutines.ExperimentalCoroutinesApi
import kotlinx.coroutines.flow.flowOf
import kotlinx.coroutines.test.StandardTestDispatcher
import kotlinx.coroutines.test.TestScope
import kotlinx.coroutines.test.UnconfinedTestDispatcher
import kotlinx.coroutines.test.advanceUntilIdle
import kotlinx.coroutines.test.runTest
import org.junit.jupiter.api.Test
import kotlin.test.assertEquals
import kotlin.test.assertNull
/**
* The session cookie moved out of the Room row into a Keystore-backed vault
* (M462 #4985). What matters is that nobody is signed out by it: an install
* upgrading with a cookie in the row keeps it, a device whose Keystore will
* not work keeps the old storage, and a sign-in or 401 that lands while the
* one-time move runs is never overwritten by the value it read.
*/
@OptIn(ExperimentalCoroutinesApi::class)
class AuthStoreSessionVaultTest {
/** A DAO whose single row holds [legacyCookie] in its sessionCookie column. */
private class RowDao(var legacyCookie: String?) {
val dao: AuthSessionDao = mockk {
every { observe() } returns flowOf(null)
coEvery { get() } answers {
AuthSessionEntity(baseUrl = "http://music.local", sessionCookie = legacyCookie)
}
coEvery { upsert(any()) } answers { legacyCookie = firstArg<AuthSessionEntity>().sessionCookie }
coEvery { setSessionCookie(any()) } answers { legacyCookie = firstArg() }
}
}
@Test
fun `an upgrading install keeps its session, moved out of the row into the vault`() = runTest {
val row = RowDao(legacyCookie = "session=abc")
val vault = FakeSessionVault()
val store = AuthStore(row.dao, vault, TestScope(UnconfinedTestDispatcher(testScheduler)))
store.awaitSessionHydrated()
assertEquals("session=abc", store.sessionCookie.value)
assertEquals("session=abc", vault.stored)
assertNull(row.legacyCookie, "the plain-text copy must be cleared from the row")
}
@Test
fun `a cookie already in the vault is loaded without touching the row`() = runTest {
val row = RowDao(legacyCookie = null)
val vault = FakeSessionVault(stored = "session=xyz")
val store = AuthStore(row.dao, vault, TestScope(UnconfinedTestDispatcher(testScheduler)))
store.awaitSessionHydrated()
assertEquals("session=xyz", store.sessionCookie.value)
assertEquals(0, vault.writes)
coVerify(exactly = 0) { row.dao.setSessionCookie(any()) }
}
@Test
fun `when the Keystore will not work the cookie stays in the row instead of being lost`() = runTest {
val row = RowDao(legacyCookie = "session=abc")
val vault = FakeSessionVault(available = false)
val store = AuthStore(row.dao, vault, TestScope(UnconfinedTestDispatcher(testScheduler)))
store.awaitSessionHydrated()
assertEquals("session=abc", store.sessionCookie.value)
assertEquals("session=abc", row.legacyCookie)
store.setSessionCookie("session=new")
assertEquals("session=new", row.legacyCookie, "fallback writes go to the row")
}
@Test
fun `sign-in and sign-out write the vault, and never the row`() = runTest {
val row = RowDao(legacyCookie = null)
val vault = FakeSessionVault()
val store = AuthStore(row.dao, vault, TestScope(UnconfinedTestDispatcher(testScheduler)))
store.awaitSessionHydrated()
store.setSessionCookie("session=new")
assertEquals("session=new", vault.stored)
assertNull(row.legacyCookie)
store.setSessionCookie(null)
assertNull(vault.stored)
assertNull(store.sessionCookie.value)
}
@Test
fun `a sign-out that lands before hydration finishes is not undone by it`() = runTest {
val row = RowDao(legacyCookie = "session=stale")
val vault = FakeSessionVault()
val scope = TestScope(StandardTestDispatcher(testScheduler))
// Hydration is queued but has not run; a 401 clears the session first.
val store = AuthStore(row.dao, vault, scope)
store.setSessionCookie(null)
scope.advanceUntilIdle()
assertNull(store.sessionCookie.value, "hydration must not resurrect the stale cookie")
assertNull(vault.stored)
}
}
@@ -0,0 +1,23 @@
package com.fabledsword.minstrel.auth
/**
* In-memory [SessionVault] for JVM tests: the Android Keystore has no JVM
* implementation. [available] = false stands in for a device whose Keystore
* refuses to work, so callers' fallback paths can be exercised.
*/
class FakeSessionVault(
var stored: String? = null,
var available: Boolean = true,
) : SessionVault {
var writes = 0
private set
override fun read(): String? = if (available) stored else null
override fun write(value: String?): Boolean {
if (!available) return false
writes++
stored = value
return true
}
}
@@ -0,0 +1,48 @@
package com.fabledsword.minstrel.auth
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.assertThrows
import java.util.Base64
import javax.crypto.KeyGenerator
import javax.crypto.SecretKey
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertNotEquals
/**
* The sealing half of the session vault, with an ordinary JVM AES key in
* place of the Keystore one (the Keystore has no JVM implementation).
*/
class SealedBoxTest {
private fun newKey(): SecretKey = KeyGenerator.getInstance("AES").apply { init(256) }.generateKey()
@Test
fun `round-trips a cookie`() {
val key = newKey()
assertEquals("session=abc", SealedBox.open(key, SealedBox.seal(key, "session=abc")))
}
@Test
fun `the stored form does not contain the cookie, and differs every time`() {
val key = newKey()
val first = SealedBox.seal(key, "session=abc")
val second = SealedBox.seal(key, "session=abc")
assertNotEquals(first, second, "a fresh IV per seal")
val decoded = String(Base64.getDecoder().decode(first), Charsets.ISO_8859_1)
assertFalse(decoded.contains("session=abc"), "plaintext visible in the sealed value")
}
@Test
fun `a tampered value does not open`() {
val key = newKey()
val bytes = Base64.getDecoder().decode(SealedBox.seal(key, "session=abc"))
bytes[bytes.size - 1] = (bytes[bytes.size - 1].toInt() xor 1).toByte()
assertThrows<Exception> { SealedBox.open(key, Base64.getEncoder().encodeToString(bytes)) }
}
@Test
fun `a value sealed under another key does not open`() {
val sealed = SealedBox.seal(newKey(), "session=abc")
assertThrows<Exception> { SealedBox.open(newKey(), sealed) }
}
}
@@ -1,6 +1,8 @@
package com.fabledsword.minstrel.cache.mutations
import com.fabledsword.minstrel.cache.db.entities.CachedMutationEntity
import com.fabledsword.minstrel.settings.data.NormalizationMode
import com.fabledsword.minstrel.settings.data.NormalizationPrefs
import kotlinx.serialization.json.Json
import org.junit.jupiter.api.Test
import kotlin.test.assertEquals
@@ -129,4 +131,26 @@ class SupersededToggleIdsTest {
fun `an empty queue collapses nothing`() {
assertTrue(supersededToggleIds(emptyList(), json).isEmpty())
}
private fun normalizationRow(id: Long, mode: NormalizationMode) = CachedMutationEntity(
id = id,
kind = MutationKind.NORMALIZATION_SET,
payload = json.encodeToString(
NormalizationPrefs.serializer(),
NormalizationPrefs.DEFAULT.copy(mode = mode),
),
)
// There is one preference per user, so any two queued changes to it
// collapse, and only the newest is sent (#4998).
@Test
fun `only the newest queued normalization change survives`() {
val rows = listOf(
normalizationRow(1, NormalizationMode.TRACK),
snoozeRow(2, "mb-a", desiredSnoozed = true),
normalizationRow(3, NormalizationMode.OFF),
normalizationRow(4, NormalizationMode.ALBUM),
)
assertEquals(setOf(1L, 3L), supersededToggleIds(rows, json))
}
}
@@ -0,0 +1,111 @@
package com.fabledsword.minstrel.player.gain
import com.fabledsword.minstrel.settings.data.NormalizationBoost
import com.fabledsword.minstrel.settings.data.NormalizationMode
import com.fabledsword.minstrel.settings.data.NormalizationPrefs
import org.junit.jupiter.api.Test
import kotlin.math.log10
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertTrue
/** Mirrors web/src/lib/player/gain.test.ts: both players must level alike. */
class GainMathTest {
private fun prefs(
mode: NormalizationMode = NormalizationMode.AUTO,
target: Int = -18,
boost: NormalizationBoost = NormalizationBoost.HEADROOM,
) = NormalizationPrefs(mode = mode, targetLufs = target, boost = boost)
private val g = ReplayGain(trackGain = -6f, trackPeak = 1f, albumGain = -4f, albumPeak = 1f)
@Test
fun `off and unmeasured tracks play as mastered`() {
assertEquals(0f, GainMath.gainDb(prefs(mode = NormalizationMode.OFF), g, false))
assertEquals(0f, GainMath.gainDb(prefs(), null, false))
assertEquals(0f, GainMath.gainDb(prefs(), ReplayGain.NONE, false))
}
@Test
fun `mode picks the gain and auto follows album play`() {
assertEquals(-6f, GainMath.gainDb(prefs(mode = NormalizationMode.TRACK), g, true))
assertEquals(-4f, GainMath.gainDb(prefs(mode = NormalizationMode.ALBUM), g, false))
assertEquals(-4f, GainMath.gainDb(prefs(), g, true))
assertEquals(-6f, GainMath.gainDb(prefs(), g, false))
}
@Test
fun `album gain falls back to track gain until the album is measured`() {
val partial = ReplayGain(trackGain = -6f, trackPeak = 1f, albumGain = null, albumPeak = null)
assertEquals(-6f, GainMath.gainDb(prefs(mode = NormalizationMode.ALBUM), partial, true))
}
@Test
fun `a louder target raises every gain by the difference`() {
assertEquals(-2f, GainMath.gainDb(prefs(mode = NormalizationMode.TRACK, target = -14), g, false))
}
@Test
fun `headroom stops a boost 1 dB under the true peak and the limiter lets it through`() {
val quiet = ReplayGain(trackGain = 8f, trackPeak = 0.5f, albumGain = null, albumPeak = null)
val headroom = GainMath.gainDb(prefs(mode = NormalizationMode.TRACK), quiet, false)
assertEquals(-1f - 20f * log10(0.5f), headroom, 1e-4f)
val limited = GainMath.gainDb(
prefs(mode = NormalizationMode.TRACK, boost = NormalizationBoost.LIMITER),
quiet,
false,
)
assertEquals(8f, limited)
}
@Test
fun `no boost passes the cap`() {
val silent = ReplayGain(trackGain = 30f, trackPeak = 0.001f, albumGain = null, albumPeak = null)
val db = GainMath.gainDb(
prefs(mode = NormalizationMode.TRACK, boost = NormalizationBoost.LIMITER),
silent,
false,
)
assertEquals(GainMath.MAX_BOOST_DB, db)
}
@Test
fun `cuts are never limited by the peak`() {
val hot = ReplayGain(trackGain = -9f, trackPeak = 1.4f, albumGain = null, albumPeak = null)
assertEquals(-9f, GainMath.gainDb(prefs(mode = NormalizationMode.TRACK), hot, false))
}
private fun pos(album: String, track: Int?, disc: Int? = 1) = AlbumPosition(album, disc, track)
@Test
fun `in-order neighbours from the same album are album play`() {
assertTrue(GainMath.playingAsAlbum(null, pos("x", 1), pos("x", 2)))
assertTrue(GainMath.playingAsAlbum(pos("x", 1), pos("x", 2), pos("x", 3)))
assertTrue(GainMath.playingAsAlbum(pos("x", 2), pos("x", 3), null))
}
@Test
fun `a shuffled album, a mix and a lone track are not album play`() {
assertFalse(GainMath.playingAsAlbum(null, pos("x", 3), pos("x", 1)))
assertFalse(GainMath.playingAsAlbum(null, pos("x", 1), pos("y", 2)))
assertFalse(GainMath.playingAsAlbum(null, pos("x", 1), null))
}
@Test
fun `disc order counts`() {
assertTrue(GainMath.playingAsAlbum(pos("x", 12, disc = 1), pos("x", 1, disc = 2), null))
}
@Test
fun `a neighbour without a track number is not evidence of order`() {
assertFalse(GainMath.playingAsAlbum(pos("x", null), pos("x", 3), null))
assertFalse(GainMath.playingAsAlbum(null, pos("x", null), pos("x", null)))
}
@Test
fun `dbToLinear`() {
assertEquals(1f, GainMath.dbToLinear(0f))
assertEquals(0.1f, GainMath.dbToLinear(-20f), 1e-6f)
}
}
@@ -0,0 +1,73 @@
package com.fabledsword.minstrel.player.gain
import org.junit.jupiter.api.Test
import java.nio.ByteBuffer
import java.nio.ByteOrder
import kotlin.math.abs
import kotlin.test.assertEquals
import kotlin.test.assertTrue
class GainStageTest {
private fun floats(vararg v: Float): ByteBuffer =
ByteBuffer.allocateDirect(v.size * 4).order(ByteOrder.nativeOrder()).apply {
v.forEach { putFloat(it) }
flip()
}
private fun shorts(vararg v: Short): ByteBuffer =
ByteBuffer.allocateDirect(v.size * 2).order(ByteOrder.nativeOrder()).apply {
v.forEach { putShort(it) }
flip()
}
private fun runFloat(stage: GainStage, input: ByteBuffer, target: Float, limiting: Boolean): FloatArray {
val out = ByteBuffer.allocateDirect(input.remaining()).order(ByteOrder.nativeOrder())
stage.process(input, out, channels = 2, isFloat = true, target = target, limiting = limiting)
out.flip()
return FloatArray(out.remaining() / 4) { out.float }
}
@Test
fun `a stream that starts at its gain is scaled from the first sample`() {
val stage = GainStage().apply { reset(sampleRate = 44_100, channels = 2, startGain = 0.5f) }
val out = runFloat(stage, floats(0.8f, -0.4f), target = 0.5f, limiting = false)
assertEquals(0.4f, out[0], 1e-6f)
assertEquals(-0.2f, out[1], 1e-6f)
}
@Test
fun `a changed target ramps rather than steps`() {
val stage = GainStage().apply { reset(sampleRate = 44_100, channels = 2, startGain = 1f) }
val n = 44_100 // one second of stereo frames
val input = floats(*FloatArray(n * 2) { 0.5f })
val out = runFloat(stage, input, target = 0.5f, limiting = false)
assertTrue(out[0] > 0.49f, "first frame barely moved: ${out[0]}")
assertEquals(0.25f, out.last(), 1e-3f)
}
@Test
fun `the limiter holds a boosted peak at -1 dBFS`() {
val stage = GainStage().apply { reset(sampleRate = 44_100, channels = 2, startGain = 4f) }
val out = runFloat(stage, floats(0.9f, -0.9f, 0.1f, 0.1f), target = 4f, limiting = true)
out.forEach { assertTrue(abs(it) <= GainStage.LIMIT_CEILING + 1e-6f, "sample $it over the ceiling") }
}
@Test
fun `without the limiter a boost is clamped to full scale, 16-bit included`() {
val stage = GainStage().apply { reset(sampleRate = 44_100, channels = 2, startGain = 4f) }
val input = shorts(20_000, -20_000)
val out = ByteBuffer.allocateDirect(4).order(ByteOrder.nativeOrder())
stage.process(input, out, channels = 2, isFloat = false, target = 4f, limiting = false)
out.flip()
assertEquals(Short.MAX_VALUE, out.short)
assertEquals((-Short.MAX_VALUE).toShort(), out.short)
}
@Test
fun `unity is a pass-through`() {
val stage = GainStage().apply { reset(sampleRate = 44_100, channels = 2, startGain = 1f) }
assertTrue(stage.isUnity(target = 1f, limiting = false))
assertTrue(!stage.isUnity(target = 1f, limiting = true))
}
}
@@ -0,0 +1,74 @@
package com.fabledsword.minstrel.player.gain
import com.fabledsword.minstrel.api.endpoints.ReplayGainApi
import com.fabledsword.minstrel.cache.db.dao.CachedReplayGain
import com.fabledsword.minstrel.cache.db.dao.CachedTrackDao
import com.fabledsword.minstrel.connectivity.ServerHealth
import com.fabledsword.minstrel.models.wire.ReplayGainResponseWire
import com.fabledsword.minstrel.models.wire.ReplayGainWire
import io.mockk.coEvery
import io.mockk.coVerify
import io.mockk.mockk
import kotlinx.coroutines.test.TestScope
import kotlinx.coroutines.test.runTest
import org.junit.jupiter.api.Test
import java.io.IOException
import kotlin.test.assertEquals
import kotlin.test.assertNull
/** The fallback ladder: the library cache, then the server, then nothing. */
class ReplayGainStoreTest {
private val dao: CachedTrackDao = mockk()
private val api: ReplayGainApi = mockk()
private var health = ServerHealth.Healthy
private fun store() = ReplayGainStore(
scope = TestScope(),
trackDao = dao,
api = api,
serverHealth = { health },
clock = { 0L },
)
@Test
fun `cached gains are used and only the rest are asked of the server`() = runTest {
coEvery { dao.replayGains(listOf("a", "b", "c")) } returns listOf(
CachedReplayGain("a", trackGain = -6f, trackPeak = 1f, albumGain = -4f, albumPeak = 1f),
// Cached but unmeasured at the last sync: the server may know more.
CachedReplayGain("b", trackGain = null, trackPeak = null, albumGain = null, albumPeak = null),
)
coEvery { api.getReplayGain("b,c") } returns ReplayGainResponseWire(
items = mapOf("b" to ReplayGainWire(trackGain = -3f, trackPeak = 0.9f)),
)
val s = store()
s.load(listOf("a", "b", "c"))
assertEquals(-4f, s.get("a")?.albumGain)
assertEquals(-3f, s.get("b")?.trackGain)
assertNull(s.get("c"), "a track the server has not measured stays unknown")
}
@Test
fun `offline, the cache is all there is`() = runTest {
health = ServerHealth.Offline
coEvery { dao.replayGains(any()) } returns emptyList()
val s = store()
s.load(listOf("a"))
assertNull(s.get("a"))
coVerify(exactly = 0) { api.getReplayGain(any()) }
}
@Test
fun `a failed lookup leaves the track unknown, to be asked again`() = runTest {
coEvery { dao.replayGains(any()) } returns emptyList()
coEvery { api.getReplayGain(any()) } throws IOException("reset")
val s = store()
s.load(listOf("a"))
assertNull(s.get("a"))
}
}
@@ -0,0 +1,43 @@
package com.fabledsword.minstrel.player.output
import com.fabledsword.minstrel.models.TrackRef
import org.junit.jupiter.api.Test
import kotlin.test.assertEquals
/**
* The Sonos queue plays straight through, so a track's album play is judged
* from its neighbours in the list sent (M464 #5002).
*/
class SonosAlbumPlayTest {
private fun t(album: String, n: Int?, disc: Int? = null) =
TrackRef(id = "$album-$disc-$n", title = "", albumId = album, artistId = "", trackNumber = n, discNumber = disc)
private fun asAlbum(queue: List<TrackRef>) = queue.indices.map { SonosQueueLoader.playingAsAlbum(queue, it) }
@Test
fun `an album in order plays as an album throughout`() {
assertEquals(listOf(true, true, true), asAlbum(listOf(t("a", 1), t("a", 2), t("a", 3))))
}
@Test
fun `a mix of albums plays as singles`() {
assertEquals(listOf(false, false, false), asAlbum(listOf(t("a", 1), t("b", 1), t("c", 4))))
}
@Test
fun `album order runs across discs`() {
assertEquals(listOf(true, true), asAlbum(listOf(t("a", 12, disc = 1), t("a", 1, disc = 2))))
}
@Test
fun `shuffled tracks of one album are not album play`() {
assertEquals(listOf(false, false), asAlbum(listOf(t("a", 3), t("a", 1))))
}
@Test
fun `unnumbered tracks and missing albums are never album play`() {
assertEquals(listOf(false, false), asAlbum(listOf(t("a", null), t("a", null))))
assertEquals(listOf(false, false), asAlbum(listOf(t("", 1), t("", 2))))
}
}
@@ -0,0 +1,90 @@
package com.fabledsword.minstrel.settings.data
import com.fabledsword.minstrel.api.endpoints.MeApi
import com.fabledsword.minstrel.auth.AuthStore
import com.fabledsword.minstrel.cache.db.dao.CachedMutationDao
import com.fabledsword.minstrel.cache.mutations.MutationKind
import com.fabledsword.minstrel.cache.mutations.MutationQueue
import io.mockk.coEvery
import io.mockk.coVerify
import io.mockk.every
import io.mockk.mockk
import io.mockk.verify
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.test.runTest
import org.junit.jupiter.api.Test
import retrofit2.Retrofit
import java.io.IOException
/**
* The normalization preference is written offline-first (rule 100): the
* device shows the change at once, and a change the server has not seen
* yet is never overwritten or reordered.
*/
class NormalizationRepositoryTest {
private val api: MeApi = mockk()
private val authStore: AuthStore = mockk(relaxed = true) {
every { normalization } returns MutableStateFlow(NormalizationPrefs.DEFAULT)
}
private val queue: MutationQueue = mockk(relaxed = true)
private val dao: CachedMutationDao = mockk()
private val retrofit: Retrofit = mockk {
every { create(MeApi::class.java) } returns api
}
private val repo = NormalizationRepository(retrofit, authStore, queue, dao)
private val album = NormalizationPrefs.DEFAULT.copy(mode = NormalizationMode.ALBUM)
@Test
fun `a change is shown at once and sent`() = runTest {
coEvery { dao.hasPending(MutationKind.NORMALIZATION_SET) } returns false
coEvery { api.putNormalization(album) } returns album
repo.set(album)
verify { authStore.setNormalization(album) }
coVerify { api.putNormalization(album) }
coVerify(exactly = 0) { queue.enqueueNormalizationSet(any()) }
}
@Test
fun `a change that cannot reach the server is queued`() = runTest {
coEvery { dao.hasPending(MutationKind.NORMALIZATION_SET) } returns false
coEvery { api.putNormalization(album) } throws IOException("offline")
repo.set(album)
verify { authStore.setNormalization(album) }
coVerify { queue.enqueueNormalizationSet(album) }
}
@Test
fun `a change behind a queued one queues too, so the older one cannot land last`() = runTest {
coEvery { dao.hasPending(MutationKind.NORMALIZATION_SET) } returns true
repo.set(album)
coVerify(exactly = 0) { api.putNormalization(any()) }
coVerify { queue.enqueueNormalizationSet(album) }
}
@Test
fun `refresh takes the server value`() = runTest {
coEvery { api.getNormalization() } returns album
coEvery { dao.hasPending(MutationKind.NORMALIZATION_SET) } returns false
repo.refresh()
verify { authStore.setNormalization(album) }
}
@Test
fun `refresh leaves a queued change on screen`() = runTest {
coEvery { api.getNormalization() } returns NormalizationPrefs.DEFAULT
coEvery { dao.hasPending(MutationKind.NORMALIZATION_SET) } returns true
repo.refresh()
verify(exactly = 0) { authStore.setNormalization(any()) }
}
}
+1 -1
View File
@@ -21,7 +21,7 @@ kotlinx-serialization = "1.7.3"
kotlinx-coroutines = "1.9.0"
kotlinx-datetime = "0.6.1"
kotlinx-serialization-converter = "1.0.0"
media3 = "1.10.1"
media3 = "1.11.0"
mediarouter = "1.7.0"
coil = "3.0.0-rc02"
palette = "1.0.0"
BIN
View File
Binary file not shown.
+29 -55
View File
@@ -12,8 +12,9 @@ git.fabledsword.com/bvandeusen/ci-go:1.26
git.fabledsword.com/bvandeusen/ci-android:36
```
- `ci-go:1.26` — Go server tests (`.gitea/workflows/test-go.yml`), web SPA tests (`.gitea/workflows/test-web.yml`), and the release container build (`release.yml`'s `image-release` job).
- `ci-android:36` — native Kotlin/Compose client: ktlint + detekt + unit tests + debug APK (`.gitea/workflows/android.yml`), and the signed release APK (`release.yml`'s `android-release` job).
- `ci-go:1.26` — the `go`, `integration` and `web` lanes, `release-assets`, and the container build (`image-release`). All CI lives in one workflow, `.gitea/workflows/release.yml`, so every publishing job can depend on every lane (rule 177).
- `golang:1.26-bookworm` (Docker Hub) — the `govulncheck` lane. Deliberately the same image as the Dockerfile's builder stage rather than `ci-go`, so the standard library it checks is the one in the shipped binary. Keep the two in step.
- `ci-android:36` — native Kotlin/Compose client: ktlint + detekt + unit tests + debug APK (the `android` lane), and the signed release APK (`android-release`).
**`ci-flutter` is no longer consumed, and `ci-flutter` can now be retired.**
The M8 rewrite replaced the Flutter client with the native Android app and
@@ -30,9 +31,9 @@ split below. The label is a scheduling handle, not a toolchain assertion.
### From `ci-go:1.26`
- **Go** (1.26 toolchain) — `go vet`, `go test -race`, `go build`, `go mod`.
- **Node + npm** — `npm ci` and `npm test` / `npm run check` in `test-web.yml`.
- **golangci-lint** — lint pass in `test-go.yml`.
- **docker CLI** — bridge-IP discovery of the per-job Postgres service container in `test-go.yml` integration job (via the runner's shared `/var/run/docker.sock`).
- **Node + npm** — `npm ci`, `npm audit`, `npm test` and `npm run check` in the `web` lane.
- **golangci-lint** — lint pass in the `go` lane.
- **docker CLI** — bridge-IP discovery of the per-job Postgres service container in the `integration` lane (via the runner's shared `/var/run/docker.sock`).
- **docker buildx** — release container build + push in `release.yml`.
- **curl** — release-asset polling / upload in `release.yml`.
@@ -45,7 +46,7 @@ split below. The label is a scheduling handle, not a toolchain assertion.
No NDK: the native client has no C/C++ sources (this is why it isn't on
`ci-flutter`).
- **ktlint + detekt** — `./gradlew ktlintCheck` and `./gradlew detekt` in
`android.yml`. Image pins track `android/gradle/libs.versions.toml` so local
the `android` lane. Image pins track `android/gradle/libs.versions.toml` so local
and CI checks agree.
- **git** — `actions/checkout@v4` baseline (and any shell git operations).
- **base64 + curl** — keystore decode + release-asset upload in `release.yml`'s
@@ -58,61 +59,34 @@ None.
## Notes
- **Label/image split.** Workflows keep `runs-on: go-ci` / `runs-on: flutter-ci` as the scheduling label per the [`ci-runners.md`](https://…/FabledRulebook/ci-runners.md) "label = scheduling handle, image = `container.image`" pattern. The labels are intentional handles, not toolchain assertions — which is why the Android jobs still schedule on `flutter-ci` while pulling `ci-android:36`. Switch them to `android-ci` if that runner label is ever registered; nothing breaks either way.
- **Integration-job docker-socket dependency.** `test-go.yml`'s integration job uses the runner's shared docker socket (`/var/run/docker.sock`) to bridge-IP-discover the per-job Postgres service container by name + network intersection — the dev compose's `minstrel-postgres-*` containers are explicitly skipped as belt-and-suspenders. Depends on `act_runner.valid_volumes` whitelisting the socket; if that ever stops auto-mounting, integration tests fail at the `docker inspect` step.
- **Integration-job docker-socket dependency.** The `integration` lane uses the runner's shared docker socket (`/var/run/docker.sock`) to bridge-IP-discover the per-job Postgres service container by name + network intersection — the dev compose's `minstrel-postgres-*` containers are explicitly skipped as belt-and-suspenders. Depends on `act_runner.valid_volumes` whitelisting the socket; if that ever stops auto-mounting, integration tests fail at the `docker inspect` step.
- **Go toolchain pin.** `go.mod` is on `go 1.25.0` because `golang.org/x/crypto v0.51.0` declares 1.25 as its minimum. `ci-go:1.26` satisfies this with headroom. Future `x/crypto` bumps that move the Go floor should be paired with an image-tag bump in this file + the workflows.
- **In-app update channel — `needs:`, not polling.** `release.yml`'s `image-release` job declares `needs: [android-release]`, so on tag pushes the signed APK is guaranteed present before the image build starts — no polling window, no race. (The old cross-workflow polling against `flutter.yml` is gone with that workflow.) On non-tag `main` pushes `android-release` is skipped and `image-release` instead pulls the most recent release's APK and reconstructs its exact `versionName`, so `:latest` never ships without an update channel. It degrades to an empty `client/` — never a wrong version — if no release, asset, or tag commit-count can be resolved.
- **Cache server reachability.** `test-web.yml` does NOT use `cache: 'npm'` on `actions/setup-node` — the Gitea Actions cache server isn't reachable from this runner's container network and `setup-node` was burning ~4m41s on ETIMEDOUT before failing open. With the migration to `ci-go:1.26`, `setup-node` is removed entirely (Node is in the image). The cache concern reappears if a future change re-introduces a network-dependent action.
- **Artifacts — use the mirrored actions, never `actions/{upload,download}-artifact`.**
- **Cache server reachability.** The `web` lane does NOT use `cache: 'npm'` on `actions/setup-node` — the Gitea Actions cache server isn't reachable from this runner's container network and `setup-node` was burning ~4m41s on ETIMEDOUT before failing open. With the migration to `ci-go:1.26`, `setup-node` is removed entirely (Node is in the image). The cache concern reappears if a future change re-introduces a network-dependent action.
- **Artifacts — stock `actions/upload-artifact@v7` and `actions/download-artifact@v8`; never `@v3`.**
```yaml
uses: https://git.fabledsword.com/bvandeusen/upload-artifact@cb8afe72b42edc798abfb8fcb556cf660d894245
uses: https://git.fabledsword.com/bvandeusen/download-artifact@8d4e9521a5f7e5f8b6351f341f719f9f45a92a3a
uses: actions/upload-artifact@v7
uses: actions/download-artifact@v8
```
Upstream's `@v4+` cannot work against this instance and no server-side change
will help: `isGhes()` rejects any hostname that isn't `github.com` /
`*.ghe.com` / `*.localhost` and throws before it opens a connection, so the
server is never asked what it supports. `@v3` is worse — it reports success,
and Gitea then serves artifacts back only through the v4 API
(`content_encoding = application/zip`), so a v3 upload is stored but invisible
to every retrieval path. A green job producing nothing retrievable; that is how
72 unreachable artifacts accumulated on this repo. Scribe issues 2255 / 2270.
Stock works on this forge since the runner moved to gitea/runner 3.x, which
edits the actions' client-side `isGhes()` refusal out of their bundles. Proven
on 2026-09-10 for upload v4–v7 and download v4–v8 (Scribe spike #3843). Until
then this repo pinned SHA mirrors of the Forgejo project's forks, because
upstream threw on the hostname before it opened a connection (Scribe 2255).
Both are pull mirrors of the Forgejo project's forks
(`code.forgejo.org/forgejo/{upload,download}-artifact`, one commit on upstream
disabling that check), mirrored so CI depends on commits we hold and pinned by
SHA because the mirrors auto-sync every 8h — a moved upstream tag would
otherwise silently change what runs.
`@v3` is still broken: it reports success, and Gitea serves artifacts back only
through the v4 API (`content_encoding = application/zip`), so a v3 upload is
stored but invisible to every retrieval path. That is how 72 unreachable
artifacts accumulated on this repo (Scribe 2270).
**Match the pins on `@actions/artifact`, not on the actions' own version
numbers.** The two actions release on unrelated cadences, so equal version
numbers do NOT mean a compatible pair — upload `v5` bundles `@actions/artifact`
^4.0.0 while download `v5` bundles ^2.3.2. The pins above are upload **v5** and
download **v6**, which is the pairing that puts ^4.0.0 on both sides. This
matters because `release.yml` is a producer/consumer pair — `android-release`
uploads `minstrel-apk`, `image-release` downloads it — and a protocol mismatch
across it yields an empty listing rather than an error, exactly the silent
failure this entry exists to prevent.
| tag | `@actions/artifact` | runtime |
|---|---|---|
| upload v4 | ^2.1.1 | node20 |
| **upload v5** ← pinned | **^4.0.0** | node20 |
| download v4 | ^2.1.1 | node20 |
| download v5 | ^2.3.2 | node20 |
| **download v6** ← pinned | **^4.0.0** | node20 |
| download v7 | ^5.0.0 | **node24** |
The only true protocol break in this history was **v3 → v4** (upstream:
"Downloading artifacts that were created from `actions/upload-artifact@v3` and
below are not supported"); v4-and-up are one family. Later majors are mostly
ergonomics and runtime — upload v4 forbids re-uploading a name and caps a job
at 500 artifacts; download v5 made by-ID extraction match by-name.
**Do not jump the download pin to v7.** That major is a runner requirement, not
a feature change: it moves to `runs.using: node24` and upstream states it
"requires a minimum Actions Runner version of 2.327.1 … if you are using
self-hosted runners, ensure they are updated before upgrading." act_runner is
not GitHub's runner and makes no such version claim, so node24 is unverified
here. Everything currently pinned is node20.
**Pairing no longer needs managing.** This entry used to pin upload v5 against
download v6 so both bundled `@actions/artifact` ^4.0.0, warning that a mismatch
across `release.yml`'s producer/consumer pair would list empty. Tested, and not
true on this instance: every download major v4–v8 read the artifacts of every
upload major v4–v7, by name and by pattern (CI-runner run 6312). The only real
protocol break is v3 → v4. node24 is no longer a concern either — every
CI-runner image carries Node 24 and the runner runs actions with the image's
`node`.
Upload steps set `if-no-files-found: error` rather than the default `warn`, so
an upload that matches nothing fails its own job instead of failing the
+77 -2
View File
@@ -2,7 +2,8 @@
#
# Derives the three values a build is stamped with, and the tag that names it.
#
# name=YYYY.MM.DD.HHMM label for people, from the COMMIT's timestamp
# name=YYYY.MM.DD.HHMM label for people, from the timestamp of the newest
# commit that CHANGED SOMETHING SHIPPED (see SHIPPED)
# code=<int> ordering key, minutes since 2020-01-01 at BUILD time
# tag=v<name> what a release of this commit must be called
#
@@ -33,9 +34,83 @@ readonly REF="${1:-HEAD}"
# Both clocks are overridable so a test can pin them. Nothing but tests should
# set these — the defaults are the real derivation.
# The paths that do NOT ship, in either artifact. Everything else counts.
#
# A DENYLIST, and the direction is the whole point. As an allowlist, the list
# has to be updated by whoever adds a directory and nothing fails if they
# don't — so the failure mode is a changed artifact keeping its old version,
# silently, on a green run. That is a build lying about what it is. Inverted,
# new content counts by default and the only way to wrongly EXCLUDE something
# is to name it here deliberately.
#
# The two error directions are not symmetric, which is why this is not taste:
# wrongly excluded → changed artifact, unchanged version. A silent lie.
# wrongly included → version moves when nothing shipped. Cosmetic noise in
# a string nobody sorts.
#
# THIS REPO SHIPS TWO ARTIFACTS FROM ONE DERIVATION, and that is why the list
# is shorter than it looks like it should be. The server image ships cmd/,
# internal/, shared/, web/, config.example.yaml and client/; the APK ships
# android/. Neither ships the other's sources — but excluding android/ here
# would stop an Android-only commit from moving the APK's OWN version, which
# is the dangerous direction. So this is the union: exclude only what ships in
# NEITHER, and accept that an Android commit also nudges the server's reported
# version. Over-inclusion across the two, which is the harmless direction.
#
# The family's other repos (roundtable / roundtable-android) each keep a
# tighter list because they are separate repos with one artifact apiece. Do
# not copy theirs onto this one.
readonly SHIPPED=(
.
':!.gitea' # CI workflows — including this script's own caller
':!ci' # CI scripts — including this script
':!docs'
':!tools' # asset/font generators; their OUTPUT ships, they do not
':!deploy' # test-database bootstrap SQL
':!bin' # local `make build` output
':!*.md'
':!Makefile'
':!docker-compose.yml'
':!.env.example'
':!.gitignore'
':!.dockerignore'
':!renovate.json'
':!.golangci.yml'
# TESTS DO NOT SHIP, so they must not re-version an artifact.
#
# Named as globs rather than a directory because this repo has no tests/
# tree to exclude: Go tests sit inline beside the code they cover, and the
# web suite sits beside its modules. `go build` drops *_test.go outright and
# the Vite build never imports a .test.ts, so neither reaches an artifact.
#
# A commit touching a test AND its source still moves the version — the
# source path matches on its own. Only a test-ONLY commit is inert, which is
# the whole intent.
#
# Patterns match what exists today and nothing speculative: there are no
# .spec.* files, no __tests__/ directories and no androidTest/ tree. If any
# appear they will re-version until named here, which is the harmless
# direction and the reason this list is a denylist.
':!*_test.go' # 158 files, inline beside the code
':!*.test.ts' # 114 files
':!*.test.js'
':!android/app/src/test' # JVM unit tests; no androidTest tree exists
':!web/vitest.config.ts' # test-harness config, not build config
':!web/vitest.setup.ts'
)
commit_epoch="${MINSTREL_COMMIT_EPOCH:-}"
if [ -z "${commit_epoch}" ]; then
commit_epoch="$(git log --format=%ct -1 "${REF}")"
commit_epoch="$(git log --format=%ct -1 "${REF}" -- "${SHIPPED[@]}")"
# Loudly, on purpose. A silent fallback here is the landmine this whole
# script exists to avoid: a plausible-looking version that is quietly wrong,
# on a green run. Realistically this means a shallow clone (no commit in
# range touches the shipped set) rather than a repo of pure CI config.
if [ -z "${commit_epoch}" ]; then
echo "version.sh: no commit under '${REF}' touches the shipped file set — shallow clone? (needs fetch-depth: 0)" >&2
exit 1
fi
fi
now_epoch="${MINSTREL_NOW_EPOCH:-$(date -u +%s)}"
+8 -13
View File
@@ -55,12 +55,14 @@ func runAdmin(args []string) error {
}
}
// adminResetPassword resets a user's credentials. It updates BOTH
// password_hash (bcrypt, for /api/auth/login) and subsonic_password
// (plaintext, required for Subsonic t+s token verification) so neither
// auth path is left stale. Recovers a locked-out operator when the
// bootstrap password was missed or the DB volume was recreated (Fable
// #321) without DB surgery.
// adminResetPassword resets a user's login password (password_hash). It
// recovers a locked-out operator when the bootstrap password was missed or
// the DB volume was recreated (Fable #321) without DB surgery.
//
// It deliberately leaves subsonic_password alone. That column is stored in
// plain text, and it used to receive the new login password here, so any
// account recovered this way had its login password readable in the database
// (#5026). The Subsonic password is generated separately in Settings.
func adminResetPassword(args []string) error {
fs := flag.NewFlagSet("admin reset-password", flag.ContinueOnError)
configPath := fs.String("config", os.Getenv("MINSTREL_CONFIG"), "path to YAML config file")
@@ -112,13 +114,6 @@ func adminResetPassword(args []string) error {
}); err != nil {
return fmt.Errorf("update password_hash: %w", err)
}
sp := pw
if err := q.SetSubsonicPassword(ctx, dbq.SetSubsonicPasswordParams{
ID: user.ID,
SubsonicPassword: &sp,
}); err != nil {
return fmt.Errorf("update subsonic_password: %w", err)
}
if generated {
fmt.Printf("minstrel: password for %q reset.\nNew password: %s\n", *username, pw)
+58 -1
View File
@@ -14,6 +14,7 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/acoustid"
"git.fabledsword.com/bvandeusen/minstrel/internal/config"
"git.fabledsword.com/bvandeusen/minstrel/internal/coplay"
"git.fabledsword.com/bvandeusen/minstrel/internal/coverart"
@@ -122,7 +123,31 @@ func run() error {
}
defer pool.Close()
scanner := library.New(pool, logger, cfg.Library.ScanPaths)
// Fingerprinting settings (M400 #3913): one instance, shared by the scanner,
// the fingerprint backfill, the duplicate sweep and the admin API, so a save
// reaches all of them without a restart. A load failure is logged, not fatal:
// the service falls back to the shipped defaults.
fpSettings, fpErr := library.NewFingerprintSettingsService(ctx, pool)
if fpErr != nil {
logger.Warn("fingerprint settings: using defaults", "err", fpErr)
}
scanner := library.New(pool, logger, cfg.Library.ScanPaths, fpSettings)
// Loudness analysis settings (M464 #4995): shared by the loudness backfill
// and the admin API, and falls back to the defaults like the above.
loudSettings, loudErr := library.NewLoudnessSettingsService(ctx, pool)
if loudErr != nil {
logger.Warn("loudness settings: using defaults", "err", loudErr)
}
// AcoustID lookup settings (M401): shared by the lookup worker and the
// admin API. A load failure leaves the lookup off.
acoustIDSettings, acoustIDErr := library.NewAcoustIDSettingsService(ctx, pool)
if acoustIDErr != nil {
logger.Warn("acoustid settings: using defaults (lookup off)", "err", acoustIDErr)
}
acoustIDLookup := library.NewAcoustIDLookupWorker(pool, logger.With("component", "acoustid_lookup"),
acoustIDSettings, acoustid.New(acoustid.DefaultBaseURL))
contact := cfg.Library.ContactEmail
if contact == "" {
@@ -214,6 +239,27 @@ func run() error {
// SQL, no external calls; empty on single-user servers.
go coplay.NewWorker(pool, logger.With("component", "coplay")).Run(ctx)
// Fingerprint backfill (M400 #3908): fingerprints the tracks the scan never
// will — everything imported before fingerprinting existed, and rows derived
// by an older method. A worker of its own rather than a scan stage; see
// internal/library/fingerprint_backfill.go for why.
go library.NewFingerprintBackfillWorker(pool, logger.With("component", "fingerprint_backfill"), fpSettings).Run(ctx)
// Loudness backfill (M464 #4995): measures every track's loudness for
// normalization, new tracks included; the scan only drops a changed file's
// measurement. See internal/library/loudness_backfill.go.
go library.NewLoudnessBackfillWorker(pool, logger.With("component", "loudness_backfill"), loudSettings).Run(ctx)
// AcoustID lookup (M401): fills the recording MBIDs untagged tracks lack,
// so the similarity arms can see them. Idle until the operator adds a key
// and switches it on. See internal/library/acoustid_lookup.go.
go acoustIDLookup.Run(ctx)
// Duplicate sweep (M400 #3910): proposes groups of tracks holding one
// recording, from the fingerprints above. Sweeps only when fingerprints have
// changed since the last sweep.
go library.NewDuplicateSweepWorker(pool, logger.With("component", "duplicate_sweep"), fpSettings).Run(ctx)
// Start the tag-enrichment worker (#1490). Reconciles the compiled-in
// tag providers with tag_provider_settings, bumps the sources version if
// the provider set changed (re-opening settled rows), then drains tracks
@@ -357,11 +403,22 @@ func run() error {
srv.PlaylistScheduler = playlistScheduler
srv.RecSettings = recSettings
srv.TagSettings = tagSettings
srv.FingerprintSettings = fpSettings
srv.LoudnessSettings = loudSettings
srv.AcoustIDLookup = acoustIDLookup
// The sweeper above holds this same instance, so a save from the admin
// card changes what it does on its next tick (#3936).
srv.ReacqSettings = reacqSettings
srv.StreamSecret = cfg.StreamSecret
httpServer := &http.Server{
Addr: cfg.Server.Address,
Handler: srv.Router(),
ReadHeaderTimeout: 10 * time.Second,
// Closes keep-alive connections nobody is using. Deliberately no
// ReadTimeout or WriteTimeout: either would cut off audio streams and
// the SSE event stream. Request bodies get their own deadline in the
// server's limitRequestBody middleware instead.
IdleTimeout: 120 * time.Second,
}
errCh := make(chan error, 1)
+1 -1
View File
@@ -8,7 +8,7 @@ version: "3.9"
# tests at it:
# make test-integration
# (CI runs the same suite against its own ephemeral Postgres service —
# see .gitea/workflows/test-go.yml.)
# see the `integration` job in .gitea/workflows/release.yml.)
#
# Full stack (server + db):
# docker compose up --build
+92
View File
@@ -0,0 +1,92 @@
# Hosting Minstrel
Minstrel serves plain HTTP on port 4533 and never terminates TLS itself. On a
home network or a VPN you trust, that is all you need. Once the server is
reachable from the internet, put it behind a reverse proxy that speaks HTTPS,
and tell Minstrel about that proxy. This page covers both.
## On a LAN or VPN
Publish the port to the network and use `http://<host>:4533`:
```yaml
ports: ['4533:4533']
```
If nothing sits in front of Minstrel, set **Admin → Integrations → Client IP
detection** to 0. It defaults to 1, which assumes one proxy (see below for
why that matters).
## On the internet
Three things, all required:
1. **Don't publish 4533 to the world.** Bind it to the loopback interface so
only the proxy on the same host can reach it:
```yaml
ports: ['127.0.0.1:4533:4533']
```
If the proxy runs in another container on the same Docker network, drop
`ports:` entirely and point the proxy at `minstrel:4533`.
2. **Terminate HTTPS at a reverse proxy.** Any proxy works. With Caddy, which
gets and renews certificates by itself:
```
music.example.com {
reverse_proxy 127.0.0.1:4533
}
```
Two settings matter for every proxy:
- **Don't buffer responses.** Live updates use Server-Sent Events and audio
is streamed; a buffering proxy holds both back. Caddy streams by default.
For nginx, set `proxy_buffering off;`.
- **Allow long-lived responses.** An event stream stays open for as long as
the app is. Raise the proxy's read timeout well past a minute (nginx:
`proxy_read_timeout 1h;`).
3. **Tell Minstrel where it lives**, in **Admin → Integrations**:
- **Client IP detection:** the number of proxies in front of Minstrel. One
proxy (Caddy, Traefik, nginx) is 1; Cloudflare in front of Traefik is 2.
Minstrel uses this to find the real client address, for login rate
limits and the sessions list, and to tell whether a request arrived over
HTTPS.
- **Public address:** the URL people use, e.g. `https://music.example.com`.
Password-reset emails link here. **Until it is set, no reset email is
sent**, so a forged `Host` header can never put a reset token on a link
to someone else's server.
### Why the proxy count matters
Minstrel only believes `X-Forwarded-For` and `X-Forwarded-Proto` from the
number of proxies you configure, counted from the right. Set it too high, or
leave it at the default of 1 with no proxy in front, and a client can write
those headers itself: it could claim any address to slip past the per-address
login limit, or claim HTTPS. With no proxy, set it to 0.
When the proxy reports HTTPS, Minstrel marks the session cookie `Secure` and
sends `Strict-Transport-Security`. Over plain HTTP it does neither, and it
never redirects to HTTPS, so LAN use keeps working unchanged.
## First account
A fresh server with no accounts prints a **setup token** in its log:
```
docker compose logs minstrel | grep setup_token
```
Creating the first account (which becomes the admin) requires that token, so
whoever reaches a newly exposed server first can't claim it. The token
changes on every restart until an account exists.
## Android app
The app connects over whatever URL you give it, HTTP or HTTPS. Once the
server has a public HTTPS address, give the app that address so the session
cookie never crosses the internet unencrypted. See
[security notes](./security.md#android-allows-plain-http) for why plain HTTP
is still allowed.
+105
View File
@@ -0,0 +1,105 @@
# Security notes
What Minstrel does to protect accounts, and the reasoning behind the
decisions that look odd at first. For deployment steps, see
[hosting](./hosting.md).
## Accounts and sessions
- **Passwords** are stored as bcrypt hashes.
- **Login, registration and password reset are rate-limited:** 10 failed
sign-ins per account and 50 per address per 15 minutes, with similar limits
on register, forgot-password and reset. The Subsonic `/rest` API shares the
login limits. An unknown username takes as long to reject as a wrong
password, so timing doesn't reveal which accounts exist.
- **Sessions** are random 256-bit tokens; the server stores only their
SHA-256. A session ends after 30 days unused or 365 days in total. Changing
your password signs out your other devices; a password reset, or an admin
resetting it, signs out all of them.
- **API keys** (the OpenSubsonic `apiKey`) are stored as SHA-256 too, so a
key is shown once, when you generate it in Settings, and can only be
replaced after that.
- **The first account** on an empty server needs the setup token from the
server log (see [hosting](./hosting.md#first-account)).
- **Password-reset links** are built only from the configured public address,
never from the request's `Host` header.
## Requests
- Request bodies are capped at 4 MiB and must arrive within 30 seconds.
Streams and the live-event connection are unaffected.
- Every response carries `X-Content-Type-Options: nosniff`,
`Referrer-Policy: strict-origin-when-cross-origin`, a restrictive
`Permissions-Policy` and `X-Frame-Options: DENY`. The web app also gets a
`Content-Security-Policy` that allows only its own scripts, by hash.
- Media responses are `Cache-Control: private`, so a shared cache never keeps
one user's audio or artwork for another.
## CSRF: SameSite cookies plus JSON-only writes
There are no CSRF tokens. The session cookie is `SameSite=Strict`, so
browsers don't send it with requests that start on another site. That leaves
one gap: SameSite treats every subdomain of the same registrable domain as the
same site, so a different app on `other.example.com` could still send a
request carrying the cookie. To close it, any state-changing `/api` request
authenticated by the cookie must have a JSON body (`Content-Type:
application/json`) or no body at all. An HTML form or a script on another
origin can't send JSON without a CORS preflight, and Minstrel answers no
cross-origin preflight. Requests authenticated with a bearer token or the
Subsonic query parameters carry nothing a browser attaches on its own, so
they aren't checked.
## Subsonic sign-in, and the one password stored in plain text
Classic Subsonic clients sign in with `t` and `s`: the MD5 of the password
followed by a random salt. To check that, the server has to know the password
itself, so supporting this sign-in method means storing a password Minstrel
can read. That is the Subsonic password, and it is the only credential
Minstrel keeps unhashed.
- **The recommended way in is the API key.** Clients that support the
OpenSubsonic `apiKey` should use it. The key is stored hashed and can be
replaced at any time in Settings.
- **The Subsonic password is never your login password.** Minstrel generates
it (**Settings → Subsonic password**), shows it once, and lets you replace
or turn it off. Because it is random, a copy of the database exposes access
to this server's Subsonic API and nothing else: it can't be a password you
also use somewhere else.
- **`t`/`s` and `p=` sign-in are off for an account until it has a Subsonic
password.** Plain `p=` sign-in is additionally off server-wide unless
`subsonic.allow_plaintext_password` is enabled.
- `minstrel admin reset-password` changes only the login password. Older
versions also copied it into the Subsonic password; upgrading clears every
Subsonic password once, so those copies are gone. An account that used
`t`/`s` sign-in needs a new Subsonic password generated in Settings.
## Android allows plain HTTP
The Android app permits cleartext connections, for two reasons that can't be
narrowed to a list of hosts. The server address is whatever the user types,
and many self-hosted servers run plain HTTP on a LAN. And UPnP, DLNA and
Sonos speakers are controlled over plain HTTP at addresses that are only
known once they're discovered. The reasoning lives in
`android/app/src/main/res/xml/network_security_config.xml`.
This doesn't weaken app updates: an APK altered in transit fails the
platform's signature check. It does mean a server reached over plain HTTP
across the internet exposes the session cookie in transit, which is why the
[hosting guide](./hosting.md) puts public servers behind HTTPS.
On the phone, the session cookie is encrypted with a key held in the Android
Keystore, so a copy of the app's files doesn't yield a usable session.
## Build pipeline
Nothing is published unless every check passes: the Go, integration, web and
Android test suites, `govulncheck` against the toolchain that builds the
image, and `npm audit` on the packages that ship to the browser. See
`.gitea/workflows/release.yml`.
## Reporting a problem
Open an issue on the
[repository](https://git.fabledsword.com/bvandeusen/minstrel/issues), or
contact the maintainer privately first if it's something that shouldn't be
public until fixed.
+2 -2
View File
@@ -13,6 +13,7 @@ require (
github.com/jackc/pgx/v5 v5.9.2
github.com/stretchr/testify v1.11.1
golang.org/x/crypto v0.51.0
golang.org/x/sync v0.21.0
gopkg.in/yaml.v3 v3.0.1
)
@@ -25,7 +26,6 @@ require (
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
github.com/rogpeppe/go-internal v1.14.1 // indirect
golang.org/x/sync v0.20.0 // indirect
golang.org/x/sys v0.44.0 // indirect
golang.org/x/text v0.37.0 // indirect
golang.org/x/text v0.39.0 // indirect
)
+4 -4
View File
@@ -95,12 +95,12 @@ go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI=
golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.44.0 h1:ildZl3J4uzeKP07r2F++Op7E9B29JRUy+a27EibtBTQ=
golang.org/x/sys v0.44.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
golang.org/x/text v0.39.0 h1:UbZz4pLOvn600D6Oh6GGEI6VAmndrEBLv8/6BEXzyus=
golang.org/x/text v0.39.0/go.mod h1:3UwRclnC2g0TU9x8PZiyfOajCd1zaUNHF9cvqcQZ+ZM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
+267
View File
@@ -0,0 +1,267 @@
// Package acoustid looks up MusicBrainz recording ids by acoustic fingerprint
// through the AcoustID web service (M401).
//
// What it relies on, from https://acoustid.org/webservice (read 2026-10-06):
// - POST https://api.acoustid.org/v2/lookup with client, duration (whole
// seconds), fingerprint (fpcalc's compressed string) and meta. A gzip body
// with Content-Encoding: gzip is accepted and preferred, because
// fingerprints are long.
// - meta=recordings returns each recording's title, duration and artists,
// which the worker needs to tell several candidates apart (M401 D5).
// - "Do not make more than 3 requests per second."
// - The service is free for non-commercial use only.
//
// Error codes are the server's own, from acoustid-server's
// acoustid/api/errors.py.
package acoustid
import (
"bytes"
"cmp"
"compress/gzip"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"slices"
"strconv"
"sync"
"time"
)
// DefaultBaseURL is the lookup endpoint.
const DefaultBaseURL = "https://api.acoustid.org/v2/lookup"
const (
// minInterval keeps to AcoustID's 3 requests a second with headroom: one
// every 400ms is 2.5 a second.
minInterval = 400 * time.Millisecond
// lookupTimeout bounds one request, connection included (rule 156). A
// lookup answers in well under a second; 20s is the line past which slow
// has become never.
lookupTimeout = 20 * time.Second
// bodyLimit caps the response read. A lookup with meta=recordings for a
// popular song runs to tens of kilobytes.
bodyLimit = 2 << 20
userAgent = "Minstrel ( https://git.fabledsword.com/bvandeusen/minstrel )"
)
// The server's error codes that the worker treats differently.
const (
codeInvalidFingerprint = 3
codeInvalidAPIKey = 4
codeServiceUnavailable = 13
codeTooManyRequests = 14
codeUnknownApplication = 17
)
var (
// ErrInvalidKey: the operator's application key was refused. No lookup
// can succeed until it is changed, so the worker stops and says so.
ErrInvalidKey = errors.New("acoustid: the API key was refused")
// ErrInvalidFingerprint: AcoustID rejected this track's fingerprint. A
// verdict on the track, not on the service.
ErrInvalidFingerprint = errors.New("acoustid: the fingerprint was rejected")
// ErrUnavailable: the service could not answer now (unreachable, timed
// out, overloaded or rate limiting). Says nothing about the track, so the
// lookup is tried again later.
ErrUnavailable = errors.New("acoustid: the service is unavailable")
)
// Recording is one MusicBrainz recording a fingerprint matched.
type Recording struct {
ID string
Title string
// DurationSec is MusicBrainz's length for the recording; 0 when unknown.
DurationSec int
Artists []string
}
// Candidate is a recording with the score of the best AcoustID result that
// linked to it.
type Candidate struct {
Recording
Score float64
}
// Client calls the lookup endpoint, at most one request per minInterval.
type Client struct {
baseURL string
http *http.Client
mu sync.Mutex
lastCall time.Time
}
// New returns a client for baseURL (DefaultBaseURL in production).
func New(baseURL string) *Client {
return &Client{baseURL: baseURL, http: &http.Client{Timeout: lookupTimeout}}
}
// Lookup asks which recordings match fingerprint, a compressed chromaprint
// of audio durationSec long. It returns every candidate, best score first;
// choosing among them is the caller's job. No match is an empty slice and
// no error.
func (c *Client) Lookup(ctx context.Context, apiKey, fingerprint string, durationSec int) ([]Candidate, error) {
if err := c.wait(ctx); err != nil {
return nil, err
}
body, err := gzipForm(url.Values{
"client": {apiKey},
"duration": {strconv.Itoa(durationSec)},
"fingerprint": {fingerprint},
"meta": {"recordings"},
"format": {"json"},
})
if err != nil {
return nil, err
}
reqCtx, cancel := context.WithTimeout(ctx, lookupTimeout)
defer cancel()
req, err := http.NewRequestWithContext(reqCtx, http.MethodPost, c.baseURL, bytes.NewReader(body))
if err != nil {
return nil, fmt.Errorf("acoustid: build request: %w", err)
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.Header.Set("Content-Encoding", "gzip")
req.Header.Set("Accept", "application/json")
req.Header.Set("User-Agent", userAgent)
resp, err := c.http.Do(req)
if err != nil {
// The caller giving up is not the service failing.
if ctx.Err() != nil {
return nil, ctx.Err()
}
return nil, fmt.Errorf("%w: %v", ErrUnavailable, err)
}
defer func() { _ = resp.Body.Close() }()
raw, err := io.ReadAll(io.LimitReader(resp.Body, bodyLimit))
if err != nil {
return nil, fmt.Errorf("%w: read body: %v", ErrUnavailable, err)
}
return parseLookup(resp.StatusCode, raw)
}
// wait blocks until minInterval has passed since the last request, then
// claims the slot.
func (c *Client) wait(ctx context.Context) error {
c.mu.Lock()
defer c.mu.Unlock()
if d := time.Until(c.lastCall.Add(minInterval)); d > 0 {
t := time.NewTimer(d)
defer t.Stop()
select {
case <-ctx.Done():
return ctx.Err()
case <-t.C:
}
}
c.lastCall = time.Now()
return nil
}
func gzipForm(v url.Values) ([]byte, error) {
var buf bytes.Buffer
zw := gzip.NewWriter(&buf)
if _, err := zw.Write([]byte(v.Encode())); err != nil {
return nil, fmt.Errorf("acoustid: compress request: %w", err)
}
if err := zw.Close(); err != nil {
return nil, fmt.Errorf("acoustid: compress request: %w", err)
}
return buf.Bytes(), nil
}
type lookupResponse struct {
Status string `json:"status"`
Error *struct {
Code int `json:"code"`
Message string `json:"message"`
} `json:"error"`
Results []struct {
ID string `json:"id"`
Score float64 `json:"score"`
Recordings []struct {
ID string `json:"id"`
Title string `json:"title"`
Duration float64 `json:"duration"`
Artists []struct {
Name string `json:"name"`
} `json:"artists"`
} `json:"recordings"`
} `json:"results"`
}
// parseLookup reads a lookup response. AcoustID reports its errors in the
// body with a code, alongside a 4xx or 5xx status, so the body is read first
// and the status only decides when the body says nothing usable.
func parseLookup(status int, raw []byte) ([]Candidate, error) {
var r lookupResponse
if err := json.Unmarshal(raw, &r); err != nil {
if status != http.StatusOK {
return nil, fmt.Errorf("%w: status %d", ErrUnavailable, status)
}
return nil, fmt.Errorf("%w: unreadable response: %v", ErrUnavailable, err)
}
if r.Status != "ok" {
return nil, lookupError(status, r)
}
return candidates(r), nil
}
func lookupError(status int, r lookupResponse) error {
if r.Error == nil {
return fmt.Errorf("%w: status %d", ErrUnavailable, status)
}
switch r.Error.Code {
case codeInvalidAPIKey, codeUnknownApplication:
return fmt.Errorf("%w: %s", ErrInvalidKey, r.Error.Message)
case codeInvalidFingerprint:
return fmt.Errorf("%w: %s", ErrInvalidFingerprint, r.Error.Message)
case codeServiceUnavailable, codeTooManyRequests:
return fmt.Errorf("%w: %s", ErrUnavailable, r.Error.Message)
}
if status >= http.StatusInternalServerError {
return fmt.Errorf("%w: %s", ErrUnavailable, r.Error.Message)
}
// Any other code is a request this client built wrongly. Retrying would
// repeat it, so it is named rather than folded into unavailable.
return fmt.Errorf("acoustid: error %d: %s", r.Error.Code, r.Error.Message)
}
// candidates flattens the results into one entry per recording. The same
// recording can hang off several AcoustID results (one audio, fingerprinted
// from different sources); it keeps the best score it was seen with.
// Results with no linked recording are fingerprints AcoustID knows but nobody
// has tied to MusicBrainz, and carry nothing to fill.
func candidates(r lookupResponse) []Candidate {
out := []Candidate{}
at := map[string]int{}
for _, res := range r.Results {
for _, rec := range res.Recordings {
if rec.ID == "" {
continue
}
if i, ok := at[rec.ID]; ok {
out[i].Score = max(out[i].Score, res.Score)
continue
}
c := Candidate{
Recording: Recording{ID: rec.ID, Title: rec.Title, DurationSec: int(rec.Duration + 0.5)},
Score: res.Score,
}
for _, a := range rec.Artists {
c.Artists = append(c.Artists, a.Name)
}
at[rec.ID] = len(out)
out = append(out, c)
}
}
// Best first, keeping AcoustID's order among equals.
slices.SortStableFunc(out, func(a, b Candidate) int { return cmp.Compare(b.Score, a.Score) })
return out
}
+220
View File
@@ -0,0 +1,220 @@
package acoustid
import (
"compress/gzip"
"context"
"errors"
"io"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"time"
)
// docSample is the meta=recordings response printed in AcoustID's own
// documentation (https://acoustid.org/webservice), artists and release groups
// as given.
const docSample = `{
"status": "ok",
"results": [{
"score": 1.0,
"id": "9ff43b6a-4f16-427c-93c2-92307ca505e0",
"recordings": [{
"duration": 639,
"releasegroups": [{"type": "Album", "id": "ddaa2d4d-314e-3e7c-b1d0-f6d207f5aa2f", "title": "Before the Dawn Heals Us"}],
"title": "Lower Your Eyelids to Die With the Sun",
"id": "cd2e7c47-16f5-46c6-a37c-a1eb7bf599ff",
"artists": [{"id": "6d7b7cd4-254b-4c25-83f6-dd20f98ceacd", "name": "M83"}]
}]
}]
}`
// severalSample is the shape the worker has to disambiguate: one audio linked
// to two recordings (an album and a single release), the album recording
// reached again through a weaker second result, and a third result that
// AcoustID knows but nobody has tied to MusicBrainz.
const severalSample = `{
"status": "ok",
"results": [
{"score": 0.71, "id": "a2", "recordings": [{"id": "rec-album", "title": "Song", "duration": 200.4}]},
{"score": 0.93, "id": "a1", "recordings": [
{"id": "rec-album", "title": "Song", "duration": 200.4, "artists": [{"name": "A"}, {"name": "B"}]},
{"id": "rec-single", "title": "Song (radio edit)", "duration": 181}
]},
{"score": 0.88, "id": "a3"}
]
}`
func TestParseLookup(t *testing.T) {
cases := []struct {
name string
body string
want []Candidate
}{
{"single match", docSample, []Candidate{{
Recording: Recording{
ID: "cd2e7c47-16f5-46c6-a37c-a1eb7bf599ff", Title: "Lower Your Eyelids to Die With the Sun",
DurationSec: 639, Artists: []string{"M83"},
},
Score: 1.0,
}}},
{"several recordings", severalSample, []Candidate{
{Recording: Recording{ID: "rec-album", Title: "Song", DurationSec: 200}, Score: 0.93},
{Recording: Recording{ID: "rec-single", Title: "Song (radio edit)", DurationSec: 181}, Score: 0.93},
}},
// Choosing a threshold is the worker's; the parser carries the score.
{"below threshold", `{"status":"ok","results":[{"score":0.31,"id":"x","recordings":[{"id":"rec"}]}]}`,
[]Candidate{{Recording: Recording{ID: "rec"}, Score: 0.31}}},
{"no results", `{"status":"ok","results":[]}`, []Candidate{}},
{"results with no recordings", `{"status":"ok","results":[{"score":0.99,"id":"x"}]}`, []Candidate{}},
}
for _, c := range cases {
got, err := parseLookup(http.StatusOK, []byte(c.body))
if err != nil {
t.Errorf("%s: err = %v", c.name, err)
continue
}
if !sameCandidates(got, c.want) {
t.Errorf("%s:\n got %+v\nwant %+v", c.name, got, c.want)
}
}
}
// The first result's recording is the album one with both artists; the
// weaker repeat must not replace its fields, only fail to lower its score.
func TestParseLookup_RepeatedRecordingKeepsBestScoreAndFirstFields(t *testing.T) {
got, err := parseLookup(http.StatusOK, []byte(severalSample))
if err != nil {
t.Fatal(err)
}
if got[0].ID != "rec-album" || got[0].Score != 0.93 {
t.Errorf("album recording = %+v, want score 0.93", got[0])
}
}
func TestParseLookup_Errors(t *testing.T) {
cases := []struct {
name string
status int
body string
want error
}{
{"invalid key", 400, `{"status":"error","error":{"code":4,"message":"invalid API key"}}`, ErrInvalidKey},
{"unknown application", 400, `{"status":"error","error":{"code":17,"message":"unknown application"}}`, ErrInvalidKey},
{"bad fingerprint", 400, `{"status":"error","error":{"code":3,"message":"invalid fingerprint"}}`, ErrInvalidFingerprint},
{"rate limited", 429, `{"status":"error","error":{"code":14,"message":"rate limit (3.000000 requests per second) exceeded, try again later"}}`, ErrUnavailable},
{"down", 503, `{"status":"error","error":{"code":13,"message":"service currently unavailable, try again later"}}`, ErrUnavailable},
{"internal error", 500, `{"status":"error","error":{"code":5,"message":"internal error"}}`, ErrUnavailable},
{"proxy page", 502, `<html>Bad Gateway</html>`, ErrUnavailable},
}
for _, c := range cases {
_, err := parseLookup(c.status, []byte(c.body))
if !errors.Is(err, c.want) {
t.Errorf("%s: err = %v, want %v", c.name, err, c.want)
}
}
// A code this client caused (here a missing parameter) is neither the
// key nor the service, so it must not read as either: retrying repeats it.
_, err := parseLookup(400, []byte(`{"status":"error","error":{"code":2,"message":"missing required parameter \"fingerprint\""}}`))
if err == nil || errors.Is(err, ErrUnavailable) || errors.Is(err, ErrInvalidKey) || !strings.Contains(err.Error(), "missing required parameter") {
t.Errorf("missing parameter: err = %v", err)
}
}
// The request is what AcoustID documents: a gzip form POST carrying the key,
// whole-second duration, fingerprint and meta=recordings.
func TestLookup_SendsACompressedForm(t *testing.T) {
var form url.Values
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost || r.Header.Get("Content-Encoding") != "gzip" {
t.Errorf("request %s with Content-Encoding %q, want a gzip POST", r.Method, r.Header.Get("Content-Encoding"))
}
zr, err := gzip.NewReader(r.Body)
if err != nil {
t.Fatalf("body is not gzip: %v", err)
}
raw, _ := io.ReadAll(zr)
form, _ = url.ParseQuery(string(raw))
_, _ = io.WriteString(w, docSample)
}))
defer srv.Close()
got, err := New(srv.URL).Lookup(context.Background(), "key123", "AQADtE", 241)
if err != nil || len(got) != 1 {
t.Fatalf("Lookup = %+v, %v", got, err)
}
for k, want := range map[string]string{
"client": "key123", "duration": "241", "fingerprint": "AQADtE", "meta": "recordings", "format": "json",
} {
if form.Get(k) != want {
t.Errorf("form %s = %q, want %q", k, form.Get(k), want)
}
}
}
func TestLookup_DeadlineIsUnavailable(t *testing.T) {
release := make(chan struct{})
srv := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) { <-release }))
defer srv.Close()
defer close(release)
c := New(srv.URL)
c.http.Timeout = 50 * time.Millisecond
_, err := c.Lookup(context.Background(), "k", "f", 1)
if !errors.Is(err, ErrUnavailable) {
t.Errorf("err = %v, want ErrUnavailable", err)
}
}
// A cancelled caller is reported as such, never as the service being down.
func TestLookup_CancelledCallerIsNotUnavailable(t *testing.T) {
release := make(chan struct{})
srv := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) { <-release }))
defer srv.Close()
defer close(release)
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
defer cancel()
_, err := New(srv.URL).Lookup(ctx, "k", "f", 1)
if !errors.Is(err, context.DeadlineExceeded) || errors.Is(err, ErrUnavailable) {
t.Errorf("err = %v, want the caller's own deadline", err)
}
}
func TestLookup_KeepsToTheRateLimit(t *testing.T) {
var at []time.Time
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
at = append(at, time.Now())
_, _ = io.WriteString(w, `{"status":"ok","results":[]}`)
}))
defer srv.Close()
c := New(srv.URL)
for range 3 {
if _, err := c.Lookup(context.Background(), "k", "f", 1); err != nil {
t.Fatal(err)
}
}
for i := 1; i < len(at); i++ {
// A little slack for the server's own clock reading.
if gap := at[i].Sub(at[i-1]); gap < minInterval-10*time.Millisecond {
t.Errorf("requests %d and %d were %s apart, want at least %s", i-1, i, gap, minInterval)
}
}
}
func sameCandidates(a, b []Candidate) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i].ID != b[i].ID || a[i].Title != b[i].Title || a[i].DurationSec != b[i].DurationSec ||
a[i].Score != b[i].Score || strings.Join(a[i].Artists, "|") != strings.Join(b[i].Artists, "|") {
return false
}
}
return true
}
+171
View File
@@ -0,0 +1,171 @@
package api
import (
"encoding/json"
"errors"
"net/http"
"time"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/library"
)
// AcoustID recording-id lookup, admin side (M401 #3922).
// acoustIDUnsettledLimit caps the admin list of tracks a lookup could not
// settle. It is a list to read, not to page through.
const acoustIDUnsettledLimit = 200
// acoustIDSettingsResp never carries the key, only whether one is stored.
type acoustIDSettingsResp struct {
Enabled bool `json:"enabled"`
MinScore float32 `json:"min_score"`
APIKeySet bool `json:"api_key_set"`
}
func acoustIDSettingsRespOf(s library.AcoustIDSettings) acoustIDSettingsResp {
return acoustIDSettingsResp{Enabled: s.Enabled, MinScore: s.MinScore, APIKeySet: s.APIKey != ""}
}
type acoustIDStatusResp struct {
Running bool `json:"running"`
LastPassAt *time.Time `json:"last_pass_at"`
// Problem is why the last pass stopped short, in words for the card.
Problem string `json:"problem"`
}
// acoustIDCoverageResp: from_tag + from_acoustid + the tracks with no MBID =
// total. ambiguous, no_match and failed are lookup states of tracks with no
// MBID; pending is what the worker still has to look at.
type acoustIDCoverageResp struct {
Total int64 `json:"total"`
FromTag int64 `json:"from_tag"`
FromAcoustID int64 `json:"from_acoustid"`
Ambiguous int64 `json:"ambiguous"`
NoMatch int64 `json:"no_match"`
Failed int64 `json:"failed"`
Pending int64 `json:"pending"`
}
type acoustIDResp struct {
Settings acoustIDSettingsResp `json:"settings"`
// Ready is whether the worker will call AcoustID at all: switched on
// and holding a key.
Ready bool `json:"ready"`
Status acoustIDStatusResp `json:"status"`
Coverage acoustIDCoverageResp `json:"coverage"`
}
// handleGetAcoustID implements GET /api/admin/library/acoustid: the settings,
// what the worker is doing, and how much of the library has an MBID by source.
func (h *handlers) handleGetAcoustID(w http.ResponseWriter, r *http.Request) {
row, err := library.AcoustIDCoverage(r.Context(), h.pool)
if err != nil {
writeErrWithLog(w, h.logger, "admin: get acoustid coverage", apierror.InternalMsg("lookup failed", err))
return
}
cfg := h.acoustIDLookup.Settings().Get()
st := h.acoustIDLookup.Status()
resp := acoustIDResp{
Settings: acoustIDSettingsRespOf(cfg),
Ready: cfg.Ready(),
Status: acoustIDStatusResp{Running: st.Running, Problem: st.Problem},
Coverage: acoustIDCoverageResp{
Total: row.Total,
FromTag: row.FromTag,
FromAcoustID: row.FromAcoustid,
Ambiguous: row.Ambiguous,
NoMatch: row.NoMatch,
Failed: row.Failed,
Pending: row.Pending,
},
}
if !st.LastPassAt.IsZero() {
at := st.LastPassAt.UTC()
resp.Status.LastPassAt = &at
}
writeJSON(w, http.StatusOK, resp)
}
// acoustIDSettingsReq is the PUT body. api_key left out keeps the stored key;
// an empty string clears it.
type acoustIDSettingsReq struct {
Enabled bool `json:"enabled"`
MinScore float32 `json:"min_score"`
APIKey *string `json:"api_key"`
}
// handleUpdateAcoustIDSettings implements PUT /api/admin/library/acoustid-settings.
func (h *handlers) handleUpdateAcoustIDSettings(w http.ResponseWriter, r *http.Request) {
var req acoustIDSettingsReq
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, apierror.BadRequest("invalid_body", "malformed JSON"))
return
}
saved, err := h.acoustIDLookup.Settings().Set(r.Context(), library.AcoustIDSettingsUpdate{
Enabled: req.Enabled, MinScore: req.MinScore, APIKey: req.APIKey,
})
if err != nil {
if errors.Is(err, library.ErrAcoustIDSettingOutOfRange) {
writeErr(w, apierror.BadRequest("invalid_setting", err.Error()))
return
}
writeErrWithLog(w, h.logger, "admin acoustid settings: update failed", apierror.Internal(err))
return
}
writeJSON(w, http.StatusOK, acoustIDSettingsRespOf(saved))
}
// handleRunAcoustIDLookup implements POST /api/admin/library/acoustid/run:
// start a pass now rather than at the next tick. 409 when the lookup is off
// or has no key, so the card can say so rather than appear to do nothing.
func (h *handlers) handleRunAcoustIDLookup(w http.ResponseWriter, _ *http.Request) {
if !h.acoustIDLookup.Settings().Get().Ready() {
writeErr(w, apierror.Conflict("not_ready",
"Switch the AcoustID lookup on and add an API key first."))
return
}
h.acoustIDLookup.Kick()
w.WriteHeader(http.StatusAccepted)
}
type acoustIDUnsettledResp struct {
TrackID string `json:"track_id"`
Title string `json:"title"`
Artist string `json:"artist"`
Album string `json:"album"`
FilePath string `json:"file_path"`
State string `json:"state"`
BestScore *float32 `json:"best_score"`
Candidates int32 `json:"candidates"`
Detail *string `json:"detail"`
LookedUpAt time.Time `json:"looked_up_at"`
}
// handleListUnsettledAcoustID implements GET /api/admin/library/acoustid/unsettled:
// the tracks a lookup could not settle (ambiguous or failed), so they can be
// found and tagged by hand.
func (h *handlers) handleListUnsettledAcoustID(w http.ResponseWriter, r *http.Request) {
rows, err := dbq.New(h.pool).ListUnsettledAcoustIDLookups(r.Context(), acoustIDUnsettledLimit)
if err != nil {
writeErrWithLog(w, h.logger, "admin: list unsettled acoustid lookups", apierror.InternalMsg("lookup failed", err))
return
}
out := make([]acoustIDUnsettledResp, 0, len(rows))
for _, row := range rows {
out = append(out, acoustIDUnsettledResp{
TrackID: uuidToString(row.ID),
Title: row.Title,
Artist: row.ArtistName,
Album: row.AlbumTitle,
FilePath: row.FilePath,
State: row.State,
BestScore: row.BestScore,
Candidates: row.Candidates,
Detail: row.Detail,
LookedUpAt: row.LookedUpAt.Time.UTC(),
})
}
writeJSON(w, http.StatusOK, out)
}
+45
View File
@@ -1,10 +1,12 @@
package api
import (
"context"
"net/http"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/library"
)
// coverageRollupResp is the wire shape for GET /api/admin/library/coverage.
@@ -36,3 +38,46 @@ func (h *handlers) handleGetLibraryCoverage(w http.ResponseWriter, r *http.Reque
PendingNoMbid: row.PendingNoMbid,
})
}
// fingerprintCoverageResp is the wire shape for GET /api/admin/library/fingerprints.
// fingerprinted + rejected + pending = total. Missing tracks are not counted:
// there is no file to fingerprint. Enabled travels with the counts because with
// fingerprinting off (#3913) pending never shrinks, and a gauge that implies
// progress would be promising work nothing is doing.
type fingerprintCoverageResp struct {
Total int64 `json:"total"`
Fingerprinted int64 `json:"fingerprinted"`
Rejected int64 `json:"rejected"`
Pending int64 `json:"pending"`
Enabled bool `json:"enabled"`
}
// fingerprintCoverage reads the gauge against the current settings: a print at
// another length counts as pending, because the backfill will re-derive it.
func (h *handlers) fingerprintCoverage(ctx context.Context) (fingerprintCoverageResp, error) {
cfg := h.fingerprintSettings.Get()
row, err := library.FingerprintCoverage(ctx, h.pool, cfg)
if err != nil {
return fingerprintCoverageResp{}, err
}
return fingerprintCoverageResp{
Total: row.Total,
Fingerprinted: row.Fingerprinted,
Rejected: row.Rejected,
Pending: row.Pending,
Enabled: cfg.Enabled,
}, nil
}
// handleGetFingerprintCoverage implements GET /api/admin/library/fingerprints:
// how far the fingerprint backfill (#3908) has got. The backfill is its own
// worker spanning many passes, with no scan run to attach a tally to, so its
// progress is read live here. Always 200; zeros on an empty library.
func (h *handlers) handleGetFingerprintCoverage(w http.ResponseWriter, r *http.Request) {
cov, err := h.fingerprintCoverage(r.Context())
if err != nil {
writeErrWithLog(w, h.logger, "admin: get fingerprint coverage", apierror.InternalMsg("lookup failed", err))
return
}
writeJSON(w, http.StatusOK, cov)
}
+316
View File
@@ -0,0 +1,316 @@
package api
import (
"context"
"encoding/json"
"errors"
"io"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/library"
)
// duplicateMemberView is one copy in a proposed duplicate group. LikeCount and
// PlayCount span every user: the report is admin-only, and what a copy carries
// is the fact the operator weighs when choosing which to keep.
type duplicateMemberView struct {
TrackID string `json:"track_id"`
Title string `json:"title"`
ArtistName string `json:"artist_name"`
AlbumID string `json:"album_id"`
AlbumTitle string `json:"album_title"`
FilePath string `json:"file_path"`
FileFormat string `json:"file_format"`
FileSize int64 `json:"file_size"`
DurationSec int32 `json:"duration_sec"`
AddedAt string `json:"added_at"`
LikeCount int64 `json:"like_count"`
PlayCount int64 `json:"play_count"`
}
// duplicateGroupView is one proposal. SurvivorTrackID and SurvivorReason are
// the copy the report proposes keeping and the rule that chose it
// (library.ProposeSurvivor) — a default the merge (#3911) lets the operator
// override.
type duplicateGroupView struct {
ID string `json:"id"`
Tier string `json:"tier"`
WorstBitErrorRate *float32 `json:"worst_bit_error_rate"`
DetectedAt string `json:"detected_at"`
SurvivorTrackID string `json:"survivor_track_id"`
SurvivorReason string `json:"survivor_reason"`
Members []duplicateMemberView `json:"members"`
}
// duplicateSweepView is the latest sweep. State is "never" when none has run,
// which is what lets the page tell an empty report apart from a sweep that
// found nothing.
type duplicateSweepView struct {
State string `json:"state"`
StartedAt *string `json:"started_at"`
FinishedAt *string `json:"finished_at"`
Candidates *int32 `json:"candidates"`
GroupsFound *int32 `json:"groups_found"`
OversizeClusters *int32 `json:"oversize_clusters"`
ErrorMessage *string `json:"error_message"`
}
// adminDuplicatesResponse is the paged report. Total counts groups.
type adminDuplicatesResponse struct {
Sweep duplicateSweepView `json:"sweep"`
Fingerprints fingerprintCoverageResp `json:"fingerprints"`
Total int64 `json:"total"`
Limit int `json:"limit"`
Offset int `json:"offset"`
Groups []duplicateGroupView `json:"groups"`
}
// handleListDuplicates implements GET /api/admin/library/duplicates (#3912).
//
// Read-only. The sweep's state and the fingerprint backfill's progress travel
// with the groups because an empty report means three different things — still
// fingerprinting, never swept, or swept and clean — and the page has to say which.
func (h *handlers) handleListDuplicates(w http.ResponseWriter, r *http.Request) {
limit, offset, err := parsePaging(r.URL.Query())
if err != nil {
writeAdminJSONErr(w, http.StatusBadRequest, "invalid_paging")
return
}
ctx := r.Context()
q := dbq.New(h.pool)
sweep := duplicateSweepView{State: "never"}
last, err := q.GetLatestDuplicateSweep(ctx)
switch {
case err == nil:
sweep = duplicateSweepViewOf(last)
case !errors.Is(err, pgx.ErrNoRows):
h.logger.Error("admin: latest duplicate sweep", "err", err)
writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
return
}
cov, err := h.fingerprintCoverage(ctx)
if err != nil {
h.logger.Error("admin: fingerprint coverage", "err", err)
writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
return
}
total, err := q.CountPendingDuplicateGroups(ctx)
if err != nil {
h.logger.Error("admin: count duplicate groups", "err", err)
writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
return
}
rows, err := q.ListPendingDuplicateGroupMembers(ctx, dbq.ListPendingDuplicateGroupMembersParams{
PageLimit: int32(limit), PageOffset: int32(offset),
})
if err != nil {
h.logger.Error("admin: list duplicate groups", "err", err)
writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
return
}
writeJSON(w, http.StatusOK, adminDuplicatesResponse{
Sweep: sweep,
Fingerprints: cov,
Total: total,
Limit: limit,
Offset: offset,
Groups: foldDuplicateGroups(rows),
})
}
func duplicateSweepViewOf(s dbq.DuplicateSweep) duplicateSweepView {
v := duplicateSweepView{
State: "running",
Candidates: s.Candidates,
GroupsFound: s.GroupsFound,
OversizeClusters: s.OversizeClusters,
ErrorMessage: s.ErrorMessage,
}
started := formatTimestamp(s.StartedAt)
v.StartedAt = &started
if s.FinishedAt.Valid {
finished := formatTimestamp(s.FinishedAt)
v.FinishedAt = &finished
v.State = "finished"
}
return v
}
// foldDuplicateGroups folds the one-row-per-member query result into groups and
// proposes each group's survivor. It relies on the query ordering members of a
// group together, so a run-length fold is enough and the page order holds.
func foldDuplicateGroups(rows []dbq.ListPendingDuplicateGroupMembersRow) []duplicateGroupView {
groups := make([]duplicateGroupView, 0, 8)
var candidates [][]library.SurvivorCandidate
for _, row := range rows {
id := uuidToString(row.GroupID)
if n := len(groups); n == 0 || groups[n-1].ID != id {
groups = append(groups, duplicateGroupView{
ID: id,
Tier: row.Tier,
WorstBitErrorRate: row.WorstBitErrorRate,
DetectedAt: formatTimestamp(row.DetectedAt),
})
candidates = append(candidates, nil)
}
n := len(groups) - 1
trackID := uuidToString(row.TrackID)
groups[n].Members = append(groups[n].Members, duplicateMemberView{
TrackID: trackID,
Title: row.Title,
ArtistName: row.ArtistName,
AlbumID: uuidToString(row.AlbumID),
AlbumTitle: row.AlbumTitle,
FilePath: row.FilePath,
FileFormat: row.FileFormat,
FileSize: row.FileSize,
DurationSec: row.DurationMs / 1000,
AddedAt: formatTimestamp(row.AddedAt),
LikeCount: row.LikeCount,
PlayCount: row.PlayCount,
})
candidates[n] = append(candidates[n], library.SurvivorCandidate{
TrackID: trackID, FileFormat: row.FileFormat, FileSize: row.FileSize, AddedAt: row.AddedAt.Time,
})
}
for i := range groups {
groups[i].SurvivorTrackID, groups[i].SurvivorReason = library.ProposeSurvivor(candidates[i])
}
return groups
}
// handleRunDuplicateSweep implements POST /api/admin/library/duplicates/sweep:
// 202 when a sweep starts, 409 sweep_in_progress when one is already running.
// The sweep outlives the request, so it runs on a background context, as
// handleTriggerScan's scan does.
func (h *handlers) handleRunDuplicateSweep(w http.ResponseWriter, _ *http.Request) {
// Runs whatever the sweep interval says: the interval paces the automatic
// sweep, and an operator pressing the button has already decided.
started, err := library.TryStartDuplicateSweep(
context.Background(), h.pool, h.logger.With("source", "manual"), h.fingerprintSettings.Get(),
)
if err != nil {
h.logger.Error("admin: start duplicate sweep", "err", err)
writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
return
}
if !started {
writeAdminJSONErr(w, http.StatusConflict, "sweep_in_progress")
return
}
writeJSON(w, http.StatusAccepted, map[string]bool{"started": true})
}
// handleDismissDuplicateGroup implements POST
// /api/admin/library/duplicates/{id}/dismiss: "these are not duplicates". The
// sweep keeps the dismissal and will not propose that set of tracks again. 404
// duplicate_group_not_pending when the group was already resolved or is gone.
func (h *handlers) handleDismissDuplicateGroup(w http.ResponseWriter, r *http.Request) {
id, ok := parseUUID(chi.URLParam(r, "id"))
if !ok {
writeAdminJSONErr(w, http.StatusBadRequest, "invalid_id")
return
}
n, err := dbq.New(h.pool).DismissDuplicateGroup(r.Context(), id)
if err != nil {
h.logger.Error("admin: dismiss duplicate group", "err", err)
writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
return
}
if n == 0 {
writeAdminJSONErr(w, http.StatusNotFound, "duplicate_group_not_pending")
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "dismissed"})
}
// mergeDuplicateRequest chooses the copy to keep. An empty survivor_track_id
// keeps the report's proposal.
type mergeDuplicateRequest struct {
SurvivorTrackID string `json:"survivor_track_id"`
Unmonitor bool `json:"unmonitor"`
}
// mergeDuplicateResponse reports what the merge removed. RemovedPaths are files
// deleted from disk; the operator reads them to know exactly what went.
type mergeDuplicateResponse struct {
SurvivorTrackID string `json:"survivor_track_id"`
RemovedPaths []string `json:"removed_paths"`
LidarrUnmonitorFailed *bool `json:"lidarr_unmonitor_failed,omitempty"`
}
// mergeRequestBodyLimit bounds the request body. It holds one id and a flag.
const mergeRequestBodyLimit = 1 << 16
// handleMergeDuplicateGroup implements POST /api/admin/library/duplicates/{id}/merge
// (#3911): keep one copy, move the others' likes, plays and playlist entries onto
// it, and delete the others' files and rows.
//
// Errors:
// - 409 library_not_writable / 500 file_delete_failed when a file could not be
// removed — nothing was changed (fileRemoveAPIError)
// - 404 duplicate_group_not_pending when the group was already resolved
// - 400 survivor_not_in_group, invalid_id, invalid_body
func (h *handlers) handleMergeDuplicateGroup(w http.ResponseWriter, r *http.Request) {
admin, ok := requireUser(w, r)
if !ok {
return
}
groupID, ok := parseUUID(chi.URLParam(r, "id"))
if !ok {
writeAdminJSONErr(w, http.StatusBadRequest, "invalid_id")
return
}
var body mergeDuplicateRequest
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, mergeRequestBodyLimit)).Decode(&body); err != nil && !errors.Is(err, io.EOF) {
writeAdminJSONErr(w, http.StatusBadRequest, "invalid_body")
return
}
var survivorID pgtype.UUID // invalid: keep the proposal
if body.SurvivorTrackID != "" {
if survivorID, ok = parseUUID(body.SurvivorTrackID); !ok {
writeAdminJSONErr(w, http.StatusBadRequest, "invalid_id")
return
}
}
res, unmonitorFailed, err := h.tracks.MergeDuplicates(r.Context(), groupID, survivorID, admin.ID, body.Unmonitor)
if err != nil {
if apiErr, ok := fileRemoveAPIError(err); ok {
logFileRemoveFailure(h.logger, apiErr, "group_id", uuidToString(groupID))
writeErr(w, apiErr)
return
}
switch {
case errors.Is(err, library.ErrDuplicateGroupNotPending):
writeAdminJSONErr(w, http.StatusNotFound, "duplicate_group_not_pending")
case errors.Is(err, library.ErrSurvivorNotInGroup):
writeAdminJSONErr(w, http.StatusBadRequest, "survivor_not_in_group")
default:
h.logger.Error("admin: merge duplicate group", "group_id", uuidToString(groupID), "err", err)
writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
}
return
}
resp := mergeDuplicateResponse{
SurvivorTrackID: uuidToString(res.Survivor.TrackID),
RemovedPaths: make([]string, 0, len(res.Removed)),
}
for _, c := range res.Removed {
resp.RemovedPaths = append(resp.RemovedPaths, c.FilePath)
}
if body.Unmonitor && unmonitorFailed {
failed := true
resp.LidarrUnmonitorFailed = &failed
}
writeJSON(w, http.StatusOK, resp)
}
+65
View File
@@ -0,0 +1,65 @@
package api
import (
"testing"
"time"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
func dupUUID(b byte) pgtype.UUID {
var u pgtype.UUID
u.Bytes[15] = b
u.Valid = true
return u
}
func dupTS(t time.Time) pgtype.Timestamptz { return pgtype.Timestamptz{Time: t, Valid: true} }
// Rows arrive one per member, members of a group together. The fold must keep
// groups apart, keep the query's order, and propose each group's survivor from
// its own members only.
func TestFoldDuplicateGroups(t *testing.T) {
older := time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC)
newer := older.Add(48 * time.Hour)
ber := float32(0.04)
rows := []dbq.ListPendingDuplicateGroupMembersRow{
// Group 1: identical audio, sizes tie, the older copy should be kept.
{GroupID: dupUUID(1), Tier: "exact", DetectedAt: dupTS(newer), TrackID: dupUUID(10),
Title: "WWW", FileFormat: "mp3", FileSize: 6_900_000, DurationMs: 215_400, AddedAt: dupTS(newer), PlayCount: 3},
{GroupID: dupUUID(1), Tier: "exact", DetectedAt: dupTS(newer), TrackID: dupUUID(11),
Title: "WWW", FileFormat: "mp3", FileSize: 6_900_000, DurationMs: 215_400, AddedAt: dupTS(older), LikeCount: 1},
// Group 2: the same recording, FLAC against MP3.
{GroupID: dupUUID(2), Tier: "acoustic", WorstBitErrorRate: &ber, DetectedAt: dupTS(older), TrackID: dupUUID(20),
Title: "Lovesick", FileFormat: "mp3", FileSize: 9_000_000, DurationMs: 198_000, AddedAt: dupTS(older)},
{GroupID: dupUUID(2), Tier: "acoustic", WorstBitErrorRate: &ber, DetectedAt: dupTS(older), TrackID: dupUUID(21),
Title: "Lovesick", FileFormat: "flac", FileSize: 30_000_000, DurationMs: 198_000, AddedAt: dupTS(newer)},
}
got := foldDuplicateGroups(rows)
if len(got) != 2 {
t.Fatalf("folded %d groups, want 2", len(got))
}
g1, g2 := got[0], got[1]
if g1.ID != uuidToString(dupUUID(1)) || len(g1.Members) != 2 || g1.WorstBitErrorRate != nil {
t.Fatalf("group 1 = %+v, want the exact pair with no score", g1)
}
if g1.SurvivorTrackID != uuidToString(dupUUID(11)) || g1.SurvivorReason != "in the library longest" {
t.Errorf("group 1 survivor = (%s, %q), want the older copy", g1.SurvivorTrackID, g1.SurvivorReason)
}
if g1.Members[0].DurationSec != 215 || g1.Members[0].PlayCount != 3 || g1.Members[1].LikeCount != 1 {
t.Errorf("group 1 members lost their facts: %+v", g1.Members)
}
if g2.Tier != "acoustic" || g2.WorstBitErrorRate == nil || *g2.WorstBitErrorRate != ber {
t.Fatalf("group 2 = %+v, want the acoustic pair with its score", g2)
}
// Chosen from group 2's own members: a survivor leaking across groups is
// exactly what a wrong fold boundary would produce.
if g2.SurvivorTrackID != uuidToString(dupUUID(21)) || g2.SurvivorReason != "lossless (flac)" {
t.Errorf("group 2 survivor = (%s, %q), want the FLAC copy", g2.SurvivorTrackID, g2.SurvivorReason)
}
}
@@ -0,0 +1,67 @@
package api
import (
"encoding/json"
"errors"
"net/http"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/library"
)
// fingerprintSettingsBody is the wire shape for GET and PUT
// /api/admin/library/fingerprint-settings (M400 #3913). The threshold travels as
// the bit-error rate the matcher uses; the card presents it as a match percentage.
type fingerprintSettingsBody struct {
Enabled bool `json:"enabled"`
ChromaprintLengthSec int32 `json:"chromaprint_length_sec"`
AcousticMaxBitErrorRate float64 `json:"acoustic_max_bit_error_rate"`
BackfillConcurrency int32 `json:"backfill_concurrency"`
SweepIntervalHours int32 `json:"sweep_interval_hours"`
}
func fingerprintSettingsBodyOf(s library.FingerprintSettings) fingerprintSettingsBody {
return fingerprintSettingsBody{
Enabled: s.Enabled,
ChromaprintLengthSec: s.ChromaprintLengthSec,
AcousticMaxBitErrorRate: s.AcousticMaxBitErrorRate,
BackfillConcurrency: s.BackfillConcurrency,
SweepIntervalHours: s.SweepIntervalHours,
}
}
// handleGetFingerprintSettings implements GET /api/admin/library/fingerprint-settings.
func (h *handlers) handleGetFingerprintSettings(w http.ResponseWriter, _ *http.Request) {
writeJSON(w, http.StatusOK, fingerprintSettingsBodyOf(h.fingerprintSettings.Get()))
}
// handleUpdateFingerprintSettings implements PUT /api/admin/library/fingerprint-settings.
//
// A whole-row write. A body that leaves a field out decodes it as zero, which no
// field accepts, so a partial save is refused rather than zeroing what it omitted.
// The saved settings reach the scanner and both workers at once: they share the
// service instance.
func (h *handlers) handleUpdateFingerprintSettings(w http.ResponseWriter, r *http.Request) {
var req fingerprintSettingsBody
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, apierror.BadRequest("invalid_body", "malformed JSON"))
return
}
saved, err := h.fingerprintSettings.Set(r.Context(), library.FingerprintSettings{
Enabled: req.Enabled,
ChromaprintLengthSec: req.ChromaprintLengthSec,
AcousticMaxBitErrorRate: req.AcousticMaxBitErrorRate,
BackfillConcurrency: req.BackfillConcurrency,
SweepIntervalHours: req.SweepIntervalHours,
})
if err != nil {
// Validation mirrors migration 0061's CHECKs and names the field.
if errors.Is(err, library.ErrFingerprintSettingOutOfRange) {
writeErr(w, apierror.BadRequest("invalid_setting", err.Error()))
return
}
writeErrWithLog(w, h.logger, "admin fingerprint settings: update failed", apierror.Internal(err))
return
}
writeJSON(w, http.StatusOK, fingerprintSettingsBodyOf(saved))
}
@@ -0,0 +1,55 @@
package api
import (
"encoding/json"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"strings"
"testing"
"git.fabledsword.com/bvandeusen/minstrel/internal/library"
)
func TestGetFingerprintSettings_ServesDefaultsWithoutAService(t *testing.T) {
h := &handlers{logger: slog.New(slog.NewTextHandler(io.Discard, nil))}
rec := httptest.NewRecorder()
h.handleGetFingerprintSettings(rec, httptest.NewRequest(http.MethodGet, "/api/admin/library/fingerprint-settings", nil))
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rec.Code)
}
var got fingerprintSettingsBody
if err := json.NewDecoder(rec.Body).Decode(&got); err != nil {
t.Fatalf("decode: %v", err)
}
if want := fingerprintSettingsBodyOf(library.DefaultFingerprintSettings); got != want {
t.Fatalf("body = %+v, want the defaults %+v", got, want)
}
}
func TestUpdateFingerprintSettings_Rejects(t *testing.T) {
h := &handlers{logger: slog.New(slog.NewTextHandler(io.Discard, nil))}
for name, tc := range map[string]struct {
body string
code string
mentions string
}{
"a value out of range, naming the field": {
body: `{"enabled":true,"chromaprint_length_sec":5,"acoustic_max_bit_error_rate":0.15,"backfill_concurrency":2,"sweep_interval_hours":1}`,
code: "invalid_setting",
mentions: "chromaprint_length_sec",
},
// A partial body would otherwise zero every field it left out.
"a body missing fields": {body: `{"enabled":false}`, code: "invalid_setting"},
"malformed JSON": {body: `{"enabled":`, code: "invalid_body"},
} {
rec := httptest.NewRecorder()
h.handleUpdateFingerprintSettings(rec, httptest.NewRequest(
http.MethodPut, "/api/admin/library/fingerprint-settings", strings.NewReader(tc.body)))
body := rec.Body.String()
if rec.Code != http.StatusBadRequest || !strings.Contains(body, `"`+tc.code+`"`) || !strings.Contains(body, tc.mentions) {
t.Errorf("%s: status %d body %s; want 400 %s mentioning %q", name, rec.Code, body, tc.code, tc.mentions)
}
}
}
+88
View File
@@ -0,0 +1,88 @@
package api
import (
"encoding/json"
"errors"
"net/http"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/library"
)
// loudnessCoverageResp is the wire shape for GET /api/admin/library/loudness
// (M464 #4995). measured + silent + unreadable + pending = total; missing
// tracks are not counted. Enabled travels with the counts because with
// analysis off, pending never shrinks, and a gauge implying progress would be
// promising work nothing is doing.
type loudnessCoverageResp struct {
Total int64 `json:"total"`
Measured int64 `json:"measured"`
Silent int64 `json:"silent"`
Unreadable int64 `json:"unreadable"`
Pending int64 `json:"pending"`
Enabled bool `json:"enabled"`
}
// handleGetLoudnessCoverage implements GET /api/admin/library/loudness: how
// far the loudness backfill has got. Always 200; zeros on an empty library.
func (h *handlers) handleGetLoudnessCoverage(w http.ResponseWriter, r *http.Request) {
row, err := library.LoudnessCoverage(r.Context(), h.pool)
if err != nil {
writeErrWithLog(w, h.logger, "admin: get loudness coverage", apierror.InternalMsg("lookup failed", err))
return
}
writeJSON(w, http.StatusOK, loudnessCoverageResp{
Total: row.Total,
Measured: row.Measured,
Silent: row.Silent,
Unreadable: row.Unreadable,
Pending: row.Pending,
Enabled: h.loudnessSettings.Get().Enabled,
})
}
// loudnessSettingsBody is the wire shape for GET and PUT
// /api/admin/library/loudness-settings.
type loudnessSettingsBody struct {
Enabled bool `json:"enabled"`
BackfillConcurrency int32 `json:"backfill_concurrency"`
LeveledCacheMB int32 `json:"leveled_cache_mb"`
}
func loudnessSettingsBodyOf(s library.LoudnessSettings) loudnessSettingsBody {
return loudnessSettingsBody{
Enabled: s.Enabled,
BackfillConcurrency: s.BackfillConcurrency,
LeveledCacheMB: s.LeveledCacheMB,
}
}
// handleGetLoudnessSettings implements GET /api/admin/library/loudness-settings.
func (h *handlers) handleGetLoudnessSettings(w http.ResponseWriter, _ *http.Request) {
writeJSON(w, http.StatusOK, loudnessSettingsBodyOf(h.loudnessSettings.Get()))
}
// handleUpdateLoudnessSettings implements PUT /api/admin/library/loudness-settings.
// A whole-row write: a body that leaves out the concurrency decodes it as zero,
// which is refused rather than saved.
func (h *handlers) handleUpdateLoudnessSettings(w http.ResponseWriter, r *http.Request) {
var req loudnessSettingsBody
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, apierror.BadRequest("invalid_body", "malformed JSON"))
return
}
saved, err := h.loudnessSettings.Set(r.Context(), library.LoudnessSettings{
Enabled: req.Enabled,
BackfillConcurrency: req.BackfillConcurrency,
LeveledCacheMB: req.LeveledCacheMB,
})
if err != nil {
if errors.Is(err, library.ErrLoudnessSettingOutOfRange) {
writeErr(w, apierror.BadRequest("invalid_setting", err.Error()))
return
}
writeErrWithLog(w, h.logger, "admin loudness settings: update failed", apierror.Internal(err))
return
}
writeJSON(w, http.StatusOK, loudnessSettingsBodyOf(saved))
}
+37 -6
View File
@@ -23,10 +23,16 @@ type networkSettingsResp struct {
// arrived and count them rather than guess.
ForwardedChain string `json:"forwarded_chain"`
RemoteAddr string `json:"remote_addr"`
// PublicURL is where users reach Minstrel; reset emails link to it and
// are not sent while it is empty.
PublicURL string `json:"public_url"`
}
// Both fields are optional so the proxy card and the public-address card can
// each save their own value without overwriting the other's.
type updateNetworkSettingsReq struct {
TrustedProxyHops int `json:"trusted_proxy_hops"`
TrustedProxyHops *int `json:"trusted_proxy_hops"`
PublicURL *string `json:"public_url"`
}
func (h *handlers) handleGetNetworkSettings(w http.ResponseWriter, r *http.Request) {
@@ -39,13 +45,37 @@ func (h *handlers) handleUpdateNetworkSettings(w http.ResponseWriter, r *http.Re
writeErr(w, apierror.BadRequest("invalid_body", "malformed JSON"))
return
}
if err := h.netSettings.SetHops(r.Context(), req.TrustedProxyHops); err != nil {
if errors.Is(err, netsettings.ErrHopsOutOfRange) {
writeErr(w, apierror.BadRequest("invalid_hops", err.Error()))
if req.TrustedProxyHops == nil && req.PublicURL == nil {
writeErr(w, apierror.BadRequest("invalid_body", "nothing to update"))
return
}
// Validate everything before writing anything, so a bad public URL can't
// leave the hops half-saved.
if req.TrustedProxyHops != nil && (*req.TrustedProxyHops < 0 || *req.TrustedProxyHops > netsettings.MaxTrustedProxyHops) {
writeErr(w, apierror.BadRequest("invalid_hops", netsettings.ErrHopsOutOfRange.Error()))
return
}
if req.PublicURL != nil {
if _, err := netsettings.NormalizePublicURL(*req.PublicURL); err != nil {
writeErr(w, apierror.BadRequest("invalid_public_url", err.Error()))
return
}
}
if req.TrustedProxyHops != nil {
if err := h.netSettings.SetHops(r.Context(), *req.TrustedProxyHops); err != nil {
if errors.Is(err, netsettings.ErrHopsOutOfRange) {
writeErr(w, apierror.BadRequest("invalid_hops", err.Error()))
return
}
writeErrWithLog(w, h.logger, "admin network: update failed", apierror.Internal(err))
return
}
}
if req.PublicURL != nil {
if err := h.netSettings.SetPublicURL(r.Context(), *req.PublicURL); err != nil {
writeErrWithLog(w, h.logger, "admin network: public URL update failed", apierror.Internal(err))
return
}
writeErrWithLog(w, h.logger, "admin network: update failed", apierror.Internal(err))
return
}
// Echo the payload recomputed under the NEW value, so the card can show
// immediately what the change did to this request's own address rather
@@ -61,5 +91,6 @@ func (h *handlers) networkSettingsPayload(r *http.Request) networkSettingsResp {
DetectedClientIP: auth.ClientIP(r, hops),
ForwardedChain: r.Header.Get("X-Forwarded-For"),
RemoteAddr: r.RemoteAddr,
PublicURL: h.netSettings.PublicURL(),
}
}
+7
View File
@@ -133,6 +133,13 @@ func (h *handlers) handleDeleteQuarantineFile(w http.ResponseWriter, r *http.Req
}
action, err := h.lidarrQuarantine.DeleteFile(r.Context(), id, admin.ID)
if err != nil {
// Written in the enveloped shape, not writeAdminJSONErr's bare code: the
// message is the part that tells the operator which directory and uid.
if apiErr, ok := fileRemoveAPIError(err); ok {
logFileRemoveFailure(h.logger, apiErr, "track_id", uuidToString(id))
writeErr(w, apiErr)
return
}
switch {
case errors.Is(err, lidarrquarantine.ErrTrackNotFound):
writeAdminJSONErr(w, http.StatusNotFound, "track_not_found")
+1 -1
View File
@@ -69,7 +69,7 @@ func installQuarantineClientFn(t *testing.T, h *handlers) {
}
return lidarr.NewClient(c.BaseURL, c.APIKey)
}
h.lidarrQuarantine = lidarrquarantine.NewService(h.pool, cfg, clientFn)
h.lidarrQuarantine = lidarrquarantine.NewService(h.pool, cfg, clientFn, h.dataDir)
}
// flagDirect bypasses the HTTP handler to seed a quarantine row via the
+6 -4
View File
@@ -97,14 +97,16 @@ type tuningSnapshot struct {
func (h *handlers) tuningSnapshot() tuningSnapshot {
var out tuningSnapshot
out.Profiles = map[string]weightsResp{
recsettings.ScopeRadio: weightsRespFrom(h.recSettings.Weights(recsettings.ScopeRadio)),
recsettings.ScopeDailyMix: weightsRespFrom(h.recSettings.Weights(recsettings.ScopeDailyMix)),
recsettings.ScopeRadio: weightsRespFrom(h.recSettings.Weights(recsettings.ScopeRadio)),
recsettings.ScopeDailyMix: weightsRespFrom(h.recSettings.Weights(recsettings.ScopeDailyMix)),
recsettings.ScopeSongsLike: weightsRespFrom(h.recSettings.Weights(recsettings.ScopeSongsLike)),
}
out.Taste = tasteRespFrom(h.recSettings.Taste())
out.Discover = discoverRespFrom(h.recSettings.Discover())
out.Shipped.Profiles = map[string]weightsResp{
recsettings.ScopeRadio: weightsRespFrom(recsettings.ShippedRadioWeights()),
recsettings.ScopeDailyMix: weightsRespFrom(recsettings.ShippedDailyMixWeights()),
recsettings.ScopeRadio: weightsRespFrom(recsettings.ShippedRadioWeights()),
recsettings.ScopeDailyMix: weightsRespFrom(recsettings.ShippedDailyMixWeights()),
recsettings.ScopeSongsLike: weightsRespFrom(recsettings.ShippedSongsLikeWeights()),
}
out.Shipped.Taste = tasteRespFrom(recsettings.ShippedTasteTuning())
out.Shipped.Discover = discoverRespFrom(recsettings.ShippedDiscoverTuning())
+11 -4
View File
@@ -23,15 +23,17 @@ type removeTrackResponse struct {
// handleRemoveTrack implements DELETE /api/admin/tracks/{id}?unmonitor=true|false.
//
// Admin-only (gated by auth.RequireAdmin on the /admin route group). Always
// deletes the file + DB row and runs the album/artist cascade tidy-up. When
// Admin-only (gated by auth.RequireAdmin on the /admin route group). Deletes the
// file, then the DB row, and runs the album/artist cascade tidy-up — and deletes
// nothing at all when the file cannot be removed (#3918). When
// unmonitor=true and the track has an mbid, also calls Lidarr.UnmonitorTrack
// — failure there is non-fatal (the destructive part already completed) and
// surfaces as `lidarr_unmonitor_failed: true` in the success envelope.
//
// Per spec §5, Lidarr-side errors during the unmonitor step do NOT map to
// wire error codes; the only error codes this handler emits are not_found,
// server_error, plus the auth codes the middleware emits upstream.
// wire error codes. The codes this handler emits are not_found,
// library_not_writable (409) and file_delete_failed when the file could not be
// removed, server_error, plus the auth codes the middleware emits upstream.
func (h *handlers) handleRemoveTrack(w http.ResponseWriter, r *http.Request) {
idStr := chi.URLParam(r, "id")
trackID, ok := parseUUID(idStr)
@@ -66,6 +68,11 @@ func (h *handlers) handleRemoveTrack(w http.ResponseWriter, r *http.Request) {
writeErr(w, &apierror.Error{Status: http.StatusNotFound, Code: "not_found", Message: "track not found"})
return
}
if apiErr, ok := fileRemoveAPIError(err); ok {
logFileRemoveFailure(h.logger, apiErr, "track_id", idStr)
writeErr(w, apiErr)
return
}
h.logger.Error("api: remove track failed", "err", err, "track_id", idStr)
writeErr(w, apierror.InternalMsg("remove failed", err))
return
+14 -1
View File
@@ -169,7 +169,7 @@ func (h *handlers) handleAdminCreateUser(w http.ResponseWriter, r *http.Request)
user, err := q.CreateUserAdmin(r.Context(), dbq.CreateUserAdminParams{
Username: req.Username,
PasswordHash: string(hash),
ApiToken: apiToken,
ApiTokenHash: auth.HashAPIToken(apiToken),
IsAdmin: req.IsAdmin,
DisplayName: req.DisplayName,
})
@@ -285,6 +285,19 @@ func (h *handlers) handleAdminResetPassword(w http.ResponseWriter, r *http.Reque
return
}
// The target's existing sessions end with the old password. An admin
// resetting their own password keeps the session they're using, the
// same as a self-service change.
sessID, ownSession := auth.SessionIDFromContext(r.Context())
if targetID == caller.ID && ownSession {
_, err = q.DeleteOtherSessionsForUser(r.Context(), dbq.DeleteOtherSessionsForUserParams{UserID: targetID, ID: sessID})
} else {
_, err = q.DeleteSessionsForUser(r.Context(), targetID)
}
if err != nil {
h.logger.Error("admin reset password: revoke sessions failed", "err", err)
}
audit.WriteOrLog(r.Context(), h.pool, h.logger, caller.ID, targetID, audit.ActionPasswordResetAdmin, nil)
w.WriteHeader(http.StatusNoContent)
+120 -27
View File
@@ -7,6 +7,8 @@ package api
import (
"log/slog"
"math/rand"
"path/filepath"
"time"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5/pgxpool"
@@ -24,6 +26,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/playevents"
"git.fabledsword.com/bvandeusen/minstrel/internal/playlists"
"git.fabledsword.com/bvandeusen/minstrel/internal/reacquisition"
"git.fabledsword.com/bvandeusen/minstrel/internal/recommendation"
"git.fabledsword.com/bvandeusen/minstrel/internal/recsettings"
"git.fabledsword.com/bvandeusen/minstrel/internal/tags"
"git.fabledsword.com/bvandeusen/minstrel/internal/tracks"
@@ -32,34 +35,54 @@ import (
// Mount attaches /api/* handlers to r. Public endpoints (login) are outside
// RequireUser; everything else is gated by the middleware. The events writer
// is shared with the Subsonic mount so /rest/scrobble feeds the same store.
func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playevents.Writer, recCfg config.RecommendationConfig, recSettings *recsettings.Service, lidarrCfg *lidarrconfig.Service, lidarrReqs *lidarrrequests.Service, lidarrQuar *lidarrquarantine.Service, tracksSvc *tracks.Service, playlistsSvc *playlists.Service, coverEnricher *coverart.Enricher, coverSettings *coverart.SettingsService, tagSettings *tags.SettingsService, scanner *library.Scanner, scanCfg library.RunScanConfig, dataDir string, sender mailer.Sender, bus *eventbus.Bus, playlistScheduler *playlists.Scheduler, streamSecret []byte, netSettings *netsettings.Service, reacqSettings *reacquisition.SettingsService) {
func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playevents.Writer, recCfg config.RecommendationConfig, recSettings *recsettings.Service, lidarrCfg *lidarrconfig.Service, lidarrReqs *lidarrrequests.Service, lidarrQuar *lidarrquarantine.Service, tracksSvc *tracks.Service, playlistsSvc *playlists.Service, coverEnricher *coverart.Enricher, coverSettings *coverart.SettingsService, tagSettings *tags.SettingsService, scanner *library.Scanner, scanCfg library.RunScanConfig, dataDir string, sender mailer.Sender, bus *eventbus.Bus, playlistScheduler *playlists.Scheduler, streamSecret []byte, netSettings *netsettings.Service, reacqSettings *reacquisition.SettingsService, fpSettings *library.FingerprintSettingsService, loudSettings *library.LoudnessSettingsService, acoustIDLookup *library.AcoustIDLookupWorker) {
rng := rand.New(rand.NewSource(rand.Int63()))
setupToken, err := auth.NewSetupToken()
if err != nil {
// crypto/rand failing means the platform can't make secrets at all;
// sessions would be minted from the same source. Nothing to degrade to.
panic("api: mint setup token: " + err.Error())
}
logSetupTokenIfNeeded(pool, logger, setupToken)
h := &handlers{
pool: pool, logger: logger, events: events, recCfg: recCfg,
recSettings: recSettings,
rng: rng.Float64,
lidarrCfg: lidarrCfg,
lidarrRequests: lidarrReqs,
lidarrQuarantine: lidarrQuar,
tracks: tracksSvc,
playlists: playlistsSvc,
coverart: coverEnricher,
coverSettings: coverSettings,
tagSettings: tagSettings,
scanner: scanner,
scanCfg: scanCfg,
dataDir: dataDir,
mailer: sender,
eventbus: bus,
playlistScheduler: playlistScheduler,
streamSecret: streamSecret,
netSettings: netSettings,
reacqSettings: reacqSettings,
recSettings: recSettings,
rng: rng.Float64,
lidarrCfg: lidarrCfg,
lidarrRequests: lidarrReqs,
lidarrQuarantine: lidarrQuar,
tracks: tracksSvc,
playlists: playlistsSvc,
coverart: coverEnricher,
coverSettings: coverSettings,
tagSettings: tagSettings,
scanner: scanner,
scanCfg: scanCfg,
dataDir: dataDir,
mailer: sender,
eventbus: bus,
playlistScheduler: playlistScheduler,
streamSecret: streamSecret,
netSettings: netSettings,
reacqSettings: reacqSettings,
fingerprintSettings: fpSettings,
loudnessSettings: loudSettings,
acoustIDLookup: acoustIDLookup,
leveled: newLeveledRenderer(dataDir, loudSettings, logger),
librarySize: recommendation.NewLibrarySize(nil),
loginGuard: auth.NewLoginGuard(),
setupToken: setupToken,
requireSetupToken: true,
registerLimit: auth.NewAttemptLimiter(registerPerAddressMax, time.Hour),
forgotAddressLimit: auth.NewAttemptLimiter(forgotPerAddressMax, time.Hour),
forgotEmailLimit: auth.NewAttemptLimiter(forgotPerEmailMax, time.Hour),
resetLimit: auth.NewAttemptLimiter(resetFailuresPerAddressMax, 15*time.Minute),
}
r.Route("/api", func(api chi.Router) {
api.Post("/auth/login", h.handleLogin)
api.Post("/auth/register", h.handleRegister)
api.Get("/auth/setup-status", h.handleSetupStatus)
api.Post("/auth/forgot-password", h.handleForgotPassword)
api.Post("/auth/reset-password", h.handleResetPassword)
@@ -76,6 +99,9 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
// audio format from the path. The {ext} param is consumed by chi
// and ignored by the handler (which keys off {id}). See task #610.
api.With(auth.OptionalUser(pool, logger)).Get("/tracks/{id}/stream.{ext}", h.handleGetStream)
// The leveled stream for Sonos/UPnP (M464 #5001): session or a
// leveled token, like the plain stream.
api.With(auth.OptionalUser(pool, logger)).Get("/tracks/{id}/leveled.flac", h.handleGetLeveledStream)
api.Group(func(authed chi.Router) {
authed.Use(auth.RequireUser(pool, netSettings.Hops))
@@ -89,8 +115,12 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
authed.Put("/me/password", h.handleChangePassword)
authed.Put("/me/profile", h.handleUpdateMyProfile)
authed.Put("/me/timezone", h.handlePutTimezone)
authed.Get("/me/api-token", h.handleGetMyAPIToken)
authed.Get("/me/normalization", h.handleGetMyNormalization)
authed.Put("/me/normalization", h.handlePutMyNormalization)
authed.Post("/me/api-token", h.handleRegenerateMyAPIToken)
authed.Get("/me/subsonic-password", h.handleGetMySubsonicPassword)
authed.Post("/me/subsonic-password", h.handleGenerateMySubsonicPassword)
authed.Delete("/me/subsonic-password", h.handleClearMySubsonicPassword)
authed.Get("/me/sessions", h.handleListMySessions)
authed.Delete("/me/sessions/{id}", h.handleRevokeMySession)
authed.Post("/me/sessions/logout-others", h.handleRevokeMyOtherSessions)
@@ -110,6 +140,9 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
authed.Get("/library/genres", h.handleListGenres)
authed.Get("/library/years", h.handleListAlbumYears)
authed.Get("/library/sync", h.handleLibrarySync)
// Before /tracks/{id} for readability; chi prefers the static
// segment either way.
authed.Get("/tracks/replay-gain", h.handleGetReplayGain)
authed.Get("/tracks/{id}", h.handleGetTrack)
// /tracks/{id}/stream is mounted above with OptionalUser so
// it can accept either a session or a signed token.
@@ -213,6 +246,23 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
admin.Get("/library/missing", h.handleListMissingTracks)
admin.Get("/library/coverage", h.handleGetLibraryCoverage)
admin.Get("/library/fingerprints", h.handleGetFingerprintCoverage)
admin.Get("/library/fingerprint-settings", h.handleGetFingerprintSettings)
admin.Put("/library/fingerprint-settings", h.handleUpdateFingerprintSettings)
admin.Get("/library/loudness", h.handleGetLoudnessCoverage)
admin.Get("/library/loudness-settings", h.handleGetLoudnessSettings)
admin.Put("/library/loudness-settings", h.handleUpdateLoudnessSettings)
admin.Get("/library/acoustid", h.handleGetAcoustID)
admin.Put("/library/acoustid-settings", h.handleUpdateAcoustIDSettings)
admin.Post("/library/acoustid/run", h.handleRunAcoustIDLookup)
admin.Get("/library/acoustid/unsettled", h.handleListUnsettledAcoustID)
// Duplicates report (#3912): proposals from the duplicate sweep, a
// trigger to sweep now, dismissal, and the merge (#3911), which deletes
// the removed copies' files after moving their history onto the kept one.
admin.Get("/library/duplicates", h.handleListDuplicates)
admin.Post("/library/duplicates/sweep", h.handleRunDuplicateSweep)
admin.Post("/library/duplicates/{id}/dismiss", h.handleDismissDuplicateGroup)
admin.Post("/library/duplicates/{id}/merge", h.handleMergeDuplicateGroup)
admin.Get("/invites", h.handleListInvites)
admin.Post("/invites", h.handleCreateInvite)
@@ -268,12 +318,16 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
}
type handlers struct {
pool *pgxpool.Pool
logger *slog.Logger
events *playevents.Writer
recCfg config.RecommendationConfig
recSettings *recsettings.Service
rng func() float64
pool *pgxpool.Pool
logger *slog.Logger
events *playevents.Writer
recCfg config.RecommendationConfig
recSettings *recsettings.Service
rng func() float64
// librarySize memoises the track count that sizes the candidate pool
// (#3880). Held here rather than counted per request: the count is a
// full table scan, and library size only moves when a scan runs.
librarySize *recommendation.LibrarySize
lidarrCfg *lidarrconfig.Service
lidarrRequests *lidarrrequests.Service
lidarrQuarantine *lidarrquarantine.Service
@@ -292,6 +346,34 @@ type handlers struct {
// missing files (milestone #290) — grace window, backoff, attempt caps.
// Cached in the service, so the admin card reads it without a query.
reacqSettings *reacquisition.SettingsService
// fingerprintSettings is the fingerprinting policy (M400 #3913), the same
// instance the scanner and the fingerprint workers read, so a save from the
// admin card reaches them without a restart. Nil serves the defaults.
fingerprintSettings *library.FingerprintSettingsService
// loudnessSettings is the loudness analysis policy (M464 #4995), the same
// instance the loudness backfill reads. Nil serves the defaults.
loudnessSettings *library.LoudnessSettingsService
// acoustIDLookup is the AcoustID lookup worker (M401): its settings, its
// status and "look up now". Nil serves the defaults (off).
acoustIDLookup *library.AcoustIDLookupWorker
// leveled renders the leveled streams handed to Sonos/UPnP speakers
// (M464 #5001). Nil when its cache directory cannot be made: a level
// request then gets the plain stream.
leveled *library.LeveledRenderer
// setupToken must accompany the first registration while no users exist
// (see auth.SetupToken). requireSetupToken is set by Mount, the only
// production constructor; tests that build handlers directly leave it
// off unless they are testing it.
setupToken *auth.SetupToken
requireSetupToken bool
// loginGuard throttles failed logins per account and per address, and
// the limiters below cap the other unauthenticated auth routes. All are
// nil-safe, so tests that build handlers directly run unthrottled.
loginGuard *auth.LoginGuard
registerLimit *auth.AttemptLimiter
forgotAddressLimit *auth.AttemptLimiter
forgotEmailLimit *auth.AttemptLimiter
resetLimit *auth.AttemptLimiter
// netSettings caches the trusted reverse-proxy depth read by the auth
// middleware on every request and edited from the admin network card.
netSettings *netsettings.Service
@@ -305,3 +387,14 @@ type handlers struct {
// anything a client mints), which is the desired slice-1 default.
streamSecret []byte
}
// newLeveledRenderer makes the leveled-stream renderer, caching under the
// data directory. A failure is logged and leaves leveling off for speakers.
func newLeveledRenderer(dataDir string, settings *library.LoudnessSettingsService, logger *slog.Logger) *library.LeveledRenderer {
r, err := library.NewLeveledRenderer(filepath.Join(dataDir, "leveled-cache"), settings, logger)
if err != nil {
logger.Error("api: leveled streams unavailable", "err", err)
return nil
}
return r
}
+30 -2
View File
@@ -19,6 +19,20 @@ import (
// so an abandoned laptop doesn't stay logged in forever.
const sessionCookieMaxAge = 30 * 24 * time.Hour
// Limits on the unauthenticated auth routes other than login (which uses
// auth.LoginGuard). Per address, per window as wired in Mount.
const (
// registerPerAddressMax: 10 registrations an hour. Every attempt counts,
// typos included, which is still far above a household's need.
registerPerAddressMax = 10
// forgotPerAddressMax / forgotPerEmailMax: reset emails an hour. The
// per-email cap is what keeps one inbox from being mailbombed.
forgotPerAddressMax = 5
forgotPerEmailMax = 3
// resetFailuresPerAddressMax: wrong or expired reset tokens per 15 min.
resetFailuresPerAddressMax = 20
)
func (h *handlers) handleLogout(w http.ResponseWriter, r *http.Request) {
// The session token can be on the cookie OR bearer header — RequireUser
// accepted either. Re-resolve it here so we can delete the row.
@@ -70,10 +84,22 @@ func (h *handlers) handleLogin(w http.ResponseWriter, r *http.Request) {
return
}
// Checked before any lookup or bcrypt, so a throttled guess costs the
// server nothing and learns nothing.
addr := auth.ClientIP(r, h.netSettings.Hops())
if blocked, wait := h.loginGuard.Blocked(req.Username, addr); blocked {
writeRateLimited(w, wait)
return
}
q := dbq.New(h.pool)
user, err := q.GetUserByUsername(r.Context(), req.Username)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
// Same bcrypt time as a wrong password, so timing doesn't say
// which usernames exist.
auth.DummyVerify(req.Password)
h.loginGuard.Fail(req.Username, addr)
writeErr(w, apierror.Unauthorized("invalid_credentials", "invalid username or password"))
return
}
@@ -82,9 +108,11 @@ func (h *handlers) handleLogin(w http.ResponseWriter, r *http.Request) {
return
}
if !auth.VerifyPassword(user.PasswordHash, req.Password) {
h.loginGuard.Fail(req.Username, addr)
writeErr(w, apierror.Unauthorized("invalid_credentials", "invalid username or password"))
return
}
h.loginGuard.Succeed(req.Username)
token, err := auth.MintSessionToken()
if err != nil {
@@ -100,7 +128,7 @@ func (h *handlers) handleLogin(w http.ResponseWriter, r *http.Request) {
// the active-sessions surface: a session that was born somewhere the
// user recognises but is being used from somewhere they don't is the
// case this whole surface exists to surface.
Ip: auth.ClientIP(r, h.netSettings.Hops()),
Ip: addr,
}); err != nil {
h.logger.Error("api: insert session failed", "err", err)
writeErr(w, apierror.InternalMsg("insert failed", err))
@@ -112,7 +140,7 @@ func (h *handlers) handleLogin(w http.ResponseWriter, r *http.Request) {
Value: token,
Path: "/",
HttpOnly: true,
Secure: r.TLS != nil, // dev over http stays functional; prod over https gets Secure
Secure: auth.IsHTTPS(r, h.netSettings.Hops()), // plain HTTP and LAN stay functional (rule 94)
SameSite: http.SameSiteStrictMode,
MaxAge: int(sessionCookieMaxAge.Seconds()),
})
+33 -12
View File
@@ -14,6 +14,7 @@ import (
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/audit"
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/mailer"
)
@@ -47,6 +48,19 @@ func (h *handlers) handleForgotPassword(w http.ResponseWriter, r *http.Request)
}
email := strings.ToLower(strings.TrimSpace(req.Email))
// Throttled per address (a spray) and per email (a mailbomb of one
// inbox). Applied whether or not the email matches, so a 429 says
// nothing about which addresses are registered.
addr := auth.ClientIP(r, h.netSettings.Hops())
blockedAddr, waitAddr := h.forgotAddressLimit.Blocked(addr)
blockedEmail, waitEmail := h.forgotEmailLimit.Blocked(email)
if blockedAddr || blockedEmail {
writeRateLimited(w, max(waitAddr, waitEmail))
return
}
h.forgotAddressLimit.Record(addr)
h.forgotEmailLimit.Record(email)
q := dbq.New(h.pool)
var matched bool
var auditTarget pgtype.UUID
@@ -55,7 +69,7 @@ func (h *handlers) handleForgotPassword(w http.ResponseWriter, r *http.Request)
if err == nil && user.Email != nil && *user.Email != "" {
matched = true
auditTarget = user.ID
if sendErr := h.sendResetEmail(r.Context(), r, user); sendErr != nil {
if sendErr := h.sendResetEmail(r.Context(), user); sendErr != nil {
h.logger.Warn("forgot-password: send failed",
"email", email, "err", sendErr)
// fall through; response is still 200
@@ -75,7 +89,7 @@ func (h *handlers) handleForgotPassword(w http.ResponseWriter, r *http.Request)
// sendResetEmail generates a token, persists it, renders + sends the
// email. Returns the underlying error (caller logs it but doesn't
// surface to the HTTP response).
func (h *handlers) sendResetEmail(ctx context.Context, r *http.Request, user dbq.User) error {
func (h *handlers) sendResetEmail(ctx context.Context, user dbq.User) error {
if h.mailer == nil {
return errors.New("forgot-password: no mailer configured")
}
@@ -95,7 +109,10 @@ func (h *handlers) sendResetEmail(ctx context.Context, r *http.Request, user dbq
return err
}
resetURL := buildResetURL(r, token)
resetURL, err := buildResetURL(h.netSettings.PublicURL(), token)
if err != nil {
return err
}
textBody, htmlBody, err := mailer.RenderResetEmail(mailer.ResetEmailVars{
Username: user.Username,
ResetURL: resetURL,
@@ -113,14 +130,18 @@ func (h *handlers) sendResetEmail(ctx context.Context, r *http.Request, user dbq
return h.mailer.Send(ctx, *user.Email, mailer.ResetEmailSubject, textBody, htmlBody)
}
func buildResetURL(r *http.Request, token string) string {
scheme := "http"
if r.TLS != nil {
scheme = "https"
// errNoPublicURL stops a reset email from going out before the operator has
// said where Minstrel lives. It surfaces in the log, not the response, which
// stays the same opaque 200 either way.
var errNoPublicURL = errors.New("forgot-password: no public URL set (Admin → Integrations → Public address); reset email not sent")
// buildResetURL builds the emailed link from the operator-set public URL.
// It deliberately ignores the request: the Host header is whatever the
// requester sent, and building from it let a forged Host plant a real reset
// token on a link to someone else's server.
func buildResetURL(publicURL, token string) (string, error) {
if publicURL == "" {
return "", errNoPublicURL
}
host := r.Host
if host == "" {
host = "localhost"
}
return scheme + "://" + host + "/reset-password/" + token
return publicURL + "/reset-password/" + token, nil
}
+57 -1
View File
@@ -7,11 +7,59 @@ import (
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/mailer"
"git.fabledsword.com/bvandeusen/minstrel/internal/netsettings"
)
// withPublicURL gives h a network-settings service holding url, restoring an
// empty value afterwards so other tests see the default.
func withPublicURL(t *testing.T, h *handlers, pool *pgxpool.Pool, url string) {
t.Helper()
ns, err := netsettings.New(context.Background(), pool, nil)
if err != nil {
t.Fatalf("netsettings: %v", err)
}
if err := ns.SetPublicURL(context.Background(), url); err != nil {
t.Fatalf("set public url: %v", err)
}
t.Cleanup(func() { _ = ns.SetPublicURL(context.Background(), "") })
h.netSettings = ns
}
// With no public URL set, a reset email is not sent at all, and the response
// is still the same opaque 200.
func TestForgotPassword_NoPublicURL_SendsNothing(t *testing.T) {
if os.Getenv("MINSTREL_TEST_DATABASE_URL") == "" {
t.Skip("MINSTREL_TEST_DATABASE_URL not set")
}
h, pool := testHandlers(t)
fake := &mailer.FakeSender{}
h.mailer = fake
withPublicURL(t, h, pool, "")
user := seedUser(t, pool, "nourl", "pw", false)
if _, err := pool.Exec(context.Background(),
"UPDATE users SET email = 'nourl@example.com' WHERE id = $1", user.ID); err != nil {
t.Fatalf("seed email: %v", err)
}
req := httptest.NewRequest(http.MethodPost, "/api/auth/forgot-password",
bytes.NewReader([]byte(`{"email":"nourl@example.com"}`)))
rec := httptest.NewRecorder()
h.handleForgotPassword(rec, req)
if rec.Code != http.StatusOK {
t.Errorf("status = %d, want 200", rec.Code)
}
if len(fake.Sent) != 0 {
t.Errorf("sent %d emails with no public URL set, want 0", len(fake.Sent))
}
}
func TestForgotPassword_KnownEmail_SendsMail(t *testing.T) {
if os.Getenv("MINSTREL_TEST_DATABASE_URL") == "" {
t.Skip("MINSTREL_TEST_DATABASE_URL not set")
@@ -19,6 +67,7 @@ func TestForgotPassword_KnownEmail_SendsMail(t *testing.T) {
h, pool := testHandlers(t)
fake := &mailer.FakeSender{}
h.mailer = fake
withPublicURL(t, h, pool, "https://music.example.com")
user := seedUser(t, pool, "forgotuser", "pw", false)
if _, err := pool.Exec(context.Background(),
@@ -29,7 +78,8 @@ func TestForgotPassword_KnownEmail_SendsMail(t *testing.T) {
body := `{"email":"forgot@example.com"}`
req := httptest.NewRequest(http.MethodPost, "/api/auth/forgot-password",
bytes.NewReader([]byte(body)))
req.Host = "minstrel.example.com"
// A forged Host must not reach the link: it comes from the public URL.
req.Host = "attacker.example.net"
rec := httptest.NewRecorder()
h.handleForgotPassword(rec, req)
@@ -43,6 +93,12 @@ func TestForgotPassword_KnownEmail_SendsMail(t *testing.T) {
if got.To != "forgot@example.com" {
t.Errorf("To = %q, want forgot@example.com", got.To)
}
if !strings.Contains(got.TextBody, "https://music.example.com/reset-password/") {
t.Errorf("reset link not built from the public URL:\n%s", got.TextBody)
}
if strings.Contains(got.TextBody+got.HTMLBody, "attacker.example.net") {
t.Error("the request's Host header reached the emailed link")
}
// Token row was inserted.
var tokenCount int
if err := pool.QueryRow(context.Background(),
+62 -6
View File
@@ -1,8 +1,10 @@
package api
import (
"context"
"encoding/json"
"errors"
"log/slog"
"net/http"
"regexp"
"time"
@@ -10,6 +12,7 @@ import (
"github.com/jackc/pgerrcode"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"github.com/jackc/pgx/v5/pgxpool"
"golang.org/x/crypto/bcrypt"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
@@ -26,9 +29,12 @@ var usernameRe = regexp.MustCompile(`^[a-zA-Z0-9_-]{3,32}$`)
const minPasswordLength = 8
type registerReq struct {
Username string `json:"username"`
Password string `json:"password"`
InviteToken string `json:"invite_token"`
Username string `json:"username"`
Password string `json:"password"`
InviteToken string `json:"invite_token"`
// SetupToken is required only for the very first account; see
// auth.SetupToken.
SetupToken string `json:"setup_token"`
DisplayName *string `json:"display_name"`
}
@@ -55,6 +61,15 @@ type registerReq struct {
// - 409 username_taken (PG unique violation on users.username)
// - 500 server_error otherwise
func (h *handlers) handleRegister(w http.ResponseWriter, r *http.Request) {
// Every attempt counts, not only failures: a successful registration is
// the thing being sprayed when the mode is open.
addr := auth.ClientIP(r, h.netSettings.Hops())
if blocked, wait := h.registerLimit.Blocked(addr); blocked {
writeRateLimited(w, wait)
return
}
h.registerLimit.Record(addr)
var req registerReq
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, apierror.BadRequest("invalid_body", "invalid JSON body"))
@@ -79,6 +94,19 @@ func (h *handlers) handleRegister(w http.ResponseWriter, r *http.Request) {
return
}
// The first account becomes admin, so it must prove it can read the
// server log. Checked before the invite logic, which the empty-users
// state skips.
if userCount == 0 && h.requireSetupToken && !h.setupToken.Matches(req.SetupToken) {
// Repeat the token in the log at the moment someone needs it: the
// boot line may have scrolled away, or users may have been deleted
// since boot.
h.logger.Warn("register: first-admin registration needs the setup token",
"setup_token", h.setupToken.Value())
writeErr(w, apierror.Forbidden("setup_token_invalid", "the setup token from the server log is required to create the first account"))
return
}
// Validate invite (skipped on empty-users state; skipped in 'open' mode).
usedInviteToken := ""
if userCount > 0 {
@@ -133,7 +161,7 @@ func (h *handlers) handleRegister(w http.ResponseWriter, r *http.Request) {
user, err := q.CreateUserFirstAdminRace(r.Context(), dbq.CreateUserFirstAdminRaceParams{
Username: req.Username,
PasswordHash: string(hash),
ApiToken: apiToken,
ApiTokenHash: auth.HashAPIToken(apiToken),
DisplayName: req.DisplayName,
})
if err != nil {
@@ -175,7 +203,7 @@ func (h *handlers) handleRegister(w http.ResponseWriter, r *http.Request) {
UserID: user.ID,
TokenHash: auth.HashSessionToken(sessionToken),
UserAgent: r.UserAgent(),
Ip: auth.ClientIP(r, h.netSettings.Hops()),
Ip: addr,
}); err != nil {
h.logger.Error("register: insert session failed", "err", err)
writeErr(w, apierror.Internal(err))
@@ -186,7 +214,7 @@ func (h *handlers) handleRegister(w http.ResponseWriter, r *http.Request) {
Value: sessionToken,
Path: "/",
HttpOnly: true,
Secure: r.TLS != nil,
Secure: auth.IsHTTPS(r, h.netSettings.Hops()), // plain HTTP and LAN stay functional (rule 94)
SameSite: http.SameSiteStrictMode,
MaxAge: int(sessionCookieMaxAge.Seconds()),
})
@@ -223,3 +251,31 @@ func (h *handlers) handleRegister(w http.ResponseWriter, r *http.Request) {
},
})
}
// handleSetupStatus implements GET /api/auth/setup-status. It tells the
// register screen whether to ask for the setup token, i.e. whether no
// account exists yet. Public, since it is needed before anyone can sign in;
// "this server has no users" is not worth hiding from someone who could
// simply try to register.
func (h *handlers) handleSetupStatus(w http.ResponseWriter, r *http.Request) {
n, err := dbq.New(h.pool).CountUsers(r.Context())
if err != nil {
writeErrWithLog(w, h.logger, "setup status: count users failed", apierror.Internal(err))
return
}
writeJSON(w, http.StatusOK, map[string]bool{"setup_required": n == 0})
}
// logSetupTokenIfNeeded writes the setup token to the log at boot when the
// instance has no accounts yet, with the instruction for using it.
func logSetupTokenIfNeeded(pool *pgxpool.Pool, logger *slog.Logger, token *auth.SetupToken) {
if pool == nil || logger == nil {
return
}
n, err := dbq.New(pool).CountUsers(context.Background())
if err != nil || n > 0 {
return
}
logger.Warn("no accounts yet: open the web app, choose Create account, and enter this setup token to become the admin",
"setup_token", token.Value())
}
+72
View File
@@ -273,3 +273,75 @@ func TestRegister_FirstAdminRace(t *testing.T) {
// not "exactly one" (which would require serializable isolation).
t.Logf("admin count after race = %d (>=1 is the invariant)", adminCount)
}
// With the gate on (as Mount sets it), the first account needs the setup
// token from the log; a missing or wrong one is refused and creates nothing.
func TestRegister_FirstUserNeedsSetupToken(t *testing.T) {
if os.Getenv("MINSTREL_TEST_DATABASE_URL") == "" {
t.Skip("MINSTREL_TEST_DATABASE_URL not set")
}
h, _ := testHandlers(t)
resetUsers(t, h)
token, err := auth.NewSetupToken()
if err != nil {
t.Fatalf("mint: %v", err)
}
h.setupToken = token
h.requireSetupToken = true
register := func(body string) int {
req := httptest.NewRequest(http.MethodPost, "/api/auth/register", bytes.NewReader([]byte(body)))
rec := httptest.NewRecorder()
h.handleRegister(rec, req)
return rec.Code
}
if code := register(`{"username":"squatter","password":"abcd1234"}`); code != http.StatusForbidden {
t.Errorf("no token: status = %d, want 403", code)
}
if code := register(`{"username":"squatter","password":"abcd1234","setup_token":"wrong"}`); code != http.StatusForbidden {
t.Errorf("wrong token: status = %d, want 403", code)
}
var n int
if err := h.pool.QueryRow(context.Background(), `SELECT count(*) FROM users`).Scan(&n); err != nil {
t.Fatalf("count: %v", err)
}
if n != 0 {
t.Fatalf("a refused registration created %d account(s)", n)
}
if code := register(`{"username":"operator","password":"abcd1234","setup_token":"` + token.Value() + `"}`); code != http.StatusOK {
t.Fatalf("right token: status = %d, want 200", code)
}
// Once an account exists the token plays no part: the second user is
// governed by registration mode, not the setup token.
if _, err := h.pool.Exec(context.Background(),
`UPDATE registration_settings SET mode = 'open' WHERE id = true`); err != nil {
t.Fatalf("open mode: %v", err)
}
t.Cleanup(func() {
_, _ = h.pool.Exec(context.Background(),
`UPDATE registration_settings SET mode = 'invite_only' WHERE id = true`)
})
if code := register(`{"username":"second","password":"abcd1234"}`); code != http.StatusOK {
t.Errorf("second user without token: status = %d, want 200", code)
}
}
func TestSetupToken_MatchesOnlyItself(t *testing.T) {
tok, err := auth.NewSetupToken()
if err != nil {
t.Fatalf("mint: %v", err)
}
if !tok.Matches(tok.Value()) {
t.Error("token does not match itself")
}
if tok.Matches("") || tok.Matches(tok.Value()+"x") {
t.Error("token matched something else")
}
var none *auth.SetupToken
if none.Matches("") || none.Matches("anything") {
t.Error("a nil token must fail closed")
}
}
+19
View File
@@ -10,6 +10,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/audit"
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
@@ -42,6 +43,15 @@ func (h *handlers) handleResetPassword(w http.ResponseWriter, r *http.Request) {
return
}
// Tokens carry 256 bits, so guessing one is hopeless; the limit is on
// failed tries per address so that nobody gets to find that out at our
// expense.
addr := auth.ClientIP(r, h.netSettings.Hops())
if blocked, wait := h.resetLimit.Blocked(addr); blocked {
writeRateLimited(w, wait)
return
}
q := dbq.New(h.pool)
// Look up the reset record so we know which user to update before
@@ -50,6 +60,7 @@ func (h *handlers) handleResetPassword(w http.ResponseWriter, r *http.Request) {
reset, err := q.GetPasswordReset(r.Context(), req.Token)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
h.resetLimit.Record(addr)
writeErr(w, apierror.BadRequest("invalid_token", ""))
return
}
@@ -69,6 +80,7 @@ func (h *handlers) handleResetPassword(w http.ResponseWriter, r *http.Request) {
return
}
if rows == 0 {
h.resetLimit.Record(addr)
writeErr(w, apierror.BadRequest("invalid_token", ""))
return
}
@@ -92,6 +104,13 @@ func (h *handlers) handleResetPassword(w http.ResponseWriter, r *http.Request) {
return
}
// End every session the account has. Whoever needed a reset isn't signed
// in anywhere they rely on, and whoever may have learned the old password
// must not stay signed in on it.
if _, err := q.DeleteSessionsForUser(r.Context(), reset.UserID); err != nil {
h.logger.Error("reset password: revoke sessions failed", "err", err)
}
audit.WriteOrLog(r.Context(), h.pool, h.logger, reset.UserID, reset.UserID, audit.ActionPasswordResetByEmail, nil)
w.WriteHeader(http.StatusNoContent)
+32 -2
View File
@@ -65,7 +65,7 @@ func testHandlers(t *testing.T) (*handlers, *pgxpool.Pool) {
}
lidarrCfg := lidarrconfig.New(pool)
lidarrReqs := lidarrrequests.NewService(pool, lidarrCfg, nil, nil)
lidarrQuar := lidarrquarantine.NewService(pool, lidarrCfg, nil)
lidarrQuar := lidarrquarantine.NewService(pool, lidarrCfg, nil, "")
// tracks.Service has no Lidarr unmonitorer in tests by default; the
// admin-tracks tests below override h.tracks via installTracksLidarrStub
// when they need a stubbed Lidarr.
@@ -91,7 +91,7 @@ func seedUser(t *testing.T, pool *pgxpool.Pool, username, password string, isAdm
u, err := dbq.New(pool).CreateUser(context.Background(), dbq.CreateUserParams{
Username: prefixed,
PasswordHash: string(hash),
ApiToken: "test-api-token-" + prefixed,
ApiTokenHash: "test-api-token-" + prefixed,
IsAdmin: isAdmin,
})
if err != nil {
@@ -179,6 +179,36 @@ func TestHandleLogin_UnknownUserReturns401(t *testing.T) {
}
}
// A guesser who reaches the account limit is refused with 429 before any
// password check, so even the right password is turned away until the window
// passes, and the response says when to come back.
func TestHandleLogin_ThrottledAfterRepeatedFailures(t *testing.T) {
h, pool := testHandlers(t)
h.loginGuard = auth.NewLoginGuard()
seedUser(t, pool, "alice", "hunter2", false)
login := func(password string) *httptest.ResponseRecorder {
body := strings.NewReader(`{"username":"test-alice","password":"` + password + `"}`)
req := httptest.NewRequest(http.MethodPost, "/api/auth/login", body)
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
h.handleLogin(w, req)
return w
}
for i := 0; i < 10; i++ {
if w := login("wrong"); w.Code != http.StatusUnauthorized {
t.Fatalf("attempt %d: status = %d, want 401", i+1, w.Code)
}
}
w := login("hunter2")
if w.Code != http.StatusTooManyRequests {
t.Fatalf("status = %d, want 429 once the account limit is reached", w.Code)
}
if w.Header().Get("Retry-After") == "" {
t.Error("429 without Retry-After")
}
}
func TestHandleLogin_MalformedBodyReturns400(t *testing.T) {
h, _ := testHandlers(t)
req := httptest.NewRequest(http.MethodPost, "/api/auth/login",

Some files were not shown because too many files have changed in this diff Show More