After the X-Forwarded-Proto fix Sonos now gets a clean https:// URL
but returns vendor error 1023 - empty CurrentURIMetaData. Sonos
requires DIDL-Lite metadata with at minimum <res protocolInfo>
carrying the audio MIME type so it can validate the source before
playback. The original spec said 'Sonos accepts empty DIDL; recoverable
if a device rejects' - that was wrong for Sonos.
Server (cast_token.go):
- Look up the track and return mime (from tracks.file_format) +
title in the cast-token response. mimeForFormat covers the common
formats - mp3, flac, m4a/aac, ogg, opus, wav - falling through to
audio/mpeg for unknowns.
- Missing track returns 404 (apierror.NotFound) instead of letting the
caller mint a token for nothing.
Client (CastApi.kt, AVTransportClient.kt, OutputPickerController.kt):
- StreamTokenResponse gains mime + title (defaulted so old contracts
stay parseable).
- AVTransportClient.setAVTransportURIWithMetadata builds minimal Sonos-
acceptable DIDL-Lite around the URL + MIME + title. xml-escaped.
- selectUpnp calls the new overload; Timber.i now logs the MIME so the
next on-device test shows it.
Generic UPnP renderers tolerate the DIDL shape too - no downside to
sending it everywhere.
On-device test against Sonos showed SetAVTransportURI returning UPnP
error 714 (IllegalMimeType). Logcat:
POST /api/cast/stream-token -> 200 (token minted)
SetAVTransportURI to http://minstrel.fabledsword.com/...
<-- 500 from Sonos: SoapFaultException SOAP fault 714
The server is behind a TLS-terminating reverse proxy, so r.TLS is
nil and the URL builder emitted http://. Sonos does a HEAD probe to
detect the audio MIME type; against an http:// URL that 301s to
https://, the probe finds no audio body and bails with 714.
The Task 2 code-quality reviewer flagged this exact scenario at the
time. Closing it now: honor X-Forwarded-Proto + X-Forwarded-Host
before falling back to r.TLS + r.Host. Public URL the speaker
fetches now matches the scheme/host the client used to reach the
endpoint.
TestRoutesRegisteredInMount failed because handleGetStream did the
DB lookup (404 on missing track) BEFORE streamAuthOk (401 on
unauth). For an unauth request to a non-existent track, the test
saw 404 and concluded the route wasn't registered when actually it
was - the handler just bailed at the lookup before auth.
Reorder: extract trackID via chi.URLParam, run streamAuthOk on the
raw path id first (the HMAC token is signed over the same id
string so we don't need the resolved row yet), then do the DB
lookup. Test now sees 401 on the unauth probe as it expected.
Also closes a small info-leak: previously a 404/401 differential
let unauth callers probe which track IDs exist. Now both unknown
and known IDs return 401 for unauth requests.
golangci-lint flagged three errcheck:
- stream_token.go: fmt.Fprintf(mac, ...) - hash.Hash never errors
per documented contract, but errcheck wants explicit discard.
Discard via _, _ assignment with a WHY comment.
- config_test.go: os.Unsetenv calls in tests - discard the error
via _ assignment. Test cleanup paths.
Reviewers flagged the Fprintf one during Task 1 quality review but
golangci-lint runs in a separate CI step that wasn't exercised on
the per-task pushes (cancelled by subsequent push concurrency).
go vet caught the test's Mount call missing the trailing []byte
streamSecret arg added by the UPnP slice's Task 2. The test passed nil
for *playlists.Scheduler but didn't pass anything for []byte, so the
arg count was one short.
Added nil for the streamSecret position - the test exercises route
registration only, not the cast-token endpoint, so the secret value
doesn't matter for what this test asserts.
Operator feedback on the prior unification commit (7473e98d):
1. NewForYou should daily-rotate alongside Rediscover and FirstListens.
The 'newest album first regardless of day' intent was the wrong
call - operator wants visible day-over-day movement on every
deterministic mix surface. Spec flipped to dailyRotate: true.
2. Diversity caps (<=2 per album / <=3 per artist) on every mix, not
just the historically-diverse ones. The 2-per-album limit has
helped a lot on the operator's library; extending it to NewForYou
and FirstListens (previously album-coherent / no cap) surfaces
more distinct albums per day. Spec flipped to diversify: true on
all five.
3. Fallback when diversity caps strip the pool below the 100-track
target: finishMix now calls topUpFromRaw, which appends non-capped
tracks from the raw SQL pool (preserving original ranked order +
skipping duplicates) until the target is hit or the pool runs out.
On rich libraries the cap yields >= 100 and top-up never runs; on
thin / album-heavy libraries we ship a partly-diversified 100
instead of a strictly-diversified 40.
Net effect: every deterministic mix now rotates day-over-day, every
mix gets the same diversity treatment (with graceful degradation),
and the producer surface stays a single factory over a spec list.
The five discovery-mix producers (Deep Cuts, Rediscover, New for you,
On this day, First listens) were near-identical boilerplate that
differed only in (a) which SQL query they ran and (b) whether to
diversity-cap the result. Folded into one produceDiscoveryMix(spec)
factory + a per-mix discoveryMixSpec slice. The registry composes the
factory over the spec list so adding a new mix is one struct literal
+ a SQL query, never a new func.
Also fixes the user-reported bug that several mixes 'show the same
content from yesterday'. Audit of the SQL queries:
- Deep Cuts: ORDER BY md5(t.id::text || $2::text) → day-keyed
- On this day: ORDER BY w.c DESC, md5(...) → day-keyed
- Rediscover: ORDER BY tier, c DESC, id → invariant
- New for you: ORDER BY al.created_at DESC, disc, track → invariant
- First listens: ORDER BY tier, al.id, disc, track → invariant
The three invariant ones produced identical content day-over-day. The
unified spec carries a dailyRotate bool: when set, the producer
applies a daily-deterministic offset rotate-left of the candidate
pool BEFORE diversify+truncate. Rotation (not shuffle) preserves
contiguous-block ordering inside each day's slice — matters for First
listens which is album-coherent.
Set on Rediscover + First listens (where same-content-every-day is
clearly a bug). Left off New for you because 'newest album first
regardless of day' is the intended UX for that surface — daily
rotation there would feel wrong.
Daily rotation seed: rand.New(NewSource(int64(userIDHash(userID,
dateStr)))) — same primitive used by For-You's pickHeadAndTail
sampling so behavior is consistent across the system playlist family.
No test file referenced the deleted produceXxx functions directly,
only the registry, so this is a closed refactor.
Adds the client-facing endpoint that issues a signed stream URL for
the current track. Authenticated via the standard session cookie.
Returns {token, exp, url} where url is a fully-formed stream URL
the client passes verbatim to a UPnP / Sonos device's
AVTransport.SetAVTransportURI call.
expSeconds clamped to [60, 86400]; default 21600 (6h) - long enough
to play through any typical track without re-minting mid-playback.
MINSTREL_STREAM_SECRET is loaded from env var with a per-machine
fallback persisted at <Storage.DataDir>/stream_secret (auto-generated
on first boot via 64 random bytes, base64-url-encoded, 0600). The
file-based fallback is operator-machine-scoped runtime state, not a
user-facing setting - chosen over a DB column to avoid a migration
and keep the secret out of cross-instance restores. Operator can
override at any time via the env var; default path requires zero
config.
Tests cover happy-path token issuance + URL formatting, bad-UUID
rejection, unauthenticated rejection, the expSeconds clamp at all
boundaries, secret env override, auto-gen + file persistence at 0600,
second-boot reuse of the persisted file, and rejection of a malformed
env value.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds SignStreamToken / VerifyStreamToken (HMAC-SHA256 over
trackID|exp) and modifies handleGetStream to accept either the
existing session cookie OR a valid signed token. Stream route
moved out of the authed group so the handler's own auth check
runs and the token bypass is reachable.
Enables Sonos / UPnP speakers to fetch the stream URL without
carrying the user's session cookie - they cannot. The token is
short-lived (max 24h per the design); expiry checked at request
time only, not per-byte, so long tracks play through.
streamSecret field on handlers is nil for now; Task 2 wires the
loader (env var with auto-generated fallback persisted in
app_preferences).
Adds auth.OptionalUser - the permissive sibling of RequireUser
that attaches the user to context when a valid cookie / bearer is
present but does NOT 401 on absence. The stream route is wrapped
with it so the handler can fall through to the token path when
no session is present.
newLibraryRouter (test fixture) gets a synthetic-user middleware
on the stream route so existing media_test tests keep passing
without seeding a real session row - production traffic uses
auth.OptionalUser, the test path uses auth.UserCtxKeyForTest().
Five tests cover round-trip, tampered token rejection, expiry,
wrong-track-ID, and wrong-secret rejection. CI verifies.
Second go-round of the same shape of bug: tracks has file_size +
file_format NOT NULL (0002_core_library.up.sql) and my GC test seed
omitted both. The previous fix only addressed the artists.sort_name
column; the tracks INSERT was missing two more.
Use plausible stub values — the GC sweep only joins on track_id,
none of these columns affect what the test exercises.
The artists table requires sort_name (NOT NULL constraint added by
0009_artist_sort.up.sql). My GC integration test was inserting only
name + relying on a separate SELECT to pull the id back, which both
(a) violated the NOT NULL constraint and (b) was unnecessarily
indirect. RETURNING the id directly is the standard pattern used
everywhere else in the test suite.
Test now matches the real-world insert pattern in api.search +
library scan (sort_name mirrors name when no MBID-driven sort hint
is available). Other GC tests in this file don't touch artists so
they were already fine.
New `internal/gc` package with a single Worker that runs all five
lifecycle / retention sweeps from the 2026-06-02 drift audit on a
1-hour tick. Each sweep is small, idempotent (re-running on
already-clean rows is a no-op), and logs its affected-row count.
Sweeps (Scribe parent #552):
- **#566** GcCloseStalePlayEvents — play_events rows opened > 24h
ago that never got a play_ended (client crash, network drop).
Synthesizes ended_at from duration_played_ms when known, falls
back to now() so the row stops looking "open" to downstream
filters (ended_at IS NULL).
- **#565** GcClosePlaySessionsWithNoRecentEvents — play_sessions
with last_event_at older than 6h get ended_at = last_event_at
("user moved on"); empty sessions older than 1h get closed
too (stale handshakes from clients that never recorded a play).
The audit caught that the column was added but never populated
by any writer — every session row was "open" forever, breaking
downstream dedup queries that assume closed semantics.
- **#567** GcExpireScrobbleQueueFailedRows — drops scrobble_queue
rows in status='failed' older than 14 days. The worker stops
retrying after maxAttempts so these otherwise accumulate
forever on a persistent ListenBrainz outage / revoked token.
- **#574** GcResetStuckSystemPlaylistRuns — flips
system_playlist_runs.in_flight back to false on rows whose
last_run_at is older than 10 minutes. Catches goroutine-panic
wedges where the generator died between SET in_flight=true and
SET in_flight=false; the duplicate-prevention check refuses to
start a fresh regen while in_flight, so a stuck row would
otherwise deadlock all future regens for that user. Records
"stuck-row auto-reset by gc" in last_error so the operator can
tell auto-reset from a recent real failure.
- **#575** GcDeleteExpiredPasswordResets — deletes expired
password_resets rows. Unused expired rows go after a 1h grace
(gives the operator time to debug an active reset attempt);
used rows are kept 7 days for audit.
Wiring:
- main.go `go gcWorker.Run(ctx)` alongside the other periodic
workers (scrobble, similarity, lidarr).
- tickOnce fires once at start so a freshly-deployed server does
its initial sweep without waiting a full tick, matching the
scrobble worker pattern.
- Errors per sweep are logged but do NOT abort the remaining
ones — a transient pgx error from one query shouldn't prevent
the others from running.
Tests:
- 4 integration tests, one per UPDATE/DELETE sweep, that seed
rows-to-sweep + rows-to-leave-alone and assert the right rows
changed state. Skip unless MINSTREL_TEST_DATABASE_URL is set
(mirrors the api package pattern).
- Empty-tables no-op smoke test.
- Run() cancellation honoured (no spinning goroutine at
test-runner exit).
That's all five remaining server-side lifecycle findings from the
audit. The Android LOCAL_USER_ID hardcode (#576) is a separate
refactor that needs auth-store wiring and stays in the queue.
The docstring claimed "the next library scan reconciles missing
files by removing their tracks rows" — but scanner.go only does
filepath.WalkDir + UpsertTrack; it never enumerates existing rows
to check file_path presence, and it never DELETEs orphan rows. The
audit verified this — repo-wide grep finds no orphan-sweep code.
The lie is load-bearing: lidarrquarantine/service.go:270 leans on
this guarantee, so downstream code thinks the orphan case heals
itself. Fix the comment to state reality (admin re-trigger or
manual cleanup) and reference the open follow-up for adding a real
sweep. The actual reconcile pass is a separate piece of work
(needs scanrun integration + retention semantics + tests) and
stays in the Scribe audit queue.
Server-side fix for the drift audit finding (Scribe #578, parent
#552). Mirrored on Android (and Flutter) but the root cause and
the smallest blast-radius fix both live here.
The bug:
- Android MeApi.getProfile() calls GET /api/me and deserializes
into MyProfileWire which has nullable display_name + email.
- Server's handleGetMe was emitting the narrower UserView shape
(id, username, is_admin only).
- Android always saw displayName=null, email=null. The Settings →
Profile screen rendered BLANK form fields for users with stored
values.
- Saving from the blank state submitted empty strings to
PUT /api/me/profile, which interprets empty as "clear to NULL"
(me_profile.go:53-65) — DESTROYING the user's saved profile.
- Flutter (flutter_client/lib/api/endpoints/settings.dart:9-12)
has the identical bug pattern.
The fix:
- handleGetMe now emits profileViewFromUser(user) — the same
shape PUT /api/me/profile already returns (meProfileResp:
id, username, display_name, email, is_admin).
- auth.UserFromContext already returns a full dbq.User row, so no
extra DB lookup needed.
- Web's User TypeScript type is narrower than this response but
doesn't care about the extra fields (TS structural typing).
- LoginResp.User still uses UserView; login response unchanged.
New test asserts the regression directly: a user with stored
display_name + email sees them in /api/me. Old test updated to
decode into meProfileResp and assert the nullable fields are
correctly null for an unset profile.
Android side needs no change — the existing wire shape already
expected display_name + email; this just delivers them.
Five findings + one cancelled duplicate from the 2026-06-02 drift
audit (Scribe parent task #552):
- **#561 (Android)** PlayerController.playbackErrorEventsChannel was
Channel.CONFLATED. The PlaybackErrorReporter coroutine reads it in
a debounce loop that buffers events to coalesce into "Skipped N
unplayable tracks" — but CONFLATED silently dropped every emission
except the latest each time the reader wasn't actively pulling.
A network blip that failed 5 tracks back-to-back surfaced only the
last failure to the snackbar AND only POSTed one playback_errors
row to the admin inbox. Switch to BUFFERED (default capacity 64,
well above any plausible burst rate). Coalescing path now reaches
N > 1 and the admin inbox sees every failure.
- **#563 (server)** systemPlaylistSources rotation whitelist in
playevents/writer.go had drifted behind the migrations. It listed
only for_you + discover; migrations 0021 + 0028 added 6 more
variants (deep_cuts, rediscover, new_for_you, on_this_day,
first_listens, songs_like_artist) that ship as refreshable system
mixes. Plays from those surfaces never advanced the per-user
rotation, so "unplayed first" ordering staled — the same tracks
kept resurfacing. Add all 6 to the map; comment now points at the
migration's CHECK list as the canonical source so future variants
notice the requirement. #573 was the duplicate auditor hit for
the same drift; cancelled in Scribe.
- **#564 (Android)** Android emitted source = "playlist:<variant>"
for system-mix plays from Home and PlaylistDetail, but the
server's rotation matcher keys on the BARE variant string (web
sends the bare form — PlaylistCard.svelte:83). Misalignment meant
system-mix plays from Android never advanced rotation; switching
from web to Android effectively reset the perceived "unplayed
next" ordering. Fix HomeScreen.kt:291 to send bare variant and
PlaylistDetailScreen.kt's play() to prefer systemVariant over the
playlist:<id> tag when the playlist is a refreshable system mix.
User playlists keep playlist:<id> (intentional — rotation only
applies to system mixes anyway).
- **#568 + #569 (Android)** AuthCookieInterceptor was unconditionally
attaching the Minstrel session cookie to every outgoing request
AND wiping the session on any 401. The shared OkHttpClient is also
used by Coil for external image fetches (artwork.musicbrainz.org,
coverartarchive.org, Lidarr /MediaCover URLs); this leaked the
session cookie to those hosts (privacy posture) AND silently
signed users out of Minstrel if any external image host returned
401. Scope both attach + clear to the placeholder.invalid sentinel
host the same way BaseUrlInterceptor was scoped in aec10ce7. Two
new regression tests cover the external-host pass-through. Existing
tests rewritten to make requests through the placeholder URL so
they exercise the in-scope path explicitly.
All five Scribe tasks updated to in_progress at start, will flip to
done after CI green on this push.
Six findings from the 2026-06-02 multi-system drift audit (Scribe
parent task #552):
- **#553 (web)** Tailwind class fix: web admin playback-errors Delete
confirm button was using `bg-action-danger`, an undefined token —
swap to `bg-action-destructive` to match every other destructive
button. Restored the Oxblood signal that distinguishes Delete from
Cancel.
- **#555 (web)** Type the `source` field on `play_started` in the
EventRequest discriminated union. Server's eventRequest accepts it;
web's TS type was missing the slot, so a "drop extra properties"
refactor could silently strip the source tag and break system-
playlist rotation attribution.
- **#556 + #557 (server)** Coverage rollup whitelist was pinned to
('sidecar','embedded','mbcaa','theaudiodb'); migration 0020 added
'deezer' and 'lastfm' as valid cover_art_source values but those
never got wired in, so albums with art from those providers
silently counted as MISSING in the admin Coverage dashboard. The
rollup test was seeding only the pre-0020 sources, masking the
gap in CI. Extend the query to include deezer + lastfm; seed the
test with one row per valid source (regression-guards future
additions).
- **#558 (web)** Auth gate was blocking /forgot-password and
/reset-password/<token> — both are entered without a session by
definition, so the email-link reset flow was bouncing signed-out
users to /login. Add /forgot-password to the public set and a
/reset-password/ prefix matcher. New tests assert both routes
reach their pages without redirect.
- **#571 (server)** Library scanner was indexing only .mp3/.m4a/.flac
/.ogg while the stream handler (media.go) had been extended to
serve .opus, .aac, and .wav. A user with .opus files in their
library never saw them in artist/album listings because the
scanner skipped indexing — silent data loss. Aligned the scanner
to match the media handler.
Scribe statuses updated to in_progress; flipping to done after the
push since these are mechanical and verified directly against the
cited file:lines.
New client-reported playback-error log. Surfaces zero-duration
tracks (and future load_failed / stalled kinds) into an admin
inbox so the operator can hide / delete / re-request the
offending track.
Schema (migration 0032):
- playback_errors table with CHECK constraints on the kind +
resolution enums (per the standing rule that new enum values
need a migration to add)
- Partial index on unresolved rows for fast inbox lookup
- ON DELETE CASCADE from tracks + users so cleanup is automatic
Endpoints:
- POST /api/playback-errors: any signed-in user reports. Body
validates track existence + kind whitelist; client_id required
so support can correlate reports from the same device.
- GET /api/admin/playback-errors?resolved=false&offset=&limit=:
admin list with join to track/album/artist for table render
without per-row round-trips. Pagination capped at 200/page.
- POST /api/admin/playback-errors/{id}/resolve: admin marks
resolved with a resolution enum string.
Auto-resolve on Hide/Delete/Re-request from the inbox row is
driven from the web client (two sequential calls) — keeps the
existing track-action endpoints unchanged.
CI on 8b586c2e caught it: the $1::text cast in the md5 ORDER BY made
sqlc infer the parameter as plain string instead of pgtype.UUID,
generating Column1 string instead of UserID pgtype.UUID on the
ListRediscover*ForUserParams structs. go vet flagged both call
sites in internal/recommendation/home.go where the Go code passes
UserID by name.
Fix: hash on album_id (or artist_id) + current_date only. The
eligibility filter already differs per user (different liked sets),
so the daily-rotation goal is preserved; we just lose per-user salt
in the within-day ordering of overlapping items - acceptable.
Applies to both primary queries AND both fallback queries (all four
had the same cast).
User request 2026-06-01: the previous Rediscover only fired on
explicit album/artist likes, so users who only liked at the track
level got an empty Rediscover row. Also no daily rotation - the
section never changed between data updates - and 25 items felt long
for the Home carousel.
SQL (internal/db/queries/recommendation.sql):
- ListRediscoverAlbumsForUser UNIONs the existing explicit album-like
signal with a new track-derived path: an album qualifies if it has
>=2 liked tracks where the earliest like is >30 days old.
- ListRediscoverArtistsForUser does the same with threshold 3 (artists
span more releases, so the bar is higher to avoid one-hit-wonder
affinities).
- Both ordering switched from longest-since-last-play to a daily-
stable random hash md5(entity_id || user_id || current_date) so
the row randomizes but stays consistent within a day.
- Fallback queries also switched to the daily-stable hash so the
fallback rows don't reshuffle on every refresh.
Go (internal/recommendation/home.go):
- HomeRediscoverLimit dropped from 25 to 10 (carousel-sized).
- rediscoverInnerLimit (30) is the per-query cap; gives headroom so
the Go-layer filters don't undershoot.
- applyRediscoverAlbumFilters does cross-section dedup vs Most Played
(no point surfacing albums the user is actively spinning) plus a
diversity cap of max 2 albums per artist (prevents one liked-but-
forgotten artist dominating the row).
- applyRediscoverArtistFilters does cross-section dedup vs Most
Played's artist set; no diversity cap needed (one row per artist).
Tests (internal/recommendation/home_integration_test.go):
- TrackDerived path: 2 liked tracks >30d old + no recent plays = album
appears in Rediscover.
- Threshold guard: 1 liked track = album does NOT appear.
- Diversity cap: 4 explicit album-likes from same artist = 2 in output.
- Dedup vs MostPlayed: eligible album whose track is in MostPlayed
gets filtered out.
- Existing FallbackWhenSparse test preserved (semantics unchanged for
recent likes that miss the >30d primary filter).
Clients unchanged - they render whatever /api/home/index returns. No
parity-map row needed (this is a server-internal recommendation
change, not a layout divergence).
staticcheck S1016 in the new golangci-lint v2 flagged four sites
copying field-by-field between two types with identical struct
shapes. Direct type conversion is the canonical form:
- internal/library/scanrun.go:
- LibraryStageTallies copy from Stats → LibraryStageTallies(lastStats)
- MBIDBackfillStageTallies copy from BackfillMBIDsResult →
MBIDBackfillStageTallies(lastRes)
- internal/recommendation/home.go:
- dbq.ListRediscoverAlbumsForUserRow copy from the Fallback row
type → dbq.ListRediscoverAlbumsForUserRow(r)
- Same pattern for the Artists rediscover pair.
No behavior change; the underlying struct shapes are identical
(staticcheck verified the conversion is valid). Net -18 +4 lines.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
player: setQueueFromTracks fast-starts a single source at player-index 0
while the full queue is broadcast, so the transient currentIndexStream→0
emission clobbered the correct mediaItem with queue.value[0] (the first
track). Mini bar / playlist marker pinned to the wrong track until a
later index event (~the "passive ~30s recovery"). Track a logical-index
base so the player→queue mapping stays correct during the fill window;
also fixes the latent forward-fill auto-advance off-by-base.
lidarr #50: approving no longer fails when Lidarr is down. Approve
records the decision durably first, then best-effort adds; the
reconciler idempotently (re)sends unconfirmed adds every tick until they
stick (new additive lidarr_add_confirmed_at; AddArtist/AddAlbum map
Lidarr's "already exists" 400 → ErrAlreadyExists). No failed-state or
expiry by design — Lidarr keeps trying, operator monitors.
lidarr #51: Create() is now idempotent — a non-terminal request for the
same MBID returns the existing row instead of inserting a duplicate.
Rewrites the obsolete LidarrUnreachable_503 test to assert the durable-
approve contract; threads a client factory into NewReconciler.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operator decision: the enricher is canonical. No MBID still runs the
provider chain (name-based providers — Deezer/Last.fm — resolve
without an MBID); if every provider returns ErrNotFound the row
settles cover/artist source 'none' at the current sources version
(re-eligible only when the registered provider set changes). It does
NOT skip-and-leave-NULL.
The two _NoMBID_LeavesNull tests predated the name-based providers
(0020 slice) and asserted the old skip→NULL contract. Updated:
- TestEnrichArtist_NoMBID_SettlesNone: stub now returns ErrNotFound
(realistic MBID-only-provider-with-empty-MBID), expect source 'none'.
- TestEnrichAlbum_NoSidecarNoMBID_SettlesNone: empty registry →
allWere404 stays true → expect 'none'.
Last failing cluster from the CI-integration initiative; suite should
now be fully green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The test marked id2 'none' via SetAlbumCover, which (covers.sql:42)
does NOT set cover_art_sources_version. ListAlbumsMissingCover treats
'none' AND version != current as eligible, so id2 (version 0, current
1) was wrongly drained → processed=2. The comment's intent ("'none'
with current version → not drained") requires SetAlbumCoverWithVersion
(albums.sql) stamped with the live GetCurrentSourcesVersion — the same
value EnrichBatch compares against. Test-only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two distinct pre-existing test bugs in the last failing cluster:
1. newTestEnricher() called resetRegistryForTests() itself, wiping the
fake providers callers Register() right before calling it (its own
doc says callers register first and reconcile() picks them up). The
~8 TestEnrichArtist_* failures (source stayed NULL, no thumb
written) all stem from reconcile() seeing an empty registry. Remove
the internal reset; make every caller that lacked one own the
registry lifecycle explicitly (resetRegistryForTests + t.Cleanup):
SidecarFound, NoSidecarNoMBID, AlreadySidecar_NoOp,
DrainsNullSourceOnly.
2. apiTestAlbumProvider.FetchAlbumCover had a stale signature
(context, string) predating the AlbumRef refactor; it no longer
satisfied coverart.AlbumCoverProvider, so the p.(AlbumCoverProvider)
capability assertion failed and TestAdminListCoverSources got
supports=[]. Fix the param to coverart.AlbumRef.
Test-only. (A1/A2 were correct; they unmasked these. Any residual
album-enricher semantics failures will be root-caused from the next
clean run, not bundled speculatively.)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A2 — migration 0030: the albums/artists art `*_source` CHECK was a
fixed provider-ID allowlist fighting the extensible coverart.Register
registry (per-provider migration churn at 0016/0018/0020; rejected
test stub providers → ~11 enricher tests failed). Relax to "NULL or
non-empty"; the registry is the source of truth. Same brittleness
class as the #433 discovery-mix CHECK.
D — lidarr Approve resolves metadata/quality profiles (JSON arrays)
before the add; the test stubs returned {"id":1} for every path, so
ListMetadataProfiles failed to unmarshal → 500/Approve errors. Make
the lidarrrequests + admin_requests stubs path-aware (arrays for
/metadataprofile, /qualityprofile).
E:
- auth: requireUser (prelude.go) emitted code "auth_required"; the
canonical code is "unauthenticated" (operator decision). Change the
code; drop the now-dead "auth_required" web error-copy key
("unauthenticated" already has copy).
- playlists_system_test wrapped the real auth.RequireUser middleware
but only injected withUser() context → 401. Like every other api
handler test, drop the middleware and rely on requireUser().
- admin_users dup-username test seeded "test-existing" (seedUser
prefixes) but POSTed "existing" → no collision; POST the prefixed
name.
- me_timezone test decoded a top-level {"code"} but the envelope is
{"error":{"code"}}; decode the nested shape.
- audit test asserted compact JSON; Postgres jsonb::text is spaced.
- put-lidarr-config tests predated the missing_defaults gate (correct
handler behavior); supply the required defaults in the bodies.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A redo: the prior commit truncated cover_art_sources_meta, but
SettingsService.reconcile() only READS that singleton (seeded once by
migration 0018) and never recreates it → ~45 coverart/api tests hard-
failed "get current sources version: no rows". Correct reset: keep
cover_art_provider_settings in the truncate set (boot idempotently
re-UpsertProviderSettings), drop cover_art_sources_meta from it, and
instead `UPDATE cover_art_sources_meta SET current_version = 1` so the
row survives while cross-test version accumulation is cleared.
B residual (exposed once the play_events FK fix let these run):
- seedQuarantine used reason 'test-hide', invalid for the
lidarr_quarantine_reason enum (bad_rip/wrong_file/wrong_tags/
duplicate/other) → use 'other'.
- TestBuildSystemPlaylists_SufficientActivity predated #411/#352: the
variant switch errored on the 5 new seedless mixes and capped
track_count at 25. Accept deep_cuts/rediscover/new_for_you/
on_this_day/first_listens as seedless; raise the cap to 100.
Test/test-harness only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Latent failures exposed now that integration tests run in CI (#339).
All test/test-harness only — no production code changes.
A (dbtest.ResetDB): also truncate cover_art_provider_settings +
cover_art_sources_meta. The monotonic source-version counter (seeded 1
by 0018) accumulated across internal/coverart tests → CurrentVersion=4
want 1, key-only-bump assertions, enricher source skips. SettingsService
re-seeds both at boot, so a truncated start is the correct fresh state.
B (playlists system_test.seedPlayEvent): inserted play_events with a
random session_id → play_events_session_id_fkey violation (7 tests).
Create the parent play_sessions row in the same statement (CTE).
C (similarity worker_integration_test.newTestWorker): built the client
with only BaseURL. Post-4fca0e6 similarity hits the Labs API via
LabsBaseURL, so an unset LabsBaseURL fell through to the real labs.api
and the stub never ran → 0 similarity rows. Set LabsBaseURL to the stub.
F (library scanner_test): NOT a flake — deterministic. Synthetic
ID3-only MP3s have no decodable duration; the scanner intentionally
won't skip duration_ms=0 rows (retries duration backfill), so every
re-scan reported Updated not Skipped. Seed duration_ms>0 before the
second scan so the mtime-based incremental-skip path under test is
actually exercised. Production scanner behavior unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Out-of-library suggestion artists (artist_similarity_unmatched rows)
have no local art row, so the Discover card always showed a
placeholder. Resolve art on-demand from Lidarr's artist lookup,
matched by MBID (foreignArtistId == candidate_mbid), and pass the
remote image URL straight through — no caching (a suggestion may
never be viewed; the browser fetches the URL directly).
- suggestionView gains image_url (omitempty); handler resolves it via
bounded-concurrency Lidarr LookupArtist, best-effort, never fails
the request.
- Lidarr is the sole source: disabled / unreachable / no-match →
empty → existing placeholder. (No inline TheAudioDB fallback — it's
externally rate-limited and there's no cache to amortize it.)
- web: ArtistSuggestion.image_url?; SuggestionFeed passes it to
DiscoverResultCard (already supports imageUrl).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follows 005965d (#388), which removed the coverArtBackfillCap param
from server.New. server_test.go is in package server so it calls New()
unqualified — missed by the signature-caller grep. Updated all 6
positional calls.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operator feedback (2026-05-09): the 500-album cap meant cover art rolled
in over many nightly runs even when local sources (sidecar/embedded)
could finish in minutes. Remove the global cap; rely on the existing
per-provider HTTP throttle (coverart httpClient MinInterval) so local
art is disk-speed and remote providers stay TOS-friendly.
- enricher.go / artist_enricher.go: EnrichBatch, EnrichArtistBatch,
EnrichRetryMissing now treat limit<0 as unbounded (0 still = stage
disabled). The cap was a SQL LIMIT; unbounded uses max int32.
- main.go: RunScanConfig EnrichCap/ArtistEnrichCap = -1 (unbounded).
- Drop LibraryConfig.CoverArtBackfillCap + the
MINSTREL_LIBRARY_COVERART_BACKFILL_CAP env var.
- Drop the now-dead coverBackfillCap param threaded through
server.New + api.Mount + the handlers struct.
- Admin bulk refetch (/api/admin/covers/refetch-missing) now drains
unbounded; response {queued:int} → {started:bool} (the count is
unknowable synchronously for a fire-and-forget drain). Web copy +
client type + Go/web tests updated to match.
No doc refs existed (config.example.yaml / docker-compose / README
never documented the env var).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root cause of zero LB recommendations (every similar-recordings AND
similar-artists call returned HTTP 404, worker logs):
- Wrong host/path: client called
api.listenbrainz.org/1/explore/similar-{recordings,artists}/{mbid}.
/explore/... is a WEBSITE route, not an API endpoint — it 308s then
404s. Similarity datasets live on the separate Labs API.
- Invalid algorithm: the hardcoded
session_…_session_30_…_limit_100_filter_True_… is not a permitted
Labs enum member (400s) regardless of host.
Verified against the live Labs API:
GET labs.api.listenbrainz.org/similar-recordings/json
?recording_mbids=<mbid>&algorithm=<algo>
GET labs.api.listenbrainz.org/similar-artists/json
?artist_mbids=<mbid>&algorithm=<algo>
algorithm=session_based_days_9000_session_300_contribution_5_threshold_15_limit_50_skip_30
→ 200 for both. Response field names (recording_mbid/artist_mbid/
name/score) already match the existing structs — parsing unchanged.
- Add defaultLabsBaseURL + Client.LabsBaseURL (separate from the main
BaseURL; scrobble submission still uses api.listenbrainz.org).
- Drop the count/limit query param — result size is encoded in the
algorithm name (limit_50); caller still applies its own top-K.
- Tests: newTestClient sets LabsBaseURL; the two *_LimitParamSet tests
become *_MbidParamSet (assert the Labs path + mbid query param).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follows ca1bc5a, which raised the worker batch default. The defaults
test pinned the old value; align it with the intended new default.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- similarity.Worker batch 5→25 (tracks AND artists per 1h tick). At
5/h a freshly-scrobbled library took days to build a usable
similarity pool; 25/h converges in hours, still well under
ListenBrainz rate limits (429 aborts the tick).
- scanrun Stage 2b track backfill now runs unbounded (-1) instead of
reusing the album backfill's 5000 staged cap. It's a one-time
whole-library heal with no progress UI; a cap just left tracks.mbid
partially NULL (3634/18056 after one pass) until several future
scans caught up, re-reading untagged files each time. One uncapped
pass converges; later scans only re-read the remaining NULL rows.
Album backfill keeps its 5000 cap (it has staged scan_runs UX).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tracks.mbid was 100% NULL: the scanner only extracted album + artist
MBIDs, never the recording MBID. The ListenBrainz similarity worker is
gated on tracks.mbid IS NOT NULL, so track_similarity could never
populate — starving For-You/radio's strongest candidate source
(lb_similar) on every library.
- mbids.go: add extractRecordingMBID (mbz.Recording / Picard
musicbrainz_recordingid). Separate fn so extractMBIDs' signature +
unit tests stay untouched.
- scanner.go: persist recording MBID via UpsertTrack (heals on the
file_path conflict, so re-scans backfill for free).
- BackfillTrackMBIDs: one-shot pass mirroring BackfillMBIDs, wired as
scan Stage 2b (idempotent via SetTrackMbidIfNull, gated by
BackfillCap, log-only progress).
- migration 0029: tracks_mbid_unique (0002, written when the column
was always NULL) wrongly assumes one MBID == one track. A recording
appears on multiple releases, so rows legitimately share a recording
MBID. Replace with a non-unique partial index. Zero-risk: column is
100% NULL at migration time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
For-You silently vanished after ~7 days of not listening (seed query
required a non-skip play in the last 7 days) and capped at ~40 on
self-hosted libraries with no ListenBrainz similarity data.
- PickTopPlayedTracksForUser: tiered seed — last 30d top plays, else
all-time top plays, else liked tracks. For-You only disappears now
if the account has zero plays AND zero likes.
- produceForYou uses deeper candidate source limits (raised random/
tag/similar K) so it reaches ~100 even with empty lb_similar /
similar_artists; richer when LB enrichment is present.
- Rediscover: tiered — 6-month-dormant, else ≥5-play 30-day-dormant.
- On This Day: floor 60→30 days, window ±7→±10 doy; still skips
cleanly (no rows → no playlist) on insufficient history.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HOTFIX for v2026.05.15.0. R3 added deep_cuts/rediscover/new_for_you/
on_this_day/first_listens producers + registry entries but not their
system_variant values to the playlists_kind_variant_consistent /
playlists_seed_consistent CHECK constraints (0021's whitelist).
insertSystemPlaylist for any new mix → SQLSTATE 23514, and since
BuildSystemPlaylists is one all-or-nothing txn that aborts the
ENTIRE build — For-You/Discover refresh 500s and the daily lazy
build fails too. System playlists are fully broken in prod.
Migration 0028 drops + re-adds both constraints with all five new
seedless variants (mirrors the 0021 drop-and-readd pattern).
CHECK-only — sqlc/dbq unaffected (regen produced no drift).
Bumps to 2026.5.15+7 for the v2026.05.15.1 patch release.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Headline of S4. "Shuffle all" is always present (home app-bar
shuffle icon); the pool degrades with reachability:
- online → GET /api/library/shuffle?limit=N (new): N random
library tracks server-side, per-user quarantine filtered
(ListRandomTracksForUser, ORDER BY random()).
- offline → ShuffleSource shuffles the whole local cache index
(audio_cache_index ∩ cached_tracks, names from cached_artists/
albums) — a UNION over liked AND recently-played, since the
two-bucket split is storage-only and never filters playback.
Offline gating: refreshable (singleton) system playlists need the
live build/shuffle endpoints, so their tile play is disabled when
offlineProvider is true — Shuffle all is the offline path. User
playlists still play from cache.
playlist_card now reads offlineProvider in build → wrapped the
direct-render widget test in ProviderScope (offlineProvider is
smoke-safe from S1: tracked timers, no connectivity mount).
S4b (offline Recently-played / Liked browsable surfaces over the
cache index) next.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each is one candidate query + one registry entry; zero client work
(R2 made tiles/refresh/shuffle generic, all are singleton kinds so
web's server `refreshable` flag and Flutter's derived getter both
light up automatically).
- deep_cuts (#419): <=2-play tracks from liked / heavily-played
artists; diversity-capped.
- rediscover (#420): >=5-play tracks not heard in 6 months, by
historical affection.
- new_for_you (#421): tracks from albums added <=30d whose artist
the user likes/plays; album-coherent (no cap).
- on_this_day (#422): tracks played within ±7 day-of-year in prior
windows (>60d ago), weighted by play count.
- first_listens (#423): never-played albums, tiered liked-artist →
played-artist → rest; album-coherent.
system_mixes.go producers mirror the Discover model (SQL gives the
ranking; finishMix caps+truncates to 100 to match For-You/Discover
shuffle depth; album-coherent mixes skip the cap). Builder query
failure is non-fatal (logged, yields no playlist) like Discover.
Existing system_test existence checks are unaffected.
Closes the #411 system-playlists-v2 umbrella's new-types thread.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The commented Refreshable field broke gofmt's struct-tag column
alignment in playlistRowView. Pure formatting.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
go vet broke because playlists_{discover,foryou}_refresh_test.go
referenced the handlers/types deleted in R2 (d67c0de). Consolidated
into playlists_system_test.go covering the generic
/playlists/system/{kind}/{refresh} endpoint: for_you + discover
200/shape, non-singleton & unknown kind → 404, no-auth → 401.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the per-kind refresh/shuffle handlers with one generic
pair driven off the kind registry, in lockstep across both clients.
Server:
- systemPlaylistKind gains Singleton; RefreshableSystemKind(key)
exported. for_you/discover singleton; songs_like_artist not.
- New generic POST /api/playlists/system/{kind}/refresh and
GET /api/playlists/system/{kind}/shuffle ({kind} = raw
system_variant). Non-singleton/unknown kind → 404. Deleted
playlists_{foryou,discover}_refresh.go and the per-kind shuffle
wrappers; serveSystemPlaylistShuffle core kept.
- playlistRowView.refreshable: server-derived flag so clients show
the refresh affordance generically without hardcoding kinds.
Web (not drift-cached → uses the server flag):
- refreshSystem(variant) replaces refreshForYou/refreshDiscover;
systemShuffle drops the for_you→for-you mapping (raw variant).
- PlaylistCard + detail page gate the kebab/Refresh button on
playlist.refreshable; label is "Refresh {name}". Tests reworked;
obsolete refresh-foryou/discover api tests deleted.
Flutter (list tiles are drift-cache-sourced → derive, no migration):
- Playlist.refreshable getter = isSystem && variant !=
songs_like_artist (holds for all current + planned kinds).
- refreshSystem/systemShuffle use the raw variant; PlaylistCard +
detail screen gate kebab/Regenerate/source-tagging on refreshable
so songs_like_artist plays via get() (no by-kind endpoint).
Pure-plumbing refactor; CI verifies parity. Next (R3): the five
discovery mixes — each a candidate query + one registry entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
revive unused-parameter: produceDiscover keys off dateStr, not now,
but must keep the uniform systemPlaylistProducer signature. Blank
the unused param (param names don't affect func-type identity).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Behavior-preserving prep for the new mix types. Extracts the three
inline candidate computations in BuildSystemPlaylists into
producers (produceForYou / produceSeedMixes / produceDiscover) and
drives the build off a systemPlaylistRegistry. The shared
machinery (run-claim guard, atomic delete+insert tx, post-commit
collages) is now generic over a []builtPlaylist.
Fatal-vs-skip error semantics unchanged: a base query failure
(PickTopPlayedTracksForUser, PickSeedArtists) still aborts the
whole build; candidate-load / per-seed-artist / Discover-bucket
failures are still logged and just yield fewer playlists.
Materialize order (for_you, songs_like_artist, discover) is
unchanged and functionally irrelevant.
No API/client/schema change — CI's system/foryou/service tests
verify For You / Songs-like-X / Discover parity. Adding a new mix
is now: a producer + one registry entry + its candidate query.
Next (R2): generic /api/playlists/system/{kind}/{refresh,shuffle}
off the registry; then the new kinds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Server half of the offline-replay capture. New writer path
RecordOfflinePlay: writes a complete start+end play in one txn from
a caller-supplied `at` + duration_played_ms, applying the spec §6
skip rule (same AND-of-thresholds as RecordPlayEnded) and threading
`source` so #415 rotation advances for system-playlist plays just
like the live path. Generalizes RecordSyntheticCompletedPlay (which
hard-codes full completion). duration clamped to [0, track len].
New /api/events type "play_offline" → handleEventPlayOffline:
validates track + duration, reuses the existing req.At parse so the
play lands on the original timeline, not replay time. Subsonic
shim + live 3-call lifecycle untouched.
Flutter half next: EventsApi.playOffline, a play.offline
MutationQueue kind, and PlayEventsReporter capturing the completed
play + enqueuing it when the live calls have no server id / fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GET /api/playlists/system/{discover,for-you}/shuffle returns the
caller's system playlist with tracks ordered: unplayed-this-rotation
first (shuffled), then already-heard (shuffled). When the whole
snapshot has been heard, ResetRotationState fires and the full list
reshuffles fresh.
Option A (operator's choice): a separate, intentionally-uncached
endpoint. The cached GET /api/playlists/{id} detail path stays pure
for "open to view"; this varies per play. Same JSON shape as the
detail GET so Stage 3 clients reuse track parsing with no new model.
Two explicit static routes per variant mirror the refresh handlers
and avoid chi static-vs-param ambiguity under /playlists/system/.
Empty/absent snapshot → 200 with empty track list (nothing to play,
not an error). Rotation reset failure is non-fatal — still returns a
playable reshuffled list.
No client wiring yet — Stage 3 makes web + Flutter call this on the
play/tile gesture and send `source` on play_started. Handler-level
test deferred to Stage 3 (needs the full service+pool harness; the
end-to-end path is exercised there).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First of three stages for system-playlist sample-history dedup
(Fable #415, server-side rotation per the operator's choice).
Schema (migration 0027):
- play_events.source (nullable text): which surface a play came
from. 'for_you' / 'discover' feed rotation; NULL for library /
user-playlist / radio / Subsonic.
- system_playlist_rotation_state(user_id, playlist_kind,
played_track_ids uuid[], rotation_started_at, updated_at): the
per-(user,kind) set of already-heard tracks this rotation.
Ingest:
- New RecordPlayStartedWithSource on the writer; RecordPlayStarted
is now a thin source="" wrapper so the frozen Subsonic shim is
untouched (no signature ripple).
- When source is a known system kind, the same txn appends the
track to rotation state (AppendRotationPlayed keeps the array a
set via the conflict CASE).
- /api/events play_started accepts an optional "source".
No serve-behavior change yet — Stage 2 makes shuffle prefer the
unplayed tail + resets on exhaustion; Stage 3 wires the clients to
send source and consume the rotation-aware order.
Tests: rotation appends + dedupes for a system source; source-less
play writes no rotation row. (Existing RecordPlayStarted tests are
unchanged — same wrapper signature, identical behavior at source="".)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes Fable #414. forYouHeadN/forYouTailN go 12/13 → 50/50 so the
For-You snapshot is 100 tracks, matching Discover. Motivated by the
shuffle-on-play default that just shipped (#413): a 25-track shuffle
pool repeats fast; 100 makes re-plays within a day feel varied.
pickHeadAndTail already degrades gracefully when the candidate pool
is too thin for a full head/tail split (returns top-N-by-score),
mirroring how Discover returns <100 when its buckets are thin — no
new edge-case handling needed. No build-path test asserts the
For-You total; pickHeadAndTail unit tests pass their own head/tail
values so they're unaffected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Discover playlists could surface the same track twice with the
duplicates landing back-to-back — a "first song plays, then plays
again, skip works" symptom user reported on v2026.05.13.0. Root
cause: interleaveBuckets rotates one track per pass per bucket but
never tracks which IDs it has already emitted, so a track that's
both a dormant-artist pick AND a random-unheard pick comes out
once from each bucket.
On a single-user server the crossUser bucket is empty, so the
redistribute step rolls its slots into dormant + random. Their
output then interleaves d0, r0, d1, r1, … — and when d0 == r0
(common: a dormant-artist track is also valid for random-unheard)
the result is [X, X, …] with adjacent duplicates.
Fix: track seen track IDs across all buckets while interleaving;
skip already-taken IDs and advance to the next index in that
bucket. Dedup priority is bucket order, so a track in both
dormant and random comes from dormant.
Regression test covers the single-user case directly. The existing
round-robin test still passes — no shared IDs in that fixture.
Note: stale duplicates already written to drift / served as cached
playlists will clear naturally on the next playlist rebuild (the
03:00-local refresh, or any manual /api/me/playlists/refresh).
bvandeusen