New diagnostic_events table + per-account users.debug_mode_enabled flag.
When an account's flag is on, its client(s) POST a batch timeseries of
connectivity / UPnP-sync / power / lifecycle events to /api/diagnostics
(no-op 204 when off, kind whitelist mirrors the CHECK constraint).
Admin surface: GET /api/admin/diagnostics (optional account/device/kind/
time-window filters, RFC3339-or-epoch-ms, export-sized paging) + a
/diagnostics/devices overview + PUT /api/admin/users/{id}/debug-mode to
flip an account remotely while a bug is live. debug_mode_enabled is now
exposed on /api/me (client gate) and the admin user views.
Retention: a 30-day gc-worker sweep (GcPruneDiagnostics), keyed on the
server clock so a skewed device clock can't keep rows alive.
Refs Scribe M9 (#119), tasks #1172#1173.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K55iTxn95BtshocgdE1shW
The daily 03:00 scheduler rebuild (and the manual refresh endpoint) replace
a user's system playlists + You-might-like rows but published no event, so a
client left open across the rebuild served yesterday's snapshot until a
manual reload — the stale-tab case behind #968. Add a user-scoped
playlist.system_rebuilt event (envelope {kind,user_id,data:{}}) from both the
scheduler (bus threaded into NewScheduler) and handleSystemPlaylistRefresh.
Clients consume it to invalidate home / system-playlist views and proactively
re-pull a stale active queue. Issue #968.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make a fresh install usable out of the box and document the first-run flow
for the public-facing repo.
- Default scan_on_startup to true (Default() + config.example.yaml, which is
the live config baked into the image). Previously false, so a fresh stack
came up with an empty library and no hint to scan. Scans are incremental
(mtime skip), so the per-restart cost is just a directory walk. Re-point
the env-override test to exercise the override against the new default.
- README: add a "First run" walkthrough (register -> scan -> integrations ->
install Android app -> invite users), each grounded in a real route.
- Add docs/screenshots/ with six captures, referenced via width-constrained
<img> wrapped in a link (shrink inline + click to open full size).
API token and invite token were cropped/redacted out of the captures
before commit so no live credential lands in the public history.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per-source play outcomes so the operator can see whether each recommendation
surface is landing and tune the now-operator-tunable taste weights.
Server:
- query RecommendationSourceMetricsForUser: groups the user's play_events by
source (system-playlist surface), reporting plays / skips / avg completion
over a window; NULL-source (library/radio) plays excluded.
- GET /api/me/recommendation-metrics?days=30 (default 30, capped 365) →
{window_days, sources:[{source, plays, skips, skip_rate, avg_completion}]}.
- handler test: 401 unauth; per-source aggregation + NULL-source exclusion +
skip_rate / avg_completion math.
Web:
- lib/api/metrics.ts: query + friendly source labels.
- settings page gains a "Recommendation metrics" card (table of surface / plays
/ skip rate / avg completion), with loading/error/empty states.
- settings tests mock the new query (manual subscribe-store, hoisting-safe).
Note: You-might-like plays aren't source-tagged (it's a Home row, not a system
playlist), so this covers For-You / Discover / the mixes. Tagging YML plays
would be a client follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2a re-ranks the existing pool by TasteMatch; this ensures taste-relevant tracks
ARE in the pool. Adds a 6th arm to LoadRadioCandidatesV2: in-library tracks by
the user's top positively-weighted taste-profile artists ($10 K, weight > 0,
deterministic weight-DESC,id order so it doesn't reintroduce same-day
nondeterminism). Pool-inclusion only (sim_score 0) — TasteMatch already scores
the fit. Empty for cold-start users (no profile).
- CandidateSourceLimits.TasteOverlap; default 20 (radio), 80 for For-You via
systemForYouSourceLimits.
- You-might-like deliberately sets TasteOverlap=0: it surfaces NOT-actively-
engaged artists, so flooding its pool with top-taste (mostly already-played)
artists would just feed the read-time dedup.
- Test: positive-weight artist's track enters via the arm; negative-weight one
is excluded (weight > 0). Existing pool tests unaffected (no profile seeded).
Deferred within 2b: profile-seeded For-You — marginal given the arm + TasteMatch
already inject taste broadly (top-played seed ≈ top-taste artist).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The fallback pulled artists only from explicit artist-likes (general_likes_artists),
but most users like albums and tracks far more than artists — so the artists row
still came up thin (a couple of tiles) even with a rich library, while the albums
row filled fine.
Broaden both fallbacks to "entities you've shown affinity for":
- artist fallback = explicit artist-likes ∪ artists of liked albums ∪ artists of
liked tracks.
- album fallback = explicit album-likes ∪ albums of liked tracks.
New dedicated queries (ListYouMightLike{Artist,Album}FallbackForUser) replace the
narrow Rediscover-fallback reuse; same projection so the Go layer still converts
directly. (Aliased + fully-qualified the UNION arms — sqlc merges UNION scopes,
so unqualified user_id was ambiguous across the three like tables.)
Test: 12 liked TRACKS by distinct artists, no artist-likes → the artist row now
fills from their artists (was empty before).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
golangci-lint v2 (CI-only; local is v1) flagged the field-by-field struct
literals — the fallback and you-might-like row types are identical, so convert
directly instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The taste roll-up surfaces top-similar albums/artists, which for a heavy
listener are mostly ones they already play — so the read-time dedup (vs Most
Played + Rediscover + Last Played) can strip the section down to a single tile
(reported on the artists row). The code was sound; the section was just starved.
Adds a read-time fallback: when a You-might-like row comes up short after dedup,
top it up from the user's LIKED artists/albums — a far larger pool than the
12-entity similarity roll-up, so the same exclusions still leave plenty. Reuses
the existing Rediscover-fallback queries (no new SQL), applies the same
exclusions (already-shown + Rediscover + Most/Last Played) so it never
duplicates a tile or suggests an actively-played entity, and is best-effort
(a query error leaves the section as-is). Takes effect immediately — no rebuild.
A cold-start user with no likes gets nothing from the fallback, so the
new-user-empty behaviour is preserved (test still passes).
Test: 20 liked artists, none played → Rediscover fills 10, You-might-like
fallback fills the other 10, disjoint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The profile built in phase 1 now changes what gets surfaced. Adds a TasteMatch
term to the weighted-shuffle score so candidates are re-ranked by their fit to
the user's learned taste (positive draws toward it; negative reflects passive
avoidance; 0 at cold start).
- recommendation/score.go: ScoringInputs.TasteMatchScore ([-1,+1]) +
ScoringWeights.TasteWeight + the term in Score.
- recommendation/taste.go: LoadTasteProfile reads the taste_profile_* tables;
TasteProfile.Match blends the candidate's artist weight (0.7) and avg genre-tag
weight (0.3), each tanh-squashed by a fixed scale so one outlier artist can't
compress the rest. Unknown artist/tags and empty profiles → 0 (neutral).
- candidates.go: both candidate loaders set TasteMatchScore per candidate, so
every Score caller (system playlists incl. You-might-like, radio) becomes
taste-aware automatically.
- weights: systemMixWeights.TasteWeight = 1.5 (daily mixes are the primary
taste surface); config.RecommendationConfig gains taste_weight (default 1.0,
lighter — radio is seed-directed) wired into the radio handler.
- tests: pure (Match curve incl. saturation/clamp/empty-neutral, Score term
add+subtract) + DB round-trip (seed taste rows → Match positive). All green
vs real Postgres; existing playlist/radio tests unaffected (empty profile →
zero taste effect).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
golangci-lint v2 (CI-only; local is v1) flagged two unused-parameter issues:
- BuildTasteProfile's `now` was genuinely dead — decay/windowing are computed
DB-side via now(), so no Go-side timestamp is threaded. Removed it (a
phase-3 context model that needs a pinned reference time would re-add it);
updated the scheduler call site.
- the degenerate-params engagement test ignored t; reworked it to assert the
result stays in [-1,1], which also strengthens the test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Build a persistent, decaying model of each user's taste, recomputed daily,
that later phases consume across every recommendation surface. Phase 1 only
BUILDS the object — no behaviour change to what's surfaced yet.
Core mechanic — graded engagement (replaces binary was_skipped for learning;
was_skipped stays for History): a play's completion ratio maps to a signal in
[-1,+1] via two linear ramps (instant-skip → -1, ~0.30 neutral, ≥0.90 → +1).
Time-decayed (half-life ~75d) so recent behaviour dominates and the profile
tracks drift.
Per operator constraints:
- No explicit dislike button — negatives come only from passive behaviour
(early skips). Nothing recorded to regret or opt out of.
- Negatives are track-scoped; artist/tag weight is the decayed SUM of their
tracks' engagement, so one skip nets out against many good plays (a
DB test asserts a liked artist stays positive despite an early-skipped
track). A floor clamp bounds how negative any single entity can get.
- migration 0035: taste_profile_artists / taste_profile_tags (signed weight,
indexed by (user, weight DESC)).
- internal/taste: engagement.go (pure curve + decay) + profile.go
(accumulate plays + like bonuses, floor damping, size caps, atomic-replace).
- scheduler: rebuildUserDaily recomputes the profile before the playlist
build (so phase 2 can read it), best-effort — a taste failure never blocks
playlist building. Wired into the daily job + startup catch-up only (not
manual/lazy rebuilds).
- tests: pure (engagement curve, decay, ranking, floor, genre split) +
DB-backed (positive/negative weights, aggregation-protects-artist, like
bonus, atomic replace). All green vs real Postgres.
Config knobs live in taste.DefaultConfig() for now; wiring them into the
server RecommendationConfig is a later follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scoreAndSortCandidates drew per-candidate jitter by slice position, but
the candidate query (LoadRadioCandidatesV2) has ORDER BY random() arms and
no stable outer ordering, so DB row order varies call-to-call. When the
recency spread between candidates is smaller than the ±jitter (small or
recency-clustered libraries), two same-day rebuilds assigned jitter to
different tracks and reordered near-ties — so the build was not actually
deterministic-within-a-day as documented.
Pre-existing latent flake in TestBuildSystemPlaylists_DailyNonceDeterminism
(passed in isolation / by luck in CI; deterministically reproduced when the
system-build tests run in sequence). Confirmed independent of the
You-might-like change by neutralizing buildYouMightLike — the flake
persisted.
Fix: sort the candidate slice by track id before assigning jitter, so the
jitter for a track is a function of (track, day) alone, independent of DB
return order. Verified: full playlists package green 4/4 and the build-test
sequence green 5/5 (was 0/4 before).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surface in-library albums/artists the listener doesn't actively spin but
is predicted to enjoy, derived from the same similarity + like-weighted
candidate engine that powers For-You — rolled up from track scores to
album/artist granularity. Built in the daily 3am BuildSystemPlaylists
pass, atomic-replaced alongside the system playlists, and read back by
/api/home (+ /api/home/index).
Cold-start gate: skips generation entirely below 20 distinct unskipped
tracks AND 5 distinct artists, so a thin profile ships empty rows rather
than near-random tiles.
- migration 0034: you_might_like_albums / you_might_like_artists (id+rank,
CASCADE, per-user rank index).
- playlists/you_might_like.go: cold-start gate + similarity roll-up
(sum-of-top-3 aggregation, per-artist album cap, daily-rotating via the
same userIDHash jitter as For-You) + atomic-replace persist in the tx.
- recommendation/home.go: two new HomePayload sections with read-time
cross-section dedup vs Most Played / Rediscover / Last Played, trimmed
to 10 each.
- api: you_might_like_albums / you_might_like_artists on /api/home and
/api/home/index, reusing albumRefFrom / artistRefFromCovered.
- tests: pure roll-up/aggregation/cap unit tests + DB-backed gate,
sufficiency, and atomic-replace tests (all green vs real Postgres).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Audit of every Android↔server connection point (2026-06-11) cleared the
silent-contract class that caused the events `type` bug, but surfaced a
cluster of offline/playback-robustness defects. Fixes:
1. Playback double-count on background. PlayEventsReporter no longer
enqueues a partial play_offline + leaves the live row open on every
screen-lock. closeCurrent() now routes by whether the server has an
open row: close-by-id (durable PLAY_ENDED on failure) when it does,
offline only when no row exists, and a no-op while a play_started is
in flight (the server auto-closes that orphan). onStop only durably
closes a *paused* play — a still-playing one is left to the live path
under the foreground service. Adds the PLAY_ENDED mutation kind.
2. Replayer poison rows. MutationReplayer now classifies each replay as
SENT / DROP / RETRY: permanent 4xx (and corrupt payloads) are dropped
instead of retried forever; 408/429/5xx/transport still retry.
3. Offline-play / close-by-id idempotency (server). RecordOfflinePlay
dedups on (user, track, started_at); RecordPlayEnded skips a second
skip_events insert when re-closing an already-ended row. Makes the
at-least-once replay safe against lost-response duplicates.
4. Like-toggle collapse. Replayer drops like-toggles superseded by a
later toggle for the same entity, so partial-failure + differential
retry can't invert the final like state.
5. Connectivity-return trigger. MutationReplayer + SyncController now
also drain/sync when NetworkStatusController recovers to Healthy, so
an offline→online transition mid-session doesn't wait for a cold
start. SyncController.syncSafe gains a single-in-flight mutex.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
autoClosePriorOpen hardcoded was_skipped=true for every orphaned
play_event (a play_started whose play_ended never arrived, e.g. the
client backgrounded mid-track). That hid fully-listened tracks from
History — a play that sat open past its own length was capped to the
track duration (ratio ~1) yet still flagged skipped. Observed live:
History showed 3 plays for a day of listening because most rows were
auto-closed orphans marked skipped.
Now the auto-close applies the same skip rule as RecordPlayEnded to the
duration-capped elapsed estimate: ratio >= threshold OR elapsed >= the
duration floor -> a real play that lands in History; a genuine
quick-abandon still classifies as a skip. Still writes no skip_events
row, so the ambiguous auto-close never feeds the skip-ratio /
recommendation signal.
This is the server half. The client-side root cause (backgrounded
track transitions never closed, orphaning the rows in the first place)
is tracked separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GET /api/artists/{id}/similar — in-library artists ranked by similarity
score (deduped across sources), ArtistRef list with cover + album count.
GET /api/artists/{id}/top-tracks — current user's most-played tracks for
the artist (skips excluded, quarantine filtered).
Wire the fsnotify watcher and a fixed 12h safety-net delta walk in main;
remove the configurable scan scheduler (scheduler.go, scan_schedule table via
migration 0033, GET/PATCH /api/admin/scan/schedule, and the server/api
plumbing). Manual scan + scan status are unchanged.
Add Scanner.ScanFiles (watcher-driven targeted scan returning changed album
IDs) and a recursive fsnotify Watcher that debounces filesystem events and
enriches just the affected albums inline. Pure classifyEvent/drainPending
seams unit-tested; ScanFiles covered in the scanner integration test.
Closes Scribe #614, #615, server half of #616 surfaced by the 2026-06-04 divergent-provider audit.
- streamURL helper now used everywhere /api/tracks/{id}/stream is built (was inline concat in playlists.go and cast_token.go); add streamURLWithExt for the .ext cast variant.
- audioContentType in media.go is the canonical file_format -> MIME lookup; mimeForFormat in cast_token.go is now a thin wrapper that overrides the unknown-format fallback to audio/mpeg (Sonos rejects octet-stream). Adds mpeg/vorbis/wave aliases. Subsonic's contentTypeForFormat stays frozen per docs.
- coverart.ResolveAlbumPath extracted; api and subsonic both delegate to it.
After the X-Forwarded-Proto fix Sonos now gets a clean https:// URL
but returns vendor error 1023 - empty CurrentURIMetaData. Sonos
requires DIDL-Lite metadata with at minimum <res protocolInfo>
carrying the audio MIME type so it can validate the source before
playback. The original spec said 'Sonos accepts empty DIDL; recoverable
if a device rejects' - that was wrong for Sonos.
Server (cast_token.go):
- Look up the track and return mime (from tracks.file_format) +
title in the cast-token response. mimeForFormat covers the common
formats - mp3, flac, m4a/aac, ogg, opus, wav - falling through to
audio/mpeg for unknowns.
- Missing track returns 404 (apierror.NotFound) instead of letting the
caller mint a token for nothing.
Client (CastApi.kt, AVTransportClient.kt, OutputPickerController.kt):
- StreamTokenResponse gains mime + title (defaulted so old contracts
stay parseable).
- AVTransportClient.setAVTransportURIWithMetadata builds minimal Sonos-
acceptable DIDL-Lite around the URL + MIME + title. xml-escaped.
- selectUpnp calls the new overload; Timber.i now logs the MIME so the
next on-device test shows it.
Generic UPnP renderers tolerate the DIDL shape too - no downside to
sending it everywhere.
On-device test against Sonos showed SetAVTransportURI returning UPnP
error 714 (IllegalMimeType). Logcat:
POST /api/cast/stream-token -> 200 (token minted)
SetAVTransportURI to http://minstrel.fabledsword.com/...
<-- 500 from Sonos: SoapFaultException SOAP fault 714
The server is behind a TLS-terminating reverse proxy, so r.TLS is
nil and the URL builder emitted http://. Sonos does a HEAD probe to
detect the audio MIME type; against an http:// URL that 301s to
https://, the probe finds no audio body and bails with 714.
The Task 2 code-quality reviewer flagged this exact scenario at the
time. Closing it now: honor X-Forwarded-Proto + X-Forwarded-Host
before falling back to r.TLS + r.Host. Public URL the speaker
fetches now matches the scheme/host the client used to reach the
endpoint.
TestRoutesRegisteredInMount failed because handleGetStream did the
DB lookup (404 on missing track) BEFORE streamAuthOk (401 on
unauth). For an unauth request to a non-existent track, the test
saw 404 and concluded the route wasn't registered when actually it
was - the handler just bailed at the lookup before auth.
Reorder: extract trackID via chi.URLParam, run streamAuthOk on the
raw path id first (the HMAC token is signed over the same id
string so we don't need the resolved row yet), then do the DB
lookup. Test now sees 401 on the unauth probe as it expected.
Also closes a small info-leak: previously a 404/401 differential
let unauth callers probe which track IDs exist. Now both unknown
and known IDs return 401 for unauth requests.
golangci-lint flagged three errcheck:
- stream_token.go: fmt.Fprintf(mac, ...) - hash.Hash never errors
per documented contract, but errcheck wants explicit discard.
Discard via _, _ assignment with a WHY comment.
- config_test.go: os.Unsetenv calls in tests - discard the error
via _ assignment. Test cleanup paths.
Reviewers flagged the Fprintf one during Task 1 quality review but
golangci-lint runs in a separate CI step that wasn't exercised on
the per-task pushes (cancelled by subsequent push concurrency).
go vet caught the test's Mount call missing the trailing []byte
streamSecret arg added by the UPnP slice's Task 2. The test passed nil
for *playlists.Scheduler but didn't pass anything for []byte, so the
arg count was one short.
Added nil for the streamSecret position - the test exercises route
registration only, not the cast-token endpoint, so the secret value
doesn't matter for what this test asserts.
Operator feedback on the prior unification commit (7473e98d):
1. NewForYou should daily-rotate alongside Rediscover and FirstListens.
The 'newest album first regardless of day' intent was the wrong
call - operator wants visible day-over-day movement on every
deterministic mix surface. Spec flipped to dailyRotate: true.
2. Diversity caps (<=2 per album / <=3 per artist) on every mix, not
just the historically-diverse ones. The 2-per-album limit has
helped a lot on the operator's library; extending it to NewForYou
and FirstListens (previously album-coherent / no cap) surfaces
more distinct albums per day. Spec flipped to diversify: true on
all five.
3. Fallback when diversity caps strip the pool below the 100-track
target: finishMix now calls topUpFromRaw, which appends non-capped
tracks from the raw SQL pool (preserving original ranked order +
skipping duplicates) until the target is hit or the pool runs out.
On rich libraries the cap yields >= 100 and top-up never runs; on
thin / album-heavy libraries we ship a partly-diversified 100
instead of a strictly-diversified 40.
Net effect: every deterministic mix now rotates day-over-day, every
mix gets the same diversity treatment (with graceful degradation),
and the producer surface stays a single factory over a spec list.
The five discovery-mix producers (Deep Cuts, Rediscover, New for you,
On this day, First listens) were near-identical boilerplate that
differed only in (a) which SQL query they ran and (b) whether to
diversity-cap the result. Folded into one produceDiscoveryMix(spec)
factory + a per-mix discoveryMixSpec slice. The registry composes the
factory over the spec list so adding a new mix is one struct literal
+ a SQL query, never a new func.
Also fixes the user-reported bug that several mixes 'show the same
content from yesterday'. Audit of the SQL queries:
- Deep Cuts: ORDER BY md5(t.id::text || $2::text) → day-keyed
- On this day: ORDER BY w.c DESC, md5(...) → day-keyed
- Rediscover: ORDER BY tier, c DESC, id → invariant
- New for you: ORDER BY al.created_at DESC, disc, track → invariant
- First listens: ORDER BY tier, al.id, disc, track → invariant
The three invariant ones produced identical content day-over-day. The
unified spec carries a dailyRotate bool: when set, the producer
applies a daily-deterministic offset rotate-left of the candidate
pool BEFORE diversify+truncate. Rotation (not shuffle) preserves
contiguous-block ordering inside each day's slice — matters for First
listens which is album-coherent.
Set on Rediscover + First listens (where same-content-every-day is
clearly a bug). Left off New for you because 'newest album first
regardless of day' is the intended UX for that surface — daily
rotation there would feel wrong.
Daily rotation seed: rand.New(NewSource(int64(userIDHash(userID,
dateStr)))) — same primitive used by For-You's pickHeadAndTail
sampling so behavior is consistent across the system playlist family.
No test file referenced the deleted produceXxx functions directly,
only the registry, so this is a closed refactor.
Adds the client-facing endpoint that issues a signed stream URL for
the current track. Authenticated via the standard session cookie.
Returns {token, exp, url} where url is a fully-formed stream URL
the client passes verbatim to a UPnP / Sonos device's
AVTransport.SetAVTransportURI call.
expSeconds clamped to [60, 86400]; default 21600 (6h) - long enough
to play through any typical track without re-minting mid-playback.
MINSTREL_STREAM_SECRET is loaded from env var with a per-machine
fallback persisted at <Storage.DataDir>/stream_secret (auto-generated
on first boot via 64 random bytes, base64-url-encoded, 0600). The
file-based fallback is operator-machine-scoped runtime state, not a
user-facing setting - chosen over a DB column to avoid a migration
and keep the secret out of cross-instance restores. Operator can
override at any time via the env var; default path requires zero
config.
Tests cover happy-path token issuance + URL formatting, bad-UUID
rejection, unauthenticated rejection, the expSeconds clamp at all
boundaries, secret env override, auto-gen + file persistence at 0600,
second-boot reuse of the persisted file, and rejection of a malformed
env value.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds SignStreamToken / VerifyStreamToken (HMAC-SHA256 over
trackID|exp) and modifies handleGetStream to accept either the
existing session cookie OR a valid signed token. Stream route
moved out of the authed group so the handler's own auth check
runs and the token bypass is reachable.
Enables Sonos / UPnP speakers to fetch the stream URL without
carrying the user's session cookie - they cannot. The token is
short-lived (max 24h per the design); expiry checked at request
time only, not per-byte, so long tracks play through.
streamSecret field on handlers is nil for now; Task 2 wires the
loader (env var with auto-generated fallback persisted in
app_preferences).
Adds auth.OptionalUser - the permissive sibling of RequireUser
that attaches the user to context when a valid cookie / bearer is
present but does NOT 401 on absence. The stream route is wrapped
with it so the handler can fall through to the token path when
no session is present.
newLibraryRouter (test fixture) gets a synthetic-user middleware
on the stream route so existing media_test tests keep passing
without seeding a real session row - production traffic uses
auth.OptionalUser, the test path uses auth.UserCtxKeyForTest().
Five tests cover round-trip, tampered token rejection, expiry,
wrong-track-ID, and wrong-secret rejection. CI verifies.
Second go-round of the same shape of bug: tracks has file_size +
file_format NOT NULL (0002_core_library.up.sql) and my GC test seed
omitted both. The previous fix only addressed the artists.sort_name
column; the tracks INSERT was missing two more.
Use plausible stub values — the GC sweep only joins on track_id,
none of these columns affect what the test exercises.
The artists table requires sort_name (NOT NULL constraint added by
0009_artist_sort.up.sql). My GC integration test was inserting only
name + relying on a separate SELECT to pull the id back, which both
(a) violated the NOT NULL constraint and (b) was unnecessarily
indirect. RETURNING the id directly is the standard pattern used
everywhere else in the test suite.
Test now matches the real-world insert pattern in api.search +
library scan (sort_name mirrors name when no MBID-driven sort hint
is available). Other GC tests in this file don't touch artists so
they were already fine.
New `internal/gc` package with a single Worker that runs all five
lifecycle / retention sweeps from the 2026-06-02 drift audit on a
1-hour tick. Each sweep is small, idempotent (re-running on
already-clean rows is a no-op), and logs its affected-row count.
Sweeps (Scribe parent #552):
- **#566** GcCloseStalePlayEvents — play_events rows opened > 24h
ago that never got a play_ended (client crash, network drop).
Synthesizes ended_at from duration_played_ms when known, falls
back to now() so the row stops looking "open" to downstream
filters (ended_at IS NULL).
- **#565** GcClosePlaySessionsWithNoRecentEvents — play_sessions
with last_event_at older than 6h get ended_at = last_event_at
("user moved on"); empty sessions older than 1h get closed
too (stale handshakes from clients that never recorded a play).
The audit caught that the column was added but never populated
by any writer — every session row was "open" forever, breaking
downstream dedup queries that assume closed semantics.
- **#567** GcExpireScrobbleQueueFailedRows — drops scrobble_queue
rows in status='failed' older than 14 days. The worker stops
retrying after maxAttempts so these otherwise accumulate
forever on a persistent ListenBrainz outage / revoked token.
- **#574** GcResetStuckSystemPlaylistRuns — flips
system_playlist_runs.in_flight back to false on rows whose
last_run_at is older than 10 minutes. Catches goroutine-panic
wedges where the generator died between SET in_flight=true and
SET in_flight=false; the duplicate-prevention check refuses to
start a fresh regen while in_flight, so a stuck row would
otherwise deadlock all future regens for that user. Records
"stuck-row auto-reset by gc" in last_error so the operator can
tell auto-reset from a recent real failure.
- **#575** GcDeleteExpiredPasswordResets — deletes expired
password_resets rows. Unused expired rows go after a 1h grace
(gives the operator time to debug an active reset attempt);
used rows are kept 7 days for audit.
Wiring:
- main.go `go gcWorker.Run(ctx)` alongside the other periodic
workers (scrobble, similarity, lidarr).
- tickOnce fires once at start so a freshly-deployed server does
its initial sweep without waiting a full tick, matching the
scrobble worker pattern.
- Errors per sweep are logged but do NOT abort the remaining
ones — a transient pgx error from one query shouldn't prevent
the others from running.
Tests:
- 4 integration tests, one per UPDATE/DELETE sweep, that seed
rows-to-sweep + rows-to-leave-alone and assert the right rows
changed state. Skip unless MINSTREL_TEST_DATABASE_URL is set
(mirrors the api package pattern).
- Empty-tables no-op smoke test.
- Run() cancellation honoured (no spinning goroutine at
test-runner exit).
That's all five remaining server-side lifecycle findings from the
audit. The Android LOCAL_USER_ID hardcode (#576) is a separate
refactor that needs auth-store wiring and stays in the queue.
The docstring claimed "the next library scan reconciles missing
files by removing their tracks rows" — but scanner.go only does
filepath.WalkDir + UpsertTrack; it never enumerates existing rows
to check file_path presence, and it never DELETEs orphan rows. The
audit verified this — repo-wide grep finds no orphan-sweep code.
The lie is load-bearing: lidarrquarantine/service.go:270 leans on
this guarantee, so downstream code thinks the orphan case heals
itself. Fix the comment to state reality (admin re-trigger or
manual cleanup) and reference the open follow-up for adding a real
sweep. The actual reconcile pass is a separate piece of work
(needs scanrun integration + retention semantics + tests) and
stays in the Scribe audit queue.
Server-side fix for the drift audit finding (Scribe #578, parent
#552). Mirrored on Android (and Flutter) but the root cause and
the smallest blast-radius fix both live here.
The bug:
- Android MeApi.getProfile() calls GET /api/me and deserializes
into MyProfileWire which has nullable display_name + email.
- Server's handleGetMe was emitting the narrower UserView shape
(id, username, is_admin only).
- Android always saw displayName=null, email=null. The Settings →
Profile screen rendered BLANK form fields for users with stored
values.
- Saving from the blank state submitted empty strings to
PUT /api/me/profile, which interprets empty as "clear to NULL"
(me_profile.go:53-65) — DESTROYING the user's saved profile.
- Flutter (flutter_client/lib/api/endpoints/settings.dart:9-12)
has the identical bug pattern.
The fix:
- handleGetMe now emits profileViewFromUser(user) — the same
shape PUT /api/me/profile already returns (meProfileResp:
id, username, display_name, email, is_admin).
- auth.UserFromContext already returns a full dbq.User row, so no
extra DB lookup needed.
- Web's User TypeScript type is narrower than this response but
doesn't care about the extra fields (TS structural typing).
- LoginResp.User still uses UserView; login response unchanged.
New test asserts the regression directly: a user with stored
display_name + email sees them in /api/me. Old test updated to
decode into meProfileResp and assert the nullable fields are
correctly null for an unset profile.
Android side needs no change — the existing wire shape already
expected display_name + email; this just delivers them.
Five findings + one cancelled duplicate from the 2026-06-02 drift
audit (Scribe parent task #552):
- **#561 (Android)** PlayerController.playbackErrorEventsChannel was
Channel.CONFLATED. The PlaybackErrorReporter coroutine reads it in
a debounce loop that buffers events to coalesce into "Skipped N
unplayable tracks" — but CONFLATED silently dropped every emission
except the latest each time the reader wasn't actively pulling.
A network blip that failed 5 tracks back-to-back surfaced only the
last failure to the snackbar AND only POSTed one playback_errors
row to the admin inbox. Switch to BUFFERED (default capacity 64,
well above any plausible burst rate). Coalescing path now reaches
N > 1 and the admin inbox sees every failure.
- **#563 (server)** systemPlaylistSources rotation whitelist in
playevents/writer.go had drifted behind the migrations. It listed
only for_you + discover; migrations 0021 + 0028 added 6 more
variants (deep_cuts, rediscover, new_for_you, on_this_day,
first_listens, songs_like_artist) that ship as refreshable system
mixes. Plays from those surfaces never advanced the per-user
rotation, so "unplayed first" ordering staled — the same tracks
kept resurfacing. Add all 6 to the map; comment now points at the
migration's CHECK list as the canonical source so future variants
notice the requirement. #573 was the duplicate auditor hit for
the same drift; cancelled in Scribe.
- **#564 (Android)** Android emitted source = "playlist:<variant>"
for system-mix plays from Home and PlaylistDetail, but the
server's rotation matcher keys on the BARE variant string (web
sends the bare form — PlaylistCard.svelte:83). Misalignment meant
system-mix plays from Android never advanced rotation; switching
from web to Android effectively reset the perceived "unplayed
next" ordering. Fix HomeScreen.kt:291 to send bare variant and
PlaylistDetailScreen.kt's play() to prefer systemVariant over the
playlist:<id> tag when the playlist is a refreshable system mix.
User playlists keep playlist:<id> (intentional — rotation only
applies to system mixes anyway).
- **#568 + #569 (Android)** AuthCookieInterceptor was unconditionally
attaching the Minstrel session cookie to every outgoing request
AND wiping the session on any 401. The shared OkHttpClient is also
used by Coil for external image fetches (artwork.musicbrainz.org,
coverartarchive.org, Lidarr /MediaCover URLs); this leaked the
session cookie to those hosts (privacy posture) AND silently
signed users out of Minstrel if any external image host returned
401. Scope both attach + clear to the placeholder.invalid sentinel
host the same way BaseUrlInterceptor was scoped in aec10ce7. Two
new regression tests cover the external-host pass-through. Existing
tests rewritten to make requests through the placeholder URL so
they exercise the in-scope path explicitly.
All five Scribe tasks updated to in_progress at start, will flip to
done after CI green on this push.
Six findings from the 2026-06-02 multi-system drift audit (Scribe
parent task #552):
- **#553 (web)** Tailwind class fix: web admin playback-errors Delete
confirm button was using `bg-action-danger`, an undefined token —
swap to `bg-action-destructive` to match every other destructive
button. Restored the Oxblood signal that distinguishes Delete from
Cancel.
- **#555 (web)** Type the `source` field on `play_started` in the
EventRequest discriminated union. Server's eventRequest accepts it;
web's TS type was missing the slot, so a "drop extra properties"
refactor could silently strip the source tag and break system-
playlist rotation attribution.
- **#556 + #557 (server)** Coverage rollup whitelist was pinned to
('sidecar','embedded','mbcaa','theaudiodb'); migration 0020 added
'deezer' and 'lastfm' as valid cover_art_source values but those
never got wired in, so albums with art from those providers
silently counted as MISSING in the admin Coverage dashboard. The
rollup test was seeding only the pre-0020 sources, masking the
gap in CI. Extend the query to include deezer + lastfm; seed the
test with one row per valid source (regression-guards future
additions).
- **#558 (web)** Auth gate was blocking /forgot-password and
/reset-password/<token> — both are entered without a session by
definition, so the email-link reset flow was bouncing signed-out
users to /login. Add /forgot-password to the public set and a
/reset-password/ prefix matcher. New tests assert both routes
reach their pages without redirect.
- **#571 (server)** Library scanner was indexing only .mp3/.m4a/.flac
/.ogg while the stream handler (media.go) had been extended to
serve .opus, .aac, and .wav. A user with .opus files in their
library never saw them in artist/album listings because the
scanner skipped indexing — silent data loss. Aligned the scanner
to match the media handler.
Scribe statuses updated to in_progress; flipping to done after the
push since these are mechanical and verified directly against the
cited file:lines.
New client-reported playback-error log. Surfaces zero-duration
tracks (and future load_failed / stalled kinds) into an admin
inbox so the operator can hide / delete / re-request the
offending track.
Schema (migration 0032):
- playback_errors table with CHECK constraints on the kind +
resolution enums (per the standing rule that new enum values
need a migration to add)
- Partial index on unresolved rows for fast inbox lookup
- ON DELETE CASCADE from tracks + users so cleanup is automatic
Endpoints:
- POST /api/playback-errors: any signed-in user reports. Body
validates track existence + kind whitelist; client_id required
so support can correlate reports from the same device.
- GET /api/admin/playback-errors?resolved=false&offset=&limit=:
admin list with join to track/album/artist for table render
without per-row round-trips. Pagination capped at 200/page.
- POST /api/admin/playback-errors/{id}/resolve: admin marks
resolved with a resolution enum string.
Auto-resolve on Hide/Delete/Re-request from the inbox row is
driven from the web client (two sequential calls) — keeps the
existing track-action endpoints unchanged.
CI on 8b586c2e caught it: the $1::text cast in the md5 ORDER BY made
sqlc infer the parameter as plain string instead of pgtype.UUID,
generating Column1 string instead of UserID pgtype.UUID on the
ListRediscover*ForUserParams structs. go vet flagged both call
sites in internal/recommendation/home.go where the Go code passes
UserID by name.
Fix: hash on album_id (or artist_id) + current_date only. The
eligibility filter already differs per user (different liked sets),
so the daily-rotation goal is preserved; we just lose per-user salt
in the within-day ordering of overlapping items - acceptable.
Applies to both primary queries AND both fallback queries (all four
had the same cast).
User request 2026-06-01: the previous Rediscover only fired on
explicit album/artist likes, so users who only liked at the track
level got an empty Rediscover row. Also no daily rotation - the
section never changed between data updates - and 25 items felt long
for the Home carousel.
SQL (internal/db/queries/recommendation.sql):
- ListRediscoverAlbumsForUser UNIONs the existing explicit album-like
signal with a new track-derived path: an album qualifies if it has
>=2 liked tracks where the earliest like is >30 days old.
- ListRediscoverArtistsForUser does the same with threshold 3 (artists
span more releases, so the bar is higher to avoid one-hit-wonder
affinities).
- Both ordering switched from longest-since-last-play to a daily-
stable random hash md5(entity_id || user_id || current_date) so
the row randomizes but stays consistent within a day.
- Fallback queries also switched to the daily-stable hash so the
fallback rows don't reshuffle on every refresh.
Go (internal/recommendation/home.go):
- HomeRediscoverLimit dropped from 25 to 10 (carousel-sized).
- rediscoverInnerLimit (30) is the per-query cap; gives headroom so
the Go-layer filters don't undershoot.
- applyRediscoverAlbumFilters does cross-section dedup vs Most Played
(no point surfacing albums the user is actively spinning) plus a
diversity cap of max 2 albums per artist (prevents one liked-but-
forgotten artist dominating the row).
- applyRediscoverArtistFilters does cross-section dedup vs Most
Played's artist set; no diversity cap needed (one row per artist).
Tests (internal/recommendation/home_integration_test.go):
- TrackDerived path: 2 liked tracks >30d old + no recent plays = album
appears in Rediscover.
- Threshold guard: 1 liked track = album does NOT appear.
- Diversity cap: 4 explicit album-likes from same artist = 2 in output.
- Dedup vs MostPlayed: eligible album whose track is in MostPlayed
gets filtered out.
- Existing FallbackWhenSparse test preserved (semantics unchanged for
recent likes that miss the >30d primary filter).
Clients unchanged - they render whatever /api/home/index returns. No
parity-map row needed (this is a server-internal recommendation
change, not a layout divergence).
staticcheck S1016 in the new golangci-lint v2 flagged four sites
copying field-by-field between two types with identical struct
shapes. Direct type conversion is the canonical form:
- internal/library/scanrun.go:
- LibraryStageTallies copy from Stats → LibraryStageTallies(lastStats)
- MBIDBackfillStageTallies copy from BackfillMBIDsResult →
MBIDBackfillStageTallies(lastRes)
- internal/recommendation/home.go:
- dbq.ListRediscoverAlbumsForUserRow copy from the Fallback row
type → dbq.ListRediscoverAlbumsForUserRow(r)
- Same pattern for the Artists rediscover pair.
No behavior change; the underlying struct shapes are identical
(staticcheck verified the conversion is valid). Net -18 +4 lines.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
player: setQueueFromTracks fast-starts a single source at player-index 0
while the full queue is broadcast, so the transient currentIndexStream→0
emission clobbered the correct mediaItem with queue.value[0] (the first
track). Mini bar / playlist marker pinned to the wrong track until a
later index event (~the "passive ~30s recovery"). Track a logical-index
base so the player→queue mapping stays correct during the fill window;
also fixes the latent forward-fill auto-advance off-by-base.
lidarr #50: approving no longer fails when Lidarr is down. Approve
records the decision durably first, then best-effort adds; the
reconciler idempotently (re)sends unconfirmed adds every tick until they
stick (new additive lidarr_add_confirmed_at; AddArtist/AddAlbum map
Lidarr's "already exists" 400 → ErrAlreadyExists). No failed-state or
expiry by design — Lidarr keeps trying, operator monitors.
lidarr #51: Create() is now idempotent — a non-terminal request for the
same MBID returns the existing row instead of inserting a duplicate.
Rewrites the obsolete LidarrUnreachable_503 test to assert the durable-
approve contract; threads a client factory into NewReconciler.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operator decision: the enricher is canonical. No MBID still runs the
provider chain (name-based providers — Deezer/Last.fm — resolve
without an MBID); if every provider returns ErrNotFound the row
settles cover/artist source 'none' at the current sources version
(re-eligible only when the registered provider set changes). It does
NOT skip-and-leave-NULL.
The two _NoMBID_LeavesNull tests predated the name-based providers
(0020 slice) and asserted the old skip→NULL contract. Updated:
- TestEnrichArtist_NoMBID_SettlesNone: stub now returns ErrNotFound
(realistic MBID-only-provider-with-empty-MBID), expect source 'none'.
- TestEnrichAlbum_NoSidecarNoMBID_SettlesNone: empty registry →
allWere404 stays true → expect 'none'.
Last failing cluster from the CI-integration initiative; suite should
now be fully green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The test marked id2 'none' via SetAlbumCover, which (covers.sql:42)
does NOT set cover_art_sources_version. ListAlbumsMissingCover treats
'none' AND version != current as eligible, so id2 (version 0, current
1) was wrongly drained → processed=2. The comment's intent ("'none'
with current version → not drained") requires SetAlbumCoverWithVersion
(albums.sql) stamped with the live GetCurrentSourcesVersion — the same
value EnrichBatch compares against. Test-only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two distinct pre-existing test bugs in the last failing cluster:
1. newTestEnricher() called resetRegistryForTests() itself, wiping the
fake providers callers Register() right before calling it (its own
doc says callers register first and reconcile() picks them up). The
~8 TestEnrichArtist_* failures (source stayed NULL, no thumb
written) all stem from reconcile() seeing an empty registry. Remove
the internal reset; make every caller that lacked one own the
registry lifecycle explicitly (resetRegistryForTests + t.Cleanup):
SidecarFound, NoSidecarNoMBID, AlreadySidecar_NoOp,
DrainsNullSourceOnly.
2. apiTestAlbumProvider.FetchAlbumCover had a stale signature
(context, string) predating the AlbumRef refactor; it no longer
satisfied coverart.AlbumCoverProvider, so the p.(AlbumCoverProvider)
capability assertion failed and TestAdminListCoverSources got
supports=[]. Fix the param to coverart.AlbumRef.
Test-only. (A1/A2 were correct; they unmasked these. Any residual
album-enricher semantics failures will be root-caused from the next
clean run, not bundled speculatively.)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A2 — migration 0030: the albums/artists art `*_source` CHECK was a
fixed provider-ID allowlist fighting the extensible coverart.Register
registry (per-provider migration churn at 0016/0018/0020; rejected
test stub providers → ~11 enricher tests failed). Relax to "NULL or
non-empty"; the registry is the source of truth. Same brittleness
class as the #433 discovery-mix CHECK.
D — lidarr Approve resolves metadata/quality profiles (JSON arrays)
before the add; the test stubs returned {"id":1} for every path, so
ListMetadataProfiles failed to unmarshal → 500/Approve errors. Make
the lidarrrequests + admin_requests stubs path-aware (arrays for
/metadataprofile, /qualityprofile).
E:
- auth: requireUser (prelude.go) emitted code "auth_required"; the
canonical code is "unauthenticated" (operator decision). Change the
code; drop the now-dead "auth_required" web error-copy key
("unauthenticated" already has copy).
- playlists_system_test wrapped the real auth.RequireUser middleware
but only injected withUser() context → 401. Like every other api
handler test, drop the middleware and rely on requireUser().
- admin_users dup-username test seeded "test-existing" (seedUser
prefixes) but POSTed "existing" → no collision; POST the prefixed
name.
- me_timezone test decoded a top-level {"code"} but the envelope is
{"error":{"code"}}; decode the nested shape.
- audit test asserted compact JSON; Postgres jsonb::text is spaced.
- put-lidarr-config tests predated the missing_defaults gate (correct
handler behavior); supply the required defaults in the bodies.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A redo: the prior commit truncated cover_art_sources_meta, but
SettingsService.reconcile() only READS that singleton (seeded once by
migration 0018) and never recreates it → ~45 coverart/api tests hard-
failed "get current sources version: no rows". Correct reset: keep
cover_art_provider_settings in the truncate set (boot idempotently
re-UpsertProviderSettings), drop cover_art_sources_meta from it, and
instead `UPDATE cover_art_sources_meta SET current_version = 1` so the
row survives while cross-test version accumulation is cleared.
B residual (exposed once the play_events FK fix let these run):
- seedQuarantine used reason 'test-hide', invalid for the
lidarr_quarantine_reason enum (bad_rip/wrong_file/wrong_tags/
duplicate/other) → use 'other'.
- TestBuildSystemPlaylists_SufficientActivity predated #411/#352: the
variant switch errored on the 5 new seedless mixes and capped
track_count at 25. Accept deep_cuts/rediscover/new_for_you/
on_this_day/first_listens as seedless; raise the cap to 100.
Test/test-harness only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bvandeusen