The verify half of the tune→verify loop (#1251), on the same admin
Tuning page as the knobs:
- RecommendationWeeklyTrends: weekly per-source outcomes aggregated
across all users (the knobs are global, so judging a turn needs
global outcomes — rows carry rates only, no track/user identity),
with a taste-hit count per bucket: plays whose track's artist has a
positive weight in the player's current taste profile. That's the
"cheap recompute" reading — retroactive over the whole window, at
the cost of profile drift.
- GET /api/admin/recommendation-trends?weeks=N (default 12, cap 52):
per-family weekly series (skip rate, sample-weighted completion,
taste-hit rate) plus the tuning-audit markers inside the window.
- Web: sparkline table under the tuning cards — skip rate per week on
a shared axis with dashed ticks at knob turns, latest-week columns,
window taste-hit rate, low-volume rows dimmed as anecdote, and a
plain-text list of the window's tuning changes.
Also fixes the revive unused-parameter lint on the tuning GET handler
that failed CI run 1903 on the previous commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
The recommendation scoring knobs move out of YAML (radio profile) and
out of the systemMixWeights hard-code (daily_mix profile) into
DB-backed settings with live effect (#1250) — the defaults-discovery
lab per decision #1247: the operator turns knobs to find good values,
which then get baked back into shipped defaults; end users and other
operators should never need the card.
- Migration 0040: recommendation_weight_profiles (radio / daily_mix,
8 weight columns), taste_tuning singleton (engagement half-life +
completion-curve points), recommendation_tuning_audit (one row per
change with a {field, old, new} diff — the trend view's markers,
#1251).
- internal/recsettings: boot reconcile seeds shipped defaults without
clobbering tuned rows (coverart SettingsService pattern), validates
patches (bounds, curve ordering), writes audit rows, and pushes
daily_mix weights + taste config into package playlists. No-op
patches write no audit row.
- playlists gains SetSystemMixWeights / SetTasteConfig swap points
under a RWMutex — no signature threading through the producers; the
scheduler's taste rebuild reads the pushed config.
- Radio reads its weight profile from the service per request; the 8
weight fields leave config.RecommendationConfig (YAML keeps only
RecentlyPlayedHours / RadioSize / RadioSizeMax).
- Admin API: GET/PATCH/reset under /api/admin/recommendation-tuning,
echoing current + shipped values.
- Web: new admin Tuning tab — two weight profiles side by side, taste
card, per-scope save (changed fields only) + reset, deviation dots
against shipped defaults.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
Two approved composition changes (#1269), mechanism only — the
taste/fresh share stays data-decided (#1252) and pick_kind
attribution is unchanged.
Multi-seed blending: each day's build now seeds from up to 3 of the
user's top-5 tracks (pickDailySeeds, the generalized daily shuffle)
instead of one rotating anchor, so the mix spans neighborhoods within
a day and stops feeling bipolar as the rotation swings between
dissimilar seeds. Per-seed pools merge first-seen-deduped; the head
is filled best-first under 50/30/20 per-seed quotas (60/40 for two
seeds) so one neighborhood can't monopolize it, with thin-seed quota
spilling best-first.
Score-weighted fresh tail: the tail sample (rank 2*headN onward) was
uniform — the 380th-best candidate as likely as the 101st. It now
uses deterministic Efraimidis-Spirakis keys with weight halving every
50 ranks, so freshness keeps its "you'll probably enjoy this" half
while still rotating daily.
The retired single-seed picker's one other caller, You-might-like,
moves to pickDailySeeds(n=1) — a single neighborhood per day is right
for a short shelf, and the behavior note is inline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
Both mixes move from a single hard eligibility rule to the tiered
ladder, with their tier stamped onto playlist_tracks.pick_kind via the
#1270 provenance pipeline.
New for you (#1267) — consume on play, degrade by stepping back:
- "Consumed" = any track attempted >=30s; played albums leave the mix
at the next build instead of crowding it until the calendar window
expires.
- Tier 1: unconsumed albums added <30d by direct-affinity artists.
Tier 2: unconsumed affinity albums from the wider 30-90d window —
added while you weren't looking. Tier 3: any unconsumed album added
<90d, newest first.
First listens (#1268) — track-level "attempted" threshold:
- A 2-second accidental brush no longer disqualifies a whole album;
"attempted" is duration_played_ms >= 30000 per track.
- Tier 1: albums with zero attempted tracks. Tier 2: barely-attempted
albums (<=25% of tracks reached 30s), minus the attempted tracks
themselves. The artist-affinity ordering signal also moves to the
>=30s definition so skip-only contact doesn't read as trust.
Producer plumbing: fetch adapters map the tier column onto pick kinds,
finishMix propagates PickKind into the persisted candidates, and
rotateForDay now rotates within contiguous same-pick-kind blocks so
daily rotation can't hoist tier-3 filler above tier-1's exact fits
(untiered pools are one block — original behavior).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
Three discovery-mix defects from the intent audit (Scribe note #1254),
all sharing the same root pattern — skips treated as non-events:
- Deep Cuts (#1257): eligibility counted only unskipped plays, so a
track skipped twice with zero completed listens read as "barely
heard" and kept being re-offered. Tracks with >=2 skips no longer
qualify; a single accidental skip doesn't banish.
- Rediscover (#1258): a skip on a rediscover-sourced play — the user
explicitly declining the resurfacing invitation — changed nothing,
so declined tracks re-qualified the next day forever. Such tracks
now sit out 90 days.
- On This Day (#1256): day-of-year distance used plain ABS, so
Dec 28 vs Jan 3 read as 359 days apart and the window silently
gutted itself for ~3 weeks around every New Year. Now circular
(LEAST(d, 365-d)), anchored on the build-date parameter instead of
now() so it's testable and consistent with the mix's daily
determinism.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
PickSeedArtists had a hard 7-day window with no fallback: a week
without listening emptied the seed pool, produceSeedMixes returned
zero playlists, and the daily atomic-replace build deleted every
existing "Songs like X" mix until the user played something again
(#1255).
The query now falls back through widening engagement windows — 7d →
30d → all-time → liked artists — the same tiered shape that fixed the
identical vanish for For You's seeds (PickTopPlayedTracksForUser).
Like-boost scoring is preserved in every tier.
All returned rows share the winning tier, and produceSeedMixes maps it
onto the rule-#131 pick-kind ladder (7d = tier1 exact, 30d = tier2,
all-time/liked = tier3) and stamps the built tracks — the #1270
provenance pipeline then attributes plays and skips to seed freshness,
so the metrics card can say whether stale-seeded mixes actually
perform worse.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
The #1249 mechanism (stamp WHY a track is in the snapshot at build
time, freeze it onto the play at ingestion, break it down in metrics)
generalizes from a For You one-off to the standard for every system
mix (#1270):
- Migration 0039 widens both pick_kind CHECKs (drop + re-add in the
same change) to taste/fresh + Discover's dormant/cross_user/random
+ tier1-3 for the rule-#131 eligibility ladders.
- GetForYouPickKindForTrack becomes GetSystemPickKindForTrack
(user, variant, track); ingestion stamps any systemPlaylistSources
play from its own variant's live snapshot, live + offline paths.
- Discover stamps its candidate bucket on discoverTrack before the
interleave, making the 40/30/30 allocation measurable; dedup keeps
the taking bucket's stamp.
- Metrics replace the for_you special-case with one pick-kind
vocabulary — any family with attributed plays gets a breakdown,
future stamping mixes need no metrics change.
- Web: breakdown sub-rows are now toggled per surface (collapsed by
default) so eight stamping mixes don't swamp the card.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
Milestone #127 step 2 (#1249). For You deliberately blends two
populations — a head of top-scored taste picks and a tail sampled from
deeper ranking (the freshness injection) — but the metrics judged it as
one blob, so its skip rate couldn't distinguish "the taste engine is
missing" from "the freshness tax is too high". That number decides the
exploration share before we tune it.
- Migration 0038: nullable pick_kind ('taste'|'fresh') on both
playlist_tracks (stamped at snapshot build) and play_events (frozen at
play-ingestion — the snapshot rebuilds daily, so attribution cannot be
reconstructed at read time).
- Builder: pickHeadAndTail marks head=taste / tail=fresh; the small-pool
fallback is all taste (top-N-by-score IS the taste mechanism). Other
variants persist NULL.
- Ingestion: for_you plays (live + offline replay) look the track up in
the user's current snapshot; not found → unattributed, never guessed.
- Metrics: For You's row gains a breakdown (taste / fresh / earlier
unattributed plays), parent row stays the sum; web card renders the
sub-rows indented with the same baseline deltas + low-data dimming.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
The recommendation metrics table was observable but not actionable: raw
source strings (album:<uuid> one-offs) drowned the stable surfaces, and
manual plays were excluded so skip rates had no control group.
- SQL: include NULL-source rows (the baseline) and carry completion_n
so family merges can weight avg_completion correctly.
- Handler buckets raw sources into stable families (radio:<uuid> →
Radio, album:/artist: → direct plays, etc.) grouped by surface
intent: go-to / discovery / direct — each band judged against its
job, since discovery mixes are expected to skip hotter. Families
under 20 plays are flagged low-confidence, not hidden.
- Settings card renders the baseline row and per-surface deltas in
percentage points vs baseline (worse-than-baseline deltas in danger
color), intent hint copy per group, low-data rows dimmed.
- Pure-unit test for the bucketing/merge; DB test updated to the new
contract (baseline included, radio:<uuid> collapse).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
Confirmed against prod: exactly one track (17 plays, cold since May 21)
met the c>=5 + 30d-cold bar, and three process defects turned that into
a 1-track playlist instead of the locked placeholder.
- ListRediscoverTracks: collapse the two-tier UNION into one blended
pool. The old shallow-tier gate (WHERE NOT EXISTS deep) was
all-or-nothing — one 6-month row suppressed the entire 30-day tier —
and deep was a strict subset of shallow anyway. Eligibility drops to
>=3 non-skip plays (on a weeks-old history the >=5-play tracks are
precisely the ones still in rotation); ordering prefers >=6mo cold,
then >=5 plays, then raw count.
- Minimum viable mix floor for all five discovery mixes: below
minLen (15; 5 for the album-coherent NewForYou/FirstListens) the
variant is withheld so Home renders the 'listen more to unlock'
placeholder instead of a mix that reads as built-wrong.
- /api/events: clamp client-supplied 'at' to [user.created_at,
now+5m]. Unbounded client clocks could write arbitrarily old plays
and poison the 6-month ordering (prod data verified clean — no
scrub needed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
Relabel (#1204): route + player_state events fire for every output route,
not just UPnP — split them into a new 'playback' kind; 'upnp_sync' now
means genuinely UPnP/Sonos signal (drops, resync). Migration 0037 adds
'playback' to the kind CHECK; server whitelist, Android reporter labels,
and the web kind filter updated.
Web sort: the diagnostics list gains a Newest/Oldest-first sort (default
newest at top); export follows the displayed order.
Fix (#1205): OutputRoute.isConnected was derived from RouteInfo.connectionState,
which stays DISCONNECTED for local SYSTEM routes even when active — so a
connected Bluetooth device showed "Available" and reported connected:false.
The picker subtitle now uses isSelected (route == selected route); the dead
isConnected field is removed and the misleading `connected` field dropped
from the diagnostics route event (it only ever logs the active route).
Refs Scribe M9 (#119), tasks #1204#1205.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K55iTxn95BtshocgdE1shW
New diagnostic_events table + per-account users.debug_mode_enabled flag.
When an account's flag is on, its client(s) POST a batch timeseries of
connectivity / UPnP-sync / power / lifecycle events to /api/diagnostics
(no-op 204 when off, kind whitelist mirrors the CHECK constraint).
Admin surface: GET /api/admin/diagnostics (optional account/device/kind/
time-window filters, RFC3339-or-epoch-ms, export-sized paging) + a
/diagnostics/devices overview + PUT /api/admin/users/{id}/debug-mode to
flip an account remotely while a bug is live. debug_mode_enabled is now
exposed on /api/me (client gate) and the admin user views.
Retention: a 30-day gc-worker sweep (GcPruneDiagnostics), keyed on the
server clock so a skewed device clock can't keep rows alive.
Refs Scribe M9 (#119), tasks #1172#1173.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K55iTxn95BtshocgdE1shW
The daily 03:00 scheduler rebuild (and the manual refresh endpoint) replace
a user's system playlists + You-might-like rows but published no event, so a
client left open across the rebuild served yesterday's snapshot until a
manual reload — the stale-tab case behind #968. Add a user-scoped
playlist.system_rebuilt event (envelope {kind,user_id,data:{}}) from both the
scheduler (bus threaded into NewScheduler) and handleSystemPlaylistRefresh.
Clients consume it to invalidate home / system-playlist views and proactively
re-pull a stale active queue. Issue #968.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make a fresh install usable out of the box and document the first-run flow
for the public-facing repo.
- Default scan_on_startup to true (Default() + config.example.yaml, which is
the live config baked into the image). Previously false, so a fresh stack
came up with an empty library and no hint to scan. Scans are incremental
(mtime skip), so the per-restart cost is just a directory walk. Re-point
the env-override test to exercise the override against the new default.
- README: add a "First run" walkthrough (register -> scan -> integrations ->
install Android app -> invite users), each grounded in a real route.
- Add docs/screenshots/ with six captures, referenced via width-constrained
<img> wrapped in a link (shrink inline + click to open full size).
API token and invite token were cropped/redacted out of the captures
before commit so no live credential lands in the public history.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per-source play outcomes so the operator can see whether each recommendation
surface is landing and tune the now-operator-tunable taste weights.
Server:
- query RecommendationSourceMetricsForUser: groups the user's play_events by
source (system-playlist surface), reporting plays / skips / avg completion
over a window; NULL-source (library/radio) plays excluded.
- GET /api/me/recommendation-metrics?days=30 (default 30, capped 365) →
{window_days, sources:[{source, plays, skips, skip_rate, avg_completion}]}.
- handler test: 401 unauth; per-source aggregation + NULL-source exclusion +
skip_rate / avg_completion math.
Web:
- lib/api/metrics.ts: query + friendly source labels.
- settings page gains a "Recommendation metrics" card (table of surface / plays
/ skip rate / avg completion), with loading/error/empty states.
- settings tests mock the new query (manual subscribe-store, hoisting-safe).
Note: You-might-like plays aren't source-tagged (it's a Home row, not a system
playlist), so this covers For-You / Discover / the mixes. Tagging YML plays
would be a client follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2a re-ranks the existing pool by TasteMatch; this ensures taste-relevant tracks
ARE in the pool. Adds a 6th arm to LoadRadioCandidatesV2: in-library tracks by
the user's top positively-weighted taste-profile artists ($10 K, weight > 0,
deterministic weight-DESC,id order so it doesn't reintroduce same-day
nondeterminism). Pool-inclusion only (sim_score 0) — TasteMatch already scores
the fit. Empty for cold-start users (no profile).
- CandidateSourceLimits.TasteOverlap; default 20 (radio), 80 for For-You via
systemForYouSourceLimits.
- You-might-like deliberately sets TasteOverlap=0: it surfaces NOT-actively-
engaged artists, so flooding its pool with top-taste (mostly already-played)
artists would just feed the read-time dedup.
- Test: positive-weight artist's track enters via the arm; negative-weight one
is excluded (weight > 0). Existing pool tests unaffected (no profile seeded).
Deferred within 2b: profile-seeded For-You — marginal given the arm + TasteMatch
already inject taste broadly (top-played seed ≈ top-taste artist).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The fallback pulled artists only from explicit artist-likes (general_likes_artists),
but most users like albums and tracks far more than artists — so the artists row
still came up thin (a couple of tiles) even with a rich library, while the albums
row filled fine.
Broaden both fallbacks to "entities you've shown affinity for":
- artist fallback = explicit artist-likes ∪ artists of liked albums ∪ artists of
liked tracks.
- album fallback = explicit album-likes ∪ albums of liked tracks.
New dedicated queries (ListYouMightLike{Artist,Album}FallbackForUser) replace the
narrow Rediscover-fallback reuse; same projection so the Go layer still converts
directly. (Aliased + fully-qualified the UNION arms — sqlc merges UNION scopes,
so unqualified user_id was ambiguous across the three like tables.)
Test: 12 liked TRACKS by distinct artists, no artist-likes → the artist row now
fills from their artists (was empty before).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
golangci-lint v2 (CI-only; local is v1) flagged the field-by-field struct
literals — the fallback and you-might-like row types are identical, so convert
directly instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The taste roll-up surfaces top-similar albums/artists, which for a heavy
listener are mostly ones they already play — so the read-time dedup (vs Most
Played + Rediscover + Last Played) can strip the section down to a single tile
(reported on the artists row). The code was sound; the section was just starved.
Adds a read-time fallback: when a You-might-like row comes up short after dedup,
top it up from the user's LIKED artists/albums — a far larger pool than the
12-entity similarity roll-up, so the same exclusions still leave plenty. Reuses
the existing Rediscover-fallback queries (no new SQL), applies the same
exclusions (already-shown + Rediscover + Most/Last Played) so it never
duplicates a tile or suggests an actively-played entity, and is best-effort
(a query error leaves the section as-is). Takes effect immediately — no rebuild.
A cold-start user with no likes gets nothing from the fallback, so the
new-user-empty behaviour is preserved (test still passes).
Test: 20 liked artists, none played → Rediscover fills 10, You-might-like
fallback fills the other 10, disjoint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The profile built in phase 1 now changes what gets surfaced. Adds a TasteMatch
term to the weighted-shuffle score so candidates are re-ranked by their fit to
the user's learned taste (positive draws toward it; negative reflects passive
avoidance; 0 at cold start).
- recommendation/score.go: ScoringInputs.TasteMatchScore ([-1,+1]) +
ScoringWeights.TasteWeight + the term in Score.
- recommendation/taste.go: LoadTasteProfile reads the taste_profile_* tables;
TasteProfile.Match blends the candidate's artist weight (0.7) and avg genre-tag
weight (0.3), each tanh-squashed by a fixed scale so one outlier artist can't
compress the rest. Unknown artist/tags and empty profiles → 0 (neutral).
- candidates.go: both candidate loaders set TasteMatchScore per candidate, so
every Score caller (system playlists incl. You-might-like, radio) becomes
taste-aware automatically.
- weights: systemMixWeights.TasteWeight = 1.5 (daily mixes are the primary
taste surface); config.RecommendationConfig gains taste_weight (default 1.0,
lighter — radio is seed-directed) wired into the radio handler.
- tests: pure (Match curve incl. saturation/clamp/empty-neutral, Score term
add+subtract) + DB round-trip (seed taste rows → Match positive). All green
vs real Postgres; existing playlist/radio tests unaffected (empty profile →
zero taste effect).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
golangci-lint v2 (CI-only; local is v1) flagged two unused-parameter issues:
- BuildTasteProfile's `now` was genuinely dead — decay/windowing are computed
DB-side via now(), so no Go-side timestamp is threaded. Removed it (a
phase-3 context model that needs a pinned reference time would re-add it);
updated the scheduler call site.
- the degenerate-params engagement test ignored t; reworked it to assert the
result stays in [-1,1], which also strengthens the test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Build a persistent, decaying model of each user's taste, recomputed daily,
that later phases consume across every recommendation surface. Phase 1 only
BUILDS the object — no behaviour change to what's surfaced yet.
Core mechanic — graded engagement (replaces binary was_skipped for learning;
was_skipped stays for History): a play's completion ratio maps to a signal in
[-1,+1] via two linear ramps (instant-skip → -1, ~0.30 neutral, ≥0.90 → +1).
Time-decayed (half-life ~75d) so recent behaviour dominates and the profile
tracks drift.
Per operator constraints:
- No explicit dislike button — negatives come only from passive behaviour
(early skips). Nothing recorded to regret or opt out of.
- Negatives are track-scoped; artist/tag weight is the decayed SUM of their
tracks' engagement, so one skip nets out against many good plays (a
DB test asserts a liked artist stays positive despite an early-skipped
track). A floor clamp bounds how negative any single entity can get.
- migration 0035: taste_profile_artists / taste_profile_tags (signed weight,
indexed by (user, weight DESC)).
- internal/taste: engagement.go (pure curve + decay) + profile.go
(accumulate plays + like bonuses, floor damping, size caps, atomic-replace).
- scheduler: rebuildUserDaily recomputes the profile before the playlist
build (so phase 2 can read it), best-effort — a taste failure never blocks
playlist building. Wired into the daily job + startup catch-up only (not
manual/lazy rebuilds).
- tests: pure (engagement curve, decay, ranking, floor, genre split) +
DB-backed (positive/negative weights, aggregation-protects-artist, like
bonus, atomic replace). All green vs real Postgres.
Config knobs live in taste.DefaultConfig() for now; wiring them into the
server RecommendationConfig is a later follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scoreAndSortCandidates drew per-candidate jitter by slice position, but
the candidate query (LoadRadioCandidatesV2) has ORDER BY random() arms and
no stable outer ordering, so DB row order varies call-to-call. When the
recency spread between candidates is smaller than the ±jitter (small or
recency-clustered libraries), two same-day rebuilds assigned jitter to
different tracks and reordered near-ties — so the build was not actually
deterministic-within-a-day as documented.
Pre-existing latent flake in TestBuildSystemPlaylists_DailyNonceDeterminism
(passed in isolation / by luck in CI; deterministically reproduced when the
system-build tests run in sequence). Confirmed independent of the
You-might-like change by neutralizing buildYouMightLike — the flake
persisted.
Fix: sort the candidate slice by track id before assigning jitter, so the
jitter for a track is a function of (track, day) alone, independent of DB
return order. Verified: full playlists package green 4/4 and the build-test
sequence green 5/5 (was 0/4 before).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surface in-library albums/artists the listener doesn't actively spin but
is predicted to enjoy, derived from the same similarity + like-weighted
candidate engine that powers For-You — rolled up from track scores to
album/artist granularity. Built in the daily 3am BuildSystemPlaylists
pass, atomic-replaced alongside the system playlists, and read back by
/api/home (+ /api/home/index).
Cold-start gate: skips generation entirely below 20 distinct unskipped
tracks AND 5 distinct artists, so a thin profile ships empty rows rather
than near-random tiles.
- migration 0034: you_might_like_albums / you_might_like_artists (id+rank,
CASCADE, per-user rank index).
- playlists/you_might_like.go: cold-start gate + similarity roll-up
(sum-of-top-3 aggregation, per-artist album cap, daily-rotating via the
same userIDHash jitter as For-You) + atomic-replace persist in the tx.
- recommendation/home.go: two new HomePayload sections with read-time
cross-section dedup vs Most Played / Rediscover / Last Played, trimmed
to 10 each.
- api: you_might_like_albums / you_might_like_artists on /api/home and
/api/home/index, reusing albumRefFrom / artistRefFromCovered.
- tests: pure roll-up/aggregation/cap unit tests + DB-backed gate,
sufficiency, and atomic-replace tests (all green vs real Postgres).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Audit of every Android↔server connection point (2026-06-11) cleared the
silent-contract class that caused the events `type` bug, but surfaced a
cluster of offline/playback-robustness defects. Fixes:
1. Playback double-count on background. PlayEventsReporter no longer
enqueues a partial play_offline + leaves the live row open on every
screen-lock. closeCurrent() now routes by whether the server has an
open row: close-by-id (durable PLAY_ENDED on failure) when it does,
offline only when no row exists, and a no-op while a play_started is
in flight (the server auto-closes that orphan). onStop only durably
closes a *paused* play — a still-playing one is left to the live path
under the foreground service. Adds the PLAY_ENDED mutation kind.
2. Replayer poison rows. MutationReplayer now classifies each replay as
SENT / DROP / RETRY: permanent 4xx (and corrupt payloads) are dropped
instead of retried forever; 408/429/5xx/transport still retry.
3. Offline-play / close-by-id idempotency (server). RecordOfflinePlay
dedups on (user, track, started_at); RecordPlayEnded skips a second
skip_events insert when re-closing an already-ended row. Makes the
at-least-once replay safe against lost-response duplicates.
4. Like-toggle collapse. Replayer drops like-toggles superseded by a
later toggle for the same entity, so partial-failure + differential
retry can't invert the final like state.
5. Connectivity-return trigger. MutationReplayer + SyncController now
also drain/sync when NetworkStatusController recovers to Healthy, so
an offline→online transition mid-session doesn't wait for a cold
start. SyncController.syncSafe gains a single-in-flight mutex.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
autoClosePriorOpen hardcoded was_skipped=true for every orphaned
play_event (a play_started whose play_ended never arrived, e.g. the
client backgrounded mid-track). That hid fully-listened tracks from
History — a play that sat open past its own length was capped to the
track duration (ratio ~1) yet still flagged skipped. Observed live:
History showed 3 plays for a day of listening because most rows were
auto-closed orphans marked skipped.
Now the auto-close applies the same skip rule as RecordPlayEnded to the
duration-capped elapsed estimate: ratio >= threshold OR elapsed >= the
duration floor -> a real play that lands in History; a genuine
quick-abandon still classifies as a skip. Still writes no skip_events
row, so the ambiguous auto-close never feeds the skip-ratio /
recommendation signal.
This is the server half. The client-side root cause (backgrounded
track transitions never closed, orphaning the rows in the first place)
is tracked separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GET /api/artists/{id}/similar — in-library artists ranked by similarity
score (deduped across sources), ArtistRef list with cover + album count.
GET /api/artists/{id}/top-tracks — current user's most-played tracks for
the artist (skips excluded, quarantine filtered).
Wire the fsnotify watcher and a fixed 12h safety-net delta walk in main;
remove the configurable scan scheduler (scheduler.go, scan_schedule table via
migration 0033, GET/PATCH /api/admin/scan/schedule, and the server/api
plumbing). Manual scan + scan status are unchanged.
Add Scanner.ScanFiles (watcher-driven targeted scan returning changed album
IDs) and a recursive fsnotify Watcher that debounces filesystem events and
enriches just the affected albums inline. Pure classifyEvent/drainPending
seams unit-tested; ScanFiles covered in the scanner integration test.
Closes Scribe #614, #615, server half of #616 surfaced by the 2026-06-04 divergent-provider audit.
- streamURL helper now used everywhere /api/tracks/{id}/stream is built (was inline concat in playlists.go and cast_token.go); add streamURLWithExt for the .ext cast variant.
- audioContentType in media.go is the canonical file_format -> MIME lookup; mimeForFormat in cast_token.go is now a thin wrapper that overrides the unknown-format fallback to audio/mpeg (Sonos rejects octet-stream). Adds mpeg/vorbis/wave aliases. Subsonic's contentTypeForFormat stays frozen per docs.
- coverart.ResolveAlbumPath extracted; api and subsonic both delegate to it.
After the X-Forwarded-Proto fix Sonos now gets a clean https:// URL
but returns vendor error 1023 - empty CurrentURIMetaData. Sonos
requires DIDL-Lite metadata with at minimum <res protocolInfo>
carrying the audio MIME type so it can validate the source before
playback. The original spec said 'Sonos accepts empty DIDL; recoverable
if a device rejects' - that was wrong for Sonos.
Server (cast_token.go):
- Look up the track and return mime (from tracks.file_format) +
title in the cast-token response. mimeForFormat covers the common
formats - mp3, flac, m4a/aac, ogg, opus, wav - falling through to
audio/mpeg for unknowns.
- Missing track returns 404 (apierror.NotFound) instead of letting the
caller mint a token for nothing.
Client (CastApi.kt, AVTransportClient.kt, OutputPickerController.kt):
- StreamTokenResponse gains mime + title (defaulted so old contracts
stay parseable).
- AVTransportClient.setAVTransportURIWithMetadata builds minimal Sonos-
acceptable DIDL-Lite around the URL + MIME + title. xml-escaped.
- selectUpnp calls the new overload; Timber.i now logs the MIME so the
next on-device test shows it.
Generic UPnP renderers tolerate the DIDL shape too - no downside to
sending it everywhere.
On-device test against Sonos showed SetAVTransportURI returning UPnP
error 714 (IllegalMimeType). Logcat:
POST /api/cast/stream-token -> 200 (token minted)
SetAVTransportURI to http://minstrel.fabledsword.com/...
<-- 500 from Sonos: SoapFaultException SOAP fault 714
The server is behind a TLS-terminating reverse proxy, so r.TLS is
nil and the URL builder emitted http://. Sonos does a HEAD probe to
detect the audio MIME type; against an http:// URL that 301s to
https://, the probe finds no audio body and bails with 714.
The Task 2 code-quality reviewer flagged this exact scenario at the
time. Closing it now: honor X-Forwarded-Proto + X-Forwarded-Host
before falling back to r.TLS + r.Host. Public URL the speaker
fetches now matches the scheme/host the client used to reach the
endpoint.
TestRoutesRegisteredInMount failed because handleGetStream did the
DB lookup (404 on missing track) BEFORE streamAuthOk (401 on
unauth). For an unauth request to a non-existent track, the test
saw 404 and concluded the route wasn't registered when actually it
was - the handler just bailed at the lookup before auth.
Reorder: extract trackID via chi.URLParam, run streamAuthOk on the
raw path id first (the HMAC token is signed over the same id
string so we don't need the resolved row yet), then do the DB
lookup. Test now sees 401 on the unauth probe as it expected.
Also closes a small info-leak: previously a 404/401 differential
let unauth callers probe which track IDs exist. Now both unknown
and known IDs return 401 for unauth requests.
golangci-lint flagged three errcheck:
- stream_token.go: fmt.Fprintf(mac, ...) - hash.Hash never errors
per documented contract, but errcheck wants explicit discard.
Discard via _, _ assignment with a WHY comment.
- config_test.go: os.Unsetenv calls in tests - discard the error
via _ assignment. Test cleanup paths.
Reviewers flagged the Fprintf one during Task 1 quality review but
golangci-lint runs in a separate CI step that wasn't exercised on
the per-task pushes (cancelled by subsequent push concurrency).
go vet caught the test's Mount call missing the trailing []byte
streamSecret arg added by the UPnP slice's Task 2. The test passed nil
for *playlists.Scheduler but didn't pass anything for []byte, so the
arg count was one short.
Added nil for the streamSecret position - the test exercises route
registration only, not the cast-token endpoint, so the secret value
doesn't matter for what this test asserts.
Operator feedback on the prior unification commit (7473e98d):
1. NewForYou should daily-rotate alongside Rediscover and FirstListens.
The 'newest album first regardless of day' intent was the wrong
call - operator wants visible day-over-day movement on every
deterministic mix surface. Spec flipped to dailyRotate: true.
2. Diversity caps (<=2 per album / <=3 per artist) on every mix, not
just the historically-diverse ones. The 2-per-album limit has
helped a lot on the operator's library; extending it to NewForYou
and FirstListens (previously album-coherent / no cap) surfaces
more distinct albums per day. Spec flipped to diversify: true on
all five.
3. Fallback when diversity caps strip the pool below the 100-track
target: finishMix now calls topUpFromRaw, which appends non-capped
tracks from the raw SQL pool (preserving original ranked order +
skipping duplicates) until the target is hit or the pool runs out.
On rich libraries the cap yields >= 100 and top-up never runs; on
thin / album-heavy libraries we ship a partly-diversified 100
instead of a strictly-diversified 40.
Net effect: every deterministic mix now rotates day-over-day, every
mix gets the same diversity treatment (with graceful degradation),
and the producer surface stays a single factory over a spec list.
The five discovery-mix producers (Deep Cuts, Rediscover, New for you,
On this day, First listens) were near-identical boilerplate that
differed only in (a) which SQL query they ran and (b) whether to
diversity-cap the result. Folded into one produceDiscoveryMix(spec)
factory + a per-mix discoveryMixSpec slice. The registry composes the
factory over the spec list so adding a new mix is one struct literal
+ a SQL query, never a new func.
Also fixes the user-reported bug that several mixes 'show the same
content from yesterday'. Audit of the SQL queries:
- Deep Cuts: ORDER BY md5(t.id::text || $2::text) → day-keyed
- On this day: ORDER BY w.c DESC, md5(...) → day-keyed
- Rediscover: ORDER BY tier, c DESC, id → invariant
- New for you: ORDER BY al.created_at DESC, disc, track → invariant
- First listens: ORDER BY tier, al.id, disc, track → invariant
The three invariant ones produced identical content day-over-day. The
unified spec carries a dailyRotate bool: when set, the producer
applies a daily-deterministic offset rotate-left of the candidate
pool BEFORE diversify+truncate. Rotation (not shuffle) preserves
contiguous-block ordering inside each day's slice — matters for First
listens which is album-coherent.
Set on Rediscover + First listens (where same-content-every-day is
clearly a bug). Left off New for you because 'newest album first
regardless of day' is the intended UX for that surface — daily
rotation there would feel wrong.
Daily rotation seed: rand.New(NewSource(int64(userIDHash(userID,
dateStr)))) — same primitive used by For-You's pickHeadAndTail
sampling so behavior is consistent across the system playlist family.
No test file referenced the deleted produceXxx functions directly,
only the registry, so this is a closed refactor.
Adds the client-facing endpoint that issues a signed stream URL for
the current track. Authenticated via the standard session cookie.
Returns {token, exp, url} where url is a fully-formed stream URL
the client passes verbatim to a UPnP / Sonos device's
AVTransport.SetAVTransportURI call.
expSeconds clamped to [60, 86400]; default 21600 (6h) - long enough
to play through any typical track without re-minting mid-playback.
MINSTREL_STREAM_SECRET is loaded from env var with a per-machine
fallback persisted at <Storage.DataDir>/stream_secret (auto-generated
on first boot via 64 random bytes, base64-url-encoded, 0600). The
file-based fallback is operator-machine-scoped runtime state, not a
user-facing setting - chosen over a DB column to avoid a migration
and keep the secret out of cross-instance restores. Operator can
override at any time via the env var; default path requires zero
config.
Tests cover happy-path token issuance + URL formatting, bad-UUID
rejection, unauthenticated rejection, the expSeconds clamp at all
boundaries, secret env override, auto-gen + file persistence at 0600,
second-boot reuse of the persisted file, and rejection of a malformed
env value.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds SignStreamToken / VerifyStreamToken (HMAC-SHA256 over
trackID|exp) and modifies handleGetStream to accept either the
existing session cookie OR a valid signed token. Stream route
moved out of the authed group so the handler's own auth check
runs and the token bypass is reachable.
Enables Sonos / UPnP speakers to fetch the stream URL without
carrying the user's session cookie - they cannot. The token is
short-lived (max 24h per the design); expiry checked at request
time only, not per-byte, so long tracks play through.
streamSecret field on handlers is nil for now; Task 2 wires the
loader (env var with auto-generated fallback persisted in
app_preferences).
Adds auth.OptionalUser - the permissive sibling of RequireUser
that attaches the user to context when a valid cookie / bearer is
present but does NOT 401 on absence. The stream route is wrapped
with it so the handler can fall through to the token path when
no session is present.
newLibraryRouter (test fixture) gets a synthetic-user middleware
on the stream route so existing media_test tests keep passing
without seeding a real session row - production traffic uses
auth.OptionalUser, the test path uses auth.UserCtxKeyForTest().
Five tests cover round-trip, tampered token rejection, expiry,
wrong-track-ID, and wrong-secret rejection. CI verifies.
Second go-round of the same shape of bug: tracks has file_size +
file_format NOT NULL (0002_core_library.up.sql) and my GC test seed
omitted both. The previous fix only addressed the artists.sort_name
column; the tracks INSERT was missing two more.
Use plausible stub values — the GC sweep only joins on track_id,
none of these columns affect what the test exercises.
The artists table requires sort_name (NOT NULL constraint added by
0009_artist_sort.up.sql). My GC integration test was inserting only
name + relying on a separate SELECT to pull the id back, which both
(a) violated the NOT NULL constraint and (b) was unnecessarily
indirect. RETURNING the id directly is the standard pattern used
everywhere else in the test suite.
Test now matches the real-world insert pattern in api.search +
library scan (sort_name mirrors name when no MBID-driven sort hint
is available). Other GC tests in this file don't touch artists so
they were already fine.
New `internal/gc` package with a single Worker that runs all five
lifecycle / retention sweeps from the 2026-06-02 drift audit on a
1-hour tick. Each sweep is small, idempotent (re-running on
already-clean rows is a no-op), and logs its affected-row count.
Sweeps (Scribe parent #552):
- **#566** GcCloseStalePlayEvents — play_events rows opened > 24h
ago that never got a play_ended (client crash, network drop).
Synthesizes ended_at from duration_played_ms when known, falls
back to now() so the row stops looking "open" to downstream
filters (ended_at IS NULL).
- **#565** GcClosePlaySessionsWithNoRecentEvents — play_sessions
with last_event_at older than 6h get ended_at = last_event_at
("user moved on"); empty sessions older than 1h get closed
too (stale handshakes from clients that never recorded a play).
The audit caught that the column was added but never populated
by any writer — every session row was "open" forever, breaking
downstream dedup queries that assume closed semantics.
- **#567** GcExpireScrobbleQueueFailedRows — drops scrobble_queue
rows in status='failed' older than 14 days. The worker stops
retrying after maxAttempts so these otherwise accumulate
forever on a persistent ListenBrainz outage / revoked token.
- **#574** GcResetStuckSystemPlaylistRuns — flips
system_playlist_runs.in_flight back to false on rows whose
last_run_at is older than 10 minutes. Catches goroutine-panic
wedges where the generator died between SET in_flight=true and
SET in_flight=false; the duplicate-prevention check refuses to
start a fresh regen while in_flight, so a stuck row would
otherwise deadlock all future regens for that user. Records
"stuck-row auto-reset by gc" in last_error so the operator can
tell auto-reset from a recent real failure.
- **#575** GcDeleteExpiredPasswordResets — deletes expired
password_resets rows. Unused expired rows go after a 1h grace
(gives the operator time to debug an active reset attempt);
used rows are kept 7 days for audit.
Wiring:
- main.go `go gcWorker.Run(ctx)` alongside the other periodic
workers (scrobble, similarity, lidarr).
- tickOnce fires once at start so a freshly-deployed server does
its initial sweep without waiting a full tick, matching the
scrobble worker pattern.
- Errors per sweep are logged but do NOT abort the remaining
ones — a transient pgx error from one query shouldn't prevent
the others from running.
Tests:
- 4 integration tests, one per UPDATE/DELETE sweep, that seed
rows-to-sweep + rows-to-leave-alone and assert the right rows
changed state. Skip unless MINSTREL_TEST_DATABASE_URL is set
(mirrors the api package pattern).
- Empty-tables no-op smoke test.
- Run() cancellation honoured (no spinning goroutine at
test-runner exit).
That's all five remaining server-side lifecycle findings from the
audit. The Android LOCAL_USER_ID hardcode (#576) is a separate
refactor that needs auth-store wiring and stays in the queue.
The docstring claimed "the next library scan reconciles missing
files by removing their tracks rows" — but scanner.go only does
filepath.WalkDir + UpsertTrack; it never enumerates existing rows
to check file_path presence, and it never DELETEs orphan rows. The
audit verified this — repo-wide grep finds no orphan-sweep code.
The lie is load-bearing: lidarrquarantine/service.go:270 leans on
this guarantee, so downstream code thinks the orphan case heals
itself. Fix the comment to state reality (admin re-trigger or
manual cleanup) and reference the open follow-up for adding a real
sweep. The actual reconcile pass is a separate piece of work
(needs scanrun integration + retention semantics + tests) and
stays in the Scribe audit queue.
Server-side fix for the drift audit finding (Scribe #578, parent
#552). Mirrored on Android (and Flutter) but the root cause and
the smallest blast-radius fix both live here.
The bug:
- Android MeApi.getProfile() calls GET /api/me and deserializes
into MyProfileWire which has nullable display_name + email.
- Server's handleGetMe was emitting the narrower UserView shape
(id, username, is_admin only).
- Android always saw displayName=null, email=null. The Settings →
Profile screen rendered BLANK form fields for users with stored
values.
- Saving from the blank state submitted empty strings to
PUT /api/me/profile, which interprets empty as "clear to NULL"
(me_profile.go:53-65) — DESTROYING the user's saved profile.
- Flutter (flutter_client/lib/api/endpoints/settings.dart:9-12)
has the identical bug pattern.
The fix:
- handleGetMe now emits profileViewFromUser(user) — the same
shape PUT /api/me/profile already returns (meProfileResp:
id, username, display_name, email, is_admin).
- auth.UserFromContext already returns a full dbq.User row, so no
extra DB lookup needed.
- Web's User TypeScript type is narrower than this response but
doesn't care about the extra fields (TS structural typing).
- LoginResp.User still uses UserView; login response unchanged.
New test asserts the regression directly: a user with stored
display_name + email sees them in /api/me. Old test updated to
decode into meProfileResp and assert the nullable fields are
correctly null for an unset profile.
Android side needs no change — the existing wire shape already
expected display_name + email; this just delivers them.
Five findings + one cancelled duplicate from the 2026-06-02 drift
audit (Scribe parent task #552):
- **#561 (Android)** PlayerController.playbackErrorEventsChannel was
Channel.CONFLATED. The PlaybackErrorReporter coroutine reads it in
a debounce loop that buffers events to coalesce into "Skipped N
unplayable tracks" — but CONFLATED silently dropped every emission
except the latest each time the reader wasn't actively pulling.
A network blip that failed 5 tracks back-to-back surfaced only the
last failure to the snackbar AND only POSTed one playback_errors
row to the admin inbox. Switch to BUFFERED (default capacity 64,
well above any plausible burst rate). Coalescing path now reaches
N > 1 and the admin inbox sees every failure.
- **#563 (server)** systemPlaylistSources rotation whitelist in
playevents/writer.go had drifted behind the migrations. It listed
only for_you + discover; migrations 0021 + 0028 added 6 more
variants (deep_cuts, rediscover, new_for_you, on_this_day,
first_listens, songs_like_artist) that ship as refreshable system
mixes. Plays from those surfaces never advanced the per-user
rotation, so "unplayed first" ordering staled — the same tracks
kept resurfacing. Add all 6 to the map; comment now points at the
migration's CHECK list as the canonical source so future variants
notice the requirement. #573 was the duplicate auditor hit for
the same drift; cancelled in Scribe.
- **#564 (Android)** Android emitted source = "playlist:<variant>"
for system-mix plays from Home and PlaylistDetail, but the
server's rotation matcher keys on the BARE variant string (web
sends the bare form — PlaylistCard.svelte:83). Misalignment meant
system-mix plays from Android never advanced rotation; switching
from web to Android effectively reset the perceived "unplayed
next" ordering. Fix HomeScreen.kt:291 to send bare variant and
PlaylistDetailScreen.kt's play() to prefer systemVariant over the
playlist:<id> tag when the playlist is a refreshable system mix.
User playlists keep playlist:<id> (intentional — rotation only
applies to system mixes anyway).
- **#568 + #569 (Android)** AuthCookieInterceptor was unconditionally
attaching the Minstrel session cookie to every outgoing request
AND wiping the session on any 401. The shared OkHttpClient is also
used by Coil for external image fetches (artwork.musicbrainz.org,
coverartarchive.org, Lidarr /MediaCover URLs); this leaked the
session cookie to those hosts (privacy posture) AND silently
signed users out of Minstrel if any external image host returned
401. Scope both attach + clear to the placeholder.invalid sentinel
host the same way BaseUrlInterceptor was scoped in aec10ce7. Two
new regression tests cover the external-host pass-through. Existing
tests rewritten to make requests through the placeholder URL so
they exercise the in-scope path explicitly.
All five Scribe tasks updated to in_progress at start, will flip to
done after CI green on this push.
bvandeusen