minstrel/docs/superpowers/specs/2026-04-27-m3-shuffle-design.md

M3 Weighted Shuffle v1 — Design Spec

Status: approved 2026-04-27 Slice: M3 sub-plan #1 of 3 (recommendation engine v1 + contextual likes). Spec §13 step 7. Step 8 (session vectors + contextual_match_score) is the next sub-plan; this slice ships the scoring foundation without the contextual term. Fable task: #340.

Goal

Replace the M2 stub /api/radio (currently returns just the seed track) with a real weighted-shuffle implementation that scores user library tracks against a per-track-stats formula and returns a top-N queue ordered by score. After this slice, clicking a track in the SPA's search/track-row "play radio" surface produces a meaningful 50-track queue instead of a one-track stub.

Non-goals

• Session vectors / contextual_match_score (sub-plan #2 / Fable #341 covers the write path; sub-plan #3 / Fable #342 folds the score into this function).
• Real "radio" candidate pools from ListenBrainz similarity / similar artists / similar tags (M4).
• Album-seed or artist-seed radio (?seed_album=, ?seed_artist=). v1 is track-seeded only; the Fable task covers that scope explicitly.
• Periodic radio refresh (the spec's "regenerate at 80% consumed"). Player-side concern; lands later.
• Persistent radio queues. Each /api/radio call returns a fresh selection — stateless on the server.
• Performance optimization for very large libraries (>50k tracks). At v1 scale the wide-pool scan is fine; M3.5 / M4 can add sampling if telemetry shows it matters.
• Subsonic star-radio compatibility. Subsonic's getRandomSongs is a different shape; we don't try to map weighted shuffle into it for v1.

Architecture

A new internal/recommendation package owns:

• A pure scoring function Score(inputs, weights, now, rng) → float64 with no DB dependency. The rng parameter is injectable so tests pin jitter to deterministic values.
• A candidate loader LoadCandidates(ctx, q, userID, seedID, recentlyPlayedHours) → []Candidate that runs ONE SQL query joining tracks, general_likes, and aggregated play_events to produce the per-track stats the scoring function needs. Excludes the seed and any track played within the recently-played window.
• A Shuffle(candidates, weights, now, rng, limit) orchestrator that scores each candidate, sorts descending, truncates to limit. Pure.

The /api/radio handler becomes a thin shim: validate the seed → call LoadCandidates → call Shuffle → prepend the seed to the result → project to []TrackRef → return JSON.

Why split into three layers:

• score.go is the math; trivially unit-testable, no infra.
• candidates.go is the data access; integration-tested against a live DB.
• shuffle.go is the composition; pure-function tests against fake candidate sets.
• The HTTP handler is the I/O glue; lives in api/radio.go like the existing pattern.

This separation also sets up sub-plan #3 (contextual scoring) cleanly — we'll add LoadContextualSimilarity next to LoadCandidates, extend ScoringInputs with ContextualMatchScore, and the rest of the chain doesn't change.

Schema

No migration. All inputs are computed on demand from existing tables:

• general_likes for is_general_liked.
• play_events for last_played_at, play_count, skip_count. Aggregations done in the candidate-loader query.
• tracks provides the candidate set itself.

The recently-played-in-the-last-hour filter is applied at candidate-load time (a single WHERE NOT EXISTS (... AND started_at > now() - interval '1 hour') clause). Hard suppression — these tracks aren't even scored.

Scoring math

type ScoringInputs struct {
    IsGeneralLiked bool
    LastPlayedAt   *time.Time  // nil = never played
    PlayCount      int           // total play_events
    SkipCount      int           // play_events with was_skipped=true
}

type ScoringWeights struct {
    BaseWeight      float64  // default 1.0
    LikeBoost       float64  // default 2.0
    RecencyWeight   float64  // default 1.0
    SkipPenalty     float64  // default 1.0
    JitterMagnitude float64  // default 0.1
}

func Score(in ScoringInputs, w ScoringWeights, now time.Time, rng func() float64) float64 {
    s := w.BaseWeight
    if in.IsGeneralLiked {
        s += w.LikeBoost
    }
    s += recencyDecay(in.LastPlayedAt, now) * w.RecencyWeight
    s -= skipRatio(in.PlayCount, in.SkipCount) * w.SkipPenalty
    s += (rng()*2 - 1) * w.JitterMagnitude
    return s
}

recencyDecay(lastPlayed *time.Time, now time.Time) float64 — returns [0, 1]:

• Never played (lastPlayed == nil) → 1.0. Cold-start tracks compete favorably with stale ones; otherwise the recommendation engine never surfaces music the user hasn't tried.
• Played within last hour: doesn't reach this function (filtered earlier at the candidate-pool stage).
• Otherwise: min(age_days / 30.0, 1.0). Linear ramp; tracks ≥ 30 days stale hit max recency boost.

skipRatio(plays, skips int) float64 — returns [0, 1]:

• plays == 0 → 0.0. Don't penalize never-played tracks.
• Otherwise → float64(skips) / float64(plays).

Random jitter — (rng()*2 - 1) * JitterMagnitude produces values in [-magnitude, +magnitude]. The rng parameter is func() float64 (matches math/rand.Float64); tests inject a fixed-value version for deterministic ordering.

Score range under defaults:

• Min (unliked, recent, all-skips): 1.0 + 0 + 0 - 1.0 - 0.1 = -0.1
• Max (liked, ≥30d stale, never skipped): 1.0 + 2.0 + 1.0 - 0 + 0.1 = 4.1
• Liked-track structural advantage (LikeBoost = 2.0) dominates the jitter band (±0.1), so liked tracks always rank above identical unliked tracks.

Configurable via YAML / env. Operators can crank LikeBoost up if their library is dominated by stuff they don't actually like, or JitterMagnitude up if they want more variety.

API contracts

Request:

GET /api/radio?seed_track=<uuid>&limit=<int>

• seed_track (required) — UUID of the track that seeds the radio.
• limit (optional, default 50, max 200) — total number of tracks to return including the seed.

Response (shape unchanged from M2 stub, only contents grew):

type RadioResponse = { tracks: TrackRef[] };

• tracks[0] is always the seed track (so playQueue(resp.tracks, 0) plays the seed).
• tracks[1..] are the top-scored candidates from the user's library, descending. Length up to limit - 1.
• Cold-start (empty library beyond the seed): returns { tracks: [<seed>] }.

Errors (unchanged from M2 stub):

• 400 bad_request — missing seed_track, malformed UUID, or limit < 1.
• 404 not_found — seed_track doesn't exist.
• 500 server_error — DB issue.

No web client changes. The existing playRadio(seedTrackId) already calls this endpoint and feeds resp.tracks into playQueue(tracks, 0). After this slice the queue is fuller; the call shape is identical.

No Subsonic changes. Track-seeded radio isn't part of Subsonic's surface in our v1 scope.

Components & files

New server files

Path	Responsibility
internal/recommendation/score.go	Pure scoring function + recencyDecay + skipRatio helpers. No DB.
internal/recommendation/score_test.go	Boundary cases: every term, cold-start, jitter determinism, score ranges.
internal/recommendation/candidates.go	LoadCandidates(...) — single SQL query returning []Candidate (Track + ScoringInputs).
internal/recommendation/candidates_test.go	Live-DB tests: seed exclusion, recently-played exclusion, stat-join correctness, cross-user isolation.
internal/recommendation/shuffle.go	Shuffle(candidates, weights, now, rng, limit) []Candidate — composes Score + sort + truncate. Pure.
internal/recommendation/shuffle_test.go	Pure-function tests: liked-rank-higher, high-skip-rejected, jitter-doesn't-reorder-structural-winners, limit.
internal/db/queries/recommendation.sql	sqlc query: LoadRadioCandidates — SELECT tracks WITH LEFT JOIN general_likes, LEFT JOIN aggregated play_events stats. WHERE clause excludes seed_id and last-hour plays.
internal/db/dbq/recommendation.sql.go	Generated bindings.

Modified server files

Path	Change
internal/api/radio.go	Replace stub. New flow: validate seed_track and limit, call recommendation.LoadCandidates, call recommendation.Shuffle, prepend seed, project to []TrackRef, return JSON.
internal/api/radio_test.go	Replace stub tests with: cold-start, typical (seed + scored picks), 404 on unknown seed, 400 on bad seed/limit, cross-user isolation.
internal/config/config.go + config.example.yaml	Add RecommendationConfig struct: BaseWeight (1.0), LikeBoost (2.0), RecencyWeight (1.0), SkipPenalty (1.0), JitterMagnitude (0.1), RecentlyPlayedHours (1), RadioSize (50), RadioSizeMax (200). YAML key recommendation:.
internal/api/api.go	handlers struct gains recCfg config.RecommendationConfig. Mount signature gains the config arg; constructs the handler with it; the radio handler builds ScoringWeights from it per request.
internal/server/server.go + cmd/minstrel/main.go	Pass cfg.Recommendation through Mount.

No web changes

The existing playRadio(seedTrackId) already consumes RadioResponse. The web slice for this PR is empty — server-only.

Data flow

1. SPA calls playRadio(seedTrackId) (existing player-store action). It POSTs GET /api/radio?seed_track=<id> (no explicit limit, server defaults to 50).
2. handleRadio validates auth + seed UUID. Looks up the track to confirm it exists (404 otherwise).
3. Reads limit from query (default cfg.Recommendation.RadioSize, clamped to cfg.Recommendation.RadioSizeMax).
4. Calls recommendation.LoadCandidates(ctx, q, userID, seedID, cfg.Recommendation.RecentlyPlayedHours). The SQL query joins:
   • tracks t — base set
   • LEFT JOIN general_likes l ON l.user_id = $1 AND l.track_id = t.id → is_liked
   • LEFT JOIN LATERAL (SELECT max(started_at) FROM play_events WHERE user_id = $1 AND track_id = t.id) pe_last → last_played_at
   • LEFT JOIN LATERAL (SELECT count(*), count(*) FILTER (WHERE was_skipped) FROM play_events WHERE user_id = $1 AND track_id = t.id) pe_stats → play_count, skip_count
   • WHERE t.id <> seed_id AND NOT EXISTS (SELECT 1 FROM play_events WHERE user_id = $1 AND track_id = t.id AND started_at > now() - interval '$2 hours')
5. Iterates the candidates, calls Score(...) per track, sorts descending, truncates to limit - 1.
6. Prepends the seed track, projects each Candidate.Track to TrackRef (existing trackRefFrom helper), writes RadioResponse{ Tracks: ... }.
7. SPA receives 50 tracks, plays from index 0 (the seed).

Stateless — every call recomputes from scratch. No persisted radio queue, no cursor tracking, no "you already heard this" memory beyond the recently-played-hours window.

Testing

Server (go test)

internal/recommendation/score_test.go (pure unit tests):

• Base case (never played, not liked, no skip data) with deterministic RNG () => 0.5 → score = BaseWeight + RecencyWeight + 0 exactly.
• Liked boost: same inputs but IsGeneralLiked=true → score increases by LikeBoost.
• Recency ramp: lastPlayedAt = now - 15d → recencyDecay = 0.5. lastPlayedAt = now - 60d → 1.0 (capped).
• Skip ratio: PlayCount=4, SkipCount=2 → ratio 0.5, score loses 0.5 * SkipPenalty.
• Cold-start skip: PlayCount=0, SkipCount=0 → ratio 0.0, no penalty.
• Jitter bounds: 1000 Score(...) calls with math/rand.Float64, every result within [mid - JitterMagnitude, mid + JitterMagnitude] where mid is the deterministic-RNG score.
• Determinism: same (inputs, weights, now, rng) returns the same score (regression guard for hidden global state).

internal/recommendation/shuffle_test.go (pure unit tests):

• Liked-vs-not: two otherwise-identical candidates → liked one ranks higher.
• High-skip rejected: candidate with skipRatio=1.0 ranks last among otherwise-identical candidates.
• Limit truncates: 100 candidates, limit=10 → result has 10.
• Jitter doesn't reorder structural winners: 100 random RNG seeds → liked track ALWAYS ranks above an equivalent unliked track.
• Empty input → empty output.

internal/recommendation/candidates_test.go (live DB):

• Seed exclusion: 5 tracks in library, ask for radio with one as seed → returns the other 4.
• Recently-played exclusion: seed a play_events row with started_at = now - 30 minutes for track A → radio excludes A.
• Stat join: track with one play + one skip → play_count=1, skip_count=1. Liked track → is_liked=true. Never-played → last_played_at IS NULL.
• Cross-user: Alice's plays / likes do NOT appear in Bob's stats.

internal/api/radio_test.go (HTTP integration, replacing existing stub tests):

• Cold-start: seed only in library → response is { tracks: [seed] }.
• Typical: seed + 5 other tracks → response is [seed, 5 ranked]. Seed always at index 0.
• 404 on unknown seed.
• 400 on missing/malformed seed.
• 400 on limit=0 or limit < 0.
• limit > RadioSizeMax clamped (returns at most RadioSizeMax, no error).
• Cross-user isolation: Alice has many plays + likes → Bob's radio uses Bob's clean stats.

Coverage: go test -coverprofile=cover.out ./internal/recommendation/... → ≥ 70% per the M3 milestone description. The pure-function tests should hit ~100% on score.go + shuffle.go; candidates.go reaches similar coverage via the integration tests.

End-to-end manual

Final task in the implementation plan:

1. Sign in. Play 3 tracks all the way through (build play_count history).
2. Skip a 4th track within 10 seconds (creates a high skip_ratio).
3. Like 2 tracks via the heart button.
4. Click radio on a 5th track.
5. Inspect the response in dev tools network tab — 50 tracks, seed first.
6. The 2 liked tracks appear early in the list (within first ~20).
7. The just-skipped track appears late or absent.
8. The 3 just-played tracks are absent (recently-played exclusion).
9. Trigger radio again with a different seed; ordering varies (jitter), liked tracks still rank prominently.

Risks & mitigations

• Wide-pool scan performance. For a 50k-track library: 50k rows × scoring overhead × sort ≈ a few hundred ms in Go. Below the user-perceptible threshold for a click-to-play. If telemetry later shows >1s P95 latency, M3.5 / M4 can add sampling (random-N candidate pre-filter) or a partial index. The candidate-pool function is the swap point; the scoring function and HTTP handler don't change.
• Recently-played window edge effects. If the user plays 50 tracks in an hour, all of them get excluded from the next radio call. Library < 100 tracks could leave the candidate pool too thin to fill RadioSize. Mitigation: handler returns whatever it can fit (length might be < RadioSize); SPA's playQueue works on any non-empty array. Document as acceptable degradation; users with tiny libraries get short radios.
• Skip-ratio gaming. A user who skips a track once (during initial library discovery) gets a skipRatio=1.0 for that track. With SkipPenalty=1.0 that's a -1.0 hit — could permanently demote the track even if they like it. Mitigation: weights are tunable; user can lower SkipPenalty. Future: smoothing (e.g. (skips + α) / (plays + β)) instead of raw ratio. Out of scope for v1.
• Cold-start RNG dominance. A brand-new user with zero plays / likes gets every track scored at BaseWeight + RecencyWeight + jitter. Top-N is essentially random. That's fine — it matches user expectation ("I haven't told you anything about me, give me random music"). The recency-decay max-for-never-played avoids amplifying new tracks just because they're new.
• Score formula evolution. Adding contextual_match_score in sub-plan #3 means changing ScoringInputs and the Score function signature. Mitigation: the function is internal-package only; sub-plan #3 changes both ends of the call atomically. No external consumers.