Four arms of the candidate query ended in a bare `ORDER BY random()` with no
seed: similar_artists, likes_overlap, coplay_artists and random_fill.
Such an arm returns a STABLE set only while its LIMIT exceeds the rows
eligible for it — at that point it returns all of them and the order stops
mattering, because scoreAndSortCandidates sorts by track id before drawing
jitter. Below that threshold it returns a random SUBSET, and two builds on
the same day draw different ones.
So daily determinism held BY ACCIDENT, and only for libraries smaller than
the limits. Any real library is larger, which means same-day rebuilds have
been producing different mixes since those arms were written — invisible,
because a mix that changes after a refresh looks like a feature rather than
a broken promise.
Found by breaking it: cutting RandomFill to 10 while tuning Songs-like
turned TestBuildSystemPlaylists_DailyNonceDeterminism red. That test seeds
~20 tracks against a default RandomFill of 30, so its determinism came from
the limit exceeding the library, not from the code being right. It is now a
real guard.
The arms order by md5(id || $12) instead. The CALLER decides what that
means, which is the point: system mixes pass a per-(user, day) seed and get
the determinism they promise, radio passes a fresh value per request and
keeps varying, which is what a radio should do. Same shape the browse
queries in this file already use (`md5(id::text || current_date::text)`) —
existing idiom, not a new one.
This also unblocks the trim that #3881 wanted and could not have. Shrinking
a randomly-ordered arm was what broke membership; a seeded one takes a
smaller but REPRODUCIBLE slice. Songs-like's seed-independent share drops
from 29% to 12%, which was the original intent before determinism forced it
back to 20%.
TestSongsLikeLimits_DoNotShrinkTheUnseededRandomArms is DELETED rather than
kept passing. It existed to stop anyone trimming those arms while the
ordering was broken; the ordering is fixed, so the constraint is gone and a
guard enforcing it would now forbid correct code.
Was filed as blocked on tooling. It was not: `make generate-go` runs sqlc as
a pinned Go tool and is the same path CI takes.
One thing worth knowing for next time: three files in internal/db/dbq are
owned by root, left by `make generate` running sqlc in Docker. sqlc errored
on the first it could not write. They are untouched by this change and the
regeneration of recommendation.sql.go completed, but `make generate` will
keep failing until they are chowned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
Operator, 2026-09-10: "is the pool that we draw from somehow scaled to the
amount of music in the library... my earlier understanding of the tuning and
work may have been skewed by what was in my library."
It was not. DefaultCandidateSourceLimits returns what its own comment calls
"the v1 hardcoded constants per spec" — ~170 candidates for a 500-track
library and a 100,000-track one alike. The pool therefore samples a
shrinking FRACTION of a growing collection: 17% of 1,000 tracks, 1.7% of
10,000, 0.17% of 100,000. RandomFill, whose whole job is exploration,
becomes a thinner and noisier slice at exactly the moment a library gets
more diverse — which is the "starting to feel weird" being reported.
1,000 tracks -> pool 170 (unchanged)
5,000 -> pool 170 (unchanged)
20,000 -> pool 280
80,000 -> pool 500 (ceiling)
THE SCALING IS PER-ARM, and that is the substance rather than a refinement.
A limit only matters if there are rows for it to cut off, so what an arm is
BOUNDED BY decides whether library size can help it. LBSimilar,
SimilarArtist, TagOverlap and RandomFill grow: they are bounded by
similarity/tag data and by the library itself. LikesOverlap, UserCoplay and
TasteOverlap do not: they are bounded by the user's likes, the instance's
co-play graph and the taste profile, none of which grow when the library
does. Raising those would sample more of a set that did not change — churn,
not reach. It also keeps this from inflating the sim_score-0 share, since
TasteOverlap is one of the two zero-similarity arms.
sqrt, not linear: linear would put a 100,000-track library at a
3,400-candidate pool, long past where more candidates improve the answer.
A 4x ceiling bounds it at ~500.
Never shrinks an arm. The base limits are a floor, and #3889 makes that
load-bearing rather than tidy — shrinking an arm ordered by unseeded
random() changes pool membership between same-day rebuilds.
Library size comes from a TTL-cached count reusing CountTracksMatching with
an empty pattern (rule 28 — a new query would need sqlc regeneration, which
is blocked). The ILIKE defeats every index, so it is a full scan and must
not run per request. It degrades rather than fails: an error keeps the last
known value, a never-counted cache returns 0, and 0 scales to the base
limits — today's behaviour exactly. Nothing about sizing a pool justifies
failing the request it is sizing. Bounded by a 3s deadline (rule 156), and
a failed refresh does not stamp the clock, so a blip cannot pin a stale
value for the whole TTL.
THE REFERENCE IS ASSUMED, NOT MEASURED. libraryScaleReference = 5000 is
where growth starts, and the size the v1 constants were really tuned against
is unrecorded. #3879 should replace it; until then that constant is the one
thing to change. Deliberately conservative: below it nothing scales at all,
so no existing install changes behaviour.
Falsification caught a weak guard: the sqrt-vs-linear assertion was written
at SIXTEEN times the reference, where linear has already been clamped by the
ceiling and both curves land on 4x. It proved nothing. Moved to four times
the reference, below the ceiling for both, where sqrt gives 2x and linear
would give 4x.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
Operator, 2026-09-10: started radio from a song and "literally all of the
songs in the playlist after that were from a single artist which was not
expected."
There was no per-artist cap anywhere in the radio path. radio.go built the
pool and handed it straight to Shuffle, which scores, sorts and takes the
top N — nothing between those steps bounded any artist's share, so a pool
dominated by one artist produced an output dominated by it. The asymmetry
was the tell: discover.go, you_might_like.go and home.go all cap; radio
never got one.
With the fixture that reproduces it — 20 liked tracks by one artist plus 10
by ten others — the old path returns 10 tracks from 1 artist. It now returns
10 from 8.
TWO PASSES, and that is the whole design. A hard cap was the easy mistake:
radio asks for 50 tracks by default and 200 at most, so capping at three per
artist over a concentrated pool would hand back a six-track "radio". Pass
one takes candidates that fit under the caps; pass two fills any remaining
slots from those it skipped, still in score order. The result always holds
min(limit, len(candidates)) — the caps change WHICH tracks are picked, never
HOW MANY. Rule 131's principle past the system mixes it was written for.
The caps SCALE with the requested length rather than being a constant.
Three-per-artist is a sensible 12% of a 25-track mix and an absurd 1.5% of a
200-track radio, where every selection would sit in the relaxation path and
the cap would be decorative. RadioDiversityCaps holds the system mixes'
proportion at any length: 3/2 at 25, 6/4 at 50, 24/16 at 200, with floors so
a very short radio is not capped down to one track per artist.
A BOUND, NOT AN EXCLUSION — the operator asked for the opposite of removal:
"again it should be able to add songs from the same artist." The dominant
artist still appears, just not exclusively. Guarded, because the tempting
wrong fix is the filter songs-like used to carry.
Shuffle grew the parameter rather than gaining a capped twin: radio is its
only production caller, so a second function would have left the original
dead (rule 22).
Falsified against each named regression: uncapped gives 10/10 to one artist;
a hard cap returns 3 of 10 on a single-artist pool; a cap-as-exclusion drops
the artist entirely; a fixed cap stays 3 where the scaled one reaches 24.
Caught while writing the guards: the artist-key constant was hand-written
hex and wrong — the fixture's artist UUID carries 0001 in its fourth group,
so the lookup missed and the assertion measured nothing. Derived from the
same construction the fixture uses now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
Milestone #160 Opt 3b. Adds device class as a third context axis on top
of the #1531 time-of-day/weekday affinity: on the radio path, a candidate
is boosted when its artist concentrates in the current (daypart × weekday
× device) cell. Client-sent (client_id is opaque; no UA stored), so it's
captured going forward and applies to radio only (daily mixes are
cron-built with no device → stay device-agnostic).
Server:
- Migration 0048: play_events.device_class text NULL (no CHECK; normalized
in Go — one whitelist entry per new client class, not a migration).
- events.go: eventRequest.device_class + normalizeDeviceClass (whitelist →
mobile/web/…, else "other", empty → NULL); threaded through both
RecordPlayStartedWithSource and RecordOfflinePlay into InsertPlayEvent.
- ListArtistContextPlayCountsForUser gains a current-device param; the cell
FILTER adds AND ($2='' OR device_class=$2) — '' reproduces the #1531
time-only behaviour exactly (used by mixes). SessionVector.DeviceClass
carries it; the radio handler derives the current device from the user's
latest play (GetLatestPlayDeviceClassForUser) — request-free proxy.
- No new tuning knob: device narrows the existing ContextAffinityScore
(reuses context_time_weight).
Clients:
- web: play_started sends device_class 'web'.
- android: play_started + offline replay send 'mobile' (EventsWire +
PlayOfflinePayload + MutationReplayer + PlayEventsReporter).
Test: LoadContextAffinity device-narrowing integration test (mobile vs web
artist separation; device-agnostic parity).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The recommendation scoring knobs move out of YAML (radio profile) and
out of the systemMixWeights hard-code (daily_mix profile) into
DB-backed settings with live effect (#1250) — the defaults-discovery
lab per decision #1247: the operator turns knobs to find good values,
which then get baked back into shipped defaults; end users and other
operators should never need the card.
- Migration 0040: recommendation_weight_profiles (radio / daily_mix,
8 weight columns), taste_tuning singleton (engagement half-life +
completion-curve points), recommendation_tuning_audit (one row per
change with a {field, old, new} diff — the trend view's markers,
#1251).
- internal/recsettings: boot reconcile seeds shipped defaults without
clobbering tuned rows (coverart SettingsService pattern), validates
patches (bounds, curve ordering), writes audit rows, and pushes
daily_mix weights + taste config into package playlists. No-op
patches write no audit row.
- playlists gains SetSystemMixWeights / SetTasteConfig swap points
under a RWMutex — no signature threading through the producers; the
scheduler's taste rebuild reads the pushed config.
- Radio reads its weight profile from the service per request; the 8
weight fields leave config.RecommendationConfig (YAML keeps only
RecentlyPlayedHours / RadioSize / RadioSizeMax).
- Admin API: GET/PATCH/reset under /api/admin/recommendation-tuning,
echoing current + shipped values.
- Web: new admin Tuning tab — two weight profiles side by side, taste
card, per-scope save (changed fields only) + reset, deviation dots
against shipped defaults.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
The profile built in phase 1 now changes what gets surfaced. Adds a TasteMatch
term to the weighted-shuffle score so candidates are re-ranked by their fit to
the user's learned taste (positive draws toward it; negative reflects passive
avoidance; 0 at cold start).
- recommendation/score.go: ScoringInputs.TasteMatchScore ([-1,+1]) +
ScoringWeights.TasteWeight + the term in Score.
- recommendation/taste.go: LoadTasteProfile reads the taste_profile_* tables;
TasteProfile.Match blends the candidate's artist weight (0.7) and avg genre-tag
weight (0.3), each tanh-squashed by a fixed scale so one outlier artist can't
compress the rest. Unknown artist/tags and empty profiles → 0 (neutral).
- candidates.go: both candidate loaders set TasteMatchScore per candidate, so
every Score caller (system playlists incl. You-might-like, radio) becomes
taste-aware automatically.
- weights: systemMixWeights.TasteWeight = 1.5 (daily mixes are the primary
taste surface); config.RecommendationConfig gains taste_weight (default 1.0,
lighter — radio is seed-directed) wired into the radio handler.
- tests: pure (Match curve incl. saturation/clamp/empty-neutral, Score term
add+subtract) + DB round-trip (seed taste rows → Match positive). All green
vs real Postgres; existing playlist/radio tests unaffected (empty profile →
zero taste effect).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire LoadCandidatesFromSimilarity as the primary candidate loader with
an ?exclude= query param for comma-separated UUID filtering; fall back
to LoadCandidates on error. Thread SimilarityWeight into ScoringWeights
and update testHandlers recCfg accordingly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wire contextual scoring end-to-end: add ContextWeight to RecommendationConfig
(struct + Default()), fetch the user's current session vector in handleRadio
via loadCurrentSessionVector (cold-start returns Seed sentinel), and pass it
as the 6th arg to LoadCandidates so ContextualMatchScore flows into Shuffle.
Validates seed_track + optional limit (default cfg.Recommendation.RadioSize,
clamped to RadioSizeMax). Calls recommendation.LoadCandidates +
recommendation.Shuffle. Prepends seed to result. Server seeds a
math/rand source at startup; handlers package threads that as a
func() float64 so tests inject deterministic RNGs.
Mount + server.New gain a RecommendationConfig parameter.
M6 stub. Validates seed_track param (400 on missing/blank/bad UUID),
looks up the track (404 on miss), returns RadioResponse with a single
TrackRef. M4 will replace the body with similarity-driven candidate
pool + scoring; the request/response shape is final.
bvandeusenandClaude Opus 5