test-go / test (push) Successful in 1m18s
test-go / integration (push) Successful in 4m52s
release / Build signed APK (releases and dev) (push) Successful in 6m9s
release / Build + push container image (push) Successful in 2m5s
release / Verify release artifacts (tag releases only) (push) Skipped
Four arms of the candidate query ended in a bare `ORDER BY random()` with no seed: similar_artists, likes_overlap, coplay_artists and random_fill. Such an arm returns a STABLE set only while its LIMIT exceeds the rows eligible for it — at that point it returns all of them and the order stops mattering, because scoreAndSortCandidates sorts by track id before drawing jitter. Below that threshold it returns a random SUBSET, and two builds on the same day draw different ones. So daily determinism held BY ACCIDENT, and only for libraries smaller than the limits. Any real library is larger, which means same-day rebuilds have been producing different mixes since those arms were written — invisible, because a mix that changes after a refresh looks like a feature rather than a broken promise. Found by breaking it: cutting RandomFill to 10 while tuning Songs-like turned TestBuildSystemPlaylists_DailyNonceDeterminism red. That test seeds ~20 tracks against a default RandomFill of 30, so its determinism came from the limit exceeding the library, not from the code being right. It is now a real guard. The arms order by md5(id || $12) instead. The CALLER decides what that means, which is the point: system mixes pass a per-(user, day) seed and get the determinism they promise, radio passes a fresh value per request and keeps varying, which is what a radio should do. Same shape the browse queries in this file already use (`md5(id::text || current_date::text)`) — existing idiom, not a new one. This also unblocks the trim that #3881 wanted and could not have. Shrinking a randomly-ordered arm was what broke membership; a seeded one takes a smaller but REPRODUCIBLE slice. Songs-like's seed-independent share drops from 29% to 12%, which was the original intent before determinism forced it back to 20%. TestSongsLikeLimits_DoNotShrinkTheUnseededRandomArms is DELETED rather than kept passing. It existed to stop anyone trimming those arms while the ordering was broken; the ordering is fixed, so the constraint is gone and a guard enforcing it would now forbid correct code. Was filed as blocked on tooling. It was not: `make generate-go` runs sqlc as a pinned Go tool and is the same path CI takes. One thing worth knowing for next time: three files in internal/db/dbq are owned by root, left by `make generate` running sqlc in Docker. sqlc errored on the first it could not write. They are untouched by this change and the regeneration of recommendation.sql.go completed, but `make generate` will keep failing until they are chowned. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
240 lines
8.7 KiB
Go
240 lines
8.7 KiB
Go
// you_might_like.go builds the per-user "You might like" Home rows
|
|
// (#790): in-library albums/artists the listener doesn't actively spin
|
|
// but is predicted to enjoy. It reuses the For-You candidate engine
|
|
// (similarity + like-weighted scoring) and rolls the per-track scores up
|
|
// to album/artist granularity, gated on a minimum listening history so a
|
|
// near-empty profile ships nothing rather than noise.
|
|
//
|
|
// Built in the same daily BuildSystemPlaylists pass and atomic-replaced
|
|
// alongside the system playlists; read back by /api/home.
|
|
package playlists
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"log/slog"
|
|
"math/rand"
|
|
"sort"
|
|
"time"
|
|
|
|
"github.com/jackc/pgx/v5/pgtype"
|
|
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/recommendation"
|
|
)
|
|
|
|
const (
|
|
// Cold-start gate. Below this much *real* listening the similarity
|
|
// roll-up degrades toward random fill, so You-might-like ships
|
|
// nothing rather than arbitrary tiles. Both bars must clear: distinct
|
|
// tracks rules out "played 3 songs on repeat," distinct artists rules
|
|
// out "hammered one album." Conservative defaults; raise if early
|
|
// rows feel random on thin libraries.
|
|
youMightLikeMinDistinctTracks = 20
|
|
youMightLikeMinDistinctArtists = 5
|
|
|
|
// Persisted depth — a few more than the rendered HomeYouMightLikeLimit
|
|
// so the read-time cross-section dedup (vs Most Played / Rediscover)
|
|
// has headroom before trimming.
|
|
youMightLikeAlbumsN = 12
|
|
youMightLikeArtistsN = 12
|
|
|
|
// One strongly-matched artist shouldn't fill the albums row.
|
|
youMightLikeMaxAlbumsPerArtist = 2
|
|
|
|
// Aggregation: sum of an entity's top-K track scores. Favors albums/
|
|
// artists with several good matches over a single outlier track.
|
|
youMightLikeAggTopK = 3
|
|
)
|
|
|
|
// youMightLikeResult carries one day's roll-up. built=false means the
|
|
// computation failed and the caller must leave any existing rows intact;
|
|
// built=true with nil slices means "explicitly empty" (cold-start gate or
|
|
// no candidates) and the caller should clear the user's rows.
|
|
type youMightLikeResult struct {
|
|
albumIDs []pgtype.UUID
|
|
artistIDs []pgtype.UUID
|
|
built bool
|
|
}
|
|
|
|
// buildYouMightLike computes the user's daily album/artist rolls. Reads
|
|
// only — the caller persists inside the build tx. Never returns an error:
|
|
// a failure logs and yields built=false so the prior day's rows survive.
|
|
func buildYouMightLike(
|
|
ctx context.Context, q *dbq.Queries, logger *slog.Logger,
|
|
userID pgtype.UUID, dateStr string, now time.Time,
|
|
) youMightLikeResult {
|
|
signal, err := q.CountListeningSignalForUser(ctx, userID)
|
|
if err != nil {
|
|
logger.Warn("you-might-like: listening-signal query failed; skipping",
|
|
"user_id", uuidStringPL(userID), "err", err)
|
|
return youMightLikeResult{built: false}
|
|
}
|
|
if signal.DistinctTracks < youMightLikeMinDistinctTracks ||
|
|
signal.DistinctArtists < youMightLikeMinDistinctArtists {
|
|
// Cold start: not enough listening to recommend from. built=true
|
|
// clears any stale rows (normally none) and ships an empty row.
|
|
return youMightLikeResult{built: true}
|
|
}
|
|
|
|
seeds, err := q.PickTopPlayedTracksForUser(ctx, userID)
|
|
if err != nil {
|
|
logger.Warn("you-might-like: seed query failed; skipping",
|
|
"user_id", uuidStringPL(userID), "err", err)
|
|
return youMightLikeResult{built: false}
|
|
}
|
|
// One rotating seed is right here (unlike For-You's multi-seed
|
|
// blend, #1269): the row is a short shelf, not a mix, and a single
|
|
// neighborhood per day keeps it coherent.
|
|
daily := pickDailySeeds(seeds, userID, dateStr, 1)
|
|
if len(daily) == 0 {
|
|
return youMightLikeResult{built: true}
|
|
}
|
|
seed := daily[0]
|
|
|
|
zeroVec := recommendation.SessionVector{Seed: true}
|
|
// You-might-like surfaces in-library artists the user does NOT actively
|
|
// engage with, so it deliberately skips the taste_overlap arm — that arm
|
|
// pulls top-taste (mostly already-played) artists, which would crowd the
|
|
// pool with entities the read-time dedup then strips. For-You/radio keep it.
|
|
ymlLimits := systemForYouSourceLimits()
|
|
ymlLimits.TasteOverlap = 0
|
|
cands, err := recommendation.LoadCandidatesFromSimilarity(
|
|
ctx, q, userID, seed, 1, zeroVec,
|
|
[]pgtype.UUID{seed}, ymlLimits,
|
|
dailyOrderSeed(userID, dateStr),
|
|
)
|
|
if err != nil {
|
|
logger.Warn("you-might-like: candidate load failed; skipping",
|
|
"user_id", uuidStringPL(userID), "err", err)
|
|
return youMightLikeResult{built: false}
|
|
}
|
|
|
|
albumIDs, artistIDs := rollUpCandidates(cands, userID, dateStr, now)
|
|
return youMightLikeResult{albumIDs: albumIDs, artistIDs: artistIDs, built: true}
|
|
}
|
|
|
|
// scoredEntity is an album or artist with its aggregated score.
|
|
type scoredEntity struct {
|
|
id pgtype.UUID
|
|
score float64
|
|
}
|
|
|
|
// rollUpCandidates scores every candidate track (systemMixWeights, jitter
|
|
// seeded by userIDHash so near-ties rotate day-to-day) and aggregates the
|
|
// scores up to album and artist via sum-of-top-K. Returns the top-N album
|
|
// and artist IDs, with a per-artist cap on the album list.
|
|
func rollUpCandidates(
|
|
cands []recommendation.Candidate, userID pgtype.UUID, dateStr string, now time.Time,
|
|
) (albumIDs, artistIDs []pgtype.UUID) {
|
|
rng := rand.New(rand.NewSource(int64(userIDHash(userID, dateStr))))
|
|
weights := currentSystemMixWeights()
|
|
albumScores := map[pgtype.UUID][]float64{}
|
|
artistScores := map[pgtype.UUID][]float64{}
|
|
albumArtist := map[pgtype.UUID]pgtype.UUID{}
|
|
for _, c := range cands {
|
|
s := recommendation.Score(c.Inputs, weights, now, rng.Float64)
|
|
if c.Track.AlbumID.Valid {
|
|
albumScores[c.Track.AlbumID] = append(albumScores[c.Track.AlbumID], s)
|
|
albumArtist[c.Track.AlbumID] = c.Track.ArtistID
|
|
}
|
|
if c.Track.ArtistID.Valid {
|
|
artistScores[c.Track.ArtistID] = append(artistScores[c.Track.ArtistID], s)
|
|
}
|
|
}
|
|
|
|
rankedAlbums := capAlbumsPerArtist(
|
|
rankEntities(albumScores, dateStr), albumArtist, youMightLikeMaxAlbumsPerArtist)
|
|
albumIDs = topNEntityIDs(rankedAlbums, youMightLikeAlbumsN)
|
|
artistIDs = topNEntityIDs(rankEntities(artistScores, dateStr), youMightLikeArtistsN)
|
|
return albumIDs, artistIDs
|
|
}
|
|
|
|
// rankEntities aggregates each entity's track scores (sum-of-top-K) and
|
|
// returns them sorted by score DESC, ties broken deterministically by
|
|
// tieBreakHash(id, dateStr) so ordering is stable within a day.
|
|
func rankEntities(scoresByEntity map[pgtype.UUID][]float64, dateStr string) []scoredEntity {
|
|
out := make([]scoredEntity, 0, len(scoresByEntity))
|
|
for id, scores := range scoresByEntity {
|
|
out = append(out, scoredEntity{id: id, score: sumTopK(scores, youMightLikeAggTopK)})
|
|
}
|
|
sort.SliceStable(out, func(i, j int) bool {
|
|
if out[i].score != out[j].score {
|
|
return out[i].score > out[j].score
|
|
}
|
|
return tieBreakHash(out[i].id, dateStr) < tieBreakHash(out[j].id, dateStr)
|
|
})
|
|
return out
|
|
}
|
|
|
|
// sumTopK sorts scores DESC and sums the highest k.
|
|
func sumTopK(scores []float64, k int) float64 {
|
|
sort.Sort(sort.Reverse(sort.Float64Slice(scores)))
|
|
sum := 0.0
|
|
for i := 0; i < len(scores) && i < k; i++ {
|
|
sum += scores[i]
|
|
}
|
|
return sum
|
|
}
|
|
|
|
// capAlbumsPerArtist drops albums beyond maxPerArtist for any one artist,
|
|
// preserving input (score) order.
|
|
func capAlbumsPerArtist(
|
|
albums []scoredEntity, albumArtist map[pgtype.UUID]pgtype.UUID, maxPerArtist int,
|
|
) []scoredEntity {
|
|
perArtist := map[pgtype.UUID]int{}
|
|
out := make([]scoredEntity, 0, len(albums))
|
|
for _, a := range albums {
|
|
art := albumArtist[a.id]
|
|
if art.Valid {
|
|
if perArtist[art] >= maxPerArtist {
|
|
continue
|
|
}
|
|
perArtist[art]++
|
|
}
|
|
out = append(out, a)
|
|
}
|
|
return out
|
|
}
|
|
|
|
// topNEntityIDs truncates to n and projects the IDs in rank order.
|
|
func topNEntityIDs(entities []scoredEntity, n int) []pgtype.UUID {
|
|
if len(entities) > n {
|
|
entities = entities[:n]
|
|
}
|
|
out := make([]pgtype.UUID, 0, len(entities))
|
|
for _, e := range entities {
|
|
out = append(out, e.id)
|
|
}
|
|
return out
|
|
}
|
|
|
|
// persistYouMightLike atomic-replaces the user's You-might-like rows
|
|
// inside the build tx. Called only when the roll-up was freshly built;
|
|
// a failed computation leaves the prior rows untouched.
|
|
func persistYouMightLike(
|
|
ctx context.Context, qtx *dbq.Queries, userID pgtype.UUID, r youMightLikeResult,
|
|
) error {
|
|
if err := qtx.DeleteYouMightLikeAlbumsForUser(ctx, userID); err != nil {
|
|
return fmt.Errorf("delete you-might-like albums: %w", err)
|
|
}
|
|
for i, id := range r.albumIDs {
|
|
if err := qtx.InsertYouMightLikeAlbum(ctx, dbq.InsertYouMightLikeAlbumParams{
|
|
UserID: userID, AlbumID: id, Rank: int32(i),
|
|
}); err != nil {
|
|
return fmt.Errorf("insert you-might-like album: %w", err)
|
|
}
|
|
}
|
|
if err := qtx.DeleteYouMightLikeArtistsForUser(ctx, userID); err != nil {
|
|
return fmt.Errorf("delete you-might-like artists: %w", err)
|
|
}
|
|
for i, id := range r.artistIDs {
|
|
if err := qtx.InsertYouMightLikeArtist(ctx, dbq.InsertYouMightLikeArtistParams{
|
|
UserID: userID, ArtistID: id, Rank: int32(i),
|
|
}); err != nil {
|
|
return fmt.Errorf("insert you-might-like artist: %w", err)
|
|
}
|
|
}
|
|
return nil
|
|
}
|