bvandeusen/minstrel
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 52 Wiki Activity

docs: query comments say what each query selects and why, not how it came to be (#6277)

Browse Source 
Rewrites the internal/db/queries comments to rule 32 and regenerates
dbq, whose doc comments and embedded SQL carry them. The bugs behind the
tiered mixes, the md5 order seed, the score bands and the round-robin
arms are restated as the constraint each one enforces; counts from one
library and stale cross-references are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
bvandeusen
2026-10-23 12:31:13 -04:00
co-authored by Claude Opus 5.5
parent 7e6228c13f
commit 1d4ef1a23f
100 changed files with 648 additions and 698 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/db/queries/recommendation.sql
+42 -49
View File
@@ -24,7 +24,7 @@ LEFT JOIN LATERAL (
WHERE user_id = $1 AND track_id = t.id
) pe ON true
WHERE t.id <> $2
AND t.missing_since IS NULL -- #2523: never offer a file that is gone
AND t.missing_since IS NULL -- never offer a file that is gone
AND NOT EXISTS (
SELECT 1 FROM play_events
WHERE user_id = $1 AND track_id = t.id
@@ -36,25 +36,22 @@ WHERE t.id <> $2
);
-- name: LoadRadioCandidatesV2 :many
-- M4c: similarity-driven candidate pool. 6-way UNION:
-- Similarity-driven candidate pool. 6-way UNION:
-- $1 user_id, $2 seed_track_id, $3 recently_played_hours,
-- $4 exclude (uuid[]), $5 lb_similar K, $6 similar_artists K,
-- $7 tag_overlap K, $8 likes_overlap K, $9 random_fill K,
-- $10 taste_overlap K (#796 phase 2b — tracks by the user's top
-- $10 taste_overlap K (tracks by the user's top
-- positively-weighted taste-profile artists, so taste-relevant tracks
-- enter the pool even when the similarity/random arms miss them; scored
-- in Go via TasteMatch, so sim_score here is 0 pool-inclusion),
-- $11 coplay_artists K (#1533 — tracks by artists co-played across the
-- $11 coplay_artists K (tracks by artists co-played across the
-- instance with the seed's artist; source='user_cooccurrence'),
-- $12 order_seed (text) — see below.
--
-- $12 REPLACES `ORDER BY random()` IN FOUR ARMS (#3889). Those arms returned
-- a stable set only while their LIMIT exceeded the rows eligible for them: at
-- that point they returned all of them and the order stopped mattering,
-- because the caller sorts by track id before scoring. Below that threshold
-- they returned a random SUBSET, and two builds on the same day drew
-- different ones — so "daily determinism" held by accident, and only for
-- libraries smaller than the limits.
-- $12 orders four arms by md5(id || seed), not random(). An arm whose LIMIT is
-- below the rows eligible for it returns a SUBSET, and with random() two
-- builds on the same day would draw different ones; the caller sorts by track
-- id before scoring, so only the subset matters.
--
-- md5(id || seed) keeps the intent — an arbitrary spread that changes when
-- the seed does — while making it reproducible for a given seed. The CALLER
@@ -63,7 +60,7 @@ WHERE t.id <> $2
-- keeps varying, which is what a radio should do.
-- Returns same shape as LoadRadioCandidates plus similarity_score column.
--
-- sim_score BANDS keep the tiers in order (rule 131, #6177). Similarity edges
-- sim_score BANDS keep the eligibility tiers in order. Similarity edges
-- are stored relative to the seed's best match, 0 to 1, so the arms are placed
-- in bands rather than compared raw:
-- tier 1 lb_similar 0.5 + 0.5 × edge (0.5, 1.0]
@@ -71,9 +68,9 @@ WHERE t.id <> $2
-- tag_overlap 0.5 × share of the seed's tags
-- likes_overlap 0.3, flat
-- tier 3 taste_overlap, random_fill 0
-- Without the bands, the seed's weaker LB edges fell below a shared genre tag
-- and the flat likes score, and the one arm that measures the seed lost to
-- the ones that guess.
-- Without the bands, the seed's weaker LB edges would fall below a shared
-- genre tag and the flat likes score, and the one arm that measures the seed
-- would lose to the ones that guess.
WITH
seed_artist AS (
@@ -108,11 +105,10 @@ lb_similar AS (
LIMIT $5
),
similar_artists AS (
-- Round-robin across the similar artists (#5297): each artist's first
-- Round-robin across the similar artists: each artist's first
-- track, best-scoring artist first, then each one's second, and so on.
-- Ordering by artist score alone let the closest similar artist's
-- catalogue fill the whole LIMIT; on the operator's library this arm held
-- exactly one artist for all 17 seeds measured (#3879).
-- Ordering by artist score alone would let the closest similar artist's
-- catalogue fill the whole LIMIT.
SELECT track_id, sim_score
FROM (
SELECT t.id AS track_id, asim.score * 0.5 AS sim_score, asim.score AS artist_score,
@@ -138,11 +134,10 @@ tag_overlap AS (
AND t.id <> $2
GROUP BY t.id
HAVING count(DISTINCT trim(g_split.g)) > 0
-- md5 breaks the ties (#3880). A seed with one tag gives every track of
-- that genre the same score, 7,804 tracks on the operator's library, and
-- with no tie-break the LIMIT took whichever rows Postgres reached first:
-- the same slice for every seed of the genre, and not one the order seed
-- controls.
-- md5 breaks the ties. A seed with one tag gives every track of that
-- genre the same score, and with no tie-break the LIMIT would take
-- whichever rows Postgres reached first: the same slice for every seed of
-- the genre, and not one the order seed controls.
ORDER BY sim_score DESC, md5(t.id::text || $12::text)
LIMIT $7
),
@@ -161,12 +156,10 @@ likes_overlap AS (
LIMIT $8
),
taste_overlap AS (
-- Round-robin across the taste artists (#3880), as similar_artists is
-- (#5297): each artist's first track, strongest artist first, then each
-- one's second. Ordered by weight then track id, the strongest artist's
-- catalogue filled the whole arm with the same lowest-id tracks every
-- time; on the operator's library that was 20 tracks of one artist of
-- 174, whose catalogue is 197 tracks.
-- Round-robin across the taste artists, as similar_artists is: each
-- artist's first track, strongest artist first, then each one's second.
-- Ordered by weight then track id, the strongest artist's catalogue would
-- fill the whole arm with the same lowest-id tracks every time.
SELECT track_id, sim_score
FROM (
SELECT t.id AS track_id, 0.0::float8 AS sim_score, tpa.weight,
@@ -183,7 +176,7 @@ taste_overlap AS (
LIMIT $10
),
coplay_artists AS (
-- Household co-play (#1533): tracks by artists co-played across the
-- Household co-play: tracks by artists co-played across the
-- instance with the seed's artist (source='user_cooccurrence', built by
-- the coplay worker). Mirrors similar_artists but from local co-occurrence
-- instead of ListenBrainz; empty on single-user servers. Same 0.5 damp as
@@ -237,7 +230,7 @@ FROM (
UNION ALL SELECT track_id, sim_score FROM coplay_artists
UNION ALL SELECT track_id, sim_score FROM random_fill
) u
JOIN tracks t ON t.id = u.track_id AND t.missing_since IS NULL -- #2523: never offer a file that is gone
JOIN tracks t ON t.id = u.track_id AND t.missing_since IS NULL -- never offer a file that is gone
JOIN albums al ON al.id = t.album_id
LEFT JOIN general_likes l ON l.user_id = $1 AND l.track_id = t.id
LEFT JOIN LATERAL (
@@ -255,10 +248,10 @@ GROUP BY t.id, t.title, t.album_id, t.artist_id, t.duration_ms, t.file_path,
-- name: ListArtistContextPlayCountsForUser :many
-- Per-artist completed-play counts split by whether each play falls in the
-- CURRENT context cell, in the user's local timezone (#1531). The cell is
-- daypart × weekday-type, optionally narrowed by device class (#1551): when
-- $2 (the current device) is '', the device dimension is ignored (identical to
-- the time-only #1531 behaviour, used by the daily mixes which have no device);
-- CURRENT context cell, in the user's local timezone. The cell is
-- daypart × weekday-type, optionally narrowed by device class: when
-- $2 (the current device) is '', the device dimension is ignored (time only,
-- for the daily mixes, which have no device);
-- when $2 is set (radio), a play only counts in the cell if its device_class
-- matches. Feeds the context-affinity scoring term. Skips excluded; a 365-day
-- window bounds cost. Daypart buckets: night [22,5) morning [5,12)
@@ -308,7 +301,7 @@ FROM plays p
GROUP BY p.artist_id;
-- name: GetLatestPlayDeviceClassForUser :one
-- The device_class of the user's most recent play (#1551), used as the
-- The device_class of the user's most recent play, used as the
-- "current device" for radio context conditioning. NULL when the latest play
-- predates device capture or came from a client that didn't send one.
SELECT device_class
@@ -321,24 +314,24 @@ LIMIT 1;
-- Per-user artist suggestions ranked by taste signal x similarity, projected
-- through artist_similarity_unmatched (out-of-library candidates only).
--
-- Seeds are TIERED (rule #131) so the surface never empties:
-- Seeds are TIERED so the surface never empties:
-- tier 1 - taste_profile_artists.weight: engagement-graded, time-decayed and
-- SIGNED by internal/taste, so an artist the user has drifted away
-- from stops contributing instead of accumulating forever.
-- tier 2 - likes + completed plays, used ONLY when the profile has no rows
-- (new account, or before the first daily recompute).
--
-- The signal is log-damped: contribution is signal x similarity, and the old
-- undamped sum let one heavily-played artist's neighbours take every slot --
-- entrenching harder the MORE the user listened (issue #2367 mechanism 2).
-- The signal is log-damped: contribution is signal x similarity, and an
-- undamped sum would let one heavily-played artist's neighbours take every
-- slot -- entrenching harder the MORE the user listened.
--
-- Candidates already in the library, already requested and not terminal, or
-- snoozed by this user, are excluded. $1=user_id, $2=half_life_days (tier 2
-- decay), $3=limit.
WITH artist_plays AS (
-- Completed plays only. The previous seed query counted every play_event,
-- so skipping an artist repeatedly INCREASED its signal and pushed more of
-- its neighbours at the user (issue #2367 mechanism 3).
-- Completed plays only: counting every play_event would make skipping an
-- artist repeatedly INCREASE its signal and push more of its neighbours
-- at the user.
SELECT t.artist_id, count(*)::bigint AS play_count
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
@@ -395,7 +388,7 @@ contributions AS (
AND r.artist_mbid = u.candidate_mbid
AND r.status NOT IN ('rejected', 'failed')
)
-- Snoozed by this user and not yet expired (#2374). Time-boxed and
-- Snoozed by this user and not yet expired. Time-boxed and
-- per-user: the candidate returns on its own once snoozed_until
-- passes, and stays visible to everyone else meanwhile. Deliberately
-- filtered at the candidate stage, NOT folded into the score — a
@@ -421,7 +414,7 @@ ORDER BY total_score DESC
LIMIT $3;
-- name: ListMostPlayedTracksForUser :many
-- M6a: top-N tracks by completed-play count for the user. was_skipped
-- Top-N tracks by completed-play count for the user. was_skipped
-- excludes skips so a user spamming next doesn't fabricate a top track.
-- Quarantined tracks (per-user soft-hide from M5b) are filtered out.
--
@@ -442,7 +435,7 @@ FROM plays p
JOIN tracks t ON t.id = p.track_id
JOIN albums ON albums.id = t.album_id
JOIN artists ON artists.id = t.artist_id
WHERE t.missing_since IS NULL -- #2523: never offer a file that is gone
WHERE t.missing_since IS NULL -- never offer a file that is gone
AND NOT EXISTS (
SELECT 1 FROM quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
@@ -469,7 +462,7 @@ JOIN tracks t ON t.id = p.track_id
JOIN albums ON albums.id = t.album_id
JOIN artists ON artists.id = t.artist_id
WHERE t.artist_id = sqlc.arg(artist_id)
AND t.missing_since IS NULL -- #2523: never offer a file that is gone
AND t.missing_since IS NULL -- never offer a file that is gone
AND NOT EXISTS (
SELECT 1 FROM quarantine q
WHERE q.user_id = sqlc.arg(user_id) AND q.track_id = t.id
@@ -478,7 +471,7 @@ ORDER BY p.cnt DESC, t.id
LIMIT sqlc.arg(result_limit);
-- name: ListLastPlayedArtistsForUser :many
-- M6a: artists ranked by max(play_events.started_at) for the user, with
-- Artists ranked by max(play_events.started_at) for the user, with
-- a derived cover_album_id via a representative-album lateral join (most
-- recent album that has cover_art_path set). album_count joined for the
-- ArtistRef wire shape.