Milestone 127: recommendation quality — provenance, tiered mixes, For You v2, tuning lab #107
Merged
bvandeusen
merged 11 commits from 2026-07-03 10:13:50 -04:00
dev into main
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
29e1e7c64c |
test(web/tuning): marker summary appears in tooltips too — use getAllByText
test-web / test (push) Successful in 32s
The knob-turn summary renders both as the sparkline tick tooltips (one per series) and in the list under the chart; the single-element query tripped on the duplicates in CI run 1906. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
|
9ad4343c76 |
feat(tuning): weekly trend view — per-surface series + knob-turn markers
The verify half of the tune→verify loop (#1251), on the same admin Tuning page as the knobs: - RecommendationWeeklyTrends: weekly per-source outcomes aggregated across all users (the knobs are global, so judging a turn needs global outcomes — rows carry rates only, no track/user identity), with a taste-hit count per bucket: plays whose track's artist has a positive weight in the player's current taste profile. That's the "cheap recompute" reading — retroactive over the whole window, at the cost of profile drift. - GET /api/admin/recommendation-trends?weeks=N (default 12, cap 52): per-family weekly series (skip rate, sample-weighted completion, taste-hit rate) plus the tuning-audit markers inside the window. - Web: sparkline table under the tuning cards — skip rate per week on a shared axis with dashed ticks at knob turns, latest-week columns, window taste-hit rate, low-volume rows dimmed as anecdote, and a plain-text list of the window's tuning changes. Also fixes the revive unused-parameter lint on the tuning GET handler that failed CI run 1903 on the previous commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
|
0d0a8f46b1 |
feat(tuning): scoring weights → DB-backed admin tuning lab
The recommendation scoring knobs move out of YAML (radio profile) and out of the systemMixWeights hard-code (daily_mix profile) into DB-backed settings with live effect (#1250) — the defaults-discovery lab per decision #1247: the operator turns knobs to find good values, which then get baked back into shipped defaults; end users and other operators should never need the card. - Migration 0040: recommendation_weight_profiles (radio / daily_mix, 8 weight columns), taste_tuning singleton (engagement half-life + completion-curve points), recommendation_tuning_audit (one row per change with a {field, old, new} diff — the trend view's markers, #1251). - internal/recsettings: boot reconcile seeds shipped defaults without clobbering tuned rows (coverart SettingsService pattern), validates patches (bounds, curve ordering), writes audit rows, and pushes daily_mix weights + taste config into package playlists. No-op patches write no audit row. - playlists gains SetSystemMixWeights / SetTasteConfig swap points under a RWMutex — no signature threading through the producers; the scheduler's taste rebuild reads the pushed config. - Radio reads its weight profile from the service per request; the 8 weight fields leave config.RecommendationConfig (YAML keeps only RecentlyPlayedHours / RadioSize / RadioSizeMax). - Admin API: GET/PATCH/reset under /api/admin/recommendation-tuning, echoing current + shipped values. - Web: new admin Tuning tab — two weight profiles side by side, taste card, per-scope save (changed fields only) + reset, deviation dots against shipped defaults. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
|
9e02878b61 |
feat(playlists): For You composition v2 — multi-seed blend + weighted fresh tail
Two approved composition changes (#1269), mechanism only — the taste/fresh share stays data-decided (#1252) and pick_kind attribution is unchanged. Multi-seed blending: each day's build now seeds from up to 3 of the user's top-5 tracks (pickDailySeeds, the generalized daily shuffle) instead of one rotating anchor, so the mix spans neighborhoods within a day and stops feeling bipolar as the rotation swings between dissimilar seeds. Per-seed pools merge first-seen-deduped; the head is filled best-first under 50/30/20 per-seed quotas (60/40 for two seeds) so one neighborhood can't monopolize it, with thin-seed quota spilling best-first. Score-weighted fresh tail: the tail sample (rank 2*headN onward) was uniform — the 380th-best candidate as likely as the 101st. It now uses deterministic Efraimidis-Spirakis keys with weight halving every 50 ranks, so freshness keeps its "you'll probably enjoy this" half while still rotating daily. The retired single-seed picker's one other caller, You-might-like, moves to pickDailySeeds(n=1) — a single neighborhood per day is right for a short shelf, and the behavior note is inline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
|
48f288e2e5 |
feat(mixes): tiered rebuilds for New for you + First listens (rule #131)
Both mixes move from a single hard eligibility rule to the tiered ladder, with their tier stamped onto playlist_tracks.pick_kind via the #1270 provenance pipeline. New for you (#1267) — consume on play, degrade by stepping back: - "Consumed" = any track attempted >=30s; played albums leave the mix at the next build instead of crowding it until the calendar window expires. - Tier 1: unconsumed albums added <30d by direct-affinity artists. Tier 2: unconsumed affinity albums from the wider 30-90d window — added while you weren't looking. Tier 3: any unconsumed album added <90d, newest first. First listens (#1268) — track-level "attempted" threshold: - A 2-second accidental brush no longer disqualifies a whole album; "attempted" is duration_played_ms >= 30000 per track. - Tier 1: albums with zero attempted tracks. Tier 2: barely-attempted albums (<=25% of tracks reached 30s), minus the attempted tracks themselves. The artist-affinity ordering signal also moves to the >=30s definition so skip-only contact doesn't read as trust. Producer plumbing: fetch adapters map the tier column onto pick kinds, finishMix propagates PickKind into the persisted candidates, and rotateForDay now rotates within contiguous same-pick-kind blocks so daily rotation can't hoist tier-3 filler above tier-1's exact fits (untiered pools are one block — original behavior). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
|
2be07ef271 |
fix(mixes): close three intent gaps found in the system-playlists audit
Three discovery-mix defects from the intent audit (Scribe note #1254), all sharing the same root pattern — skips treated as non-events: - Deep Cuts (#1257): eligibility counted only unskipped plays, so a track skipped twice with zero completed listens read as "barely heard" and kept being re-offered. Tracks with >=2 skips no longer qualify; a single accidental skip doesn't banish. - Rediscover (#1258): a skip on a rediscover-sourced play — the user explicitly declining the resurfacing invitation — changed nothing, so declined tracks re-qualified the next day forever. Such tracks now sit out 90 days. - On This Day (#1256): day-of-year distance used plain ABS, so Dec 28 vs Jan 3 read as 359 days apart and the window silently gutted itself for ~3 weeks around every New Year. Now circular (LEAST(d, 365-d)), anchored on the build-date parameter instead of now() so it's testable and consistent with the mix's daily determinism. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
|
a670840114 |
fix(playlists): Songs-like mixes no longer vanish after a quiet week
PickSeedArtists had a hard 7-day window with no fallback: a week without listening emptied the seed pool, produceSeedMixes returned zero playlists, and the daily atomic-replace build deleted every existing "Songs like X" mix until the user played something again (#1255). The query now falls back through widening engagement windows — 7d → 30d → all-time → liked artists — the same tiered shape that fixed the identical vanish for For You's seeds (PickTopPlayedTracksForUser). Like-boost scoring is preserved in every tier. All returned rows share the winning tier, and produceSeedMixes maps it onto the rule-#131 pick-kind ladder (7d = tier1 exact, 30d = tier2, all-time/liked = tier3) and stamps the built tracks — the #1270 provenance pipeline then attributes plays and skips to seed freshness, so the metrics card can say whether stale-seeded mixes actually perform worse. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
|
5faa57634b |
feat(metrics): provenance as standard — pick_kind for all system mixes
The #1249 mechanism (stamp WHY a track is in the snapshot at build time, freeze it onto the play at ingestion, break it down in metrics) generalizes from a For You one-off to the standard for every system mix (#1270): - Migration 0039 widens both pick_kind CHECKs (drop + re-add in the same change) to taste/fresh + Discover's dormant/cross_user/random + tier1-3 for the rule-#131 eligibility ladders. - GetForYouPickKindForTrack becomes GetSystemPickKindForTrack (user, variant, track); ingestion stamps any systemPlaylistSources play from its own variant's live snapshot, live + offline paths. - Discover stamps its candidate bucket on discoverTrack before the interleave, making the 40/30/30 allocation measurable; dedup keeps the taking bucket's stamp. - Metrics replace the for_you special-case with one pick-kind vocabulary — any family with attributed plays gets a breakdown, future stamping mixes need no metrics change. - Web: breakdown sub-rows are now toggled per surface (collapsed by default) so eight stamping mixes don't swamp the card. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
|
fb4431207d |
feat(recommendation): For You exploration attribution — taste vs fresh picks
Milestone #127 step 2 (#1249). For You deliberately blends two populations — a head of top-scored taste picks and a tail sampled from deeper ranking (the freshness injection) — but the metrics judged it as one blob, so its skip rate couldn't distinguish "the taste engine is missing" from "the freshness tax is too high". That number decides the exploration share before we tune it. - Migration 0038: nullable pick_kind ('taste'|'fresh') on both playlist_tracks (stamped at snapshot build) and play_events (frozen at play-ingestion — the snapshot rebuilds daily, so attribution cannot be reconstructed at read time). - Builder: pickHeadAndTail marks head=taste / tail=fresh; the small-pool fallback is all taste (top-N-by-score IS the taste mechanism). Other variants persist NULL. - Ingestion: for_you plays (live + offline replay) look the track up in the user's current snapshot; not found → unattributed, never guessed. - Metrics: For You's row gains a breakdown (taste / fresh / earlier unattributed plays), parent row stays the sum; web card renders the sub-rows indented with the same baseline deltas + low-data dimming. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
|
60533073ad |
feat(metrics): bucketed surface families + manual-plays baseline (#1248, milestone 127)
The recommendation metrics table was observable but not actionable: raw source strings (album:<uuid> one-offs) drowned the stable surfaces, and manual plays were excluded so skip rates had no control group. - SQL: include NULL-source rows (the baseline) and carry completion_n so family merges can weight avg_completion correctly. - Handler buckets raw sources into stable families (radio:<uuid> → Radio, album:/artist: → direct plays, etc.) grouped by surface intent: go-to / discovery / direct — each band judged against its job, since discovery mixes are expected to skip hotter. Families under 20 plays are flagged low-confidence, not hidden. - Settings card renders the baseline row and per-surface deltas in percentage points vs baseline (worse-than-baseline deltas in danger color), intent hint copy per group, low-data rows dimmed. - Pure-unit test for the bucketing/merge; DB test updated to the new contract (baseline included, radio:<uuid> collapse). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
|
4b150a277e |
fix(recommendation): Rediscover no longer ships a one-song playlist (#1246)
Confirmed against prod: exactly one track (17 plays, cold since May 21) met the c>=5 + 30d-cold bar, and three process defects turned that into a 1-track playlist instead of the locked placeholder. - ListRediscoverTracks: collapse the two-tier UNION into one blended pool. The old shallow-tier gate (WHERE NOT EXISTS deep) was all-or-nothing — one 6-month row suppressed the entire 30-day tier — and deep was a strict subset of shallow anyway. Eligibility drops to >=3 non-skip plays (on a weeks-old history the >=5-play tracks are precisely the ones still in rotation); ordering prefers >=6mo cold, then >=5 plays, then raw count. - Minimum viable mix floor for all five discovery mixes: below minLen (15; 5 for the album-coherent NewForYou/FirstListens) the variant is withheld so Home renders the 'listen more to unlock' placeholder instead of a mix that reads as built-wrong. - /api/events: clamp client-supplied 'at' to [user.created_at, now+5m]. Unbounded client clocks could write arbitrarily old plays and poison the 6-month ordering (prod data verified clean — no scrub needed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |