304de88c50c81119e9f2f8b2e7b85c6b8dd7ed05
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 bvandeusen
|
481f906059 |
feat(metrics): publish margin of error on every delta — #2495, #2524
The metrics card had one volume threshold doing two jobs. recMetricsLowVolume = 20 is a DISPLAY floor — below that a skip rate is anecdote — but the card then presented deltas as though it were also a DECISION floor. Those differ by an order of magnitude: detecting the ~13pp differences that matter needs ~133 plays per arm for 80% power at a=0.05. So Discover's taste-matched (59 plays) and random-unheard (70) both rendered as full-confidence rows with a bold delta beside them, and that comparison sits at p ~ 0.06. The card said "signal"; the arithmetic said "maybe". It produced a recommendation the data didn't support, and any reader with the same numbers would have made the same call. Deltas now carry a 95% margin of error and a `distinguishable` flag, computed server-side so both clients read the same arithmetic instead of each re-deriving it. Skip rate is a two-proportion difference; completion is Welch, which needs a variance — hence completion_sqsum in the query. It is the sum of squares rather than stddev_samp on purpose: raw source rows are merged into surface families in Go, and sums of squares combine across groups exactly whereas standard deviations cannot. recMetricsLowVolume is untouched. "Too thin to show" and "too thin to act on" are different questions. Web renders an indistinguishable delta as dimmed and prefixed "≈", with the range on hover and a legend explaining the glyph. Colour is withheld unless the delta clears its margin — colouring noise red is what made the old card misleading. Breakdown rows go through the same path; those are the thinnest samples on screen and where the old card misled most. Also fixes the admin trends view, which had the same problem worse: its "Latest skip"/"Latest completion" columns are one WEEK while the adjacent Plays column is the whole window. I misread exactly that and briefly concluded Deep cuts was the worst surface, from ~17 plays in a single week — over 180 days it is one of the best. Headers now name their period and the skip cell carries that week's play count. #2524: resolveArtist now recognises a duplicate-MBID unique violation as the expected condition it is, matching resolveAlbum. Two rows mapping to one MusicBrainz artist is a merge candidate, not a fault; without the branch it logged a generic warning plus a Postgres ERROR line on every scan, which teaches an operator to ignore database errors. |
||
|
bvandeusen
|
7226dab9ff |
feat(discover): taste-targeted novelty bucket + rebalance toward discovery
The 2-week metrics review found Discover beating the manual baseline on skip rate — which for a discovery surface means it plays it safe. On a single-user server it's effectively dormant + crude-random, and the per-user taste profile (taste_profile_tags, #796) went unused (#1254 gap). Add a fourth Discover candidate bucket, ListTasteUnheardTracksForDiscover: unheard / non-liked / non-quarantined tracks ranked by summed taste-tag weight over tracks.genre (split like the radio tag_overlap arm), md5 tiebreak. Its picks are stamped pick_kind = 'taste_unheard' (migration 0041 widens the CHECK on playlist_tracks + play_events, rule #36). Rebalance the slot allocation 40/30/30 → taste_unheard 35 / dormant 30 / cross_user 20 / random 15, and lead the interleave with taste_unheard so a track shared with another bucket keeps the taste stamp and the targeted-novelty arm stays measurable. Metrics label "Taste-matched" + order entry added to the single server-side pickKindLabels map, so web and Android surface the new breakdown row with no client change. Cold start (empty taste_profile_tags) yields an empty taste bucket that redistributes to the survivors, so Discover still fills. Scribe #1488. Companion review outcomes: Songs-like starvation already fixed (#1255); For You v2 ratified as-is (fresh-injection cost disproven). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 bvandeusenandClaude Fable 5
|
5faa57634b |
feat(metrics): provenance as standard — pick_kind for all system mixes
The #1249 mechanism (stamp WHY a track is in the snapshot at build time, freeze it onto the play at ingestion, break it down in metrics) generalizes from a For You one-off to the standard for every system mix (#1270): - Migration 0039 widens both pick_kind CHECKs (drop + re-add in the same change) to taste/fresh + Discover's dormant/cross_user/random + tier1-3 for the rule-#131 eligibility ladders. - GetForYouPickKindForTrack becomes GetSystemPickKindForTrack (user, variant, track); ingestion stamps any systemPlaylistSources play from its own variant's live snapshot, live + offline paths. - Discover stamps its candidate bucket on discoverTrack before the interleave, making the 40/30/30 allocation measurable; dedup keeps the taking bucket's stamp. - Metrics replace the for_you special-case with one pick-kind vocabulary — any family with attributed plays gets a breakdown, future stamping mixes need no metrics change. - Web: breakdown sub-rows are now toggled per surface (collapsed by default) so eight stamping mixes don't swamp the card. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
bvandeusenandClaude Fable 5
|
fb4431207d |
feat(recommendation): For You exploration attribution — taste vs fresh picks
Milestone #127 step 2 (#1249). For You deliberately blends two populations — a head of top-scored taste picks and a tail sampled from deeper ranking (the freshness injection) — but the metrics judged it as one blob, so its skip rate couldn't distinguish "the taste engine is missing" from "the freshness tax is too high". That number decides the exploration share before we tune it. - Migration 0038: nullable pick_kind ('taste'|'fresh') on both playlist_tracks (stamped at snapshot build) and play_events (frozen at play-ingestion — the snapshot rebuilds daily, so attribution cannot be reconstructed at read time). - Builder: pickHeadAndTail marks head=taste / tail=fresh; the small-pool fallback is all taste (top-N-by-score IS the taste mechanism). Other variants persist NULL. - Ingestion: for_you plays (live + offline replay) look the track up in the user's current snapshot; not found → unattributed, never guessed. - Metrics: For You's row gains a breakdown (taste / fresh / earlier unattributed plays), parent row stays the sum; web card renders the sub-rows indented with the same baseline deltas + low-data dimming. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
bvandeusenandClaude Fable 5
|
60533073ad |
feat(metrics): bucketed surface families + manual-plays baseline (#1248, milestone 127)
The recommendation metrics table was observable but not actionable: raw source strings (album:<uuid> one-offs) drowned the stable surfaces, and manual plays were excluded so skip rates had no control group. - SQL: include NULL-source rows (the baseline) and carry completion_n so family merges can weight avg_completion correctly. - Handler buckets raw sources into stable families (radio:<uuid> → Radio, album:/artist: → direct plays, etc.) grouped by surface intent: go-to / discovery / direct — each band judged against its job, since discovery mixes are expected to skip hotter. Families under 20 plays are flagged low-confidence, not hidden. - Settings card renders the baseline row and per-surface deltas in percentage points vs baseline (worse-than-baseline deltas in danger color), intent hint copy per group, low-data rows dimmed. - Pure-unit test for the bucketing/merge; DB test updated to the new contract (baseline included, radio:<uuid> collapse). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6 |
||
|
bvandeusenandClaude Opus 4.8
|
1a7515e6ea |
feat(taste): phase 4 — recommendation observability (#796)
Per-source play outcomes so the operator can see whether each recommendation
surface is landing and tune the now-operator-tunable taste weights.
Server:
- query RecommendationSourceMetricsForUser: groups the user's play_events by
source (system-playlist surface), reporting plays / skips / avg completion
over a window; NULL-source (library/radio) plays excluded.
- GET /api/me/recommendation-metrics?days=30 (default 30, capped 365) →
{window_days, sources:[{source, plays, skips, skip_rate, avg_completion}]}.
- handler test: 401 unauth; per-source aggregation + NULL-source exclusion +
skip_rate / avg_completion math.
Web:
- lib/api/metrics.ts: query + friendly source labels.
- settings page gains a "Recommendation metrics" card (table of surface / plays
/ skip rate / avg completion), with loading/error/empty states.
- settings tests mock the new query (manual subscribe-store, hoisting-safe).
Note: You-might-like plays aren't source-tagged (it's a Home row, not a system
playlist), so this covers For-You / Discover / the mixes. Tagging YML plays
would be a client follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|