Milestone 127: recommendation quality — provenance, tiered mixes, For You v2, tuning lab #107

Merged

bvandeusen merged 11 commits from dev into main 2026-07-03 10:13:50 -04:00

Conversation 0 Commits 11 Files Changed 59

+5508 -504

bvandeusen commented 2026-07-03 10:13:41 -04:00

Owner

Copy Link

Copy Source

Everything from milestone 127 that doesn't require accumulated listening data (the remaining #1252 substance-tuning pass waits on ~1–2 weeks of attributed plays by design), plus the diagnostics work already sitting on dev.

Recommendation quality (this session)

• Provenance as standard (#1270) — pick_kind stamping generalized from a For You one-off to every system mix: migration 0039 widens the vocabulary (taste/fresh + Discover's buckets + tier1–3), ingestion resolves any system source against its own live snapshot, Discover stamps its dormant/cross_user/random buckets, metrics grow a generic per-family breakdown behind a per-surface toggle.
• Audit bug fixes — Songs-like seed vanish after a quiet week (#1255, tiered 7d→30d→all-time→likes fallback with tier stamped); Deep Cuts re-offering actively-skipped tracks (#1257); Rediscover re-offering declined resurfacings (#1258, 90-day cooldown); On This Day year-boundary wraparound (#1256).
• Tiered rebuilds (rule #131) — New for you (#1267): consume-on-play (any ≥30s listen) + affinity/window tiers; First Listens (#1268): track-level ≥30s "attempted" threshold, brushed albums step back to tier 2. Daily rotation is tier-aware.
• For You v2 (#1269) — three blended daily seeds under 50/30/20 head quotas; rank-weighted fresh tail (Efraimidis–Spirakis, half-life 50 ranks).
• Tuning lab (#1250) — scoring weights move out of YAML/hard-code into DB-backed admin settings with live effect (migration 0040): radio + daily-mix profiles, taste half-life + engagement curve, reset-to-shipped, audited changes. New admin Tuning tab.
• Trend view (#1251) — weekly per-surface skip-rate sparklines with knob-turn markers and taste-hit rate, closing the tune→verify loop.

Also included (earlier on dev)

• Android diagnostics: track-identity enrichment + zero stale Sonos state; per-skip track-change events; playback event kind, newest-first sort, active-route subtitle fix; web diagnostics defaults.

All commits CI-green on dev (latest: 29e1e7c6).

🤖 Generated with Claude Code

https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

Everything from milestone 127 that doesn't require accumulated listening data (the remaining #1252 substance-tuning pass waits on ~1–2 weeks of attributed plays by design), plus the diagnostics work already sitting on dev. ## Recommendation quality (this session) - **Provenance as standard (#1270)** — pick_kind stamping generalized from a For You one-off to every system mix: migration 0039 widens the vocabulary (taste/fresh + Discover's buckets + tier1–3), ingestion resolves any system source against its own live snapshot, Discover stamps its dormant/cross_user/random buckets, metrics grow a generic per-family breakdown behind a per-surface toggle. - **Audit bug fixes** — Songs-like seed vanish after a quiet week (#1255, tiered 7d→30d→all-time→likes fallback with tier stamped); Deep Cuts re-offering actively-skipped tracks (#1257); Rediscover re-offering declined resurfacings (#1258, 90-day cooldown); On This Day year-boundary wraparound (#1256). - **Tiered rebuilds (rule #131)** — New for you (#1267): consume-on-play (any ≥30s listen) + affinity/window tiers; First Listens (#1268): track-level ≥30s "attempted" threshold, brushed albums step back to tier 2. Daily rotation is tier-aware. - **For You v2 (#1269)** — three blended daily seeds under 50/30/20 head quotas; rank-weighted fresh tail (Efraimidis–Spirakis, half-life 50 ranks). - **Tuning lab (#1250)** — scoring weights move out of YAML/hard-code into DB-backed admin settings with live effect (migration 0040): radio + daily-mix profiles, taste half-life + engagement curve, reset-to-shipped, audited changes. New admin **Tuning** tab. - **Trend view (#1251)** — weekly per-surface skip-rate sparklines with knob-turn markers and taste-hit rate, closing the tune→verify loop. ## Also included (earlier on dev) - Android diagnostics: track-identity enrichment + zero stale Sonos state; per-skip track-change events; playback event kind, newest-first sort, active-route subtitle fix; web diagnostics defaults. All commits CI-green on dev (latest: 29e1e7c6). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

bvandeusen added 11 commits 2026-07-03 10:13:41 -04:00

fix(recommendation): Rediscover no longer ships a one-song playlist (#1246) ... 4b150a277e

Confirmed against prod: exactly one track (17 plays, cold since May 21)
met the c>=5 + 30d-cold bar, and three process defects turned that into
a 1-track playlist instead of the locked placeholder.

- ListRediscoverTracks: collapse the two-tier UNION into one blended
  pool. The old shallow-tier gate (WHERE NOT EXISTS deep) was
  all-or-nothing — one 6-month row suppressed the entire 30-day tier —
  and deep was a strict subset of shallow anyway. Eligibility drops to
  >=3 non-skip plays (on a weeks-old history the >=5-play tracks are
  precisely the ones still in rotation); ordering prefers >=6mo cold,
  then >=5 plays, then raw count.
- Minimum viable mix floor for all five discovery mixes: below
  minLen (15; 5 for the album-coherent NewForYou/FirstListens) the
  variant is withheld so Home renders the 'listen more to unlock'
  placeholder instead of a mix that reads as built-wrong.
- /api/events: clamp client-supplied 'at' to [user.created_at,
  now+5m]. Unbounded client clocks could write arbitrarily old plays
  and poison the 6-month ordering (prod data verified clean — no
  scrub needed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

feat(metrics): bucketed surface families + manual-plays baseline (#1248, milestone 127) ... 60533073ad

The recommendation metrics table was observable but not actionable: raw
source strings (album:<uuid> one-offs) drowned the stable surfaces, and
manual plays were excluded so skip rates had no control group.

- SQL: include NULL-source rows (the baseline) and carry completion_n
  so family merges can weight avg_completion correctly.
- Handler buckets raw sources into stable families (radio:<uuid> →
  Radio, album:/artist: → direct plays, etc.) grouped by surface
  intent: go-to / discovery / direct — each band judged against its
  job, since discovery mixes are expected to skip hotter. Families
  under 20 plays are flagged low-confidence, not hidden.
- Settings card renders the baseline row and per-surface deltas in
  percentage points vs baseline (worse-than-baseline deltas in danger
  color), intent hint copy per group, low-data rows dimmed.
- Pure-unit test for the bucketing/merge; DB test updated to the new
  contract (baseline included, radio:<uuid> collapse).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

feat(recommendation): For You exploration attribution — taste vs fresh picks ... fb4431207d

Milestone #127 step 2 (#1249). For You deliberately blends two
populations — a head of top-scored taste picks and a tail sampled from
deeper ranking (the freshness injection) — but the metrics judged it as
one blob, so its skip rate couldn't distinguish "the taste engine is
missing" from "the freshness tax is too high". That number decides the
exploration share before we tune it.

- Migration 0038: nullable pick_kind ('taste'|'fresh') on both
  playlist_tracks (stamped at snapshot build) and play_events (frozen at
  play-ingestion — the snapshot rebuilds daily, so attribution cannot be
  reconstructed at read time).
- Builder: pickHeadAndTail marks head=taste / tail=fresh; the small-pool
  fallback is all taste (top-N-by-score IS the taste mechanism). Other
  variants persist NULL.
- Ingestion: for_you plays (live + offline replay) look the track up in
  the user's current snapshot; not found → unattributed, never guessed.
- Metrics: For You's row gains a breakdown (taste / fresh / earlier
  unattributed plays), parent row stays the sum; web card renders the
  sub-rows indented with the same baseline deltas + low-data dimming.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

feat(metrics): provenance as standard — pick_kind for all system mixes ... 5faa57634b

The #1249 mechanism (stamp WHY a track is in the snapshot at build
time, freeze it onto the play at ingestion, break it down in metrics)
generalizes from a For You one-off to the standard for every system
mix (#1270):

- Migration 0039 widens both pick_kind CHECKs (drop + re-add in the
  same change) to taste/fresh + Discover's dormant/cross_user/random
  + tier1-3 for the rule-#131 eligibility ladders.
- GetForYouPickKindForTrack becomes GetSystemPickKindForTrack
  (user, variant, track); ingestion stamps any systemPlaylistSources
  play from its own variant's live snapshot, live + offline paths.
- Discover stamps its candidate bucket on discoverTrack before the
  interleave, making the 40/30/30 allocation measurable; dedup keeps
  the taking bucket's stamp.
- Metrics replace the for_you special-case with one pick-kind
  vocabulary — any family with attributed plays gets a breakdown,
  future stamping mixes need no metrics change.
- Web: breakdown sub-rows are now toggled per surface (collapsed by
  default) so eight stamping mixes don't swamp the card.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

fix(playlists): Songs-like mixes no longer vanish after a quiet week ... a670840114

PickSeedArtists had a hard 7-day window with no fallback: a week
without listening emptied the seed pool, produceSeedMixes returned
zero playlists, and the daily atomic-replace build deleted every
existing "Songs like X" mix until the user played something again
(#1255).

The query now falls back through widening engagement windows — 7d →
30d → all-time → liked artists — the same tiered shape that fixed the
identical vanish for For You's seeds (PickTopPlayedTracksForUser).
Like-boost scoring is preserved in every tier.

All returned rows share the winning tier, and produceSeedMixes maps it
onto the rule-#131 pick-kind ladder (7d = tier1 exact, 30d = tier2,
all-time/liked = tier3) and stamps the built tracks — the #1270
provenance pipeline then attributes plays and skips to seed freshness,
so the metrics card can say whether stale-seeded mixes actually
perform worse.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

fix(mixes): close three intent gaps found in the system-playlists audit ... 2be07ef271

Three discovery-mix defects from the intent audit (Scribe note #1254),
all sharing the same root pattern — skips treated as non-events:

- Deep Cuts (#1257): eligibility counted only unskipped plays, so a
  track skipped twice with zero completed listens read as "barely
  heard" and kept being re-offered. Tracks with >=2 skips no longer
  qualify; a single accidental skip doesn't banish.

- Rediscover (#1258): a skip on a rediscover-sourced play — the user
  explicitly declining the resurfacing invitation — changed nothing,
  so declined tracks re-qualified the next day forever. Such tracks
  now sit out 90 days.

- On This Day (#1256): day-of-year distance used plain ABS, so
  Dec 28 vs Jan 3 read as 359 days apart and the window silently
  gutted itself for ~3 weeks around every New Year. Now circular
  (LEAST(d, 365-d)), anchored on the build-date parameter instead of
  now() so it's testable and consistent with the mix's daily
  determinism.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

feat(mixes): tiered rebuilds for New for you + First listens (rule #131) ... 48f288e2e5

Both mixes move from a single hard eligibility rule to the tiered
ladder, with their tier stamped onto playlist_tracks.pick_kind via the
#1270 provenance pipeline.

New for you (#1267) — consume on play, degrade by stepping back:
- "Consumed" = any track attempted >=30s; played albums leave the mix
  at the next build instead of crowding it until the calendar window
  expires.
- Tier 1: unconsumed albums added <30d by direct-affinity artists.
  Tier 2: unconsumed affinity albums from the wider 30-90d window —
  added while you weren't looking. Tier 3: any unconsumed album added
  <90d, newest first.

First listens (#1268) — track-level "attempted" threshold:
- A 2-second accidental brush no longer disqualifies a whole album;
  "attempted" is duration_played_ms >= 30000 per track.
- Tier 1: albums with zero attempted tracks. Tier 2: barely-attempted
  albums (<=25% of tracks reached 30s), minus the attempted tracks
  themselves. The artist-affinity ordering signal also moves to the
  >=30s definition so skip-only contact doesn't read as trust.

Producer plumbing: fetch adapters map the tier column onto pick kinds,
finishMix propagates PickKind into the persisted candidates, and
rotateForDay now rotates within contiguous same-pick-kind blocks so
daily rotation can't hoist tier-3 filler above tier-1's exact fits
(untiered pools are one block — original behavior).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

feat(playlists): For You composition v2 — multi-seed blend + weighted fresh tail ... 9e02878b61

Two approved composition changes (#1269), mechanism only — the
taste/fresh share stays data-decided (#1252) and pick_kind
attribution is unchanged.

Multi-seed blending: each day's build now seeds from up to 3 of the
user's top-5 tracks (pickDailySeeds, the generalized daily shuffle)
instead of one rotating anchor, so the mix spans neighborhoods within
a day and stops feeling bipolar as the rotation swings between
dissimilar seeds. Per-seed pools merge first-seen-deduped; the head
is filled best-first under 50/30/20 per-seed quotas (60/40 for two
seeds) so one neighborhood can't monopolize it, with thin-seed quota
spilling best-first.

Score-weighted fresh tail: the tail sample (rank 2*headN onward) was
uniform — the 380th-best candidate as likely as the 101st. It now
uses deterministic Efraimidis-Spirakis keys with weight halving every
50 ranks, so freshness keeps its "you'll probably enjoy this" half
while still rotating daily.

The retired single-seed picker's one other caller, You-might-like,
moves to pickDailySeeds(n=1) — a single neighborhood per day is right
for a short shelf, and the behavior note is inline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

feat(tuning): scoring weights → DB-backed admin tuning lab ... 0d0a8f46b1

The recommendation scoring knobs move out of YAML (radio profile) and
out of the systemMixWeights hard-code (daily_mix profile) into
DB-backed settings with live effect (#1250) — the defaults-discovery
lab per decision #1247: the operator turns knobs to find good values,
which then get baked back into shipped defaults; end users and other
operators should never need the card.

- Migration 0040: recommendation_weight_profiles (radio / daily_mix,
  8 weight columns), taste_tuning singleton (engagement half-life +
  completion-curve points), recommendation_tuning_audit (one row per
  change with a {field, old, new} diff — the trend view's markers,
  #1251).
- internal/recsettings: boot reconcile seeds shipped defaults without
  clobbering tuned rows (coverart SettingsService pattern), validates
  patches (bounds, curve ordering), writes audit rows, and pushes
  daily_mix weights + taste config into package playlists. No-op
  patches write no audit row.
- playlists gains SetSystemMixWeights / SetTasteConfig swap points
  under a RWMutex — no signature threading through the producers; the
  scheduler's taste rebuild reads the pushed config.
- Radio reads its weight profile from the service per request; the 8
  weight fields leave config.RecommendationConfig (YAML keeps only
  RecentlyPlayedHours / RadioSize / RadioSizeMax).
- Admin API: GET/PATCH/reset under /api/admin/recommendation-tuning,
  echoing current + shipped values.
- Web: new admin Tuning tab — two weight profiles side by side, taste
  card, per-scope save (changed fields only) + reset, deviation dots
  against shipped defaults.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

feat(tuning): weekly trend view — per-surface series + knob-turn markers ... 9ad4343c76

The verify half of the tune→verify loop (#1251), on the same admin
Tuning page as the knobs:

- RecommendationWeeklyTrends: weekly per-source outcomes aggregated
  across all users (the knobs are global, so judging a turn needs
  global outcomes — rows carry rates only, no track/user identity),
  with a taste-hit count per bucket: plays whose track's artist has a
  positive weight in the player's current taste profile. That's the
  "cheap recompute" reading — retroactive over the whole window, at
  the cost of profile drift.
- GET /api/admin/recommendation-trends?weeks=N (default 12, cap 52):
  per-family weekly series (skip rate, sample-weighted completion,
  taste-hit rate) plus the tuning-audit markers inside the window.
- Web: sparkline table under the tuning cards — skip rate per week on
  a shared axis with dashed ticks at knob turns, latest-week columns,
  window taste-hit rate, low-volume rows dimmed as anecdote, and a
  plain-text list of the window's tuning changes.

Also fixes the revive unused-parameter lint on the tuning GET handler
that failed CI run 1903 on the previous commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

test(web/tuning): marker summary appears in tooltips too — use getAllByText ... 29e1e7c64c

The knob-turn summary renders both as the sparkline tick tooltips (one
per series) and in the list under the chart; the single-element query
tripped on the duplicates in CI run 1906.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6

bvandeusen merged commit 725ddca950 into main 2026-07-03 10:13:50 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

deps

Dependency update (Renovate)

deps:major

Major dependency bump — held for manual review

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/minstrel#107