Compare commits

..

26 Commits

Author SHA1 Message Date
bvandeusen 725ddca950 Merge pull request 'Milestone 127: recommendation quality — provenance, tiered mixes, For You v2, tuning lab' (#107) from dev into main
release / Build signed APK (tag releases only) (push) Has been skipped
test-go / test (push) Successful in 34s
release / Build + push container image (push) Successful in 36s
test-web / test (push) Successful in 41s
test-go / integration (push) Successful in 4m41s
2026-07-03 10:13:49 -04:00
bvandeusen 29e1e7c64c test(web/tuning): marker summary appears in tooltips too — use getAllByText
test-web / test (push) Successful in 32s
The knob-turn summary renders both as the sparkline tick tooltips (one
per series) and in the list under the chart; the single-element query
tripped on the duplicates in CI run 1906.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-03 09:35:51 -04:00
bvandeusen 9ad4343c76 feat(tuning): weekly trend view — per-surface series + knob-turn markers
test-go / test (push) Successful in 33s
test-web / test (push) Failing after 38s
test-go / integration (push) Successful in 4m44s
The verify half of the tune→verify loop (#1251), on the same admin
Tuning page as the knobs:

- RecommendationWeeklyTrends: weekly per-source outcomes aggregated
  across all users (the knobs are global, so judging a turn needs
  global outcomes — rows carry rates only, no track/user identity),
  with a taste-hit count per bucket: plays whose track's artist has a
  positive weight in the player's current taste profile. That's the
  "cheap recompute" reading — retroactive over the whole window, at
  the cost of profile drift.
- GET /api/admin/recommendation-trends?weeks=N (default 12, cap 52):
  per-family weekly series (skip rate, sample-weighted completion,
  taste-hit rate) plus the tuning-audit markers inside the window.
- Web: sparkline table under the tuning cards — skip rate per week on
  a shared axis with dashed ticks at knob turns, latest-week columns,
  window taste-hit rate, low-volume rows dimmed as anecdote, and a
  plain-text list of the window's tuning changes.

Also fixes the revive unused-parameter lint on the tuning GET handler
that failed CI run 1903 on the previous commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-03 09:29:33 -04:00
bvandeusen 0d0a8f46b1 feat(tuning): scoring weights → DB-backed admin tuning lab
test-go / test (push) Failing after 14s
test-web / test (push) Successful in 34s
test-go / integration (push) Successful in 4m42s
The recommendation scoring knobs move out of YAML (radio profile) and
out of the systemMixWeights hard-code (daily_mix profile) into
DB-backed settings with live effect (#1250) — the defaults-discovery
lab per decision #1247: the operator turns knobs to find good values,
which then get baked back into shipped defaults; end users and other
operators should never need the card.

- Migration 0040: recommendation_weight_profiles (radio / daily_mix,
  8 weight columns), taste_tuning singleton (engagement half-life +
  completion-curve points), recommendation_tuning_audit (one row per
  change with a {field, old, new} diff — the trend view's markers,
  #1251).
- internal/recsettings: boot reconcile seeds shipped defaults without
  clobbering tuned rows (coverart SettingsService pattern), validates
  patches (bounds, curve ordering), writes audit rows, and pushes
  daily_mix weights + taste config into package playlists. No-op
  patches write no audit row.
- playlists gains SetSystemMixWeights / SetTasteConfig swap points
  under a RWMutex — no signature threading through the producers; the
  scheduler's taste rebuild reads the pushed config.
- Radio reads its weight profile from the service per request; the 8
  weight fields leave config.RecommendationConfig (YAML keeps only
  RecentlyPlayedHours / RadioSize / RadioSizeMax).
- Admin API: GET/PATCH/reset under /api/admin/recommendation-tuning,
  echoing current + shipped values.
- Web: new admin Tuning tab — two weight profiles side by side, taste
  card, per-scope save (changed fields only) + reset, deviation dots
  against shipped defaults.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-03 09:22:03 -04:00
bvandeusen 9e02878b61 feat(playlists): For You composition v2 — multi-seed blend + weighted fresh tail
test-go / test (push) Successful in 29s
test-go / integration (push) Successful in 4m37s
Two approved composition changes (#1269), mechanism only — the
taste/fresh share stays data-decided (#1252) and pick_kind
attribution is unchanged.

Multi-seed blending: each day's build now seeds from up to 3 of the
user's top-5 tracks (pickDailySeeds, the generalized daily shuffle)
instead of one rotating anchor, so the mix spans neighborhoods within
a day and stops feeling bipolar as the rotation swings between
dissimilar seeds. Per-seed pools merge first-seen-deduped; the head
is filled best-first under 50/30/20 per-seed quotas (60/40 for two
seeds) so one neighborhood can't monopolize it, with thin-seed quota
spilling best-first.

Score-weighted fresh tail: the tail sample (rank 2*headN onward) was
uniform — the 380th-best candidate as likely as the 101st. It now
uses deterministic Efraimidis-Spirakis keys with weight halving every
50 ranks, so freshness keeps its "you'll probably enjoy this" half
while still rotating daily.

The retired single-seed picker's one other caller, You-might-like,
moves to pickDailySeeds(n=1) — a single neighborhood per day is right
for a short shelf, and the behavior note is inline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-03 09:02:35 -04:00
bvandeusen 48f288e2e5 feat(mixes): tiered rebuilds for New for you + First listens (rule #131)
test-go / test (push) Successful in 29s
test-go / integration (push) Successful in 4m39s
Both mixes move from a single hard eligibility rule to the tiered
ladder, with their tier stamped onto playlist_tracks.pick_kind via the
#1270 provenance pipeline.

New for you (#1267) — consume on play, degrade by stepping back:
- "Consumed" = any track attempted >=30s; played albums leave the mix
  at the next build instead of crowding it until the calendar window
  expires.
- Tier 1: unconsumed albums added <30d by direct-affinity artists.
  Tier 2: unconsumed affinity albums from the wider 30-90d window —
  added while you weren't looking. Tier 3: any unconsumed album added
  <90d, newest first.

First listens (#1268) — track-level "attempted" threshold:
- A 2-second accidental brush no longer disqualifies a whole album;
  "attempted" is duration_played_ms >= 30000 per track.
- Tier 1: albums with zero attempted tracks. Tier 2: barely-attempted
  albums (<=25% of tracks reached 30s), minus the attempted tracks
  themselves. The artist-affinity ordering signal also moves to the
  >=30s definition so skip-only contact doesn't read as trust.

Producer plumbing: fetch adapters map the tier column onto pick kinds,
finishMix propagates PickKind into the persisted candidates, and
rotateForDay now rotates within contiguous same-pick-kind blocks so
daily rotation can't hoist tier-3 filler above tier-1's exact fits
(untiered pools are one block — original behavior).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-03 08:54:20 -04:00
bvandeusen 2be07ef271 fix(mixes): close three intent gaps found in the system-playlists audit
test-go / test (push) Successful in 29s
test-go / integration (push) Successful in 4m36s
Three discovery-mix defects from the intent audit (Scribe note #1254),
all sharing the same root pattern — skips treated as non-events:

- Deep Cuts (#1257): eligibility counted only unskipped plays, so a
  track skipped twice with zero completed listens read as "barely
  heard" and kept being re-offered. Tracks with >=2 skips no longer
  qualify; a single accidental skip doesn't banish.

- Rediscover (#1258): a skip on a rediscover-sourced play — the user
  explicitly declining the resurfacing invitation — changed nothing,
  so declined tracks re-qualified the next day forever. Such tracks
  now sit out 90 days.

- On This Day (#1256): day-of-year distance used plain ABS, so
  Dec 28 vs Jan 3 read as 359 days apart and the window silently
  gutted itself for ~3 weeks around every New Year. Now circular
  (LEAST(d, 365-d)), anchored on the build-date parameter instead of
  now() so it's testable and consistent with the mix's daily
  determinism.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-03 08:47:01 -04:00
bvandeusen a670840114 fix(playlists): Songs-like mixes no longer vanish after a quiet week
test-go / test (push) Successful in 29s
test-go / integration (push) Successful in 4m30s
PickSeedArtists had a hard 7-day window with no fallback: a week
without listening emptied the seed pool, produceSeedMixes returned
zero playlists, and the daily atomic-replace build deleted every
existing "Songs like X" mix until the user played something again
(#1255).

The query now falls back through widening engagement windows — 7d →
30d → all-time → liked artists — the same tiered shape that fixed the
identical vanish for For You's seeds (PickTopPlayedTracksForUser).
Like-boost scoring is preserved in every tier.

All returned rows share the winning tier, and produceSeedMixes maps it
onto the rule-#131 pick-kind ladder (7d = tier1 exact, 30d = tier2,
all-time/liked = tier3) and stamps the built tracks — the #1270
provenance pipeline then attributes plays and skips to seed freshness,
so the metrics card can say whether stale-seeded mixes actually
perform worse.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-03 08:38:54 -04:00
bvandeusen 5faa57634b feat(metrics): provenance as standard — pick_kind for all system mixes
test-go / test (push) Successful in 31s
test-web / test (push) Successful in 38s
test-go / integration (push) Successful in 4m36s
The #1249 mechanism (stamp WHY a track is in the snapshot at build
time, freeze it onto the play at ingestion, break it down in metrics)
generalizes from a For You one-off to the standard for every system
mix (#1270):

- Migration 0039 widens both pick_kind CHECKs (drop + re-add in the
  same change) to taste/fresh + Discover's dormant/cross_user/random
  + tier1-3 for the rule-#131 eligibility ladders.
- GetForYouPickKindForTrack becomes GetSystemPickKindForTrack
  (user, variant, track); ingestion stamps any systemPlaylistSources
  play from its own variant's live snapshot, live + offline paths.
- Discover stamps its candidate bucket on discoverTrack before the
  interleave, making the 40/30/30 allocation measurable; dedup keeps
  the taking bucket's stamp.
- Metrics replace the for_you special-case with one pick-kind
  vocabulary — any family with attributed plays gets a breakdown,
  future stamping mixes need no metrics change.
- Web: breakdown sub-rows are now toggled per surface (collapsed by
  default) so eight stamping mixes don't swamp the card.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-02 23:26:52 -04:00
bvandeusen fb4431207d feat(recommendation): For You exploration attribution — taste vs fresh picks
test-go / test (push) Successful in 31s
test-web / test (push) Successful in 37s
test-go / integration (push) Successful in 4m37s
Milestone #127 step 2 (#1249). For You deliberately blends two
populations — a head of top-scored taste picks and a tail sampled from
deeper ranking (the freshness injection) — but the metrics judged it as
one blob, so its skip rate couldn't distinguish "the taste engine is
missing" from "the freshness tax is too high". That number decides the
exploration share before we tune it.

- Migration 0038: nullable pick_kind ('taste'|'fresh') on both
  playlist_tracks (stamped at snapshot build) and play_events (frozen at
  play-ingestion — the snapshot rebuilds daily, so attribution cannot be
  reconstructed at read time).
- Builder: pickHeadAndTail marks head=taste / tail=fresh; the small-pool
  fallback is all taste (top-N-by-score IS the taste mechanism). Other
  variants persist NULL.
- Ingestion: for_you plays (live + offline replay) look the track up in
  the user's current snapshot; not found → unattributed, never guessed.
- Metrics: For You's row gains a breakdown (taste / fresh / earlier
  unattributed plays), parent row stays the sum; web card renders the
  sub-rows indented with the same baseline deltas + low-data dimming.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-02 20:39:41 -04:00
bvandeusen 60533073ad feat(metrics): bucketed surface families + manual-plays baseline (#1248, milestone 127)
test-go / test (push) Successful in 31s
test-web / test (push) Successful in 37s
test-go / integration (push) Successful in 4m30s
The recommendation metrics table was observable but not actionable: raw
source strings (album:<uuid> one-offs) drowned the stable surfaces, and
manual plays were excluded so skip rates had no control group.

- SQL: include NULL-source rows (the baseline) and carry completion_n
  so family merges can weight avg_completion correctly.
- Handler buckets raw sources into stable families (radio:<uuid> →
  Radio, album:/artist: → direct plays, etc.) grouped by surface
  intent: go-to / discovery / direct — each band judged against its
  job, since discovery mixes are expected to skip hotter. Families
  under 20 plays are flagged low-confidence, not hidden.
- Settings card renders the baseline row and per-surface deltas in
  percentage points vs baseline (worse-than-baseline deltas in danger
  color), intent hint copy per group, low-data rows dimmed.
- Pure-unit test for the bucketing/merge; DB test updated to the new
  contract (baseline included, radio:<uuid> collapse).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-02 18:00:40 -04:00
bvandeusen 4b150a277e fix(recommendation): Rediscover no longer ships a one-song playlist (#1246)
test-go / test (push) Successful in 29s
test-go / integration (push) Successful in 4m35s
Confirmed against prod: exactly one track (17 plays, cold since May 21)
met the c>=5 + 30d-cold bar, and three process defects turned that into
a 1-track playlist instead of the locked placeholder.

- ListRediscoverTracks: collapse the two-tier UNION into one blended
  pool. The old shallow-tier gate (WHERE NOT EXISTS deep) was
  all-or-nothing — one 6-month row suppressed the entire 30-day tier —
  and deep was a strict subset of shallow anyway. Eligibility drops to
  >=3 non-skip plays (on a weeks-old history the >=5-play tracks are
  precisely the ones still in rotation); ordering prefers >=6mo cold,
  then >=5 plays, then raw count.
- Minimum viable mix floor for all five discovery mixes: below
  minLen (15; 5 for the album-coherent NewForYou/FirstListens) the
  variant is withheld so Home renders the 'listen more to unlock'
  placeholder instead of a mix that reads as built-wrong.
- /api/events: clamp client-supplied 'at' to [user.created_at,
  now+5m]. Unbounded client clocks could write arbitrarily old plays
  and poison the 6-month ordering (prod data verified clean — no
  scrub needed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-02 17:29:41 -04:00
bvandeusen d145fee35d Merge pull request 'fix(android/resilience): auto-recover failed loads on reconnect' (#106) from dev into main
android / Build + lint + test (push) Successful in 4m44s
release / Build signed APK (tag releases only) (push) Successful in 4m31s
release / Build + push container image (push) Successful in 16s
2026-07-02 16:45:13 -04:00
bvandeusen 7628330f72 test(android/library): stub SyncController.lastSyncError with a real StateFlow
android / Build + lint + test (push) Successful in 3m45s
The VM now combines lastSyncError into uiState; a relaxed-mock flow
never emits, so the combine never fired and the state sat at Loading.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-02 16:10:45 -04:00
bvandeusen 09102dc615 fix(android/resilience): auto-recover failed loads on reconnect (#1245)
android / Build + lint + test (push) Failing after 3m3s
Failed loads used to stay failed forever: no screen ViewModel listened
to server-health recovery, and the cache-first screens swallowed refresh
errors so a cold load against a down server looked like an empty account.

- connectivity/Recovery.kt: recoveries() — per-collector flow of
  down→Healthy transitions; the screen-level half of the idiom
  SyncController/MutationReplayer/DiagnosticsUploader already use.
- Every screen VM now re-runs its load on recovery (cache-first screens
  unconditionally; direct-load screens when sitting in Error).
- Cache-first error surfacing: Home / Playlists / Liked track refresh
  failure; Library reads SyncController.lastSyncError (new) — empty
  cache + failed refresh now renders Error-with-Retry, not welcome copy.
- Requests: 12s poll also retries from Error (was structurally unable
  to escape it — the in-flight predicate required a Success state).
- Search: retry() bypasses the distinctUntilChanged query pipeline so a
  same-text resubmit after a transient failure actually re-runs.
- ArtistDetail: secondary sections (similar artists / top tracks)
  re-fetch on recovery instead of staying silently absent.
- ErrorRetry: LazyColumn wrapper (pull-to-refresh works on error states,
  same rationale as EmptyState) + optional title; adopted on every error
  branch; PlaylistDetail's one-shot ErrorBlock removed in its favor.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TsF3cNoKrqCYsU78cXC8U6
2026-07-02 16:02:14 -04:00
bvandeusen 611715154b Merge pull request 'M9 diagnostics follow-ups: playback relabel, sort, connected fix, per-skip + track-identity' (#105) from dev into main
test-go / test (push) Successful in 38s
test-web / test (push) Successful in 47s
android / Build + lint + test (push) Successful in 4m18s
test-go / integration (push) Successful in 4m39s
release / Build signed APK (tag releases only) (push) Successful in 3m46s
release / Build + push container image (push) Successful in 16s
2026-06-30 19:19:15 -04:00
bvandeusen 392454b249 feat(android/diagnostics): track-identity enrichment + zero stale Sonos state
android / Build + lint + test (push) Successful in 3m27s
Numeric indices wobble across re-casts (offset +1↔0 seen during output
toggling), making "same track?" ambiguous. Enrich both the track_change
event and the heartbeat with local_track_id (TrackRef.id) and sonos_uri
(RemotePlayerState.currentTrackUri — the URL the speaker is actually
streaming), so a desync is unambiguous.

Also fixes the cast→phone stale-state pollution (#1211): sonos_* is now
zeroed unless a remote route is active, via a shared putSonos() helper —
so a just-ended cast's RemotePlayerState can't masquerade as live Sonos
data in the diagnostics.

Refs Scribe M9 (#119), tasks #1210 #1211.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K55iTxn95BtshocgdE1shW
2026-06-30 19:15:12 -04:00
bvandeusen cdfc79e6ab feat(android/diagnostics): per-skip track-change event
android / Build + lint + test (push) Successful in 3m35s
Heartbeats are 45s apart and missed a rapid skip burst (local_index
16→22 in one gap). Add a 'playback' track_change event emitted on each
queue-index / current-track change, snapshotting local vs Sonos
index+position + server_health + upnp_loading + route — so a transient
skip-induced desync is captured at the instant it happens. (uiState is a
conflated StateFlow, so a very rapid burst may coalesce intermediate
indices; we still get the boundaries + the snapshot.)

Refs Scribe M9 (#119), task #1210.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K55iTxn95BtshocgdE1shW
2026-06-30 18:48:35 -04:00
bvandeusen 79f2d79a2e feat(diagnostics): 'playback' kind, newest-first sort, fix active-route subtitle
test-go / test (push) Successful in 32s
test-web / test (push) Successful in 41s
android / Build + lint + test (push) Successful in 3m40s
test-go / integration (push) Successful in 4m30s
Relabel (#1204): route + player_state events fire for every output route,
not just UPnP — split them into a new 'playback' kind; 'upnp_sync' now
means genuinely UPnP/Sonos signal (drops, resync). Migration 0037 adds
'playback' to the kind CHECK; server whitelist, Android reporter labels,
and the web kind filter updated.

Web sort: the diagnostics list gains a Newest/Oldest-first sort (default
newest at top); export follows the displayed order.

Fix (#1205): OutputRoute.isConnected was derived from RouteInfo.connectionState,
which stays DISCONNECTED for local SYSTEM routes even when active — so a
connected Bluetooth device showed "Available" and reported connected:false.
The picker subtitle now uses isSelected (route == selected route); the dead
isConnected field is removed and the misleading `connected` field dropped
from the diagnostics route event (it only ever logs the active route).

Refs Scribe M9 (#119), tasks #1204 #1205.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K55iTxn95BtshocgdE1shW
2026-06-30 16:33:01 -04:00
bvandeusen 96b15b75e6 feat(web/diagnostics): default to recent 500, move time window to Advanced
test-web / test (push) Successful in 39s
The diagnostics view already defaulted to the most recent 500 events (no
window); make that the obvious path. Device/Kind stay primary; the
start/end window + row cap move into a collapsed "Advanced filters"
disclosure (auto-opens when a window is active) with a "Reset to recent
500" action. Caption now states whether you're seeing the recent default
or a windowed slice.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K55iTxn95BtshocgdE1shW
2026-06-30 12:42:25 -04:00
bvandeusen 23a82fb38d Merge pull request 'M9 — Device diagnostics & debug reporting (connectivity + UPnP desync)' (#104) from dev into main
test-go / test (push) Successful in 35s
test-web / test (push) Successful in 47s
android / Build + lint + test (push) Successful in 4m6s
test-go / integration (push) Successful in 4m37s
release / Build signed APK (tag releases only) (push) Successful in 3m56s
release / Build + push container image (push) Successful in 1m38s
2026-06-29 19:24:16 -04:00
bvandeusen 782f152d37 test(web/admin): AdminTabs now has seven tabs (Diagnostics added)
test-web / test (push) Successful in 31s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K55iTxn95BtshocgdE1shW
2026-06-29 19:11:39 -04:00
bvandeusen bffa5b28bd fix(diagnostics): StateFlow distinctUntilChanged build error + AdminUser test fixtures
test-web / test (push) Failing after 33s
android / Build + lint + test (push) Successful in 3m29s
- DiagnosticsReporter.collectServerHealth: drop distinctUntilChanged() on
  networkStatus.state (StateFlow is already distinct; the deprecation
  warning is a hard error under allWarningsAsErrors).
- web users.test.ts: add debug_mode_enabled to the alice/bob AdminUser
  fixtures now that the field is required on the type.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K55iTxn95BtshocgdE1shW
2026-06-29 19:05:50 -04:00
bvandeusen 8a58f07237 fix(android/diagnostics): keep uploader drain within ReturnCount gate
android / Build + lint + test (push) Failing after 2m28s
drainSafe/drain each had 3 returns (detekt ReturnCount ≤ 2). Collapse the
guard clauses and convert drain's loop to a `more` flag — same behavior,
zero/two returns.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K55iTxn95BtshocgdE1shW
2026-06-29 18:59:05 -04:00
bvandeusen 4d42e298dd feat(android+web/diagnostics): on-device debug reporter + admin timeline (M9)
test-web / test (push) Failing after 11s
android / Build + lint + test (push) Failing after 1m19s
Android: a gated DiagnosticsReporter taps connectivity, server-health,
UPnP drops/player-state/route, power (Doze/battery-opt/screen), and
app fg/bg, plus a heartbeat snapshotting Sonos-vs-local position — the
locked-phone desync signal. Events buffer in a Room ring buffer
(deliberately NOT the MutationQueue: high-volume best-effort telemetry
that must survive the dead zone being debugged) and DiagnosticsUploader
drains them on a tick / health-recovery / sign-in.

Gating: the account flag (users.debug_mode_enabled) reaches the device
via a new /api/me refresh in AuthController; a per-device local OFF
switch lives in Settings. Reporter runs only when enabled && !optOut;
disabling drops the unsent buffer.

Web admin: /admin/diagnostics — pick account+device+kind+time-window,
see a chronological timeline, flip an account's debug mode remotely, and
Copy-JSON / Download-NDJSON the slice for analysis.

Room schema 6→7 (new diagnostic_events table + auth_session.diagnosticsOptOut;
pre-v1 destructive fallback).

Refs Scribe M9 (#119), tasks #1174 #1175 #1176 #1177.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K55iTxn95BtshocgdE1shW
2026-06-29 18:56:36 -04:00
bvandeusen 4ed831d9c3 feat(server/diagnostics): device debug-reporting ingest + admin timeline + retention (M9)
test-go / test (push) Successful in 37s
test-go / integration (push) Successful in 4m44s
New diagnostic_events table + per-account users.debug_mode_enabled flag.
When an account's flag is on, its client(s) POST a batch timeseries of
connectivity / UPnP-sync / power / lifecycle events to /api/diagnostics
(no-op 204 when off, kind whitelist mirrors the CHECK constraint).

Admin surface: GET /api/admin/diagnostics (optional account/device/kind/
time-window filters, RFC3339-or-epoch-ms, export-sized paging) + a
/diagnostics/devices overview + PUT /api/admin/users/{id}/debug-mode to
flip an account remotely while a bug is live. debug_mode_enabled is now
exposed on /api/me (client gate) and the admin user views.

Retention: a 30-day gc-worker sweep (GcPruneDiagnostics), keyed on the
server clock so a skewed device clock can't keep rows alive.

Refs Scribe M9 (#119), tasks #1172 #1173.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K55iTxn95BtshocgdE1shW
2026-06-29 18:38:56 -04:00
121 changed files with 7918 additions and 647 deletions
@@ -10,6 +10,8 @@ import com.fabledsword.minstrel.cache.CacheIndexer
import com.fabledsword.minstrel.cache.mutations.MutationReplayer
import com.fabledsword.minstrel.cache.sync.SyncController
import com.fabledsword.minstrel.di.ApplicationScope
import com.fabledsword.minstrel.diagnostics.DiagnosticsReporter
import com.fabledsword.minstrel.diagnostics.DiagnosticsUploader
import com.fabledsword.minstrel.events.EventsStream
import com.fabledsword.minstrel.events.LiveEventsDispatcher
import com.fabledsword.minstrel.metadata.FreshnessSweeper
@@ -155,6 +157,11 @@ class MinstrelApplication :
*/
@Suppress("unused") @Inject lateinit var liveEventsDispatcher: LiveEventsDispatcher
// Device diagnostics (M9). The reporter gates itself on the account's
// debug flag; the uploader drains its buffer on a tick / recovery.
@Suppress("unused") @Inject lateinit var diagnosticsReporter: DiagnosticsReporter
@Suppress("unused") @Inject lateinit var diagnosticsUploader: DiagnosticsUploader
@Inject @ApplicationScope lateinit var appScope: CoroutineScope
override fun onCreate() {
@@ -4,6 +4,8 @@ import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.admin.data.AdminInvitesRepository
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.models.Invite
import dagger.hilt.android.lifecycle.HiltViewModel
import kotlinx.coroutines.channels.Channel
@@ -25,6 +27,7 @@ data class AdminInvitesUiState(
@HiltViewModel
class AdminInvitesViewModel @Inject constructor(
private val repository: AdminInvitesRepository,
networkStatus: NetworkStatusController,
) : ViewModel() {
private val internal = MutableStateFlow(AdminInvitesUiState())
@@ -36,6 +39,13 @@ class AdminInvitesViewModel @Inject constructor(
init {
refresh()
// Screen-level auto-recovery (issue #1245): reload a failed list
// when server health returns instead of waiting for a manual pull.
viewModelScope.launch {
networkStatus.recoveries().collect {
if (internal.value.message != null) refresh()
}
}
}
fun refresh() {
@@ -29,6 +29,7 @@ import androidx.navigation.NavHostController
import com.fabledsword.minstrel.models.AdminQuarantineItemRef
import com.fabledsword.minstrel.nav.AdminQuarantine
import com.fabledsword.minstrel.shared.widgets.EmptyState
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.LoadingCentered
import com.fabledsword.minstrel.shared.widgets.MinstrelTopAppBar
import com.fabledsword.minstrel.shared.widgets.PullToRefreshScaffold
@@ -62,9 +63,10 @@ fun AdminQuarantineScreen(
body = "When users flag tracks as bad rips, wrong tags, or " +
"duplicates, their reports get aggregated and surfaced here.",
)
is AdminQuarantineUiState.Error -> EmptyState(
is AdminQuarantineUiState.Error -> ErrorRetry(
title = "Couldn't load queue",
body = s.message,
message = s.message,
onRetry = { viewModel.refresh() },
)
is AdminQuarantineUiState.Success -> QueueList(
rows = s.rows,
@@ -4,6 +4,8 @@ import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.admin.data.AdminQuarantineRepository
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.events.EventsStream
import com.fabledsword.minstrel.models.AdminQuarantineItemRef
import dagger.hilt.android.lifecycle.HiltViewModel
@@ -26,6 +28,7 @@ sealed interface AdminQuarantineUiState {
class AdminQuarantineViewModel @Inject constructor(
private val repository: AdminQuarantineRepository,
private val eventsStream: EventsStream,
networkStatus: NetworkStatusController,
) : ViewModel() {
private val internal = MutableStateFlow<AdminQuarantineUiState>(AdminQuarantineUiState.Loading)
@@ -38,6 +41,13 @@ class AdminQuarantineViewModel @Inject constructor(
.filter { it.kind.startsWith("quarantine.") }
.collect { refresh() }
}
// Screen-level auto-recovery (issue #1245): reload a failed list
// when server health returns instead of waiting for a manual pull.
viewModelScope.launch {
networkStatus.recoveries().collect {
if (internal.value is AdminQuarantineUiState.Error) refresh()
}
}
}
fun refresh(): Job = viewModelScope.launch {
@@ -28,6 +28,7 @@ import androidx.navigation.NavHostController
import com.fabledsword.minstrel.models.RequestRef
import com.fabledsword.minstrel.nav.AdminRequests
import com.fabledsword.minstrel.shared.widgets.EmptyState
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.LoadingCentered
import com.fabledsword.minstrel.shared.widgets.MinstrelTopAppBar
import com.fabledsword.minstrel.shared.widgets.PullToRefreshScaffold
@@ -61,9 +62,10 @@ fun AdminRequestsScreen(
body = "When users ask Lidarr for new music, their pending " +
"requests show up here for approval.",
)
is AdminRequestsUiState.Error -> EmptyState(
is AdminRequestsUiState.Error -> ErrorRetry(
title = "Couldn't load requests",
body = s.message,
message = s.message,
onRetry = { viewModel.refresh() },
)
is AdminRequestsUiState.Success -> RequestList(
rows = s.rows,
@@ -5,6 +5,8 @@ import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.admin.data.AdminRequestsRepository
import com.fabledsword.minstrel.admin.data.AdminUsersRepository
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.events.EventsStream
import com.fabledsword.minstrel.models.RequestRef
import dagger.hilt.android.lifecycle.HiltViewModel
@@ -35,6 +37,7 @@ class AdminRequestsViewModel @Inject constructor(
private val repository: AdminRequestsRepository,
private val usersRepository: AdminUsersRepository,
private val eventsStream: EventsStream,
networkStatus: NetworkStatusController,
) : ViewModel() {
private val internal = MutableStateFlow<AdminRequestsUiState>(AdminRequestsUiState.Loading)
@@ -47,6 +50,13 @@ class AdminRequestsViewModel @Inject constructor(
.filter { it.kind in RELEVANT_EVENT_KINDS }
.collect { refresh() }
}
// Screen-level auto-recovery (issue #1245): reload a failed list
// when server health returns instead of waiting for a manual pull.
viewModelScope.launch {
networkStatus.recoveries().collect {
if (internal.value is AdminRequestsUiState.Error) refresh()
}
}
}
fun refresh(): Job = viewModelScope.launch {
@@ -4,6 +4,8 @@ import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.admin.data.AdminUsersRepository
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.models.AdminUserRef
import dagger.hilt.android.lifecycle.HiltViewModel
import kotlinx.coroutines.Job
@@ -23,6 +25,7 @@ sealed interface AdminUsersUiState {
@HiltViewModel
class AdminUsersViewModel @Inject constructor(
private val repository: AdminUsersRepository,
networkStatus: NetworkStatusController,
) : ViewModel() {
private val internal = MutableStateFlow<AdminUsersUiState>(AdminUsersUiState.Loading)
@@ -30,6 +33,13 @@ class AdminUsersViewModel @Inject constructor(
init {
refresh()
// Screen-level auto-recovery (issue #1245): reload a failed list
// when server health returns instead of waiting for a manual pull.
viewModelScope.launch {
networkStatus.recoveries().collect {
if (internal.value is AdminUsersUiState.Error) refresh()
}
}
}
fun refresh(): Job = viewModelScope.launch {
@@ -0,0 +1,36 @@
package com.fabledsword.minstrel.api.endpoints
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.JsonElement
import retrofit2.http.Body
import retrofit2.http.POST
/**
* Retrofit interface for device diagnostics ingest (M9). One call
* uploads a batch of buffered events. The server stores them only when
* the account's debug_mode_enabled flag is on (it returns 204 otherwise,
* which the uploader treats as success so it can drop the batch).
*/
interface DiagnosticsApi {
@POST("api/diagnostics")
suspend fun report(@Body body: DiagnosticsReportRequest)
}
@Serializable
data class DiagnosticsReportRequest(
@SerialName("client_id") val clientId: String,
@SerialName("app_version") val appVersion: String? = null,
@SerialName("os_version") val osVersion: String? = null,
val events: List<DiagnosticEventWire>,
)
@Serializable
data class DiagnosticEventWire(
val kind: String,
// Device-clock epoch milliseconds. The server stamps its own
// received_at; both are stored so a skewed device clock is visible.
@SerialName("occurred_at") val occurredAt: Long,
// Opaque structured payload carrying the event sub-type + fields.
val payload: JsonElement,
)
@@ -1,6 +1,7 @@
package com.fabledsword.minstrel.auth
import com.fabledsword.minstrel.api.endpoints.AuthApi
import com.fabledsword.minstrel.api.endpoints.MeApi
import com.fabledsword.minstrel.di.ApplicationScope
import com.fabledsword.minstrel.models.UserRef
import com.fabledsword.minstrel.models.wire.LoginRequestBody
@@ -35,6 +36,7 @@ class AuthController @Inject constructor(
retrofit: Retrofit,
) {
private val api: AuthApi = retrofit.create()
private val meApi: MeApi = retrofit.create()
private val currentUserState = MutableStateFlow<UserRef?>(null)
val currentUser: StateFlow<UserRef?> = currentUserState.asStateFlow()
@@ -55,6 +57,10 @@ class AuthController @Inject constructor(
currentUserState.value = raw?.let { decodeUser(it) }
}
}
// Refresh from /api/me on startup so a remotely-changed account
// flag (e.g. admin enabling debug mode, M9) reaches the device
// without a re-login. Best-effort; failures keep the cached value.
scope.launch { refreshProfile() }
}
/**
@@ -72,9 +78,32 @@ class AuthController @Inject constructor(
)
currentUserState.value = user
authStore.setUserJson(json.encodeToString(UserRef.serializer(), user))
// Pull the fuller /me shape (carries debug_mode_enabled) right
// after login so the diagnostics gate is correct without waiting
// for the next startup refresh.
scope.launch { refreshProfile() }
return user
}
/**
* Re-fetch the caller's profile from /api/me and update currentUser
* + persisted userJson. Carries account-level flags (debug mode)
* that aren't in the login response. Best-effort: a network failure
* leaves the cached identity untouched. No-op when signed out.
*/
suspend fun refreshProfile() {
if (!isSignedIn) return
val p = runCatching { meApi.getProfile() }.getOrNull() ?: return
val user = UserRef(
id = p.id,
username = p.username,
isAdmin = p.isAdmin,
debugModeEnabled = p.debugModeEnabled,
)
currentUserState.value = user
authStore.setUserJson(json.encodeToString(UserRef.serializer(), user))
}
/**
* Clears the local session immediately and best-effort hits
* `/api/auth/logout` so the server can drop its session row.
@@ -59,6 +59,9 @@ class AuthStore @Inject constructor(
private val cacheSettingsState = MutableStateFlow(CacheSettings.DEFAULT)
val cacheSettings: StateFlow<CacheSettings> = cacheSettingsState.asStateFlow()
private val diagnosticsOptOutState = MutableStateFlow(false)
val diagnosticsOptOut: StateFlow<Boolean> = diagnosticsOptOutState.asStateFlow()
private val json = Json { ignoreUnknownKeys = true }
init {
@@ -70,6 +73,7 @@ class AuthStore @Inject constructor(
themeModeState.value = row?.themeMode
clientIdState.value = row?.clientId
cacheSettingsState.value = decodeCacheSettings(row?.cacheSettingsJson)
diagnosticsOptOutState.value = row?.diagnosticsOptOut ?: false
}
}
}
@@ -112,6 +116,11 @@ class AuthStore @Inject constructor(
scope.launch { persistCacheSettings(encoded) }
}
fun setDiagnosticsOptOut(value: Boolean) {
diagnosticsOptOutState.value = value
scope.launch { persistDiagnosticsOptOut(value) }
}
private suspend fun persistCookie(value: String?) {
if (dao.get() == null) {
dao.upsert(currentEntity().copy(sessionCookie = value))
@@ -160,6 +169,14 @@ class AuthStore @Inject constructor(
}
}
private suspend fun persistDiagnosticsOptOut(value: Boolean) {
if (dao.get() == null) {
dao.upsert(currentEntity().copy(diagnosticsOptOut = value))
} else {
dao.setDiagnosticsOptOut(value)
}
}
private fun currentEntity(): AuthSessionEntity = AuthSessionEntity(
id = ROW_ID,
sessionCookie = sessionCookieState.value,
@@ -171,6 +188,7 @@ class AuthStore @Inject constructor(
CacheSettings.serializer(),
cacheSettingsState.value,
),
diagnosticsOptOut = diagnosticsOptOutState.value,
)
companion object {
@@ -16,6 +16,7 @@ import com.fabledsword.minstrel.cache.db.dao.CachedResumeStateDao
import com.fabledsword.minstrel.cache.db.dao.CachedPlaylistTrackDao
import com.fabledsword.minstrel.cache.db.dao.CachedQuarantineDao
import com.fabledsword.minstrel.cache.db.dao.CachedTrackDao
import com.fabledsword.minstrel.cache.db.dao.DiagnosticEventDao
import com.fabledsword.minstrel.cache.db.dao.SyncMetadataDao
import com.fabledsword.minstrel.cache.db.entities.AudioCacheIndexEntity
import com.fabledsword.minstrel.cache.db.entities.AuthSessionEntity
@@ -30,6 +31,7 @@ import com.fabledsword.minstrel.cache.db.entities.CachedResumeStateEntity
import com.fabledsword.minstrel.cache.db.entities.CachedPlaylistTrackEntity
import com.fabledsword.minstrel.cache.db.entities.CachedQuarantineEntity
import com.fabledsword.minstrel.cache.db.entities.CachedTrackEntity
import com.fabledsword.minstrel.cache.db.entities.DiagnosticEventEntity
import com.fabledsword.minstrel.cache.db.entities.SyncMetadataEntity
/**
@@ -61,8 +63,11 @@ import com.fabledsword.minstrel.cache.db.entities.SyncMetadataEntity
CachedHomeIndexEntity::class,
CachedHistorySnapshotEntity::class,
AuthSessionEntity::class,
DiagnosticEventEntity::class,
],
version = 6,
// v7: + diagnostic_events table (M9) and the diagnosticsOptOut column
// on auth_session. Pre-v1 destructive fallback rebuilds on mismatch.
version = 7,
exportSchema = true,
)
@TypeConverters(MinstrelTypeConverters::class)
@@ -81,4 +86,5 @@ abstract class AppDatabase : RoomDatabase() {
abstract fun cachedHomeIndexDao(): CachedHomeIndexDao
abstract fun cachedHistorySnapshotDao(): CachedHistorySnapshotDao
abstract fun authSessionDao(): AuthSessionDao
abstract fun diagnosticEventDao(): DiagnosticEventDao
}
@@ -10,6 +10,7 @@ import com.fabledsword.minstrel.cache.db.dao.CachedHistorySnapshotDao
import com.fabledsword.minstrel.cache.db.dao.CachedHomeIndexDao
import com.fabledsword.minstrel.cache.db.dao.CachedLikeDao
import com.fabledsword.minstrel.cache.db.dao.CachedMutationDao
import com.fabledsword.minstrel.cache.db.dao.DiagnosticEventDao
import com.fabledsword.minstrel.cache.db.dao.CachedPlaylistDao
import com.fabledsword.minstrel.cache.db.dao.CachedPlaylistTrackDao
import com.fabledsword.minstrel.cache.db.dao.CachedQuarantineDao
@@ -105,5 +106,10 @@ object DatabaseModule {
fun provideAudioCacheIndexDao(db: AppDatabase): AudioCacheIndexDao =
db.audioCacheIndexDao()
@Provides
@Singleton
fun provideDiagnosticEventDao(db: AppDatabase): DiagnosticEventDao =
db.diagnosticEventDao()
private const val DATABASE_NAME = "minstrel.db"
}
@@ -42,4 +42,8 @@ interface AuthSessionDao {
/** Partial update: change only the serialized cache settings. */
@Query("UPDATE auth_session SET cacheSettingsJson = :json WHERE id = 0")
suspend fun setCacheSettingsJson(json: String?)
/** Partial update: change only the per-device diagnostics opt-out. */
@Query("UPDATE auth_session SET diagnosticsOptOut = :optOut WHERE id = 0")
suspend fun setDiagnosticsOptOut(optOut: Boolean)
}
@@ -0,0 +1,35 @@
package com.fabledsword.minstrel.cache.db.dao
import androidx.room.Dao
import androidx.room.Insert
import androidx.room.Query
import com.fabledsword.minstrel.cache.db.entities.DiagnosticEventEntity
@Dao
interface DiagnosticEventDao {
@Insert
suspend fun insert(row: DiagnosticEventEntity): Long
/** FIFO drain order so the uploader sends oldest-first. */
@Query("SELECT * FROM diagnostic_events ORDER BY id ASC LIMIT :limit")
suspend fun takeBatch(limit: Int): List<DiagnosticEventEntity>
@Query("SELECT COUNT(*) FROM diagnostic_events")
suspend fun count(): Int
@Query("DELETE FROM diagnostic_events WHERE id IN (:ids)")
suspend fun deleteByIds(ids: List<Long>)
/**
* Ring-buffer trim: drop the oldest rows beyond [keep]. Called after
* insert so a long offline stretch can't grow the buffer unbounded.
*/
@Query(
"DELETE FROM diagnostic_events WHERE id NOT IN " +
"(SELECT id FROM diagnostic_events ORDER BY id DESC LIMIT :keep)",
)
suspend fun trimToNewest(keep: Int)
@Query("DELETE FROM diagnostic_events")
suspend fun clear()
}
@@ -36,4 +36,11 @@ data class AuthSessionEntity(
* CacheSettings shape evolves.
*/
val cacheSettingsJson: String? = null,
/**
* Per-device opt-out of diagnostics reporting (M9). When the
* account's debug mode is enabled by an admin, the user can still
* turn reporting OFF on this device (battery/privacy) — that local
* choice lives here. Default false = honor the account flag.
*/
val diagnosticsOptOut: Boolean = false,
)
@@ -0,0 +1,28 @@
package com.fabledsword.minstrel.cache.db.entities
import androidx.room.Entity
import androidx.room.PrimaryKey
/**
* One buffered device-diagnostics event (M9). The DiagnosticsReporter
* writes rows here when the account's debug mode is on; the
* DiagnosticsUploader drains them to POST /api/diagnostics and deletes
* on success.
*
* This is a deliberate ring buffer that is NOT routed through the offline
* MutationQueue: diagnostics are high-volume, best-effort telemetry, and
* the bug we're chasing (roaming / dead-zone recovery) happens WHILE
* offline — so events must persist locally through the dead zone and
* upload on recovery, without clogging the user-data mutation replay.
*
* `occurredAtMillis` is the device-clock epoch-ms when the event happened
* (the server also stamps its own received_at). `payloadJson` is an
* opaque JSON object carrying the event sub-type + fields.
*/
@Entity(tableName = "diagnostic_events")
data class DiagnosticEventEntity(
@PrimaryKey(autoGenerate = true) val id: Long = 0,
val kind: String,
val occurredAtMillis: Long,
val payloadJson: String,
)
@@ -1,5 +1,6 @@
package com.fabledsword.minstrel.cache.sync
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.api.endpoints.SyncApi
import com.fabledsword.minstrel.auth.AuthStore
import com.fabledsword.minstrel.connectivity.NetworkStatusController
@@ -17,6 +18,9 @@ import com.fabledsword.minstrel.models.wire.SyncAlbumWire
import com.fabledsword.minstrel.models.wire.SyncArtistWire
import com.fabledsword.minstrel.models.wire.SyncTrackWire
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.flow.distinctUntilChanged
import kotlinx.coroutines.flow.filter
import kotlinx.coroutines.flow.filterNotNull
@@ -64,6 +68,17 @@ class SyncController @Inject constructor(
// pull-to-refresh) can fire concurrently; coalesce them into one pass.
private val mutex = Mutex()
private val lastSyncErrorInternal = MutableStateFlow<String?>(null)
/**
* Human copy for the most recent sync failure; null after any clean
* pass. Lets the Library screen distinguish "empty because the first
* sync failed" (show error + retry) from "genuinely empty library"
* (show welcome copy) — a failed sync over a populated cache stays
* silent, since the cached content is still the better surface.
*/
val lastSyncError: StateFlow<String?> = lastSyncErrorInternal.asStateFlow()
init {
scope.launch {
authStore.sessionCookie
@@ -82,11 +97,13 @@ class SyncController @Inject constructor(
}
}
/** Public entry-point for "Sync now" affordances. Swallows errors. */
/** Public entry-point for "Sync now" affordances. Swallows errors into [lastSyncError]. */
suspend fun syncSafe() {
if (!mutex.tryLock()) return
try {
runCatching { sync() }
.onSuccess { lastSyncErrorInternal.value = null }
.onFailure { lastSyncErrorInternal.value = ErrorCopy.fromThrowable(it) }
} finally {
mutex.unlock()
}
@@ -0,0 +1,27 @@
package com.fabledsword.minstrel.connectivity
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.distinctUntilChanged
import kotlinx.coroutines.flow.drop
import kotlinx.coroutines.flow.filter
import kotlinx.coroutines.flow.map
/**
* Emits once each time server health RETURNS to [ServerHealth.Healthy]
* after the collector subscribed — the StateFlow's replayed current value
* is dropped, so only genuine down→up transitions fire (a screen that
* subscribes while already Healthy doesn't double-load).
*
* This is the screen-level half of the recovery idiom the app-lifetime
* singletons (SyncController / MutationReplayer / DiagnosticsUploader)
* already use: a ViewModel collects this in its viewModelScope and re-runs
* its load, so a surface that failed while the server was unreachable
* heals itself the moment connectivity returns instead of sitting in the
* failed state until a manual pull-to-refresh.
*/
fun NetworkStatusController.recoveries(): Flow<Unit> = state
.map { it == ServerHealth.Healthy }
.distinctUntilChanged()
.drop(1)
.filter { it }
.map { }
@@ -0,0 +1,328 @@
package com.fabledsword.minstrel.diagnostics
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.os.PowerManager
import androidx.lifecycle.DefaultLifecycleObserver
import androidx.lifecycle.LifecycleOwner
import androidx.core.content.ContextCompat
import androidx.lifecycle.ProcessLifecycleOwner
import com.fabledsword.minstrel.auth.AuthController
import com.fabledsword.minstrel.auth.AuthStore
import com.fabledsword.minstrel.cache.db.dao.DiagnosticEventDao
import com.fabledsword.minstrel.cache.db.entities.DiagnosticEventEntity
import com.fabledsword.minstrel.connectivity.ConnectivityObserver
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.di.ApplicationScope
import com.fabledsword.minstrel.player.PlayerController
import com.fabledsword.minstrel.player.RemotePlayerState
import com.fabledsword.minstrel.player.output.OutputPickerController
import com.fabledsword.minstrel.player.output.OutputRoute
import dagger.hilt.android.qualifiers.ApplicationContext
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.combine
import kotlinx.coroutines.flow.distinctUntilChanged
import kotlinx.coroutines.flow.map
import kotlinx.coroutines.launch
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import javax.inject.Inject
import javax.inject.Singleton
/**
* On-device diagnostics collector (M9). Active only while the account's
* debug mode is enabled (server flag, read via AuthController.currentUser)
* AND the user hasn't opted out locally. When active it taps existing
* connectivity / player / UPnP / power seams READ-ONLY and writes a
* timeseries of events into the [DiagnosticEventDao] ring buffer; the
* [DiagnosticsUploader] drains that buffer to the server.
*
* The aim is to explain two failure classes after the fact: roaming /
* poor-data recovery (connectivity + server-health transitions) and
* locked-phone UPnP desync (power/Doze transitions correlated with the
* Sonos-vs-local position deltas captured in the heartbeat).
*/
@Suppress("TooManyFunctions") // one collector + one (un)register pair per signal; cohesive
@Singleton
class DiagnosticsReporter @Inject constructor(
@ApplicationContext private val context: Context,
@ApplicationScope private val scope: CoroutineScope,
private val dao: DiagnosticEventDao,
private val authController: AuthController,
private val authStore: AuthStore,
private val connectivity: ConnectivityObserver,
private val networkStatus: NetworkStatusController,
private val playerController: PlayerController,
private val outputPicker: OutputPickerController,
private val remoteState: RemotePlayerState,
private val json: Json,
) {
@Volatile private var enabled = false
private var collectorJob: Job? = null
private var powerReceiver: BroadcastReceiver? = null
private var lifecycleObserver: DefaultLifecycleObserver? = null
init {
// Gate = account flag (currentUser.debugModeEnabled) AND not the
// per-device local opt-out.
scope.launch {
combine(authController.currentUser, authStore.diagnosticsOptOut) { user, optOut ->
(user?.debugModeEnabled == true) && !optOut
}.distinctUntilChanged().collect { setEnabled(it) }
}
// Periodically re-pull /me so a remote admin flip propagates
// without a re-login or app restart.
scope.launch {
while (true) {
delay(GATE_REFRESH_MS)
authController.refreshProfile()
}
}
}
private fun setEnabled(value: Boolean) {
if (value == enabled) return
enabled = value
if (value) start() else stop()
}
private fun start() {
registerPowerReceiver()
registerLifecycleObserver()
collectorJob = scope.launch {
record("lifecycle", buildJsonObject { put("event", "reporting_started") })
launch { collectConnectivity() }
launch { collectServerHealth() }
launch { collectUpnpDrops() }
launch { collectPlayerState() }
launch { collectTrackChanges() }
launch { collectRoutes() }
launch { heartbeatLoop() }
}
}
private fun stop() {
collectorJob?.cancel()
collectorJob = null
unregisterPowerReceiver()
unregisterLifecycleObserver()
// Off means off: drop any unsent buffer (honors the user OFF
// intent and avoids leaking data captured before disable).
scope.launch { runCatching { dao.clear() } }
}
private suspend fun collectConnectivity() {
connectivity.online.distinctUntilChanged().collect { online ->
record("connectivity", buildJsonObject {
put("event", "online_changed")
put("online", online)
})
}
}
private suspend fun collectServerHealth() {
// state is a StateFlow — already conflated/distinct, so no
// distinctUntilChanged (it's a deprecation warning = build error).
networkStatus.state.collect { s ->
record("connectivity", buildJsonObject {
put("event", "server_health")
put("state", s.name)
})
}
}
private suspend fun collectUpnpDrops() {
playerController.dropEvents.collect { msg ->
record("upnp_sync", buildJsonObject {
put("event", "drop")
put("message", msg)
})
}
}
private suspend fun collectPlayerState() {
// 'playback', not 'upnp_sync' — player state applies to every output
// route (phone speaker, Bluetooth, UPnP), not just casting.
playerController.uiState
.map { Triple(it.currentSource, it.isUpnpLoading, it.playbackError) }
.distinctUntilChanged()
.collect { (source, upnpLoading, err) ->
record("playback", buildJsonObject {
put("event", "player_state")
put("source", source ?: "")
put("upnp_loading", upnpLoading)
if (err != null) put("error", err)
})
}
}
// Emit a snapshot at each track/queue-index change — captures a
// skip-induced local↔Sonos desync at the INSTANT it happens, which the
// 45s heartbeat misses. Note: uiState is a conflated StateFlow, so a
// very rapid skip burst may coalesce intermediate indices (we still get
// the boundaries + the local-vs-Sonos snapshot).
private suspend fun collectTrackChanges() {
playerController.uiState
.map { it.queueIndex to it.currentTrack?.id }
.distinctUntilChanged()
.collect { (index, _) ->
val ui = playerController.uiState.value
val casting = outputPicker.routesState.value.current.protocol !=
OutputRoute.Protocol.SYSTEM
record("playback", buildJsonObject {
put("event", "track_change")
put("local_index", index)
// Track IDENTITY, not just index — indices wobble across
// re-casts, so the id + Sonos URI make a desync unambiguous.
put("local_track_id", ui.currentTrack?.id ?: "")
put("local_pos_ms", ui.positionMs)
putSonos(this, casting)
put("upnp_loading", ui.isUpnpLoading)
put("server_health", networkStatus.state.value.name)
put("route", outputPicker.routesState.value.current.name)
})
}
}
private suspend fun collectRoutes() {
// 'playback' — route changes happen for all outputs. This only ever
// logs the ACTIVE route (routesState.current), so no "connected" flag.
outputPicker.routesState.map { it.current }.distinctUntilChanged().collect { r ->
record("playback", buildJsonObject {
put("event", "route")
put("id", r.id)
put("name", r.name)
put("kind", r.kind.name)
put("protocol", r.protocol.name)
})
}
}
private suspend fun heartbeatLoop() {
while (true) {
delay(HEARTBEAT_MS)
val ui = playerController.uiState.value
val route = outputPicker.routesState.value.current
val routeActive = route.protocol != OutputRoute.Protocol.SYSTEM
// Idle = nothing worth sampling. Skip to keep the buffer lean.
if (!ui.isPlaying && !routeActive) continue
record("heartbeat", buildJsonObject {
put("server_health", networkStatus.state.value.name)
put("is_playing", ui.isPlaying)
put("source", ui.currentSource ?: "")
put("local_index", ui.queueIndex)
put("local_track_id", ui.currentTrack?.id ?: "")
put("local_pos_ms", ui.positionMs)
put("route", route.name)
put("route_protocol", route.protocol.name)
putSonos(this, routeActive)
addPowerFields(this)
})
}
}
// --- power + lifecycle taps -------------------------------------------
private fun registerPowerReceiver() {
if (powerReceiver != null) return
val receiver = object : BroadcastReceiver() {
override fun onReceive(c: Context?, intent: Intent?) {
val action = intent?.action ?: return
if (!enabled) return
scope.launch { record("power", powerEvent(action)) }
}
}
val filter = IntentFilter().apply {
addAction(Intent.ACTION_SCREEN_ON)
addAction(Intent.ACTION_SCREEN_OFF)
addAction(PowerManager.ACTION_DEVICE_IDLE_MODE_CHANGED)
}
// System-protected broadcasts, but pass NOT_EXPORTED explicitly so
// the API 34+ registerReceiver flag requirement is satisfied.
ContextCompat.registerReceiver(
context, receiver, filter, ContextCompat.RECEIVER_NOT_EXPORTED,
)
powerReceiver = receiver
}
private fun unregisterPowerReceiver() {
powerReceiver?.let { runCatching { context.unregisterReceiver(it) } }
powerReceiver = null
}
private fun registerLifecycleObserver() {
if (lifecycleObserver != null) return
val obs = object : DefaultLifecycleObserver {
override fun onStart(owner: LifecycleOwner) = logLifecycle("app_foreground")
override fun onStop(owner: LifecycleOwner) = logLifecycle("app_background")
}
lifecycleObserver = obs
scope.launch(Dispatchers.Main) {
ProcessLifecycleOwner.get().lifecycle.addObserver(obs)
}
}
private fun unregisterLifecycleObserver() {
val obs = lifecycleObserver ?: return
lifecycleObserver = null
scope.launch(Dispatchers.Main) {
ProcessLifecycleOwner.get().lifecycle.removeObserver(obs)
}
}
private fun logLifecycle(event: String) {
if (!enabled) return
scope.launch { record("lifecycle", buildJsonObject { put("event", event) }) }
}
private fun powerEvent(action: String): JsonObject = buildJsonObject {
put("event", action.substringAfterLast('.'))
addPowerFields(this)
}
// Sonos/UPnP remote-vs-local desync fields. Only meaningful while a
// remote route is active; zeroed otherwise so a stale RemotePlayerState
// from a just-ended cast can't masquerade as live Sonos data (the
// cast→phone handoff artifact). currentTrackUri is the desync ground
// truth — it carries the track the speaker is actually streaming.
private fun putSonos(builder: kotlinx.serialization.json.JsonObjectBuilder, casting: Boolean) {
builder.put("sonos_track", if (casting) remoteState.trackNumber else 0)
builder.put("sonos_pos_ms", if (casting) remoteState.positionMs else 0)
builder.put("sonos_playing", casting && remoteState.isPlaying)
if (casting) builder.put("sonos_uri", remoteState.currentTrackUri)
}
private fun addPowerFields(builder: kotlinx.serialization.json.JsonObjectBuilder) {
val pm = context.getSystemService(Context.POWER_SERVICE) as PowerManager
builder.put("doze", pm.isDeviceIdleMode)
builder.put("screen_on", pm.isInteractive)
builder.put("battery_opt_ignored", pm.isIgnoringBatteryOptimizations(context.packageName))
}
private suspend fun record(kind: String, payload: JsonObject) {
runCatching {
val id = dao.insert(
DiagnosticEventEntity(
kind = kind,
occurredAtMillis = System.currentTimeMillis(),
payloadJson = json.encodeToString(JsonObject.serializer(), payload),
),
)
if (id % TRIM_INTERVAL == 0L) dao.trimToNewest(MAX_BUFFER)
}
}
private companion object {
const val HEARTBEAT_MS = 45_000L
const val GATE_REFRESH_MS = 300_000L // re-pull /me every 5 min
const val MAX_BUFFER = 5_000 // ring-buffer ceiling, oldest dropped
const val TRIM_INTERVAL = 50L // trim every N inserts, not every one
}
}
@@ -0,0 +1,118 @@
package com.fabledsword.minstrel.diagnostics
import android.os.Build
import com.fabledsword.minstrel.BuildConfig
import com.fabledsword.minstrel.api.endpoints.DiagnosticEventWire
import com.fabledsword.minstrel.api.endpoints.DiagnosticsApi
import com.fabledsword.minstrel.api.endpoints.DiagnosticsReportRequest
import com.fabledsword.minstrel.auth.AuthStore
import com.fabledsword.minstrel.cache.db.dao.DiagnosticEventDao
import com.fabledsword.minstrel.cache.db.entities.DiagnosticEventEntity
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.ServerHealth
import com.fabledsword.minstrel.di.ApplicationScope
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.distinctUntilChanged
import kotlinx.coroutines.flow.filter
import kotlinx.coroutines.flow.filterNotNull
import kotlinx.coroutines.flow.map
import kotlinx.coroutines.launch
import kotlinx.coroutines.sync.Mutex
import kotlinx.serialization.json.Json
import retrofit2.Retrofit
import retrofit2.create
import javax.inject.Inject
import javax.inject.Singleton
/**
* Drains the [DiagnosticEventDao] ring buffer to POST /api/diagnostics
* (M9). Triggers mirror [com.fabledsword.minstrel.cache.mutations.MutationReplayer]:
* a periodic tick, a sign-in (sessionCookie non-null), and a server-health
* recovery to [ServerHealth.Healthy] — so a dead-zone's buffered events
* upload the moment the link comes back.
*
* Deliberately NOT the MutationQueue: diagnostics are high-volume,
* best-effort telemetry. A failed upload just retries on the next tick;
* the server no-ops (204) when the account's debug flag is off, which the
* uploader treats as success so the buffer drains either way.
*/
@Singleton
class DiagnosticsUploader @Inject constructor(
private val dao: DiagnosticEventDao,
private val authStore: AuthStore,
private val json: Json,
@ApplicationScope private val scope: CoroutineScope,
networkStatus: NetworkStatusController,
retrofit: Retrofit,
) {
private val api: DiagnosticsApi = retrofit.create()
private val mutex = Mutex()
init {
scope.launch {
while (true) {
delay(UPLOAD_INTERVAL_MS)
drainSafe()
}
}
scope.launch {
authStore.sessionCookie.filterNotNull().distinctUntilChanged().collect { drainSafe() }
}
scope.launch {
networkStatus.state
.map { it == ServerHealth.Healthy }
.distinctUntilChanged()
.filter { it }
.collect { drainSafe() }
}
}
/** Coalesces concurrent triggers via tryLock — a drain in flight wins. */
private suspend fun drainSafe() {
val clientId = authStore.clientId.value
// Signed out / no client id yet → nothing to do. Single guard keeps
// the return count within the detekt gate.
if (authStore.sessionCookie.value.isNullOrEmpty() || clientId == null) return
if (!mutex.tryLock()) return
try {
drain(clientId)
} finally {
mutex.unlock()
}
}
private suspend fun drain(clientId: String) {
var more = true
while (more) {
val batch = runCatching { dao.takeBatch(BATCH_SIZE) }.getOrNull().orEmpty()
val sent = batch.isNotEmpty() && runCatching { upload(clientId, batch) }.isSuccess
if (sent) runCatching { dao.deleteByIds(batch.map { it.id }) }
// Keep going only while a full batch sent cleanly; otherwise stop
// (empty buffer, or a failure to retry on the next trigger).
more = sent && batch.size >= BATCH_SIZE
}
}
private suspend fun upload(clientId: String, batch: List<DiagnosticEventEntity>) {
api.report(
DiagnosticsReportRequest(
clientId = clientId,
appVersion = BuildConfig.VERSION_NAME,
osVersion = "Android ${Build.VERSION.RELEASE} (${Build.MODEL})",
events = batch.map { row ->
DiagnosticEventWire(
kind = row.kind,
occurredAt = row.occurredAtMillis,
payload = json.parseToJsonElement(row.payloadJson),
)
},
),
)
}
private companion object {
const val UPLOAD_INTERVAL_MS = 60_000L
const val BATCH_SIZE = 100
}
}
@@ -41,6 +41,7 @@ import com.fabledsword.minstrel.models.ArtistSuggestionRef
import com.fabledsword.minstrel.models.LidarrRequestKind
import com.fabledsword.minstrel.models.LidarrSearchResultRef
import com.fabledsword.minstrel.nav.Discover
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.LoadingCentered
import com.fabledsword.minstrel.shared.widgets.MinstrelTopAppBar
import com.fabledsword.minstrel.shared.widgets.PullToRefreshScaffold
@@ -109,6 +110,7 @@ private fun DiscoverBody(
snackbar.showSnackbar(snackbarFor(outcome, s.name))
}
},
onRetry = { viewModel.loadSuggestions() },
)
ResultsState.Loading -> LoadingCentered()
is ResultsState.Loaded -> ResultsList(
@@ -121,7 +123,11 @@ private fun DiscoverBody(
}
},
)
is ResultsState.Error -> CenteredMessage("Search failed: ${r.message}")
is ResultsState.Error -> ErrorRetry(
title = "Search failed",
message = r.message,
onRetry = viewModel::runSearch,
)
}
}
}
@@ -177,10 +183,15 @@ private fun SuggestionsPane(
state: SuggestionState,
locallyRequestedMbids: Set<String>,
onRequest: (ArtistSuggestionRef) -> Unit,
onRetry: () -> Unit,
) {
when (state) {
SuggestionState.Loading -> LoadingCentered()
is SuggestionState.Error -> CenteredMessage("Couldn't load suggestions.")
is SuggestionState.Error -> ErrorRetry(
title = "Couldn't load suggestions",
message = state.message,
onRetry = onRetry,
)
is SuggestionState.Loaded -> SuggestionsList(
items = state.items.filter { it.mbid !in locallyRequestedMbids },
onRequest = onRequest,
@@ -3,6 +3,8 @@ package com.fabledsword.minstrel.discover.ui
import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.discover.data.DiscoverRepository
import com.fabledsword.minstrel.discover.data.RequestOutcome
import com.fabledsword.minstrel.models.ArtistSuggestionRef
@@ -46,6 +48,7 @@ sealed interface ResultsState {
@HiltViewModel
class DiscoverViewModel @Inject constructor(
private val repository: DiscoverRepository,
networkStatus: NetworkStatusController,
) : ViewModel() {
private val internal = MutableStateFlow(DiscoverState())
@@ -53,6 +56,15 @@ class DiscoverViewModel @Inject constructor(
init {
loadSuggestions()
// Screen-level auto-recovery (issue #1245): re-run whichever pane
// failed while the server was unreachable once health returns.
viewModelScope.launch {
networkStatus.recoveries().collect {
val s = internal.value
if (s.suggestions is SuggestionState.Error) loadSuggestions()
if (s.results is ResultsState.Error) runSearch()
}
}
}
fun setQuery(value: String) {
@@ -19,6 +19,8 @@ import androidx.lifecycle.ViewModel
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.history.data.HistoryEntry
import com.fabledsword.minstrel.history.data.HistoryRepository
import com.fabledsword.minstrel.models.TrackRef
@@ -26,6 +28,7 @@ import com.fabledsword.minstrel.player.PlayerController
import com.fabledsword.minstrel.shared.UiState
import com.fabledsword.minstrel.shared.widgets.TrackRow
import com.fabledsword.minstrel.shared.widgets.EmptyState
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.LoadingCentered
import com.fabledsword.minstrel.shared.widgets.PullToRefreshScaffold
import com.fabledsword.minstrel.shared.widgets.TrackCoverThumb
@@ -56,6 +59,7 @@ private const val SHARE_STOP_TIMEOUT_MS = 5_000L
class HistoryTabViewModel @Inject constructor(
private val repository: HistoryRepository,
private val player: PlayerController,
networkStatus: NetworkStatusController,
) : ViewModel() {
private val refreshError = MutableStateFlow<String?>(null)
@@ -82,6 +86,11 @@ class HistoryTabViewModel @Inject constructor(
init {
refresh()
// Screen-level auto-recovery (issue #1245): re-pull when server
// health returns, so a load that failed offline heals unprompted.
viewModelScope.launch {
networkStatus.recoveries().collect { refresh() }
}
}
fun refresh(): Job = viewModelScope.launch {
@@ -116,9 +125,10 @@ fun HistoryTab(
title = "No listening history yet",
body = "Play something — your recent plays will show up here.",
)
is UiState.Error -> EmptyState(
is UiState.Error -> ErrorRetry(
title = "Couldn't load history",
body = s.message,
message = s.message,
onRetry = { viewModel.refresh() },
)
is UiState.Success -> HistoryList(
entries = s.data,
@@ -55,6 +55,7 @@ import com.composables.icons.lucide.Lucide
import com.composables.icons.lucide.Music
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.ServerHealth
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.home.data.HomeRepository
import com.fabledsword.minstrel.library.data.LibraryRepository
import com.fabledsword.minstrel.library.widgets.AlbumCard
@@ -77,6 +78,7 @@ import com.fabledsword.minstrel.playlists.widgets.PlaylistPlaceholderCard
import com.fabledsword.minstrel.shared.UiState
import com.fabledsword.minstrel.shared.asCacheFirstStateFlow
import com.fabledsword.minstrel.shared.widgets.EmptyState
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.HorizontalScrollRow
import com.fabledsword.minstrel.shared.widgets.MinstrelTopAppBar
import com.fabledsword.minstrel.shared.widgets.PullToRefreshScaffold
@@ -167,8 +169,22 @@ class HomeViewModel @Inject constructor(
/** Transient snackbar messages from offline-pool taps. */
val transientMessages: Flow<String> = poolMessages.receiveAsFlow()
/**
* Copy for the most recent /home/index refresh failure; null once a
* refresh succeeds. Folded into [uiState] so an empty cache + failed
* refresh renders as Error (with retry) instead of masquerading as
* the "Welcome to Minstrel" empty state.
*/
private val refreshError = MutableStateFlow<String?>(null)
init {
refresh()
// Screen-level auto-recovery (issue #1245): a Home that failed to
// load while the server was unreachable re-pulls itself the moment
// health returns — same idiom as SyncController, one layer up.
viewModelScope.launch {
networkStatus.recoveries().collect { refresh() }
}
// #968: the daily 03:00 rebuild (and manual refresh) emit
// playlist.system_rebuilt; re-pull Home so the system-playlist tiles
// and You-might-like rows reflect the new snapshot without a manual
@@ -289,7 +305,14 @@ class HomeViewModel @Inject constructor(
* actual completion before hiding the indicator.
*/
fun refresh(): Job = viewModelScope.launch {
val home = launch { runCatching { homeRepository.refreshIndex() } }
refreshError.value = null
val home = launch {
// /home/index is the load-bearing pull: its failure drives the
// empty-cache Error state. A failure over a populated cache
// stays silent — cached sections beat a full-screen error.
runCatching { homeRepository.refreshIndex() }
.onFailure { refreshError.value = ErrorCopy.fromThrowable(it) }
}
val lists = launch { runCatching { playlistsRepository.refreshList() } }
val status = launch {
runCatching { homeRepository.getSystemPlaylistsStatus() }
@@ -336,9 +359,17 @@ class HomeViewModel @Inject constructor(
}
private fun combineHomeFlows() =
observeHomeSections().combine(playlistsRepository.observeAll()) { sections, playlists ->
combine(
observeHomeSections(),
playlistsRepository.observeAll(),
refreshError,
) { sections, playlists, err ->
val merged = sections.copy(playlists = playlists)
if (merged.isAllEmpty) UiState.Empty else UiState.Success(merged)
when {
!merged.isAllEmpty -> UiState.Success(merged)
err != null -> UiState.Error(err)
else -> UiState.Empty
}
}
}
@@ -389,9 +420,10 @@ fun HomeScreen(
"settings, then come back here for system playlists " +
"and recommendations.",
)
is UiState.Error -> EmptyState(
is UiState.Error -> ErrorRetry(
title = "Couldn't load home",
body = s.message,
message = s.message,
onRetry = { viewModel.refresh() },
)
is UiState.Success -> HomeSuccessContent(
sections = s.data,
@@ -53,7 +53,7 @@ import com.fabledsword.minstrel.nav.AlbumDetail
import com.fabledsword.minstrel.nav.ArtistDetail
import com.fabledsword.minstrel.shared.formatDuration
import com.fabledsword.minstrel.shared.widgets.TrackRow
import com.fabledsword.minstrel.shared.widgets.EmptyState
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.LikeButton
import com.fabledsword.minstrel.shared.widgets.PullToRefreshScaffold
import com.fabledsword.minstrel.shared.widgets.ServerImage
@@ -116,9 +116,10 @@ private fun AlbumDetailStateContent(
when (val s = state) {
is AlbumDetailUiState.Loading ->
if (s.seed != null) SeededAlbumLoading(s.seed) else SkeletonTrackList()
is AlbumDetailUiState.Error -> EmptyState(
is AlbumDetailUiState.Error -> ErrorRetry(
title = "Couldn't load album",
body = s.message,
message = s.message,
onRetry = { viewModel.refresh() },
)
is AlbumDetailUiState.Success -> {
val albumLiked by viewModel.albumLiked.collectAsStateWithLifecycle()
@@ -5,6 +5,8 @@ import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import androidx.navigation.toRoute
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.library.data.LibraryRepository
import com.fabledsword.minstrel.likes.data.LikesRepository
import com.fabledsword.minstrel.models.AlbumDetailRef
@@ -36,6 +38,7 @@ class AlbumDetailViewModel @Inject constructor(
private val likes: LikesRepository,
private val player: PlayerController,
private val seedCache: DetailSeedCache,
networkStatus: NetworkStatusController,
savedStateHandle: SavedStateHandle,
) : ViewModel() {
@@ -64,6 +67,14 @@ class AlbumDetailViewModel @Inject constructor(
init {
refresh()
// Screen-level auto-recovery (issue #1245): a detail that failed to
// load while the server was unreachable re-fetches when health
// returns, so coming back to the screen shows content, not an error.
viewModelScope.launch {
networkStatus.recoveries().collect {
if (internal.value is AlbumDetailUiState.Error) refresh()
}
}
}
fun refresh(): Job = viewModelScope.launch {
@@ -56,7 +56,7 @@ import com.fabledsword.minstrel.models.albumCoverPath
import com.fabledsword.minstrel.nav.AlbumDetail
import com.fabledsword.minstrel.nav.ArtistDetail
import com.fabledsword.minstrel.shared.formatDuration
import com.fabledsword.minstrel.shared.widgets.EmptyState
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.HorizontalScrollRow
import com.fabledsword.minstrel.shared.widgets.LikeButton
import com.fabledsword.minstrel.shared.widgets.PullToRefreshScaffold
@@ -107,9 +107,10 @@ fun ArtistDetailScreen(
} else {
SkeletonArtistAlbumsGrid()
}
is ArtistDetailUiState.Error -> EmptyState(
is ArtistDetailUiState.Error -> ErrorRetry(
title = "Couldn't load artist",
body = s.message,
message = s.message,
onRetry = { viewModel.refresh() },
)
is ArtistDetailUiState.Success ->
ArtistSuccessBody(s, viewModel, playerViewModel, navController)
@@ -5,6 +5,8 @@ import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import androidx.navigation.toRoute
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.library.data.LibraryRepository
import com.fabledsword.minstrel.likes.data.LikesRepository
import com.fabledsword.minstrel.models.ArtistDetailRef
@@ -45,6 +47,7 @@ class ArtistDetailViewModel @Inject constructor(
private val likes: LikesRepository,
private val player: PlayerController,
private val seedCache: DetailSeedCache,
networkStatus: NetworkStatusController,
savedStateHandle: SavedStateHandle,
) : ViewModel() {
@@ -69,6 +72,19 @@ class ArtistDetailViewModel @Inject constructor(
init {
refresh()
// Screen-level auto-recovery (issue #1245). From Error, redo the
// whole load; from Success, re-pull just the secondary sections —
// their fetch failures are swallowed to emptyList, so a dead zone
// during the first load leaves them permanently absent otherwise.
viewModelScope.launch {
networkStatus.recoveries().collect {
when (internal.value) {
is ArtistDetailUiState.Error -> refresh()
is ArtistDetailUiState.Success -> loadSecondarySections()
is ArtistDetailUiState.Loading -> Unit
}
}
}
}
fun toggleLikeArtist() {
@@ -24,11 +24,17 @@ class LibraryViewModel @Inject constructor(
combine(
repository.observeArtists(),
repository.observeAlbums(),
) { artists, albums ->
if (artists.isEmpty() && albums.isEmpty()) {
UiState.Empty
} else {
UiState.Success(LibraryData(artists, albums))
syncController.lastSyncError,
) { artists, albums, syncError ->
when {
artists.isNotEmpty() || albums.isNotEmpty() ->
UiState.Success(LibraryData(artists, albums))
// Empty cache + failed sync is a load failure, not an empty
// library — surface it so the user gets a Retry instead of
// the welcome copy. Heals automatically: SyncController
// re-syncs on health recovery, clearing lastSyncError.
syncError != null -> UiState.Error(syncError)
else -> UiState.Empty
}
}
.asCacheFirstStateFlow(viewModelScope)
@@ -25,6 +25,9 @@ import androidx.lifecycle.ViewModel
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import androidx.lifecycle.viewModelScope
import androidx.navigation.NavHostController
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.likes.data.LikesRepository
import com.fabledsword.minstrel.library.widgets.AlbumCard
import com.fabledsword.minstrel.library.widgets.ArtistCard
@@ -38,6 +41,7 @@ import com.fabledsword.minstrel.shared.UiState
import com.fabledsword.minstrel.shared.asCacheFirstStateFlow
import com.fabledsword.minstrel.shared.widgets.TrackRow
import com.fabledsword.minstrel.shared.widgets.EmptyState
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.HorizontalScrollRow
import com.fabledsword.minstrel.shared.widgets.LikeButton
import com.fabledsword.minstrel.shared.widgets.LoadingCentered
@@ -46,6 +50,7 @@ import com.fabledsword.minstrel.shared.widgets.TrackCoverThumb
import com.fabledsword.minstrel.shared.widgets.trackactions.TrackActionsButton
import dagger.hilt.android.lifecycle.HiltViewModel
import kotlinx.coroutines.Job
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.combine
import kotlinx.coroutines.launch
@@ -68,14 +73,25 @@ data class LikedSections(
class LikedTabViewModel @Inject constructor(
private val repository: LikesRepository,
private val player: PlayerController,
networkStatus: NetworkStatusController,
) : ViewModel() {
/** Copy for the most recent refresh failure; null once one succeeds. */
private val refreshError = MutableStateFlow<String?>(null)
init {
refresh()
// Screen-level auto-recovery (issue #1245): re-pull when server
// health returns, so a load that failed offline heals unprompted.
viewModelScope.launch {
networkStatus.recoveries().collect { refresh() }
}
}
fun refresh(): Job = viewModelScope.launch {
refreshError.value = null
runCatching { repository.refreshIds() }
.onFailure { refreshError.value = ErrorCopy.fromThrowable(it) }
}
/**
@@ -92,9 +108,16 @@ class LikedTabViewModel @Inject constructor(
repository.observeLikedArtists(),
repository.observeLikedAlbums(),
repository.observeLikedTracks(),
) { artists, albums, tracks ->
refreshError,
) { artists, albums, tracks, err ->
val sections = LikedSections(artists, albums, tracks)
if (sections.isAllEmpty) UiState.Empty else UiState.Success(sections)
when {
!sections.isAllEmpty -> UiState.Success(sections)
// Empty cache + failed refresh is a load failure, not "no
// likes yet" — surface Error so the user gets a Retry.
err != null -> UiState.Error(err)
else -> UiState.Empty
}
}.asCacheFirstStateFlow(viewModelScope)
/**
@@ -130,9 +153,10 @@ fun LikedTab(
body = "Tap the heart on an artist, album, or track to start " +
"building your liked collection.",
)
is UiState.Error -> EmptyState(
is UiState.Error -> ErrorRetry(
title = "Couldn't load likes",
body = s.message,
message = s.message,
onRetry = { viewModel.refresh() },
)
is UiState.Success -> LikedContent(
sections = s.data,
@@ -16,4 +16,11 @@ data class UserRef(
val id: String,
val username: String,
val isAdmin: Boolean = false,
/**
* Account-level diagnostics opt-in (M9). NOT present in the login
* response (that's the narrow UserView) — populated by the /api/me
* refresh in AuthController so a remote admin flip reaches the
* device. Default false until the first refresh.
*/
val debugModeEnabled: Boolean = false,
)
@@ -18,4 +18,5 @@ data class MyProfileWire(
@SerialName("display_name") val displayName: String? = null,
val email: String? = null,
@SerialName("is_admin") val isAdmin: Boolean = false,
@SerialName("debug_mode_enabled") val debugModeEnabled: Boolean = false,
)
@@ -133,7 +133,7 @@ private fun RouteRow(
maxLines = 2,
overflow = TextOverflow.Ellipsis,
)
val subtitle = route.description ?: defaultSubtitle(route)
val subtitle = route.description ?: defaultSubtitle(route, isSelected)
Text(
text = subtitle,
style = MaterialTheme.typography.bodySmall,
@@ -198,10 +198,13 @@ private fun MulticastHintRow() {
}
}
private fun defaultSubtitle(route: OutputRoute): String = when (route.kind) {
// isSelected (route == the active/selected route) drives the Bluetooth
// "Connected" subtitle. RouteInfo.connectionState can't — it stays
// DISCONNECTED for local SYSTEM routes even when they're in use.
private fun defaultSubtitle(route: OutputRoute, isSelected: Boolean): String = when (route.kind) {
OutputRoute.Kind.BuiltIn -> "Phone speaker"
OutputRoute.Kind.Wired -> "Wired"
OutputRoute.Kind.Bluetooth -> if (route.isConnected) "Connected" else "Available"
OutputRoute.Kind.Bluetooth -> if (isSelected) "Connected" else "Available"
OutputRoute.Kind.Cast -> "Cast"
OutputRoute.Kind.Other -> "Available"
}
@@ -21,7 +21,6 @@ data class OutputRoute(
val description: String?,
val kind: Kind,
val protocol: Protocol,
val isConnected: Boolean,
) {
enum class Kind { BuiltIn, Wired, Bluetooth, Cast, Other }
@@ -43,9 +42,11 @@ data class OutputRoute(
/**
* Lift a MediaRouter [route] into the domain model. Kind is
* inferred from [MediaRouter.RouteInfo.getDeviceType]; unknown
* device types fall through to [Kind.Other]. The
* `connectionState` proxy is good enough for the chip's
* "Connected"/"Available" subtitle.
* device types fall through to [Kind.Other]. "Active" is NOT an
* intrinsic of the route — the caller derives it by comparing to
* the selected route (RouteSnapshot.current), because
* RouteInfo.connectionState only reflects remote-route handshakes
* and stays DISCONNECTED for local SYSTEM routes even when in use.
*/
fun fromRouteInfo(route: MediaRouter.RouteInfo): OutputRoute {
val kind = when (route.deviceType) {
@@ -58,15 +59,12 @@ data class OutputRoute(
MediaRouter.RouteInfo.DEVICE_TYPE_SPEAKER -> Kind.Other
else -> Kind.Other
}
val connected =
route.connectionState == MediaRouter.RouteInfo.CONNECTION_STATE_CONNECTED
return OutputRoute(
id = route.id,
name = route.name,
description = route.description,
kind = kind,
protocol = Protocol.SYSTEM,
isConnected = connected,
)
}
@@ -76,10 +74,10 @@ data class OutputRoute(
* network speakers into the same `OutputPickerController`
* routes stream the system routes come through.
*
* `isConnected = false` because UPnP devices have no
* MediaRouter connection-state concept — they're always
* "available" on the LAN, and the picker's selected-route
* rendering handles the "currently playing" indicator.
* UPnP devices have no MediaRouter connection-state concept —
* they're always "available" on the LAN, and the picker's
* selected-route rendering handles the "currently playing"
* indicator.
*
* Subtitle is `manufacturer modelName` joined by a single
* space, falling back to "Network speaker" when both fields
@@ -96,7 +94,6 @@ data class OutputRoute(
description = description,
kind = Kind.Other,
protocol = Protocol.UPNP,
isConnected = false,
)
}
}
@@ -38,9 +38,7 @@ import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
@@ -60,6 +58,8 @@ import com.composables.icons.lucide.Play
import com.composables.icons.lucide.RefreshCw
import com.composables.icons.lucide.Shuffle
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.events.EventsStream
import com.fabledsword.minstrel.events.LiveEvent
import com.fabledsword.minstrel.models.PlaylistRef
@@ -76,6 +76,7 @@ import com.fabledsword.minstrel.playlists.data.PlaylistsRepository
import com.fabledsword.minstrel.playlists.data.toPlayableTrackRefs
import com.fabledsword.minstrel.shared.formatDuration
import com.fabledsword.minstrel.shared.widgets.TrackRow
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.LikeButton
import com.fabledsword.minstrel.shared.widgets.PullToRefreshScaffold
import com.fabledsword.minstrel.shared.widgets.SkeletonTrackRow
@@ -117,6 +118,7 @@ class PlaylistDetailViewModel @Inject constructor(
private val player: PlayerController,
private val eventsStream: EventsStream,
private val seedCache: DetailSeedCache,
networkStatus: NetworkStatusController,
savedStateHandle: SavedStateHandle,
) : ViewModel() {
@@ -167,6 +169,14 @@ class PlaylistDetailViewModel @Inject constructor(
.filter { it.kind.startsWith("playlist.") }
.collect { handlePlaylistEvent(it) }
}
// Screen-level auto-recovery (issue #1245): a detail that failed to
// load while the server was unreachable re-fetches when health
// returns, instead of waiting for the Retry tap.
viewModelScope.launch {
networkStatus.recoveries().collect {
if (internal.value is PlaylistDetailUiState.Error) refresh()
}
}
}
/**
@@ -369,7 +379,11 @@ private fun PlaylistDetailContent(
Crossfade(targetState = state::class, label = "playlist-detail") { _ -> when (val s = state) {
is PlaylistDetailUiState.Loading ->
s.seed?.let { SeededPlaylistLoading(it) } ?: SkeletonPlaylistTrackList()
is PlaylistDetailUiState.Error -> ErrorBlock(s.message, viewModel::refresh)
is PlaylistDetailUiState.Error -> ErrorRetry(
title = "Couldn't load playlist",
message = s.message,
onRetry = { viewModel.refresh() },
)
is PlaylistDetailUiState.Success -> {
val likedTrackIds by viewModel.likedTrackIds.collectAsState()
val playerState by playerViewModel.uiState.collectAsState()
@@ -653,33 +667,6 @@ private fun SeededPlaylistLoading(seed: PlaylistRef) {
}
}
@Composable
private fun ErrorBlock(message: String, onRetry: () -> Unit) {
var retried by remember { mutableStateOf(false) }
Column(
modifier = Modifier
.fillMaxSize()
.padding(24.dp),
verticalArrangement = Arrangement.Center,
horizontalAlignment = Alignment.CenterHorizontally,
) {
Text(
text = message,
color = MaterialTheme.colorScheme.error,
style = MaterialTheme.typography.bodyLarge,
)
Spacer(Modifier.height(12.dp))
OutlinedButton(
onClick = {
if (!retried) {
retried = true
onRetry()
}
},
) { Text("Retry") }
}
}
// ─── Helpers ─────────────────────────────────────────────────────────
private const val UNAVAILABLE_ALPHA = 0.4f
@@ -27,8 +27,10 @@ import androidx.lifecycle.ViewModel
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import androidx.lifecycle.viewModelScope
import androidx.navigation.NavHostController
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.ServerHealth
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.events.EventsStream
import com.fabledsword.minstrel.models.PlaylistRef
import com.fabledsword.minstrel.nav.PlaylistDetail
@@ -40,6 +42,7 @@ import com.fabledsword.minstrel.shared.UiState
import com.fabledsword.minstrel.shared.asCacheFirstStateFlow
import com.fabledsword.minstrel.playlists.widgets.PlaylistCard
import com.fabledsword.minstrel.shared.widgets.EmptyState
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.LoadingCentered
import com.fabledsword.minstrel.shared.widgets.MinstrelTopAppBar
import com.fabledsword.minstrel.shared.widgets.PullToRefreshScaffold
@@ -47,8 +50,10 @@ import dagger.hilt.android.lifecycle.HiltViewModel
import kotlinx.coroutines.Job
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.SharingStarted
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.combine
import kotlinx.coroutines.flow.filter
import kotlinx.coroutines.flow.map
import kotlinx.coroutines.flow.receiveAsFlow
@@ -84,6 +89,9 @@ class PlaylistsListViewModel @Inject constructor(
initialValue = false,
)
/** Copy for the most recent list-refresh failure; null once one succeeds. */
private val refreshError = MutableStateFlow<String?>(null)
init {
refresh()
// Live updates: a playlist created/updated/deleted from another
@@ -93,11 +101,18 @@ class PlaylistsListViewModel @Inject constructor(
.filter { it.kind.startsWith("playlist.") }
.collect { refresh() }
}
// Screen-level auto-recovery (issue #1245): re-pull when server
// health returns, so a load that failed offline heals unprompted.
viewModelScope.launch {
networkStatus.recoveries().collect { refresh() }
}
}
/** Re-pull the playlists list. Returns the Job for pull-to-refresh awaits. */
fun refresh(): Job = viewModelScope.launch {
refreshError.value = null
runCatching { repository.refreshList() }
.onFailure { refreshError.value = ErrorCopy.fromThrowable(it) }
}
/** Tile play button: shuffle the playlist's tracks and start at index 0. */
@@ -110,14 +125,15 @@ class PlaylistsListViewModel @Inject constructor(
}
val uiState: StateFlow<UiState<List<PlaylistRef>>> =
repository.observeAll()
.map { list ->
if (list.isEmpty()) {
UiState.Empty
} else {
UiState.Success(list)
}
combine(repository.observeAll(), refreshError) { list, err ->
when {
list.isNotEmpty() -> UiState.Success(list)
// Empty cache + failed refresh is a load failure, not "no
// playlists yet" — surface Error so the user gets a Retry.
err != null -> UiState.Error(err)
else -> UiState.Empty
}
}
.asCacheFirstStateFlow(viewModelScope)
}
@@ -157,9 +173,10 @@ fun PlaylistsListScreen(
"appear once your library has enough plays. Create your " +
"own playlist from any album or track in the meantime.",
)
is UiState.Error -> EmptyState(
is UiState.Error -> ErrorRetry(
title = "Couldn't load playlists",
body = s.message,
message = s.message,
onRetry = { viewModel.refresh() },
)
is UiState.Success -> PlaylistsGrid(
playlists = s.data,
@@ -27,6 +27,7 @@ import com.composables.icons.lucide.Lucide
import com.fabledsword.minstrel.models.QuarantineRef
import com.fabledsword.minstrel.shared.UiState
import com.fabledsword.minstrel.shared.widgets.EmptyState
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.LoadingCentered
import com.fabledsword.minstrel.shared.widgets.PullToRefreshScaffold
import com.fabledsword.minstrel.shared.widgets.TrackCoverThumb
@@ -43,9 +44,10 @@ fun HiddenTab(viewModel: HiddenTabViewModel = hiltViewModel()) {
"and it'll show up here. The track stays out of system " +
"playlists until you unhide it.",
)
is UiState.Error -> EmptyState(
is UiState.Error -> ErrorRetry(
title = "Couldn't load hidden tracks",
body = s.message,
message = s.message,
onRetry = { viewModel.refresh() },
)
is UiState.Success -> HiddenList(
rows = s.data,
@@ -3,6 +3,8 @@ package com.fabledsword.minstrel.quarantine.ui
import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.events.EventsStream
import com.fabledsword.minstrel.models.QuarantineRef
import com.fabledsword.minstrel.shared.UiState
@@ -24,6 +26,7 @@ private const val SHARE_STOP_TIMEOUT_MS = 5_000L
class HiddenTabViewModel @Inject constructor(
private val repository: QuarantineRepository,
private val eventsStream: EventsStream,
networkStatus: NetworkStatusController,
) : ViewModel() {
private val refreshError = MutableStateFlow<String?>(null)
@@ -61,6 +64,11 @@ class HiddenTabViewModel @Inject constructor(
.filter { it.kind.startsWith("quarantine.") }
.collect { refresh() }
}
// Screen-level auto-recovery (issue #1245): re-pull when server
// health returns, so a load that failed offline heals unprompted.
viewModelScope.launch {
networkStatus.recoveries().collect { refresh() }
}
}
fun refresh(): Job = viewModelScope.launch {
@@ -46,6 +46,7 @@ import com.fabledsword.minstrel.shared.UiState
import com.fabledsword.minstrel.nav.ArtistDetail
import com.fabledsword.minstrel.nav.Requests
import com.fabledsword.minstrel.shared.widgets.EmptyState
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.LoadingCentered
import com.fabledsword.minstrel.shared.widgets.MinstrelTopAppBar
import com.fabledsword.minstrel.shared.widgets.PullToRefreshScaffold
@@ -78,9 +79,10 @@ fun RequestsScreen(
body = "Use Discover to ask Lidarr for new music; your " +
"requests show up here.",
)
is UiState.Error -> EmptyState(
is UiState.Error -> ErrorRetry(
title = "Couldn't load requests",
body = s.message,
message = s.message,
onRetry = { viewModel.refresh() },
)
is UiState.Success -> RequestList(
rows = s.data,
@@ -3,6 +3,8 @@ package com.fabledsword.minstrel.requests.ui
import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.events.EventsStream
import com.fabledsword.minstrel.models.RequestRef
import com.fabledsword.minstrel.models.RequestStatus
@@ -30,6 +32,7 @@ private const val POLL_INTERVAL_MS = 12_000L
class RequestsViewModel @Inject constructor(
private val repository: RequestsRepository,
private val eventsStream: EventsStream,
networkStatus: NetworkStatusController,
) : ViewModel() {
private val internal = MutableStateFlow<UiState<List<RequestRef>>>(UiState.Loading)
@@ -45,6 +48,14 @@ class RequestsViewModel @Inject constructor(
.collect { silentReload() }
}
viewModelScope.launch { pollWhileInFlight() }
// Screen-level auto-recovery (issue #1245): a load that failed while
// the server was unreachable heals the moment health returns, without
// waiting for the next poll tick.
viewModelScope.launch {
networkStatus.recoveries().collect {
if (internal.value is UiState.Error) silentReload()
}
}
}
fun refresh(): Job = viewModelScope.launch {
@@ -71,7 +82,11 @@ class RequestsViewModel @Inject constructor(
*/
private suspend fun pollWhileInFlight() {
while (true) {
if (hasInFlightRequest()) {
// Also retry from a failed load (issue #1245): from UiState.Error
// the in-flight predicate can never become true, so without this
// the poll never self-heals the screen — e.g. after a transient
// 500 that health monitoring (transport-level) never saw.
if (hasInFlightRequest() || internal.value is UiState.Error) {
silentReload()
}
delay(POLL_INTERVAL_MS)
@@ -56,6 +56,7 @@ import com.fabledsword.minstrel.nav.AlbumDetail
import com.fabledsword.minstrel.nav.ArtistDetail
import com.fabledsword.minstrel.nav.Search as SearchRoute
import com.fabledsword.minstrel.shared.widgets.TrackRow
import com.fabledsword.minstrel.shared.widgets.ErrorRetry
import com.fabledsword.minstrel.shared.widgets.LoadingCentered
import com.fabledsword.minstrel.shared.widgets.MainAppBarActions
import com.fabledsword.minstrel.shared.widgets.TrackCoverThumb
@@ -113,6 +114,7 @@ fun SearchScreen(
onTrackPlay = viewModel::playTrack,
onNavigateToAlbum = { id -> navController.navigate(AlbumDetail(id)) },
onNavigateToArtist = { id -> navController.navigate(ArtistDetail(id)) },
onRetry = viewModel::retry,
)
}
}
@@ -151,11 +153,16 @@ private fun ResultsPane(
onTrackPlay: (TrackRef) -> Unit,
onNavigateToAlbum: (String) -> Unit,
onNavigateToArtist: (String) -> Unit,
onRetry: () -> Unit,
) {
when (state) {
SearchResultsState.Idle -> CenteredHint("Type to search your library.")
SearchResultsState.Loading -> LoadingCentered()
is SearchResultsState.Error -> CenteredHint("Search failed: ${state.message}")
is SearchResultsState.Error -> ErrorRetry(
title = "Search failed",
message = state.message,
onRetry = onRetry,
)
is SearchResultsState.Loaded -> {
if (state.response.isEmpty) {
CenteredHint(
@@ -3,6 +3,8 @@ package com.fabledsword.minstrel.search.ui
import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.connectivity.recoveries
import com.fabledsword.minstrel.models.SearchResponseRef
import com.fabledsword.minstrel.models.TrackRef
import com.fabledsword.minstrel.player.PlayerController
@@ -43,6 +45,7 @@ data class SearchState(
class SearchViewModel @Inject constructor(
private val repository: SearchRepository,
private val player: PlayerController,
networkStatus: NetworkStatusController,
) : ViewModel() {
private val queryFlow = MutableStateFlow("")
@@ -69,6 +72,14 @@ class SearchViewModel @Inject constructor(
runSearch(q)
}
}
// Screen-level auto-recovery (issue #1245): a search that failed
// while the server was unreachable re-runs when health returns —
// the queryFlow pipeline can't, since the text hasn't changed.
viewModelScope.launch {
networkStatus.recoveries().collect {
if (internal.value.results is SearchResultsState.Error) retry()
}
}
}
fun setQuery(value: String) {
@@ -80,6 +91,18 @@ class SearchViewModel @Inject constructor(
setQuery("")
}
/**
* Re-run the current query after a failure. Bypasses the queryFlow
* pipeline, whose distinctUntilChanged drops a same-text resubmit —
* without this, a transient error pins "Search failed" until the
* user edits the query.
*/
fun retry() {
val q = internal.value.query.trim()
if (q.isEmpty()) return
viewModelScope.launch { runSearch(q) }
}
/**
* Tapping a search result builds a queue from the full visible
* track-results list starting at the tapped entry, matching Flutter.
@@ -28,6 +28,7 @@ import androidx.compose.material3.Scaffold
import androidx.compose.material3.SegmentedButton
import androidx.compose.material3.SegmentedButtonDefaults
import androidx.compose.material3.SingleChoiceSegmentedButtonRow
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
@@ -94,6 +95,7 @@ fun SettingsScreen(
onPickTheme = themeVm::setThemeMode,
onNavToRequests = { navController.navigate(Requests) },
onNavToAdmin = { navController.navigate(Admin) },
onToggleDiagnostics = viewModel::setDiagnosticsOptOut,
onSignOutClick = { showSignOutConfirm = true },
)
}
@@ -116,6 +118,7 @@ private fun SettingsList(
onPickTheme: (ThemeMode) -> Unit,
onNavToRequests: () -> Unit,
onNavToAdmin: () -> Unit,
onToggleDiagnostics: (Boolean) -> Unit,
onSignOutClick: () -> Unit,
) {
Column(
@@ -148,6 +151,12 @@ private fun SettingsList(
ProfileCard()
PasswordCard()
ListenBrainzCard()
if (state.diagnosticsEnabledByAdmin) {
DiagnosticsCard(
optOut = state.diagnosticsOptOut,
onToggle = onToggleDiagnostics,
)
}
AppearanceCard(themeMode = themeMode, onPick = onPickTheme)
StorageCard()
AboutCard()
@@ -262,6 +271,46 @@ private fun AccountCard(username: String, isAdmin: Boolean, serverUrl: String) {
}
}
// Shown only while an admin has enabled debug mode on this account. Lets
// the user turn reporting OFF on this device (battery/privacy) without
// touching the account flag. Checked = reporting on = NOT opted out.
@Composable
private fun DiagnosticsCard(optOut: Boolean, onToggle: (Boolean) -> Unit) {
ElevatedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(
text = "Diagnostics",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
)
Row(verticalAlignment = Alignment.CenterVertically) {
Column(modifier = Modifier.weight(1f)) {
Text(
text = "Send diagnostic reports",
style = MaterialTheme.typography.bodyLarge,
color = MaterialTheme.colorScheme.onSurface,
)
Text(
text = "Your admin enabled debug mode to investigate an " +
"issue. This device streams connectivity and playback " +
"diagnostics. Turn off any time.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
Spacer(Modifier.size(12.dp))
Switch(
checked = !optOut,
onCheckedChange = { checked -> onToggle(!checked) },
)
}
}
}
}
@OptIn(ExperimentalMaterial3Api::class)
@Composable
private fun AppearanceCard(themeMode: ThemeMode, onPick: (ThemeMode) -> Unit) {
@@ -19,12 +19,15 @@ data class SettingsState(
val isAdmin: Boolean = false,
val isSigningOut: Boolean = false,
val signedOut: Boolean = false,
// M9: account debug flag (admin-set) + this device's local opt-out.
val diagnosticsEnabledByAdmin: Boolean = false,
val diagnosticsOptOut: Boolean = false,
)
@HiltViewModel
class SettingsViewModel @Inject constructor(
private val authController: AuthController,
authStore: AuthStore,
private val authStore: AuthStore,
) : ViewModel() {
private val transient = MutableStateFlow(TransientState())
@@ -41,19 +44,27 @@ class SettingsViewModel @Inject constructor(
combine(
authStore.baseUrl,
authController.currentUser,
authStore.diagnosticsOptOut,
transient,
) { url, user, t ->
) { url, user, optOut, t ->
SettingsState(
serverUrl = url,
username = user?.username.orEmpty(),
isAdmin = user?.isAdmin == true,
isSigningOut = t.isSigningOut,
signedOut = t.signedOut,
diagnosticsEnabledByAdmin = user?.debugModeEnabled == true,
diagnosticsOptOut = optOut,
)
}.collect { internal.value = it }
}
}
/** Per-device opt-out of diagnostics while the account flag is on. */
fun setDiagnosticsOptOut(optOut: Boolean) {
authStore.setDiagnosticsOptOut(optOut)
}
fun signOut() {
if (transient.value.isSigningOut) return
viewModelScope.launch {
@@ -5,6 +5,7 @@ import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.material3.Button
import androidx.compose.material3.ButtonDefaults
import androidx.compose.material3.Icon
@@ -23,44 +24,63 @@ import com.fabledsword.minstrel.theme.LocalActionColors
* Shared error-with-retry widget. Surfaces a brief message and a
* primary action button. Per the design system, the button uses
* `LocalActionColors.primary` (Moss) — NEVER the accent.
*
* Renders inside a single-item LazyColumn (same rationale as
* [EmptyState]) so the widget participates in nested-scroll dispatch
* under PullToRefreshBox — both the Retry button and a pull gesture
* can recover from the error state.
*/
@Composable
fun ErrorRetry(
message: String,
onRetry: () -> Unit,
modifier: Modifier = Modifier,
title: String? = null,
retryLabel: String = "Retry",
) {
val actionColors = LocalActionColors.current
Column(
modifier = modifier
.fillMaxSize()
.padding(32.dp),
verticalArrangement = Arrangement.Center,
horizontalAlignment = Alignment.CenterHorizontally,
) {
Icon(
imageVector = Lucide.TriangleAlert,
contentDescription = null,
modifier = Modifier.size(48.dp),
tint = MaterialTheme.colorScheme.error,
)
Text(
text = message,
modifier = Modifier.padding(top = 16.dp),
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurface,
textAlign = TextAlign.Center,
)
Button(
onClick = onRetry,
modifier = Modifier.padding(top = 16.dp),
colors = ButtonDefaults.buttonColors(
containerColor = actionColors.primary,
contentColor = actionColors.onAction,
),
) {
Text(retryLabel)
LazyColumn(modifier = modifier.fillMaxSize()) {
item {
Column(
modifier = Modifier
.fillParentMaxSize()
.padding(32.dp),
verticalArrangement = Arrangement.Center,
horizontalAlignment = Alignment.CenterHorizontally,
) {
Icon(
imageVector = Lucide.TriangleAlert,
contentDescription = null,
modifier = Modifier.size(48.dp),
tint = MaterialTheme.colorScheme.error,
)
if (title != null) {
Text(
text = title,
modifier = Modifier.padding(top = 16.dp),
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
textAlign = TextAlign.Center,
)
}
Text(
text = message,
modifier = Modifier.padding(top = if (title != null) 8.dp else 16.dp),
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurface,
textAlign = TextAlign.Center,
)
Button(
onClick = onRetry,
modifier = Modifier.padding(top = 16.dp),
colors = ButtonDefaults.buttonColors(
containerColor = actionColors.primary,
contentColor = actionColors.onAction,
),
) {
Text(retryLabel)
}
}
}
}
}
@@ -11,6 +11,7 @@ import com.fabledsword.minstrel.testutil.MainDispatcherExtension
import io.mockk.every
import io.mockk.mockk
import kotlinx.coroutines.flow.MutableSharedFlow
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.flow
import kotlinx.coroutines.flow.flowOf
import kotlinx.coroutines.test.runTest
@@ -21,6 +22,13 @@ import kotlin.test.assertEquals
@ExtendWith(MainDispatcherExtension::class)
class LibraryViewModelTest {
// lastSyncError needs a real StateFlow: the VM combines it into uiState,
// and a relaxed-mock flow never emits, so the combine would never fire.
private fun syncControllerMock(): SyncController =
mockk<SyncController>(relaxed = true) {
every { lastSyncError } returns MutableStateFlow(null)
}
@Test
fun `initial state is Loading before any DAO emission`() = runTest {
val repo = mockk<LibraryRepository>()
@@ -29,7 +37,7 @@ class LibraryViewModelTest {
val vm = LibraryViewModel(
repo,
mockk<SyncController>(relaxed = true),
syncControllerMock(),
mockk<PlayerController>(relaxed = true),
)
@@ -51,7 +59,7 @@ class LibraryViewModelTest {
val vm = LibraryViewModel(
repo,
mockk<SyncController>(relaxed = true),
syncControllerMock(),
mockk<PlayerController>(relaxed = true),
)
@@ -71,7 +79,7 @@ class LibraryViewModelTest {
val vm = LibraryViewModel(
repo,
mockk<SyncController>(relaxed = true),
syncControllerMock(),
mockk<PlayerController>(relaxed = true),
)
@@ -94,7 +102,7 @@ class LibraryViewModelTest {
val vm = LibraryViewModel(
repo,
mockk<SyncController>(relaxed = true),
syncControllerMock(),
mockk<PlayerController>(relaxed = true),
)
+12
View File
@@ -25,6 +25,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/lidarrrequests"
"git.fabledsword.com/bvandeusen/minstrel/internal/logging"
"git.fabledsword.com/bvandeusen/minstrel/internal/playlists"
"git.fabledsword.com/bvandeusen/minstrel/internal/recsettings"
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble"
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble/listenbrainz"
"git.fabledsword.com/bvandeusen/minstrel/internal/server"
@@ -146,6 +147,16 @@ func run() error {
coverEnricher := coverart.NewEnricher(pool, logger.With("component", "coverart"), coverSettings)
coverEnricher.DataDir = cfg.Storage.DataDir
// Recommendation tuning lab (#1250): seeds shipped defaults on first
// boot and pushes the daily_mix weights + taste config into package
// playlists — must precede the scheduler so the first builds score
// with the operator's tuned values, not the pre-push literals.
recSettings, err := recsettings.New(ctx, pool, logger.With("component", "recsettings"))
if err != nil {
logger.Error("recommendation settings service init failed", "err", err)
os.Exit(1)
}
// One unified scan chain: library walk → MBID backfill → cover enrich.
// Boot-time scan and manual-trigger scans share this path; results land
// in scan_runs for the admin overview.
@@ -294,6 +305,7 @@ func run() error {
}, cfg.Events, cfg.Recommendation, cfg.Storage.DataDir, cfg.Branding, coverEnricher, coverSettings, scanner, scanCfg)
srv.Bus = bus
srv.PlaylistScheduler = playlistScheduler
srv.RecSettings = recSettings
srv.StreamSecret = cfg.StreamSecret
httpServer := &http.Server{
Addr: cfg.Server.Address,
+202
View File
@@ -0,0 +1,202 @@
// Admin recommendation-trends endpoint (#1251): weekly per-surface
// outcome series with tuning-audit markers — the verify half of the
// tune→verify loop the tuning lab (#1250) opens. Aggregated across
// all users because the knobs are global; rows carry rates only.
package api
import (
"encoding/json"
"net/http"
"sort"
"strconv"
"time"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
const (
trendsDefaultWeeks = 12
trendsMaxWeeks = 52
// trendsAuditFetchCap bounds the audit fetch; markers older than
// the window are dropped in Go. Far above any real knob-turn count
// inside a year.
trendsAuditFetchCap = 500
)
// trendPoint is one week of one surface family's outcomes.
type trendPoint struct {
WeekStart string `json:"week_start"` // ISO date (Monday)
Plays int64 `json:"plays"`
Skips int64 `json:"skips"`
SkipRate float64 `json:"skip_rate"`
AvgCompletion float64 `json:"avg_completion"`
// TasteHitRate is the share of plays whose track's artist carries a
// positive weight in the player's current taste profile — a drifted
// but retroactive read on whether the surface feeds taste-fitting
// tracks.
TasteHitRate float64 `json:"taste_hit_rate"`
// completionN carries the completion sample count through same-week
// merges so avg_completion stays sample-weighted; not serialized.
completionN int64
}
// trendSeries is one surface family's weekly series.
type trendSeries struct {
Key string `json:"key"`
Label string `json:"label"`
Intent string `json:"intent"` // go_to | discovery | direct; "" for the manual baseline
Plays int64 `json:"plays"` // window total, for sorting/volume-gating
Points []trendPoint `json:"points"`
}
// trendMarker is one tuning-audit event annotated on the timeline.
type trendMarker struct {
ChangedAt string `json:"changed_at"`
Scope string `json:"scope"`
Action string `json:"action"`
Changes json.RawMessage `json:"changes"`
}
type trendsResp struct {
Weeks int `json:"weeks"`
Series []trendSeries `json:"series"`
Markers []trendMarker `json:"markers"`
}
// handleGetRecommendationTrends implements
// GET /api/admin/recommendation-trends?weeks=N (default 12, cap 52).
func (h *handlers) handleGetRecommendationTrends(w http.ResponseWriter, r *http.Request) {
weeks := trendsDefaultWeeks
if v := r.URL.Query().Get("weeks"); v != "" {
n, err := strconv.Atoi(v)
if err != nil || n < 1 {
writeErr(w, apierror.BadRequest("bad_request", "invalid weeks"))
return
}
if n > trendsMaxWeeks {
n = trendsMaxWeeks
}
weeks = n
}
q := dbq.New(h.pool)
rows, err := q.RecommendationWeeklyTrends(r.Context(), int32(weeks))
if err != nil {
h.logger.Error("api: recommendation trends", "err", err)
writeErr(w, apierror.InternalMsg("lookup failed", err))
return
}
audits, err := q.ListTuningAudit(r.Context(), trendsAuditFetchCap)
if err != nil {
h.logger.Error("api: recommendation trends audit", "err", err)
writeErr(w, apierror.InternalMsg("lookup failed", err))
return
}
writeJSON(w, http.StatusOK, buildTrendsResponse(weeks, time.Now().UTC(), rows, audits))
}
// buildTrendsResponse folds weekly rows into per-family series and
// windows the audit markers. Split from the handler for pure-unit
// testability.
func buildTrendsResponse(
weeks int, now time.Time,
rows []dbq.RecommendationWeeklyTrendsRow,
audits []dbq.RecommendationTuningAudit,
) trendsResp {
type accum struct {
fam recFamily
plays int64
points []trendPoint
}
families := map[string]*accum{}
for _, row := range rows {
fam := recFamily{key: "manual", label: "Manual library plays"}
if row.Source != nil && *row.Source != "" {
fam = bucketRecSource(*row.Source)
}
acc, ok := families[fam.key]
if !ok {
acc = &accum{fam: fam}
families[fam.key] = acc
}
acc.plays += row.Plays
p := trendPoint{
WeekStart: row.WeekStart.Time.Format("2006-01-02"),
Plays: row.Plays,
Skips: row.Skips,
AvgCompletion: row.AvgCompletion,
completionN: row.CompletionN,
}
if row.Plays > 0 {
p.SkipRate = float64(row.Skips) / float64(row.Plays)
p.TasteHitRate = float64(row.TasteHits) / float64(row.Plays)
}
// Same family can arrive as several raw sources (radio:<uuid>);
// merge same-week points play-weighted.
if n := len(acc.points); n > 0 && acc.points[n-1].WeekStart == p.WeekStart {
acc.points[n-1] = mergeTrendPoints(acc.points[n-1], p)
} else {
acc.points = append(acc.points, p)
}
}
resp := trendsResp{Weeks: weeks, Series: []trendSeries{}, Markers: []trendMarker{}}
for _, acc := range families {
resp.Series = append(resp.Series, trendSeries{
Key: acc.fam.key,
Label: acc.fam.label,
Intent: acc.fam.intent,
Plays: acc.plays,
Points: acc.points,
})
}
sort.Slice(resp.Series, func(i, j int) bool {
if resp.Series[i].Plays != resp.Series[j].Plays {
return resp.Series[i].Plays > resp.Series[j].Plays
}
return resp.Series[i].Key < resp.Series[j].Key
})
cutoff := now.Add(-time.Duration(weeks) * 7 * 24 * time.Hour)
for _, a := range audits {
if a.ChangedAt.Time.Before(cutoff) {
continue
}
resp.Markers = append(resp.Markers, trendMarker{
ChangedAt: a.ChangedAt.Time.UTC().Format(time.RFC3339),
Scope: a.Scope,
Action: a.Action,
Changes: json.RawMessage(a.Changes),
})
}
// ListTuningAudit returns newest-first; the timeline reads better
// oldest-first.
sort.Slice(resp.Markers, func(i, j int) bool {
return resp.Markers[i].ChangedAt < resp.Markers[j].ChangedAt
})
return resp
}
// mergeTrendPoints combines two same-week points of one family:
// counts add, skip/taste rates re-derive from the merged counts, and
// avg_completion is weighted by each side's completion sample count.
func mergeTrendPoints(a, b trendPoint) trendPoint {
out := trendPoint{
WeekStart: a.WeekStart,
Plays: a.Plays + b.Plays,
Skips: a.Skips + b.Skips,
completionN: a.completionN + b.completionN,
}
if out.Plays > 0 {
out.SkipRate = float64(out.Skips) / float64(out.Plays)
out.TasteHitRate = (a.TasteHitRate*float64(a.Plays) + b.TasteHitRate*float64(b.Plays)) /
float64(out.Plays)
}
if out.completionN > 0 {
out.AvgCompletion = (a.AvgCompletion*float64(a.completionN) + b.AvgCompletion*float64(b.completionN)) /
float64(out.completionN)
}
return out
}
@@ -0,0 +1,152 @@
package api
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
func date(s string) pgtype.Date {
t, _ := time.Parse("2006-01-02", s)
return pgtype.Date{Time: t, Valid: true}
}
func TestBuildTrendsResponse_SeriesMergingAndMarkers(t *testing.T) {
src := func(s string) *string { return &s }
now := time.Date(2026, 7, 3, 12, 0, 0, 0, time.UTC)
rows := []dbq.RecommendationWeeklyTrendsRow{
// Two radio session sources in the same week collapse into one
// family point; completion weighting by sample count.
{WeekStart: date("2026-06-22"), Source: src("radio:aaaa"),
Plays: 3, Skips: 1, CompletionN: 3, AvgCompletion: 0.6, TasteHits: 3},
{WeekStart: date("2026-06-22"), Source: src("radio:bbbb"),
Plays: 1, Skips: 1, CompletionN: 1, AvgCompletion: 0.2, TasteHits: 0},
{WeekStart: date("2026-06-29"), Source: src("radio:aaaa"),
Plays: 2, Skips: 0, CompletionN: 2, AvgCompletion: 0.9, TasteHits: 1},
// NULL source = manual baseline family.
{WeekStart: date("2026-06-29"), Source: nil,
Plays: 5, Skips: 1, CompletionN: 5, AvgCompletion: 0.8, TasteHits: 4},
}
audits := []dbq.RecommendationTuningAudit{
{ID: 2, ChangedAt: pgtype.Timestamptz{Time: now.Add(-24 * time.Hour), Valid: true},
Scope: "radio", Action: "update", Changes: []byte(`[{"field":"taste_weight","old":1,"new":2}]`)},
// Older than the window → dropped.
{ID: 1, ChangedAt: pgtype.Timestamptz{Time: now.Add(-100 * 7 * 24 * time.Hour), Valid: true},
Scope: "taste", Action: "reset", Changes: []byte(`[]`)},
}
resp := buildTrendsResponse(12, now, rows, audits)
if len(resp.Series) != 2 {
t.Fatalf("series = %d, want 2 (radio + manual)", len(resp.Series))
}
radio := resp.Series[0] // 6 plays > manual's 5 → sorted first
if radio.Key != "radio" || radio.Plays != 6 {
t.Fatalf("series[0] = %s/%d, want radio/6", radio.Key, radio.Plays)
}
if len(radio.Points) != 2 {
t.Fatalf("radio points = %d, want 2 weeks", len(radio.Points))
}
wk1 := radio.Points[0]
if wk1.WeekStart != "2026-06-22" || wk1.Plays != 4 || wk1.Skips != 2 {
t.Errorf("week1 = %+v, want 2026-06-22 with 4 plays / 2 skips", wk1)
}
if wk1.SkipRate != 0.5 {
t.Errorf("week1 skip_rate = %v, want 0.5", wk1.SkipRate)
}
// Completion weighted by sample count: (0.6*3 + 0.2*1) / 4 = 0.5.
if wk1.AvgCompletion < 0.49 || wk1.AvgCompletion > 0.51 {
t.Errorf("week1 avg_completion = %v, want 0.5", wk1.AvgCompletion)
}
// Taste hits: 3 of 4 plays.
if wk1.TasteHitRate != 0.75 {
t.Errorf("week1 taste_hit_rate = %v, want 0.75", wk1.TasteHitRate)
}
if manual := resp.Series[1]; manual.Key != "manual" || manual.Intent != "" {
t.Errorf("series[1] = %+v, want the manual baseline family", manual)
}
if len(resp.Markers) != 1 {
t.Fatalf("markers = %d, want 1 (out-of-window marker dropped)", len(resp.Markers))
}
if resp.Markers[0].Scope != "radio" || resp.Markers[0].Action != "update" {
t.Errorf("marker = %+v, want radio/update", resp.Markers[0])
}
}
func newTrendsRouter(h *handlers) chi.Router {
r := chi.NewRouter()
r.Get("/api/admin/recommendation-trends", h.handleGetRecommendationTrends)
return r
}
func TestRecommendationTrends_EndToEnd(t *testing.T) {
h, pool := testHandlers(t)
user := seedUser(t, pool, "trends", "pw", true)
artist := seedArtist(t, pool, "TrendArtist")
album := seedAlbum(t, pool, artist.ID, "TrendAlbum", 2020)
tk := seedTrack(t, pool, album.ID, artist.ID, "TrendTrack", 1, 200000)
session := seedPlaySession(t, pool, user.ID, time.Now())
// Positive taste weight for the artist → plays count as taste hits.
if _, err := pool.Exec(context.Background(),
`INSERT INTO taste_profile_artists (user_id, artist_id, weight) VALUES ($1, $2, 1.5)`,
user.ID, artist.ID); err != nil {
t.Fatalf("seed taste profile: %v", err)
}
radio := "radio"
seedSourcedPlay(t, h, user.ID, tk.ID, session, &radio, nil, 0.9, false)
// A knob turn to mark the timeline.
if err := h.recSettings.UpdateProfile(context.Background(), "radio",
map[string]float64{"taste_weight": 2}); err != nil {
t.Fatalf("UpdateProfile: %v", err)
}
req := httptest.NewRequest(http.MethodGet, "/api/admin/recommendation-trends", nil)
rec := httptest.NewRecorder()
newTrendsRouter(h).ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 (%s)", rec.Code, rec.Body.String())
}
var resp trendsResp
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v", err)
}
if resp.Weeks != trendsDefaultWeeks {
t.Errorf("weeks = %d, want %d", resp.Weeks, trendsDefaultWeeks)
}
var radioSeries *trendSeries
for i := range resp.Series {
if resp.Series[i].Key == "radio" {
radioSeries = &resp.Series[i]
}
}
if radioSeries == nil || len(radioSeries.Points) != 1 {
t.Fatalf("radio series = %+v, want one point", radioSeries)
}
if radioSeries.Points[0].TasteHitRate != 1.0 {
t.Errorf("taste_hit_rate = %v, want 1.0 (positive-weight artist)",
radioSeries.Points[0].TasteHitRate)
}
if len(resp.Markers) != 1 || resp.Markers[0].Scope != "radio" {
t.Errorf("markers = %+v, want the radio knob turn", resp.Markers)
}
}
func TestRecommendationTrends_InvalidWeeks(t *testing.T) {
h, _ := testHandlers(t)
req := httptest.NewRequest(http.MethodGet, "/api/admin/recommendation-trends?weeks=zero", nil)
rec := httptest.NewRecorder()
newTrendsRouter(h).ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Errorf("status = %d, want 400", rec.Code)
}
}
+152
View File
@@ -0,0 +1,152 @@
// Admin recommendation-tuning endpoints (#1250): the defaults-
// discovery lab. GET returns current values + shipped defaults for
// every scope; PATCH applies a partial update to one scope; reset
// restores a scope to shipped defaults. Every change writes an audit
// row (consumed by the metrics trend view, #1251).
package api
import (
"encoding/json"
"errors"
"net/http"
"github.com/go-chi/chi/v5"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/recommendation"
"git.fabledsword.com/bvandeusen/minstrel/internal/recsettings"
)
// weightsResp is one weight profile on the wire, keyed by the same
// snake_case field names the PATCH body accepts.
type weightsResp struct {
BaseWeight float64 `json:"base_weight"`
LikeBoost float64 `json:"like_boost"`
RecencyWeight float64 `json:"recency_weight"`
SkipPenalty float64 `json:"skip_penalty"`
JitterMagnitude float64 `json:"jitter_magnitude"`
ContextWeight float64 `json:"context_weight"`
SimilarityWeight float64 `json:"similarity_weight"`
TasteWeight float64 `json:"taste_weight"`
}
func weightsRespFrom(w recommendation.ScoringWeights) weightsResp {
return weightsResp{
BaseWeight: w.BaseWeight,
LikeBoost: w.LikeBoost,
RecencyWeight: w.RecencyWeight,
SkipPenalty: w.SkipPenalty,
JitterMagnitude: w.JitterMagnitude,
ContextWeight: w.ContextWeight,
SimilarityWeight: w.SimilarityWeight,
TasteWeight: w.TasteWeight,
}
}
type tasteTuningResp struct {
HalfLifeDays float64 `json:"half_life_days"`
EngagementHardSkip float64 `json:"engagement_hard_skip"`
EngagementNeutral float64 `json:"engagement_neutral"`
EngagementFull float64 `json:"engagement_full"`
}
func tasteRespFrom(t recsettings.TasteTuning) tasteTuningResp {
return tasteTuningResp{
HalfLifeDays: t.HalfLifeDays,
EngagementHardSkip: t.EngagementHardSkip,
EngagementNeutral: t.EngagementNeutral,
EngagementFull: t.EngagementFull,
}
}
// tuningSnapshot is both the GET response and the post-mutation echo:
// current values alongside shipped defaults so the card can mark
// which knobs deviate.
type tuningSnapshot struct {
Profiles map[string]weightsResp `json:"profiles"`
Taste tasteTuningResp `json:"taste"`
Shipped struct {
Profiles map[string]weightsResp `json:"profiles"`
Taste tasteTuningResp `json:"taste"`
} `json:"shipped"`
}
func (h *handlers) tuningSnapshot() tuningSnapshot {
var out tuningSnapshot
out.Profiles = map[string]weightsResp{
recsettings.ScopeRadio: weightsRespFrom(h.recSettings.Weights(recsettings.ScopeRadio)),
recsettings.ScopeDailyMix: weightsRespFrom(h.recSettings.Weights(recsettings.ScopeDailyMix)),
}
out.Taste = tasteRespFrom(h.recSettings.Taste())
out.Shipped.Profiles = map[string]weightsResp{
recsettings.ScopeRadio: weightsRespFrom(recsettings.ShippedRadioWeights()),
recsettings.ScopeDailyMix: weightsRespFrom(recsettings.ShippedDailyMixWeights()),
}
out.Shipped.Taste = tasteRespFrom(recsettings.ShippedTasteTuning())
return out
}
// handleGetRecommendationTuning implements GET /api/admin/recommendation-tuning.
func (h *handlers) handleGetRecommendationTuning(w http.ResponseWriter, _ *http.Request) {
writeJSON(w, http.StatusOK, h.tuningSnapshot())
}
// patchTuningReq carries the partial update: field name → new value,
// using the same snake_case names the GET response emits.
type patchTuningReq struct {
Values map[string]float64 `json:"values"`
}
// handlePatchRecommendationTuning implements
// PATCH /api/admin/recommendation-tuning/{scope}.
func (h *handlers) handlePatchRecommendationTuning(w http.ResponseWriter, r *http.Request) {
scope := chi.URLParam(r, "scope")
var body patchTuningReq
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeErr(w, apierror.BadRequest("bad_body", "invalid JSON"))
return
}
if len(body.Values) == 0 {
writeErr(w, apierror.BadRequest("bad_body", "values is empty"))
return
}
var err error
if scope == recsettings.ScopeTaste {
err = h.recSettings.UpdateTaste(r.Context(), body.Values)
} else {
err = h.recSettings.UpdateProfile(r.Context(), scope, body.Values)
}
if err != nil {
writeTuningErr(w, h, scope, err)
return
}
writeJSON(w, http.StatusOK, h.tuningSnapshot())
}
// handleResetRecommendationTuning implements
// POST /api/admin/recommendation-tuning/{scope}/reset.
func (h *handlers) handleResetRecommendationTuning(w http.ResponseWriter, r *http.Request) {
scope := chi.URLParam(r, "scope")
if err := h.recSettings.Reset(r.Context(), scope); err != nil {
writeTuningErr(w, h, scope, err)
return
}
writeJSON(w, http.StatusOK, h.tuningSnapshot())
}
// writeTuningErr maps recsettings validation errors to 400s and
// everything else to a logged 500.
func writeTuningErr(w http.ResponseWriter, h *handlers, scope string, err error) {
switch {
case errors.Is(err, recsettings.ErrUnknownScope):
writeErr(w, &apierror.Error{
Status: http.StatusNotFound, Code: "not_found", Message: "no such tuning scope",
})
case errors.Is(err, recsettings.ErrUnknownField), errors.Is(err, recsettings.ErrOutOfRange):
writeErr(w, apierror.BadRequest("invalid_tuning", err.Error()))
default:
h.logger.Error("admin: recommendation tuning", "scope", scope, "err", err)
writeErr(w, apierror.InternalMsg("tuning update failed", err))
}
}
@@ -0,0 +1,110 @@
package api
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/go-chi/chi/v5"
"git.fabledsword.com/bvandeusen/minstrel/internal/recsettings"
)
func newTuningRouter(h *handlers) chi.Router {
r := chi.NewRouter()
r.Get("/api/admin/recommendation-tuning", h.handleGetRecommendationTuning)
r.Patch("/api/admin/recommendation-tuning/{scope}", h.handlePatchRecommendationTuning)
r.Post("/api/admin/recommendation-tuning/{scope}/reset", h.handleResetRecommendationTuning)
return r
}
func decodeTuning(t *testing.T, rec *httptest.ResponseRecorder) tuningSnapshot {
t.Helper()
var snap tuningSnapshot
if err := json.Unmarshal(rec.Body.Bytes(), &snap); err != nil {
t.Fatalf("decode: %v", err)
}
return snap
}
func TestRecommendationTuning_GetReturnsShippedDefaults(t *testing.T) {
h, _ := testHandlers(t)
req := httptest.NewRequest(http.MethodGet, "/api/admin/recommendation-tuning", nil)
rec := httptest.NewRecorder()
newTuningRouter(h).ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rec.Code)
}
snap := decodeTuning(t, rec)
if snap.Profiles["radio"].TasteWeight != 1.0 || snap.Profiles["daily_mix"].TasteWeight != 1.5 {
t.Errorf("profiles = %+v, want shipped taste weights 1.0 / 1.5", snap.Profiles)
}
if snap.Taste.HalfLifeDays != 75 {
t.Errorf("taste half-life = %v, want shipped 75", snap.Taste.HalfLifeDays)
}
if snap.Shipped.Profiles["radio"] != snap.Profiles["radio"] {
t.Error("untouched values must equal shipped defaults")
}
}
func TestRecommendationTuning_PatchAndReset(t *testing.T) {
h, _ := testHandlers(t)
r := newTuningRouter(h)
req := httptest.NewRequest(http.MethodPatch, "/api/admin/recommendation-tuning/radio",
strings.NewReader(`{"values":{"taste_weight": 2.5}}`))
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("patch status = %d, want 200 (%s)", rec.Code, rec.Body.String())
}
snap := decodeTuning(t, rec)
if snap.Profiles["radio"].TasteWeight != 2.5 {
t.Errorf("patched taste_weight = %v, want 2.5", snap.Profiles["radio"].TasteWeight)
}
// The change is live for the radio scoring path.
if got := h.recSettings.Weights(recsettings.ScopeRadio).TasteWeight; got != 2.5 {
t.Errorf("service taste_weight = %v, want 2.5 (live effect)", got)
}
req = httptest.NewRequest(http.MethodPost, "/api/admin/recommendation-tuning/radio/reset", nil)
rec = httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("reset status = %d, want 200", rec.Code)
}
snap = decodeTuning(t, rec)
if snap.Profiles["radio"].TasteWeight != 1.0 {
t.Errorf("reset taste_weight = %v, want shipped 1.0", snap.Profiles["radio"].TasteWeight)
}
}
func TestRecommendationTuning_PatchErrors(t *testing.T) {
h, _ := testHandlers(t)
r := newTuningRouter(h)
cases := []struct {
name, path, body string
want int
}{
{"unknown scope", "/api/admin/recommendation-tuning/banana",
`{"values":{"taste_weight":1}}`, http.StatusNotFound},
{"unknown field", "/api/admin/recommendation-tuning/radio",
`{"values":{"vibes":1}}`, http.StatusBadRequest},
{"out of range", "/api/admin/recommendation-tuning/taste",
`{"values":{"engagement_neutral":2}}`, http.StatusBadRequest},
{"empty values", "/api/admin/recommendation-tuning/radio",
`{"values":{}}`, http.StatusBadRequest},
{"bad json", "/api/admin/recommendation-tuning/radio",
`{`, http.StatusBadRequest},
}
for _, c := range cases {
req := httptest.NewRequest(http.MethodPatch, c.path, strings.NewReader(c.body))
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != c.want {
t.Errorf("%s: status = %d, want %d", c.name, rec.Code, c.want)
}
}
}
+5
View File
@@ -22,6 +22,7 @@ type adminUserView struct {
DisplayName *string `json:"display_name"`
IsAdmin bool `json:"is_admin"`
AutoApproveRequests bool `json:"auto_approve_requests"`
DebugModeEnabled bool `json:"debug_mode_enabled"`
CreatedAt string `json:"created_at"`
}
@@ -44,6 +45,7 @@ func (h *handlers) handleAdminListUsers(w http.ResponseWriter, r *http.Request)
DisplayName: row.DisplayName,
IsAdmin: row.IsAdmin,
AutoApproveRequests: row.AutoApproveRequests,
DebugModeEnabled: row.DebugModeEnabled,
CreatedAt: row.CreatedAt.Time.UTC().Format(time.RFC3339),
}
out = append(out, v)
@@ -120,6 +122,7 @@ func (h *handlers) handleUpdateUserAdmin(w http.ResponseWriter, r *http.Request)
DisplayName: updated.DisplayName,
IsAdmin: updated.IsAdmin,
AutoApproveRequests: updated.AutoApproveRequests,
DebugModeEnabled: updated.DebugModeEnabled,
CreatedAt: updated.CreatedAt.Time.UTC().Format(time.RFC3339),
})
}
@@ -189,6 +192,7 @@ func (h *handlers) handleAdminCreateUser(w http.ResponseWriter, r *http.Request)
DisplayName: user.DisplayName,
IsAdmin: user.IsAdmin,
AutoApproveRequests: user.AutoApproveRequests,
DebugModeEnabled: user.DebugModeEnabled,
CreatedAt: user.CreatedAt.Time.UTC().Format(time.RFC3339),
})
}
@@ -329,6 +333,7 @@ func (h *handlers) handleAdminAutoApproveToggle(w http.ResponseWriter, r *http.R
DisplayName: updated.DisplayName,
IsAdmin: updated.IsAdmin,
AutoApproveRequests: updated.AutoApproveRequests,
DebugModeEnabled: updated.DebugModeEnabled,
CreatedAt: updated.CreatedAt.Time.UTC().Format(time.RFC3339),
})
}
+23 -1
View File
@@ -22,16 +22,18 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/mailer"
"git.fabledsword.com/bvandeusen/minstrel/internal/playevents"
"git.fabledsword.com/bvandeusen/minstrel/internal/playlists"
"git.fabledsword.com/bvandeusen/minstrel/internal/recsettings"
"git.fabledsword.com/bvandeusen/minstrel/internal/tracks"
)
// Mount attaches /api/* handlers to r. Public endpoints (login) are outside
// RequireUser; everything else is gated by the middleware. The events writer
// is shared with the Subsonic mount so /rest/scrobble feeds the same store.
func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playevents.Writer, recCfg config.RecommendationConfig, lidarrCfg *lidarrconfig.Service, lidarrReqs *lidarrrequests.Service, lidarrQuar *lidarrquarantine.Service, tracksSvc *tracks.Service, playlistsSvc *playlists.Service, coverEnricher *coverart.Enricher, coverSettings *coverart.SettingsService, scanner *library.Scanner, scanCfg library.RunScanConfig, dataDir string, sender mailer.Sender, bus *eventbus.Bus, playlistScheduler *playlists.Scheduler, streamSecret []byte) {
func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playevents.Writer, recCfg config.RecommendationConfig, recSettings *recsettings.Service, lidarrCfg *lidarrconfig.Service, lidarrReqs *lidarrrequests.Service, lidarrQuar *lidarrquarantine.Service, tracksSvc *tracks.Service, playlistsSvc *playlists.Service, coverEnricher *coverart.Enricher, coverSettings *coverart.SettingsService, scanner *library.Scanner, scanCfg library.RunScanConfig, dataDir string, sender mailer.Sender, bus *eventbus.Bus, playlistScheduler *playlists.Scheduler, streamSecret []byte) {
rng := rand.New(rand.NewSource(rand.Int63()))
h := &handlers{
pool: pool, logger: logger, events: events, recCfg: recCfg,
recSettings: recSettings,
rng: rng.Float64,
lidarrCfg: lidarrCfg,
lidarrRequests: lidarrReqs,
@@ -135,6 +137,12 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
// load failures). Admin-only inbox; any user can report.
authed.Post("/playback-errors", h.handleReportPlaybackError)
// Device diagnostics ingest (M9). Any signed-in user can
// POST a batch, but events are only stored when the
// account's debug_mode_enabled flag is on (handler no-ops
// otherwise). Admin views live under /admin/diagnostics.
authed.Post("/diagnostics", h.handleReportDiagnostics)
// Self-hosted in-app update channel (#397). Auth-gated to
// prevent anonymous bandwidth abuse on the APK stream;
// /apk additionally per-user rate-limited.
@@ -181,6 +189,11 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
admin.Delete("/users/{id}", h.handleAdminDeleteUser)
admin.Post("/users/{id}/reset-password", h.handleAdminResetPassword)
admin.Put("/users/{id}/auto-approve", h.handleAdminAutoApproveToggle)
admin.Put("/users/{id}/debug-mode", h.handleAdminDebugModeToggle)
// Device diagnostics timeline + device overview (M9).
admin.Get("/diagnostics", h.handleListAdminDiagnostics)
admin.Get("/diagnostics/devices", h.handleListAdminDiagnosticDevices)
admin.Get("/cover-sources", h.handleListCoverSources)
admin.Patch("/cover-sources/{provider_id}", h.handleUpdateCoverSource)
@@ -190,6 +203,14 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
admin.Get("/smtp-config", h.handleGetSMTPConfig)
admin.Put("/smtp-config", h.handleUpdateSMTPConfig)
admin.Post("/smtp-config/test", h.handleTestSMTPConfig)
// Recommendation tuning lab (#1250): scoring-weight
// profiles + taste-build knobs, DB-backed, live effect.
admin.Get("/recommendation-tuning", h.handleGetRecommendationTuning)
admin.Patch("/recommendation-tuning/{scope}", h.handlePatchRecommendationTuning)
admin.Post("/recommendation-tuning/{scope}/reset", h.handleResetRecommendationTuning)
// Weekly outcome trends + knob-turn markers (#1251).
admin.Get("/recommendation-trends", h.handleGetRecommendationTrends)
})
authed.Get("/playlists", h.handleListPlaylists)
@@ -212,6 +233,7 @@ type handlers struct {
logger *slog.Logger
events *playevents.Writer
recCfg config.RecommendationConfig
recSettings *recsettings.Service
rng func() float64
lidarrCfg *lidarrconfig.Service
lidarrRequests *lidarrrequests.Service
+6 -4
View File
@@ -29,6 +29,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/mailer"
"git.fabledsword.com/bvandeusen/minstrel/internal/playevents"
"git.fabledsword.com/bvandeusen/minstrel/internal/playlists"
"git.fabledsword.com/bvandeusen/minstrel/internal/recsettings"
"git.fabledsword.com/bvandeusen/minstrel/internal/tracks"
)
@@ -56,11 +57,12 @@ func testHandlers(t *testing.T) (*handlers, *pgxpool.Pool) {
dbtest.ResetDB(t, pool)
w := playevents.NewWriter(pool, logger, 30*time.Minute, 0.5, 30000)
recCfg := config.RecommendationConfig{
BaseWeight: 1.0, LikeBoost: 2.0, RecencyWeight: 1.0,
SkipPenalty: 1.0, JitterMagnitude: 0.1,
ContextWeight: 2.0, SimilarityWeight: 2.0,
RecentlyPlayedHours: 1, RadioSize: 50, RadioSizeMax: 200,
}
recSettings, err := recsettings.New(context.Background(), pool, logger)
if err != nil {
t.Fatalf("recsettings: %v", err)
}
lidarrCfg := lidarrconfig.New(pool)
lidarrReqs := lidarrrequests.NewService(pool, lidarrCfg, nil, nil)
lidarrQuar := lidarrquarantine.NewService(pool, lidarrCfg, nil)
@@ -70,7 +72,7 @@ func testHandlers(t *testing.T) (*handlers, *pgxpool.Pool) {
dataDir := t.TempDir()
tracksSvc := tracks.NewService(pool, logger, nil, dataDir)
playlistsSvc := playlists.NewService(pool, logger, dataDir)
h := &handlers{pool: pool, logger: logger, events: w, recCfg: recCfg, rng: func() float64 { return 0.5 }, lidarrCfg: lidarrCfg, lidarrRequests: lidarrReqs, lidarrQuarantine: lidarrQuar, tracks: tracksSvc, playlists: playlistsSvc, dataDir: dataDir, scanner: nil, scanCfg: library.RunScanConfig{}, mailer: &mailer.FakeSender{}}
h := &handlers{pool: pool, logger: logger, events: w, recCfg: recCfg, recSettings: recSettings, rng: func() float64 { return 0.5 }, lidarrCfg: lidarrCfg, lidarrRequests: lidarrReqs, lidarrQuarantine: lidarrQuar, tracks: tracksSvc, playlists: playlistsSvc, dataDir: dataDir, scanner: nil, scanCfg: library.RunScanConfig{}, mailer: &mailer.FakeSender{}}
return h, pool
}
+346
View File
@@ -0,0 +1,346 @@
package api
import (
"encoding/json"
"net/http"
"strconv"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
// validDiagnosticKinds is the COARSE category whitelist. Mirrors the
// CHECK constraint in migration 0036 — keep both in sync (per the
// enum-CHECK-whitelist rule). The finer event discriminator + all
// event-specific fields live inside each event's payload, so new event
// variants do NOT require a new category here.
var validDiagnosticKinds = map[string]struct{}{
"connectivity": {},
"upnp_sync": {}, // genuinely UPnP/Sonos-specific (drops, resync)
"playback": {}, // route + player-state, any output (phone/BT/UPnP)
"power": {},
"lifecycle": {},
"heartbeat": {},
"http": {},
}
// maxDiagnosticBatch caps a single ingest call. The client buffers and
// flushes in batches of ~100; 500 leaves generous headroom while
// bounding a misbehaving client's per-request work.
const maxDiagnosticBatch = 500
// Diagnostics list paging. Larger than the generic parsePaging caps
// because the admin timeline is meant to be exported as a slice and
// handed off for analysis — a heartbeat-dense window has many rows.
const (
defaultDiagnosticPageSize = 500
maxDiagnosticPageSize = 5000
)
// diagnosticEventIn is one event in a POST /api/diagnostics batch.
// OccurredAt is the client device clock as epoch milliseconds (may be
// skewed — the server also stamps received_at). Payload is opaque JSON
// carrying the event's sub-type + fields.
type diagnosticEventIn struct {
Kind string `json:"kind"`
OccurredAt int64 `json:"occurred_at"`
Payload json.RawMessage `json:"payload"`
}
// reportDiagnosticsRequest is the body of POST /api/diagnostics — one
// batch from one device.
type reportDiagnosticsRequest struct {
ClientID string `json:"client_id"`
AppVersion *string `json:"app_version,omitempty"`
OsVersion *string `json:"os_version,omitempty"`
Events []diagnosticEventIn `json:"events"`
}
// handleReportDiagnostics implements POST /api/diagnostics. Any signed-in
// user can call it, but events are only stored when the account's
// debug_mode_enabled flag is on; otherwise the call is a 204 no-op so the
// client can safely drop the batch (it gates locally on /api/me, this is
// the race-safe backstop for "admin just disabled debug").
func (h *handlers) handleReportDiagnostics(w http.ResponseWriter, r *http.Request) {
user, ok := requireUser(w, r)
if !ok {
return
}
if !user.DebugModeEnabled {
w.WriteHeader(http.StatusNoContent)
return
}
var req reportDiagnosticsRequest
if !decodeBody(w, r, &req) {
return
}
if strings.TrimSpace(req.ClientID) == "" {
writeErr(w, apierror.BadRequest("bad_request", "client_id required"))
return
}
if len(req.Events) == 0 {
writeJSON(w, http.StatusAccepted, map[string]int{"accepted": 0})
return
}
if len(req.Events) > maxDiagnosticBatch {
writeErr(w, apierror.BadRequest("batch_too_large", "too many events in one batch"))
return
}
for i := range req.Events {
if _, valid := validDiagnosticKinds[req.Events[i].Kind]; !valid {
writeErr(w, apierror.BadRequest("invalid_kind", "unknown diagnostic kind"))
return
}
}
err := pgx.BeginFunc(r.Context(), h.pool, func(tx pgx.Tx) error {
q := dbq.New(tx)
for i := range req.Events {
ev := req.Events[i]
payload := ev.Payload
if len(payload) == 0 {
payload = json.RawMessage("{}")
}
if _, err := q.InsertDiagnosticEvent(r.Context(), dbq.InsertDiagnosticEventParams{
UserID: user.ID,
ClientID: req.ClientID,
AppVersion: req.AppVersion,
OsVersion: req.OsVersion,
Kind: ev.Kind,
Payload: payload,
OccurredAt: pgtype.Timestamptz{Time: time.UnixMilli(ev.OccurredAt).UTC(), Valid: true},
}); err != nil {
return err
}
}
return nil
})
if err != nil {
writeErrWithLog(w, h.logger, "diagnostics: insert batch failed", apierror.Internal(err))
return
}
writeJSON(w, http.StatusAccepted, map[string]int{"accepted": len(req.Events)})
}
// adminDiagnosticView is one row in the admin timeline response. Payload
// is passed through verbatim as raw JSON so the SPA can render / export
// the structured event without the server re-shaping it.
type adminDiagnosticView struct {
ID string `json:"id"`
UserID string `json:"user_id"`
Username string `json:"username"`
ClientID string `json:"client_id"`
AppVersion *string `json:"app_version,omitempty"`
OsVersion *string `json:"os_version,omitempty"`
Kind string `json:"kind"`
Payload json.RawMessage `json:"payload"`
OccurredAt string `json:"occurred_at"`
ReceivedAt string `json:"received_at"`
}
// handleListAdminDiagnostics implements GET /api/admin/diagnostics with
// optional filters user_id, client_id, kind, from, to (RFC3339 or epoch
// millis) plus limit/offset. Newest-first; the SPA reverses for a
// chronological timeline + export.
func (h *handlers) handleListAdminDiagnostics(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
params, ok := h.buildDiagnosticsFilter(w, q)
if !ok {
return
}
rows, err := dbq.New(h.pool).ListAdminDiagnostics(r.Context(), params)
if err != nil {
writeErrWithLog(w, h.logger, "admin: list diagnostics failed", apierror.Internal(err))
return
}
out := make([]adminDiagnosticView, 0, len(rows))
for _, row := range rows {
out = append(out, adminDiagnosticView{
ID: uuidToString(row.ID),
UserID: uuidToString(row.UserID),
Username: row.Username,
ClientID: row.ClientID,
AppVersion: row.AppVersion,
OsVersion: row.OsVersion,
Kind: row.Kind,
Payload: json.RawMessage(row.Payload),
OccurredAt: formatTimestamp(row.OccurredAt),
ReceivedAt: formatTimestamp(row.ReceivedAt),
})
}
writeJSON(w, http.StatusOK, out)
}
// buildDiagnosticsFilter parses the query string into a query param
// struct, writing a 400 and returning ok=false on a malformed time.
func (h *handlers) buildDiagnosticsFilter(
w http.ResponseWriter, q map[string][]string,
) (dbq.ListAdminDiagnosticsParams, bool) {
get := func(k string) string {
if v, present := q[k]; present && len(v) > 0 {
return v[0]
}
return ""
}
limit, offset, err := parseDiagPaging(get("limit"), get("offset"))
if err != nil {
writeErr(w, apierror.BadRequest("bad_request", "invalid limit or offset"))
return dbq.ListAdminDiagnosticsParams{}, false
}
from, okFrom := parseDiagTime(get("from"))
to, okTo := parseDiagTime(get("to"))
if !okFrom || !okTo {
writeErr(w, apierror.BadRequest("bad_request", "invalid from/to timestamp"))
return dbq.ListAdminDiagnosticsParams{}, false
}
params := dbq.ListAdminDiagnosticsParams{
ClientID: optionalQuery(get("client_id")),
Kind: optionalQuery(get("kind")),
FromTs: from,
ToTs: to,
Off: int32(offset),
Lim: int32(limit),
}
if uid, valid := parseUUID(get("user_id")); valid {
params.UserID = uid
}
return params, true
}
// adminDiagnosticDeviceView is one device in the "who is reporting"
// overview / device filter dropdown.
type adminDiagnosticDeviceView struct {
ClientID string `json:"client_id"`
UserID string `json:"user_id"`
Username string `json:"username"`
AppVersion string `json:"app_version"`
OsVersion string `json:"os_version"`
LastSeen string `json:"last_seen"`
EventCount int64 `json:"event_count"`
}
// handleListAdminDiagnosticDevices implements GET
// /api/admin/diagnostics/devices?user_id=. Lists distinct reporting
// devices so the admin UI can populate its device filter.
func (h *handlers) handleListAdminDiagnosticDevices(w http.ResponseWriter, r *http.Request) {
var userID pgtype.UUID
if uid, valid := parseUUID(r.URL.Query().Get("user_id")); valid {
userID = uid
}
rows, err := dbq.New(h.pool).ListDiagnosticDevices(r.Context(), userID)
if err != nil {
writeErrWithLog(w, h.logger, "admin: list diagnostic devices failed", apierror.Internal(err))
return
}
out := make([]adminDiagnosticDeviceView, 0, len(rows))
for _, row := range rows {
out = append(out, adminDiagnosticDeviceView{
ClientID: row.ClientID,
UserID: uuidToString(row.UserID),
Username: row.Username,
AppVersion: row.AppVersion,
OsVersion: row.OsVersion,
LastSeen: formatTimestamp(row.LastSeen),
EventCount: row.EventCount,
})
}
writeJSON(w, http.StatusOK, out)
}
// adminDebugModeReq is the body of PUT /api/admin/users/{id}/debug-mode.
type adminDebugModeReq struct {
Enabled bool `json:"enabled"`
}
// handleAdminDebugModeToggle implements PUT
// /api/admin/users/{id}/debug-mode — flip an account's diagnostics
// opt-in remotely while a bug is live. Echoes the updated user.
func (h *handlers) handleAdminDebugModeToggle(w http.ResponseWriter, r *http.Request) {
targetID, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
var req adminDebugModeReq
if !decodeBody(w, r, &req) {
return
}
updated, err := dbq.New(h.pool).SetDebugMode(r.Context(), dbq.SetDebugModeParams{
ID: targetID,
DebugModeEnabled: req.Enabled,
})
if err != nil {
writeErrWithLog(w, h.logger, "admin: set debug mode failed", apierror.Internal(err))
return
}
h.logger.Info("admin: debug mode toggled",
"target_user", uuidToString(targetID), "enabled", req.Enabled)
writeJSON(w, http.StatusOK, adminUserView{
ID: uuidToString(updated.ID),
Username: updated.Username,
DisplayName: updated.DisplayName,
IsAdmin: updated.IsAdmin,
AutoApproveRequests: updated.AutoApproveRequests,
DebugModeEnabled: updated.DebugModeEnabled,
CreatedAt: updated.CreatedAt.Time.UTC().Format(time.RFC3339),
})
}
// parseDiagPaging reads limit/offset with diagnostics-specific caps.
// Blank → defaults; out-of-range clamps; non-numeric → error.
func parseDiagPaging(rawLimit, rawOffset string) (limit, offset int, err error) {
limit = defaultDiagnosticPageSize
if s := strings.TrimSpace(rawLimit); s != "" {
n, perr := strconv.Atoi(s)
if perr != nil {
return 0, 0, perr
}
if n < 1 {
n = 1
}
if n > maxDiagnosticPageSize {
n = maxDiagnosticPageSize
}
limit = n
}
if s := strings.TrimSpace(rawOffset); s != "" {
n, perr := strconv.Atoi(s)
if perr != nil {
return 0, 0, perr
}
if n > 0 {
offset = n
}
}
return limit, offset, nil
}
// optionalQuery returns nil for an absent/blank query value so the sqlc
// NULL-or-equals filter treats it as "no filter."
func optionalQuery(raw string) *string {
raw = strings.TrimSpace(raw)
if raw == "" {
return nil
}
return &raw
}
// parseDiagTime accepts RFC3339 or epoch-millis. Empty = absent (NULL
// filter, ok=true). Returns ok=false only on a non-empty unparseable value.
func parseDiagTime(raw string) (pgtype.Timestamptz, bool) {
raw = strings.TrimSpace(raw)
if raw == "" {
return pgtype.Timestamptz{}, true
}
if t, err := time.Parse(time.RFC3339, raw); err == nil {
return pgtype.Timestamptz{Time: t.UTC(), Valid: true}, true
}
if ms, err := strconv.ParseInt(raw, 10, 64); err == nil {
return pgtype.Timestamptz{Time: time.UnixMilli(ms).UTC(), Valid: true}, true
}
return pgtype.Timestamptz{}, false
}
+21 -1
View File
@@ -31,6 +31,26 @@ type playStartedResponse struct {
SessionID string `json:"session_id"`
}
// eventFutureSkew is the tolerated client-clock drift into the future
// before a timestamp is treated as bogus and replaced with now.
const eventFutureSkew = 5 * time.Minute
// clampEventTime bounds a client-supplied event timestamp to sanity:
// no earlier than the account's creation (offline replays can
// legitimately be days old, but no play can predate the user) and no
// later than now + a small skew allowance. Unbounded client clocks
// previously let a skewed device write arbitrarily old plays, which
// poisoned Rediscover's "not played in 6 months" ordering (#1246).
func clampEventTime(at, userCreatedAt, now time.Time) time.Time {
if at.Before(userCreatedAt) {
return userCreatedAt
}
if at.After(now.Add(eventFutureSkew)) {
return now
}
return at
}
type okResponse struct {
OK bool `json:"ok"`
}
@@ -52,7 +72,7 @@ func (h *handlers) handleEvents(w http.ResponseWriter, r *http.Request) {
writeErr(w, apierror.BadRequest("bad_request", "invalid `at` timestamp"))
return
}
at = parsed
at = clampEventTime(parsed.UTC(), user.CreatedAt.Time, time.Now().UTC())
}
clientID := ""
if req.ClientID != nil {
+1 -1
View File
@@ -465,7 +465,7 @@ func TestRoutesRegisteredInMount(t *testing.T) {
r := chi.NewRouter()
w := playevents.NewWriter(h.pool, slog.New(slog.NewTextHandler(io.Discard, nil)),
30*time.Minute, 0.5, 30000)
Mount(r, h.pool, h.logger, w, config.RecommendationConfig{RadioSize: 50, RadioSizeMax: 200, RecentlyPlayedHours: 1}, h.lidarrCfg, h.lidarrRequests, h.lidarrQuarantine, h.tracks, h.playlists, h.coverart, h.coverSettings, h.scanner, h.scanCfg, h.dataDir, nil, eventbus.New(), nil, nil)
Mount(r, h.pool, h.logger, w, config.RecommendationConfig{RadioSize: 50, RadioSizeMax: 200, RecentlyPlayedHours: 1}, h.recSettings, h.lidarrCfg, h.lidarrRequests, h.lidarrQuarantine, h.tracks, h.playlists, h.coverart, h.coverSettings, h.scanner, h.scanCfg, h.dataDir, nil, eventbus.New(), nil, nil)
paths := []string{
"/api/artists",
+10 -5
View File
@@ -35,6 +35,10 @@ type meProfileResp struct {
DisplayName *string `json:"display_name"`
Email *string `json:"email"`
IsAdmin bool `json:"is_admin"`
// DebugModeEnabled is the account's diagnostics opt-in. The client
// reads it here to decide whether to run the on-device diagnostics
// reporter (M9). Admin-set; the client also has a local OFF switch.
DebugModeEnabled bool `json:"debug_mode_enabled"`
}
func (h *handlers) handleUpdateMyProfile(w http.ResponseWriter, r *http.Request) {
@@ -120,10 +124,11 @@ func (h *handlers) handleUpdateMyProfile(w http.ResponseWriter, r *http.Request)
func profileViewFromUser(u dbq.User) meProfileResp {
return meProfileResp{
ID: uuidToString(u.ID),
Username: u.Username,
DisplayName: u.DisplayName,
Email: u.Email,
IsAdmin: u.IsAdmin,
ID: uuidToString(u.ID),
Username: u.Username,
DisplayName: u.DisplayName,
Email: u.Email,
IsAdmin: u.IsAdmin,
DebugModeEnabled: u.DebugModeEnabled,
}
}
+257 -28
View File
@@ -2,7 +2,9 @@ package api
import (
"net/http"
"sort"
"strconv"
"strings"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
@@ -11,28 +13,142 @@ import (
const (
recMetricsDefaultDays = 30
recMetricsMaxDays = 365
// recMetricsLowVolume marks a family as low-confidence rather than
// hiding it: with fewer plays than this a skip rate is anecdote, not
// signal, but silently dropping the row would misread as "surface
// unused". The web renders low-confidence rows dimmed.
recMetricsLowVolume = 20
)
// recommendationMetric is one recommendation surface's outcomes.
type recommendationMetric struct {
Source string `json:"source"` // 'for_you' | 'discover' | mixes
Plays int64 `json:"plays"` // plays launched from this surface
// Surface intents (milestone #127): each family is judged against its
// job, not one global bar — discovery mixes are EXPECTED to run higher
// skip rates than the go-to surfaces.
const (
intentGoTo = "go_to"
intentDiscovery = "discovery"
intentDirect = "direct"
)
// surfaceMetric is one bucketed surface family's outcomes.
type surfaceMetric struct {
Key string `json:"key"` // stable family key ("for_you", "radio", …)
Label string `json:"label"` // display label
Plays int64 `json:"plays"` // plays launched from this family
Skips int64 `json:"skips"` // of those, marked skipped
SkipRate float64 `json:"skip_rate"` // skips / plays, [0,1]
AvgCompletion float64 `json:"avg_completion"` // mean completion ratio, [0,1]
LowConfidence bool `json:"low_confidence"` // plays < recMetricsLowVolume
// Breakdown splits the family into the pick-kind populations its
// builder stamped (#1249, generalized #1270): For You's taste/fresh,
// Discover's buckets, tier1-3 for tiered mixes — plus earlier plays
// that predate attribution. Present only when the family has at
// least one attributed play; the parent row remains the sum of its
// breakdown.
Breakdown []surfaceMetric `json:"breakdown,omitempty"`
}
// surfaceGroup is one intent band of surface families.
type surfaceGroup struct {
Intent string `json:"intent"` // go_to | discovery | direct
Label string `json:"label"`
Surfaces []surfaceMetric `json:"surfaces"`
}
type recommendationMetricsResp struct {
WindowDays int `json:"window_days"`
Sources []recommendationMetric `json:"sources"`
WindowDays int `json:"window_days"`
// Baseline is the control group: plays the user picked manually
// (source IS NULL). Surfaces are judged as deltas against it; nil
// when the window holds no manual plays.
Baseline *surfaceMetric `json:"baseline"`
Groups []surfaceGroup `json:"groups"`
}
// recFamily is the bucketing target for a raw play_events.source value.
type recFamily struct {
key string
label string
intent string
}
// bucketRecSource maps a raw client-stamped source string to its stable
// family. One-off sources (album:<uuid>, radio:<uuid>) collapse into
// their family so the table stays readable at any library size.
func bucketRecSource(src string) recFamily {
switch {
case src == "for_you":
return recFamily{"for_you", "For You", intentGoTo}
case src == "songs_like_artist":
return recFamily{"songs_like_artist", "Songs like…", intentGoTo}
case src == "radio" || strings.HasPrefix(src, "radio:"):
return recFamily{"radio", "Radio", intentGoTo}
case src == "discover":
return recFamily{"discover", "Discover", intentDiscovery}
case src == "deep_cuts":
return recFamily{"deep_cuts", "Deep cuts", intentDiscovery}
case src == "rediscover":
return recFamily{"rediscover", "Rediscover", intentDiscovery}
case src == "new_for_you":
return recFamily{"new_for_you", "New for you", intentDiscovery}
case src == "on_this_day":
return recFamily{"on_this_day", "On this day", intentDiscovery}
case src == "first_listens":
return recFamily{"first_listens", "First listens", intentDiscovery}
case strings.HasPrefix(src, "album:"):
return recFamily{"direct_album", "Album plays", intentDirect}
case strings.HasPrefix(src, "artist:"):
return recFamily{"direct_artist", "Artist plays", intentDirect}
case strings.HasPrefix(src, "offline:"):
return recFamily{"offline", "Offline pools", intentDirect}
case strings.HasPrefix(src, "home:"):
return recFamily{"home", "Home sections", intentDirect}
case src == "history":
return recFamily{"history", "History", intentDirect}
default:
return recFamily{"other", "Other", intentDirect}
}
}
// familyAccum merges raw source rows into one family, carrying the
// completion sample count so the merged average stays play-weighted.
type familyAccum struct {
fam recFamily
plays int64
skips int64
completionN int64
// completionSum is avg*count re-expanded, so merging N raw rows
// reduces to a single weighted division at the end.
completionSum float64
}
func (a *familyAccum) add(row dbq.RecommendationSourceMetricsForUserRow) {
a.plays += row.Plays
a.skips += row.Skips
a.completionN += row.CompletionN
a.completionSum += row.AvgCompletion * float64(row.CompletionN)
}
func (a *familyAccum) metric() surfaceMetric {
m := surfaceMetric{
Key: a.fam.key,
Label: a.fam.label,
Plays: a.plays,
Skips: a.skips,
LowConfidence: a.plays < recMetricsLowVolume,
}
if a.plays > 0 {
m.SkipRate = float64(a.skips) / float64(a.plays)
}
if a.completionN > 0 {
m.AvgCompletion = a.completionSum / float64(a.completionN)
}
return m
}
// handleGetRecommendationMetrics implements GET /api/me/recommendation-metrics.
// Per-source play outcomes (plays / skips / skip-rate / avg-completion) for the
// caller over the last `days` (default 30, capped at 365), so the operator can
// see which recommendation surfaces are landing and tune the taste weights.
// Only plays tagged with a system-playlist source count; library/radio plays
// (no source) are excluded.
// Bucketed per-surface-family outcomes for the caller over the last `days`
// (default 30, capped at 365), grouped by surface intent and anchored by the
// manual-plays baseline so the numbers are judgeable, not just observable.
func (h *handlers) handleGetRecommendationMetrics(w http.ResponseWriter, r *http.Request) {
caller, ok := requireUser(w, r)
if !ok {
@@ -51,28 +167,141 @@ func (h *handlers) handleGetRecommendationMetrics(w http.ResponseWriter, r *http
return
}
out := recommendationMetricsResp{
WindowDays: days,
Sources: make([]recommendationMetric, 0, len(rows)),
writeJSON(w, http.StatusOK, bucketMetricsResponse(days, rows))
}
// pickKindLabels is the display vocabulary for play_events.pick_kind
// values (mirrors the CHECK in migration 0039). Every system mix that
// stamps provenance gets its breakdown from this one map — adding a
// stamping mix needs no metrics change.
var pickKindLabels = map[string]string{
"taste": "Taste picks",
"fresh": "Fresh picks",
"dormant": "Dormant artists",
"cross_user": "Liked by others",
"random": "Random unheard",
"tier1": "Tier 1 (exact)",
"tier2": "Tier 2 (relaxed)",
"tier3": "Tier 3 (stretched)",
}
// pickKindOrder fixes breakdown row order; unattributed ("", i.e. NULL
// pick_kind — plays recorded before the mix stamped provenance, or
// whose track had rotated out of the snapshot at ingestion) renders
// last, kept visible so the parent row's sums stay transparent instead
// of silently shrinking. The DB CHECK gates pick_kind to exactly this
// vocabulary, so iterating the list is exhaustive.
var pickKindOrder = []string{
"taste", "fresh", "dormant", "cross_user", "random",
"tier1", "tier2", "tier3", "",
}
// pickKindFamily derives the sub-family for one (family, pick_kind)
// population, e.g. ("for_you", "taste") → for_you_taste "Taste picks".
func pickKindFamily(parent recFamily, kind string) recFamily {
if kind == "" {
return recFamily{parent.key + "_unattributed", "Earlier plays", parent.intent}
}
label, ok := pickKindLabels[kind]
if !ok {
label = kind
}
return recFamily{parent.key + "_" + kind, label, parent.intent}
}
// pickKindBreakdown folds a family's per-pick-kind accums into its
// Breakdown rows. Attached only when at least one attributed play
// exists — an all-unattributed breakdown would just repeat the parent
// row, and families that never stamp (radio, direct plays) stay flat.
func pickKindBreakdown(picks map[string]*familyAccum) []surfaceMetric {
attributed := int64(0)
for kind, acc := range picks {
if kind != "" {
attributed += acc.plays
}
}
if attributed == 0 {
return nil
}
out := make([]surfaceMetric, 0, len(picks))
for _, kind := range pickKindOrder {
if acc, ok := picks[kind]; ok && acc.plays > 0 {
out = append(out, acc.metric())
}
}
return out
}
// bucketMetricsResponse folds the raw per-source rows into the grouped,
// baseline-anchored response shape. Split from the handler for pure-unit
// testability.
func bucketMetricsResponse(
days int, rows []dbq.RecommendationSourceMetricsForUserRow,
) recommendationMetricsResp {
baseline := &familyAccum{fam: recFamily{"manual", "Manual library plays", ""}}
families := map[string]*familyAccum{}
picks := map[string]map[string]*familyAccum{}
for _, row := range rows {
source := ""
if row.Source != nil {
source = *row.Source
if row.Source == nil || *row.Source == "" {
baseline.add(row)
continue
}
var skipRate float64
if row.Plays > 0 {
skipRate = float64(row.Skips) / float64(row.Plays)
fam := bucketRecSource(*row.Source)
acc, exists := families[fam.key]
if !exists {
acc = &familyAccum{fam: fam}
families[fam.key] = acc
}
out.Sources = append(out.Sources, recommendationMetric{
Source: source,
Plays: row.Plays,
Skips: row.Skips,
SkipRate: skipRate,
AvgCompletion: row.AvgCompletion,
})
acc.add(row)
// Accumulate the pick-kind population unconditionally; families
// that never stamp end up all-unattributed and get no breakdown.
kind := ""
if row.PickKind != nil {
kind = *row.PickKind
}
byKind, ok := picks[fam.key]
if !ok {
byKind = map[string]*familyAccum{}
picks[fam.key] = byKind
}
pick, ok := byKind[kind]
if !ok {
pick = &familyAccum{fam: pickKindFamily(fam, kind)}
byKind[kind] = pick
}
pick.add(row)
}
writeJSON(w, http.StatusOK, out)
resp := recommendationMetricsResp{WindowDays: days, Groups: []surfaceGroup{}}
if baseline.plays > 0 {
m := baseline.metric()
resp.Baseline = &m
}
for _, g := range []struct{ intent, label string }{
{intentGoTo, "Go-to surfaces"},
{intentDiscovery, "Discovery mixes"},
{intentDirect, "Direct plays"},
} {
group := surfaceGroup{Intent: g.intent, Label: g.label}
for _, acc := range families {
if acc.fam.intent == g.intent {
m := acc.metric()
m.Breakdown = pickKindBreakdown(picks[acc.fam.key])
group.Surfaces = append(group.Surfaces, m)
}
}
if len(group.Surfaces) == 0 {
continue
}
sort.Slice(group.Surfaces, func(i, j int) bool {
if group.Surfaces[i].Plays != group.Surfaces[j].Plays {
return group.Surfaces[i].Plays > group.Surfaces[j].Plays
}
return group.Surfaces[i].Key < group.Surfaces[j].Key
})
resp.Groups = append(resp.Groups, group)
}
return resp
}
// parseMetricsDays reads the `days` query param (default 30, capped at 365).
+197 -24
View File
@@ -11,6 +11,8 @@ import (
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
func newMetricsRouter(h *handlers) chi.Router {
@@ -20,17 +22,19 @@ func newMetricsRouter(h *handlers) chi.Router {
}
// seedSourcedPlay inserts a play_event with an explicit source + completion +
// skip flag. A nil source inserts NULL (library/radio play).
// skip flag. A nil source inserts NULL (manual library play → baseline).
// pickKind is the For You taste/fresh attribution (#1249); nil everywhere
// except attributed for_you plays.
func seedSourcedPlay(
t *testing.T, h *handlers, userID, trackID, sessionID pgtype.UUID,
source *string, completion float64, skipped bool,
source, pickKind *string, completion float64, skipped bool,
) {
t.Helper()
if _, err := h.pool.Exec(context.Background(),
`INSERT INTO play_events
(user_id, track_id, session_id, started_at, source, completion_ratio, was_skipped)
VALUES ($1, $2, $3, now(), $4, $5, $6)`,
userID, trackID, sessionID, source, completion, skipped); err != nil {
(user_id, track_id, session_id, started_at, source, pick_kind, completion_ratio, was_skipped)
VALUES ($1, $2, $3, now(), $4, $5, $6, $7)`,
userID, trackID, sessionID, source, pickKind, completion, skipped); err != nil {
t.Fatalf("seed sourced play: %v", err)
}
}
@@ -45,7 +49,168 @@ func TestRecommendationMetrics_NoSession401(t *testing.T) {
}
}
func TestRecommendationMetrics_AggregatesBySourceExcludingNull(t *testing.T) {
// findSurface returns the named family from any group, or nil.
func findSurface(resp recommendationMetricsResp, key string) *surfaceMetric {
for _, g := range resp.Groups {
for i := range g.Surfaces {
if g.Surfaces[i].Key == key {
return &g.Surfaces[i]
}
}
}
return nil
}
func TestBucketMetricsResponse_FamiliesGroupsBaseline(t *testing.T) {
src := func(s string) *string { return &s }
rows := []dbq.RecommendationSourceMetricsForUserRow{
{Source: src("for_you"), Plays: 3, Skips: 1, CompletionN: 3, AvgCompletion: 2.0 / 3},
// Two radio sessions collapse into one "radio" family; weighted
// completion = (0.2*1 + 0.8*1) / 2 = 0.5.
{Source: src("radio:aaaa"), Plays: 1, Skips: 1, CompletionN: 1, AvgCompletion: 0.2},
{Source: src("radio:bbbb"), Plays: 1, Skips: 0, CompletionN: 1, AvgCompletion: 0.8},
{Source: src("album:cccc"), Plays: 1, Skips: 0, CompletionN: 0, AvgCompletion: 0},
{Source: src("discover"), Plays: 1, Skips: 0, CompletionN: 1, AvgCompletion: 0.8},
// NULL source = manual plays → baseline, not a group row.
{Source: nil, Plays: 25, Skips: 5, CompletionN: 20, AvgCompletion: 0.9},
}
resp := bucketMetricsResponse(recMetricsDefaultDays, rows)
if resp.Baseline == nil {
t.Fatal("baseline missing")
}
if resp.Baseline.Plays != 25 || resp.Baseline.SkipRate != 0.2 {
t.Errorf("baseline = %+v, want plays=25 skip_rate=0.2", resp.Baseline)
}
if resp.Baseline.LowConfidence {
t.Error("baseline with 25 plays should not be low-confidence")
}
radio := findSurface(resp, "radio")
if radio == nil {
t.Fatal("radio family missing")
}
if radio.Plays != 2 || radio.Skips != 1 {
t.Errorf("radio plays/skips = %d/%d, want 2/1", radio.Plays, radio.Skips)
}
if radio.AvgCompletion < 0.49 || radio.AvgCompletion > 0.51 {
t.Errorf("radio avg_completion = %.3f, want 0.5 (play-weighted merge)", radio.AvgCompletion)
}
if !radio.LowConfidence {
t.Error("radio with 2 plays should be low-confidence")
}
if s := findSurface(resp, "direct_album"); s == nil || s.Plays != 1 {
t.Errorf("direct_album = %+v, want plays=1", s)
}
if s := findSurface(resp, ""); s != nil {
t.Error("NULL source must not appear as a surface family")
}
// No attributed (taste/fresh) plays in this fixture → no breakdown;
// an all-unattributed breakdown would just repeat the parent row.
if fy := findSurface(resp, "for_you"); fy == nil || fy.Breakdown != nil {
t.Errorf("for_you breakdown = %+v, want nil without attributed plays", fy)
}
// Group ordering is intent-banded: go_to before discovery before direct.
wantOrder := []string{intentGoTo, intentDiscovery, intentDirect}
if len(resp.Groups) != len(wantOrder) {
t.Fatalf("groups = %d, want %d", len(resp.Groups), len(wantOrder))
}
for i, g := range resp.Groups {
if g.Intent != wantOrder[i] {
t.Errorf("group[%d].intent = %s, want %s", i, g.Intent, wantOrder[i])
}
}
}
func TestBucketMetricsResponse_ForYouBreakdown(t *testing.T) {
src := func(s string) *string { return &s }
kind := func(s string) *string { return &s }
rows := []dbq.RecommendationSourceMetricsForUserRow{
{Source: src("for_you"), PickKind: kind("taste"), Plays: 30, Skips: 3,
CompletionN: 30, AvgCompletion: 0.9},
{Source: src("for_you"), PickKind: kind("fresh"), Plays: 10, Skips: 4,
CompletionN: 10, AvgCompletion: 0.5},
// NULL pick_kind = plays that predate attribution.
{Source: src("for_you"), Plays: 5, Skips: 1, CompletionN: 5, AvgCompletion: 0.7},
}
resp := bucketMetricsResponse(recMetricsDefaultDays, rows)
fy := findSurface(resp, "for_you")
if fy == nil {
t.Fatal("for_you family missing")
}
// The parent row stays the sum of its breakdown.
if fy.Plays != 45 || fy.Skips != 8 {
t.Errorf("for_you plays/skips = %d/%d, want 45/8", fy.Plays, fy.Skips)
}
if len(fy.Breakdown) != 3 {
t.Fatalf("breakdown rows = %d, want 3 (taste, fresh, earlier)", len(fy.Breakdown))
}
wantKeys := []string{"for_you_taste", "for_you_fresh", "for_you_unattributed"}
for i, k := range wantKeys {
if fy.Breakdown[i].Key != k {
t.Errorf("breakdown[%d].key = %s, want %s", i, fy.Breakdown[i].Key, k)
}
}
taste, fresh := fy.Breakdown[0], fy.Breakdown[1]
if taste.Plays != 30 || taste.SkipRate != 0.1 || taste.LowConfidence {
t.Errorf("taste = %+v, want plays=30 skip_rate=0.1 confident", taste)
}
if fresh.Plays != 10 || fresh.SkipRate != 0.4 || !fresh.LowConfidence {
t.Errorf("fresh = %+v, want plays=10 skip_rate=0.4 low-confidence", fresh)
}
// Breakdown rows never appear as their own surface families.
if s := findSurface(resp, "for_you_taste"); s != nil {
t.Error("for_you_taste must not be a top-level surface")
}
}
func TestBucketMetricsResponse_DiscoverBucketBreakdown(t *testing.T) {
// Provenance is standard (#1270): Discover's bucket stamps surface as
// a breakdown exactly like For You's taste/fresh — this is what makes
// the 40/30/30 allocation judgeable instead of a guess.
src := func(s string) *string { return &s }
kind := func(s string) *string { return &s }
rows := []dbq.RecommendationSourceMetricsForUserRow{
{Source: src("discover"), PickKind: kind("dormant"), Plays: 8, Skips: 2,
CompletionN: 8, AvgCompletion: 0.8},
{Source: src("discover"), PickKind: kind("cross_user"), Plays: 6, Skips: 3,
CompletionN: 6, AvgCompletion: 0.6},
{Source: src("discover"), PickKind: kind("random"), Plays: 4, Skips: 3,
CompletionN: 4, AvgCompletion: 0.4},
// NULL pick_kind = plays that predate bucket stamping.
{Source: src("discover"), Plays: 2, Skips: 0, CompletionN: 2, AvgCompletion: 0.9},
}
resp := bucketMetricsResponse(recMetricsDefaultDays, rows)
d := findSurface(resp, "discover")
if d == nil {
t.Fatal("discover family missing")
}
if d.Plays != 20 || d.Skips != 8 {
t.Errorf("discover plays/skips = %d/%d, want 20/8", d.Plays, d.Skips)
}
wantKeys := []string{
"discover_dormant", "discover_cross_user", "discover_random",
"discover_unattributed",
}
if len(d.Breakdown) != len(wantKeys) {
t.Fatalf("breakdown rows = %d, want %d", len(d.Breakdown), len(wantKeys))
}
for i, k := range wantKeys {
if d.Breakdown[i].Key != k {
t.Errorf("breakdown[%d].key = %s, want %s", i, d.Breakdown[i].Key, k)
}
}
if b := d.Breakdown[0]; b.Label != "Dormant artists" || b.Plays != 8 {
t.Errorf("dormant row = %+v, want label=Dormant artists plays=8", b)
}
}
func TestRecommendationMetrics_BucketsWithBaseline(t *testing.T) {
if os.Getenv("MINSTREL_TEST_DATABASE_URL") == "" {
t.Skip("MINSTREL_TEST_DATABASE_URL not set")
}
@@ -57,15 +222,17 @@ func TestRecommendationMetrics_AggregatesBySourceExcludingNull(t *testing.T) {
session := seedPlaySession(t, pool, user.ID, time.Now())
forYou := "for_you"
discover := "discover"
radioA := "radio:11111111-1111-1111-1111-111111111111"
taste, fresh := "taste", "fresh"
// for_you: 3 plays, 1 skipped; completions 1.0, 0.95, 0.05 → mean 0.6667.
seedSourcedPlay(t, h, user.ID, tk.ID, session, &forYou, 1.0, false)
seedSourcedPlay(t, h, user.ID, tk.ID, session, &forYou, 0.95, false)
seedSourcedPlay(t, h, user.ID, tk.ID, session, &forYou, 0.05, true)
// discover: 1 play.
seedSourcedPlay(t, h, user.ID, tk.ID, session, &discover, 0.8, false)
// library play (NULL source) — must be excluded.
seedSourcedPlay(t, h, user.ID, tk.ID, session, nil, 1.0, false)
// Two attributed as taste picks, the skipped one as a fresh pick (#1249).
seedSourcedPlay(t, h, user.ID, tk.ID, session, &forYou, &taste, 1.0, false)
seedSourcedPlay(t, h, user.ID, tk.ID, session, &forYou, &taste, 0.95, false)
seedSourcedPlay(t, h, user.ID, tk.ID, session, &forYou, &fresh, 0.05, true)
// A radio session play collapses into the "radio" family.
seedSourcedPlay(t, h, user.ID, tk.ID, session, &radioA, nil, 0.8, false)
// Manual play (NULL source) — the baseline row.
seedSourcedPlay(t, h, user.ID, tk.ID, session, nil, nil, 1.0, false)
req := httptest.NewRequest(http.MethodGet, "/api/me/recommendation-metrics", nil)
req = withUser(req, user)
@@ -82,15 +249,11 @@ func TestRecommendationMetrics_AggregatesBySourceExcludingNull(t *testing.T) {
if resp.WindowDays != recMetricsDefaultDays {
t.Errorf("window_days = %d, want %d", resp.WindowDays, recMetricsDefaultDays)
}
bySource := map[string]recommendationMetric{}
for _, m := range resp.Sources {
bySource[m.Source] = m
if resp.Baseline == nil || resp.Baseline.Plays != 1 {
t.Fatalf("baseline = %+v, want plays=1", resp.Baseline)
}
if _, present := bySource[""]; present {
t.Error("NULL-source (library) plays should be excluded")
}
fy, ok := bySource["for_you"]
if !ok {
fy := findSurface(resp, "for_you")
if fy == nil {
t.Fatal("for_you metrics missing")
}
if fy.Plays != 3 || fy.Skips != 1 {
@@ -102,7 +265,17 @@ func TestRecommendationMetrics_AggregatesBySourceExcludingNull(t *testing.T) {
if fy.AvgCompletion < 0.66 || fy.AvgCompletion > 0.67 {
t.Errorf("for_you avg_completion = %.4f, want ~0.6667", fy.AvgCompletion)
}
if d, ok := bySource["discover"]; !ok || d.Plays != 1 || d.Skips != 0 {
t.Errorf("discover metrics = %+v, want plays=1 skips=0", d)
// Pick-kind attribution surfaces as the For You breakdown (#1249).
if len(fy.Breakdown) != 2 {
t.Fatalf("for_you breakdown rows = %d, want 2 (taste, fresh)", len(fy.Breakdown))
}
if b := fy.Breakdown[0]; b.Key != "for_you_taste" || b.Plays != 2 || b.Skips != 0 {
t.Errorf("breakdown[0] = %+v, want for_you_taste plays=2 skips=0", b)
}
if b := fy.Breakdown[1]; b.Key != "for_you_fresh" || b.Plays != 1 || b.Skips != 1 {
t.Errorf("breakdown[1] = %+v, want for_you_fresh plays=1 skips=1", b)
}
if radio := findSurface(resp, "radio"); radio == nil || radio.Plays != 1 {
t.Errorf("radio family = %+v, want plays=1 (collapsed from radio:<uuid>)", radio)
}
}
+4 -10
View File
@@ -15,6 +15,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/recommendation"
"git.fabledsword.com/bvandeusen/minstrel/internal/recsettings"
)
// RadioResponse is the body of GET /api/radio.
@@ -100,16 +101,9 @@ func (h *handlers) handleRadio(w http.ResponseWriter, r *http.Request) {
}
}
weights := recommendation.ScoringWeights{
BaseWeight: h.recCfg.BaseWeight,
LikeBoost: h.recCfg.LikeBoost,
RecencyWeight: h.recCfg.RecencyWeight,
SkipPenalty: h.recCfg.SkipPenalty,
JitterMagnitude: h.recCfg.JitterMagnitude,
ContextWeight: h.recCfg.ContextWeight,
SimilarityWeight: h.recCfg.SimilarityWeight,
TasteWeight: h.recCfg.TasteWeight,
}
// Scoring weights come from the DB-backed tuning lab (#1250) —
// read per request so an admin change takes effect live.
weights := h.recSettings.Weights(recsettings.ScopeRadio)
picks := recommendation.Shuffle(candidates, weights, time.Now().UTC(), h.rng, limit-1)
out := make([]TrackRef, 0, len(picks)+1)
+7 -23
View File
@@ -88,20 +88,14 @@ type EventsConfig struct {
SkipMaxDurationPlayedMs int `yaml:"skip_max_duration_played_ms"`
}
// RecommendationConfig governs the M3 weighted-shuffle scoring (spec §6).
// All weights are operator-tunable; defaults match the spec recommendations.
// RecommendationConfig holds the radio path's operational knobs. The
// scoring WEIGHTS moved to DB-backed admin settings (#1250,
// internal/recsettings) — YAML is bootstrap-only; anything an operator
// tunes lives in the UI with live effect.
type RecommendationConfig struct {
BaseWeight float64 `yaml:"base_weight"`
LikeBoost float64 `yaml:"like_boost"`
RecencyWeight float64 `yaml:"recency_weight"`
SkipPenalty float64 `yaml:"skip_penalty"`
JitterMagnitude float64 `yaml:"jitter_magnitude"`
ContextWeight float64 `yaml:"context_weight"`
SimilarityWeight float64 `yaml:"similarity_weight"`
TasteWeight float64 `yaml:"taste_weight"`
RecentlyPlayedHours int `yaml:"recently_played_hours"`
RadioSize int `yaml:"radio_size"`
RadioSizeMax int `yaml:"radio_size_max"`
RecentlyPlayedHours int `yaml:"recently_played_hours"`
RadioSize int `yaml:"radio_size"`
RadioSizeMax int `yaml:"radio_size_max"`
}
func Default() Config {
@@ -120,16 +114,6 @@ func Default() Config {
SkipMaxDurationPlayedMs: 30000,
},
Recommendation: RecommendationConfig{
BaseWeight: 1.0,
LikeBoost: 2.0,
RecencyWeight: 1.0,
SkipPenalty: 1.0,
JitterMagnitude: 0.1,
ContextWeight: 2.0,
SimilarityWeight: 2.0,
// Radio is seed-directed (the user picked a direction), so taste
// is a lighter nudge here than in the daily mixes (1.5).
TasteWeight: 1.0,
RecentlyPlayedHours: 1,
RadioSize: 50,
RadioSizeMax: 200,
+221
View File
@@ -0,0 +1,221 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.31.1
// source: diagnostics.sql
package dbq
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const gcPruneDiagnostics = `-- name: GcPruneDiagnostics :execrows
DELETE FROM diagnostic_events WHERE received_at < now() - INTERVAL '30 days'
`
// Retention sweep for the gc worker. Deletes diagnostic_events older
// than 30 days by server clock (received_at) so a skewed client clock
// can't keep rows alive. Matches the other gc lifecycle sweeps'
// hardcoded-threshold style; diagnostics are debug telemetry, not user
// data, and 30 days is ample to investigate a live incident after the
// fact. Returns the affected row count for the sweep log line.
func (q *Queries) GcPruneDiagnostics(ctx context.Context) (int64, error) {
result, err := q.db.Exec(ctx, gcPruneDiagnostics)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const insertDiagnosticEvent = `-- name: InsertDiagnosticEvent :one
INSERT INTO diagnostic_events (
user_id, client_id, app_version, os_version, kind, payload, occurred_at
)
VALUES ($1, $2, $3, $4, $5, $6, $7)
RETURNING id, received_at
`
type InsertDiagnosticEventParams struct {
UserID pgtype.UUID
ClientID string
AppVersion *string
OsVersion *string
Kind string
Payload []byte
OccurredAt pgtype.Timestamptz
}
type InsertDiagnosticEventRow struct {
ID pgtype.UUID
ReceivedAt pgtype.Timestamptz
}
// Records one client-reported diagnostic event. The writer validates the
// `kind` category against its whitelist before this runs; the CHECK
// constraint is the belt-and-braces guard. payload carries the finer
// event discriminator + all event-specific fields. Returns received_at
// so the caller can confirm ingest.
func (q *Queries) InsertDiagnosticEvent(ctx context.Context, arg InsertDiagnosticEventParams) (InsertDiagnosticEventRow, error) {
row := q.db.QueryRow(ctx, insertDiagnosticEvent,
arg.UserID,
arg.ClientID,
arg.AppVersion,
arg.OsVersion,
arg.Kind,
arg.Payload,
arg.OccurredAt,
)
var i InsertDiagnosticEventRow
err := row.Scan(&i.ID, &i.ReceivedAt)
return i, err
}
const listAdminDiagnostics = `-- name: ListAdminDiagnostics :many
SELECT
de.id AS id,
de.user_id AS user_id,
u.username AS username,
de.client_id AS client_id,
de.app_version AS app_version,
de.os_version AS os_version,
de.kind AS kind,
de.payload AS payload,
de.occurred_at AS occurred_at,
de.received_at AS received_at
FROM diagnostic_events de
JOIN users u ON u.id = de.user_id
WHERE ($1::uuid IS NULL OR de.user_id = $1::uuid)
AND ($2::text IS NULL OR de.client_id = $2::text)
AND ($3::text IS NULL OR de.kind = $3::text)
AND ($4::timestamptz IS NULL
OR de.occurred_at >= $4::timestamptz)
AND ($5::timestamptz IS NULL
OR de.occurred_at <= $5::timestamptz)
ORDER BY de.occurred_at DESC
LIMIT $7::int OFFSET $6::int
`
type ListAdminDiagnosticsParams struct {
UserID pgtype.UUID
ClientID *string
Kind *string
FromTs pgtype.Timestamptz
ToTs pgtype.Timestamptz
Off int32
Lim int32
}
type ListAdminDiagnosticsRow struct {
ID pgtype.UUID
UserID pgtype.UUID
Username string
ClientID string
AppVersion *string
OsVersion *string
Kind string
Payload []byte
OccurredAt pgtype.Timestamptz
ReceivedAt pgtype.Timestamptz
}
// Admin timeline query. All filters are optional (NULL = no filter):
// account (user_id), device (client_id), category (kind), and the
// occurred_at window (from_ts/to_ts). Newest-first; the SPA reverses for
// a chronological timeline / export. Joined with users for the username
// so the view renders without a second round-trip.
func (q *Queries) ListAdminDiagnostics(ctx context.Context, arg ListAdminDiagnosticsParams) ([]ListAdminDiagnosticsRow, error) {
rows, err := q.db.Query(ctx, listAdminDiagnostics,
arg.UserID,
arg.ClientID,
arg.Kind,
arg.FromTs,
arg.ToTs,
arg.Off,
arg.Lim,
)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListAdminDiagnosticsRow
for rows.Next() {
var i ListAdminDiagnosticsRow
if err := rows.Scan(
&i.ID,
&i.UserID,
&i.Username,
&i.ClientID,
&i.AppVersion,
&i.OsVersion,
&i.Kind,
&i.Payload,
&i.OccurredAt,
&i.ReceivedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listDiagnosticDevices = `-- name: ListDiagnosticDevices :many
SELECT
de.client_id AS client_id,
de.user_id AS user_id,
u.username AS username,
max(de.app_version)::text AS app_version,
max(de.os_version)::text AS os_version,
max(de.occurred_at)::timestamptz AS last_seen,
count(*) AS event_count
FROM diagnostic_events de
JOIN users u ON u.id = de.user_id
WHERE ($1::uuid IS NULL OR de.user_id = $1::uuid)
GROUP BY de.client_id, de.user_id, u.username
ORDER BY last_seen DESC
`
type ListDiagnosticDevicesRow struct {
ClientID string
UserID pgtype.UUID
Username string
AppVersion string
OsVersion string
LastSeen pgtype.Timestamptz
EventCount int64
}
// Distinct devices that have reported, for the admin device filter +
// "who is currently reporting" overview. Optionally scoped to one account.
func (q *Queries) ListDiagnosticDevices(ctx context.Context, userID pgtype.UUID) ([]ListDiagnosticDevicesRow, error) {
rows, err := q.db.Query(ctx, listDiagnosticDevices, userID)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListDiagnosticDevicesRow
for rows.Next() {
var i ListDiagnosticDevicesRow
if err := rows.Scan(
&i.ClientID,
&i.UserID,
&i.Username,
&i.AppVersion,
&i.OsVersion,
&i.LastSeen,
&i.EventCount,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
+47 -6
View File
@@ -54,7 +54,7 @@ func (q *Queries) GetMostRecentPlaySessionForUser(ctx context.Context, userID pg
}
const getOpenPlayEventForUser = `-- name: GetOpenPlayEventForUser :one
SELECT id, user_id, track_id, session_id, started_at, ended_at, duration_played_ms, completion_ratio, was_skipped, client_id, session_vector_at_play, scrobbled_at, source FROM play_events
SELECT id, user_id, track_id, session_id, started_at, ended_at, duration_played_ms, completion_ratio, was_skipped, client_id, session_vector_at_play, scrobbled_at, source, pick_kind FROM play_events
WHERE user_id = $1 AND ended_at IS NULL
ORDER BY started_at DESC
LIMIT 1
@@ -79,12 +79,13 @@ func (q *Queries) GetOpenPlayEventForUser(ctx context.Context, userID pgtype.UUI
&i.SessionVectorAtPlay,
&i.ScrobbledAt,
&i.Source,
&i.PickKind,
)
return i, err
}
const getPlayEventByID = `-- name: GetPlayEventByID :one
SELECT id, user_id, track_id, session_id, started_at, ended_at, duration_played_ms, completion_ratio, was_skipped, client_id, session_vector_at_play, scrobbled_at, source FROM play_events WHERE id = $1
SELECT id, user_id, track_id, session_id, started_at, ended_at, duration_played_ms, completion_ratio, was_skipped, client_id, session_vector_at_play, scrobbled_at, source, pick_kind FROM play_events WHERE id = $1
`
func (q *Queries) GetPlayEventByID(ctx context.Context, id pgtype.UUID) (PlayEvent, error) {
@@ -104,15 +105,48 @@ func (q *Queries) GetPlayEventByID(ctx context.Context, id pgtype.UUID) (PlayEve
&i.SessionVectorAtPlay,
&i.ScrobbledAt,
&i.Source,
&i.PickKind,
)
return i, err
}
const getSystemPickKindForTrack = `-- name: GetSystemPickKindForTrack :one
SELECT pt.pick_kind
FROM playlist_tracks pt
JOIN playlists p ON p.id = pt.playlist_id
WHERE p.user_id = $1
AND p.system_variant = $2
AND pt.track_id = $3
LIMIT 1
`
type GetSystemPickKindForTrackParams struct {
UserID pgtype.UUID
SystemVariant *string
TrackID pgtype.UUID
}
// Looks a track up in the user's CURRENT snapshot of the given system
// variant and returns its pick_kind. Used at play-ingestion time to
// freeze provenance onto the play_event — snapshots rebuild daily, so
// attribution can't be reconstructed at read time (#1249/#1270). No
// row = track not in today's snapshot (caller stores NULL); a row with
// NULL pick_kind = the variant doesn't stamp (yet). songs_like_artist
// is non-singleton (up to 3 mixes/user); a track in two of them takes
// whichever LIMIT 1 hits — acceptable, tier stamps there describe the
// same eligibility ladder.
func (q *Queries) GetSystemPickKindForTrack(ctx context.Context, arg GetSystemPickKindForTrackParams) (*string, error) {
row := q.db.QueryRow(ctx, getSystemPickKindForTrack, arg.UserID, arg.SystemVariant, arg.TrackID)
var pick_kind *string
err := row.Scan(&pick_kind)
return pick_kind, err
}
const insertPlayEvent = `-- name: InsertPlayEvent :one
INSERT INTO play_events (
user_id, track_id, session_id, started_at, client_id, source
) VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id, user_id, track_id, session_id, started_at, ended_at, duration_played_ms, completion_ratio, was_skipped, client_id, session_vector_at_play, scrobbled_at, source
user_id, track_id, session_id, started_at, client_id, source, pick_kind
) VALUES ($1, $2, $3, $4, $5, $6, $7::text)
RETURNING id, user_id, track_id, session_id, started_at, ended_at, duration_played_ms, completion_ratio, was_skipped, client_id, session_vector_at_play, scrobbled_at, source, pick_kind
`
type InsertPlayEventParams struct {
@@ -122,8 +156,12 @@ type InsertPlayEventParams struct {
StartedAt pgtype.Timestamptz
ClientID *string
Source *string
PickKind *string
}
// pick_kind is non-NULL only for system-playlist plays whose track was
// found (with a stamped kind) in the user's live snapshot for that
// variant at ingestion time (#1249, generalized in #1270).
func (q *Queries) InsertPlayEvent(ctx context.Context, arg InsertPlayEventParams) (PlayEvent, error) {
row := q.db.QueryRow(ctx, insertPlayEvent,
arg.UserID,
@@ -132,6 +170,7 @@ func (q *Queries) InsertPlayEvent(ctx context.Context, arg InsertPlayEventParams
arg.StartedAt,
arg.ClientID,
arg.Source,
arg.PickKind,
)
var i PlayEvent
err := row.Scan(
@@ -148,6 +187,7 @@ func (q *Queries) InsertPlayEvent(ctx context.Context, arg InsertPlayEventParams
&i.SessionVectorAtPlay,
&i.ScrobbledAt,
&i.Source,
&i.PickKind,
)
return i, err
}
@@ -291,7 +331,7 @@ SET ended_at = $2,
completion_ratio = $4,
was_skipped = $5
WHERE id = $1
RETURNING id, user_id, track_id, session_id, started_at, ended_at, duration_played_ms, completion_ratio, was_skipped, client_id, session_vector_at_play, scrobbled_at, source
RETURNING id, user_id, track_id, session_id, started_at, ended_at, duration_played_ms, completion_ratio, was_skipped, client_id, session_vector_at_play, scrobbled_at, source, pick_kind
`
type UpdatePlayEventEndedParams struct {
@@ -328,6 +368,7 @@ func (q *Queries) UpdatePlayEventEnded(ctx context.Context, arg UpdatePlayEventE
&i.SessionVectorAtPlay,
&i.ScrobbledAt,
&i.Source,
&i.PickKind,
)
return i, err
}
+45
View File
@@ -265,6 +265,18 @@ type CoverArtSourcesMetum struct {
LastRegisteredProvidersHash string
}
type DiagnosticEvent struct {
ID pgtype.UUID
UserID pgtype.UUID
ClientID string
AppVersion *string
OsVersion *string
Kind string
Payload []byte
OccurredAt pgtype.Timestamptz
ReceivedAt pgtype.Timestamptz
}
type GeneralLike struct {
UserID pgtype.UUID
TrackID pgtype.UUID
@@ -371,6 +383,7 @@ type PlayEvent struct {
SessionVectorAtPlay []byte
ScrobbledAt pgtype.Timestamptz
Source *string
PickKind *string
}
type PlaySession struct {
@@ -421,6 +434,28 @@ type PlaylistTrack struct {
AlbumTitle string
DurationSec int32
AddedAt pgtype.Timestamptz
PickKind *string
}
type RecommendationTuningAudit struct {
ID int64
ChangedAt pgtype.Timestamptz
Scope string
Action string
Changes []byte
}
type RecommendationWeightProfile struct {
Profile string
BaseWeight float64
LikeBoost float64
RecencyWeight float64
SkipPenalty float64
JitterMagnitude float64
ContextWeight float64
SimilarityWeight float64
TasteWeight float64
UpdatedAt pgtype.Timestamptz
}
type RegistrationSetting struct {
@@ -511,6 +546,15 @@ type TasteProfileTag struct {
UpdatedAt pgtype.Timestamptz
}
type TasteTuning struct {
Singleton bool
HalfLifeDays float64
EngagementHardSkip float64
EngagementNeutral float64
EngagementFull float64
UpdatedAt pgtype.Timestamptz
}
type Track struct {
ID pgtype.UUID
Title string
@@ -552,6 +596,7 @@ type User struct {
Email *string
Timezone string
TimezoneUpdatedAt pgtype.Timestamptz
DebugModeEnabled bool
}
type UserInvite struct {
+13 -7
View File
@@ -12,7 +12,7 @@ import (
)
const appendPlaylistTrack = `-- name: AppendPlaylistTrack :one
INSERT INTO playlist_tracks (playlist_id, position, track_id, title, artist_name, album_title, duration_sec)
INSERT INTO playlist_tracks (playlist_id, position, track_id, title, artist_name, album_title, duration_sec, pick_kind)
SELECT
$1::uuid,
COALESCE((SELECT MAX(position) + 1 FROM playlist_tracks WHERE playlist_id = $1::uuid), 0),
@@ -20,24 +20,27 @@ SELECT
t.title,
artists.name,
albums.title,
(t.duration_ms / 1000)::integer
(t.duration_ms / 1000)::integer,
$2::text
FROM tracks t
JOIN albums ON albums.id = t.album_id
JOIN artists ON artists.id = t.artist_id
WHERE t.id = $2::uuid
RETURNING playlist_id, position, track_id, title, artist_name, album_title, duration_sec, added_at
WHERE t.id = $3::uuid
RETURNING playlist_id, position, track_id, title, artist_name, album_title, duration_sec, added_at, pick_kind
`
type AppendPlaylistTrackParams struct {
PlaylistID pgtype.UUID
PickKind *string
TrackID pgtype.UUID
}
// Inserts at the next available position. Snapshot fields are copied
// from the tracks/albums/artists join at insert time. tracks.duration_ms
// is converted to seconds for the snapshot.
// is converted to seconds for the snapshot. pick_kind is NULL for manual
// playlists; the For You builder stamps 'taste'/'fresh' (#1249).
func (q *Queries) AppendPlaylistTrack(ctx context.Context, arg AppendPlaylistTrackParams) (PlaylistTrack, error) {
row := q.db.QueryRow(ctx, appendPlaylistTrack, arg.PlaylistID, arg.TrackID)
row := q.db.QueryRow(ctx, appendPlaylistTrack, arg.PlaylistID, arg.PickKind, arg.TrackID)
var i PlaylistTrack
err := row.Scan(
&i.PlaylistID,
@@ -48,6 +51,7 @@ func (q *Queries) AppendPlaylistTrack(ctx context.Context, arg AppendPlaylistTra
&i.AlbumTitle,
&i.DurationSec,
&i.AddedAt,
&i.PickKind,
)
return i, err
}
@@ -257,7 +261,7 @@ func (q *Queries) ListAllPlaylistTracksForCollage(ctx context.Context, arg ListA
}
const listPlaylistTracks = `-- name: ListPlaylistTracks :many
SELECT pt.playlist_id, pt.position, pt.track_id, pt.title, pt.artist_name, pt.album_title, pt.duration_sec, pt.added_at,
SELECT pt.playlist_id, pt.position, pt.track_id, pt.title, pt.artist_name, pt.album_title, pt.duration_sec, pt.added_at, pt.pick_kind,
t.id AS live_track_id,
albums.id AS album_id,
artists.id AS artist_id
@@ -278,6 +282,7 @@ type ListPlaylistTracksRow struct {
AlbumTitle string
DurationSec int32
AddedAt pgtype.Timestamptz
PickKind *string
LiveTrackID pgtype.UUID
AlbumID pgtype.UUID
ArtistID pgtype.UUID
@@ -305,6 +310,7 @@ func (q *Queries) ListPlaylistTracks(ctx context.Context, playlistID pgtype.UUID
&i.AlbumTitle,
&i.DurationSec,
&i.AddedAt,
&i.PickKind,
&i.LiveTrackID,
&i.AlbumID,
&i.ArtistID,
+91 -12
View File
@@ -12,19 +12,17 @@ import (
)
const recommendationSourceMetricsForUser = `-- name: RecommendationSourceMetricsForUser :many
SELECT
pe.source,
pe.pick_kind,
count(*)::bigint AS plays,
count(*) FILTER (WHERE pe.was_skipped)::bigint AS skips,
COALESCE(
avg(pe.completion_ratio) FILTER (WHERE pe.completion_ratio IS NOT NULL),
0)::float8 AS avg_completion
count(pe.completion_ratio)::bigint AS completion_n,
COALESCE(avg(pe.completion_ratio), 0)::float8 AS avg_completion
FROM play_events pe
WHERE pe.user_id = $1
AND pe.source IS NOT NULL
AND pe.started_at > now() - ($2::float8 * INTERVAL '1 day')
GROUP BY pe.source
GROUP BY pe.source, pe.pick_kind
ORDER BY plays DESC
`
@@ -35,18 +33,17 @@ type RecommendationSourceMetricsForUserParams struct {
type RecommendationSourceMetricsForUserRow struct {
Source *string
PickKind *string
Plays int64
Skips int64
CompletionN int64
AvgCompletion float64
}
// Recommendation observability (#796 phase 4). Per-source play outcomes so the
// operator can see whether each recommendation surface is landing and tune the
// taste weights. Source is stamped on play_events when a play is launched from
// a system-playlist surface ('for_you' | 'discover' | the discovery mixes);
// NULL for library / radio / user-playlist plays, which are excluded here.
// $1 user_id, $2 window_days. plays/skips are counts; avg_completion is the
// mean completion ratio over plays that recorded one (0 when none did).
// mean completion ratio over the completion_n plays that recorded one.
// pick_kind splits For You plays into taste/fresh/unattributed (#1249);
// it is NULL for every other source, so those still group to one row.
func (q *Queries) RecommendationSourceMetricsForUser(ctx context.Context, arg RecommendationSourceMetricsForUserParams) ([]RecommendationSourceMetricsForUserRow, error) {
rows, err := q.db.Query(ctx, recommendationSourceMetricsForUser, arg.UserID, arg.Column2)
if err != nil {
@@ -58,8 +55,10 @@ func (q *Queries) RecommendationSourceMetricsForUser(ctx context.Context, arg Re
var i RecommendationSourceMetricsForUserRow
if err := rows.Scan(
&i.Source,
&i.PickKind,
&i.Plays,
&i.Skips,
&i.CompletionN,
&i.AvgCompletion,
); err != nil {
return nil, err
@@ -71,3 +70,83 @@ func (q *Queries) RecommendationSourceMetricsForUser(ctx context.Context, arg Re
}
return items, nil
}
const recommendationWeeklyTrends = `-- name: RecommendationWeeklyTrends :many
SELECT
date_trunc('week', pe.started_at)::date AS week_start,
pe.source,
count(*)::bigint AS plays,
count(*) FILTER (WHERE pe.was_skipped)::bigint AS skips,
count(pe.completion_ratio)::bigint AS completion_n,
COALESCE(avg(pe.completion_ratio), 0)::float8 AS avg_completion,
count(*) FILTER (WHERE tpa.artist_id IS NOT NULL)::bigint AS taste_hits
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
LEFT JOIN taste_profile_artists tpa
ON tpa.user_id = pe.user_id
AND tpa.artist_id = t.artist_id
AND tpa.weight > 0
WHERE pe.started_at > now() - ($1::int * INTERVAL '1 week')
GROUP BY 1, 2
ORDER BY 1, 2
`
type RecommendationWeeklyTrendsRow struct {
WeekStart pgtype.Date
Source *string
Plays int64
Skips int64
CompletionN int64
AvgCompletion float64
TasteHits int64
}
// Recommendation observability (#796 phase 4). Per-source play outcomes so the
// operator can see whether each recommendation surface is landing and tune the
// taste weights. Source is stamped on play_events when a play is launched from
// a recommendation surface; NULL means the user picked the track manually —
// those rows are INCLUDED here as the baseline control group the surfaces are
// judged against (milestone #127: delta-vs-baseline is what makes the numbers
// actionable). Raw source strings are bucketed into stable surface families in
// the Go handler; completion_n is carried so family merges can weight
// avg_completion correctly.
// Weekly per-source outcome series for the tuning lab's trend view
// (#1251). Aggregated across ALL users: the tuning knobs are global,
// so judging a knob turn needs global outcomes — rows carry rates
// only, no track or user identity. NULL-source (manual) rows are
// included as the baseline family.
//
// taste_hits counts plays whose track's artist has a positive weight
// in that user's CURRENT taste profile — the "cheap recompute" option:
// retroactive over the whole window, at the cost of drift (the profile
// is today's, the play may be weeks old). Good enough to read whether
// a surface is feeding taste-fitting tracks.
// $1 window in weeks.
func (q *Queries) RecommendationWeeklyTrends(ctx context.Context, weeks int32) ([]RecommendationWeeklyTrendsRow, error) {
rows, err := q.db.Query(ctx, recommendationWeeklyTrends, weeks)
if err != nil {
return nil, err
}
defer rows.Close()
var items []RecommendationWeeklyTrendsRow
for rows.Next() {
var i RecommendationWeeklyTrendsRow
if err := rows.Scan(
&i.WeekStart,
&i.Source,
&i.Plays,
&i.Skips,
&i.CompletionN,
&i.AvgCompletion,
&i.TasteHits,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
@@ -0,0 +1,272 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.31.1
// source: recommendation_tuning.sql
package dbq
import (
"context"
)
const getTasteTuning = `-- name: GetTasteTuning :one
SELECT singleton, half_life_days, engagement_hard_skip, engagement_neutral, engagement_full, updated_at FROM taste_tuning WHERE singleton = true
`
func (q *Queries) GetTasteTuning(ctx context.Context) (TasteTuning, error) {
row := q.db.QueryRow(ctx, getTasteTuning)
var i TasteTuning
err := row.Scan(
&i.Singleton,
&i.HalfLifeDays,
&i.EngagementHardSkip,
&i.EngagementNeutral,
&i.EngagementFull,
&i.UpdatedAt,
)
return i, err
}
const insertTuningAudit = `-- name: InsertTuningAudit :exec
INSERT INTO recommendation_tuning_audit (scope, action, changes)
VALUES ($1, $2, $3)
`
type InsertTuningAuditParams struct {
Scope string
Action string
Changes []byte
}
// changes is a jsonb array of {field, old, new} objects.
func (q *Queries) InsertTuningAudit(ctx context.Context, arg InsertTuningAuditParams) error {
_, err := q.db.Exec(ctx, insertTuningAudit, arg.Scope, arg.Action, arg.Changes)
return err
}
const listTuningAudit = `-- name: ListTuningAudit :many
SELECT id, changed_at, scope, action, changes
FROM recommendation_tuning_audit
ORDER BY changed_at DESC, id DESC
LIMIT $1
`
// Newest first; consumed by the metrics trend view (#1251) to annotate
// knob turns on the timeline.
func (q *Queries) ListTuningAudit(ctx context.Context, limit int32) ([]RecommendationTuningAudit, error) {
rows, err := q.db.Query(ctx, listTuningAudit, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []RecommendationTuningAudit
for rows.Next() {
var i RecommendationTuningAudit
if err := rows.Scan(
&i.ID,
&i.ChangedAt,
&i.Scope,
&i.Action,
&i.Changes,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listWeightProfiles = `-- name: ListWeightProfiles :many
SELECT profile, base_weight, like_boost, recency_weight, skip_penalty, jitter_magnitude, context_weight, similarity_weight, taste_weight, updated_at FROM recommendation_weight_profiles ORDER BY profile
`
func (q *Queries) ListWeightProfiles(ctx context.Context) ([]RecommendationWeightProfile, error) {
rows, err := q.db.Query(ctx, listWeightProfiles)
if err != nil {
return nil, err
}
defer rows.Close()
var items []RecommendationWeightProfile
for rows.Next() {
var i RecommendationWeightProfile
if err := rows.Scan(
&i.Profile,
&i.BaseWeight,
&i.LikeBoost,
&i.RecencyWeight,
&i.SkipPenalty,
&i.JitterMagnitude,
&i.ContextWeight,
&i.SimilarityWeight,
&i.TasteWeight,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const updateTasteTuning = `-- name: UpdateTasteTuning :one
UPDATE taste_tuning
SET half_life_days = $1,
engagement_hard_skip = $2,
engagement_neutral = $3,
engagement_full = $4,
updated_at = now()
WHERE singleton = true
RETURNING singleton, half_life_days, engagement_hard_skip, engagement_neutral, engagement_full, updated_at
`
type UpdateTasteTuningParams struct {
HalfLifeDays float64
EngagementHardSkip float64
EngagementNeutral float64
EngagementFull float64
}
func (q *Queries) UpdateTasteTuning(ctx context.Context, arg UpdateTasteTuningParams) (TasteTuning, error) {
row := q.db.QueryRow(ctx, updateTasteTuning,
arg.HalfLifeDays,
arg.EngagementHardSkip,
arg.EngagementNeutral,
arg.EngagementFull,
)
var i TasteTuning
err := row.Scan(
&i.Singleton,
&i.HalfLifeDays,
&i.EngagementHardSkip,
&i.EngagementNeutral,
&i.EngagementFull,
&i.UpdatedAt,
)
return i, err
}
const updateWeightProfile = `-- name: UpdateWeightProfile :one
UPDATE recommendation_weight_profiles
SET base_weight = $2,
like_boost = $3,
recency_weight = $4,
skip_penalty = $5,
jitter_magnitude = $6,
context_weight = $7,
similarity_weight = $8,
taste_weight = $9,
updated_at = now()
WHERE profile = $1
RETURNING profile, base_weight, like_boost, recency_weight, skip_penalty, jitter_magnitude, context_weight, similarity_weight, taste_weight, updated_at
`
type UpdateWeightProfileParams struct {
Profile string
BaseWeight float64
LikeBoost float64
RecencyWeight float64
SkipPenalty float64
JitterMagnitude float64
ContextWeight float64
SimilarityWeight float64
TasteWeight float64
}
func (q *Queries) UpdateWeightProfile(ctx context.Context, arg UpdateWeightProfileParams) (RecommendationWeightProfile, error) {
row := q.db.QueryRow(ctx, updateWeightProfile,
arg.Profile,
arg.BaseWeight,
arg.LikeBoost,
arg.RecencyWeight,
arg.SkipPenalty,
arg.JitterMagnitude,
arg.ContextWeight,
arg.SimilarityWeight,
arg.TasteWeight,
)
var i RecommendationWeightProfile
err := row.Scan(
&i.Profile,
&i.BaseWeight,
&i.LikeBoost,
&i.RecencyWeight,
&i.SkipPenalty,
&i.JitterMagnitude,
&i.ContextWeight,
&i.SimilarityWeight,
&i.TasteWeight,
&i.UpdatedAt,
)
return i, err
}
const upsertTasteTuningDefaults = `-- name: UpsertTasteTuningDefaults :exec
INSERT INTO taste_tuning (
singleton, half_life_days, engagement_hard_skip,
engagement_neutral, engagement_full
) VALUES (true, $1, $2, $3, $4)
ON CONFLICT (singleton) DO NOTHING
`
type UpsertTasteTuningDefaultsParams struct {
HalfLifeDays float64
EngagementHardSkip float64
EngagementNeutral float64
EngagementFull float64
}
func (q *Queries) UpsertTasteTuningDefaults(ctx context.Context, arg UpsertTasteTuningDefaultsParams) error {
_, err := q.db.Exec(ctx, upsertTasteTuningDefaults,
arg.HalfLifeDays,
arg.EngagementHardSkip,
arg.EngagementNeutral,
arg.EngagementFull,
)
return err
}
const upsertWeightProfileDefaults = `-- name: UpsertWeightProfileDefaults :exec
INSERT INTO recommendation_weight_profiles (
profile, base_weight, like_boost, recency_weight, skip_penalty,
jitter_magnitude, context_weight, similarity_weight, taste_weight
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
ON CONFLICT (profile) DO NOTHING
`
type UpsertWeightProfileDefaultsParams struct {
Profile string
BaseWeight float64
LikeBoost float64
RecencyWeight float64
SkipPenalty float64
JitterMagnitude float64
ContextWeight float64
SimilarityWeight float64
TasteWeight float64
}
// Recommendation tuning lab queries (#1250). Seeding happens via the
// recsettings boot reconcile; shipped defaults live in Go only.
// Boot reconcile: insert the shipped defaults for a profile if the row
// doesn't exist yet. Never overwrites operator-tuned values.
func (q *Queries) UpsertWeightProfileDefaults(ctx context.Context, arg UpsertWeightProfileDefaultsParams) error {
_, err := q.db.Exec(ctx, upsertWeightProfileDefaults,
arg.Profile,
arg.BaseWeight,
arg.LikeBoost,
arg.RecencyWeight,
arg.SkipPenalty,
arg.JitterMagnitude,
arg.ContextWeight,
arg.SimilarityWeight,
arg.TasteWeight,
)
return err
}
+170 -70
View File
@@ -27,12 +27,20 @@ play_counts AS (
FROM play_events
WHERE user_id = $1 AND was_skipped = false
GROUP BY track_id
),
skip_counts AS (
SELECT track_id, COUNT(*) AS c
FROM play_events
WHERE user_id = $1 AND was_skipped = true
GROUP BY track_id
)
SELECT t.id, t.album_id, t.artist_id
FROM tracks t
JOIN affinity_artists aa ON aa.artist_id = t.artist_id
LEFT JOIN play_counts pc ON pc.track_id = t.id
LEFT JOIN skip_counts sc ON sc.track_id = t.id
WHERE COALESCE(pc.c, 0) <= 2
AND COALESCE(sc.c, 0) < 2
AND NOT EXISTS (
SELECT 1 FROM lidarr_quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
@@ -60,6 +68,11 @@ type ListDeepCutsTracksRow struct {
// gives per-play variety on top.
// #419 Deep Cuts: low-play tracks (<=2 plays) from artists the user
// has liked OR played heavily (>=5 non-skip plays across the artist).
// Tracks the user has skipped twice or more don't qualify (#1257):
// eligibility used to count only unskipped plays, so the most actively
// rejected tracks read as "barely heard" and kept being re-offered.
// Threshold 2 so a single accidental skip doesn't banish a track;
// passive-signal only (no dislike UI, rule #101).
// $1 user_id, $2 date string.
func (q *Queries) ListDeepCutsTracks(ctx context.Context, arg ListDeepCutsTracksParams) ([]ListDeepCutsTracksRow, error) {
rows, err := q.db.Query(ctx, listDeepCutsTracks, arg.UserID, arg.Column2)
@@ -82,31 +95,49 @@ func (q *Queries) ListDeepCutsTracks(ctx context.Context, arg ListDeepCutsTracks
}
const listFirstListensTracks = `-- name: ListFirstListensTracks :many
WITH heard_albums AS (
SELECT DISTINCT t.album_id
FROM play_events pe JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1
WITH attempted AS (
SELECT DISTINCT pe.track_id
FROM play_events pe
WHERE pe.user_id = $1 AND pe.duration_played_ms >= 30000
),
played_artists AS (
album_attempts AS (
SELECT t.album_id,
COUNT(a.track_id) AS attempted_count,
COUNT(*) AS track_count
FROM tracks t
LEFT JOIN attempted a ON a.track_id = t.id
GROUP BY t.album_id
),
attempted_artists AS (
SELECT DISTINCT t.artist_id
FROM play_events pe JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1
FROM attempted a JOIN tracks t ON t.id = a.track_id
),
albums_tiered AS (
SELECT aa.album_id,
CASE
WHEN aa.attempted_count = 0 THEN 1
WHEN aa.attempted_count::float / GREATEST(aa.track_count, 1) <= 0.25 THEN 2
END AS tier
FROM album_attempts aa
)
SELECT t.id, t.album_id, t.artist_id
SELECT t.id, t.album_id, t.artist_id, alt.tier::int AS tier
FROM tracks t
JOIN albums al ON al.id = t.album_id
WHERE NOT EXISTS (SELECT 1 FROM heard_albums h WHERE h.album_id = al.id)
JOIN albums_tiered alt ON alt.album_id = al.id
WHERE alt.tier IS NOT NULL
AND NOT EXISTS (SELECT 1 FROM attempted a WHERE a.track_id = t.id)
AND NOT EXISTS (
SELECT 1 FROM lidarr_quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
)
ORDER BY
alt.tier,
(CASE
WHEN EXISTS (
SELECT 1 FROM general_likes_artists gla
WHERE gla.user_id = $1 AND gla.artist_id = al.artist_id
) THEN 0
WHEN al.artist_id IN (SELECT artist_id FROM played_artists) THEN 1
WHEN al.artist_id IN (SELECT artist_id FROM attempted_artists) THEN 1
ELSE 2
END),
al.id, t.disc_number NULLS FIRST, t.track_number NULLS FIRST
@@ -117,12 +148,25 @@ type ListFirstListensTracksRow struct {
ID pgtype.UUID
AlbumID pgtype.UUID
ArtistID pgtype.UUID
Tier int32
}
// #423 First Listens: albums the user has never played any track of.
// Tiered: liked-artist albums first, then played-artist albums, then
// the rest — album-coherent within each tier. Not diversity-capped
// (whole-album discovery). $1 user_id.
// #423 First Listens: songs the user has never heard or even
// attempted. "Attempted" is track-level with a >=30s listen threshold
// (#1268, operator decision 2026-07-03) — the old version disqualified
// a whole album on ANY play_event, so a 2-second accidental brush
// banished it permanently.
//
// tier 1 the exact desire: albums with ZERO attempted tracks
// tier 2 step back: barely-attempted albums (<=25% of tracks
// reached 30s) — brushed, never explored. The already-
// attempted tracks themselves are excluded; they're not
// first listens.
//
// Within each tier, affinity-ordered as before: liked artist, then
// attempted-played artist (>=30s — skip-only contact isn't trust),
// then the rest; album-coherent. The tier lands on
// playlist_tracks.pick_kind via the producer. $1 user_id.
func (q *Queries) ListFirstListensTracks(ctx context.Context, userID pgtype.UUID) ([]ListFirstListensTracksRow, error) {
rows, err := q.db.Query(ctx, listFirstListensTracks, userID)
if err != nil {
@@ -132,7 +176,12 @@ func (q *Queries) ListFirstListensTracks(ctx context.Context, userID pgtype.UUID
var items []ListFirstListensTracksRow
for rows.Next() {
var i ListFirstListensTracksRow
if err := rows.Scan(&i.ID, &i.AlbumID, &i.ArtistID); err != nil {
if err := rows.Scan(
&i.ID,
&i.AlbumID,
&i.ArtistID,
&i.Tier,
); err != nil {
return nil, err
}
items = append(items, i)
@@ -144,39 +193,76 @@ func (q *Queries) ListFirstListensTracks(ctx context.Context, userID pgtype.UUID
}
const listNewForYouTracks = `-- name: ListNewForYouTracks :many
WITH affinity_artists AS (
SELECT artist_id FROM general_likes_artists WHERE user_id = $1
WITH attempted AS (
SELECT DISTINCT pe.track_id
FROM play_events pe
WHERE pe.user_id = $1 AND pe.duration_played_ms >= 30000
),
consumed_albums AS (
SELECT DISTINCT t.album_id
FROM attempted a JOIN tracks t ON t.id = a.track_id
),
affinity_artists AS (
SELECT gla.artist_id FROM general_likes_artists gla WHERE gla.user_id = $1
UNION
SELECT t.artist_id
FROM play_events pe JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1 AND pe.was_skipped = false
FROM attempted a JOIN tracks t ON t.id = a.track_id
GROUP BY t.artist_id
HAVING COUNT(*) >= 3
),
albums_tiered AS (
SELECT al.id AS album_id, al.created_at,
CASE
WHEN al.created_at >= now() - interval '30 days'
AND al.artist_id IN (SELECT artist_id FROM affinity_artists) THEN 1
WHEN al.artist_id IN (SELECT artist_id FROM affinity_artists) THEN 2
ELSE 3
END AS tier
FROM albums al
WHERE al.created_at >= now() - interval '90 days'
AND al.id NOT IN (SELECT album_id FROM consumed_albums)
)
SELECT t.id, t.album_id, t.artist_id
SELECT t.id, t.album_id, t.artist_id, alt.tier::int AS tier
FROM tracks t
JOIN albums al ON al.id = t.album_id
JOIN affinity_artists aa ON aa.artist_id = al.artist_id
WHERE al.created_at >= now() - interval '30 days'
AND NOT EXISTS (
JOIN albums_tiered alt ON alt.album_id = t.album_id
WHERE NOT EXISTS (
SELECT 1 FROM lidarr_quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
)
ORDER BY al.created_at DESC, t.disc_number NULLS FIRST, t.track_number NULLS FIRST
LIMIT 200
ORDER BY alt.tier, alt.created_at DESC, t.album_id,
t.disc_number NULLS FIRST, t.track_number NULLS FIRST
LIMIT 300
`
type ListNewForYouTracksRow struct {
ID pgtype.UUID
AlbumID pgtype.UUID
ArtistID pgtype.UUID
Tier int32
}
// #421 New for you: tracks from albums added in the last 30 days
// whose artist the user has liked OR played (>=3 non-skip). Album-
// coherent (newest album first, then disc/track) — the producer does
// NOT diversity-cap these; they're meant as whole-album discovery.
// $1 user_id.
// #421 New for you, tiered per project rule #131 (#1267). The old
// single hard rule (added <30d AND direct affinity) had two failure
// modes with one root: no notion of consumption — the album you
// devoured in week one crowded the mix for three more weeks, and
// after a quiet month nothing qualified at all.
//
// "Consumed" = any track attempted for >=30s (the First Listens
// threshold, #1268): once you've meaningfully engaged, the album is
// no longer news. Applies to every tier — played albums leave the
// mix the next build.
//
// tier 1 the exact desire: unconsumed albums added <30d by
// direct-affinity artists (liked, or >=3 attempted plays)
// tier 2 step back a little: unconsumed affinity albums from the
// wider 30-90d window — added while you weren't looking
// tier 3 step back more: any unconsumed album added <90d
// regardless of affinity, newest first
//
// Album-coherent within tiers (newest album first, then disc/track);
// the producer fills top-down and rotates within tier blocks. The
// tier lands on playlist_tracks.pick_kind so metrics can price what
// each step-back trades away. $1 user_id.
func (q *Queries) ListNewForYouTracks(ctx context.Context, userID pgtype.UUID) ([]ListNewForYouTracksRow, error) {
rows, err := q.db.Query(ctx, listNewForYouTracks, userID)
if err != nil {
@@ -186,7 +272,12 @@ func (q *Queries) ListNewForYouTracks(ctx context.Context, userID pgtype.UUID) (
var items []ListNewForYouTracksRow
for rows.Next() {
var i ListNewForYouTracksRow
if err := rows.Scan(&i.ID, &i.AlbumID, &i.ArtistID); err != nil {
if err := rows.Scan(
&i.ID,
&i.AlbumID,
&i.ArtistID,
&i.Tier,
); err != nil {
return nil, err
}
items = append(items, i)
@@ -203,7 +294,10 @@ WITH windowed AS (
FROM play_events
WHERE user_id = $1 AND was_skipped = false
AND started_at < now() - interval '30 days'
AND ABS(EXTRACT(DOY FROM started_at) - EXTRACT(DOY FROM now())) <= 10
AND LEAST(
ABS(EXTRACT(DOY FROM started_at) - EXTRACT(DOY FROM $2::date)),
365 - ABS(EXTRACT(DOY FROM started_at) - EXTRACT(DOY FROM $2::date))
) <= 10
GROUP BY track_id
)
SELECT t.id, t.album_id, t.artist_id
@@ -235,6 +329,13 @@ type ListOnThisDayTracksRow struct {
// and window ±7→±10 so it surfaces on a months-old library instead
// of needing a full year of history; still skips cleanly (no rows →
// no playlist) when there's no qualifying history yet.
// Day-of-year distance is circular (#1256): plain ABS made Dec 28 vs
// Jan 3 read as 359 days apart, silently gutting the window for ~3
// weeks around every New Year — precisely when holiday nostalgia is
// the point. LEAST(d, 365-d) wraps the boundary; leap-year drift of
// ±1 is absorbed by the ±10 window. The build date ($2, already here
// for the md5 rotation) anchors "today" instead of now() so the
// window is testable and consistent with the mix's daily determinism.
// $1 user_id, $2 date string.
func (q *Queries) ListOnThisDayTracks(ctx context.Context, arg ListOnThisDayTracksParams) ([]ListOnThisDayTracksRow, error) {
rows, err := q.db.Query(ctx, listOnThisDayTracks, arg.UserID, arg.Column2)
@@ -262,37 +363,32 @@ WITH stats AS (
FROM play_events pe
WHERE pe.user_id = $1 AND pe.was_skipped = false
GROUP BY pe.track_id
),
deep AS (
SELECT t.id, t.album_id, t.artist_id, s.c, 0 AS tier
FROM tracks t
JOIN stats s ON s.track_id = t.id
WHERE s.c >= 5
AND s.last_at <= now() - interval '6 months'
AND NOT EXISTS (
SELECT 1 FROM lidarr_quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
)
),
shallow AS (
SELECT t.id, t.album_id, t.artist_id, s.c, 1 AS tier
FROM tracks t
JOIN stats s ON s.track_id = t.id
WHERE s.c >= 5
AND s.last_at <= now() - interval '30 days'
AND NOT EXISTS (
SELECT 1 FROM lidarr_quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
)
)
SELECT id, album_id, artist_id
FROM (
SELECT id, album_id, artist_id, c, tier FROM deep
UNION ALL
SELECT id, album_id, artist_id, c, tier FROM shallow
WHERE NOT EXISTS (SELECT 1 FROM deep)
) u
ORDER BY tier, c DESC, id
SELECT t.id, t.album_id, t.artist_id
FROM tracks t
JOIN stats s ON s.track_id = t.id
WHERE s.c >= 3
AND s.last_at <= now() - interval '30 days'
AND NOT EXISTS (
SELECT 1 FROM lidarr_quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
)
-- Declined-resurfacing cooldown (#1258): a skip on a
-- rediscover-sourced play is the user explicitly saying "I've moved
-- on" to the exact invitation this mix extends — such tracks sit
-- out ~90 days instead of re-qualifying the next day. Passive
-- signal only (rule #101).
AND NOT EXISTS (
SELECT 1 FROM play_events rp
WHERE rp.user_id = $1
AND rp.track_id = t.id
AND rp.source = 'rediscover'
AND rp.was_skipped = true
AND rp.started_at > now() - interval '90 days'
)
ORDER BY (s.last_at <= now() - interval '6 months') DESC,
(s.c >= 5) DESC,
s.c DESC, t.id
LIMIT 200
`
@@ -302,13 +398,17 @@ type ListRediscoverTracksRow struct {
ArtistID pgtype.UUID
}
// #420 Rediscover: tracks the user played a lot (>=5 non-skip) but
// has drifted away from. Tiered so a young library still gets a mix:
//
// tier 0 not played in the last 6 months (true rediscovery)
// tier 1 (only if tier 0 empty) not played in the last 30 days
//
// Ordered by historical affection. $1 user_id.
// #420 Rediscover: tracks the user played a lot but has drifted away
// from. One blended pool (issue #1246: the old two-tier UNION was
// all-or-nothing — a single ">=6 months cold" row suppressed the whole
// ">=30 days" tier, which is how a one-song playlist shipped; the 6mo
// tier was also a strict subset of the 30d tier). Eligibility is >=3
// non-skip plays and >=30 days cold; the bar sits at 3 rather than 5
// so a weeks-old library still fields a pool — on young histories the
// >=5-play tracks are precisely the ones still in rotation. Ordering
// prefers true rediscoveries (>=6 months cold), then strong affection
// (>=5 plays), then raw play count; the producer's minimum floor
// decides whether the pool is big enough to ship at all. $1 user_id.
func (q *Queries) ListRediscoverTracks(ctx context.Context, userID pgtype.UUID) ([]ListRediscoverTracksRow, error) {
rows, err := q.db.Query(ctx, listRediscoverTracks, userID)
if err != nil {
+75 -19
View File
@@ -290,9 +290,13 @@ func (q *Queries) ListPlaylistsByUserAndKind(ctx context.Context, arg ListPlayli
}
const pickSeedArtists = `-- name: PickSeedArtists :many
WITH plays AS (
WITH liked AS (
SELECT gla.artist_id FROM general_likes_artists gla WHERE gla.user_id = $1
),
recent7 AS (
SELECT t.artist_id,
COUNT(*) FILTER (WHERE pe.was_skipped = false) AS play_count
COUNT(*) FILTER (WHERE pe.was_skipped = false) AS play_count,
0 AS tier
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1
@@ -300,27 +304,79 @@ WITH plays AS (
AND t.artist_id IS NOT NULL
GROUP BY t.artist_id
),
liked AS (
SELECT artist_id FROM general_likes_artists WHERE user_id = $1
recent30 AS (
SELECT t.artist_id,
COUNT(*) FILTER (WHERE pe.was_skipped = false) AS play_count,
1 AS tier
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1
AND pe.started_at > now() - INTERVAL '30 days'
AND t.artist_id IS NOT NULL
GROUP BY t.artist_id
),
alltime AS (
SELECT t.artist_id,
COUNT(*) FILTER (WHERE pe.was_skipped = false) AS play_count,
2 AS tier
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1
AND t.artist_id IS NOT NULL
GROUP BY t.artist_id
),
likedonly AS (
SELECT l.artist_id, 0::bigint AS play_count, 3 AS tier
FROM liked l
),
chosen AS (
SELECT artist_id, play_count, tier FROM recent7
UNION ALL
SELECT artist_id, play_count, tier FROM recent30
WHERE NOT EXISTS (SELECT 1 FROM recent7)
UNION ALL
SELECT artist_id, play_count, tier FROM alltime
WHERE NOT EXISTS (SELECT 1 FROM recent7)
AND NOT EXISTS (SELECT 1 FROM recent30)
UNION ALL
SELECT artist_id, play_count, tier FROM likedonly
WHERE NOT EXISTS (SELECT 1 FROM recent7)
AND NOT EXISTS (SELECT 1 FROM recent30)
AND NOT EXISTS (SELECT 1 FROM alltime)
)
SELECT p.artist_id,
(p.play_count + CASE WHEN l.artist_id IS NOT NULL THEN 5 ELSE 0 END)::bigint AS score
FROM plays p
LEFT JOIN liked l ON l.artist_id = p.artist_id
ORDER BY score DESC, p.artist_id
SELECT c.artist_id,
(c.play_count + CASE WHEN l.artist_id IS NOT NULL THEN 5 ELSE 0 END)::bigint AS score,
c.tier::int AS tier
FROM chosen c
LEFT JOIN liked l ON l.artist_id = c.artist_id
ORDER BY score DESC, c.artist_id
LIMIT 5
`
type PickSeedArtistsRow struct {
ArtistID pgtype.UUID
Score int64
Tier int32
}
// Top-5 most-engaged distinct artist candidates in the user's last 7
// days. The Go-side picker (pickSeedArtistsForDay) shuffles these
// daily-deterministically and takes the first 3 so the set of
// "Songs like X" mixes rotates day-to-day.
// Score = unskipped-play count + 5 if user has liked the artist.
// Top-5 most-engaged distinct artist candidates, tiered so the
// "Songs like X" mixes never silently vanish (#1255): the old hard
// 7-day window emptied the seed pool after a quiet week, and the
// daily atomic-replace build then deleted every existing mix until
// the user played something again. Same fallback shape as
// PickTopPlayedTracksForUser (For You's seeds):
//
// tier 0 engagement in the last 7 days
// tier 1 (only if tier 0 empty) last 30 days
// tier 2 (only if tiers 0+1 empty) all-time
// tier 3 (only if 0-2 empty) liked artists with no play history
//
// All returned rows share one tier; produceSeedMixes maps it onto the
// rule-#131 pick-kind ladder and stamps the built tracks, so metrics
// can compare mixes seeded from fresh vs stale engagement.
// Score = unskipped-play count + 5 if user has liked the artist. The
// Go-side picker (pickSeedArtistsForDay) shuffles the 5
// daily-deterministically and takes 3 so the mix set rotates.
func (q *Queries) PickSeedArtists(ctx context.Context, userID pgtype.UUID) ([]PickSeedArtistsRow, error) {
rows, err := q.db.Query(ctx, pickSeedArtists, userID)
if err != nil {
@@ -330,7 +386,7 @@ func (q *Queries) PickSeedArtists(ctx context.Context, userID pgtype.UUID) ([]Pi
var items []PickSeedArtistsRow
for rows.Next() {
var i PickSeedArtistsRow
if err := rows.Scan(&i.ArtistID, &i.Score); err != nil {
if err := rows.Scan(&i.ArtistID, &i.Score, &i.Tier); err != nil {
return nil, err
}
items = append(items, i)
@@ -457,10 +513,10 @@ SELECT id
// tier 2 (only if tiers 0+1 empty) liked tracks
//
// Returns up to 5 ids; tie-break by track_id for determinism. The
// Go-side picker (pickForYouSeedForDay) rotates one per day via
// userIDHash. Widened from a hard 7-day window, which made For-You
// disappear after a week of not listening and never recover on a
// self-hosted library with sparse history.
// Go-side picker (pickDailySeeds) draws the day's seeds from these.
// Widened from a hard 7-day window, which made For-You disappear
// after a week of not listening and never recover on a self-hosted
// library with sparse history.
func (q *Queries) PickTopPlayedTracksForUser(ctx context.Context, userID pgtype.UUID) ([]pgtype.UUID, error) {
rows, err := q.db.Query(ctx, pickTopPlayedTracksForUser, userID)
if err != nil {
+64 -12
View File
@@ -54,7 +54,7 @@ func (q *Queries) CountUsers(ctx context.Context) (int64, error) {
const createUser = `-- name: CreateUser :one
INSERT INTO users (username, password_hash, api_token, is_admin, display_name)
VALUES ($1, $2, $3, $4, $5)
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled
`
type CreateUserParams struct {
@@ -89,6 +89,7 @@ func (q *Queries) CreateUser(ctx context.Context, arg CreateUserParams) (User, e
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
)
return i, err
}
@@ -96,7 +97,7 @@ func (q *Queries) CreateUser(ctx context.Context, arg CreateUserParams) (User, e
const createUserAdmin = `-- name: CreateUserAdmin :one
INSERT INTO users (username, password_hash, api_token, is_admin, display_name)
VALUES ($1, $2, $3, $4, $5)
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled
`
type CreateUserAdminParams struct {
@@ -134,6 +135,7 @@ func (q *Queries) CreateUserAdmin(ctx context.Context, arg CreateUserAdminParams
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
)
return i, err
}
@@ -145,7 +147,7 @@ VALUES (
(SELECT NOT EXISTS (SELECT 1 FROM users)),
$4
)
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled
`
type CreateUserFirstAdminRaceParams struct {
@@ -191,6 +193,7 @@ func (q *Queries) CreateUserFirstAdminRace(ctx context.Context, arg CreateUserFi
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
)
return i, err
}
@@ -237,7 +240,7 @@ func (q *Queries) GetListenBrainzConfig(ctx context.Context, id pgtype.UUID) (Ge
}
const getUserByAPIToken = `-- name: GetUserByAPIToken :one
SELECT id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at FROM users WHERE api_token = $1
SELECT id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled FROM users WHERE api_token = $1
`
func (q *Queries) GetUserByAPIToken(ctx context.Context, apiToken string) (User, error) {
@@ -258,12 +261,13 @@ func (q *Queries) GetUserByAPIToken(ctx context.Context, apiToken string) (User,
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
)
return i, err
}
const getUserByEmail = `-- name: GetUserByEmail :one
SELECT id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at FROM users WHERE lower(email) = lower($1)
SELECT id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled FROM users WHERE lower(email) = lower($1)
`
// Used by forgot-password lookup. Lowercase comparison both sides
@@ -287,12 +291,13 @@ func (q *Queries) GetUserByEmail(ctx context.Context, lower string) (User, error
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
)
return i, err
}
const getUserByID = `-- name: GetUserByID :one
SELECT id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at FROM users WHERE id = $1
SELECT id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled FROM users WHERE id = $1
`
func (q *Queries) GetUserByID(ctx context.Context, id pgtype.UUID) (User, error) {
@@ -313,12 +318,13 @@ func (q *Queries) GetUserByID(ctx context.Context, id pgtype.UUID) (User, error)
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
)
return i, err
}
const getUserByUsername = `-- name: GetUserByUsername :one
SELECT id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at FROM users WHERE username = $1
SELECT id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled FROM users WHERE username = $1
`
func (q *Queries) GetUserByUsername(ctx context.Context, username string) (User, error) {
@@ -339,6 +345,7 @@ func (q *Queries) GetUserByUsername(ctx context.Context, username string) (User,
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
)
return i, err
}
@@ -382,7 +389,8 @@ func (q *Queries) ListActiveUsersWithTimezones(ctx context.Context) ([]ListActiv
}
const listUsers = `-- name: ListUsers :many
SELECT id, username, display_name, is_admin, auto_approve_requests, created_at
SELECT id, username, display_name, is_admin, auto_approve_requests,
debug_mode_enabled, created_at
FROM users
ORDER BY created_at DESC
`
@@ -393,6 +401,7 @@ type ListUsersRow struct {
DisplayName *string
IsAdmin bool
AutoApproveRequests bool
DebugModeEnabled bool
CreatedAt pgtype.Timestamptz
}
@@ -412,6 +421,7 @@ func (q *Queries) ListUsers(ctx context.Context) ([]ListUsersRow, error) {
&i.DisplayName,
&i.IsAdmin,
&i.AutoApproveRequests,
&i.DebugModeEnabled,
&i.CreatedAt,
); err != nil {
return nil, err
@@ -426,7 +436,7 @@ func (q *Queries) ListUsers(ctx context.Context) ([]ListUsersRow, error) {
const regenerateApiToken = `-- name: RegenerateApiToken :one
UPDATE users SET api_token = $2 WHERE id = $1
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled
`
type RegenerateApiTokenParams struct {
@@ -454,6 +464,7 @@ func (q *Queries) RegenerateApiToken(ctx context.Context, arg RegenerateApiToken
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
)
return i, err
}
@@ -478,6 +489,44 @@ func (q *Queries) ResetUserPassword(ctx context.Context, arg ResetUserPasswordPa
return err
}
const setDebugMode = `-- name: SetDebugMode :one
UPDATE users
SET debug_mode_enabled = $2
WHERE id = $1
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled
`
type SetDebugModeParams struct {
ID pgtype.UUID
DebugModeEnabled bool
}
// Toggle the per-account diagnostics/debug-reporting opt-in. Admin-driven
// from /admin (flip remotely while a bug is live) or self-driven OFF from
// the client. Returns the updated row so the handler can echo it.
func (q *Queries) SetDebugMode(ctx context.Context, arg SetDebugModeParams) (User, error) {
row := q.db.QueryRow(ctx, setDebugMode, arg.ID, arg.DebugModeEnabled)
var i User
err := row.Scan(
&i.ID,
&i.Username,
&i.PasswordHash,
&i.ApiToken,
&i.IsAdmin,
&i.CreatedAt,
&i.SubsonicPassword,
&i.ListenbrainzToken,
&i.ListenbrainzEnabled,
&i.DisplayName,
&i.AutoApproveRequests,
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
)
return i, err
}
const setListenBrainzEnabled = `-- name: SetListenBrainzEnabled :exec
UPDATE users
SET listenbrainz_enabled = $2
@@ -531,7 +580,7 @@ const updateUserAdmin = `-- name: UpdateUserAdmin :one
UPDATE users
SET is_admin = $2
WHERE id = $1
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled
`
type UpdateUserAdminParams struct {
@@ -559,6 +608,7 @@ func (q *Queries) UpdateUserAdmin(ctx context.Context, arg UpdateUserAdminParams
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
)
return i, err
}
@@ -567,7 +617,7 @@ const updateUserAutoApprove = `-- name: UpdateUserAutoApprove :one
UPDATE users
SET auto_approve_requests = $2
WHERE id = $1
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled
`
type UpdateUserAutoApproveParams struct {
@@ -594,6 +644,7 @@ func (q *Queries) UpdateUserAutoApprove(ctx context.Context, arg UpdateUserAutoA
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
)
return i, err
}
@@ -603,7 +654,7 @@ UPDATE users
SET display_name = $2,
email = $3
WHERE id = $1
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at
RETURNING id, username, password_hash, api_token, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled
`
type UpdateUserProfileParams struct {
@@ -632,6 +683,7 @@ func (q *Queries) UpdateUserProfile(ctx context.Context, arg UpdateUserProfilePa
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
)
return i, err
}
@@ -0,0 +1,2 @@
ALTER TABLE users DROP COLUMN IF EXISTS debug_mode_enabled;
DROP TABLE IF EXISTS diagnostic_events;
@@ -0,0 +1,54 @@
-- Device diagnostics / debug-reporting subsystem (M9).
--
-- When an account has users.debug_mode_enabled = true, that account's
-- client(s) stream a timeseries of device state here: connectivity +
-- server-health transitions (roaming / poor-data recovery), UPnP/Sonos
-- sync events (locked-phone desync), and power/lifecycle state (Doze /
-- battery-optimization, the prime suspect for the desync). The admin
-- /admin/diagnostics view filters by account + device + time window and
-- exports the slice as JSON for offline analysis.
--
-- `kind` is a COARSE category gated by a CHECK whitelist; the finer
-- event discriminator + all event-specific fields live in the `payload`
-- jsonb. This keeps the enum small and stable so new event variants
-- don't require a migration (per the enum-CHECK-whitelist rule, only a
-- new *category* would). If you add a category here, mirror it in the
-- handler whitelist in internal/api/diagnostics.go.
--
-- Two timestamps on purpose: occurred_at is the client device clock
-- (when the event happened on the phone — may be skewed/wrong, e.g. in
-- a dead zone), received_at is the server clock at ingest. Retention
-- pruning is keyed on received_at so a bad client clock can't keep rows
-- alive forever.
CREATE TABLE diagnostic_events (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
client_id text NOT NULL,
app_version text,
os_version text,
kind text NOT NULL,
payload jsonb NOT NULL DEFAULT '{}'::jsonb,
occurred_at timestamptz NOT NULL,
received_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT diagnostic_events_kind_check
CHECK (kind IN (
'connectivity', 'upnp_sync', 'power', 'lifecycle',
'heartbeat', 'http'
))
);
-- Admin reads filter by account then device, ordered newest-first.
CREATE INDEX idx_diagnostic_events_user
ON diagnostic_events (user_id, occurred_at DESC);
CREATE INDEX idx_diagnostic_events_client
ON diagnostic_events (client_id, occurred_at DESC);
-- Retention pruner deletes by server-clock age.
CREATE INDEX idx_diagnostic_events_received
ON diagnostic_events (received_at);
-- Per-account opt-in for diagnostics reporting. Admin-set primarily
-- (flip remotely while a bug is live); the client also exposes a local
-- OFF switch. Default false: no account reports until explicitly enabled.
ALTER TABLE users
ADD COLUMN IF NOT EXISTS debug_mode_enabled boolean NOT NULL DEFAULT false;
@@ -0,0 +1,9 @@
-- Drop rows using the new value first so they don't violate the restored
-- (narrower) constraint, then revert the whitelist.
DELETE FROM diagnostic_events WHERE kind = 'playback';
ALTER TABLE diagnostic_events DROP CONSTRAINT diagnostic_events_kind_check;
ALTER TABLE diagnostic_events ADD CONSTRAINT diagnostic_events_kind_check
CHECK (kind IN (
'connectivity', 'upnp_sync', 'power', 'lifecycle',
'heartbeat', 'http'
));
@@ -0,0 +1,19 @@
-- Add 'playback' to the diagnostic_events kind whitelist (M9 follow-up).
--
-- `route` and `player_state` events fire for EVERY output route (phone
-- speaker, Bluetooth, UPnP/Sonos), so bucketing them under 'upnp_sync'
-- was misleading — an operator on Bluetooth earbuds saw "upnp_sync"
-- rows. 'upnp_sync' now means genuinely UPnP/Sonos-specific signal
-- (drops, and future resync/desync); general playback + route telemetry
-- moves to the new 'playback' kind.
--
-- Per the enum-CHECK-whitelist rule, the new value lands by dropping and
-- re-adding the constraint in the same change. Existing rows tagged
-- 'upnp_sync' stay as-is (debug telemetry on a 30-day retention).
ALTER TABLE diagnostic_events DROP CONSTRAINT diagnostic_events_kind_check;
ALTER TABLE diagnostic_events ADD CONSTRAINT diagnostic_events_kind_check
CHECK (kind IN (
'connectivity', 'upnp_sync', 'power', 'lifecycle',
'heartbeat', 'http', 'playback'
));
@@ -0,0 +1,2 @@
ALTER TABLE play_events DROP COLUMN pick_kind;
ALTER TABLE playlist_tracks DROP COLUMN pick_kind;
@@ -0,0 +1,34 @@
-- For You exploration attribution (milestone #127 step 2, Scribe #1249).
--
-- For You is deliberately composed of two populations: a head of
-- top-scored similarity matches ("taste" picks) and a tail sampled from
-- deeper in the candidate ranking ("fresh" picks — the freshness
-- injection). Until now the metrics could only judge For You as one
-- blob, so a high skip rate couldn't be read as "the taste engine is
-- missing" vs "the freshness tax is too high".
--
-- Two denormalized columns, both nullable:
--
-- 1. playlist_tracks.pick_kind — stamped by the system-playlist builder
-- at snapshot build time. NULL for manual playlists and for system
-- variants that don't split (Discover, Songs like X, discovery
-- mixes). System snapshots are atomically replaced daily, so the
-- column never goes stale within a snapshot.
--
-- 2. play_events.pick_kind — stamped at play-ingestion time by looking
-- the track up in the user's CURRENT For You snapshot. Attribution
-- must be frozen at play time: For You rebuilds daily, so a 30-day
-- metrics window spans ~30 snapshots and a join-at-read against the
-- live snapshot would misattribute nearly everything. Plays that
-- predate this feature (or whose track already rotated out, e.g. a
-- late offline replay) stay NULL → reported as unattributed.
ALTER TABLE playlist_tracks
ADD COLUMN pick_kind text
CONSTRAINT playlist_tracks_pick_kind_check
CHECK (pick_kind IN ('taste', 'fresh'));
ALTER TABLE play_events
ADD COLUMN pick_kind text
CONSTRAINT play_events_pick_kind_check
CHECK (pick_kind IN ('taste', 'fresh'));
@@ -0,0 +1,19 @@
-- Rows stamped with the widened vocabulary must be cleared before the
-- narrow CHECK can be re-added; NULL reads as "unattributed", which is
-- the honest downgrade.
UPDATE play_events SET pick_kind = NULL
WHERE pick_kind NOT IN ('taste', 'fresh');
UPDATE playlist_tracks SET pick_kind = NULL
WHERE pick_kind NOT IN ('taste', 'fresh');
ALTER TABLE play_events
DROP CONSTRAINT play_events_pick_kind_check;
ALTER TABLE play_events
ADD CONSTRAINT play_events_pick_kind_check
CHECK (pick_kind IN ('taste', 'fresh'));
ALTER TABLE playlist_tracks
DROP CONSTRAINT playlist_tracks_pick_kind_check;
ALTER TABLE playlist_tracks
ADD CONSTRAINT playlist_tracks_pick_kind_check
CHECK (pick_kind IN ('taste', 'fresh'));
@@ -0,0 +1,41 @@
-- Provenance as standard (milestone #127, Scribe #1270).
--
-- #1249 introduced pick_kind as a For You one-off ('taste'/'fresh').
-- The mechanism — stamp WHY a track is in the snapshot at build time,
-- freeze it onto the play at ingestion, break it down in metrics — is
-- now the standard for every system mix, so the CHECK vocabulary
-- widens to cover:
--
-- taste / fresh For You head vs freshness-injection tail
-- dormant / cross_user / random
-- Discover's three candidate buckets — makes
-- the 40/30/30 allocation measurable instead
-- of a guess
-- tier1 / tier2 / tier3 the tiered-eligibility ladder (project rule
-- #131): tier 1 pins the mix's exact desire,
-- each higher tier steps back a little. Tier
-- provenance is how we measure what the
-- step-back trades away.
--
-- Postgres CHECK whitelists can't be altered in place: DROP + re-ADD
-- with the expanded list, in the same migration.
ALTER TABLE playlist_tracks
DROP CONSTRAINT playlist_tracks_pick_kind_check;
ALTER TABLE playlist_tracks
ADD CONSTRAINT playlist_tracks_pick_kind_check
CHECK (pick_kind IN (
'taste', 'fresh',
'dormant', 'cross_user', 'random',
'tier1', 'tier2', 'tier3'
));
ALTER TABLE play_events
DROP CONSTRAINT play_events_pick_kind_check;
ALTER TABLE play_events
ADD CONSTRAINT play_events_pick_kind_check
CHECK (pick_kind IN (
'taste', 'fresh',
'dormant', 'cross_user', 'random',
'tier1', 'tier2', 'tier3'
));
@@ -0,0 +1,3 @@
DROP TABLE recommendation_tuning_audit;
DROP TABLE taste_tuning;
DROP TABLE recommendation_weight_profiles;
@@ -0,0 +1,57 @@
-- Recommendation tuning lab (milestone #127, Scribe #1250).
--
-- The scoring weights move out of YAML (radio profile) and out of the
-- systemMixWeights hard-code (daily_mix profile) into DB-backed
-- settings with live effect — the defaults-discovery lab (decision
-- #1247): the operator turns knobs here to FIND good values, which
-- then get baked into shipped defaults; end users and other operators
-- should never need this card.
--
-- Rows are seeded by the recsettings service's boot reconcile (the
-- coverart SettingsService pattern), not by this migration, so shipped
-- defaults live in exactly one place (Go).
CREATE TABLE recommendation_weight_profiles (
profile text PRIMARY KEY
CONSTRAINT recommendation_weight_profiles_profile_check
CHECK (profile IN ('radio', 'daily_mix')),
base_weight double precision NOT NULL,
like_boost double precision NOT NULL,
recency_weight double precision NOT NULL,
skip_penalty double precision NOT NULL,
jitter_magnitude double precision NOT NULL,
context_weight double precision NOT NULL,
similarity_weight double precision NOT NULL,
taste_weight double precision NOT NULL,
updated_at timestamptz NOT NULL DEFAULT now()
);
-- Taste-profile build knobs: engagement half-life + the completion→
-- engagement curve points (taste.Config's tunable subset).
CREATE TABLE taste_tuning (
singleton boolean PRIMARY KEY DEFAULT true
CONSTRAINT taste_tuning_singleton_check CHECK (singleton),
half_life_days double precision NOT NULL,
engagement_hard_skip double precision NOT NULL,
engagement_neutral double precision NOT NULL,
engagement_full double precision NOT NULL,
updated_at timestamptz NOT NULL DEFAULT now()
);
-- Every knob turn writes one audit row; the metrics trend view
-- (#1251) annotates these on its timeline so cause→effect is visible
-- after a change. changes is a jsonb array of {field, old, new}.
CREATE TABLE recommendation_tuning_audit (
id bigserial PRIMARY KEY,
changed_at timestamptz NOT NULL DEFAULT now(),
scope text NOT NULL
CONSTRAINT recommendation_tuning_audit_scope_check
CHECK (scope IN ('radio', 'daily_mix', 'taste')),
action text NOT NULL
CONSTRAINT recommendation_tuning_audit_action_check
CHECK (action IN ('update', 'reset')),
changes jsonb NOT NULL
);
CREATE INDEX recommendation_tuning_audit_changed_at_idx
ON recommendation_tuning_audit (changed_at DESC);
+66
View File
@@ -0,0 +1,66 @@
-- name: InsertDiagnosticEvent :one
-- Records one client-reported diagnostic event. The writer validates the
-- `kind` category against its whitelist before this runs; the CHECK
-- constraint is the belt-and-braces guard. payload carries the finer
-- event discriminator + all event-specific fields. Returns received_at
-- so the caller can confirm ingest.
INSERT INTO diagnostic_events (
user_id, client_id, app_version, os_version, kind, payload, occurred_at
)
VALUES ($1, $2, $3, $4, $5, $6, $7)
RETURNING id, received_at;
-- name: ListAdminDiagnostics :many
-- Admin timeline query. All filters are optional (NULL = no filter):
-- account (user_id), device (client_id), category (kind), and the
-- occurred_at window (from_ts/to_ts). Newest-first; the SPA reverses for
-- a chronological timeline / export. Joined with users for the username
-- so the view renders without a second round-trip.
SELECT
de.id AS id,
de.user_id AS user_id,
u.username AS username,
de.client_id AS client_id,
de.app_version AS app_version,
de.os_version AS os_version,
de.kind AS kind,
de.payload AS payload,
de.occurred_at AS occurred_at,
de.received_at AS received_at
FROM diagnostic_events de
JOIN users u ON u.id = de.user_id
WHERE (sqlc.narg(user_id)::uuid IS NULL OR de.user_id = sqlc.narg(user_id)::uuid)
AND (sqlc.narg(client_id)::text IS NULL OR de.client_id = sqlc.narg(client_id)::text)
AND (sqlc.narg(kind)::text IS NULL OR de.kind = sqlc.narg(kind)::text)
AND (sqlc.narg(from_ts)::timestamptz IS NULL
OR de.occurred_at >= sqlc.narg(from_ts)::timestamptz)
AND (sqlc.narg(to_ts)::timestamptz IS NULL
OR de.occurred_at <= sqlc.narg(to_ts)::timestamptz)
ORDER BY de.occurred_at DESC
LIMIT sqlc.arg(lim)::int OFFSET sqlc.arg(off)::int;
-- name: ListDiagnosticDevices :many
-- Distinct devices that have reported, for the admin device filter +
-- "who is currently reporting" overview. Optionally scoped to one account.
SELECT
de.client_id AS client_id,
de.user_id AS user_id,
u.username AS username,
max(de.app_version)::text AS app_version,
max(de.os_version)::text AS os_version,
max(de.occurred_at)::timestamptz AS last_seen,
count(*) AS event_count
FROM diagnostic_events de
JOIN users u ON u.id = de.user_id
WHERE (sqlc.narg(user_id)::uuid IS NULL OR de.user_id = sqlc.narg(user_id)::uuid)
GROUP BY de.client_id, de.user_id, u.username
ORDER BY last_seen DESC;
-- name: GcPruneDiagnostics :execrows
-- Retention sweep for the gc worker. Deletes diagnostic_events older
-- than 30 days by server clock (received_at) so a skewed client clock
-- can't keep rows alive. Matches the other gc lifecycle sweeps'
-- hardcoded-threshold style; diagnostics are debug telemetry, not user
-- data, and 30 days is ample to investigate a live incident after the
-- fact. Returns the affected row count for the sweep log line.
DELETE FROM diagnostic_events WHERE received_at < now() - INTERVAL '30 days';
+23 -2
View File
@@ -24,11 +24,32 @@ ORDER BY started_at DESC
LIMIT 1;
-- name: InsertPlayEvent :one
-- pick_kind is non-NULL only for system-playlist plays whose track was
-- found (with a stamped kind) in the user's live snapshot for that
-- variant at ingestion time (#1249, generalized in #1270).
INSERT INTO play_events (
user_id, track_id, session_id, started_at, client_id, source
) VALUES ($1, $2, $3, $4, $5, $6)
user_id, track_id, session_id, started_at, client_id, source, pick_kind
) VALUES ($1, $2, $3, $4, $5, $6, sqlc.narg(pick_kind)::text)
RETURNING *;
-- name: GetSystemPickKindForTrack :one
-- Looks a track up in the user's CURRENT snapshot of the given system
-- variant and returns its pick_kind. Used at play-ingestion time to
-- freeze provenance onto the play_event — snapshots rebuild daily, so
-- attribution can't be reconstructed at read time (#1249/#1270). No
-- row = track not in today's snapshot (caller stores NULL); a row with
-- NULL pick_kind = the variant doesn't stamp (yet). songs_like_artist
-- is non-singleton (up to 3 mixes/user); a track in two of them takes
-- whichever LIMIT 1 hits — acceptable, tier stamps there describe the
-- same eligibility ladder.
SELECT pt.pick_kind
FROM playlist_tracks pt
JOIN playlists p ON p.id = pt.playlist_id
WHERE p.user_id = $1
AND p.system_variant = $2
AND pt.track_id = $3
LIMIT 1;
-- name: UpdatePlayEventEnded :one
-- Closes a play_event by id with the given ended_at, duration, and skip flag.
-- completion_ratio is computed from duration_played_ms and the track duration
+5 -3
View File
@@ -68,8 +68,9 @@ ORDER BY pt.position;
-- name: AppendPlaylistTrack :one
-- Inserts at the next available position. Snapshot fields are copied
-- from the tracks/albums/artists join at insert time. tracks.duration_ms
-- is converted to seconds for the snapshot.
INSERT INTO playlist_tracks (playlist_id, position, track_id, title, artist_name, album_title, duration_sec)
-- is converted to seconds for the snapshot. pick_kind is NULL for manual
-- playlists; the For You builder stamps 'taste'/'fresh' (#1249).
INSERT INTO playlist_tracks (playlist_id, position, track_id, title, artist_name, album_title, duration_sec, pick_kind)
SELECT
sqlc.arg(playlist_id)::uuid,
COALESCE((SELECT MAX(position) + 1 FROM playlist_tracks WHERE playlist_id = sqlc.arg(playlist_id)::uuid), 0),
@@ -77,7 +78,8 @@ SELECT
t.title,
artists.name,
albums.title,
(t.duration_ms / 1000)::integer
(t.duration_ms / 1000)::integer,
sqlc.narg(pick_kind)::text
FROM tracks t
JOIN albums ON albums.id = t.album_id
JOIN artists ON artists.id = t.artist_id
+46 -10
View File
@@ -1,22 +1,58 @@
-- Recommendation observability (#796 phase 4). Per-source play outcomes so the
-- operator can see whether each recommendation surface is landing and tune the
-- taste weights. Source is stamped on play_events when a play is launched from
-- a system-playlist surface ('for_you' | 'discover' | the discovery mixes);
-- NULL for library / radio / user-playlist plays, which are excluded here.
-- a recommendation surface; NULL means the user picked the track manually —
-- those rows are INCLUDED here as the baseline control group the surfaces are
-- judged against (milestone #127: delta-vs-baseline is what makes the numbers
-- actionable). Raw source strings are bucketed into stable surface families in
-- the Go handler; completion_n is carried so family merges can weight
-- avg_completion correctly.
-- name: RecommendationSourceMetricsForUser :many
-- $1 user_id, $2 window_days. plays/skips are counts; avg_completion is the
-- mean completion ratio over plays that recorded one (0 when none did).
-- name: RecommendationWeeklyTrends :many
-- Weekly per-source outcome series for the tuning lab's trend view
-- (#1251). Aggregated across ALL users: the tuning knobs are global,
-- so judging a knob turn needs global outcomes — rows carry rates
-- only, no track or user identity. NULL-source (manual) rows are
-- included as the baseline family.
--
-- taste_hits counts plays whose track's artist has a positive weight
-- in that user's CURRENT taste profile — the "cheap recompute" option:
-- retroactive over the whole window, at the cost of drift (the profile
-- is today's, the play may be weeks old). Good enough to read whether
-- a surface is feeding taste-fitting tracks.
-- $1 window in weeks.
SELECT
date_trunc('week', pe.started_at)::date AS week_start,
pe.source,
count(*)::bigint AS plays,
count(*) FILTER (WHERE pe.was_skipped)::bigint AS skips,
COALESCE(
avg(pe.completion_ratio) FILTER (WHERE pe.completion_ratio IS NOT NULL),
0)::float8 AS avg_completion
count(pe.completion_ratio)::bigint AS completion_n,
COALESCE(avg(pe.completion_ratio), 0)::float8 AS avg_completion,
count(*) FILTER (WHERE tpa.artist_id IS NOT NULL)::bigint AS taste_hits
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
LEFT JOIN taste_profile_artists tpa
ON tpa.user_id = pe.user_id
AND tpa.artist_id = t.artist_id
AND tpa.weight > 0
WHERE pe.started_at > now() - (sqlc.arg(weeks)::int * INTERVAL '1 week')
GROUP BY 1, 2
ORDER BY 1, 2;
-- name: RecommendationSourceMetricsForUser :many
-- $1 user_id, $2 window_days. plays/skips are counts; avg_completion is the
-- mean completion ratio over the completion_n plays that recorded one.
-- pick_kind splits For You plays into taste/fresh/unattributed (#1249);
-- it is NULL for every other source, so those still group to one row.
SELECT
pe.source,
pe.pick_kind,
count(*)::bigint AS plays,
count(*) FILTER (WHERE pe.was_skipped)::bigint AS skips,
count(pe.completion_ratio)::bigint AS completion_n,
COALESCE(avg(pe.completion_ratio), 0)::float8 AS avg_completion
FROM play_events pe
WHERE pe.user_id = $1
AND pe.source IS NOT NULL
AND pe.started_at > now() - ($2::float8 * INTERVAL '1 day')
GROUP BY pe.source
GROUP BY pe.source, pe.pick_kind
ORDER BY plays DESC;
@@ -0,0 +1,61 @@
-- Recommendation tuning lab queries (#1250). Seeding happens via the
-- recsettings boot reconcile; shipped defaults live in Go only.
-- name: UpsertWeightProfileDefaults :exec
-- Boot reconcile: insert the shipped defaults for a profile if the row
-- doesn't exist yet. Never overwrites operator-tuned values.
INSERT INTO recommendation_weight_profiles (
profile, base_weight, like_boost, recency_weight, skip_penalty,
jitter_magnitude, context_weight, similarity_weight, taste_weight
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
ON CONFLICT (profile) DO NOTHING;
-- name: ListWeightProfiles :many
SELECT * FROM recommendation_weight_profiles ORDER BY profile;
-- name: UpdateWeightProfile :one
UPDATE recommendation_weight_profiles
SET base_weight = $2,
like_boost = $3,
recency_weight = $4,
skip_penalty = $5,
jitter_magnitude = $6,
context_weight = $7,
similarity_weight = $8,
taste_weight = $9,
updated_at = now()
WHERE profile = $1
RETURNING *;
-- name: UpsertTasteTuningDefaults :exec
INSERT INTO taste_tuning (
singleton, half_life_days, engagement_hard_skip,
engagement_neutral, engagement_full
) VALUES (true, $1, $2, $3, $4)
ON CONFLICT (singleton) DO NOTHING;
-- name: GetTasteTuning :one
SELECT * FROM taste_tuning WHERE singleton = true;
-- name: UpdateTasteTuning :one
UPDATE taste_tuning
SET half_life_days = $1,
engagement_hard_skip = $2,
engagement_neutral = $3,
engagement_full = $4,
updated_at = now()
WHERE singleton = true
RETURNING *;
-- name: InsertTuningAudit :exec
-- changes is a jsonb array of {field, old, new} objects.
INSERT INTO recommendation_tuning_audit (scope, action, changes)
VALUES ($1, $2, $3);
-- name: ListTuningAudit :many
-- Newest first; consumed by the metrics trend view (#1251) to annotate
-- knob turns on the timeline.
SELECT id, changed_at, scope, action, changes
FROM recommendation_tuning_audit
ORDER BY changed_at DESC, id DESC
LIMIT $1;
+161 -71
View File
@@ -8,6 +8,11 @@
-- name: ListDeepCutsTracks :many
-- #419 Deep Cuts: low-play tracks (<=2 plays) from artists the user
-- has liked OR played heavily (>=5 non-skip plays across the artist).
-- Tracks the user has skipped twice or more don't qualify (#1257):
-- eligibility used to count only unskipped plays, so the most actively
-- rejected tracks read as "barely heard" and kept being re-offered.
-- Threshold 2 so a single accidental skip doesn't banish a track;
-- passive-signal only (no dislike UI, rule #101).
-- $1 user_id, $2 date string.
WITH affinity_artists AS (
SELECT artist_id FROM general_likes_artists WHERE user_id = $1
@@ -23,12 +28,20 @@ play_counts AS (
FROM play_events
WHERE user_id = $1 AND was_skipped = false
GROUP BY track_id
),
skip_counts AS (
SELECT track_id, COUNT(*) AS c
FROM play_events
WHERE user_id = $1 AND was_skipped = true
GROUP BY track_id
)
SELECT t.id, t.album_id, t.artist_id
FROM tracks t
JOIN affinity_artists aa ON aa.artist_id = t.artist_id
LEFT JOIN play_counts pc ON pc.track_id = t.id
LEFT JOIN skip_counts sc ON sc.track_id = t.id
WHERE COALESCE(pc.c, 0) <= 2
AND COALESCE(sc.c, 0) < 2
AND NOT EXISTS (
SELECT 1 FROM lidarr_quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
@@ -37,76 +50,113 @@ SELECT t.id, t.album_id, t.artist_id
LIMIT 200;
-- name: ListRediscoverTracks :many
-- #420 Rediscover: tracks the user played a lot (>=5 non-skip) but
-- has drifted away from. Tiered so a young library still gets a mix:
-- tier 0 not played in the last 6 months (true rediscovery)
-- tier 1 (only if tier 0 empty) not played in the last 30 days
-- Ordered by historical affection. $1 user_id.
-- #420 Rediscover: tracks the user played a lot but has drifted away
-- from. One blended pool (issue #1246: the old two-tier UNION was
-- all-or-nothing — a single ">=6 months cold" row suppressed the whole
-- ">=30 days" tier, which is how a one-song playlist shipped; the 6mo
-- tier was also a strict subset of the 30d tier). Eligibility is >=3
-- non-skip plays and >=30 days cold; the bar sits at 3 rather than 5
-- so a weeks-old library still fields a pool — on young histories the
-- >=5-play tracks are precisely the ones still in rotation. Ordering
-- prefers true rediscoveries (>=6 months cold), then strong affection
-- (>=5 plays), then raw play count; the producer's minimum floor
-- decides whether the pool is big enough to ship at all. $1 user_id.
WITH stats AS (
SELECT pe.track_id, COUNT(*) AS c, MAX(pe.started_at) AS last_at
FROM play_events pe
WHERE pe.user_id = $1 AND pe.was_skipped = false
GROUP BY pe.track_id
),
deep AS (
SELECT t.id, t.album_id, t.artist_id, s.c, 0 AS tier
FROM tracks t
JOIN stats s ON s.track_id = t.id
WHERE s.c >= 5
AND s.last_at <= now() - interval '6 months'
AND NOT EXISTS (
SELECT 1 FROM lidarr_quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
)
),
shallow AS (
SELECT t.id, t.album_id, t.artist_id, s.c, 1 AS tier
FROM tracks t
JOIN stats s ON s.track_id = t.id
WHERE s.c >= 5
AND s.last_at <= now() - interval '30 days'
AND NOT EXISTS (
SELECT 1 FROM lidarr_quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
)
)
SELECT id, album_id, artist_id
FROM (
SELECT id, album_id, artist_id, c, tier FROM deep
UNION ALL
SELECT id, album_id, artist_id, c, tier FROM shallow
WHERE NOT EXISTS (SELECT 1 FROM deep)
) u
ORDER BY tier, c DESC, id
LIMIT 200;
-- name: ListNewForYouTracks :many
-- #421 New for you: tracks from albums added in the last 30 days
-- whose artist the user has liked OR played (>=3 non-skip). Album-
-- coherent (newest album first, then disc/track) — the producer does
-- NOT diversity-cap these; they're meant as whole-album discovery.
-- $1 user_id.
WITH affinity_artists AS (
SELECT artist_id FROM general_likes_artists WHERE user_id = $1
UNION
SELECT t.artist_id
FROM play_events pe JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1 AND pe.was_skipped = false
GROUP BY t.artist_id
HAVING COUNT(*) >= 3
)
SELECT t.id, t.album_id, t.artist_id
FROM tracks t
JOIN albums al ON al.id = t.album_id
JOIN affinity_artists aa ON aa.artist_id = al.artist_id
WHERE al.created_at >= now() - interval '30 days'
JOIN stats s ON s.track_id = t.id
WHERE s.c >= 3
AND s.last_at <= now() - interval '30 days'
AND NOT EXISTS (
SELECT 1 FROM lidarr_quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
)
ORDER BY al.created_at DESC, t.disc_number NULLS FIRST, t.track_number NULLS FIRST
-- Declined-resurfacing cooldown (#1258): a skip on a
-- rediscover-sourced play is the user explicitly saying "I've moved
-- on" to the exact invitation this mix extends — such tracks sit
-- out ~90 days instead of re-qualifying the next day. Passive
-- signal only (rule #101).
AND NOT EXISTS (
SELECT 1 FROM play_events rp
WHERE rp.user_id = $1
AND rp.track_id = t.id
AND rp.source = 'rediscover'
AND rp.was_skipped = true
AND rp.started_at > now() - interval '90 days'
)
ORDER BY (s.last_at <= now() - interval '6 months') DESC,
(s.c >= 5) DESC,
s.c DESC, t.id
LIMIT 200;
-- name: ListNewForYouTracks :many
-- #421 New for you, tiered per project rule #131 (#1267). The old
-- single hard rule (added <30d AND direct affinity) had two failure
-- modes with one root: no notion of consumption — the album you
-- devoured in week one crowded the mix for three more weeks, and
-- after a quiet month nothing qualified at all.
--
-- "Consumed" = any track attempted for >=30s (the First Listens
-- threshold, #1268): once you've meaningfully engaged, the album is
-- no longer news. Applies to every tier — played albums leave the
-- mix the next build.
--
-- tier 1 the exact desire: unconsumed albums added <30d by
-- direct-affinity artists (liked, or >=3 attempted plays)
-- tier 2 step back a little: unconsumed affinity albums from the
-- wider 30-90d window — added while you weren't looking
-- tier 3 step back more: any unconsumed album added <90d
-- regardless of affinity, newest first
--
-- Album-coherent within tiers (newest album first, then disc/track);
-- the producer fills top-down and rotates within tier blocks. The
-- tier lands on playlist_tracks.pick_kind so metrics can price what
-- each step-back trades away. $1 user_id.
WITH attempted AS (
SELECT DISTINCT pe.track_id
FROM play_events pe
WHERE pe.user_id = $1 AND pe.duration_played_ms >= 30000
),
consumed_albums AS (
SELECT DISTINCT t.album_id
FROM attempted a JOIN tracks t ON t.id = a.track_id
),
affinity_artists AS (
SELECT gla.artist_id FROM general_likes_artists gla WHERE gla.user_id = $1
UNION
SELECT t.artist_id
FROM attempted a JOIN tracks t ON t.id = a.track_id
GROUP BY t.artist_id
HAVING COUNT(*) >= 3
),
albums_tiered AS (
SELECT al.id AS album_id, al.created_at,
CASE
WHEN al.created_at >= now() - interval '30 days'
AND al.artist_id IN (SELECT artist_id FROM affinity_artists) THEN 1
WHEN al.artist_id IN (SELECT artist_id FROM affinity_artists) THEN 2
ELSE 3
END AS tier
FROM albums al
WHERE al.created_at >= now() - interval '90 days'
AND al.id NOT IN (SELECT album_id FROM consumed_albums)
)
SELECT t.id, t.album_id, t.artist_id, alt.tier::int AS tier
FROM tracks t
JOIN albums_tiered alt ON alt.album_id = t.album_id
WHERE NOT EXISTS (
SELECT 1 FROM lidarr_quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
)
ORDER BY alt.tier, alt.created_at DESC, t.album_id,
t.disc_number NULLS FIRST, t.track_number NULLS FIRST
LIMIT 300;
-- name: ListOnThisDayTracks :many
-- #422 On This Day: tracks the user played around this calendar date
-- (±10 day-of-year) in the past, excluding the very recent (>30 days
@@ -115,13 +165,23 @@ SELECT t.id, t.album_id, t.artist_id
-- and window ±7→±10 so it surfaces on a months-old library instead
-- of needing a full year of history; still skips cleanly (no rows →
-- no playlist) when there's no qualifying history yet.
-- Day-of-year distance is circular (#1256): plain ABS made Dec 28 vs
-- Jan 3 read as 359 days apart, silently gutting the window for ~3
-- weeks around every New Year — precisely when holiday nostalgia is
-- the point. LEAST(d, 365-d) wraps the boundary; leap-year drift of
-- ±1 is absorbed by the ±10 window. The build date ($2, already here
-- for the md5 rotation) anchors "today" instead of now() so the
-- window is testable and consistent with the mix's daily determinism.
-- $1 user_id, $2 date string.
WITH windowed AS (
SELECT track_id, COUNT(*) AS c
FROM play_events
WHERE user_id = $1 AND was_skipped = false
AND started_at < now() - interval '30 days'
AND ABS(EXTRACT(DOY FROM started_at) - EXTRACT(DOY FROM now())) <= 10
AND LEAST(
ABS(EXTRACT(DOY FROM started_at) - EXTRACT(DOY FROM $2::date)),
365 - ABS(EXTRACT(DOY FROM started_at) - EXTRACT(DOY FROM $2::date))
) <= 10
GROUP BY track_id
)
SELECT t.id, t.album_id, t.artist_id
@@ -135,35 +195,65 @@ SELECT t.id, t.album_id, t.artist_id
LIMIT 200;
-- name: ListFirstListensTracks :many
-- #423 First Listens: albums the user has never played any track of.
-- Tiered: liked-artist albums first, then played-artist albums, then
-- the rest — album-coherent within each tier. Not diversity-capped
-- (whole-album discovery). $1 user_id.
WITH heard_albums AS (
SELECT DISTINCT t.album_id
FROM play_events pe JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1
-- #423 First Listens: songs the user has never heard or even
-- attempted. "Attempted" is track-level with a >=30s listen threshold
-- (#1268, operator decision 2026-07-03) — the old version disqualified
-- a whole album on ANY play_event, so a 2-second accidental brush
-- banished it permanently.
--
-- tier 1 the exact desire: albums with ZERO attempted tracks
-- tier 2 step back: barely-attempted albums (<=25% of tracks
-- reached 30s) — brushed, never explored. The already-
-- attempted tracks themselves are excluded; they're not
-- first listens.
--
-- Within each tier, affinity-ordered as before: liked artist, then
-- attempted-played artist (>=30s — skip-only contact isn't trust),
-- then the rest; album-coherent. The tier lands on
-- playlist_tracks.pick_kind via the producer. $1 user_id.
WITH attempted AS (
SELECT DISTINCT pe.track_id
FROM play_events pe
WHERE pe.user_id = $1 AND pe.duration_played_ms >= 30000
),
played_artists AS (
album_attempts AS (
SELECT t.album_id,
COUNT(a.track_id) AS attempted_count,
COUNT(*) AS track_count
FROM tracks t
LEFT JOIN attempted a ON a.track_id = t.id
GROUP BY t.album_id
),
attempted_artists AS (
SELECT DISTINCT t.artist_id
FROM play_events pe JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1
FROM attempted a JOIN tracks t ON t.id = a.track_id
),
albums_tiered AS (
SELECT aa.album_id,
CASE
WHEN aa.attempted_count = 0 THEN 1
WHEN aa.attempted_count::float / GREATEST(aa.track_count, 1) <= 0.25 THEN 2
END AS tier
FROM album_attempts aa
)
SELECT t.id, t.album_id, t.artist_id
SELECT t.id, t.album_id, t.artist_id, alt.tier::int AS tier
FROM tracks t
JOIN albums al ON al.id = t.album_id
WHERE NOT EXISTS (SELECT 1 FROM heard_albums h WHERE h.album_id = al.id)
JOIN albums_tiered alt ON alt.album_id = al.id
WHERE alt.tier IS NOT NULL
AND NOT EXISTS (SELECT 1 FROM attempted a WHERE a.track_id = t.id)
AND NOT EXISTS (
SELECT 1 FROM lidarr_quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
)
ORDER BY
alt.tier,
(CASE
WHEN EXISTS (
SELECT 1 FROM general_likes_artists gla
WHERE gla.user_id = $1 AND gla.artist_id = al.artist_id
) THEN 0
WHEN al.artist_id IN (SELECT artist_id FROM played_artists) THEN 1
WHEN al.artist_id IN (SELECT artist_id FROM attempted_artists) THEN 1
ELSE 2
END),
al.id, t.disc_number NULLS FIRST, t.track_number NULLS FIRST
+71 -18
View File
@@ -47,14 +47,29 @@ UPDATE system_playlist_runs
UPDATE system_playlist_runs SET in_flight = false WHERE in_flight = true;
-- name: PickSeedArtists :many
-- Top-5 most-engaged distinct artist candidates in the user's last 7
-- days. The Go-side picker (pickSeedArtistsForDay) shuffles these
-- daily-deterministically and takes the first 3 so the set of
-- "Songs like X" mixes rotates day-to-day.
-- Score = unskipped-play count + 5 if user has liked the artist.
WITH plays AS (
-- Top-5 most-engaged distinct artist candidates, tiered so the
-- "Songs like X" mixes never silently vanish (#1255): the old hard
-- 7-day window emptied the seed pool after a quiet week, and the
-- daily atomic-replace build then deleted every existing mix until
-- the user played something again. Same fallback shape as
-- PickTopPlayedTracksForUser (For You's seeds):
-- tier 0 engagement in the last 7 days
-- tier 1 (only if tier 0 empty) last 30 days
-- tier 2 (only if tiers 0+1 empty) all-time
-- tier 3 (only if 0-2 empty) liked artists with no play history
-- All returned rows share one tier; produceSeedMixes maps it onto the
-- rule-#131 pick-kind ladder and stamps the built tracks, so metrics
-- can compare mixes seeded from fresh vs stale engagement.
-- Score = unskipped-play count + 5 if user has liked the artist. The
-- Go-side picker (pickSeedArtistsForDay) shuffles the 5
-- daily-deterministically and takes 3 so the mix set rotates.
WITH liked AS (
SELECT gla.artist_id FROM general_likes_artists gla WHERE gla.user_id = $1
),
recent7 AS (
SELECT t.artist_id,
COUNT(*) FILTER (WHERE pe.was_skipped = false) AS play_count
COUNT(*) FILTER (WHERE pe.was_skipped = false) AS play_count,
0 AS tier
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1
@@ -62,14 +77,52 @@ WITH plays AS (
AND t.artist_id IS NOT NULL
GROUP BY t.artist_id
),
liked AS (
SELECT artist_id FROM general_likes_artists WHERE user_id = $1
recent30 AS (
SELECT t.artist_id,
COUNT(*) FILTER (WHERE pe.was_skipped = false) AS play_count,
1 AS tier
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1
AND pe.started_at > now() - INTERVAL '30 days'
AND t.artist_id IS NOT NULL
GROUP BY t.artist_id
),
alltime AS (
SELECT t.artist_id,
COUNT(*) FILTER (WHERE pe.was_skipped = false) AS play_count,
2 AS tier
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1
AND t.artist_id IS NOT NULL
GROUP BY t.artist_id
),
likedonly AS (
SELECT l.artist_id, 0::bigint AS play_count, 3 AS tier
FROM liked l
),
chosen AS (
SELECT artist_id, play_count, tier FROM recent7
UNION ALL
SELECT artist_id, play_count, tier FROM recent30
WHERE NOT EXISTS (SELECT 1 FROM recent7)
UNION ALL
SELECT artist_id, play_count, tier FROM alltime
WHERE NOT EXISTS (SELECT 1 FROM recent7)
AND NOT EXISTS (SELECT 1 FROM recent30)
UNION ALL
SELECT artist_id, play_count, tier FROM likedonly
WHERE NOT EXISTS (SELECT 1 FROM recent7)
AND NOT EXISTS (SELECT 1 FROM recent30)
AND NOT EXISTS (SELECT 1 FROM alltime)
)
SELECT p.artist_id,
(p.play_count + CASE WHEN l.artist_id IS NOT NULL THEN 5 ELSE 0 END)::bigint AS score
FROM plays p
LEFT JOIN liked l ON l.artist_id = p.artist_id
ORDER BY score DESC, p.artist_id
SELECT c.artist_id,
(c.play_count + CASE WHEN l.artist_id IS NOT NULL THEN 5 ELSE 0 END)::bigint AS score,
c.tier::int AS tier
FROM chosen c
LEFT JOIN liked l ON l.artist_id = c.artist_id
ORDER BY score DESC, c.artist_id
LIMIT 5;
-- name: PickTopPlayedTracksForUser :many
@@ -78,10 +131,10 @@ SELECT p.artist_id,
-- tier 1 (only if tier 0 empty) all-time top non-skip plays
-- tier 2 (only if tiers 0+1 empty) liked tracks
-- Returns up to 5 ids; tie-break by track_id for determinism. The
-- Go-side picker (pickForYouSeedForDay) rotates one per day via
-- userIDHash. Widened from a hard 7-day window, which made For-You
-- disappear after a week of not listening and never recover on a
-- self-hosted library with sparse history.
-- Go-side picker (pickDailySeeds) draws the day's seeds from these.
-- Widened from a hard 7-day window, which made For-You disappear
-- after a week of not listening and never recover on a self-hosted
-- library with sparse history.
WITH recent AS (
SELECT t.id, COUNT(*) AS c, 0 AS tier
FROM play_events pe
+11 -1
View File
@@ -55,7 +55,8 @@ WHERE id = $1;
-- name: ListUsers :many
-- Admin user-management list. Sort newest-first.
SELECT id, username, display_name, is_admin, auto_approve_requests, created_at
SELECT id, username, display_name, is_admin, auto_approve_requests,
debug_mode_enabled, created_at
FROM users
ORDER BY created_at DESC;
@@ -115,6 +116,15 @@ UPDATE users
WHERE id = $1
RETURNING *;
-- name: SetDebugMode :one
-- Toggle the per-account diagnostics/debug-reporting opt-in. Admin-driven
-- from /admin (flip remotely while a bug is live) or self-driven OFF from
-- the client. Returns the updated row so the handler can echo it.
UPDATE users
SET debug_mode_enabled = $2
WHERE id = $1
RETURNING *;
-- name: ChangeUserPassword :exec
-- Self-service password change. Caller (HTTP handler) verifies the
-- current password before calling this. Distinct from
+5
View File
@@ -62,6 +62,11 @@ var dataTables = []string{
// (never recreates the singleton, seeded once by 0018); ResetDB
// resets its counter via UPDATE below instead.
"cover_art_provider_settings",
// recsettings.New reconciles shipped defaults on every construction
// (#1250), so truncating gives each test pristine tuning values.
"recommendation_weight_profiles",
"taste_tuning",
"recommendation_tuning_audit",
"tracks",
"albums",
"artists",
+2
View File
@@ -16,6 +16,7 @@
// - GcExpireScrobbleQueueFailedRows (#567)
// - GcResetStuckSystemPlaylistRuns (#574)
// - GcDeleteExpiredPasswordResets (#575)
// - GcPruneDiagnostics (M9 — diagnostics 30d retention)
package gc
import (
@@ -82,6 +83,7 @@ func (w *Worker) tickOnce(ctx context.Context) {
w.runSweep(ctx, "expire_scrobble_failed", q.GcExpireScrobbleQueueFailedRows)
w.runSweep(ctx, "reset_stuck_system_runs", q.GcResetStuckSystemPlaylistRuns)
w.runSweep(ctx, "delete_expired_password_resets", q.GcDeleteExpiredPasswordResets)
w.runSweep(ctx, "prune_diagnostics", q.GcPruneDiagnostics)
}
// runSweep is a small adapter so each sweep call site is a one-liner
+46
View File
@@ -97,6 +97,36 @@ var systemPlaylistSources = map[string]bool{
"songs_like_artist": true,
}
// lookupSystemPickKind resolves a system-playlist play's pick_kind
// (taste/fresh, Discover's bucket, or a rule-#131 tier) from the user's
// live snapshot of that variant at ingestion time. Attribution must be
// frozen now — snapshots rebuild daily, so it can't be reconstructed at
// metrics-read time (#1249, generalized in #1270). Source strings
// double as system_variant values (see systemPlaylistSources). Returns
// nil (unattributed) when the track isn't in the current snapshot —
// plays predating the feature, or an offline replay arriving after the
// track rotated out — and when the variant doesn't stamp yet. Real DB
// errors propagate and fail the caller's transaction.
func lookupSystemPickKind(
ctx context.Context,
q *dbq.Queries,
userID, trackID pgtype.UUID,
variant string,
) (*string, error) {
kind, err := q.GetSystemPickKindForTrack(ctx, dbq.GetSystemPickKindForTrackParams{
UserID: userID,
SystemVariant: &variant,
TrackID: trackID,
})
if errors.Is(err, pgx.ErrNoRows) {
return nil, nil
}
if err != nil {
return nil, err
}
return kind, nil
}
// RecordPlayStartedWithSource is RecordPlayStarted plus a `source`
// tag identifying which surface the play came from. When source is a
// known system-playlist kind the track is appended to that user's
@@ -127,6 +157,13 @@ func (w *Writer) RecordPlayStartedWithSource(
if source != "" {
sourcePtr = &source
}
var pickKind *string
if systemPlaylistSources[source] {
pickKind, err = lookupSystemPickKind(ctx, q, userID, trackID, source)
if err != nil {
return err
}
}
ev, err := q.InsertPlayEvent(ctx, dbq.InsertPlayEventParams{
UserID: userID,
TrackID: trackID,
@@ -134,6 +171,7 @@ func (w *Writer) RecordPlayStartedWithSource(
StartedAt: pgtype.Timestamptz{Time: at, Valid: true},
ClientID: clientIDPtr,
Source: sourcePtr,
PickKind: pickKind,
})
if err != nil {
return err
@@ -395,6 +433,13 @@ func (w *Writer) RecordOfflinePlay(
if source != "" {
sourcePtr = &source
}
var pickKind *string
if systemPlaylistSources[source] {
pickKind, err = lookupSystemPickKind(ctx, q, userID, trackID, source)
if err != nil {
return err
}
}
ev, err := q.InsertPlayEvent(ctx, dbq.InsertPlayEventParams{
UserID: userID,
TrackID: trackID,
@@ -402,6 +447,7 @@ func (w *Writer) RecordOfflinePlay(
StartedAt: pgtype.Timestamptz{Time: at, Valid: true},
ClientID: clientIDPtr,
Source: sourcePtr,
PickKind: pickKind,
})
if err != nil {
return err
+120
View File
@@ -488,3 +488,123 @@ func TestRecordPlayStarted_NoRotationWithoutSource(t *testing.T) {
t.Errorf("expected no rotation row for a source-less play")
}
}
// seedSystemSnapshot creates a system playlist of the given variant
// containing the fixture track with the given pick_kind, mimicking
// what the builder persists (#1249, generalized #1270).
func seedSystemSnapshot(t *testing.T, f fixture, variant, pickKind string) {
t.Helper()
pl, err := f.q.CreateSystemPlaylist(context.Background(), dbq.CreateSystemPlaylistParams{
UserID: f.user, Name: variant, SystemVariant: &variant,
})
if err != nil {
t.Fatalf("CreateSystemPlaylist: %v", err)
}
if _, err := f.q.AppendPlaylistTrack(context.Background(), dbq.AppendPlaylistTrackParams{
PlaylistID: pl.ID, TrackID: f.track, PickKind: &pickKind,
}); err != nil {
t.Fatalf("AppendPlaylistTrack: %v", err)
}
}
func TestRecordPlayStartedWithSource_StampsForYouPickKind(t *testing.T) {
f := newFixture(t, 200_000)
seedSystemSnapshot(t, f, "for_you", "fresh")
res, err := f.w.RecordPlayStartedWithSource(
context.Background(), f.user, f.track, "c", "for_you", time.Now().UTC(),
)
if err != nil {
t.Fatalf("RecordPlayStartedWithSource: %v", err)
}
got, err := f.q.GetPlayEventByID(context.Background(), res.PlayEventID)
if err != nil {
t.Fatalf("get: %v", err)
}
if got.PickKind == nil || *got.PickKind != "fresh" {
t.Errorf("pick_kind = %v, want fresh (frozen from live snapshot)", got.PickKind)
}
}
func TestRecordPlayStartedWithSource_NoSnapshotMatch_PickKindNull(t *testing.T) {
// A for_you play whose track isn't in the current snapshot (rotated
// out, or no snapshot exists) stays unattributed rather than guessing.
f := newFixture(t, 200_000)
res, err := f.w.RecordPlayStartedWithSource(
context.Background(), f.user, f.track, "c", "for_you", time.Now().UTC(),
)
if err != nil {
t.Fatalf("RecordPlayStartedWithSource: %v", err)
}
got, err := f.q.GetPlayEventByID(context.Background(), res.PlayEventID)
if err != nil {
t.Fatalf("get: %v", err)
}
if got.PickKind != nil {
t.Errorf("pick_kind = %q, want NULL (track not in snapshot)", *got.PickKind)
}
}
func TestRecordPlayStartedWithSource_CrossVariant_PickKindNull(t *testing.T) {
// Attribution is variant-scoped: even with a For You snapshot
// containing the track, a play from a different surface must not
// pick up For You's stamp — it reads its OWN variant's snapshot,
// which here doesn't exist.
f := newFixture(t, 200_000)
seedSystemSnapshot(t, f, "for_you", "taste")
res, err := f.w.RecordPlayStartedWithSource(
context.Background(), f.user, f.track, "c", "discover", time.Now().UTC(),
)
if err != nil {
t.Fatalf("RecordPlayStartedWithSource: %v", err)
}
got, err := f.q.GetPlayEventByID(context.Background(), res.PlayEventID)
if err != nil {
t.Fatalf("get: %v", err)
}
if got.PickKind != nil {
t.Errorf("pick_kind = %q, want NULL for a cross-variant play", *got.PickKind)
}
}
func TestRecordPlayStartedWithSource_StampsDiscoverBucket(t *testing.T) {
// Discover stamps its candidate bucket (#1270); a discover play whose
// track is in the live snapshot freezes that bucket onto the play.
f := newFixture(t, 200_000)
seedSystemSnapshot(t, f, "discover", "dormant")
res, err := f.w.RecordPlayStartedWithSource(
context.Background(), f.user, f.track, "c", "discover", time.Now().UTC(),
)
if err != nil {
t.Fatalf("RecordPlayStartedWithSource: %v", err)
}
got, err := f.q.GetPlayEventByID(context.Background(), res.PlayEventID)
if err != nil {
t.Fatalf("get: %v", err)
}
if got.PickKind == nil || *got.PickKind != "dormant" {
t.Errorf("pick_kind = %v, want dormant (frozen from live snapshot)", got.PickKind)
}
}
func TestRecordOfflinePlay_StampsForYouPickKind(t *testing.T) {
// The offline replay path threads source the same way; a replayed
// for_you play whose track is still in the snapshot gets attributed.
f := newFixture(t, 200_000)
seedSystemSnapshot(t, f, "for_you", "taste")
at := time.Now().UTC().Add(-time.Hour)
if err := f.w.RecordOfflinePlay(
context.Background(), f.user, f.track, "c", "for_you", at, 180_000,
); err != nil {
t.Fatalf("RecordOfflinePlay: %v", err)
}
var pickKind *string
if err := f.pool.QueryRow(context.Background(),
`SELECT pick_kind FROM play_events WHERE user_id = $1 AND track_id = $2`,
f.user, f.track,
).Scan(&pickKind); err != nil {
t.Fatalf("select: %v", err)
}
if pickKind == nil || *pickKind != "taste" {
t.Errorf("pick_kind = %v, want taste", pickKind)
}
}
+18 -5
View File
@@ -20,11 +20,15 @@ const (
// discoverTrack is the common shape used by the bucket allocator. The
// three sqlc-generated row types collapse into this internal struct so
// downstream functions don't need to be generic over them.
// downstream functions don't need to be generic over them. PickKind is
// the originating bucket (dormant/cross_user/random, #1270), stamped in
// the row adapters so it survives the interleave — bucket identity is
// what makes the 40/30/30 allocation measurable in the metrics.
type discoverTrack struct {
ID pgtype.UUID
AlbumID pgtype.UUID
ArtistID pgtype.UUID
PickKind string
}
// buildDiscoverCandidates assembles the Discover playlist track list.
@@ -91,7 +95,7 @@ func buildDiscoverCandidates(ctx context.Context, q *dbq.Queries, logger *slog.L
// already gave us the ranking.
ranked := make([]rankedCandidate, 0, len(out))
for _, t := range out {
ranked = append(ranked, rankedCandidate{TrackID: t.ID})
ranked = append(ranked, rankedCandidate{TrackID: t.ID, PickKind: t.PickKind})
}
return ranked, nil
}
@@ -99,7 +103,10 @@ func buildDiscoverCandidates(ctx context.Context, q *dbq.Queries, logger *slog.L
func dormantRowsToTracks(rows []dbq.ListDormantArtistTracksForDiscoverRow) []discoverTrack {
out := make([]discoverTrack, len(rows))
for i, r := range rows {
out[i] = discoverTrack{ID: r.ID, AlbumID: r.AlbumID, ArtistID: r.ArtistID}
out[i] = discoverTrack{
ID: r.ID, AlbumID: r.AlbumID, ArtistID: r.ArtistID,
PickKind: pickKindDormant,
}
}
return out
}
@@ -107,7 +114,10 @@ func dormantRowsToTracks(rows []dbq.ListDormantArtistTracksForDiscoverRow) []dis
func crossUserRowsToTracks(rows []dbq.ListCrossUserLikedTracksForDiscoverRow) []discoverTrack {
out := make([]discoverTrack, len(rows))
for i, r := range rows {
out[i] = discoverTrack{ID: r.ID, AlbumID: r.AlbumID, ArtistID: r.ArtistID}
out[i] = discoverTrack{
ID: r.ID, AlbumID: r.AlbumID, ArtistID: r.ArtistID,
PickKind: pickKindCrossUser,
}
}
return out
}
@@ -115,7 +125,10 @@ func crossUserRowsToTracks(rows []dbq.ListCrossUserLikedTracksForDiscoverRow) []
func randomRowsToTracks(rows []dbq.ListRandomUnheardTracksForDiscoverRow) []discoverTrack {
out := make([]discoverTrack, len(rows))
for i, r := range rows {
out[i] = discoverTrack{ID: r.ID, AlbumID: r.AlbumID, ArtistID: r.ArtistID}
out[i] = discoverTrack{
ID: r.ID, AlbumID: r.AlbumID, ArtistID: r.ArtistID,
PickKind: pickKindRandom,
}
}
return out
}
+24
View File
@@ -161,3 +161,27 @@ func TestInterleaveBuckets_DedupsAcrossBuckets(t *testing.T) {
}
}
}
func TestInterleaveBuckets_PreservesBucketPickKind(t *testing.T) {
// Bucket provenance (#1270) is stamped on discoverTrack before the
// interleave and must survive it — including dedup, where a track in
// two buckets keeps the stamp of the bucket it was taken from.
dormant := []discoverTrack{
{ID: uuidN(1), PickKind: pickKindDormant},
{ID: uuidN(2), PickKind: pickKindDormant},
}
random := []discoverTrack{
{ID: uuidN(1), PickKind: pickKindRandom}, // shared with dormant
{ID: uuidN(3), PickKind: pickKindRandom},
}
got := interleaveBuckets(dormant, nil, random)
if len(got) != 3 {
t.Fatalf("len = %d, want 3", len(got))
}
want := map[byte]string{1: pickKindDormant, 2: pickKindDormant, 3: pickKindRandom}
for _, tr := range got {
if w := want[tr.ID.Bytes[15]]; tr.PickKind != w {
t.Errorf("track %d pick_kind = %q, want %q", tr.ID.Bytes[15], tr.PickKind, w)
}
}
}
+195 -8
View File
@@ -99,7 +99,7 @@ func TestPickHeadAndTail_SmallPool(t *testing.T) {
makeCand(2, 11, 101, 0.9),
makeCand(3, 12, 102, 0.8),
}
got := pickHeadAndTail(in, testUserID, "2026-05-07", time.Now(), 5, 2)
got := pickHeadAndTail(in, nil, 1, testUserID, "2026-05-07", time.Now(), 5, 2)
if len(got) != 3 {
t.Errorf("len = %d, want 3 (pool too small for head/tail split)", len(got))
}
@@ -112,7 +112,7 @@ func TestPickHeadAndTail_ExactlyTotal(t *testing.T) {
for i := 0; i < 7; i++ {
in = append(in, makeCand(i+1, i+1, i+1, float64(7-i)))
}
got := pickHeadAndTail(in, testUserID, "2026-05-07", time.Now(), 5, 2)
got := pickHeadAndTail(in, nil, 1, testUserID, "2026-05-07", time.Now(), 5, 2)
if len(got) != 7 {
t.Errorf("len = %d, want 7 (pool == total)", len(got))
}
@@ -126,7 +126,7 @@ func TestPickHeadAndTail_HeadAndTailSplit(t *testing.T) {
for i := 0; i < 100; i++ {
in = append(in, makeCand(i+1, i+1, i+1, float64(100-i)))
}
got := pickHeadAndTail(in, testUserID, "2026-05-07", time.Now(), 20, 5)
got := pickHeadAndTail(in, nil, 1, testUserID, "2026-05-07", time.Now(), 20, 5)
if len(got) != 25 {
t.Errorf("len = %d, want 25 (20 head + 5 tail)", len(got))
}
@@ -139,8 +139,8 @@ func TestPickHeadAndTail_Determinism(t *testing.T) {
in = append(in, makeCand(i+1, i+1, i+1, float64(100-i)))
}
now := time.Now()
got1 := pickHeadAndTail(in, testUserID, "2026-05-07", now, 20, 5)
got2 := pickHeadAndTail(in, testUserID, "2026-05-07", now, 20, 5)
got1 := pickHeadAndTail(in, nil, 1, testUserID, "2026-05-07", now, 20, 5)
got2 := pickHeadAndTail(in, nil, 1, testUserID, "2026-05-07", now, 20, 5)
if len(got1) != len(got2) {
t.Fatalf("len mismatch: %d vs %d", len(got1), len(got2))
}
@@ -161,8 +161,8 @@ func TestPickHeadAndTail_HeadStable_TailVariesAcrossDays(t *testing.T) {
in = append(in, makeCand(i+1, i+1, i+1, float64(100-i)))
}
now := time.Now()
day1 := pickHeadAndTail(in, testUserID, "2026-05-07", now, 20, 5)
day2 := pickHeadAndTail(in, testUserID, "2026-05-08", now, 20, 5)
day1 := pickHeadAndTail(in, nil, 1, testUserID, "2026-05-07", now, 20, 5)
day2 := pickHeadAndTail(in, nil, 1, testUserID, "2026-05-08", now, 20, 5)
if len(day1) != 25 || len(day2) != 25 {
t.Fatalf("len mismatch: day1=%d day2=%d", len(day1), len(day2))
}
@@ -205,7 +205,7 @@ func TestPickHeadAndTail_TailFromBeyond2xHeadN(t *testing.T) {
sim := float64(50 - i)
in = append(in, makeCand(i+1, i+1, i+1, sim))
}
got := pickHeadAndTail(in, testUserID, "2026-05-07", time.Now(), 5, 3)
got := pickHeadAndTail(in, nil, 1, testUserID, "2026-05-07", time.Now(), 5, 3)
if len(got) != 8 {
t.Fatalf("len = %d, want 8 (5 head + 3 tail)", len(got))
}
@@ -243,3 +243,190 @@ func TestPickTopN_DiversityCap(t *testing.T) {
t.Errorf("len = %d, want 3 (artist 100 capped at 3)", len(got))
}
}
func TestPickHeadAndTail_MarksPickKinds(t *testing.T) {
// 100-deep pool with headN=20, tailN=5: the 20 head entries are the
// taste picks, the 5 tail entries the freshness injection (#1249).
in := make([]recommendation.Candidate, 0, 100)
for i := 0; i < 100; i++ {
in = append(in, makeCand(i+1, i+1, i+1, float64(100-i)))
}
got := pickHeadAndTail(in, nil, 1, testUserID, "2026-05-07", time.Now(), 20, 5)
if len(got) != 25 {
t.Fatalf("len = %d, want 25", len(got))
}
for i := 0; i < 20; i++ {
if got[i].PickKind != pickKindTaste {
t.Errorf("head[%d].PickKind = %q, want %q", i, got[i].PickKind, pickKindTaste)
}
}
for i := 20; i < 25; i++ {
if got[i].PickKind != pickKindFresh {
t.Errorf("tail[%d].PickKind = %q, want %q", i-20, got[i].PickKind, pickKindFresh)
}
}
}
func TestPickHeadAndTail_SmallPoolAllTaste(t *testing.T) {
// The small-pool fallback is pure top-N-by-score — that IS the taste
// mechanism, so nothing on this path is an exploration pick.
in := []recommendation.Candidate{
makeCand(1, 10, 100, 1.0),
makeCand(2, 11, 101, 0.9),
makeCand(3, 12, 102, 0.8),
}
got := pickHeadAndTail(in, nil, 1, testUserID, "2026-05-07", time.Now(), 5, 2)
for i, rc := range got {
if rc.PickKind != pickKindTaste {
t.Errorf("got[%d].PickKind = %q, want %q (fallback is all taste)",
i, rc.PickKind, pickKindTaste)
}
}
}
func TestPickTopN_NoPickKind(t *testing.T) {
// Songs-like-X and the discovery mixes don't split; their rows must
// persist pick_kind NULL (empty string here).
in := []recommendation.Candidate{makeCand(1, 10, 100, 1.0)}
got := pickTopN(in, testUserID, "2026-05-07", time.Now(), 25)
if len(got) != 1 || got[0].PickKind != "" {
t.Errorf("pickTopN PickKind = %q, want empty (persists as NULL)", got[0].PickKind)
}
}
func TestHeadQuotas(t *testing.T) {
cases := []struct {
seeds, headN int
want []int
}{
{1, 50, []int{50}},
{2, 50, []int{30, 20}},
{3, 50, []int{25, 15, 10}},
{3, 7, []int{4, 2, 1}}, // rounding remainder → primary seed
}
for _, c := range cases {
got := headQuotas(c.seeds, c.headN)
if len(got) != len(c.want) {
t.Fatalf("headQuotas(%d, %d) len = %d, want %d", c.seeds, c.headN, len(got), len(c.want))
}
sum := 0
for i := range got {
sum += got[i]
if got[i] != c.want[i] {
t.Errorf("headQuotas(%d, %d)[%d] = %d, want %d", c.seeds, c.headN, i, got[i], c.want[i])
}
}
if sum != c.headN {
t.Errorf("headQuotas(%d, %d) sums to %d, want %d", c.seeds, c.headN, sum, c.headN)
}
}
}
func TestPickQuotaHead_BlendsSeeds(t *testing.T) {
// Seed 0's neighborhood out-scores everything (#1269): without
// quotas the head would be all seed-0. With headN=10 and quotas
// [5,3,2], the walk takes the best 5 from seed 0, then seeds 1 and
// 2 get their guaranteed slots from deeper ranks.
in := make([]recommendation.Candidate, 0, 30)
seedOf := map[pgtype.UUID]int{}
for i := 0; i < 30; i++ {
c := makeCand(i+1, i+1, i+1, float64(30-i))
si := 0
if i >= 20 && i < 25 {
si = 1
} else if i >= 25 {
si = 2
}
seedOf[c.Track.ID] = si
in = append(in, c)
}
// Input is already score-descending and cap-free.
head := pickQuotaHead(in, seedOf, 3, 10)
if len(head) != 10 {
t.Fatalf("head len = %d, want 10", len(head))
}
want := []byte{1, 2, 3, 4, 5, 21, 22, 23, 26, 27}
for i, w := range want {
if head[i].Track.ID.Bytes[15] != w {
t.Errorf("head[%d] = track %d, want %d", i, head[i].Track.ID.Bytes[15], w)
}
}
}
func TestPickQuotaHead_ThinSeedSpillsBestFirst(t *testing.T) {
// Seed 1 has a single candidate; its unfilled quota goes to the
// best remaining candidates regardless of seed.
in := make([]recommendation.Candidate, 0, 12)
seedOf := map[pgtype.UUID]int{}
for i := 0; i < 12; i++ {
c := makeCand(i+1, i+1, i+1, float64(12-i))
si := 0
if i == 11 {
si = 1 // the worst-ranked candidate is seed 1's only one
}
seedOf[c.Track.ID] = si
in = append(in, c)
}
head := pickQuotaHead(in, seedOf, 2, 10)
if len(head) != 10 {
t.Fatalf("head len = %d, want 10", len(head))
}
// Quotas [6,4]: seed 0 fills 6 (tracks 1-6), seed 1 fills 1
// (track 12), spill tops up with tracks 7-9.
got := map[byte]bool{}
for _, c := range head {
got[c.Track.ID.Bytes[15]] = true
}
for _, w := range []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 12} {
if !got[w] {
t.Errorf("head missing track %d (spill should fill best-first)", w)
}
}
}
func TestPickWeightedTail_PrefersEarlyRanks(t *testing.T) {
// The fresh tail's sample is rank-weighted (#1269): across many
// days, the mean selected rank must sit well below the uniform
// draw's ~99.5 for a 200-deep pool.
pool := make([]recommendation.Candidate, 0, 200)
for i := 0; i < 200; i++ {
c := makeCand(1, i+1, i+1, 1.0)
// Distinct track IDs beyond one byte: bytes wrap at 256.
c.Track.ID.Bytes[14] = byte(i / 200)
c.Track.ID.Bytes[15] = byte(i % 200)
pool = append(pool, c)
}
rankOf := map[pgtype.UUID]int{}
for i, c := range pool {
rankOf[c.Track.ID] = i
}
totalRank, picks := 0, 0
for day := 1; day <= 28; day++ {
got := pickWeightedTail(pool, "2026-06-"+twoDigits(day), 10)
if len(got) != 10 {
t.Fatalf("day %d: len = %d, want 10", day, len(got))
}
for _, c := range got {
totalRank += rankOf[c.Track.ID]
picks++
}
}
mean := float64(totalRank) / float64(picks)
if mean > 80 {
t.Errorf("mean selected rank = %.1f; want well below uniform ~99.5", mean)
}
}
func TestPickWeightedTail_DeterministicWithinDay(t *testing.T) {
pool := make([]recommendation.Candidate, 0, 60)
for i := 0; i < 60; i++ {
pool = append(pool, makeCand(i+1, i+1, i+1, 1.0))
}
a := pickWeightedTail(pool, "2026-06-05", 8)
b := pickWeightedTail(pool, "2026-06-05", 8)
for i := range a {
if a[i].Track.ID != b[i].Track.ID {
t.Fatal("weighted tail must be deterministic within a day")
}
}
}
+1 -1
View File
@@ -236,7 +236,7 @@ func (s *Scheduler) runStartupCatchUp(ctx context.Context, users []dbq.ListActiv
// build can read a fresh profile (phase-2 consumption); both steps are
// best-effort and a failure in one is logged without blocking the other.
func (s *Scheduler) rebuildUserDaily(ctx context.Context, userID pgtype.UUID, now time.Time) {
if err := taste.BuildTasteProfile(ctx, s.pool, s.logger, userID, taste.DefaultConfig()); err != nil {
if err := taste.BuildTasteProfile(ctx, s.pool, s.logger, userID, currentTasteConfig()); err != nil {
s.logger.Warn("scheduler: taste profile rebuild failed",
"user_id", uuidStringPL(userID), "err", err)
}
+49 -24
View File
@@ -46,6 +46,26 @@ func TestPickSeedArtistsFromRows_Empty(t *testing.T) {
}
}
func TestPickKindForSeedTier(t *testing.T) {
// The seed query's fallback tiers map onto the rule-#131 ladder:
// fresh 7-day engagement is the exact desire, everything past the
// 30-day step-back collapses into the far tier (#1255).
cases := []struct {
tier int32
want string
}{
{0, pickKindTier1},
{1, pickKindTier2},
{2, pickKindTier3},
{3, pickKindTier3},
}
for _, c := range cases {
if got := pickKindForSeedTier(c.tier); got != c.want {
t.Errorf("pickKindForSeedTier(%d) = %q, want %q", c.tier, got, c.want)
}
}
}
// userIDHash is the per-user, per-day hash that drives the daily-
// determinism RNGs. Same family as tieBreakHash, just keyed on user
// ID instead of track ID.
@@ -76,60 +96,65 @@ func TestUserIDHash_DifferentUserChangesHash(t *testing.T) {
}
}
// pickForYouSeedForDay rotates the chosen For-You seed across the
// user's top-played candidates using userIDHash. Verifies the picker
// is deterministic within a day, varies across days, and degrades
// gracefully when fewer than 5 candidates exist.
// pickDailySeeds shuffles the candidate pool daily-deterministically
// and takes up to n — For-You uses n=forYouSeedCount for its
// multi-seed blend (#1269), Songs-like uses n=3 via the
// pickSeedArtistsForDay wrapper. Verifies determinism within a day,
// variation across days, and graceful degradation on small pools.
func TestPickForYouSeedForDay_DeterministicWithinDay(t *testing.T) {
func TestPickDailySeeds_DeterministicWithinDay(t *testing.T) {
u := pgtype.UUID{Bytes: [16]byte{1}, Valid: true}
seeds := []pgtype.UUID{
pool := []pgtype.UUID{
{Bytes: [16]byte{10}, Valid: true},
{Bytes: [16]byte{20}, Valid: true},
{Bytes: [16]byte{30}, Valid: true},
{Bytes: [16]byte{40}, Valid: true},
{Bytes: [16]byte{50}, Valid: true},
}
a := pickForYouSeedForDay(seeds, u, "2026-05-04")
b := pickForYouSeedForDay(seeds, u, "2026-05-04")
if a != b {
t.Fatalf("same day should pick same seed; got %v then %v", a, b)
a := pickDailySeeds(pool, u, "2026-05-04", 3)
b := pickDailySeeds(pool, u, "2026-05-04", 3)
if len(a) != 3 || len(b) != 3 {
t.Fatalf("lens = %d, %d; want 3, 3", len(a), len(b))
}
for i := range a {
if a[i] != b[i] {
t.Fatalf("same day should pick same seeds; got %v then %v", a, b)
}
}
}
func TestPickForYouSeedForDay_VariesAcrossDays(t *testing.T) {
func TestPickDailySeeds_VariesAcrossDays(t *testing.T) {
u := pgtype.UUID{Bytes: [16]byte{1}, Valid: true}
seeds := []pgtype.UUID{
pool := []pgtype.UUID{
{Bytes: [16]byte{10}, Valid: true},
{Bytes: [16]byte{20}, Valid: true},
{Bytes: [16]byte{30}, Valid: true},
{Bytes: [16]byte{40}, Valid: true},
{Bytes: [16]byte{50}, Valid: true},
}
picks := map[[16]byte]bool{}
firstPicks := map[[16]byte]bool{}
for i := 1; i <= 30; i++ {
date := "2026-05-" + twoDigits(i)
picks[pickForYouSeedForDay(seeds, u, date).Bytes] = true
firstPicks[pickDailySeeds(pool, u, date, 3)[0].Bytes] = true
}
if len(picks) < 2 {
t.Errorf("expected >=2 distinct seeds across 30 days; got %d", len(picks))
if len(firstPicks) < 2 {
t.Errorf("expected >=2 distinct lead seeds across 30 days; got %d", len(firstPicks))
}
}
func TestPickForYouSeedForDay_SingleCandidate(t *testing.T) {
func TestPickDailySeeds_SmallPool(t *testing.T) {
u := pgtype.UUID{Bytes: [16]byte{1}, Valid: true}
only := pgtype.UUID{Bytes: [16]byte{99}, Valid: true}
got := pickForYouSeedForDay([]pgtype.UUID{only}, u, "2026-05-04")
if got != only {
t.Errorf("single-seed pool should return that seed; got %v", got)
got := pickDailySeeds([]pgtype.UUID{only}, u, "2026-05-04", 3)
if len(got) != 1 || got[0] != only {
t.Errorf("single-entry pool should return just that entry; got %v", got)
}
}
func TestPickForYouSeedForDay_EmptyPool(t *testing.T) {
func TestPickDailySeeds_EmptyPool(t *testing.T) {
u := pgtype.UUID{Bytes: [16]byte{1}, Valid: true}
got := pickForYouSeedForDay(nil, u, "2026-05-04")
if got.Valid {
t.Errorf("empty pool should return zero UUID; got %v", got)
if got := pickDailySeeds(nil, u, "2026-05-04", 3); len(got) != 0 {
t.Errorf("empty pool should return nothing; got %v", got)
}
}
+337 -85
View File
@@ -12,8 +12,10 @@ import (
"errors"
"fmt"
"log/slog"
"math"
"math/rand"
"sort"
"sync"
"time"
"github.com/jackc/pgx/v5"
@@ -22,6 +24,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/recommendation"
"git.fabledsword.com/bvandeusen/minstrel/internal/taste"
)
// seedArtistRow mirrors the sqlc-generated PickSeedArtistsRow shape.
@@ -83,32 +86,15 @@ func userIDHash(userID pgtype.UUID, dateStr string) uint64 {
return binary.BigEndian.Uint64(sum[:8])
}
// pickForYouSeedForDay picks one of the user's top-played candidate
// tracks as today's For-You seed. With 5 candidates and SHA-256-based
// rotation, each candidate gets picked roughly 1 day in 5; the chosen
// seed drives the similarity candidate pool, so the resulting mix is
// substantially different across days for a user with diverse top-N
// plays.
//
// Degrades gracefully: 1 candidate → returns it; 0 → returns zero UUID
// (caller treats as "no seed available" and skips For-You).
func pickForYouSeedForDay(seeds []pgtype.UUID, userID pgtype.UUID, dateStr string) pgtype.UUID {
if len(seeds) == 0 {
return pgtype.UUID{}
}
idx := int(userIDHash(userID, dateStr) % uint64(len(seeds)))
return seeds[idx]
}
// pickSeedArtistsForDay takes a candidate pool of up to N artists and
// returns up to 3 of them, daily-deterministically shuffled. Each day
// gets a different ordering / selection, but within-day stability is
// preserved (same inputs always produce the same output).
// pickDailySeeds takes a candidate pool of UUIDs and returns up to n
// of them, daily-deterministically shuffled. Each day gets a different
// ordering / selection, but within-day stability is preserved (same
// inputs always produce the same output).
//
// Uses Fisher-Yates over a copy of the input, seeded by userIDHash.
// Degrades gracefully: <3 candidates returns whatever is available in
// Degrades gracefully: <n candidates returns whatever is available in
// shuffled order.
func pickSeedArtistsForDay(pool []pgtype.UUID, userID pgtype.UUID, dateStr string) []pgtype.UUID {
func pickDailySeeds(pool []pgtype.UUID, userID pgtype.UUID, dateStr string, n int) []pgtype.UUID {
if len(pool) == 0 {
return nil
}
@@ -118,41 +104,137 @@ func pickSeedArtistsForDay(pool []pgtype.UUID, userID pgtype.UUID, dateStr strin
rng.Shuffle(len(shuffled), func(i, j int) {
shuffled[i], shuffled[j] = shuffled[j], shuffled[i]
})
n := 3
if len(shuffled) < n {
n = len(shuffled)
}
return shuffled[:n]
}
// pickSeedArtistsForDay picks up to 3 seed artists for the Songs-like
// mixes; For-You uses pickDailySeeds directly with forYouSeedCount.
func pickSeedArtistsForDay(pool []pgtype.UUID, userID pgtype.UUID, dateStr string) []pgtype.UUID {
return pickDailySeeds(pool, userID, dateStr, 3)
}
// rankedCandidate is a (track_id, score) pair used during in-memory
// sorting before insert into playlist_tracks. T5 fills these from
// recommendation.Candidate scores.
// recommendation.Candidate scores. PickKind is the track's provenance
// within its mix — For-You's head/tail split (#1249) and Discover's
// buckets (#1270); empty persists as NULL (variant doesn't stamp yet).
type rankedCandidate struct {
TrackID pgtype.UUID
Score float64
TrackID pgtype.UUID
Score float64
PickKind string
}
// Pick kinds, persisted on playlist_tracks.pick_kind so plays can
// attribute skips to the population that sourced the track — For You's
// "taste engine missing" vs "freshness tax", Discover's three buckets
// (#1270). Values mirror the CHECK in migration 0039; the tier1-3
// values there land with the tiered-mix rebuilds.
const (
pickKindTaste = "taste"
pickKindFresh = "fresh"
pickKindDormant = "dormant"
pickKindCrossUser = "cross_user"
pickKindRandom = "random"
pickKindTier1 = "tier1"
pickKindTier2 = "tier2"
pickKindTier3 = "tier3"
)
// pickKindForSeedTier maps PickSeedArtists' seed-pool fallback tier
// onto the rule-#131 pick-kind ladder: fresh 7-day engagement pins the
// mix's exact desire (tier1), the 30-day window steps back a little
// (tier2), and all-time / liked-only seeds are the far fallback
// (tier3). Stamped mix-wide — seed staleness is a property of the
// whole build, and it's what the metrics breakdown attributes skips to.
func pickKindForSeedTier(tier int32) string {
switch tier {
case 0:
return pickKindTier1
case 1:
return pickKindTier2
default:
return pickKindTier3
}
}
// pickKindForMixTier maps a tiered mix query's 1-based tier column
// (numbered to match rule #131's ladder directly) onto the pick-kind
// vocabulary. Distinct from pickKindForSeedTier, whose 0-based tiers
// count fallback steps of the seed pool rather than eligibility rungs.
func pickKindForMixTier(tier int32) string {
switch tier {
case 1:
return pickKindTier1
case 2:
return pickKindTier2
default:
return pickKindTier3
}
}
const systemMixLength = 25
// systemMixWeights are the fixed scoring weights used by the cron worker.
// systemMixWeights are the scoring weights used by the daily builds.
// JitterMagnitude is small (0.1) and combined with a userIDHash-seeded
// RNG (see scoreAndSortCandidates) — same (user, day) produces same
// scores within a day, but near-tied candidates reshuffle across days
// so the playlist doesn't feel frozen.
var systemMixWeights = recommendation.ScoringWeights{
BaseWeight: 1.0,
LikeBoost: 2.0,
RecencyWeight: 1.0,
SkipPenalty: 2.0,
JitterMagnitude: 0.1,
ContextWeight: 0.5,
SimilarityWeight: 1.5,
// Taste profile (#796 phase 2): the daily mixes are the primary
// taste-driven surface, so they lean on it. TasteMatchScore is in
// [-1,+1], so 1.5 makes a strong taste fit comparable to a like boost
// while passive avoidance (negative) gently demotes.
TasteWeight: 1.5,
//
// DB-tunable since #1250: the recsettings service pushes the current
// daily_mix profile via SetSystemMixWeights at boot and on every admin
// change (coverart Configure() pattern — no signature threading, live
// effect without restart). The literal here is only the pre-push
// value; shipped defaults live in recsettings.ShippedDailyMixWeights,
// which must stay in sync with it.
var (
systemTuningMu sync.RWMutex
systemMixWeights = recommendation.ScoringWeights{
BaseWeight: 1.0,
LikeBoost: 2.0,
RecencyWeight: 1.0,
SkipPenalty: 2.0,
JitterMagnitude: 0.1,
ContextWeight: 0.5,
SimilarityWeight: 1.5,
// Taste profile (#796 phase 2): the daily mixes are the primary
// taste-driven surface, so they lean on it. TasteMatchScore is in
// [-1,+1], so 1.5 makes a strong taste fit comparable to a like boost
// while passive avoidance (negative) gently demotes.
TasteWeight: 1.5,
}
systemTasteConfig = taste.DefaultConfig()
)
// SetSystemMixWeights installs the current daily_mix scoring weights.
// Called by the recsettings service at boot and on admin updates.
func SetSystemMixWeights(w recommendation.ScoringWeights) {
systemTuningMu.Lock()
defer systemTuningMu.Unlock()
systemMixWeights = w
}
// SetTasteConfig installs the taste-profile build configuration
// (half-life + engagement curve, #1250). Same push model as
// SetSystemMixWeights.
func SetTasteConfig(c taste.Config) {
systemTuningMu.Lock()
defer systemTuningMu.Unlock()
systemTasteConfig = c
}
func currentSystemMixWeights() recommendation.ScoringWeights {
systemTuningMu.RLock()
defer systemTuningMu.RUnlock()
return systemMixWeights
}
func currentTasteConfig() taste.Config {
systemTuningMu.RLock()
defer systemTuningMu.RUnlock()
return systemTasteConfig
}
// forYouHeadN is the number of top-scored tracks that anchor the For-You
@@ -175,6 +257,122 @@ const (
forYouTailN = 50
)
// forYouSeedCount is how many of the user's top-5 tracks seed each
// day's build (#1269). A single rotating seed made the mix bipolar
// day-to-day — swinging whole neighborhoods as the rotation moved
// between dissimilar anchors; blending 3 spans neighborhoods within
// one day and smooths the swing across days.
const forYouSeedCount = 3
// headQuotas fixes each seed's share of the For-You head so one
// neighborhood can't monopolize it even when its candidates out-score
// the others (#1269): 50/30/20 for three seeds, 60/40 for two, all of
// it for one. Rounding remainder goes to the primary seed; a thin
// seed's unfilled quota spills best-first in pickQuotaHead.
func headQuotas(numSeeds, headN int) []int {
if numSeeds <= 1 {
return []int{headN}
}
fractions := []float64{0.5, 0.3, 0.2}
if numSeeds == 2 {
fractions = []float64{0.6, 0.4}
}
if numSeeds > len(fractions) {
// Not reachable at forYouSeedCount = 3; even split keeps any
// future seed-count change from silently starving seeds.
fractions = make([]float64, numSeeds)
for i := range fractions {
fractions[i] = 1.0 / float64(numSeeds)
}
}
quotas := make([]int, numSeeds)
assigned := 0
for i := 0; i < numSeeds; i++ {
quotas[i] = int(float64(headN) * fractions[i])
assigned += quotas[i]
}
quotas[0] += headN - assigned
return quotas
}
// pickQuotaHead walks the score-sorted capped pool best-first, taking
// candidates whose originating seed still has head quota; slots a thin
// seed can't fill spill best-first regardless of seed in a second
// pass. A nil seedOf (single-seed path, tests) attributes everything
// to seed 0, which reduces to plain top-headN.
func pickQuotaHead(capped []recommendation.Candidate, seedOf map[pgtype.UUID]int, numSeeds, headN int) []recommendation.Candidate {
quotas := headQuotas(numSeeds, headN)
taken := make([]int, len(quotas))
inHead := make(map[pgtype.UUID]bool, headN)
head := make([]recommendation.Candidate, 0, headN)
for _, c := range capped {
if len(head) >= headN {
break
}
si := seedOf[c.Track.ID]
if si < 0 || si >= len(quotas) {
si = 0
}
if taken[si] >= quotas[si] {
continue
}
taken[si]++
head = append(head, c)
inHead[c.Track.ID] = true
}
for _, c := range capped {
if len(head) >= headN {
break
}
if inHead[c.Track.ID] {
continue
}
head = append(head, c)
inHead[c.Track.ID] = true
}
return head
}
// forYouTailHalfLifeRanks tunes the fresh tail's rank bias (#1269): a
// candidate's sampling weight halves every 50 ranks, so the 101st-best
// candidate is far likelier than the 401st-best instead of equal —
// freshness keeps its "you'll probably enjoy this" half.
const forYouTailHalfLifeRanks = 50.0
// tailSampleUniverse quantizes tieBreakHash into a uniform (0,1] draw
// for the weighted sample; 1e6 buckets is plenty of resolution for
// pools of a few hundred.
const tailSampleUniverse uint64 = 1_000_000
// pickWeightedTail samples up to tailN candidates from the rank-
// ordered tail pool with exponentially rank-decaying weights, using
// deterministic Efraimidis-Spirakis keys (u^(1/w), u derived from
// tieBreakHash): same (track, day) → same key, so the sample is stable
// within a day and rotates across days. Replaces the uniform daily
// draw where rank 380 had the same chance as rank 101.
func pickWeightedTail(tailPool []recommendation.Candidate, dateStr string, tailN int) []recommendation.Candidate {
if len(tailPool) <= tailN {
return tailPool
}
type keyed struct {
c recommendation.Candidate
key float64
}
keys := make([]keyed, len(tailPool))
for i, c := range tailPool {
u := (float64(tieBreakHash(c.Track.ID, dateStr)%tailSampleUniverse) + 1) /
float64(tailSampleUniverse+1)
w := math.Exp2(-float64(i) / forYouTailHalfLifeRanks)
keys[i] = keyed{c: c, key: math.Pow(u, 1/w)}
}
sort.SliceStable(keys, func(i, j int) bool { return keys[i].key > keys[j].key })
out := make([]recommendation.Candidate, tailN)
for i := 0; i < tailN; i++ {
out[i] = keys[i].c
}
return out
}
// scoreAndSortCandidates scores every candidate with recommendation.Score
// and returns a new slice sorted by score DESC (ties broken by
// tieBreakHash). The scoring RNG is seeded by userIDHash so jitter is
@@ -199,9 +397,10 @@ func scoreAndSortCandidates(cands []recommendation.Candidate, userID pgtype.UUID
sort.SliceStable(ordered, func(i, j int) bool {
return uuidLessPL(ordered[i].Track.ID, ordered[j].Track.ID)
})
weights := currentSystemMixWeights()
pairs := make([]scored, len(ordered))
for i, c := range ordered {
pairs[i] = scored{c: c, score: recommendation.Score(c.Inputs, systemMixWeights, now, rng.Float64)}
pairs[i] = scored{c: c, score: recommendation.Score(c.Inputs, weights, now, rng.Float64)}
}
sort.SliceStable(pairs, func(i, j int) bool {
if pairs[i].score != pairs[j].score {
@@ -336,10 +535,11 @@ func systemForYouSourceLimits() recommendation.CandidateSourceLimits {
}
}
// produceForYou: today's seed from the user's top-5 played tracks
// (rotates daily via userIDHash), similarity candidate pool, head+
// tail composition. The base seed query failing is fatal; a
// candidate-load failure is logged and yields no For-You.
// produceForYou: blend candidate pools from up to forYouSeedCount of
// the user's top-5 played tracks (rotating daily via pickDailySeeds),
// then head+tail composition with per-seed head quotas (#1269). The
// base seed query failing is fatal; a per-seed candidate-load failure
// is logged and that seed just contributes nothing.
func produceForYou(
ctx context.Context, q *dbq.Queries, logger *slog.Logger,
userID pgtype.UUID, dateStr string, now time.Time,
@@ -348,24 +548,40 @@ func produceForYou(
if err != nil {
return nil, fmt.Errorf("pick for-you seed candidates: %w", err)
}
forYouSeed := pickForYouSeedForDay(forYouSeeds, userID, dateStr)
if !forYouSeed.Valid {
seeds := pickDailySeeds(forYouSeeds, userID, dateStr, forYouSeedCount)
if len(seeds) == 0 {
return nil, nil
}
zeroVec := recommendation.SessionVector{Seed: true}
cands, cerr := recommendation.LoadCandidatesFromSimilarity(
ctx, q, userID, forYouSeed,
1, // recentlyPlayedHours — small to avoid filtering the seed's recent neighbourhood
zeroVec,
[]pgtype.UUID{forYouSeed},
systemForYouSourceLimits(),
)
if cerr != nil {
logger.Warn("system playlist: for-you candidates load failed; skipping",
"user_id", uuidStringPL(userID), "err", cerr)
// Merge per-seed pools; first-seen wins on dedup, and seedOf
// remembers which seed sourced each track for the head quotas.
var merged []recommendation.Candidate
seedOf := map[pgtype.UUID]int{}
for i, seed := range seeds {
cands, cerr := recommendation.LoadCandidatesFromSimilarity(
ctx, q, userID, seed,
1, // recentlyPlayedHours — small to avoid filtering the seed's recent neighbourhood
zeroVec,
seeds,
systemForYouSourceLimits(),
)
if cerr != nil {
logger.Warn("system playlist: for-you candidates load failed for seed; continuing",
"user_id", uuidStringPL(userID), "seed", uuidStringPL(seed), "err", cerr)
continue
}
for _, c := range cands {
if _, seen := seedOf[c.Track.ID]; seen {
continue
}
seedOf[c.Track.ID] = i
merged = append(merged, c)
}
}
if len(merged) == 0 {
return nil, nil
}
tracks := pickHeadAndTail(cands, userID, dateStr, now, forYouHeadN, forYouTailN)
tracks := pickHeadAndTail(merged, seedOf, len(seeds), userID, dateStr, now, forYouHeadN, forYouTailN)
if len(tracks) == 0 {
return nil, nil
}
@@ -373,8 +589,11 @@ func produceForYou(
}
// produceSeedMixes: up to 3 "Songs like {artist}" mixes. Seed
// artists rotate daily-deterministically. The base seed-artist
// query failing is fatal; per-artist failures are logged + skipped.
// artists rotate daily-deterministically; the seed query falls back
// through widening engagement windows (#1255) and every returned row
// shares the winning tier, stamped onto the built tracks as their
// pick_kind. The base seed-artist query failing is fatal; per-artist
// failures are logged + skipped.
func produceSeedMixes(
ctx context.Context, q *dbq.Queries, logger *slog.Logger,
userID pgtype.UUID, dateStr string, now time.Time,
@@ -383,6 +602,10 @@ func produceSeedMixes(
if err != nil {
return nil, fmt.Errorf("pick seed artists: %w", err)
}
seedTierKind := ""
if len(seedRows) > 0 {
seedTierKind = pickKindForSeedTier(seedRows[0].Tier)
}
seedRowsLocal := make([]seedArtistRow, 0, len(seedRows))
for _, r := range seedRows {
seedRowsLocal = append(seedRowsLocal, seedArtistRow{
@@ -427,6 +650,9 @@ func produceSeedMixes(
if len(tracks) == 0 {
continue
}
for i := range tracks {
tracks[i].PickKind = seedTierKind
}
out = append(out, builtPlaylist{
Name: fmt.Sprintf("Songs like %s", artistRow.Name),
Variant: "songs_like_artist",
@@ -601,20 +827,22 @@ func pickTopN(cands []recommendation.Candidate, userID pgtype.UUID, dateStr stri
capped = capped[:n]
}
rng := rand.New(rand.NewSource(int64(userIDHash(userID, dateStr))))
weights := currentSystemMixWeights()
out := make([]rankedCandidate, len(capped))
for i, c := range capped {
out[i] = rankedCandidate{
TrackID: c.Track.ID,
Score: recommendation.Score(c.Inputs, systemMixWeights, now, rng.Float64),
Score: recommendation.Score(c.Inputs, weights, now, rng.Float64),
}
}
return out
}
// pickHeadAndTail picks headN from the score-sorted head plus tailN from
// positions 2*headN onward (the tail), with the tail sampled
// daily-deterministically via tieBreakHash. Caps applied before the
// head/tail split. Used by For-You only.
// pickHeadAndTail picks headN taste anchors from the score-sorted pool
// (under per-seed quotas when seedOf/numSeeds describe a multi-seed
// blend, #1269) plus tailN freshness picks sampled rank-weighted from
// positions 2*headN onward. Caps applied before the head/tail split.
// Used by For-You only.
//
// The "tail" — candidates ranked beyond 2*headN — is still similarity-
// related (every candidate passed the similarity filter) but isn't among
@@ -624,58 +852,76 @@ func pickTopN(cands []recommendation.Candidate, userID pgtype.UUID, dateStr stri
// Falls back to standard pickTopN behavior when the candidate pool is too
// small to support a meaningful head/tail split (capped pool <=
// headN+tailN, or no candidates at or beyond position 2*headN).
func pickHeadAndTail(cands []recommendation.Candidate, userID pgtype.UUID, dateStr string, now time.Time, headN, tailN int) []rankedCandidate {
func pickHeadAndTail(
cands []recommendation.Candidate, seedOf map[pgtype.UUID]int, numSeeds int,
userID pgtype.UUID, dateStr string, now time.Time, headN, tailN int,
) []rankedCandidate {
sorted := scoreAndSortCandidates(cands, userID, dateStr, now)
capped := capCandidatesByAlbumAndArtist(sorted)
rng := rand.New(rand.NewSource(int64(userIDHash(userID, dateStr))))
weights := currentSystemMixWeights()
total := headN + tailN
if len(capped) <= total {
// Pool too small for a head/tail split — return up to total entries.
// All marked taste: top-N-by-score IS the taste mechanism; no
// exploration sampling happens on this path.
if len(capped) < total {
total = len(capped)
}
out := make([]rankedCandidate, total)
for i := 0; i < total; i++ {
out[i] = rankedCandidate{
TrackID: capped[i].Track.ID,
Score: recommendation.Score(capped[i].Inputs, systemMixWeights, now, rng.Float64),
TrackID: capped[i].Track.ID,
Score: recommendation.Score(capped[i].Inputs, weights, now, rng.Float64),
PickKind: pickKindTaste,
}
}
return out
}
head := capped[:headN]
head := pickQuotaHead(capped, seedOf, numSeeds, headN)
inHead := make(map[pgtype.UUID]bool, len(head))
for _, c := range head {
inHead[c.Track.ID] = true
}
tailStart := 2 * headN
if tailStart >= len(capped) {
tailStart = headN
}
// Defensive copy so that sorting the tail pool does not mutate capped.
tailPool := append([]recommendation.Candidate{}, capped[tailStart:]...)
// Sort tail pool by tieBreakHash (daily-deterministic), take tailN.
// Sample is stable across requests within a day but varies across days.
sort.SliceStable(tailPool, func(i, j int) bool {
return tieBreakHash(tailPool[i].Track.ID, dateStr) < tieBreakHash(tailPool[j].Track.ID, dateStr)
})
tail := tailPool
if len(tail) > tailN {
tail = tail[:tailN]
// The tail pool keeps its rank order (position drives the sampling
// weight); head members are excluded — a quota walk can reach past
// tailStart when a seed's candidates rank deep.
tailPool := make([]recommendation.Candidate, 0, len(capped)-tailStart)
for _, c := range capped[tailStart:] {
if inHead[c.Track.ID] {
continue
}
tailPool = append(tailPool, c)
}
tail := pickWeightedTail(tailPool, dateStr, tailN)
// Combine: head order preserved (score-sorted), tail in tieBreakHash
// order. "First similar, then surprise" reads naturally in playback.
// Combine: head first (score-sorted under quotas), then the fresh
// sample. "First similar, then surprise" reads naturally in
// playback. Head entries are the taste picks; tail entries are the
// freshness injection (#1249) — the split the metrics page
// attributes skips to.
combined := make([]recommendation.Candidate, 0, len(head)+len(tail))
combined = append(combined, head...)
combined = append(combined, tail...)
out := make([]rankedCandidate, len(combined))
for i, c := range combined {
kind := pickKindTaste
if i >= len(head) {
kind = pickKindFresh
}
out[i] = rankedCandidate{
TrackID: c.Track.ID,
Score: recommendation.Score(c.Inputs, systemMixWeights, now, rng.Float64),
TrackID: c.Track.ID,
Score: recommendation.Score(c.Inputs, weights, now, rng.Float64),
PickKind: kind,
}
}
return out
@@ -709,9 +955,15 @@ func insertSystemPlaylist(ctx context.Context, qtx *dbq.Queries, userID pgtype.U
}
for _, t := range tracks {
var pickKind *string
if t.PickKind != "" {
k := t.PickKind
pickKind = &k
}
if _, err := qtx.AppendPlaylistTrack(ctx, dbq.AppendPlaylistTrackParams{
PlaylistID: p.ID,
TrackID: t.TrackID,
PickKind: pickKind,
}); err != nil {
// Track may have been deleted between candidate-load and insert;
// skip silently rather than failing the whole build.
+64 -13
View File
@@ -40,6 +40,17 @@ import (
// Discover so shuffle-on-play has a varied pool within a day.
const discoveryMixLen = 100
// Minimum viable mix sizes (issue #1246): below the floor the variant
// is withheld entirely so Home renders its "listen more to unlock"
// placeholder — a playlist with a couple of songs reads as a build
// bug, not a mix. Album-coherent mixes (NewForYou / FirstListens) get
// a lower floor because one legitimate new album (~5+ tracks) is a
// useful mix on its own; the scattered mixes need more to feel real.
const (
discoveryMixMinLen = 15
discoveryMixMinLenAlbum = 5
)
// discoveryMixSpec describes one discovery mix. The unified producer
// reads the spec and runs a single code path for all variants.
type discoveryMixSpec struct {
@@ -55,6 +66,11 @@ type discoveryMixSpec struct {
// or when day-over-day stability is the intended UX (NewForYou).
dailyRotate bool
// minLen is the minimum viable mix size: a finished pool below it
// is withheld (no playlist row) so the client renders the locked
// placeholder instead of a mix that looks built-wrong (#1246).
minLen int
// fetch returns the raw ranked rows. dateStr is supplied for
// queries that accept it (passed as the second positional arg
// historically); queries that don't accept it ignore the param.
@@ -79,25 +95,49 @@ func produceDiscoveryMix(spec discoveryMixSpec) systemPlaylistProducer {
if spec.dailyRotate {
pool = rotateForDay(pool, userID, dateStr)
}
return emit(spec.name, spec.variant, finishMix(pool, spec.diversify)), nil
tracks := finishMix(pool, spec.diversify)
if len(tracks) < spec.minLen {
logger.Info("system playlist: "+spec.variant+" below minimum viable size; withholding",
"user_id", uuidStringPL(userID), "pool", len(tracks), "min", spec.minLen)
return nil, nil
}
return emit(spec.name, spec.variant, tracks), nil
}
}
// rotateForDay rotates pool left by a daily-deterministic offset so
// each day's downstream truncate-to-N surfaces a different slice of
// the pool while contiguous-block ordering inside the slice is
// preserved. Empty / single-element pools pass through unchanged.
// rotateForDay rotates the pool left by a daily-deterministic offset
// so each day's downstream truncate-to-N surfaces a different slice
// while contiguous-block ordering inside the slice is preserved.
//
// Rotation happens WITHIN each contiguous same-pick-kind block
// (#1267): tiered mixes arrive tier-ordered, and a whole-pool
// rotation would hoist tier-3 filler above tier-1's exact fits.
// Untiered pools are a single block, which reduces to the original
// whole-pool rotation (same seed, same first draw). Empty /
// single-element pools pass through unchanged.
func rotateForDay(pool []discoverTrack, userID pgtype.UUID, dateStr string) []discoverTrack {
n := len(pool)
if n <= 1 {
return pool
}
rng := rand.New(rand.NewSource(int64(userIDHash(userID, dateStr))))
offset := rng.Intn(n)
rotated := make([]discoverTrack, 0, n)
rotated = append(rotated, pool[offset:]...)
rotated = append(rotated, pool[:offset]...)
return rotated
out := make([]discoverTrack, 0, n)
for start := 0; start < n; {
end := start + 1
for end < n && pool[end].PickKind == pool[start].PickKind {
end++
}
block := pool[start:end]
if len(block) > 1 {
offset := rng.Intn(len(block))
out = append(out, block[offset:]...)
out = append(out, block[:offset]...)
} else {
out = append(out, block...)
}
start = end
}
return out
}
// discoveryMixSpecs is the concrete spec list used by the registry in
@@ -109,6 +149,7 @@ var discoveryMixSpecs = []discoveryMixSpec{
{
name: "Deep Cuts", variant: "deep_cuts",
diversify: true, dailyRotate: false, // SQL day-keys via md5(id||$2)
minLen: discoveryMixMinLen,
fetch: func(ctx context.Context, q *dbq.Queries, uid pgtype.UUID, ds string) ([]discoverTrack, error) {
rows, err := q.ListDeepCutsTracks(ctx, dbq.ListDeepCutsTracksParams{
UserID: uid, Column2: ds,
@@ -126,6 +167,7 @@ var discoveryMixSpecs = []discoveryMixSpec{
{
name: "Rediscover", variant: "rediscover",
diversify: true, dailyRotate: true, // SQL has no date arg
minLen: discoveryMixMinLen,
fetch: func(ctx context.Context, q *dbq.Queries, uid pgtype.UUID, _ string) ([]discoverTrack, error) {
rows, err := q.ListRediscoverTracks(ctx, uid)
if err != nil {
@@ -141,6 +183,7 @@ var discoveryMixSpecs = []discoveryMixSpec{
{
name: "New for you", variant: "new_for_you",
diversify: true, dailyRotate: true, // operator wants daily rotation on all deterministic mixes
minLen: discoveryMixMinLenAlbum,
fetch: func(ctx context.Context, q *dbq.Queries, uid pgtype.UUID, _ string) ([]discoverTrack, error) {
rows, err := q.ListNewForYouTracks(ctx, uid)
if err != nil {
@@ -148,7 +191,10 @@ var discoveryMixSpecs = []discoveryMixSpec{
}
out := make([]discoverTrack, len(rows))
for i, r := range rows {
out[i] = discoverTrack{ID: r.ID, AlbumID: r.AlbumID, ArtistID: r.ArtistID}
out[i] = discoverTrack{
ID: r.ID, AlbumID: r.AlbumID, ArtistID: r.ArtistID,
PickKind: pickKindForMixTier(r.Tier),
}
}
return out, nil
},
@@ -156,6 +202,7 @@ var discoveryMixSpecs = []discoveryMixSpec{
{
name: "On this day", variant: "on_this_day",
diversify: true, dailyRotate: false, // SQL day-keys via md5(id||$2)
minLen: discoveryMixMinLen,
fetch: func(ctx context.Context, q *dbq.Queries, uid pgtype.UUID, ds string) ([]discoverTrack, error) {
rows, err := q.ListOnThisDayTracks(ctx, dbq.ListOnThisDayTracksParams{
UserID: uid, Column2: ds,
@@ -173,6 +220,7 @@ var discoveryMixSpecs = []discoveryMixSpec{
{
name: "First listens", variant: "first_listens",
diversify: true, dailyRotate: true, // SQL has no date arg; daily rotate + diversity top-up
minLen: discoveryMixMinLenAlbum,
fetch: func(ctx context.Context, q *dbq.Queries, uid pgtype.UUID, _ string) ([]discoverTrack, error) {
rows, err := q.ListFirstListensTracks(ctx, uid)
if err != nil {
@@ -180,7 +228,10 @@ var discoveryMixSpecs = []discoveryMixSpec{
}
out := make([]discoverTrack, len(rows))
for i, r := range rows {
out[i] = discoverTrack{ID: r.ID, AlbumID: r.AlbumID, ArtistID: r.ArtistID}
out[i] = discoverTrack{
ID: r.ID, AlbumID: r.AlbumID, ArtistID: r.ArtistID,
PickKind: pickKindForMixTier(r.Tier),
}
}
return out, nil
},
@@ -217,7 +268,7 @@ func finishMix(rows []discoverTrack, diversify bool) []rankedCandidate {
}
tracks := make([]rankedCandidate, len(pool))
for i, t := range pool {
tracks[i] = rankedCandidate{TrackID: t.ID}
tracks[i] = rankedCandidate{TrackID: t.ID, PickKind: t.PickKind}
}
return tracks
}

Some files were not shown because too many files have changed in this diff Show More