diff --git a/docs/superpowers/specs/2026-04-30-m5c-suggested-additions-design.md b/docs/superpowers/specs/2026-04-30-m5c-suggested-additions-design.md new file mode 100644 index 00000000..641ff1a2 --- /dev/null +++ b/docs/superpowers/specs/2026-04-30-m5c-suggested-additions-design.md @@ -0,0 +1,360 @@ +# M5c — Suggested additions on `/discover` + +> **Status:** Draft for review · 2026-04-30 +> +> **Sub-plan of:** M5 (Lidarr integration). Final slice of the M5 trilogy: +> +> - **M5a** — Lidarr connection + search/add + admin shell. Shipped on `dev`. +> - **M5b** — Quarantine workflow. Shipped on `dev`, in PR #30. +> - **M5c (this spec)** — Personalized artist suggestions surfaced on `/discover`. +> +> Note: this spec deliberately diverges from the M5a §10 carve-out, which described surfacing out-of-library MBIDs in `/api/radio` responses. Operator redirected during brainstorming: the suggestion surface is a dedicated recommendation feed on `/discover`, decoupled from radio. The §10 reference is preserved here for traceability. + +## 1. Goal + +Authenticated users land on `/discover` and see a "Suggested for you" feed by default — top-12 out-of-library artists ranked by per-user signal (likes + recency-decayed plays) projected through ListenBrainz artist-similarity. Each suggestion shows attribution ("Because you liked X, played Y, and played Z"), and one click fires an artist-kind add request through M5a's existing Lidarr-add path. + +When the user types in the search input, the suggestion feed is replaced by Lidarr search results — M5a's existing behavior. When the input clears, suggestions return. + +## 2. Goals and non-goals + +### Goals + +- Top-12 personalized artist suggestions on `/discover` when the search input is empty. +- Per-user ranking weighted by explicit likes (×5) plus implicit plays (count, recency-decayed by half-life ~30 days). +- Top-3 contributing seed artists named per suggestion: "Because you liked X, played Y, and played Z." +- One-click add via the existing M5a `POST /api/requests` artist-kind path. +- Suggestions hide automatically when the candidate is in-library or already in a non-terminal `lidarr_requests` row. +- The M4b similarity ingest worker is extended to persist unmatched similar-artist MBIDs that it currently discards. No new background worker. + +### Non-goals (this slice) + +- Album-level or track-level suggestions. Artist-only for v1; albums are a potential v2 layer once we see how artist suggestions feel in practice. +- Realtime invalidation on every like/play. The feed updates passively as user signals accumulate; no push, no immediate refresh. +- Cross-user collaborative filtering. The recommendation is the user's own seed set × ListenBrainz similarity — single-user data only. +- Pagination beyond top-12. Polish slot if the operator wants it later. +- Materialized aggregation of `play_events`. Documented as a v2 lever if on-demand performance ever becomes an issue. +- Cover art for out-of-library suggestions. v1 uses the Lucide `Disc3` fallback; a MusicBrainz Cover-Art-Archive lookup is a polish-pass slot. +- Anything in the `/api/radio` response. M5c is decoupled from radio. + +## 3. Architecture + +### Data flow + +1. **M4b similarity worker (existing)** fetches similar-artists from ListenBrainz per played artist. Today it discards MBIDs that aren't in `artists`. M5c keeps that filter for the `artist_similarity` table but ALSO writes the discarded MBIDs to a new `artist_similarity_unmatched` table — same structure but with no FK on the candidate side. +2. **Suggestion query** (on demand at `/api/discover/suggestions`) joins the user's likes + plays against `artist_similarity_unmatched` via the seeds, computes per-candidate score, ranks top-N. +3. **SPA renders** the response on `/discover` when the search input is empty. Each card reuses the existing `` with an extra `attribution` prop. +4. **One-click Request** fires the same M5a `POST /api/requests` flow used by the search-side `` — no new add machinery. + +### New Go components + +- **`internal/recommendation/suggestions.go`** — exports a single `SuggestArtists(ctx, pool, userID, halfLifeDays, limit) ([]ArtistSuggestion, error)` function. Single CTE query (see §4). Returns ranked candidates with their top-3 attribution seeds. +- **`internal/api/suggestions.go`** — `GET /api/discover/suggestions` handler. Authenticated; no admin gate. Calls the recommendation service and resolves seed artist names via a single `GetArtistsByIDs` follow-up (≤36 distinct IDs across top-12 candidates × 3 seeds each). + +### Modified Go components + +- **`internal/similarity/worker.go`** — `upsertArtistSimilar` keeps the existing matched path and adds a parallel unmatched-persist loop that calls a new `UpsertArtistSimilarityUnmatched` query. Top-K cap mirrors the matched path. +- **`internal/db/queries/similarity.sql`** — extension to ingest the new table. Reads in §4. + +### Frontend changes + +- `web/src/routes/discover/+page.svelte` — when `debouncedQ === ''`, render the new `` subcomponent. Hide the kind tabs in this state. Existing search behavior unchanged when input is non-empty. +- `web/src/lib/components/DiscoverResultCard.svelte` — add an optional `attribution?: string` prop that renders an italic Vellum line below the title. +- `web/src/lib/api/suggestions.ts` (new) — `listSuggestions(limit?)` and `createSuggestionsQuery()` factory with `staleTime: 5 * 60_000`. +- `web/src/lib/api/queries.ts` — `qk.suggestions(limit)` query key. +- `web/src/lib/api/types.ts` — `ArtistSuggestion`, `SeedContribution` types. + +### Wiring + +No `cmd/minstrel/main.go` or `internal/server/server.go` changes. The new endpoint mounts on the existing authed route group; the similarity worker is already constructed. + +## 4. Schema — migration `0012_artist_similarity_unmatched` + +```sql +-- M5c: persist unmatched-similar-artist MBIDs that the M4b worker would +-- otherwise discard. Mirrors artist_similarity shape: same composite PK +-- with source, same (seed_id, score DESC) index, same source enum check. +-- The candidate side is text + name (no FK) — that's the whole point. + +CREATE TABLE artist_similarity_unmatched ( + seed_artist_id uuid NOT NULL REFERENCES artists(id) ON DELETE CASCADE, + candidate_mbid text NOT NULL, + candidate_name text NOT NULL, + score DOUBLE PRECISION NOT NULL, + source text NOT NULL CHECK (source IN ('listenbrainz', 'musicbrainz_tag', 'user_cooccurrence')), + fetched_at timestamptz NOT NULL DEFAULT now(), + PRIMARY KEY (seed_artist_id, candidate_mbid, source) +); + +CREATE INDEX artist_similarity_unmatched_seed_score_idx + ON artist_similarity_unmatched (seed_artist_id, score DESC); +``` + +### Schema notes + +- `seed_artist_id` cascades on artist delete — consistent with `artist_similarity`. Removes orphaned suggestion rows automatically when an operator deletes a seed artist. +- `candidate_mbid` deliberately has no FK and no unique constraint by itself; the same out-of-library MBID can appear for many seed artists, and that's the whole mechanism: aggregating contributions across the user's seeds is what produces the score. +- `(seed_artist_id, candidate_mbid, source)` PK gives the worker a clean upsert target while leaving room for future similarity sources. +- `(seed_artist_id, score DESC)` index satisfies the dominant query pattern: "for seed S, give me top-K unmatched candidates by score." This is what the suggestion query exploits inside its CTE join. +- Storage estimate at v1 scale: ~2k seed artists × ~100 unmatched candidates × ~150 bytes = **~30 MB**. Negligible. + +### Down migration + +```sql +DROP INDEX IF EXISTS artist_similarity_unmatched_seed_score_idx; +DROP TABLE IF EXISTS artist_similarity_unmatched; +``` + +### Suggestion query shape + +```sql +-- name: SuggestArtistsForUser :many +WITH seeds AS ( + SELECT a.id AS artist_id, + 5.0 * (CASE WHEN gla.artist_id IS NOT NULL THEN 1 ELSE 0 END) + + COALESCE(SUM(EXP(- EXTRACT(epoch FROM now() - pe.started_at) / ($2 * 86400.0))), 0) + AS signal + FROM artists a + LEFT JOIN general_likes_artists gla ON gla.artist_id = a.id AND gla.user_id = $1 + LEFT JOIN tracks t ON t.artist_id = a.id + LEFT JOIN play_events pe ON pe.track_id = t.id AND pe.user_id = $1 + WHERE gla.artist_id IS NOT NULL OR pe.id IS NOT NULL + GROUP BY a.id +), +contributions AS ( + SELECT u.candidate_mbid, + u.candidate_name, + seeds.artist_id AS seed_id, + seeds.signal * u.score AS contribution + FROM artist_similarity_unmatched u + JOIN seeds ON seeds.artist_id = u.seed_artist_id + WHERE NOT EXISTS (SELECT 1 FROM artists WHERE mbid = u.candidate_mbid) + AND NOT EXISTS ( + SELECT 1 FROM lidarr_requests r + WHERE r.user_id = $1 + AND r.lidarr_artist_mbid = u.candidate_mbid + AND r.status NOT IN ('rejected', 'failed') + ) +) +SELECT candidate_mbid, + candidate_name, + SUM(contribution)::float AS total_score, + (array_agg(seed_id ORDER BY contribution DESC))[1:3] AS top_seed_ids, + (array_agg(contribution ORDER BY contribution DESC))[1:3] AS top_contributions +FROM contributions +GROUP BY candidate_mbid, candidate_name +ORDER BY total_score DESC +LIMIT $3; +``` + +The handler then resolves the top-3 seed UUIDs to artist names via `GetArtistsByIDs` (one extra round-trip; cheap because there are at most 36 distinct seed IDs across top-12 candidates). + +### `seeds` CTE rationale + +- Likes contribute a flat `5.0` per liked artist — explicit user signal worth more than a single play. +- Plays contribute `Σ exp(-age_days / half_life)` summed across all play events for the user × artist. Recent plays carry close to 1.0; week-old plays ~0.79; 30-day-old plays ~0.37; 90-day-old plays ~0.05. The exponential decay matches what radio's `recencyDecay` already does conceptually (different polarity). +- `WHERE gla.artist_id IS NOT NULL OR pe.id IS NOT NULL` skips artists the user has neither liked nor played — they'd contribute zero signal anyway. + +### Worker upsert query + +```sql +-- name: UpsertArtistSimilarityUnmatched :exec +INSERT INTO artist_similarity_unmatched (seed_artist_id, candidate_mbid, candidate_name, score, source) +VALUES ($1, $2, $3, $4, $5) +ON CONFLICT (seed_artist_id, candidate_mbid, source) DO UPDATE SET + candidate_name = EXCLUDED.candidate_name, + score = EXCLUDED.score, + fetched_at = now(); +``` + +Idempotent on re-fetch; `candidate_name` and `score` get refreshed when ListenBrainz returns updated values. + +## 5. API surface + +| Method | Path | Behavior | +|---|---|---| +| `GET` | `/api/discover/suggestions?limit=&half_life_days=` | Returns the caller's top-N artist suggestions. Defaults `limit=12` (capped at 50), `half_life_days=30`. Both query params are operator-tunable; the SPA only sends `limit` in v1. Returns `200 [{mbid, name, score, attribution: [{artist_id, name, contribution}]}]`. | + +### Response type + +```ts +type ArtistSuggestion = { + mbid: string; + name: string; + score: number; + attribution: SeedContribution[]; // top-3, ordered by contribution DESC +}; + +type SeedContribution = { + artist_id: string; + name: string; + contribution: number; +}; +``` + +### Empty cases + +- New user with no likes and no plays → `200 []`. Empty-state copy: "Listen to something or like an artist to start getting suggestions." +- All recommendations already in library or already requested → `200 []`. Same copy works (the operator's library exhaustively covers their taste). + +### Error codes added + +None. The endpoint is read-only and can only fail on internal DB error → `500 server_error`. + +### Performance + +- Single CTE query at request time + one follow-up `GetArtistsByIDs` lookup for attribution names. Sub-50ms end-to-end at household scale (a few thousand seeds × hundreds of candidates fits well within Postgres' happy path with the planned indexes). +- Frontend wraps the call in TanStack Query with `staleTime: 5 * 60_000` (5 minutes). Repeat visits within a session don't re-query. + +### v2 lever (deferred) + +If the per-request `seeds` CTE aggregation over `play_events` becomes the bottleneck at large-library scale (year-3 territory), materialize a per-user `artist_signal_cache` table refreshed daily. The suggestion query plugs into the cache instead of scanning `play_events`. No API surface change required. + +## 6. UI surfaces + +All against the FabledSword design tokens. Voice rule: sentence case, "understated mythic" register on errors and empty states. + +### `/discover` page-level state machine + +The page has these states (existing M5a states preserved unless noted): + +| Search input | Header copy | Tabs | Content | +|---|---|---|---| +| Empty | **"Suggested for you"** + Vellum subtitle (new) | hidden (new) | Suggestion grid, top-12 (new) | +| Empty + zero suggestions | **"Suggested for you"** | hidden | Empty-state copy (new) | +| Non-empty | "Add music to the library" (existing) | visible (existing) | Lidarr search results (existing) | + +Subtitle when suggestions are showing: "Out-of-library artists drawn from what you've liked and played." + +Empty-state copy when no suggestions: "Listen to something or like an artist to start getting suggestions." + +When the user starts typing, the suggestion feed swaps out for the search-results UI. When they clear the input, the suggestion feed comes back. TanStack `staleTime` keeps both responses cached so swaps within a session are instant. + +### `` extension + +Add an optional `attribution?: string` prop. When set, renders below the title in italic Vellum 12px. Empty / undefined → no attribution row (existing search-result rendering unchanged). + +Card layout (top to bottom) with attribution: +- Art square (1:1 aspect, Slate fallback with Lucide `Disc3`). +- **Artist name** in Parchment, font-medium 14px. +- **Attribution** (italic Vellum 12px) — only when prop is set. +- Reserved badge slot (22px min-height; empty for suggestions). +- Action button: Moss "Request" with Plus icon. + +### Attribution copy format + +The handler returns up to 3 contributors per suggestion, ordered by contribution. The SPA formats: + +| Contributors | Format | +|---|---| +| 1 | "Because you liked X." or "Because you played X." | +| 2 | "Because you liked X and played Y." | +| 3 | "Because you liked X, played Y, and played Z." (Oxford comma) | + +The verb ("liked"/"played") matches the dominant signal for each seed: if the user liked the artist, "liked"; otherwise "played." A single signal type per seed keeps the copy clean. + +In v1 the seed names are plain text, not links. Wiring them to `/artists/{id}` is a polish-pass refinement (Fable #349). + +### `` subcomponent + +Owned by `/discover/+page.svelte`. ~50 lines. Responsibilities: +- Reads `createSuggestionsQuery()` for data. +- Manages the optimistic-requested `Set` (mirrors the existing search-side machinery on the same page so suggestions and search results share the same optimistic state — a request from search hides the matching card from suggestions on next render too). +- Renders the empty state when `data.length === 0`. +- Maps suggestions → `` with `state="requestable"` and the formatted `attribution` string. + +Grid: same `grid-cols-2 sm:grid-cols-3 lg:grid-cols-4 xl:grid-cols-5 gap-4` as the search-results grid for visual continuity. + +## 7. Error handling + +### Worker-side (similarity ingest extension) + +- `UpsertArtistSimilarityUnmatched` per-row failure logged at WARN, not propagated. Mirrors the existing `UpsertArtistSimilarity` row-error policy. +- Missing `Name` field on a `SimilarArtist` LB response: skip that row at DEBUG (we can't render a suggestion without the name). Verify the LB client surface during T2 — extend the client if `Name` isn't already exposed. +- Existing rate-limit/network handling carries over (same client call, just persisting more of the response). + +### Handler-side (`/api/discover/suggestions`) + +- DB error on the CTE query → `500 server_error`. SPA renders ``. User can retry, or clear the input to fall back to search. +- Empty result is NOT an error — `200 []` with the SPA empty-state copy. +- No Lidarr-disabled branch on the read path: the suggestion feed reads pre-computed similarity data and works even when Lidarr is disconnected. The Request button on each card surfaces the M5a `lidarr_disabled` error when clicked, if Lidarr isn't configured. + +### Stale-data behavior + +- An admin adding the artist between worker re-ingest and the user's next refresh: the suggestion-query's `WHERE NOT EXISTS (SELECT 1 FROM artists WHERE mbid = ...)` filters in-library candidates at query time, so staleness window is "until next page refresh," not "until next worker tick." Acceptable. +- A user requesting an artist between page renders: same — the `WHERE NOT EXISTS (... lidarr_requests ...)` filters at query time. + +## 8. Testing + +### Unit tests (no DB) + +- Pure-helper tests if any scoring logic lands in Go (not the SQL CTE). Most logic is in SQL; expect minimal Go unit-test surface. + +### Integration tests (gated on `MINSTREL_TEST_DATABASE_URL`) + +- `internal/recommendation/suggestions_integration_test.go`: + - **TestSuggestArtists_LikesAndPlaysContributeToScore** — single user, 1 liked + 1 played seed both pointing at the same out-of-library candidate; verify combined contribution. + - **TestSuggestArtists_Top12Cap** — 30 candidates with descending scores; verify only top-12 returned, in order. + - **TestSuggestArtists_AttributionTopThree** — 5 contributing seeds for one candidate; verify `Attribution` has exactly the top-3 by contribution. + - **TestSuggestArtists_RecencyDecayDownweightsOldPlays** — two seeds with 1-day vs 90-day-old plays pointing at the same candidate; verify recent contributes more (exact ratio derives from `exp(-age/half_life)`). + - **TestSuggestArtists_FiltersInLibraryCandidates** — candidate that exists in `artists` is excluded from response. + - **TestSuggestArtists_FiltersAlreadyRequested** — non-terminal `lidarr_requests` row hides the candidate. + - **TestSuggestArtists_RejectedRequestStillShown** — `status='rejected'` does NOT hide the candidate (rejected requests are terminal; user can re-request after admin's reject). + - **TestSuggestArtists_EmptyForNewUser** — user with no likes and no plays returns `[]`. + +### Worker integration test + +- `internal/similarity/worker_test.go` — `TestUpsertArtistSimilar_PersistsUnmatchedToTable`: seed an `artists` table with 1 in-library artist; mock the LB client to return 5 similar-artists where 1 is in-library and 4 are not. Verify `artist_similarity` got 1 row and `artist_similarity_unmatched` got 4 rows. + +### HTTP handler tests + +- `internal/api/suggestions_test.go`: + - **TestSuggestions_HappyPath** — stub the recommendation service, verify JSON shape. + - **TestSuggestions_EmptyForNewUser** — `200 []`. + - **TestSuggestions_AttributionShape** — response includes `attribution` array with up to 3 entries. + +### Frontend tests (vitest) + +- `web/src/routes/discover/discover.test.ts` extension: + - **suggestion feed renders when input is empty** — mock `createSuggestionsQuery` to return 12 rows; verify 12 cards rendered, kind tabs hidden. + - **typing replaces feed with search** — input `"miles"` → search results visible, suggestions hidden. + - **clearing input restores feed**. + - **attribution copy renders correctly for 1/2/3 seeds**. + - **request button optimistically removes the card**. + - **empty state copy** when query returns `[]`. + +### Coverage targets + +- `internal/recommendation/` (new code only) ≥ 80%. +- `internal/similarity/` (worker extension) maintained at current level (≥ 80% combined per existing target). +- Combined Lidarr-suite (lidarr, lidarrconfig, lidarrrequests, lidarrquarantine, plus new suggestion code) stays ≥ 80% per the M5a/M5b cadence. + +## 9. Decisions ledger + +| # | Decision | Rationale | +|---|---|---| +| 1 | Surface on `/discover` (default state, search-input-empty), not on `/api/radio` | Operator's redirection during brainstorming — recommendations should be a deliberate "I want to add music" surface, not a radio side-channel | +| 2 | Artist-only granularity for v1 | Cleanest mental model; matches Lidarr's monitor unit; album/track suggestions deferred to v2 | +| 3 | Recency-decayed plays + likes weighted 5× | Recency matches radio's existing decay model conceptually; 5× honors the M2 distinction between explicit (like) and implicit (play) signal | +| 4 | Top-3 contributing seeds named per suggestion | Operator framing was "you might like X because you liked Y, Z, **and W**" — multi-seed attribution is the whole point of the surface | +| 5 | Search-empty replaces initial-copy state with the feed; tabs hidden | Operator wanted suggestions front-and-center for users who don't know the feature exists; search is the explicit action | +| 6 | Top-12 fixed, no pagination | Score-vs-noise drops sharply past top-12; operator workflow ends with external search engine for actual decision | +| 7 | On-demand SQL + 5-minute SPA cache; no backend caching layer | Single CTE at household scale is sub-50ms; TanStack `staleTime` handles repeat visits; v2 materialization noted as a future lever if needed | +| 8 | New `artist_similarity_unmatched` table; extend M4b worker | Mirrors `artist_similarity` shape; worker already runs and discards these MBIDs today; persisting is a small addition | +| 9 | Reuse `` with optional `attribution` prop | Cheaper than maintaining a parallel ``; the attribution slot fits cleanly above the reserved badge row | + +## 10. Out of scope (this slice) + +- Album / track suggestions. +- Cross-user collaborative filtering ("users like you also liked X"). +- Pagination / infinite scroll on the suggestion feed. +- Cover art for out-of-library candidates (would require MusicBrainz Cover-Art-Archive lookup). +- Linking attribution-seed names to `/artists/{id}` (polish-pass refinement). +- Materialized per-user signal aggregation. v2 lever if performance ever requires it. +- Tunable `half_life_days` exposed in the SPA. Backend accepts it, frontend always uses default. + +## 11. Open questions + +- **`SimilarArtist.Name` on the LB client.** Verify during T2 that the existing client surfaces the artist name alongside MBID. If not, extend the client (small, additive). +- **Cold-start duration after install.** Until the M4b worker has had a chance to run a few ticks against the user's listened-to artists, the unmatched table is small and suggestions will be sparse. Documented; not a v1 fix. +- **Cross-source diversity.** When `musicbrainz_tag` and `user_cooccurrence` source providers come online (post-M5), the same candidate MBID may appear from multiple sources with different scores. The `seeds → contributions` join already aggregates per `(seed, candidate)` regardless of source via the GROUP BY in the outer CTE — but ranking diversity (don't surface 12 candidates all sourced from one seed) is a polish-pass concern.