docs(spec): add M5c suggested-additions design

Personalized artist suggestions on /discover (search-input-empty state).
On-demand SQL ranks out-of-library MBIDs from artist_similarity_unmatched
(new table, populated by extending the M4b similarity worker) by per-user
signal: likes weighted 5x plus recency-decayed plays (exp(-age_days/30)).
Top-12 with top-3 contributing seeds attributed per card. Reuses the M5a
DiscoverResultCard + POST /api/requests artist-add path.
This commit is contained in:
2026-04-30 22:48:55 -04:00
parent 3e3ad89645
commit a0a9fb201b
@@ -0,0 +1,360 @@
# M5c — Suggested additions on `/discover`
> **Status:** Draft for review · 2026-04-30
>
> **Sub-plan of:** M5 (Lidarr integration). Final slice of the M5 trilogy:
>
> - **M5a** — Lidarr connection + search/add + admin shell. Shipped on `dev`.
> - **M5b** — Quarantine workflow. Shipped on `dev`, in PR #30.
> - **M5c (this spec)** — Personalized artist suggestions surfaced on `/discover`.
>
> Note: this spec deliberately diverges from the M5a §10 carve-out, which described surfacing out-of-library MBIDs in `/api/radio` responses. Operator redirected during brainstorming: the suggestion surface is a dedicated recommendation feed on `/discover`, decoupled from radio. The §10 reference is preserved here for traceability.
## 1. Goal
Authenticated users land on `/discover` and see a "Suggested for you" feed by default — top-12 out-of-library artists ranked by per-user signal (likes + recency-decayed plays) projected through ListenBrainz artist-similarity. Each suggestion shows attribution ("Because you liked X, played Y, and played Z"), and one click fires an artist-kind add request through M5a's existing Lidarr-add path.
When the user types in the search input, the suggestion feed is replaced by Lidarr search results — M5a's existing behavior. When the input clears, suggestions return.
## 2. Goals and non-goals
### Goals
- Top-12 personalized artist suggestions on `/discover` when the search input is empty.
- Per-user ranking weighted by explicit likes (×5) plus implicit plays (count, recency-decayed by half-life ~30 days).
- Top-3 contributing seed artists named per suggestion: "Because you liked X, played Y, and played Z."
- One-click add via the existing M5a `POST /api/requests` artist-kind path.
- Suggestions hide automatically when the candidate is in-library or already in a non-terminal `lidarr_requests` row.
- The M4b similarity ingest worker is extended to persist unmatched similar-artist MBIDs that it currently discards. No new background worker.
### Non-goals (this slice)
- Album-level or track-level suggestions. Artist-only for v1; albums are a potential v2 layer once we see how artist suggestions feel in practice.
- Realtime invalidation on every like/play. The feed updates passively as user signals accumulate; no push, no immediate refresh.
- Cross-user collaborative filtering. The recommendation is the user's own seed set × ListenBrainz similarity — single-user data only.
- Pagination beyond top-12. Polish slot if the operator wants it later.
- Materialized aggregation of `play_events`. Documented as a v2 lever if on-demand performance ever becomes an issue.
- Cover art for out-of-library suggestions. v1 uses the Lucide `Disc3` fallback; a MusicBrainz Cover-Art-Archive lookup is a polish-pass slot.
- Anything in the `/api/radio` response. M5c is decoupled from radio.
## 3. Architecture
### Data flow
1. **M4b similarity worker (existing)** fetches similar-artists from ListenBrainz per played artist. Today it discards MBIDs that aren't in `artists`. M5c keeps that filter for the `artist_similarity` table but ALSO writes the discarded MBIDs to a new `artist_similarity_unmatched` table — same structure but with no FK on the candidate side.
2. **Suggestion query** (on demand at `/api/discover/suggestions`) joins the user's likes + plays against `artist_similarity_unmatched` via the seeds, computes per-candidate score, ranks top-N.
3. **SPA renders** the response on `/discover` when the search input is empty. Each card reuses the existing `<DiscoverResultCard>` with an extra `attribution` prop.
4. **One-click Request** fires the same M5a `POST /api/requests` flow used by the search-side `<DiscoverResultCard>` — no new add machinery.
### New Go components
- **`internal/recommendation/suggestions.go`** — exports a single `SuggestArtists(ctx, pool, userID, halfLifeDays, limit) ([]ArtistSuggestion, error)` function. Single CTE query (see §4). Returns ranked candidates with their top-3 attribution seeds.
- **`internal/api/suggestions.go`** — `GET /api/discover/suggestions` handler. Authenticated; no admin gate. Calls the recommendation service and resolves seed artist names via a single `GetArtistsByIDs` follow-up (≤36 distinct IDs across top-12 candidates × 3 seeds each).
### Modified Go components
- **`internal/similarity/worker.go`** — `upsertArtistSimilar` keeps the existing matched path and adds a parallel unmatched-persist loop that calls a new `UpsertArtistSimilarityUnmatched` query. Top-K cap mirrors the matched path.
- **`internal/db/queries/similarity.sql`** — extension to ingest the new table. Reads in §4.
### Frontend changes
- `web/src/routes/discover/+page.svelte` — when `debouncedQ === ''`, render the new `<SuggestionFeed>` subcomponent. Hide the kind tabs in this state. Existing search behavior unchanged when input is non-empty.
- `web/src/lib/components/DiscoverResultCard.svelte` — add an optional `attribution?: string` prop that renders an italic Vellum line below the title.
- `web/src/lib/api/suggestions.ts` (new) — `listSuggestions(limit?)` and `createSuggestionsQuery()` factory with `staleTime: 5 * 60_000`.
- `web/src/lib/api/queries.ts``qk.suggestions(limit)` query key.
- `web/src/lib/api/types.ts``ArtistSuggestion`, `SeedContribution` types.
### Wiring
No `cmd/minstrel/main.go` or `internal/server/server.go` changes. The new endpoint mounts on the existing authed route group; the similarity worker is already constructed.
## 4. Schema — migration `0012_artist_similarity_unmatched`
```sql
-- M5c: persist unmatched-similar-artist MBIDs that the M4b worker would
-- otherwise discard. Mirrors artist_similarity shape: same composite PK
-- with source, same (seed_id, score DESC) index, same source enum check.
-- The candidate side is text + name (no FK) — that's the whole point.
CREATE TABLE artist_similarity_unmatched (
seed_artist_id uuid NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
candidate_mbid text NOT NULL,
candidate_name text NOT NULL,
score DOUBLE PRECISION NOT NULL,
source text NOT NULL CHECK (source IN ('listenbrainz', 'musicbrainz_tag', 'user_cooccurrence')),
fetched_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (seed_artist_id, candidate_mbid, source)
);
CREATE INDEX artist_similarity_unmatched_seed_score_idx
ON artist_similarity_unmatched (seed_artist_id, score DESC);
```
### Schema notes
- `seed_artist_id` cascades on artist delete — consistent with `artist_similarity`. Removes orphaned suggestion rows automatically when an operator deletes a seed artist.
- `candidate_mbid` deliberately has no FK and no unique constraint by itself; the same out-of-library MBID can appear for many seed artists, and that's the whole mechanism: aggregating contributions across the user's seeds is what produces the score.
- `(seed_artist_id, candidate_mbid, source)` PK gives the worker a clean upsert target while leaving room for future similarity sources.
- `(seed_artist_id, score DESC)` index satisfies the dominant query pattern: "for seed S, give me top-K unmatched candidates by score." This is what the suggestion query exploits inside its CTE join.
- Storage estimate at v1 scale: ~2k seed artists × ~100 unmatched candidates × ~150 bytes = **~30 MB**. Negligible.
### Down migration
```sql
DROP INDEX IF EXISTS artist_similarity_unmatched_seed_score_idx;
DROP TABLE IF EXISTS artist_similarity_unmatched;
```
### Suggestion query shape
```sql
-- name: SuggestArtistsForUser :many
WITH seeds AS (
SELECT a.id AS artist_id,
5.0 * (CASE WHEN gla.artist_id IS NOT NULL THEN 1 ELSE 0 END)
+ COALESCE(SUM(EXP(- EXTRACT(epoch FROM now() - pe.started_at) / ($2 * 86400.0))), 0)
AS signal
FROM artists a
LEFT JOIN general_likes_artists gla ON gla.artist_id = a.id AND gla.user_id = $1
LEFT JOIN tracks t ON t.artist_id = a.id
LEFT JOIN play_events pe ON pe.track_id = t.id AND pe.user_id = $1
WHERE gla.artist_id IS NOT NULL OR pe.id IS NOT NULL
GROUP BY a.id
),
contributions AS (
SELECT u.candidate_mbid,
u.candidate_name,
seeds.artist_id AS seed_id,
seeds.signal * u.score AS contribution
FROM artist_similarity_unmatched u
JOIN seeds ON seeds.artist_id = u.seed_artist_id
WHERE NOT EXISTS (SELECT 1 FROM artists WHERE mbid = u.candidate_mbid)
AND NOT EXISTS (
SELECT 1 FROM lidarr_requests r
WHERE r.user_id = $1
AND r.lidarr_artist_mbid = u.candidate_mbid
AND r.status NOT IN ('rejected', 'failed')
)
)
SELECT candidate_mbid,
candidate_name,
SUM(contribution)::float AS total_score,
(array_agg(seed_id ORDER BY contribution DESC))[1:3] AS top_seed_ids,
(array_agg(contribution ORDER BY contribution DESC))[1:3] AS top_contributions
FROM contributions
GROUP BY candidate_mbid, candidate_name
ORDER BY total_score DESC
LIMIT $3;
```
The handler then resolves the top-3 seed UUIDs to artist names via `GetArtistsByIDs` (one extra round-trip; cheap because there are at most 36 distinct seed IDs across top-12 candidates).
### `seeds` CTE rationale
- Likes contribute a flat `5.0` per liked artist — explicit user signal worth more than a single play.
- Plays contribute `Σ exp(-age_days / half_life)` summed across all play events for the user × artist. Recent plays carry close to 1.0; week-old plays ~0.79; 30-day-old plays ~0.37; 90-day-old plays ~0.05. The exponential decay matches what radio's `recencyDecay` already does conceptually (different polarity).
- `WHERE gla.artist_id IS NOT NULL OR pe.id IS NOT NULL` skips artists the user has neither liked nor played — they'd contribute zero signal anyway.
### Worker upsert query
```sql
-- name: UpsertArtistSimilarityUnmatched :exec
INSERT INTO artist_similarity_unmatched (seed_artist_id, candidate_mbid, candidate_name, score, source)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (seed_artist_id, candidate_mbid, source) DO UPDATE SET
candidate_name = EXCLUDED.candidate_name,
score = EXCLUDED.score,
fetched_at = now();
```
Idempotent on re-fetch; `candidate_name` and `score` get refreshed when ListenBrainz returns updated values.
## 5. API surface
| Method | Path | Behavior |
|---|---|---|
| `GET` | `/api/discover/suggestions?limit=&half_life_days=` | Returns the caller's top-N artist suggestions. Defaults `limit=12` (capped at 50), `half_life_days=30`. Both query params are operator-tunable; the SPA only sends `limit` in v1. Returns `200 [{mbid, name, score, attribution: [{artist_id, name, contribution}]}]`. |
### Response type
```ts
type ArtistSuggestion = {
mbid: string;
name: string;
score: number;
attribution: SeedContribution[]; // top-3, ordered by contribution DESC
};
type SeedContribution = {
artist_id: string;
name: string;
contribution: number;
};
```
### Empty cases
- New user with no likes and no plays → `200 []`. Empty-state copy: "Listen to something or like an artist to start getting suggestions."
- All recommendations already in library or already requested → `200 []`. Same copy works (the operator's library exhaustively covers their taste).
### Error codes added
None. The endpoint is read-only and can only fail on internal DB error → `500 server_error`.
### Performance
- Single CTE query at request time + one follow-up `GetArtistsByIDs` lookup for attribution names. Sub-50ms end-to-end at household scale (a few thousand seeds × hundreds of candidates fits well within Postgres' happy path with the planned indexes).
- Frontend wraps the call in TanStack Query with `staleTime: 5 * 60_000` (5 minutes). Repeat visits within a session don't re-query.
### v2 lever (deferred)
If the per-request `seeds` CTE aggregation over `play_events` becomes the bottleneck at large-library scale (year-3 territory), materialize a per-user `artist_signal_cache` table refreshed daily. The suggestion query plugs into the cache instead of scanning `play_events`. No API surface change required.
## 6. UI surfaces
All against the FabledSword design tokens. Voice rule: sentence case, "understated mythic" register on errors and empty states.
### `/discover` page-level state machine
The page has these states (existing M5a states preserved unless noted):
| Search input | Header copy | Tabs | Content |
|---|---|---|---|
| Empty | **"Suggested for you"** + Vellum subtitle (new) | hidden (new) | Suggestion grid, top-12 (new) |
| Empty + zero suggestions | **"Suggested for you"** | hidden | Empty-state copy (new) |
| Non-empty | "Add music to the library" (existing) | visible (existing) | Lidarr search results (existing) |
Subtitle when suggestions are showing: "Out-of-library artists drawn from what you've liked and played."
Empty-state copy when no suggestions: "Listen to something or like an artist to start getting suggestions."
When the user starts typing, the suggestion feed swaps out for the search-results UI. When they clear the input, the suggestion feed comes back. TanStack `staleTime` keeps both responses cached so swaps within a session are instant.
### `<DiscoverResultCard>` extension
Add an optional `attribution?: string` prop. When set, renders below the title in italic Vellum 12px. Empty / undefined → no attribution row (existing search-result rendering unchanged).
Card layout (top to bottom) with attribution:
- Art square (1:1 aspect, Slate fallback with Lucide `Disc3`).
- **Artist name** in Parchment, font-medium 14px.
- **Attribution** (italic Vellum 12px) — only when prop is set.
- Reserved badge slot (22px min-height; empty for suggestions).
- Action button: Moss "Request" with Plus icon.
### Attribution copy format
The handler returns up to 3 contributors per suggestion, ordered by contribution. The SPA formats:
| Contributors | Format |
|---|---|
| 1 | "Because you liked X." or "Because you played X." |
| 2 | "Because you liked X and played Y." |
| 3 | "Because you liked X, played Y, and played Z." (Oxford comma) |
The verb ("liked"/"played") matches the dominant signal for each seed: if the user liked the artist, "liked"; otherwise "played." A single signal type per seed keeps the copy clean.
In v1 the seed names are plain text, not links. Wiring them to `/artists/{id}` is a polish-pass refinement (Fable #349).
### `<SuggestionFeed>` subcomponent
Owned by `/discover/+page.svelte`. ~50 lines. Responsibilities:
- Reads `createSuggestionsQuery()` for data.
- Manages the optimistic-requested `Set<string>` (mirrors the existing search-side machinery on the same page so suggestions and search results share the same optimistic state — a request from search hides the matching card from suggestions on next render too).
- Renders the empty state when `data.length === 0`.
- Maps suggestions → `<DiscoverResultCard>` with `state="requestable"` and the formatted `attribution` string.
Grid: same `grid-cols-2 sm:grid-cols-3 lg:grid-cols-4 xl:grid-cols-5 gap-4` as the search-results grid for visual continuity.
## 7. Error handling
### Worker-side (similarity ingest extension)
- `UpsertArtistSimilarityUnmatched` per-row failure logged at WARN, not propagated. Mirrors the existing `UpsertArtistSimilarity` row-error policy.
- Missing `Name` field on a `SimilarArtist` LB response: skip that row at DEBUG (we can't render a suggestion without the name). Verify the LB client surface during T2 — extend the client if `Name` isn't already exposed.
- Existing rate-limit/network handling carries over (same client call, just persisting more of the response).
### Handler-side (`/api/discover/suggestions`)
- DB error on the CTE query → `500 server_error`. SPA renders `<ApiErrorBanner>`. User can retry, or clear the input to fall back to search.
- Empty result is NOT an error — `200 []` with the SPA empty-state copy.
- No Lidarr-disabled branch on the read path: the suggestion feed reads pre-computed similarity data and works even when Lidarr is disconnected. The Request button on each card surfaces the M5a `lidarr_disabled` error when clicked, if Lidarr isn't configured.
### Stale-data behavior
- An admin adding the artist between worker re-ingest and the user's next refresh: the suggestion-query's `WHERE NOT EXISTS (SELECT 1 FROM artists WHERE mbid = ...)` filters in-library candidates at query time, so staleness window is "until next page refresh," not "until next worker tick." Acceptable.
- A user requesting an artist between page renders: same — the `WHERE NOT EXISTS (... lidarr_requests ...)` filters at query time.
## 8. Testing
### Unit tests (no DB)
- Pure-helper tests if any scoring logic lands in Go (not the SQL CTE). Most logic is in SQL; expect minimal Go unit-test surface.
### Integration tests (gated on `MINSTREL_TEST_DATABASE_URL`)
- `internal/recommendation/suggestions_integration_test.go`:
- **TestSuggestArtists_LikesAndPlaysContributeToScore** — single user, 1 liked + 1 played seed both pointing at the same out-of-library candidate; verify combined contribution.
- **TestSuggestArtists_Top12Cap** — 30 candidates with descending scores; verify only top-12 returned, in order.
- **TestSuggestArtists_AttributionTopThree** — 5 contributing seeds for one candidate; verify `Attribution` has exactly the top-3 by contribution.
- **TestSuggestArtists_RecencyDecayDownweightsOldPlays** — two seeds with 1-day vs 90-day-old plays pointing at the same candidate; verify recent contributes more (exact ratio derives from `exp(-age/half_life)`).
- **TestSuggestArtists_FiltersInLibraryCandidates** — candidate that exists in `artists` is excluded from response.
- **TestSuggestArtists_FiltersAlreadyRequested** — non-terminal `lidarr_requests` row hides the candidate.
- **TestSuggestArtists_RejectedRequestStillShown** — `status='rejected'` does NOT hide the candidate (rejected requests are terminal; user can re-request after admin's reject).
- **TestSuggestArtists_EmptyForNewUser** — user with no likes and no plays returns `[]`.
### Worker integration test
- `internal/similarity/worker_test.go``TestUpsertArtistSimilar_PersistsUnmatchedToTable`: seed an `artists` table with 1 in-library artist; mock the LB client to return 5 similar-artists where 1 is in-library and 4 are not. Verify `artist_similarity` got 1 row and `artist_similarity_unmatched` got 4 rows.
### HTTP handler tests
- `internal/api/suggestions_test.go`:
- **TestSuggestions_HappyPath** — stub the recommendation service, verify JSON shape.
- **TestSuggestions_EmptyForNewUser** — `200 []`.
- **TestSuggestions_AttributionShape** — response includes `attribution` array with up to 3 entries.
### Frontend tests (vitest)
- `web/src/routes/discover/discover.test.ts` extension:
- **suggestion feed renders when input is empty** — mock `createSuggestionsQuery` to return 12 rows; verify 12 cards rendered, kind tabs hidden.
- **typing replaces feed with search** — input `"miles"` → search results visible, suggestions hidden.
- **clearing input restores feed**.
- **attribution copy renders correctly for 1/2/3 seeds**.
- **request button optimistically removes the card**.
- **empty state copy** when query returns `[]`.
### Coverage targets
- `internal/recommendation/` (new code only) ≥ 80%.
- `internal/similarity/` (worker extension) maintained at current level (≥ 80% combined per existing target).
- Combined Lidarr-suite (lidarr, lidarrconfig, lidarrrequests, lidarrquarantine, plus new suggestion code) stays ≥ 80% per the M5a/M5b cadence.
## 9. Decisions ledger
| # | Decision | Rationale |
|---|---|---|
| 1 | Surface on `/discover` (default state, search-input-empty), not on `/api/radio` | Operator's redirection during brainstorming — recommendations should be a deliberate "I want to add music" surface, not a radio side-channel |
| 2 | Artist-only granularity for v1 | Cleanest mental model; matches Lidarr's monitor unit; album/track suggestions deferred to v2 |
| 3 | Recency-decayed plays + likes weighted 5× | Recency matches radio's existing decay model conceptually; 5× honors the M2 distinction between explicit (like) and implicit (play) signal |
| 4 | Top-3 contributing seeds named per suggestion | Operator framing was "you might like X because you liked Y, Z, **and W**" — multi-seed attribution is the whole point of the surface |
| 5 | Search-empty replaces initial-copy state with the feed; tabs hidden | Operator wanted suggestions front-and-center for users who don't know the feature exists; search is the explicit action |
| 6 | Top-12 fixed, no pagination | Score-vs-noise drops sharply past top-12; operator workflow ends with external search engine for actual decision |
| 7 | On-demand SQL + 5-minute SPA cache; no backend caching layer | Single CTE at household scale is sub-50ms; TanStack `staleTime` handles repeat visits; v2 materialization noted as a future lever if needed |
| 8 | New `artist_similarity_unmatched` table; extend M4b worker | Mirrors `artist_similarity` shape; worker already runs and discards these MBIDs today; persisting is a small addition |
| 9 | Reuse `<DiscoverResultCard>` with optional `attribution` prop | Cheaper than maintaining a parallel `<SuggestionCard>`; the attribution slot fits cleanly above the reserved badge row |
## 10. Out of scope (this slice)
- Album / track suggestions.
- Cross-user collaborative filtering ("users like you also liked X").
- Pagination / infinite scroll on the suggestion feed.
- Cover art for out-of-library candidates (would require MusicBrainz Cover-Art-Archive lookup).
- Linking attribution-seed names to `/artists/{id}` (polish-pass refinement).
- Materialized per-user signal aggregation. v2 lever if performance ever requires it.
- Tunable `half_life_days` exposed in the SPA. Backend accepts it, frontend always uses default.
## 11. Open questions
- **`SimilarArtist.Name` on the LB client.** Verify during T2 that the existing client surfaces the artist name alongside MBID. If not, extend the client (small, additive).
- **Cold-start duration after install.** Until the M4b worker has had a chance to run a few ticks against the user's listened-to artists, the unmatched table is small and suggestions will be sparse. Documented; not a v1 fix.
- **Cross-source diversity.** When `musicbrainz_tag` and `user_cooccurrence` source providers come online (post-M5), the same candidate MBID may appear from multiple sources with different scores. The `seeds → contributions` join already aggregates per `(seed, candidate)` regardless of source via the GROUP BY in the outer CTE — but ranking diversity (don't surface 12 candidates all sourced from one seed) is a polish-pass concern.