Files
minstrel/docs/superpowers/specs/2026-04-30-m5c-suggested-additions-design.md
T
bvandeusen a0a9fb201b docs(spec): add M5c suggested-additions design
Personalized artist suggestions on /discover (search-input-empty state).
On-demand SQL ranks out-of-library MBIDs from artist_similarity_unmatched
(new table, populated by extending the M4b similarity worker) by per-user
signal: likes weighted 5x plus recency-decayed plays (exp(-age_days/30)).
Top-12 with top-3 contributing seeds attributed per card. Reuses the M5a
DiscoverResultCard + POST /api/requests artist-add path.
2026-04-30 22:48:55 -04:00

361 lines
22 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# M5c — Suggested additions on `/discover`
> **Status:** Draft for review · 2026-04-30
>
> **Sub-plan of:** M5 (Lidarr integration). Final slice of the M5 trilogy:
>
> - **M5a** — Lidarr connection + search/add + admin shell. Shipped on `dev`.
> - **M5b** — Quarantine workflow. Shipped on `dev`, in PR #30.
> - **M5c (this spec)** — Personalized artist suggestions surfaced on `/discover`.
>
> Note: this spec deliberately diverges from the M5a §10 carve-out, which described surfacing out-of-library MBIDs in `/api/radio` responses. Operator redirected during brainstorming: the suggestion surface is a dedicated recommendation feed on `/discover`, decoupled from radio. The §10 reference is preserved here for traceability.
## 1. Goal
Authenticated users land on `/discover` and see a "Suggested for you" feed by default — top-12 out-of-library artists ranked by per-user signal (likes + recency-decayed plays) projected through ListenBrainz artist-similarity. Each suggestion shows attribution ("Because you liked X, played Y, and played Z"), and one click fires an artist-kind add request through M5a's existing Lidarr-add path.
When the user types in the search input, the suggestion feed is replaced by Lidarr search results — M5a's existing behavior. When the input clears, suggestions return.
## 2. Goals and non-goals
### Goals
- Top-12 personalized artist suggestions on `/discover` when the search input is empty.
- Per-user ranking weighted by explicit likes (×5) plus implicit plays (count, recency-decayed by half-life ~30 days).
- Top-3 contributing seed artists named per suggestion: "Because you liked X, played Y, and played Z."
- One-click add via the existing M5a `POST /api/requests` artist-kind path.
- Suggestions hide automatically when the candidate is in-library or already in a non-terminal `lidarr_requests` row.
- The M4b similarity ingest worker is extended to persist unmatched similar-artist MBIDs that it currently discards. No new background worker.
### Non-goals (this slice)
- Album-level or track-level suggestions. Artist-only for v1; albums are a potential v2 layer once we see how artist suggestions feel in practice.
- Realtime invalidation on every like/play. The feed updates passively as user signals accumulate; no push, no immediate refresh.
- Cross-user collaborative filtering. The recommendation is the user's own seed set × ListenBrainz similarity — single-user data only.
- Pagination beyond top-12. Polish slot if the operator wants it later.
- Materialized aggregation of `play_events`. Documented as a v2 lever if on-demand performance ever becomes an issue.
- Cover art for out-of-library suggestions. v1 uses the Lucide `Disc3` fallback; a MusicBrainz Cover-Art-Archive lookup is a polish-pass slot.
- Anything in the `/api/radio` response. M5c is decoupled from radio.
## 3. Architecture
### Data flow
1. **M4b similarity worker (existing)** fetches similar-artists from ListenBrainz per played artist. Today it discards MBIDs that aren't in `artists`. M5c keeps that filter for the `artist_similarity` table but ALSO writes the discarded MBIDs to a new `artist_similarity_unmatched` table — same structure but with no FK on the candidate side.
2. **Suggestion query** (on demand at `/api/discover/suggestions`) joins the user's likes + plays against `artist_similarity_unmatched` via the seeds, computes per-candidate score, ranks top-N.
3. **SPA renders** the response on `/discover` when the search input is empty. Each card reuses the existing `<DiscoverResultCard>` with an extra `attribution` prop.
4. **One-click Request** fires the same M5a `POST /api/requests` flow used by the search-side `<DiscoverResultCard>` — no new add machinery.
### New Go components
- **`internal/recommendation/suggestions.go`** — exports a single `SuggestArtists(ctx, pool, userID, halfLifeDays, limit) ([]ArtistSuggestion, error)` function. Single CTE query (see §4). Returns ranked candidates with their top-3 attribution seeds.
- **`internal/api/suggestions.go`** — `GET /api/discover/suggestions` handler. Authenticated; no admin gate. Calls the recommendation service and resolves seed artist names via a single `GetArtistsByIDs` follow-up (≤36 distinct IDs across top-12 candidates × 3 seeds each).
### Modified Go components
- **`internal/similarity/worker.go`** — `upsertArtistSimilar` keeps the existing matched path and adds a parallel unmatched-persist loop that calls a new `UpsertArtistSimilarityUnmatched` query. Top-K cap mirrors the matched path.
- **`internal/db/queries/similarity.sql`** — extension to ingest the new table. Reads in §4.
### Frontend changes
- `web/src/routes/discover/+page.svelte` — when `debouncedQ === ''`, render the new `<SuggestionFeed>` subcomponent. Hide the kind tabs in this state. Existing search behavior unchanged when input is non-empty.
- `web/src/lib/components/DiscoverResultCard.svelte` — add an optional `attribution?: string` prop that renders an italic Vellum line below the title.
- `web/src/lib/api/suggestions.ts` (new) — `listSuggestions(limit?)` and `createSuggestionsQuery()` factory with `staleTime: 5 * 60_000`.
- `web/src/lib/api/queries.ts``qk.suggestions(limit)` query key.
- `web/src/lib/api/types.ts``ArtistSuggestion`, `SeedContribution` types.
### Wiring
No `cmd/minstrel/main.go` or `internal/server/server.go` changes. The new endpoint mounts on the existing authed route group; the similarity worker is already constructed.
## 4. Schema — migration `0012_artist_similarity_unmatched`
```sql
-- M5c: persist unmatched-similar-artist MBIDs that the M4b worker would
-- otherwise discard. Mirrors artist_similarity shape: same composite PK
-- with source, same (seed_id, score DESC) index, same source enum check.
-- The candidate side is text + name (no FK) — that's the whole point.
CREATE TABLE artist_similarity_unmatched (
seed_artist_id uuid NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
candidate_mbid text NOT NULL,
candidate_name text NOT NULL,
score DOUBLE PRECISION NOT NULL,
source text NOT NULL CHECK (source IN ('listenbrainz', 'musicbrainz_tag', 'user_cooccurrence')),
fetched_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (seed_artist_id, candidate_mbid, source)
);
CREATE INDEX artist_similarity_unmatched_seed_score_idx
ON artist_similarity_unmatched (seed_artist_id, score DESC);
```
### Schema notes
- `seed_artist_id` cascades on artist delete — consistent with `artist_similarity`. Removes orphaned suggestion rows automatically when an operator deletes a seed artist.
- `candidate_mbid` deliberately has no FK and no unique constraint by itself; the same out-of-library MBID can appear for many seed artists, and that's the whole mechanism: aggregating contributions across the user's seeds is what produces the score.
- `(seed_artist_id, candidate_mbid, source)` PK gives the worker a clean upsert target while leaving room for future similarity sources.
- `(seed_artist_id, score DESC)` index satisfies the dominant query pattern: "for seed S, give me top-K unmatched candidates by score." This is what the suggestion query exploits inside its CTE join.
- Storage estimate at v1 scale: ~2k seed artists × ~100 unmatched candidates × ~150 bytes = **~30 MB**. Negligible.
### Down migration
```sql
DROP INDEX IF EXISTS artist_similarity_unmatched_seed_score_idx;
DROP TABLE IF EXISTS artist_similarity_unmatched;
```
### Suggestion query shape
```sql
-- name: SuggestArtistsForUser :many
WITH seeds AS (
SELECT a.id AS artist_id,
5.0 * (CASE WHEN gla.artist_id IS NOT NULL THEN 1 ELSE 0 END)
+ COALESCE(SUM(EXP(- EXTRACT(epoch FROM now() - pe.started_at) / ($2 * 86400.0))), 0)
AS signal
FROM artists a
LEFT JOIN general_likes_artists gla ON gla.artist_id = a.id AND gla.user_id = $1
LEFT JOIN tracks t ON t.artist_id = a.id
LEFT JOIN play_events pe ON pe.track_id = t.id AND pe.user_id = $1
WHERE gla.artist_id IS NOT NULL OR pe.id IS NOT NULL
GROUP BY a.id
),
contributions AS (
SELECT u.candidate_mbid,
u.candidate_name,
seeds.artist_id AS seed_id,
seeds.signal * u.score AS contribution
FROM artist_similarity_unmatched u
JOIN seeds ON seeds.artist_id = u.seed_artist_id
WHERE NOT EXISTS (SELECT 1 FROM artists WHERE mbid = u.candidate_mbid)
AND NOT EXISTS (
SELECT 1 FROM lidarr_requests r
WHERE r.user_id = $1
AND r.lidarr_artist_mbid = u.candidate_mbid
AND r.status NOT IN ('rejected', 'failed')
)
)
SELECT candidate_mbid,
candidate_name,
SUM(contribution)::float AS total_score,
(array_agg(seed_id ORDER BY contribution DESC))[1:3] AS top_seed_ids,
(array_agg(contribution ORDER BY contribution DESC))[1:3] AS top_contributions
FROM contributions
GROUP BY candidate_mbid, candidate_name
ORDER BY total_score DESC
LIMIT $3;
```
The handler then resolves the top-3 seed UUIDs to artist names via `GetArtistsByIDs` (one extra round-trip; cheap because there are at most 36 distinct seed IDs across top-12 candidates).
### `seeds` CTE rationale
- Likes contribute a flat `5.0` per liked artist — explicit user signal worth more than a single play.
- Plays contribute `Σ exp(-age_days / half_life)` summed across all play events for the user × artist. Recent plays carry close to 1.0; week-old plays ~0.79; 30-day-old plays ~0.37; 90-day-old plays ~0.05. The exponential decay matches what radio's `recencyDecay` already does conceptually (different polarity).
- `WHERE gla.artist_id IS NOT NULL OR pe.id IS NOT NULL` skips artists the user has neither liked nor played — they'd contribute zero signal anyway.
### Worker upsert query
```sql
-- name: UpsertArtistSimilarityUnmatched :exec
INSERT INTO artist_similarity_unmatched (seed_artist_id, candidate_mbid, candidate_name, score, source)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (seed_artist_id, candidate_mbid, source) DO UPDATE SET
candidate_name = EXCLUDED.candidate_name,
score = EXCLUDED.score,
fetched_at = now();
```
Idempotent on re-fetch; `candidate_name` and `score` get refreshed when ListenBrainz returns updated values.
## 5. API surface
| Method | Path | Behavior |
|---|---|---|
| `GET` | `/api/discover/suggestions?limit=&half_life_days=` | Returns the caller's top-N artist suggestions. Defaults `limit=12` (capped at 50), `half_life_days=30`. Both query params are operator-tunable; the SPA only sends `limit` in v1. Returns `200 [{mbid, name, score, attribution: [{artist_id, name, contribution}]}]`. |
### Response type
```ts
type ArtistSuggestion = {
mbid: string;
name: string;
score: number;
attribution: SeedContribution[]; // top-3, ordered by contribution DESC
};
type SeedContribution = {
artist_id: string;
name: string;
contribution: number;
};
```
### Empty cases
- New user with no likes and no plays → `200 []`. Empty-state copy: "Listen to something or like an artist to start getting suggestions."
- All recommendations already in library or already requested → `200 []`. Same copy works (the operator's library exhaustively covers their taste).
### Error codes added
None. The endpoint is read-only and can only fail on internal DB error → `500 server_error`.
### Performance
- Single CTE query at request time + one follow-up `GetArtistsByIDs` lookup for attribution names. Sub-50ms end-to-end at household scale (a few thousand seeds × hundreds of candidates fits well within Postgres' happy path with the planned indexes).
- Frontend wraps the call in TanStack Query with `staleTime: 5 * 60_000` (5 minutes). Repeat visits within a session don't re-query.
### v2 lever (deferred)
If the per-request `seeds` CTE aggregation over `play_events` becomes the bottleneck at large-library scale (year-3 territory), materialize a per-user `artist_signal_cache` table refreshed daily. The suggestion query plugs into the cache instead of scanning `play_events`. No API surface change required.
## 6. UI surfaces
All against the FabledSword design tokens. Voice rule: sentence case, "understated mythic" register on errors and empty states.
### `/discover` page-level state machine
The page has these states (existing M5a states preserved unless noted):
| Search input | Header copy | Tabs | Content |
|---|---|---|---|
| Empty | **"Suggested for you"** + Vellum subtitle (new) | hidden (new) | Suggestion grid, top-12 (new) |
| Empty + zero suggestions | **"Suggested for you"** | hidden | Empty-state copy (new) |
| Non-empty | "Add music to the library" (existing) | visible (existing) | Lidarr search results (existing) |
Subtitle when suggestions are showing: "Out-of-library artists drawn from what you've liked and played."
Empty-state copy when no suggestions: "Listen to something or like an artist to start getting suggestions."
When the user starts typing, the suggestion feed swaps out for the search-results UI. When they clear the input, the suggestion feed comes back. TanStack `staleTime` keeps both responses cached so swaps within a session are instant.
### `<DiscoverResultCard>` extension
Add an optional `attribution?: string` prop. When set, renders below the title in italic Vellum 12px. Empty / undefined → no attribution row (existing search-result rendering unchanged).
Card layout (top to bottom) with attribution:
- Art square (1:1 aspect, Slate fallback with Lucide `Disc3`).
- **Artist name** in Parchment, font-medium 14px.
- **Attribution** (italic Vellum 12px) — only when prop is set.
- Reserved badge slot (22px min-height; empty for suggestions).
- Action button: Moss "Request" with Plus icon.
### Attribution copy format
The handler returns up to 3 contributors per suggestion, ordered by contribution. The SPA formats:
| Contributors | Format |
|---|---|
| 1 | "Because you liked X." or "Because you played X." |
| 2 | "Because you liked X and played Y." |
| 3 | "Because you liked X, played Y, and played Z." (Oxford comma) |
The verb ("liked"/"played") matches the dominant signal for each seed: if the user liked the artist, "liked"; otherwise "played." A single signal type per seed keeps the copy clean.
In v1 the seed names are plain text, not links. Wiring them to `/artists/{id}` is a polish-pass refinement (Fable #349).
### `<SuggestionFeed>` subcomponent
Owned by `/discover/+page.svelte`. ~50 lines. Responsibilities:
- Reads `createSuggestionsQuery()` for data.
- Manages the optimistic-requested `Set<string>` (mirrors the existing search-side machinery on the same page so suggestions and search results share the same optimistic state — a request from search hides the matching card from suggestions on next render too).
- Renders the empty state when `data.length === 0`.
- Maps suggestions → `<DiscoverResultCard>` with `state="requestable"` and the formatted `attribution` string.
Grid: same `grid-cols-2 sm:grid-cols-3 lg:grid-cols-4 xl:grid-cols-5 gap-4` as the search-results grid for visual continuity.
## 7. Error handling
### Worker-side (similarity ingest extension)
- `UpsertArtistSimilarityUnmatched` per-row failure logged at WARN, not propagated. Mirrors the existing `UpsertArtistSimilarity` row-error policy.
- Missing `Name` field on a `SimilarArtist` LB response: skip that row at DEBUG (we can't render a suggestion without the name). Verify the LB client surface during T2 — extend the client if `Name` isn't already exposed.
- Existing rate-limit/network handling carries over (same client call, just persisting more of the response).
### Handler-side (`/api/discover/suggestions`)
- DB error on the CTE query → `500 server_error`. SPA renders `<ApiErrorBanner>`. User can retry, or clear the input to fall back to search.
- Empty result is NOT an error — `200 []` with the SPA empty-state copy.
- No Lidarr-disabled branch on the read path: the suggestion feed reads pre-computed similarity data and works even when Lidarr is disconnected. The Request button on each card surfaces the M5a `lidarr_disabled` error when clicked, if Lidarr isn't configured.
### Stale-data behavior
- An admin adding the artist between worker re-ingest and the user's next refresh: the suggestion-query's `WHERE NOT EXISTS (SELECT 1 FROM artists WHERE mbid = ...)` filters in-library candidates at query time, so staleness window is "until next page refresh," not "until next worker tick." Acceptable.
- A user requesting an artist between page renders: same — the `WHERE NOT EXISTS (... lidarr_requests ...)` filters at query time.
## 8. Testing
### Unit tests (no DB)
- Pure-helper tests if any scoring logic lands in Go (not the SQL CTE). Most logic is in SQL; expect minimal Go unit-test surface.
### Integration tests (gated on `MINSTREL_TEST_DATABASE_URL`)
- `internal/recommendation/suggestions_integration_test.go`:
- **TestSuggestArtists_LikesAndPlaysContributeToScore** — single user, 1 liked + 1 played seed both pointing at the same out-of-library candidate; verify combined contribution.
- **TestSuggestArtists_Top12Cap** — 30 candidates with descending scores; verify only top-12 returned, in order.
- **TestSuggestArtists_AttributionTopThree** — 5 contributing seeds for one candidate; verify `Attribution` has exactly the top-3 by contribution.
- **TestSuggestArtists_RecencyDecayDownweightsOldPlays** — two seeds with 1-day vs 90-day-old plays pointing at the same candidate; verify recent contributes more (exact ratio derives from `exp(-age/half_life)`).
- **TestSuggestArtists_FiltersInLibraryCandidates** — candidate that exists in `artists` is excluded from response.
- **TestSuggestArtists_FiltersAlreadyRequested** — non-terminal `lidarr_requests` row hides the candidate.
- **TestSuggestArtists_RejectedRequestStillShown** — `status='rejected'` does NOT hide the candidate (rejected requests are terminal; user can re-request after admin's reject).
- **TestSuggestArtists_EmptyForNewUser** — user with no likes and no plays returns `[]`.
### Worker integration test
- `internal/similarity/worker_test.go``TestUpsertArtistSimilar_PersistsUnmatchedToTable`: seed an `artists` table with 1 in-library artist; mock the LB client to return 5 similar-artists where 1 is in-library and 4 are not. Verify `artist_similarity` got 1 row and `artist_similarity_unmatched` got 4 rows.
### HTTP handler tests
- `internal/api/suggestions_test.go`:
- **TestSuggestions_HappyPath** — stub the recommendation service, verify JSON shape.
- **TestSuggestions_EmptyForNewUser** — `200 []`.
- **TestSuggestions_AttributionShape** — response includes `attribution` array with up to 3 entries.
### Frontend tests (vitest)
- `web/src/routes/discover/discover.test.ts` extension:
- **suggestion feed renders when input is empty** — mock `createSuggestionsQuery` to return 12 rows; verify 12 cards rendered, kind tabs hidden.
- **typing replaces feed with search** — input `"miles"` → search results visible, suggestions hidden.
- **clearing input restores feed**.
- **attribution copy renders correctly for 1/2/3 seeds**.
- **request button optimistically removes the card**.
- **empty state copy** when query returns `[]`.
### Coverage targets
- `internal/recommendation/` (new code only) ≥ 80%.
- `internal/similarity/` (worker extension) maintained at current level (≥ 80% combined per existing target).
- Combined Lidarr-suite (lidarr, lidarrconfig, lidarrrequests, lidarrquarantine, plus new suggestion code) stays ≥ 80% per the M5a/M5b cadence.
## 9. Decisions ledger
| # | Decision | Rationale |
|---|---|---|
| 1 | Surface on `/discover` (default state, search-input-empty), not on `/api/radio` | Operator's redirection during brainstorming — recommendations should be a deliberate "I want to add music" surface, not a radio side-channel |
| 2 | Artist-only granularity for v1 | Cleanest mental model; matches Lidarr's monitor unit; album/track suggestions deferred to v2 |
| 3 | Recency-decayed plays + likes weighted 5× | Recency matches radio's existing decay model conceptually; 5× honors the M2 distinction between explicit (like) and implicit (play) signal |
| 4 | Top-3 contributing seeds named per suggestion | Operator framing was "you might like X because you liked Y, Z, **and W**" — multi-seed attribution is the whole point of the surface |
| 5 | Search-empty replaces initial-copy state with the feed; tabs hidden | Operator wanted suggestions front-and-center for users who don't know the feature exists; search is the explicit action |
| 6 | Top-12 fixed, no pagination | Score-vs-noise drops sharply past top-12; operator workflow ends with external search engine for actual decision |
| 7 | On-demand SQL + 5-minute SPA cache; no backend caching layer | Single CTE at household scale is sub-50ms; TanStack `staleTime` handles repeat visits; v2 materialization noted as a future lever if needed |
| 8 | New `artist_similarity_unmatched` table; extend M4b worker | Mirrors `artist_similarity` shape; worker already runs and discards these MBIDs today; persisting is a small addition |
| 9 | Reuse `<DiscoverResultCard>` with optional `attribution` prop | Cheaper than maintaining a parallel `<SuggestionCard>`; the attribution slot fits cleanly above the reserved badge row |
## 10. Out of scope (this slice)
- Album / track suggestions.
- Cross-user collaborative filtering ("users like you also liked X").
- Pagination / infinite scroll on the suggestion feed.
- Cover art for out-of-library candidates (would require MusicBrainz Cover-Art-Archive lookup).
- Linking attribution-seed names to `/artists/{id}` (polish-pass refinement).
- Materialized per-user signal aggregation. v2 lever if performance ever requires it.
- Tunable `half_life_days` exposed in the SPA. Backend accepts it, frontend always uses default.
## 11. Open questions
- **`SimilarArtist.Name` on the LB client.** Verify during T2 that the existing client surfaces the artist name alongside MBID. If not, extend the client (small, additive).
- **Cold-start duration after install.** Until the M4b worker has had a chance to run a few ticks against the user's listened-to artists, the unmatched table is small and suggestions will be sparse. Documented; not a v1 fix.
- **Cross-source diversity.** When `musicbrainz_tag` and `user_cooccurrence` source providers come online (post-M5), the same candidate MBID may appear from multiple sources with different scores. The `seeds → contributions` join already aggregates per `(seed, candidate)` regardless of source via the GROUP BY in the outer CTE — but ranking diversity (don't surface 12 candidates all sourced from one seed) is a polish-pass concern.