From 20963847c2367d5e5cf42a55991593c38b4a8ccd Mon Sep 17 00:00:00 2001 From: Bryan Van Deusen Date: Mon, 27 Apr 2026 19:57:10 -0400 Subject: [PATCH] docs(spec): add M3 session similarity + contextual_match_score design MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes M3 — adds the `contextual_match_score` term to the scoring formula via weighted-Jaccard similarity (tags 0.7, artists 0.3) over the user's contextual_likes. Reads the current session vector from the most recent open play_event (populated by sub-plan #2). Cold-start paths collapse to zero contribution, preserving v1 behavior. Co-Authored-By: Claude Opus 4.7 --- .../specs/2026-04-27-m3-similarity-design.md | 301 ++++++++++++++++++ 1 file changed, 301 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-27-m3-similarity-design.md diff --git a/docs/superpowers/specs/2026-04-27-m3-similarity-design.md b/docs/superpowers/specs/2026-04-27-m3-similarity-design.md new file mode 100644 index 00000000..44392aca --- /dev/null +++ b/docs/superpowers/specs/2026-04-27-m3-similarity-design.md @@ -0,0 +1,301 @@ +# M3 sub-plan #3 — Session similarity + contextual_match_score in scoring + +**Status:** Spec draft, 2026-04-27 +**Tracking:** Fable #342 +**Closes:** M3 milestone (recommendation engine v1) +**Builds on:** +- #340 — weighted shuffle baseline (`internal/recommendation` package, `Score`/`Shuffle`, `LoadRadioCandidates`) +- #341 — session vectors at play_started + `contextual_likes` capture on like + +## 1. Goal + +Add the `contextual_match_score` term to the recommendation scoring formula. For each +candidate track, compute the maximum weighted-Jaccard similarity between the user's +*current* session vector and the session vectors stored on that track's active +`contextual_likes` rows. Fold that scalar into `Score()` as +`+ contextual_match_score * ContextWeight`. + +When this slice ships, `/api/radio` produces context-aware recommendations: tracks the +user has previously liked while in similar musical contexts get an additive boost on +top of the v1 weighted-shuffle baseline. + +## 2. Non-goals + +- No new UI surface — `/api/radio` response shape is unchanged. +- No tag enrichment beyond `tracks.genre` (MBID tags / BPM remain post-v1). +- No similarity-axis weight exposure in YAML (hardcoded `0.7 / 0.3` for v1). +- No caching of the current session vector across requests. +- No "why this track?" debug endpoint. +- No ListenBrainz / external-similarity retrieval (M4). +- No GIN index on `play_events.session_vector_at_play` — we read the user's most + recent play by id, not by similarity. Existing `(user_id, started_at)` access + pattern is sufficient. + +## 3. Architecture overview + +Three additions to `internal/recommendation`, two adjustments to existing files, +one new sqlc query. + +### 3.1 New: `internal/recommendation/similarity.go` (pure) + +```go +type SimilarityWeights struct { + TagsWeight float64 + ArtistsWeight float64 +} + +// DefaultSimilarityWeights is the v1 axis balance per the M3 design. +// Hardcoded; not exposed via YAML — operators can't tune this for v1. +var DefaultSimilarityWeights = SimilarityWeights{ + TagsWeight: 0.7, + ArtistsWeight: 0.3, +} + +// Similarity returns weighted-Jaccard similarity in [0, 1]. Returns 0 if +// either input is Seed=true (low-confidence vectors don't contribute). +func Similarity(a, b SessionVector, w SimilarityWeights) float64 + +// ContextualMatchScore is the per-candidate scalar fed into ScoringInputs. +// Returns 0 if current.Seed is true OR likes (after filtering Seed=true +// entries) is empty. Otherwise: max(Similarity(current, l) for l in likes). +func ContextualMatchScore(current SessionVector, likes []SessionVector, w SimilarityWeights) float64 +``` + +**Per-axis semantics (classic set Jaccard):** + +- Tags axis flattens `map[string]int` keysets and computes `|A ∩ B| / |A ∪ B|`. + Bag-of-counts data is preserved on disk; we discard counts at similarity time. + Generalized Jaccard remains a one-line upgrade path if telemetry justifies it. +- Artists axis is already a `[]string` (deduplicated artist UUIDs); same Jaccard. +- Both axes empty (zero union) → axis returns 0.0, not NaN. + +### 3.2 New: `internal/db/queries/contextual_likes.sql` + +```sql +-- name: ListActiveContextualLikesForUser :many +-- Returns all the user's active (non-soft-deleted) contextual_likes with +-- non-null vectors. Cardinality bounded by the user's actual like-while- +-- playing history — typically tens to low hundreds. +SELECT track_id, session_vector +FROM contextual_likes +WHERE user_id = $1 AND deleted_at IS NULL AND session_vector IS NOT NULL; +``` + +### 3.3 New: `internal/db/queries/events.sql` (or `recommendation.sql`) + +```sql +-- name: GetCurrentSessionVectorForUser :one +-- Returns the session_vector_at_play of the user's most recent play_event +-- in a still-active (un-timed-out) session. NoRows means no current vector +-- — caller treats this as Seed=true sentinel. +SELECT pe.session_vector_at_play +FROM play_events pe +JOIN play_sessions s ON s.id = pe.session_id +WHERE pe.user_id = $1 + AND s.ended_at IS NULL +ORDER BY pe.started_at DESC +LIMIT 1; +``` + +(Final placement decided at implementation time — wherever sqlc and existing +query files line up best.) + +### 3.4 Modified: `internal/recommendation/score.go` + +```go +type ScoringInputs struct { + IsGeneralLiked bool + LastPlayedAt *time.Time + PlayCount int + SkipCount int + ContextualMatchScore float64 // NEW — in [0, 1], 0 when no signal +} + +type ScoringWeights struct { + BaseWeight float64 + LikeBoost float64 + RecencyWeight float64 + SkipPenalty float64 + JitterMagnitude float64 + ContextWeight float64 // NEW +} + +// Score gains: + in.ContextualMatchScore * w.ContextWeight +``` + +Zero-value defaults: a `ScoringInputs{}` with zero `ContextualMatchScore` and a +`ScoringWeights{}` with zero `ContextWeight` produce the v1 score. Existing callers +not passing the new fields see no behavior change. + +### 3.5 Modified: `internal/recommendation/candidates.go` + +```go +func LoadCandidates( + ctx context.Context, + q *dbq.Queries, + userID, seedID pgtype.UUID, + recentlyPlayedHours int, + currentVector SessionVector, // NEW +) ([]Candidate, error) +``` + +Body adds: + +1. Existing `LoadRadioCandidates` call. +2. New `ListActiveContextualLikesForUser(userID)` call. +3. Group result by `track_id` into `map[pgtype.UUID][]SessionVector`, unmarshaling + each `jsonb` column into `SessionVector`. Unmarshal failures are logged and + skipped (don't poison the entire response over one bad row). +4. For each candidate, set `Inputs.ContextualMatchScore = + ContextualMatchScore(currentVector, group[trackID], DefaultSimilarityWeights)`. + +### 3.6 Modified: `internal/api/radio.go` + +Before calling `LoadCandidates`, fetch the current session vector: + +```go +var currentVec recommendation.SessionVector +if raw, err := q.GetCurrentSessionVectorForUser(ctx, user.ID); err == nil && len(raw) > 0 { + if jerr := json.Unmarshal(raw, ¤tVec); jerr != nil { + h.logger.Warn("api: radio: bad session_vector_at_play", "err", jerr) + currentVec = recommendation.SessionVector{Seed: true} + } +} else { + currentVec = recommendation.SessionVector{Seed: true} +} +``` + +Pass `currentVec` into `LoadCandidates`. Pass `recCfg.ContextWeight` through into the +`ScoringWeights` struct alongside the existing weights. + +### 3.7 Modified: `internal/config/config.go` + +```go +type RecommendationConfig struct { + BaseWeight float64 `yaml:"base_weight"` + LikeBoost float64 `yaml:"like_boost"` + RecencyWeight float64 `yaml:"recency_weight"` + SkipPenalty float64 `yaml:"skip_penalty"` + JitterMagnitude float64 `yaml:"jitter_magnitude"` + ContextWeight float64 `yaml:"context_weight"` // NEW, default 2.0 + RecentlyPlayedHours int `yaml:"recently_played_hours"` + RadioSize int `yaml:"radio_size"` + RadioSizeMax int `yaml:"radio_size_max"` +} +``` + +`Default()` populates `ContextWeight: 2.0`. + +## 4. Request flow + +``` +GET /api/radio?seed_track=&limit=N + ↓ +handleRadio + 1. Auth, parse seed_track, parse limit (unchanged) + 2. q.GetCurrentSessionVectorForUser(userID) (NEW) + NoRows / NULL / unmarshal fail → SessionVector{Seed: true} + 3. recommendation.LoadCandidates(...) (extended) + a. q.LoadRadioCandidates (unchanged) + b. q.ListActiveContextualLikesForUser (NEW) + c. group by track_id → map[uuid][]SessionVector + d. per candidate: ContextualMatchScore() → ScoringInputs + 4. recommendation.Shuffle(candidates, weights, now, rng, limit-1) + Score() now folds in ContextualMatchScore * ContextWeight + 5. Resolve album/artist, build response (unchanged) +``` + +## 5. Cold-start handling + +Every cold-start path collapses to `contextual_match_score = 0` for all candidates, +so scoring degrades cleanly to v1 behavior: + +| Condition | Path | +|------------------------------------------------------|------------------------------------------------------| +| User has no `play_events` at all | `NoRows` → `Seed=true` sentinel → `ContextualMatchScore` returns 0 | +| User has plays but no active session | `NoRows` (joined `s.ended_at IS NULL` filters) | +| Active session but `session_vector_at_play` is NULL | `len(raw) == 0` → `Seed=true` sentinel | +| Vector populated but `Seed=true` | `ContextualMatchScore` short-circuits to 0 | +| Candidate has no `contextual_likes` | absent from map → empty slice → returns 0 | +| Candidate has only `Seed=true` likes | filtered out → empty → returns 0 | +| Candidate has only soft-deleted likes | excluded by `deleted_at IS NULL` in the SQL | + +## 6. Test plan + +### 6.1 `similarity_test.go` (pure, table-driven) + +- Identical vectors → `1.0`. +- Fully disjoint axes → `0.0`. +- Mixed: shared tags, no shared artists → `0.7 * tagJaccard`. +- Mixed: no shared tags, shared artists → `0.3 * artistJaccard`. +- Either input `Seed=true` → `0.0`. +- Both vectors fully empty → `0.0` (not NaN). +- One side empty on an axis, other side populated → that axis contributes 0. +- Weight balance: shared all tags, default weights → exactly `0.7`. + +### 6.2 `score_test.go` extensions + +- Perfect contextual match (`ContextualMatchScore=1.0`) at `ContextWeight=2.0` adds + exactly `+2.0` to the base score. +- Half match (`0.5`) adds `+1.0`. +- Zero match (`0.0`) leaves score unchanged from v1 behavior — guards backward compat. + +### 6.3 `candidates_test.go` (integration vs test DB) + +- Candidate with one matching `contextual_like` → `ContextualMatchScore > 0`. +- Candidate with multiple `contextual_likes` → max similarity wins. +- Candidate whose only `contextual_likes` are `Seed=true` → score 0. +- Candidate whose only `contextual_likes` are soft-deleted → score 0 (SQL filter). +- User with no `contextual_likes` anywhere → every candidate scores 0. +- User with only soft-deleted `contextual_likes` → every candidate scores 0. + +### 6.4 `radio_test.go` (integration, end-to-end) + +- Seed a current session vibe (3+ tracks of one genre/artist set) by inserting + `play_events` with populated `session_vector_at_play`. +- Insert a `contextual_like` whose `session_vector` matches that vibe, on track T. +- Insert an unrelated control track C with no contextual signal. +- Call `/api/radio` with a deterministic RNG and seed track distinct from T and C. +- Assert: T ranks above C in the response. + +### 6.5 Coverage gate + +Combined `internal/recommendation` coverage stays ≥ 70% (currently 78.5% combined +with `internal/playevents` post-#341; this slice's pure functions are highly +testable so we expect to land closer to 85%+). + +## 7. Backwards compatibility + +- `/api/radio` request and response shapes are unchanged — same query params, same + JSON output. Web client requires no edits. +- `ScoringInputs.ContextualMatchScore` and `ScoringWeights.ContextWeight` default to + `0` in zero-value structs. Pre-existing tests that construct these directly continue + passing without modification because the new term contributes nothing when both are + zero. +- `LoadCandidates` gains a `currentVector SessionVector` parameter — this is a + signature change, but the only caller is `internal/api/radio.go`, which we update + in this slice. No external consumers. +- DB schema is unchanged (migrations 0007 already shipped the table + indexes + needed for this slice). + +## 8. Out-of-scope (deferred) + +- Generalized (bag-of-counts) Jaccard if telemetry shows tag-dominance discrimination + matters. +- YAML exposure of `SimilarityWeights`. +- Per-user override of any recommendation weight. +- Caching `currentVector` across requests within a session. +- ListenBrainz / similar-artist retrieval (M4). +- `/api/radio?explain=true` style debug endpoint. +- Tag enrichment beyond `tracks.genre`. + +## 9. Milestone gate (closes M3) + +After this slice merges: + +- Recommendation engine has all three v1 components: weighted shuffle, session + vectors written, contextual matching folded into scoring. +- Manual end-to-end verification: like a track during a session of one vibe, build a + similar session later, observe the track surfaces above unrelated controls in + `/api/radio` output. +- M4 (radio refinements + scrobble polish) unblocked.