docs(spec): add M3 session similarity + contextual_match_score design
Closes M3 — adds the `contextual_match_score` term to the scoring formula via weighted-Jaccard similarity (tags 0.7, artists 0.3) over the user's contextual_likes. Reads the current session vector from the most recent open play_event (populated by sub-plan #2). Cold-start paths collapse to zero contribution, preserving v1 behavior. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,301 @@
|
||||
# M3 sub-plan #3 — Session similarity + contextual_match_score in scoring
|
||||
|
||||
**Status:** Spec draft, 2026-04-27
|
||||
**Tracking:** Fable #342
|
||||
**Closes:** M3 milestone (recommendation engine v1)
|
||||
**Builds on:**
|
||||
- #340 — weighted shuffle baseline (`internal/recommendation` package, `Score`/`Shuffle`, `LoadRadioCandidates`)
|
||||
- #341 — session vectors at play_started + `contextual_likes` capture on like
|
||||
|
||||
## 1. Goal
|
||||
|
||||
Add the `contextual_match_score` term to the recommendation scoring formula. For each
|
||||
candidate track, compute the maximum weighted-Jaccard similarity between the user's
|
||||
*current* session vector and the session vectors stored on that track's active
|
||||
`contextual_likes` rows. Fold that scalar into `Score()` as
|
||||
`+ contextual_match_score * ContextWeight`.
|
||||
|
||||
When this slice ships, `/api/radio` produces context-aware recommendations: tracks the
|
||||
user has previously liked while in similar musical contexts get an additive boost on
|
||||
top of the v1 weighted-shuffle baseline.
|
||||
|
||||
## 2. Non-goals
|
||||
|
||||
- No new UI surface — `/api/radio` response shape is unchanged.
|
||||
- No tag enrichment beyond `tracks.genre` (MBID tags / BPM remain post-v1).
|
||||
- No similarity-axis weight exposure in YAML (hardcoded `0.7 / 0.3` for v1).
|
||||
- No caching of the current session vector across requests.
|
||||
- No "why this track?" debug endpoint.
|
||||
- No ListenBrainz / external-similarity retrieval (M4).
|
||||
- No GIN index on `play_events.session_vector_at_play` — we read the user's most
|
||||
recent play by id, not by similarity. Existing `(user_id, started_at)` access
|
||||
pattern is sufficient.
|
||||
|
||||
## 3. Architecture overview
|
||||
|
||||
Three additions to `internal/recommendation`, two adjustments to existing files,
|
||||
one new sqlc query.
|
||||
|
||||
### 3.1 New: `internal/recommendation/similarity.go` (pure)
|
||||
|
||||
```go
|
||||
type SimilarityWeights struct {
|
||||
TagsWeight float64
|
||||
ArtistsWeight float64
|
||||
}
|
||||
|
||||
// DefaultSimilarityWeights is the v1 axis balance per the M3 design.
|
||||
// Hardcoded; not exposed via YAML — operators can't tune this for v1.
|
||||
var DefaultSimilarityWeights = SimilarityWeights{
|
||||
TagsWeight: 0.7,
|
||||
ArtistsWeight: 0.3,
|
||||
}
|
||||
|
||||
// Similarity returns weighted-Jaccard similarity in [0, 1]. Returns 0 if
|
||||
// either input is Seed=true (low-confidence vectors don't contribute).
|
||||
func Similarity(a, b SessionVector, w SimilarityWeights) float64
|
||||
|
||||
// ContextualMatchScore is the per-candidate scalar fed into ScoringInputs.
|
||||
// Returns 0 if current.Seed is true OR likes (after filtering Seed=true
|
||||
// entries) is empty. Otherwise: max(Similarity(current, l) for l in likes).
|
||||
func ContextualMatchScore(current SessionVector, likes []SessionVector, w SimilarityWeights) float64
|
||||
```
|
||||
|
||||
**Per-axis semantics (classic set Jaccard):**
|
||||
|
||||
- Tags axis flattens `map[string]int` keysets and computes `|A ∩ B| / |A ∪ B|`.
|
||||
Bag-of-counts data is preserved on disk; we discard counts at similarity time.
|
||||
Generalized Jaccard remains a one-line upgrade path if telemetry justifies it.
|
||||
- Artists axis is already a `[]string` (deduplicated artist UUIDs); same Jaccard.
|
||||
- Both axes empty (zero union) → axis returns 0.0, not NaN.
|
||||
|
||||
### 3.2 New: `internal/db/queries/contextual_likes.sql`
|
||||
|
||||
```sql
|
||||
-- name: ListActiveContextualLikesForUser :many
|
||||
-- Returns all the user's active (non-soft-deleted) contextual_likes with
|
||||
-- non-null vectors. Cardinality bounded by the user's actual like-while-
|
||||
-- playing history — typically tens to low hundreds.
|
||||
SELECT track_id, session_vector
|
||||
FROM contextual_likes
|
||||
WHERE user_id = $1 AND deleted_at IS NULL AND session_vector IS NOT NULL;
|
||||
```
|
||||
|
||||
### 3.3 New: `internal/db/queries/events.sql` (or `recommendation.sql`)
|
||||
|
||||
```sql
|
||||
-- name: GetCurrentSessionVectorForUser :one
|
||||
-- Returns the session_vector_at_play of the user's most recent play_event
|
||||
-- in a still-active (un-timed-out) session. NoRows means no current vector
|
||||
-- — caller treats this as Seed=true sentinel.
|
||||
SELECT pe.session_vector_at_play
|
||||
FROM play_events pe
|
||||
JOIN play_sessions s ON s.id = pe.session_id
|
||||
WHERE pe.user_id = $1
|
||||
AND s.ended_at IS NULL
|
||||
ORDER BY pe.started_at DESC
|
||||
LIMIT 1;
|
||||
```
|
||||
|
||||
(Final placement decided at implementation time — wherever sqlc and existing
|
||||
query files line up best.)
|
||||
|
||||
### 3.4 Modified: `internal/recommendation/score.go`
|
||||
|
||||
```go
|
||||
type ScoringInputs struct {
|
||||
IsGeneralLiked bool
|
||||
LastPlayedAt *time.Time
|
||||
PlayCount int
|
||||
SkipCount int
|
||||
ContextualMatchScore float64 // NEW — in [0, 1], 0 when no signal
|
||||
}
|
||||
|
||||
type ScoringWeights struct {
|
||||
BaseWeight float64
|
||||
LikeBoost float64
|
||||
RecencyWeight float64
|
||||
SkipPenalty float64
|
||||
JitterMagnitude float64
|
||||
ContextWeight float64 // NEW
|
||||
}
|
||||
|
||||
// Score gains: + in.ContextualMatchScore * w.ContextWeight
|
||||
```
|
||||
|
||||
Zero-value defaults: a `ScoringInputs{}` with zero `ContextualMatchScore` and a
|
||||
`ScoringWeights{}` with zero `ContextWeight` produce the v1 score. Existing callers
|
||||
not passing the new fields see no behavior change.
|
||||
|
||||
### 3.5 Modified: `internal/recommendation/candidates.go`
|
||||
|
||||
```go
|
||||
func LoadCandidates(
|
||||
ctx context.Context,
|
||||
q *dbq.Queries,
|
||||
userID, seedID pgtype.UUID,
|
||||
recentlyPlayedHours int,
|
||||
currentVector SessionVector, // NEW
|
||||
) ([]Candidate, error)
|
||||
```
|
||||
|
||||
Body adds:
|
||||
|
||||
1. Existing `LoadRadioCandidates` call.
|
||||
2. New `ListActiveContextualLikesForUser(userID)` call.
|
||||
3. Group result by `track_id` into `map[pgtype.UUID][]SessionVector`, unmarshaling
|
||||
each `jsonb` column into `SessionVector`. Unmarshal failures are logged and
|
||||
skipped (don't poison the entire response over one bad row).
|
||||
4. For each candidate, set `Inputs.ContextualMatchScore =
|
||||
ContextualMatchScore(currentVector, group[trackID], DefaultSimilarityWeights)`.
|
||||
|
||||
### 3.6 Modified: `internal/api/radio.go`
|
||||
|
||||
Before calling `LoadCandidates`, fetch the current session vector:
|
||||
|
||||
```go
|
||||
var currentVec recommendation.SessionVector
|
||||
if raw, err := q.GetCurrentSessionVectorForUser(ctx, user.ID); err == nil && len(raw) > 0 {
|
||||
if jerr := json.Unmarshal(raw, ¤tVec); jerr != nil {
|
||||
h.logger.Warn("api: radio: bad session_vector_at_play", "err", jerr)
|
||||
currentVec = recommendation.SessionVector{Seed: true}
|
||||
}
|
||||
} else {
|
||||
currentVec = recommendation.SessionVector{Seed: true}
|
||||
}
|
||||
```
|
||||
|
||||
Pass `currentVec` into `LoadCandidates`. Pass `recCfg.ContextWeight` through into the
|
||||
`ScoringWeights` struct alongside the existing weights.
|
||||
|
||||
### 3.7 Modified: `internal/config/config.go`
|
||||
|
||||
```go
|
||||
type RecommendationConfig struct {
|
||||
BaseWeight float64 `yaml:"base_weight"`
|
||||
LikeBoost float64 `yaml:"like_boost"`
|
||||
RecencyWeight float64 `yaml:"recency_weight"`
|
||||
SkipPenalty float64 `yaml:"skip_penalty"`
|
||||
JitterMagnitude float64 `yaml:"jitter_magnitude"`
|
||||
ContextWeight float64 `yaml:"context_weight"` // NEW, default 2.0
|
||||
RecentlyPlayedHours int `yaml:"recently_played_hours"`
|
||||
RadioSize int `yaml:"radio_size"`
|
||||
RadioSizeMax int `yaml:"radio_size_max"`
|
||||
}
|
||||
```
|
||||
|
||||
`Default()` populates `ContextWeight: 2.0`.
|
||||
|
||||
## 4. Request flow
|
||||
|
||||
```
|
||||
GET /api/radio?seed_track=<uuid>&limit=N
|
||||
↓
|
||||
handleRadio
|
||||
1. Auth, parse seed_track, parse limit (unchanged)
|
||||
2. q.GetCurrentSessionVectorForUser(userID) (NEW)
|
||||
NoRows / NULL / unmarshal fail → SessionVector{Seed: true}
|
||||
3. recommendation.LoadCandidates(...) (extended)
|
||||
a. q.LoadRadioCandidates (unchanged)
|
||||
b. q.ListActiveContextualLikesForUser (NEW)
|
||||
c. group by track_id → map[uuid][]SessionVector
|
||||
d. per candidate: ContextualMatchScore() → ScoringInputs
|
||||
4. recommendation.Shuffle(candidates, weights, now, rng, limit-1)
|
||||
Score() now folds in ContextualMatchScore * ContextWeight
|
||||
5. Resolve album/artist, build response (unchanged)
|
||||
```
|
||||
|
||||
## 5. Cold-start handling
|
||||
|
||||
Every cold-start path collapses to `contextual_match_score = 0` for all candidates,
|
||||
so scoring degrades cleanly to v1 behavior:
|
||||
|
||||
| Condition | Path |
|
||||
|------------------------------------------------------|------------------------------------------------------|
|
||||
| User has no `play_events` at all | `NoRows` → `Seed=true` sentinel → `ContextualMatchScore` returns 0 |
|
||||
| User has plays but no active session | `NoRows` (joined `s.ended_at IS NULL` filters) |
|
||||
| Active session but `session_vector_at_play` is NULL | `len(raw) == 0` → `Seed=true` sentinel |
|
||||
| Vector populated but `Seed=true` | `ContextualMatchScore` short-circuits to 0 |
|
||||
| Candidate has no `contextual_likes` | absent from map → empty slice → returns 0 |
|
||||
| Candidate has only `Seed=true` likes | filtered out → empty → returns 0 |
|
||||
| Candidate has only soft-deleted likes | excluded by `deleted_at IS NULL` in the SQL |
|
||||
|
||||
## 6. Test plan
|
||||
|
||||
### 6.1 `similarity_test.go` (pure, table-driven)
|
||||
|
||||
- Identical vectors → `1.0`.
|
||||
- Fully disjoint axes → `0.0`.
|
||||
- Mixed: shared tags, no shared artists → `0.7 * tagJaccard`.
|
||||
- Mixed: no shared tags, shared artists → `0.3 * artistJaccard`.
|
||||
- Either input `Seed=true` → `0.0`.
|
||||
- Both vectors fully empty → `0.0` (not NaN).
|
||||
- One side empty on an axis, other side populated → that axis contributes 0.
|
||||
- Weight balance: shared all tags, default weights → exactly `0.7`.
|
||||
|
||||
### 6.2 `score_test.go` extensions
|
||||
|
||||
- Perfect contextual match (`ContextualMatchScore=1.0`) at `ContextWeight=2.0` adds
|
||||
exactly `+2.0` to the base score.
|
||||
- Half match (`0.5`) adds `+1.0`.
|
||||
- Zero match (`0.0`) leaves score unchanged from v1 behavior — guards backward compat.
|
||||
|
||||
### 6.3 `candidates_test.go` (integration vs test DB)
|
||||
|
||||
- Candidate with one matching `contextual_like` → `ContextualMatchScore > 0`.
|
||||
- Candidate with multiple `contextual_likes` → max similarity wins.
|
||||
- Candidate whose only `contextual_likes` are `Seed=true` → score 0.
|
||||
- Candidate whose only `contextual_likes` are soft-deleted → score 0 (SQL filter).
|
||||
- User with no `contextual_likes` anywhere → every candidate scores 0.
|
||||
- User with only soft-deleted `contextual_likes` → every candidate scores 0.
|
||||
|
||||
### 6.4 `radio_test.go` (integration, end-to-end)
|
||||
|
||||
- Seed a current session vibe (3+ tracks of one genre/artist set) by inserting
|
||||
`play_events` with populated `session_vector_at_play`.
|
||||
- Insert a `contextual_like` whose `session_vector` matches that vibe, on track T.
|
||||
- Insert an unrelated control track C with no contextual signal.
|
||||
- Call `/api/radio` with a deterministic RNG and seed track distinct from T and C.
|
||||
- Assert: T ranks above C in the response.
|
||||
|
||||
### 6.5 Coverage gate
|
||||
|
||||
Combined `internal/recommendation` coverage stays ≥ 70% (currently 78.5% combined
|
||||
with `internal/playevents` post-#341; this slice's pure functions are highly
|
||||
testable so we expect to land closer to 85%+).
|
||||
|
||||
## 7. Backwards compatibility
|
||||
|
||||
- `/api/radio` request and response shapes are unchanged — same query params, same
|
||||
JSON output. Web client requires no edits.
|
||||
- `ScoringInputs.ContextualMatchScore` and `ScoringWeights.ContextWeight` default to
|
||||
`0` in zero-value structs. Pre-existing tests that construct these directly continue
|
||||
passing without modification because the new term contributes nothing when both are
|
||||
zero.
|
||||
- `LoadCandidates` gains a `currentVector SessionVector` parameter — this is a
|
||||
signature change, but the only caller is `internal/api/radio.go`, which we update
|
||||
in this slice. No external consumers.
|
||||
- DB schema is unchanged (migrations 0007 already shipped the table + indexes
|
||||
needed for this slice).
|
||||
|
||||
## 8. Out-of-scope (deferred)
|
||||
|
||||
- Generalized (bag-of-counts) Jaccard if telemetry shows tag-dominance discrimination
|
||||
matters.
|
||||
- YAML exposure of `SimilarityWeights`.
|
||||
- Per-user override of any recommendation weight.
|
||||
- Caching `currentVector` across requests within a session.
|
||||
- ListenBrainz / similar-artist retrieval (M4).
|
||||
- `/api/radio?explain=true` style debug endpoint.
|
||||
- Tag enrichment beyond `tracks.genre`.
|
||||
|
||||
## 9. Milestone gate (closes M3)
|
||||
|
||||
After this slice merges:
|
||||
|
||||
- Recommendation engine has all three v1 components: weighted shuffle, session
|
||||
vectors written, contextual matching folded into scoring.
|
||||
- Manual end-to-end verification: like a track during a session of one vibe, build a
|
||||
similar session later, observe the track surfaces above unrelated controls in
|
||||
`/api/radio` output.
|
||||
- M4 (radio refinements + scrobble polish) unblocked.
|
||||
Reference in New Issue
Block a user