minstrel/docs/superpowers/specs/2026-04-27-m3-similarity-design.md

# M3 sub-plan #3 — Session similarity + contextual_match_score in scoring

**Status:** Spec draft, 2026-04-27
**Tracking:** Fable #342
**Closes:** M3 milestone (recommendation engine v1)
**Builds on:**
- #340 — weighted shuffle baseline (`internal/recommendation` package, `Score`/`Shuffle`, `LoadRadioCandidates`)
- #341 — session vectors at play_started + `contextual_likes` capture on like

## 1. Goal

Add the `contextual_match_score` term to the recommendation scoring formula. For each
candidate track, compute the maximum weighted-Jaccard similarity between the user's
*current* session vector and the session vectors stored on that track's active
`contextual_likes` rows. Fold that scalar into `Score()` as
`+ contextual_match_score * ContextWeight`.

When this slice ships, `/api/radio` produces context-aware recommendations: tracks the
user has previously liked while in similar musical contexts get an additive boost on
top of the v1 weighted-shuffle baseline.

## 2. Non-goals

- No new UI surface — `/api/radio` response shape is unchanged.
- No tag enrichment beyond `tracks.genre` (MBID tags / BPM remain post-v1).
- No similarity-axis weight exposure in YAML (hardcoded `0.7 / 0.3` for v1).
- No caching of the current session vector across requests.
- No "why this track?" debug endpoint.
- No ListenBrainz / external-similarity retrieval (M4).
- No GIN index on `play_events.session_vector_at_play` — we read the user's most
  recent play by id, not by similarity. Existing `(user_id, started_at)` access
  pattern is sufficient.

## 3. Architecture overview

Three additions to `internal/recommendation`, two adjustments to existing files,
one new sqlc query.

### 3.1 New: `internal/recommendation/similarity.go` (pure)

```go
type SimilarityWeights struct {
    TagsWeight    float64
    ArtistsWeight float64
}

// DefaultSimilarityWeights is the v1 axis balance per the M3 design.
// Hardcoded; not exposed via YAML — operators can't tune this for v1.
var DefaultSimilarityWeights = SimilarityWeights{
    TagsWeight:    0.7,
    ArtistsWeight: 0.3,
}

// Similarity returns weighted-Jaccard similarity in [0, 1]. Returns 0 if
// either input is Seed=true (low-confidence vectors don't contribute).
func Similarity(a, b SessionVector, w SimilarityWeights) float64

// ContextualMatchScore is the per-candidate scalar fed into ScoringInputs.
// Returns 0 if current.Seed is true OR likes (after filtering Seed=true
// entries) is empty. Otherwise: max(Similarity(current, l) for l in likes).
func ContextualMatchScore(current SessionVector, likes []SessionVector, w SimilarityWeights) float64
```

**Per-axis semantics (classic set Jaccard):**

- Tags axis flattens `map[string]int` keysets and computes `|A ∩ B| / |A ∪ B|`.
  Bag-of-counts data is preserved on disk; we discard counts at similarity time.
  Generalized Jaccard remains a one-line upgrade path if telemetry justifies it.
- Artists axis is already a `[]string` (deduplicated artist UUIDs); same Jaccard.
- Both axes empty (zero union) → axis returns 0.0, not NaN.

### 3.2 New: `internal/db/queries/contextual_likes.sql`

```sql
-- name: ListActiveContextualLikesForUser :many
-- Returns all the user's active (non-soft-deleted) contextual_likes with
-- non-null vectors. Cardinality bounded by the user's actual like-while-
-- playing history — typically tens to low hundreds.
SELECT track_id, session_vector
FROM contextual_likes
WHERE user_id = $1 AND deleted_at IS NULL AND session_vector IS NOT NULL;
```

### 3.3 New: `internal/db/queries/events.sql` (or `recommendation.sql`)

```sql
-- name: GetCurrentSessionVectorForUser :one
-- Returns the session_vector_at_play of the user's most recent play_event
-- in a still-active (un-timed-out) session. NoRows means no current vector
-- — caller treats this as Seed=true sentinel.
SELECT pe.session_vector_at_play
FROM play_events pe
JOIN play_sessions s ON s.id = pe.session_id
WHERE pe.user_id = $1
  AND s.ended_at IS NULL
ORDER BY pe.started_at DESC
LIMIT 1;
```

(Final placement decided at implementation time — wherever sqlc and existing
query files line up best.)

### 3.4 Modified: `internal/recommendation/score.go`

```go
type ScoringInputs struct {
    IsGeneralLiked       bool
    LastPlayedAt         *time.Time
    PlayCount            int
    SkipCount            int
    ContextualMatchScore float64 // NEW — in [0, 1], 0 when no signal
}

type ScoringWeights struct {
    BaseWeight      float64
    LikeBoost       float64
    RecencyWeight   float64
    SkipPenalty     float64
    JitterMagnitude float64
    ContextWeight   float64 // NEW
}

// Score gains: + in.ContextualMatchScore * w.ContextWeight
```

Zero-value defaults: a `ScoringInputs{}` with zero `ContextualMatchScore` and a
`ScoringWeights{}` with zero `ContextWeight` produce the v1 score. Existing callers
not passing the new fields see no behavior change.

### 3.5 Modified: `internal/recommendation/candidates.go`

```go
func LoadCandidates(
    ctx context.Context,
    q *dbq.Queries,
    userID, seedID pgtype.UUID,
    recentlyPlayedHours int,
    currentVector SessionVector, // NEW
) ([]Candidate, error)
```

Body adds:

1. Existing `LoadRadioCandidates` call.
2. New `ListActiveContextualLikesForUser(userID)` call.
3. Group result by `track_id` into `map[pgtype.UUID][]SessionVector`, unmarshaling
   each `jsonb` column into `SessionVector`. Unmarshal failures are logged and
   skipped (don't poison the entire response over one bad row).
4. For each candidate, set `Inputs.ContextualMatchScore =
   ContextualMatchScore(currentVector, group[trackID], DefaultSimilarityWeights)`.

### 3.6 Modified: `internal/api/radio.go`

Before calling `LoadCandidates`, fetch the current session vector:

```go
var currentVec recommendation.SessionVector
if raw, err := q.GetCurrentSessionVectorForUser(ctx, user.ID); err == nil && len(raw) > 0 {
    if jerr := json.Unmarshal(raw, &currentVec); jerr != nil {
        h.logger.Warn("api: radio: bad session_vector_at_play", "err", jerr)
        currentVec = recommendation.SessionVector{Seed: true}
    }
} else {
    currentVec = recommendation.SessionVector{Seed: true}
}
```

Pass `currentVec` into `LoadCandidates`. Pass `recCfg.ContextWeight` through into the
`ScoringWeights` struct alongside the existing weights.

### 3.7 Modified: `internal/config/config.go`

```go
type RecommendationConfig struct {
    BaseWeight          float64 `yaml:"base_weight"`
    LikeBoost           float64 `yaml:"like_boost"`
    RecencyWeight       float64 `yaml:"recency_weight"`
    SkipPenalty         float64 `yaml:"skip_penalty"`
    JitterMagnitude     float64 `yaml:"jitter_magnitude"`
    ContextWeight       float64 `yaml:"context_weight"`         // NEW, default 2.0
    RecentlyPlayedHours int     `yaml:"recently_played_hours"`
    RadioSize           int     `yaml:"radio_size"`
    RadioSizeMax        int     `yaml:"radio_size_max"`
}
```

`Default()` populates `ContextWeight: 2.0`.

## 4. Request flow

```
GET /api/radio?seed_track=<uuid>&limit=N
  ↓
handleRadio
  1. Auth, parse seed_track, parse limit         (unchanged)
  2. q.GetCurrentSessionVectorForUser(userID)    (NEW)
        NoRows / NULL / unmarshal fail → SessionVector{Seed: true}
  3. recommendation.LoadCandidates(...)          (extended)
        a. q.LoadRadioCandidates                 (unchanged)
        b. q.ListActiveContextualLikesForUser    (NEW)
        c. group by track_id → map[uuid][]SessionVector
        d. per candidate: ContextualMatchScore() → ScoringInputs
  4. recommendation.Shuffle(candidates, weights, now, rng, limit-1)
        Score() now folds in ContextualMatchScore * ContextWeight
  5. Resolve album/artist, build response        (unchanged)
```

## 5. Cold-start handling

Every cold-start path collapses to `contextual_match_score = 0` for all candidates,
so scoring degrades cleanly to v1 behavior:

| Condition                                            | Path                                                 |
|------------------------------------------------------|------------------------------------------------------|
| User has no `play_events` at all                     | `NoRows` → `Seed=true` sentinel → `ContextualMatchScore` returns 0 |
| User has plays but no active session                 | `NoRows` (joined `s.ended_at IS NULL` filters)       |
| Active session but `session_vector_at_play` is NULL  | `len(raw) == 0` → `Seed=true` sentinel               |
| Vector populated but `Seed=true`                     | `ContextualMatchScore` short-circuits to 0           |
| Candidate has no `contextual_likes`                  | absent from map → empty slice → returns 0            |
| Candidate has only `Seed=true` likes                 | filtered out → empty → returns 0                     |
| Candidate has only soft-deleted likes                | excluded by `deleted_at IS NULL` in the SQL          |

## 6. Test plan

### 6.1 `similarity_test.go` (pure, table-driven)

- Identical vectors → `1.0`.
- Fully disjoint axes → `0.0`.
- Mixed: shared tags, no shared artists → `0.7 * tagJaccard`.
- Mixed: no shared tags, shared artists → `0.3 * artistJaccard`.
- Either input `Seed=true` → `0.0`.
- Both vectors fully empty → `0.0` (not NaN).
- One side empty on an axis, other side populated → that axis contributes 0.
- Weight balance: shared all tags, default weights → exactly `0.7`.

### 6.2 `score_test.go` extensions

- Perfect contextual match (`ContextualMatchScore=1.0`) at `ContextWeight=2.0` adds
  exactly `+2.0` to the base score.
- Half match (`0.5`) adds `+1.0`.
- Zero match (`0.0`) leaves score unchanged from v1 behavior — guards backward compat.

### 6.3 `candidates_test.go` (integration vs test DB)

- Candidate with one matching `contextual_like` → `ContextualMatchScore > 0`.
- Candidate with multiple `contextual_likes` → max similarity wins.
- Candidate whose only `contextual_likes` are `Seed=true` → score 0.
- Candidate whose only `contextual_likes` are soft-deleted → score 0 (SQL filter).
- User with no `contextual_likes` anywhere → every candidate scores 0.
- User with only soft-deleted `contextual_likes` → every candidate scores 0.

### 6.4 `radio_test.go` (integration, end-to-end)

- Seed a current session vibe (3+ tracks of one genre/artist set) by inserting
  `play_events` with populated `session_vector_at_play`.
- Insert a `contextual_like` whose `session_vector` matches that vibe, on track T.
- Insert an unrelated control track C with no contextual signal.
- Call `/api/radio` with a deterministic RNG and seed track distinct from T and C.
- Assert: T ranks above C in the response.

### 6.5 Coverage gate

Combined `internal/recommendation` coverage stays ≥ 70% (currently 78.5% combined
with `internal/playevents` post-#341; this slice's pure functions are highly
testable so we expect to land closer to 85%+).

## 7. Backwards compatibility

- `/api/radio` request and response shapes are unchanged — same query params, same
  JSON output. Web client requires no edits.
- `ScoringInputs.ContextualMatchScore` and `ScoringWeights.ContextWeight` default to
  `0` in zero-value structs. Pre-existing tests that construct these directly continue
  passing without modification because the new term contributes nothing when both are
  zero.
- `LoadCandidates` gains a `currentVector SessionVector` parameter — this is a
  signature change, but the only caller is `internal/api/radio.go`, which we update
  in this slice. No external consumers.
- DB schema is unchanged (migrations 0007 already shipped the table + indexes
  needed for this slice).

## 8. Out-of-scope (deferred)

- Generalized (bag-of-counts) Jaccard if telemetry shows tag-dominance discrimination
  matters.
- YAML exposure of `SimilarityWeights`.
- Per-user override of any recommendation weight.
- Caching `currentVector` across requests within a session.
- ListenBrainz / similar-artist retrieval (M4).
- `/api/radio?explain=true` style debug endpoint.
- Tag enrichment beyond `tracks.genre`.

## 9. Milestone gate (closes M3)

After this slice merges:

- Recommendation engine has all three v1 components: weighted shuffle, session
  vectors written, contextual matching folded into scoring.
- Manual end-to-end verification: like a track during a session of one vibe, build a
  similar session later, observe the track surfaces above unrelated controls in
  `/api/radio` output.
- M4 (radio refinements + scrobble polish) unblocked.