minstrel/docs/superpowers/specs/2026-04-27-m3-similarity-design.md

M3 sub-plan #3 — Session similarity + contextual_match_score in scoring

Status: Spec draft, 2026-04-27 Tracking: Fable #342 Closes: M3 milestone (recommendation engine v1) Builds on:

• #340 — weighted shuffle baseline (internal/recommendation package, Score/Shuffle, LoadRadioCandidates)
• #341 — session vectors at play_started + contextual_likes capture on like

1. Goal

Add the contextual_match_score term to the recommendation scoring formula. For each candidate track, compute the maximum weighted-Jaccard similarity between the user's current session vector and the session vectors stored on that track's active contextual_likes rows. Fold that scalar into Score() as + contextual_match_score * ContextWeight.

When this slice ships, /api/radio produces context-aware recommendations: tracks the user has previously liked while in similar musical contexts get an additive boost on top of the v1 weighted-shuffle baseline.

2. Non-goals

• No new UI surface — /api/radio response shape is unchanged.
• No tag enrichment beyond tracks.genre (MBID tags / BPM remain post-v1).
• No similarity-axis weight exposure in YAML (hardcoded 0.7 / 0.3 for v1).
• No caching of the current session vector across requests.
• No "why this track?" debug endpoint.
• No ListenBrainz / external-similarity retrieval (M4).
• No GIN index on play_events.session_vector_at_play — we read the user's most recent play by id, not by similarity. Existing (user_id, started_at) access pattern is sufficient.

3. Architecture overview

Three additions to internal/recommendation, two adjustments to existing files, one new sqlc query.

3.1 New: internal/recommendation/similarity.go (pure)

type SimilarityWeights struct {
    TagsWeight    float64
    ArtistsWeight float64
}

// DefaultSimilarityWeights is the v1 axis balance per the M3 design.
// Hardcoded; not exposed via YAML — operators can't tune this for v1.
var DefaultSimilarityWeights = SimilarityWeights{
    TagsWeight:    0.7,
    ArtistsWeight: 0.3,
}

// Similarity returns weighted-Jaccard similarity in [0, 1]. Returns 0 if
// either input is Seed=true (low-confidence vectors don't contribute).
func Similarity(a, b SessionVector, w SimilarityWeights) float64

// ContextualMatchScore is the per-candidate scalar fed into ScoringInputs.
// Returns 0 if current.Seed is true OR likes (after filtering Seed=true
// entries) is empty. Otherwise: max(Similarity(current, l) for l in likes).
func ContextualMatchScore(current SessionVector, likes []SessionVector, w SimilarityWeights) float64

Per-axis semantics (classic set Jaccard):

• Tags axis flattens map[string]int keysets and computes |A ∩ B| / |A ∪ B|. Bag-of-counts data is preserved on disk; we discard counts at similarity time. Generalized Jaccard remains a one-line upgrade path if telemetry justifies it.
• Artists axis is already a []string (deduplicated artist UUIDs); same Jaccard.
• Both axes empty (zero union) → axis returns 0.0, not NaN.

3.2 New: internal/db/queries/contextual_likes.sql

-- name: ListActiveContextualLikesForUser :many
-- Returns all the user's active (non-soft-deleted) contextual_likes with
-- non-null vectors. Cardinality bounded by the user's actual like-while-
-- playing history — typically tens to low hundreds.
SELECT track_id, session_vector
FROM contextual_likes
WHERE user_id = $1 AND deleted_at IS NULL AND session_vector IS NOT NULL;

3.3 New: internal/db/queries/events.sql (or recommendation.sql)

-- name: GetCurrentSessionVectorForUser :one
-- Returns the session_vector_at_play of the user's most recent play_event
-- in a still-active (un-timed-out) session. NoRows means no current vector
-- — caller treats this as Seed=true sentinel.
SELECT pe.session_vector_at_play
FROM play_events pe
JOIN play_sessions s ON s.id = pe.session_id
WHERE pe.user_id = $1
  AND s.ended_at IS NULL
ORDER BY pe.started_at DESC
LIMIT 1;

(Final placement decided at implementation time — wherever sqlc and existing query files line up best.)

3.4 Modified: internal/recommendation/score.go

type ScoringInputs struct {
    IsGeneralLiked       bool
    LastPlayedAt         *time.Time
    PlayCount            int
    SkipCount            int
    ContextualMatchScore float64 // NEW — in [0, 1], 0 when no signal
}

type ScoringWeights struct {
    BaseWeight      float64
    LikeBoost       float64
    RecencyWeight   float64
    SkipPenalty     float64
    JitterMagnitude float64
    ContextWeight   float64 // NEW
}

// Score gains: + in.ContextualMatchScore * w.ContextWeight

Zero-value defaults: a ScoringInputs{} with zero ContextualMatchScore and a ScoringWeights{} with zero ContextWeight produce the v1 score. Existing callers not passing the new fields see no behavior change.

3.5 Modified: internal/recommendation/candidates.go

func LoadCandidates(
    ctx context.Context,
    q *dbq.Queries,
    userID, seedID pgtype.UUID,
    recentlyPlayedHours int,
    currentVector SessionVector, // NEW
) ([]Candidate, error)

Body adds:

1. Existing LoadRadioCandidates call.
2. New ListActiveContextualLikesForUser(userID) call.
3. Group result by track_id into map[pgtype.UUID][]SessionVector, unmarshaling each jsonb column into SessionVector. Unmarshal failures are logged and skipped (don't poison the entire response over one bad row).
4. For each candidate, set Inputs.ContextualMatchScore = ContextualMatchScore(currentVector, group[trackID], DefaultSimilarityWeights).

3.6 Modified: internal/api/radio.go

Before calling LoadCandidates, fetch the current session vector:

var currentVec recommendation.SessionVector
if raw, err := q.GetCurrentSessionVectorForUser(ctx, user.ID); err == nil && len(raw) > 0 {
    if jerr := json.Unmarshal(raw, &currentVec); jerr != nil {
        h.logger.Warn("api: radio: bad session_vector_at_play", "err", jerr)
        currentVec = recommendation.SessionVector{Seed: true}
    }
} else {
    currentVec = recommendation.SessionVector{Seed: true}
}

Pass currentVec into LoadCandidates. Pass recCfg.ContextWeight through into the ScoringWeights struct alongside the existing weights.

3.7 Modified: internal/config/config.go

type RecommendationConfig struct {
    BaseWeight          float64 `yaml:"base_weight"`
    LikeBoost           float64 `yaml:"like_boost"`
    RecencyWeight       float64 `yaml:"recency_weight"`
    SkipPenalty         float64 `yaml:"skip_penalty"`
    JitterMagnitude     float64 `yaml:"jitter_magnitude"`
    ContextWeight       float64 `yaml:"context_weight"`         // NEW, default 2.0
    RecentlyPlayedHours int     `yaml:"recently_played_hours"`
    RadioSize           int     `yaml:"radio_size"`
    RadioSizeMax        int     `yaml:"radio_size_max"`
}

Default() populates ContextWeight: 2.0.

4. Request flow

GET /api/radio?seed_track=<uuid>&limit=N
  ↓
handleRadio
  1. Auth, parse seed_track, parse limit         (unchanged)
  2. q.GetCurrentSessionVectorForUser(userID)    (NEW)
        NoRows / NULL / unmarshal fail → SessionVector{Seed: true}
  3. recommendation.LoadCandidates(...)          (extended)
        a. q.LoadRadioCandidates                 (unchanged)
        b. q.ListActiveContextualLikesForUser    (NEW)
        c. group by track_id → map[uuid][]SessionVector
        d. per candidate: ContextualMatchScore() → ScoringInputs
  4. recommendation.Shuffle(candidates, weights, now, rng, limit-1)
        Score() now folds in ContextualMatchScore * ContextWeight
  5. Resolve album/artist, build response        (unchanged)

5. Cold-start handling

Every cold-start path collapses to contextual_match_score = 0 for all candidates, so scoring degrades cleanly to v1 behavior:

Condition	Path
User has no play_events at all	NoRows → Seed=true sentinel → ContextualMatchScore returns 0
User has plays but no active session	NoRows (joined s.ended_at IS NULL filters)
Active session but session_vector_at_play is NULL	len(raw) == 0 → Seed=true sentinel
Vector populated but Seed=true	ContextualMatchScore short-circuits to 0
Candidate has no contextual_likes	absent from map → empty slice → returns 0
Candidate has only Seed=true likes	filtered out → empty → returns 0
Candidate has only soft-deleted likes	excluded by deleted_at IS NULL in the SQL

6. Test plan

6.1 similarity_test.go (pure, table-driven)

• Identical vectors → 1.0.
• Fully disjoint axes → 0.0.
• Mixed: shared tags, no shared artists → 0.7 * tagJaccard.
• Mixed: no shared tags, shared artists → 0.3 * artistJaccard.
• Either input Seed=true → 0.0.
• Both vectors fully empty → 0.0 (not NaN).
• One side empty on an axis, other side populated → that axis contributes 0.
• Weight balance: shared all tags, default weights → exactly 0.7.

6.2 score_test.go extensions

• Perfect contextual match (ContextualMatchScore=1.0) at ContextWeight=2.0 adds exactly +2.0 to the base score.
• Half match (0.5) adds +1.0.
• Zero match (0.0) leaves score unchanged from v1 behavior — guards backward compat.

6.3 candidates_test.go (integration vs test DB)

• Candidate with one matching contextual_like → ContextualMatchScore > 0.
• Candidate with multiple contextual_likes → max similarity wins.
• Candidate whose only contextual_likes are Seed=true → score 0.
• Candidate whose only contextual_likes are soft-deleted → score 0 (SQL filter).
• User with no contextual_likes anywhere → every candidate scores 0.
• User with only soft-deleted contextual_likes → every candidate scores 0.

6.4 radio_test.go (integration, end-to-end)

• Seed a current session vibe (3+ tracks of one genre/artist set) by inserting play_events with populated session_vector_at_play.
• Insert a contextual_like whose session_vector matches that vibe, on track T.
• Insert an unrelated control track C with no contextual signal.
• Call /api/radio with a deterministic RNG and seed track distinct from T and C.
• Assert: T ranks above C in the response.

6.5 Coverage gate

Combined internal/recommendation coverage stays ≥ 70% (currently 78.5% combined with internal/playevents post-#341; this slice's pure functions are highly testable so we expect to land closer to 85%+).

7. Backwards compatibility

• /api/radio request and response shapes are unchanged — same query params, same JSON output. Web client requires no edits.
• ScoringInputs.ContextualMatchScore and ScoringWeights.ContextWeight default to 0 in zero-value structs. Pre-existing tests that construct these directly continue passing without modification because the new term contributes nothing when both are zero.
• LoadCandidates gains a currentVector SessionVector parameter — this is a signature change, but the only caller is internal/api/radio.go, which we update in this slice. No external consumers.
• DB schema is unchanged (migrations 0007 already shipped the table + indexes needed for this slice).

8. Out-of-scope (deferred)

• Generalized (bag-of-counts) Jaccard if telemetry shows tag-dominance discrimination matters.
• YAML exposure of SimilarityWeights.
• Per-user override of any recommendation weight.
• Caching currentVector across requests within a session.
• ListenBrainz / similar-artist retrieval (M4).
• /api/radio?explain=true style debug endpoint.
• Tag enrichment beyond tracks.genre.

9. Milestone gate (closes M3)

After this slice merges:

• Recommendation engine has all three v1 components: weighted shuffle, session vectors written, contextual matching folded into scoring.
• Manual end-to-end verification: like a track during a session of one vibe, build a similar session later, observe the track surfaces above unrelated controls in /api/radio output.
• M4 (radio refinements + scrobble polish) unblocked.