docs(spec): add M3 session vectors + contextual likes design
Sub-plan #2 of 3 in M3 (Fable #341). Two write-path additions on top of M2's events + likes infra: 1. play_started: compute session vector from prior 5 tracks in current play_session, write to play_events.session_vector_at_play. 2. like a track: if open play_event has populated vector, snapshot into contextual_likes with vector + session_id. Soft-delete on unlike (deleted_at column gates engine queries). Includes new migration 0007_contextual_likes — the table was missing from migration 0005 despite being mentioned in its comments. Adds GIN index for sub-plan #3's similarity queries. Backend-only slice; no UI changes.
This commit is contained in:
@@ -0,0 +1,251 @@
|
||||
# M3 Session Vectors + Contextual Likes — Design Spec
|
||||
|
||||
**Status:** approved 2026-04-27
|
||||
**Slice:** M3 sub-plan #2 of 3 (recommendation engine v1 + contextual likes). Spec §6 "Session vector" + §13 step 8.
|
||||
**Fable task:** #341.
|
||||
|
||||
## Goal
|
||||
|
||||
Add the two write paths that produce the data the recommendation engine consumes for contextual matching:
|
||||
|
||||
1. **At every play_started**, compute a session vector from the prior tracks in the user's current play_session and persist it to `play_events.session_vector_at_play` (column already exists nullable since migration 0005).
|
||||
2. **At every like**, if the user has an open play_event with a populated session_vector, snapshot it into a new `contextual_likes` row.
|
||||
|
||||
This slice is backend-only — no UI surface. It accumulates the data; sub-plan #3 (Fable #342) reads it to add the `contextual_match_score` term to the scoring formula.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Adding `contextual_match_score` to the scoring function — that's sub-plan #3.
|
||||
- Album/artist contextual likes. `contextual_likes` is track-only per spec §5.
|
||||
- MBID-derived tags. v1 uses `tracks.genre` (denormalized text from migration 0002) as the only tag source. MBID tags are a post-v1 enrichment.
|
||||
- Audio-feature tags (BPM, energy, key). Require an analysis pipeline we don't have. Post-v1.
|
||||
- Cross-session vector enrichment. Each session's vector only sees prior tracks within the same play_session row.
|
||||
- A purge/expiry policy on `contextual_likes`. Rows accumulate indefinitely. v1 problem only when a user has tens of thousands of likes; not a concern at v1 scale.
|
||||
|
||||
## Important catch from exploration
|
||||
|
||||
The `contextual_likes` table does NOT exist yet. The M2 events migration (0005) commented that "contextual_likes ships nullable now," but the actual `CREATE TABLE` was missed. This slice ships the table in a new migration (`0007`).
|
||||
|
||||
## Architecture
|
||||
|
||||
A new pure function `BuildSessionVector(priorTracks) → SessionVector` in `internal/recommendation/sessionvector.go`. The vector struct serializes to JSONB with the spec §6 shape: `seed` flag, `artists` set, `tags` bag-of-counts, `recent_track_ids` ordered list.
|
||||
|
||||
`internal/playevents.Writer.RecordPlayStarted` extends inside its existing transaction:
|
||||
1. Auto-close prior open row (existing).
|
||||
2. FindOrCreate session (existing).
|
||||
3. Insert play_event (existing).
|
||||
4. **NEW:** Query the prior tracks in the session via a new sqlc query `ListRecentSessionTracks(sessionID, beforeTime, limit)`.
|
||||
5. **NEW:** Build the vector via the pure function.
|
||||
6. **NEW:** UPDATE the just-inserted play_event's `session_vector_at_play` column (new sqlc query `UpdatePlayEventVector`).
|
||||
|
||||
`internal/api/likes.handleLikeTrack` extends:
|
||||
1. `LikeTrack` upsert (existing) — but the sqlc query becomes `:execrows` so the handler can detect "actually inserted" vs "already exists."
|
||||
2. **NEW:** if rows == 1, look up the user's open play_event. If present and its `session_vector_at_play` is non-NULL, INSERT into `contextual_likes` with the vector + session_id snapshot.
|
||||
|
||||
`internal/api/likes.handleUnlikeTrack` extends:
|
||||
1. `UnlikeTrack` (existing).
|
||||
2. **NEW:** UPDATE all the user's `contextual_likes` rows for this track to set `deleted_at = now() WHERE deleted_at IS NULL`. Idempotent.
|
||||
|
||||
Subsonic `/rest/star` and `/rest/unstar` already call into `dbq.LikeTrack` / `dbq.UnlikeTrack` through `internal/subsonic/star.go`. After this slice, those calls naturally pick up the contextual capture / soft-delete behavior — no code changes in `internal/subsonic` beyond a verification test.
|
||||
|
||||
## Schema (migration `0007_contextual_likes.up.sql`)
|
||||
|
||||
```sql
|
||||
-- contextual_likes captures the per-session-context snapshot at the time
|
||||
-- of each like. The recommendation engine in M3 sub-plan #3 uses these
|
||||
-- to compute contextual_match_score per (current session, candidate track).
|
||||
--
|
||||
-- Multiple rows per (user, track) are allowed and expected — each row
|
||||
-- represents a like in a specific session context. Soft-deleted rows
|
||||
-- remain in the table but are filtered out of the engine's queries via
|
||||
-- the partial index. Re-liking a previously-unliked track adds a NEW row
|
||||
-- (does not undelete the old one).
|
||||
|
||||
CREATE TABLE contextual_likes (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
track_id uuid NOT NULL REFERENCES tracks(id) ON DELETE CASCADE,
|
||||
liked_at timestamptz NOT NULL DEFAULT now(),
|
||||
deleted_at timestamptz,
|
||||
session_vector jsonb,
|
||||
session_id uuid REFERENCES play_sessions(id) ON DELETE SET NULL
|
||||
);
|
||||
|
||||
-- Partial index: the engine queries only active rows; this is the hot path.
|
||||
CREATE INDEX contextual_likes_active_idx
|
||||
ON contextual_likes (user_id, track_id)
|
||||
WHERE deleted_at IS NULL;
|
||||
|
||||
-- GIN index for vector similarity queries (used by sub-plan #3's
|
||||
-- contextual_match_score lookup). Even though M3 sub-plan #3 doesn't yet
|
||||
-- exist, this slice adds the index because the recommendation engine
|
||||
-- will need it; better to migrate once.
|
||||
CREATE INDEX contextual_likes_vector_idx
|
||||
ON contextual_likes USING gin (session_vector);
|
||||
|
||||
-- Lookup support for the soft-delete update (UPDATE ... WHERE user_id = $1
|
||||
-- AND track_id = $2 AND deleted_at IS NULL). Already covered by the
|
||||
-- partial index above — no separate index needed.
|
||||
```
|
||||
|
||||
`down.sql`:
|
||||
|
||||
```sql
|
||||
DROP TABLE IF EXISTS contextual_likes;
|
||||
```
|
||||
|
||||
## Session vector shape
|
||||
|
||||
```go
|
||||
type SessionVector struct {
|
||||
Seed bool `json:"seed"`
|
||||
Artists []string `json:"artists"` // distinct, ordered by first appearance
|
||||
Tags map[string]int `json:"tags"` // bag-of-counts
|
||||
RecentTrackIDs []string `json:"recent_track_ids"` // newest last
|
||||
}
|
||||
|
||||
func BuildSessionVector(priorTracks []dbq.Track) SessionVector
|
||||
```
|
||||
|
||||
Rules:
|
||||
- `Seed = len(priorTracks) < 3`. The engine in #342 filters seed vectors out of the contextual-match query (they don't represent enough context to match against).
|
||||
- `Artists` is the deduplicated list of `priorTracks[i].ArtistID` values, formatted as UUID strings. Order is "first appearance" (older tracks first).
|
||||
- `Tags` is a `map[string]int` counting how many tracks have each genre. `priorTracks[i].Genre == nil || == ""` means no contribution. Tags are case-sensitive (we don't normalize — `"Rock"` and `"rock"` are different bins; future cleanup can add normalization).
|
||||
- `RecentTrackIDs` is the ordered list, newest last (matches the spec's "ordered list of the N track ids").
|
||||
- The function does NOT truncate. The caller passes in at most N tracks; the function works with whatever it's given (extra coverage flexibility).
|
||||
|
||||
## API contracts
|
||||
|
||||
No HTTP shape changes. `POST /api/events`, `POST/DELETE /api/likes/...`, and Subsonic `/rest/star`, `/rest/unstar` all keep their current request/response contracts. The new behavior is internal data-write side effects.
|
||||
|
||||
## Components & files
|
||||
|
||||
### New server files
|
||||
|
||||
| Path | Responsibility |
|
||||
|---|---|
|
||||
| `internal/db/migrations/0007_contextual_likes.up.sql` + `.down.sql` | Schema for `contextual_likes` + partial index + GIN index. |
|
||||
| `internal/db/queries/contextual_likes.sql` | `InsertContextualLike`, `SoftDeleteContextualLikesForUserTrack`. |
|
||||
| `internal/db/dbq/contextual_likes.sql.go` | Generated bindings. |
|
||||
| `internal/recommendation/sessionvector.go` | `SessionVector` struct + `BuildSessionVector(priorTracks)` pure function. |
|
||||
| `internal/recommendation/sessionvector_test.go` | Unit tests for the pure function (boundary cases for the seed flag, dedup, tag counts, ordering, JSON round-trip). |
|
||||
|
||||
### Modified server files
|
||||
|
||||
| Path | Change |
|
||||
|---|---|
|
||||
| `internal/db/queries/events.sql` | Add `ListRecentSessionTracks(sessionID, beforeTime, limit) :many` (joins play_events ↔ tracks, orders by started_at DESC). Add `UpdatePlayEventVector(id, vector) :exec`. |
|
||||
| `internal/db/queries/likes.sql` | Change `LikeTrack` from `:exec` to `:execrows` so the handler detects insert vs already-exists. |
|
||||
| `internal/playevents/writer.go::RecordPlayStarted` | After InsertPlayEvent: ListRecentSessionTracks → BuildSessionVector → UpdatePlayEventVector. All inside the existing transaction. |
|
||||
| `internal/playevents/writer.go::RecordSyntheticCompletedPlay` | Same vector computation as RecordPlayStarted (Subsonic synthetic plays should also carry context). Single source of truth via a private helper. |
|
||||
| `internal/playevents/writer_test.go` | Three new tests covering vector persistence (seed flag, populated vector, session-scope isolation). |
|
||||
| `internal/api/likes.go::handleLikeTrack` | Capture `:execrows` result; if 1, GetOpenPlayEventForUser; if open + non-null vector, InsertContextualLike. |
|
||||
| `internal/api/likes.go::handleUnlikeTrack` | After UnlikeTrack, SoftDeleteContextualLikesForUserTrack. |
|
||||
| `internal/api/likes_test.go` | Five new tests (capture during open play, no-op without play, no duplicate on idempotent like, soft-delete on unlike, history accumulation across like/unlike/like). |
|
||||
| `internal/subsonic/star_test.go` | One new test confirming Subsonic star captures contextual_likes when a now-playing event is open. |
|
||||
|
||||
### No web changes
|
||||
|
||||
The web client already calls `/api/events` and `/api/likes/...`. After this slice, behavior is identical from the SPA's perspective; the side-effects are server-side only.
|
||||
|
||||
## Data flow
|
||||
|
||||
**Play started:**
|
||||
|
||||
1. SPA calls `POST /api/events` with `type: 'play_started', track_id`.
|
||||
2. `handleEventPlayStarted` → `events.Writer.RecordPlayStarted(ctx, userID, trackID, clientID, at)`.
|
||||
3. Inside the transaction:
|
||||
- Auto-close any prior open row (existing).
|
||||
- `playsessions.FindOrCreate(...)` → `sessionID` (existing).
|
||||
- `q.InsertPlayEvent(...)` → `playEventID` (existing).
|
||||
- **NEW:** `priorTracks := q.ListRecentSessionTracks(ctx, sessionID, at, 5)`. Returns up to 5 tracks where `started_at < at` (excluding the just-inserted row).
|
||||
- **NEW:** `vec := recommendation.BuildSessionVector(priorTracks)`.
|
||||
- **NEW:** `vecJSON, _ := json.Marshal(vec)`.
|
||||
- **NEW:** `q.UpdatePlayEventVector(ctx, UpdatePlayEventVectorParams{ID: playEventID, SessionVectorAtPlay: vecJSON})`.
|
||||
4. Returns `StartedResult{PlayEventID, SessionID}`.
|
||||
|
||||
**Subsonic synthetic completed play:** `RecordSyntheticCompletedPlay` follows the same vector-computation path so Subsonic-driven plays carry context too.
|
||||
|
||||
**Like a track:**
|
||||
|
||||
1. `POST /api/likes/tracks/:id` (or Subsonic `/rest/star?id=X`).
|
||||
2. `handleLikeTrack` validates auth + UUID + entity exists.
|
||||
3. `q.LikeTrack(...)` returns rows-affected count (sqlc `:execrows`).
|
||||
4. **If rows == 0** (already liked): return 204. No contextual capture.
|
||||
5. **If rows == 1** (freshly inserted):
|
||||
- `event, err := q.GetOpenPlayEventForUser(ctx, userID)`.
|
||||
- If `errors.Is(err, pgx.ErrNoRows)` → return 204 (no open event, no contextual capture).
|
||||
- If `event.SessionVectorAtPlay` is NULL → return 204 (event opened before this slice landed; no vector).
|
||||
- Else: `q.InsertContextualLike(ctx, InsertContextualLikeParams{UserID, TrackID, SessionVector: event.SessionVectorAtPlay, SessionID: &event.SessionID})`.
|
||||
6. Return 204.
|
||||
|
||||
**Unlike a track:**
|
||||
|
||||
1. `DELETE /api/likes/tracks/:id` (or Subsonic `/rest/unstar?id=X`).
|
||||
2. `handleUnlikeTrack` validates UUID.
|
||||
3. `q.UnlikeTrack(...)` (existing).
|
||||
4. **NEW:** `q.SoftDeleteContextualLikesForUserTrack(ctx, userID, trackID)` — `UPDATE contextual_likes SET deleted_at = now() WHERE user_id = $1 AND track_id = $2 AND deleted_at IS NULL`.
|
||||
5. Return 204.
|
||||
|
||||
**Edge cases:**
|
||||
|
||||
- **Like with no open play_event.** No contextual capture; only `general_likes`. Same outcome as today.
|
||||
- **Like during cold-start session.** Vector has `seed: true`. Row is still inserted (the seed flag is informational; the engine in #342 filters on it).
|
||||
- **Re-like of an already-liked track in the same session.** `LikeTrack` returns 0 affected rows; contextual capture skipped. No spam.
|
||||
- **Re-like after unlike, in the same session.** Unlike soft-deleted prior rows. New like inserts a fresh row. The (user, track) pair has both rows in the table; only the fresh one is `deleted_at IS NULL`.
|
||||
- **Unlike of a never-liked track.** `UnlikeTrack` is a no-op. `SoftDeleteContextualLikesForUserTrack` runs but updates 0 rows. Both safe.
|
||||
- **Album / artist likes.** `contextual_likes` is track-only. Album/artist like handlers are untouched in this slice.
|
||||
- **Subsonic `star?albumId=&artistId=`.** Album/artist branches don't trigger contextual capture (track-only). The track-id branch picks it up via the shared `dbq.LikeTrack` call.
|
||||
|
||||
## Testing
|
||||
|
||||
### Server (`go test`)
|
||||
|
||||
**`internal/recommendation/sessionvector_test.go`** (pure unit tests):
|
||||
- Empty input → seed=true, all collections empty.
|
||||
- 1 track → seed=true, artists has 1 entry, tags has 1 entry (count=1), recent_track_ids has 1 entry.
|
||||
- 2 tracks → seed=true.
|
||||
- 3 tracks → seed=false. Single-artist case: artists deduplicated to 1 entry. Multi-genre case: tags accumulates counts.
|
||||
- 5 tracks across 3 artists, mixed genres → all populated, dedup correct, counts correct.
|
||||
- Track with empty/nil genre → tag entry not added.
|
||||
- 10 tracks input → function processes all 10 (caller controls truncation; document the contract).
|
||||
- JSON round-trip via `encoding/json`: marshal, unmarshal, fields equal.
|
||||
|
||||
**`internal/playevents/writer_test.go`** (live-DB integration, three new):
|
||||
- `TestRecordPlayStarted_PersistsSessionVector_Seed`: first play in fresh session → row's `session_vector_at_play.seed == true`, empty arrays.
|
||||
- `TestRecordPlayStarted_PersistsSessionVector_Populated`: after 4 plays, the 5th's vector has `seed: false` and the 4 prior tracks' artists/tags/ids.
|
||||
- `TestRecordPlayStarted_VectorScopedToSession`: open a play_event, advance > 30 minutes, start another (new session) — second play's vector is `seed: true` with nothing from the prior session.
|
||||
|
||||
**`internal/api/likes_test.go`** (live-DB integration, five new):
|
||||
- `TestLikeTrack_DuringOpenPlayEvent_WritesContextualLike`: user starts playing track A (creates play_event with non-null vector); likes track B; assert `contextual_likes` has one row for (user, B) with the matching `session_vector` and `session_id`.
|
||||
- `TestLikeTrack_NoOpenPlayEvent_NoContextualLike`: user likes a track without playing — `general_likes` has the row, `contextual_likes` is empty.
|
||||
- `TestLikeTrack_RepeatedLike_NoDuplicate`: like the same track twice — first call writes both rows; second call no-ops (LikeTrack returns 0 rows; contextual capture skipped).
|
||||
- `TestUnlikeTrack_SoftDeletesContextualLikes`: like → unlike. `general_likes` row gone, `contextual_likes` row's `deleted_at` is set, row count unchanged.
|
||||
- `TestLikeUnlikeRelike_HistoryAccumulates`: like → unlike → like. Two `contextual_likes` rows for that (user, track): one with `deleted_at` set, one without.
|
||||
|
||||
**`internal/subsonic/star_test.go`** (live-DB, one new):
|
||||
- `TestHandleStar_DuringNowPlaying_WritesContextualLike`: send `submission=false` for track A (creates open play_event with vector), then `star?id=B` — `contextual_likes` row exists for (user, B).
|
||||
|
||||
**Migration smoke**: existing `internal/db` test runs the new 0007 migration; verify no errors.
|
||||
|
||||
### End-to-end manual
|
||||
|
||||
Final task in the implementation plan:
|
||||
|
||||
1. Sign in. Play 4 tracks all the way through.
|
||||
2. `psql ... SELECT id, session_vector_at_play FROM play_events ORDER BY started_at DESC LIMIT 5` — vectors are populated; first 3 have `seed: true`, 4th onward `seed: false`.
|
||||
3. While the 4th track is playing, like it via the heart button. `SELECT * FROM contextual_likes` shows a row with that track's session_vector.
|
||||
4. Like a different track NOT currently playing. `general_likes` row appears; no `contextual_likes` row.
|
||||
5. Unlike the first track. `general_likes` deleted; `contextual_likes` row still exists but `deleted_at` is set.
|
||||
6. Re-like that track in a new session. New `contextual_likes` row with `deleted_at IS NULL` and a different `session_vector` (different artists/tags from the new session).
|
||||
7. From Feishin: send a now-playing for track A, then star track B. Confirm contextual capture worked through the Subsonic path.
|
||||
|
||||
## Risks & mitigations
|
||||
|
||||
- **Vector size bloat**: each play_event row carries up to 5 artist UUIDs + a tag map + 5 track UUIDs. ~500 bytes per play_event in JSON. For a user with 50,000 plays, that's 25 MB of JSONB. Acceptable; PostgreSQL handles it. If telemetry shows the column dominates row width, future work can extract to a separate table or hash older vectors.
|
||||
- **Tag normalization**: `tracks.genre` is denormalized free-text. `"Rock"` and `"rock"` end up as different bins. v1 doesn't normalize. Mitigation: documented; M3.5/M4 cleanup can `lower()` consistently; the recommendation engine in #342 can do its similarity calculation on lowercased tags as a workaround.
|
||||
- **Eager column read**: extending `RecordPlayStarted` adds 1 SELECT (recent tracks) + 1 UPDATE (vector) per play. Single-digit ms overhead at v1 scale. Reasonable cost for the data we get.
|
||||
- **NULL vector after this slice ships**: existing play_events from before the migration have NULL `session_vector_at_play`. The like-handler's contextual-capture path skips when the vector is NULL — those old plays simply don't generate contextual likes. No backfill; new plays going forward populate the vector.
|
||||
- **Subsonic synthetic plays carry vector**: synthetic plays from `submission=true` use the same vector logic. Subsonic clients that scrobble in bulk after-the-fact (e.g. submit 10 plays in 5 seconds) generate vectors quickly across each new session/track — vectors may show artificial 5-track sliding windows during the catch-up. Acceptable for v1; treats Subsonic clients as first-class citizens of the recommendation pipeline.
|
||||
- **`contextual_likes` table forgotten in 0005**: this slice adds it. Existing migration history is unchanged; the new 0007 migration brings the schema up to spec.
|
||||
- **Soft-delete query without explicit unique constraint**: `SoftDeleteContextualLikesForUserTrack` updates ALL active rows for (user, track), which may be more than one if a user liked the same track in multiple sessions. That's the correct behavior — unliking means "all my historical likes for this track no longer count." Documented.
|
||||
Reference in New Issue
Block a user