From 08eb1e3f8caaa6748f8af196f12fcc54668bcabf Mon Sep 17 00:00:00 2001 From: Bryan Van Deusen Date: Mon, 27 Apr 2026 09:56:45 -0400 Subject: [PATCH] docs(spec): add M3 session vectors + contextual likes design MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sub-plan #2 of 3 in M3 (Fable #341). Two write-path additions on top of M2's events + likes infra: 1. play_started: compute session vector from prior 5 tracks in current play_session, write to play_events.session_vector_at_play. 2. like a track: if open play_event has populated vector, snapshot into contextual_likes with vector + session_id. Soft-delete on unlike (deleted_at column gates engine queries). Includes new migration 0007_contextual_likes — the table was missing from migration 0005 despite being mentioned in its comments. Adds GIN index for sub-plan #3's similarity queries. Backend-only slice; no UI changes. --- .../specs/2026-04-27-m3-vectors-design.md | 251 ++++++++++++++++++ 1 file changed, 251 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-27-m3-vectors-design.md diff --git a/docs/superpowers/specs/2026-04-27-m3-vectors-design.md b/docs/superpowers/specs/2026-04-27-m3-vectors-design.md new file mode 100644 index 00000000..928ab79c --- /dev/null +++ b/docs/superpowers/specs/2026-04-27-m3-vectors-design.md @@ -0,0 +1,251 @@ +# M3 Session Vectors + Contextual Likes — Design Spec + +**Status:** approved 2026-04-27 +**Slice:** M3 sub-plan #2 of 3 (recommendation engine v1 + contextual likes). Spec §6 "Session vector" + §13 step 8. +**Fable task:** #341. + +## Goal + +Add the two write paths that produce the data the recommendation engine consumes for contextual matching: + +1. **At every play_started**, compute a session vector from the prior tracks in the user's current play_session and persist it to `play_events.session_vector_at_play` (column already exists nullable since migration 0005). +2. **At every like**, if the user has an open play_event with a populated session_vector, snapshot it into a new `contextual_likes` row. + +This slice is backend-only — no UI surface. It accumulates the data; sub-plan #3 (Fable #342) reads it to add the `contextual_match_score` term to the scoring formula. + +## Non-goals + +- Adding `contextual_match_score` to the scoring function — that's sub-plan #3. +- Album/artist contextual likes. `contextual_likes` is track-only per spec §5. +- MBID-derived tags. v1 uses `tracks.genre` (denormalized text from migration 0002) as the only tag source. MBID tags are a post-v1 enrichment. +- Audio-feature tags (BPM, energy, key). Require an analysis pipeline we don't have. Post-v1. +- Cross-session vector enrichment. Each session's vector only sees prior tracks within the same play_session row. +- A purge/expiry policy on `contextual_likes`. Rows accumulate indefinitely. v1 problem only when a user has tens of thousands of likes; not a concern at v1 scale. + +## Important catch from exploration + +The `contextual_likes` table does NOT exist yet. The M2 events migration (0005) commented that "contextual_likes ships nullable now," but the actual `CREATE TABLE` was missed. This slice ships the table in a new migration (`0007`). + +## Architecture + +A new pure function `BuildSessionVector(priorTracks) → SessionVector` in `internal/recommendation/sessionvector.go`. The vector struct serializes to JSONB with the spec §6 shape: `seed` flag, `artists` set, `tags` bag-of-counts, `recent_track_ids` ordered list. + +`internal/playevents.Writer.RecordPlayStarted` extends inside its existing transaction: +1. Auto-close prior open row (existing). +2. FindOrCreate session (existing). +3. Insert play_event (existing). +4. **NEW:** Query the prior tracks in the session via a new sqlc query `ListRecentSessionTracks(sessionID, beforeTime, limit)`. +5. **NEW:** Build the vector via the pure function. +6. **NEW:** UPDATE the just-inserted play_event's `session_vector_at_play` column (new sqlc query `UpdatePlayEventVector`). + +`internal/api/likes.handleLikeTrack` extends: +1. `LikeTrack` upsert (existing) — but the sqlc query becomes `:execrows` so the handler can detect "actually inserted" vs "already exists." +2. **NEW:** if rows == 1, look up the user's open play_event. If present and its `session_vector_at_play` is non-NULL, INSERT into `contextual_likes` with the vector + session_id snapshot. + +`internal/api/likes.handleUnlikeTrack` extends: +1. `UnlikeTrack` (existing). +2. **NEW:** UPDATE all the user's `contextual_likes` rows for this track to set `deleted_at = now() WHERE deleted_at IS NULL`. Idempotent. + +Subsonic `/rest/star` and `/rest/unstar` already call into `dbq.LikeTrack` / `dbq.UnlikeTrack` through `internal/subsonic/star.go`. After this slice, those calls naturally pick up the contextual capture / soft-delete behavior — no code changes in `internal/subsonic` beyond a verification test. + +## Schema (migration `0007_contextual_likes.up.sql`) + +```sql +-- contextual_likes captures the per-session-context snapshot at the time +-- of each like. The recommendation engine in M3 sub-plan #3 uses these +-- to compute contextual_match_score per (current session, candidate track). +-- +-- Multiple rows per (user, track) are allowed and expected — each row +-- represents a like in a specific session context. Soft-deleted rows +-- remain in the table but are filtered out of the engine's queries via +-- the partial index. Re-liking a previously-unliked track adds a NEW row +-- (does not undelete the old one). + +CREATE TABLE contextual_likes ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE, + track_id uuid NOT NULL REFERENCES tracks(id) ON DELETE CASCADE, + liked_at timestamptz NOT NULL DEFAULT now(), + deleted_at timestamptz, + session_vector jsonb, + session_id uuid REFERENCES play_sessions(id) ON DELETE SET NULL +); + +-- Partial index: the engine queries only active rows; this is the hot path. +CREATE INDEX contextual_likes_active_idx + ON contextual_likes (user_id, track_id) + WHERE deleted_at IS NULL; + +-- GIN index for vector similarity queries (used by sub-plan #3's +-- contextual_match_score lookup). Even though M3 sub-plan #3 doesn't yet +-- exist, this slice adds the index because the recommendation engine +-- will need it; better to migrate once. +CREATE INDEX contextual_likes_vector_idx + ON contextual_likes USING gin (session_vector); + +-- Lookup support for the soft-delete update (UPDATE ... WHERE user_id = $1 +-- AND track_id = $2 AND deleted_at IS NULL). Already covered by the +-- partial index above — no separate index needed. +``` + +`down.sql`: + +```sql +DROP TABLE IF EXISTS contextual_likes; +``` + +## Session vector shape + +```go +type SessionVector struct { + Seed bool `json:"seed"` + Artists []string `json:"artists"` // distinct, ordered by first appearance + Tags map[string]int `json:"tags"` // bag-of-counts + RecentTrackIDs []string `json:"recent_track_ids"` // newest last +} + +func BuildSessionVector(priorTracks []dbq.Track) SessionVector +``` + +Rules: +- `Seed = len(priorTracks) < 3`. The engine in #342 filters seed vectors out of the contextual-match query (they don't represent enough context to match against). +- `Artists` is the deduplicated list of `priorTracks[i].ArtistID` values, formatted as UUID strings. Order is "first appearance" (older tracks first). +- `Tags` is a `map[string]int` counting how many tracks have each genre. `priorTracks[i].Genre == nil || == ""` means no contribution. Tags are case-sensitive (we don't normalize — `"Rock"` and `"rock"` are different bins; future cleanup can add normalization). +- `RecentTrackIDs` is the ordered list, newest last (matches the spec's "ordered list of the N track ids"). +- The function does NOT truncate. The caller passes in at most N tracks; the function works with whatever it's given (extra coverage flexibility). + +## API contracts + +No HTTP shape changes. `POST /api/events`, `POST/DELETE /api/likes/...`, and Subsonic `/rest/star`, `/rest/unstar` all keep their current request/response contracts. The new behavior is internal data-write side effects. + +## Components & files + +### New server files + +| Path | Responsibility | +|---|---| +| `internal/db/migrations/0007_contextual_likes.up.sql` + `.down.sql` | Schema for `contextual_likes` + partial index + GIN index. | +| `internal/db/queries/contextual_likes.sql` | `InsertContextualLike`, `SoftDeleteContextualLikesForUserTrack`. | +| `internal/db/dbq/contextual_likes.sql.go` | Generated bindings. | +| `internal/recommendation/sessionvector.go` | `SessionVector` struct + `BuildSessionVector(priorTracks)` pure function. | +| `internal/recommendation/sessionvector_test.go` | Unit tests for the pure function (boundary cases for the seed flag, dedup, tag counts, ordering, JSON round-trip). | + +### Modified server files + +| Path | Change | +|---|---| +| `internal/db/queries/events.sql` | Add `ListRecentSessionTracks(sessionID, beforeTime, limit) :many` (joins play_events ↔ tracks, orders by started_at DESC). Add `UpdatePlayEventVector(id, vector) :exec`. | +| `internal/db/queries/likes.sql` | Change `LikeTrack` from `:exec` to `:execrows` so the handler detects insert vs already-exists. | +| `internal/playevents/writer.go::RecordPlayStarted` | After InsertPlayEvent: ListRecentSessionTracks → BuildSessionVector → UpdatePlayEventVector. All inside the existing transaction. | +| `internal/playevents/writer.go::RecordSyntheticCompletedPlay` | Same vector computation as RecordPlayStarted (Subsonic synthetic plays should also carry context). Single source of truth via a private helper. | +| `internal/playevents/writer_test.go` | Three new tests covering vector persistence (seed flag, populated vector, session-scope isolation). | +| `internal/api/likes.go::handleLikeTrack` | Capture `:execrows` result; if 1, GetOpenPlayEventForUser; if open + non-null vector, InsertContextualLike. | +| `internal/api/likes.go::handleUnlikeTrack` | After UnlikeTrack, SoftDeleteContextualLikesForUserTrack. | +| `internal/api/likes_test.go` | Five new tests (capture during open play, no-op without play, no duplicate on idempotent like, soft-delete on unlike, history accumulation across like/unlike/like). | +| `internal/subsonic/star_test.go` | One new test confirming Subsonic star captures contextual_likes when a now-playing event is open. | + +### No web changes + +The web client already calls `/api/events` and `/api/likes/...`. After this slice, behavior is identical from the SPA's perspective; the side-effects are server-side only. + +## Data flow + +**Play started:** + +1. SPA calls `POST /api/events` with `type: 'play_started', track_id`. +2. `handleEventPlayStarted` → `events.Writer.RecordPlayStarted(ctx, userID, trackID, clientID, at)`. +3. Inside the transaction: + - Auto-close any prior open row (existing). + - `playsessions.FindOrCreate(...)` → `sessionID` (existing). + - `q.InsertPlayEvent(...)` → `playEventID` (existing). + - **NEW:** `priorTracks := q.ListRecentSessionTracks(ctx, sessionID, at, 5)`. Returns up to 5 tracks where `started_at < at` (excluding the just-inserted row). + - **NEW:** `vec := recommendation.BuildSessionVector(priorTracks)`. + - **NEW:** `vecJSON, _ := json.Marshal(vec)`. + - **NEW:** `q.UpdatePlayEventVector(ctx, UpdatePlayEventVectorParams{ID: playEventID, SessionVectorAtPlay: vecJSON})`. +4. Returns `StartedResult{PlayEventID, SessionID}`. + +**Subsonic synthetic completed play:** `RecordSyntheticCompletedPlay` follows the same vector-computation path so Subsonic-driven plays carry context too. + +**Like a track:** + +1. `POST /api/likes/tracks/:id` (or Subsonic `/rest/star?id=X`). +2. `handleLikeTrack` validates auth + UUID + entity exists. +3. `q.LikeTrack(...)` returns rows-affected count (sqlc `:execrows`). +4. **If rows == 0** (already liked): return 204. No contextual capture. +5. **If rows == 1** (freshly inserted): + - `event, err := q.GetOpenPlayEventForUser(ctx, userID)`. + - If `errors.Is(err, pgx.ErrNoRows)` → return 204 (no open event, no contextual capture). + - If `event.SessionVectorAtPlay` is NULL → return 204 (event opened before this slice landed; no vector). + - Else: `q.InsertContextualLike(ctx, InsertContextualLikeParams{UserID, TrackID, SessionVector: event.SessionVectorAtPlay, SessionID: &event.SessionID})`. +6. Return 204. + +**Unlike a track:** + +1. `DELETE /api/likes/tracks/:id` (or Subsonic `/rest/unstar?id=X`). +2. `handleUnlikeTrack` validates UUID. +3. `q.UnlikeTrack(...)` (existing). +4. **NEW:** `q.SoftDeleteContextualLikesForUserTrack(ctx, userID, trackID)` — `UPDATE contextual_likes SET deleted_at = now() WHERE user_id = $1 AND track_id = $2 AND deleted_at IS NULL`. +5. Return 204. + +**Edge cases:** + +- **Like with no open play_event.** No contextual capture; only `general_likes`. Same outcome as today. +- **Like during cold-start session.** Vector has `seed: true`. Row is still inserted (the seed flag is informational; the engine in #342 filters on it). +- **Re-like of an already-liked track in the same session.** `LikeTrack` returns 0 affected rows; contextual capture skipped. No spam. +- **Re-like after unlike, in the same session.** Unlike soft-deleted prior rows. New like inserts a fresh row. The (user, track) pair has both rows in the table; only the fresh one is `deleted_at IS NULL`. +- **Unlike of a never-liked track.** `UnlikeTrack` is a no-op. `SoftDeleteContextualLikesForUserTrack` runs but updates 0 rows. Both safe. +- **Album / artist likes.** `contextual_likes` is track-only. Album/artist like handlers are untouched in this slice. +- **Subsonic `star?albumId=&artistId=`.** Album/artist branches don't trigger contextual capture (track-only). The track-id branch picks it up via the shared `dbq.LikeTrack` call. + +## Testing + +### Server (`go test`) + +**`internal/recommendation/sessionvector_test.go`** (pure unit tests): +- Empty input → seed=true, all collections empty. +- 1 track → seed=true, artists has 1 entry, tags has 1 entry (count=1), recent_track_ids has 1 entry. +- 2 tracks → seed=true. +- 3 tracks → seed=false. Single-artist case: artists deduplicated to 1 entry. Multi-genre case: tags accumulates counts. +- 5 tracks across 3 artists, mixed genres → all populated, dedup correct, counts correct. +- Track with empty/nil genre → tag entry not added. +- 10 tracks input → function processes all 10 (caller controls truncation; document the contract). +- JSON round-trip via `encoding/json`: marshal, unmarshal, fields equal. + +**`internal/playevents/writer_test.go`** (live-DB integration, three new): +- `TestRecordPlayStarted_PersistsSessionVector_Seed`: first play in fresh session → row's `session_vector_at_play.seed == true`, empty arrays. +- `TestRecordPlayStarted_PersistsSessionVector_Populated`: after 4 plays, the 5th's vector has `seed: false` and the 4 prior tracks' artists/tags/ids. +- `TestRecordPlayStarted_VectorScopedToSession`: open a play_event, advance > 30 minutes, start another (new session) — second play's vector is `seed: true` with nothing from the prior session. + +**`internal/api/likes_test.go`** (live-DB integration, five new): +- `TestLikeTrack_DuringOpenPlayEvent_WritesContextualLike`: user starts playing track A (creates play_event with non-null vector); likes track B; assert `contextual_likes` has one row for (user, B) with the matching `session_vector` and `session_id`. +- `TestLikeTrack_NoOpenPlayEvent_NoContextualLike`: user likes a track without playing — `general_likes` has the row, `contextual_likes` is empty. +- `TestLikeTrack_RepeatedLike_NoDuplicate`: like the same track twice — first call writes both rows; second call no-ops (LikeTrack returns 0 rows; contextual capture skipped). +- `TestUnlikeTrack_SoftDeletesContextualLikes`: like → unlike. `general_likes` row gone, `contextual_likes` row's `deleted_at` is set, row count unchanged. +- `TestLikeUnlikeRelike_HistoryAccumulates`: like → unlike → like. Two `contextual_likes` rows for that (user, track): one with `deleted_at` set, one without. + +**`internal/subsonic/star_test.go`** (live-DB, one new): +- `TestHandleStar_DuringNowPlaying_WritesContextualLike`: send `submission=false` for track A (creates open play_event with vector), then `star?id=B` — `contextual_likes` row exists for (user, B). + +**Migration smoke**: existing `internal/db` test runs the new 0007 migration; verify no errors. + +### End-to-end manual + +Final task in the implementation plan: + +1. Sign in. Play 4 tracks all the way through. +2. `psql ... SELECT id, session_vector_at_play FROM play_events ORDER BY started_at DESC LIMIT 5` — vectors are populated; first 3 have `seed: true`, 4th onward `seed: false`. +3. While the 4th track is playing, like it via the heart button. `SELECT * FROM contextual_likes` shows a row with that track's session_vector. +4. Like a different track NOT currently playing. `general_likes` row appears; no `contextual_likes` row. +5. Unlike the first track. `general_likes` deleted; `contextual_likes` row still exists but `deleted_at` is set. +6. Re-like that track in a new session. New `contextual_likes` row with `deleted_at IS NULL` and a different `session_vector` (different artists/tags from the new session). +7. From Feishin: send a now-playing for track A, then star track B. Confirm contextual capture worked through the Subsonic path. + +## Risks & mitigations + +- **Vector size bloat**: each play_event row carries up to 5 artist UUIDs + a tag map + 5 track UUIDs. ~500 bytes per play_event in JSON. For a user with 50,000 plays, that's 25 MB of JSONB. Acceptable; PostgreSQL handles it. If telemetry shows the column dominates row width, future work can extract to a separate table or hash older vectors. +- **Tag normalization**: `tracks.genre` is denormalized free-text. `"Rock"` and `"rock"` end up as different bins. v1 doesn't normalize. Mitigation: documented; M3.5/M4 cleanup can `lower()` consistently; the recommendation engine in #342 can do its similarity calculation on lowercased tags as a workaround. +- **Eager column read**: extending `RecordPlayStarted` adds 1 SELECT (recent tracks) + 1 UPDATE (vector) per play. Single-digit ms overhead at v1 scale. Reasonable cost for the data we get. +- **NULL vector after this slice ships**: existing play_events from before the migration have NULL `session_vector_at_play`. The like-handler's contextual-capture path skips when the vector is NULL — those old plays simply don't generate contextual likes. No backfill; new plays going forward populate the vector. +- **Subsonic synthetic plays carry vector**: synthetic plays from `submission=true` use the same vector logic. Subsonic clients that scrobble in bulk after-the-fact (e.g. submit 10 plays in 5 seconds) generate vectors quickly across each new session/track — vectors may show artificial 5-track sliding windows during the catch-up. Acceptable for v1; treats Subsonic clients as first-class citizens of the recommendation pipeline. +- **`contextual_likes` table forgotten in 0005**: this slice adds it. Existing migration history is unchanged; the new 0007 migration brings the schema up to spec. +- **Soft-delete query without explicit unique constraint**: `SoftDeleteContextualLikesForUserTrack` updates ALL active rows for (user, track), which may be more than one if a user liked the same track in multiple sessions. That's the correct behavior — unliking means "all my historical likes for this track no longer count." Documented.