docs(spec): add M2 events sub-plan design
Specifies POST /api/events with start+end semantics, session service (30-min rolling), play/skip classification (spec §6 boundaries), sendBeacon + auto-close-on-next-start for abandoned plays, and Subsonic /rest/scrobble integration that writes synthetic completed plays so third-party clients feed the recommendation pipeline. Schema migration ships the full M2 set (play_events, skip_events, sessions) including M3-only nullable columns so M3 doesn't need a follow-up migration. General likes deferred to a separate M2 sub-plan; this slice covers spec §13 step 5 only.
This commit is contained in:
@@ -0,0 +1,221 @@
|
||||
# M2 Events Sub-Plan — Design Spec
|
||||
|
||||
**Status:** approved 2026-04-25
|
||||
**Slice:** M2 (Events, sessions, general likes), spec §13 step 5. General likes (step 6) deferred to a follow-up sub-plan.
|
||||
**Fable tasks:** #322 (schema), #323 (session service), #324 (events API), #325 (web UI wiring) — bundled into a single PR for this slice.
|
||||
|
||||
## Goal
|
||||
|
||||
Ship the play-event ingestion pipeline end-to-end: schema, session service, `POST /api/events` handler, Subsonic `/rest/scrobble` integration, and the SvelteKit player wiring that emits the events. Output is a populated `play_events` / `skip_events` / `sessions` set that M3's recommendation engine can read from. No recommendation logic in this slice; this is the data foundation.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- General likes table, `/api/likes` endpoint, Subsonic `star/unstar` wiring — own M2 sub-plan.
|
||||
- `contextual_likes`, `artist_preferences` — M3 (algorithm consumers).
|
||||
- `session_vector_at_play` JSONB population — M3 (the algorithm computes it). The column ships in this slice as nullable.
|
||||
- Background janitor for abandoned `ended_at IS NULL` rows. Auto-close-on-next-play handles the common case; user-never-comes-back leaves at most one open row per user, which doesn't impact recommendation queries (they filter `ended_at IS NOT NULL`).
|
||||
- Multi-device session merging. Each `play_started` opens a new row regardless of whether one is already open elsewhere; M3 decides what to do when interleaved.
|
||||
- Listen-history UI. The Web UI surfaces nothing about events in this slice — the wiring is invisible to the user.
|
||||
|
||||
## Architecture
|
||||
|
||||
A new database migration adds `play_events`, `skip_events`, and `sessions` per spec §5. A small Go package `internal/sessions` exposes a `FindOrCreate` helper implementing the 30-minute rolling-window rule from spec §6. A second package `internal/playevents` owns the write paths (start, end, skip, synthetic-completed) so both the JSON API handler and the existing Subsonic scrobble handler share one source of truth. A new HTTP handler `POST /api/events` accepts a discriminated-union JSON payload covering the three client-side event types. The web SPA gains a sibling module to the player store that watches state transitions and dispatches events.
|
||||
|
||||
**Event lifecycle:** start + end (option 1 from brainstorm). Each track produces one `play_events` row inserted at start (`ended_at` NULL) and updated at end. The skip-classification rule (spec §6) runs at end-time: a play is a SKIP if `completion_ratio < 0.5 AND duration_played_ms < 30000`. Both conditions must fail (i.e., either ≥50% OR ≥30s) for it to count as a play.
|
||||
|
||||
**Subsonic scrobble integration:** Third-party Subsonic clients call `/rest/scrobble` to record plays. Today the handler is a no-op stub. After this slice, `submission=false` writes a `play_started` (replacing the in-memory `nowPlayingMap`); `submission=true` writes a synthetic completed play (`play_started + play_ended` in one transaction). This puts mobile/desktop Subsonic clients on equal footing with the web SPA in M3's recommendation pipeline.
|
||||
|
||||
**Abandoned plays:** The web SPA uses `navigator.sendBeacon('/api/events', play_skipped)` on `pagehide` so graceful tab-close still completes the row. For ungraceful cases (network drop, browser crash, OS kill), the events handler's `play_started` path begins by auto-closing any prior `ended_at IS NULL` row for the same user. The close uses `ended_at = now()`, `duration_played_ms = min(now() - started_at, track.duration_ms)` (in ms), and `was_skipped = true`. The completion ratio is recomputed and the skip rule from spec §6 is *not* applied (auto-closed rows are flagged as skipped regardless, since we don't know how much the user actually heard). At most one open row per user at any time.
|
||||
|
||||
## Schema (migration `0005_events.up.sql`)
|
||||
|
||||
```sql
|
||||
CREATE TABLE sessions (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
started_at timestamptz NOT NULL,
|
||||
ended_at timestamptz,
|
||||
last_event_at timestamptz NOT NULL,
|
||||
track_count integer NOT NULL DEFAULT 0,
|
||||
client_id text
|
||||
);
|
||||
CREATE INDEX sessions_user_last_event_idx ON sessions (user_id, last_event_at DESC);
|
||||
|
||||
CREATE TABLE play_events (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
track_id uuid NOT NULL REFERENCES tracks(id) ON DELETE CASCADE,
|
||||
session_id uuid NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
|
||||
started_at timestamptz NOT NULL,
|
||||
ended_at timestamptz,
|
||||
duration_played_ms integer,
|
||||
completion_ratio double precision,
|
||||
was_skipped boolean NOT NULL DEFAULT false,
|
||||
client_id text,
|
||||
session_vector_at_play jsonb,
|
||||
scrobbled_at timestamptz
|
||||
);
|
||||
CREATE INDEX play_events_user_started_idx ON play_events (user_id, started_at DESC);
|
||||
CREATE INDEX play_events_user_track_idx ON play_events (user_id, track_id);
|
||||
|
||||
CREATE TABLE skip_events (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
track_id uuid NOT NULL REFERENCES tracks(id) ON DELETE CASCADE,
|
||||
session_id uuid NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
|
||||
skipped_at timestamptz NOT NULL,
|
||||
position_ms integer NOT NULL
|
||||
);
|
||||
CREATE INDEX skip_events_user_skipped_idx ON skip_events (user_id, skipped_at DESC);
|
||||
```
|
||||
|
||||
`session_vector_at_play` lands now as nullable so M3 doesn't need a follow-up migration; M2 writes NULL.
|
||||
|
||||
## API contracts
|
||||
|
||||
### `POST /api/events`
|
||||
|
||||
```ts
|
||||
type EventRequest =
|
||||
| { type: 'play_started'; track_id: string; at?: string; client_id?: string }
|
||||
| { type: 'play_ended'; play_event_id: string; duration_played_ms: number; at?: string }
|
||||
| { type: 'play_skipped'; play_event_id: string; position_ms: number; at?: string };
|
||||
|
||||
type EventResponse =
|
||||
| { play_event_id: string; session_id: string } // for play_started
|
||||
| { ok: true }; // for play_ended / play_skipped
|
||||
```
|
||||
|
||||
`at` defaults to server `now()` if omitted. `client_id` defaults to null.
|
||||
|
||||
**Errors:**
|
||||
- `400 bad_request` — missing/invalid `type`; required fields missing or malformed; UUIDs not parseable; `duration_played_ms` or `position_ms` negative.
|
||||
- `404 not_found` — `track_id` doesn't exist, or `play_event_id` doesn't exist.
|
||||
- `403 forbidden` — `play_event_id` belongs to a different user (event UUIDs aren't secrets, but writes must respect ownership).
|
||||
- `500 server_error` — DB issue.
|
||||
|
||||
### `/rest/scrobble` (Subsonic — existing endpoint, behavior change only)
|
||||
|
||||
Request shape unchanged. Behavior:
|
||||
- `submission=false&id=X&time=T` → calls `playevents.RecordPlayStarted(userID, trackID, clientID=c, at=T)` then returns the standard Subsonic `ok` envelope. Replaces the in-memory `nowPlayingMap`.
|
||||
- `submission=true&id=X&time=T` → calls `playevents.RecordSyntheticCompletedPlay(userID, trackID, clientID=c, at=T)` which writes `play_started` and `play_ended` together in a single transaction. `started_at = T`, `ended_at = T + track.duration_ms`, `duration_played_ms = track.duration_ms`, `was_skipped = false` (Subsonic clients don't tell us if a play was skipped — assume completion).
|
||||
- Other `submission` values: ignored, ack with `ok` (matches current behavior).
|
||||
|
||||
Response stays the standard Subsonic `ok` envelope; no new fields.
|
||||
|
||||
## Components & files
|
||||
|
||||
### New server files
|
||||
|
||||
| Path | Responsibility |
|
||||
|---|---|
|
||||
| `internal/db/migrations/0005_events.up.sql` + `.down.sql` | Schema for `play_events`, `skip_events`, `sessions`. |
|
||||
| `internal/db/dbq/events.sql` (sqlc input) + generated `events.sql.go` | `InsertSession`, `UpdateSessionLastEvent`, `GetMostRecentSessionForUser`, `InsertPlayEvent`, `UpdatePlayEventEnded`, `InsertSkipEvent`, `CloseAbandonedPlayEventsForUser`, `GetOpenPlayEventByUser`. |
|
||||
| `internal/sessions/service.go` | `FindOrCreate(ctx, tx, userID, eventTime, clientID, timeout) (sessionID, error)`. Composable into a larger transaction. |
|
||||
| `internal/sessions/service_test.go` | Live-DB tests covering the four scenarios above. |
|
||||
| `internal/playevents/writer.go` | `RecordPlayStarted`, `RecordPlayEnded`, `RecordPlaySkipped`, `RecordSyntheticCompletedPlay`. Owns the auto-close-prior step, skip classification rule, skip_events writes. |
|
||||
| `internal/playevents/writer_test.go` | Direct tests of rule edges (49/50% × 29/30s), auto-close logic, synthetic write. |
|
||||
| `internal/api/events.go` | `handleEvents` HTTP handler — JSON decode, dispatch by `type`, call into `playevents.writer`. |
|
||||
| `internal/api/events_test.go` | HTTP-level coverage for happy path, validation errors, ownership-mismatch. |
|
||||
|
||||
### Modified server files
|
||||
|
||||
| Path | Change |
|
||||
|---|---|
|
||||
| `internal/api/api.go` | Register `authed.Post("/events", h.handleEvents)`. |
|
||||
| `internal/subsonic/stream.go` | `handleScrobble` calls `playevents.writer` instead of `nowPlayingMap`. The `nowPlayingMap` struct + `newNowPlayingMap` + the `nowPlaying` field are deleted. The `mediaHandlers` constructor signature simplifies. |
|
||||
| `internal/subsonic/stream_test.go` | Update existing scrobble test for the new behavior; add a `submission=true` case. |
|
||||
| `internal/config/config.go` + `config.example.yaml` | New `events:` section: `session_timeout_minutes` (default 30), `skip_max_completion_ratio` (default 0.5), `skip_max_duration_played_ms` (default 30000). |
|
||||
|
||||
### New web files
|
||||
|
||||
| Path | Responsibility |
|
||||
|---|---|
|
||||
| `web/src/lib/player/events.svelte.ts` | `useEventsDispatcher()` — `$effect`-based watcher on `player.current` / `player.state`; owns `_playEventId` state; calls `api.post`; installs `pagehide` listener that uses `navigator.sendBeacon`. |
|
||||
| `web/src/lib/player/events.svelte.test.ts` | State-machine tests with mocked `api.post` and spied `navigator.sendBeacon`. |
|
||||
| `web/src/lib/player/clientId.ts` | `getOrCreateClientId(): string` — sessionStorage-backed UUID per tab. |
|
||||
|
||||
### Modified web files
|
||||
|
||||
| Path | Change |
|
||||
|---|---|
|
||||
| `web/src/lib/api/types.ts` | Add `EventRequest`, `EventResponse` types matching the server contracts. |
|
||||
| `web/src/lib/api/client.ts` | Add `api.post<T>(path, body)` helper if not already present. |
|
||||
| `web/src/routes/+layout.svelte` | Call `useEventsDispatcher()` once at mount, alongside the existing `useMediaSession()`. |
|
||||
|
||||
## Data flow
|
||||
|
||||
**Web SPA path:**
|
||||
|
||||
1. User clicks track → store mutates: `_state = 'loading'`, `_queue = tracks`, `_index = 0`, `_position = 0`.
|
||||
2. `<audio>` loads → fires `playing` → store transitions to `_state = 'playing'`.
|
||||
3. Events module's `$effect` sees `(player.current, player.state === 'playing')` for the first time on this track → `POST /api/events { type: 'play_started', track_id, client_id, at }` → server returns `{ play_event_id }` → cached in module state.
|
||||
4. Track plays. No emissions during playback.
|
||||
5. Track ends naturally → audio fires `ended` → store advances to next track or pauses → events module sees the transition out of `playing` for THIS track → `POST { type: 'play_ended', play_event_id, duration_played_ms, at }` → server updates the row, applies skip classification, optionally writes a `skip_events` row.
|
||||
6. Cycle repeats.
|
||||
|
||||
**Skip mid-track:** `skipNext()` advances queue → audio src changes → events module sees `player.current` change while `_playEventId` is still set → `POST { type: 'play_skipped', play_event_id, position_ms = _position, at }` → then on the new track's `playing` event, fires `play_started` for the new track.
|
||||
|
||||
**Tab close:** `pagehide` listener fires → `navigator.sendBeacon('/api/events', { type: 'play_skipped', play_event_id, position_ms = _position, at })`.
|
||||
|
||||
**Crash / network drop:** No `play_ended` ever sent. Row stays `ended_at IS NULL`. Next time the user starts ANY track on any client, the events handler's auto-close step closes the abandoned row before inserting the new one. Per the formula above: `ended_at = now()`, `duration_played_ms = min(now() - started_at, track.duration_ms)`, `was_skipped = true`.
|
||||
|
||||
**Subsonic path:**
|
||||
- `scrobble?submission=false` → handler calls `playevents.RecordPlayStarted`. Server has live "now playing" via `play_events WHERE ended_at IS NULL` (not used yet but available for M3+).
|
||||
- `scrobble?submission=true` → handler calls `playevents.RecordSyntheticCompletedPlay` which writes both rows in one transaction.
|
||||
|
||||
**Idempotency:** `play_ended` / `play_skipped` are naturally idempotent (each carries the row id). `play_started` is not — re-sending creates a new row. The auto-close-prior step protects the table from leaking open rows.
|
||||
|
||||
## Testing
|
||||
|
||||
### Server (`go test`)
|
||||
|
||||
- **Migration smoke** (`internal/db/db_test.go` extension): applying `0005_events` on a fresh test DB succeeds; round-trip insert+select for each new table.
|
||||
- **Session service** (`internal/sessions/service_test.go`):
|
||||
- No prior session → creates one with `started_at = eventTime`, `track_count = 0`.
|
||||
- Within window → returns existing session id, updates `last_event_at`.
|
||||
- Beyond window → creates a new session.
|
||||
- Concurrent inserts (`t.Parallel` with shared user, transactional fixture) don't double-create.
|
||||
- Uses a `Clock` interface so tests fast-forward without `time.Sleep`.
|
||||
- **playevents writer** (`internal/playevents/writer_test.go`):
|
||||
- Skip rule boundaries: `completion=0.49, duration=29s` → skip; `completion=0.5, duration=29s` → not skip; `completion=0.49, duration=30s` → not skip.
|
||||
- Auto-close-prior: prior open row gets closed with computed `ended_at` and `was_skipped=true` when a new `play_started` arrives.
|
||||
- Synthetic completed play writes both rows + the `was_skipped=false` flag in a single transaction.
|
||||
- **Events handler** (`internal/api/events_test.go`):
|
||||
- Happy path for all three event types.
|
||||
- 400 on missing `type`; 400 on `play_ended` referencing a malformed UUID; 400 on negative `duration_played_ms`.
|
||||
- 404 on `track_id` that doesn't exist; 404 on `play_event_id` that doesn't exist.
|
||||
- 403 on `play_event_id` belonging to a different user.
|
||||
- **Subsonic scrobble update** (`internal/subsonic/stream_test.go` extension):
|
||||
- Existing `submission=false` test now asserts a `play_events` row was inserted.
|
||||
- New `submission=true` test asserts both `play_started + play_ended` rows are written and the response is the standard Subsonic envelope.
|
||||
- Test-only code that referenced the deleted `nowPlayingMap` is removed.
|
||||
|
||||
### Web (`vitest`)
|
||||
|
||||
- **`events.svelte.test.ts`** with `vi.mock('$lib/api/client', ...)` and `vi.spyOn(navigator, 'sendBeacon')`:
|
||||
- Track 1 starts → exactly one `POST` with `type: 'play_started'`.
|
||||
- Server returns `play_event_id: 'pe1'`, then track ends → exactly one POST with `play_ended` carrying `pe1`.
|
||||
- User skips mid-track 2 → POST with `play_skipped` carrying that row's id.
|
||||
- Loading a new track via `playRadio()` while track 2 is still playing → POSTs `play_skipped` for track 2 BEFORE `play_started` for the new track.
|
||||
- `pagehide` event with an active play → `navigator.sendBeacon` called once with `play_skipped` payload.
|
||||
|
||||
### End-to-end manual
|
||||
|
||||
Final task in the implementation plan, against `docker compose up --build -d`:
|
||||
|
||||
1. Sign in to web SPA. Play a track. `psql -c "SELECT * FROM play_events"` shows one row, `ended_at IS NULL`, correct user/track/session.
|
||||
2. Let track finish. Same row now has `ended_at`, `was_skipped=false`.
|
||||
3. Skip mid-track (within 30s of start). New row created on next track; previous row's `was_skipped=true` and `skip_events` has the matching row.
|
||||
4. Wait > 30 minutes. Play. New `sessions` row with different id from the first session.
|
||||
5. Open Feishin/Symfonium pointed at the same instance. Play a track to completion. `play_events` shows a synthetic row.
|
||||
6. Close the web tab mid-track. `play_events` last row was closed via sendBeacon (timestamp ≈ tab close).
|
||||
7. Disconnect network mid-play, force-quit browser. Reconnect, sign back in, start a new track. Previous abandoned row gets auto-closed.
|
||||
|
||||
## Risks & mitigations
|
||||
|
||||
- **Event spam from buggy clients.** A misbehaving client could fire `play_started` in a loop, leaking rows. Mitigation: the auto-close-prior step caps each user at one open row at a time. Worst-case is one closed-row insert per `play_started` call, which is bounded by client-side sanity. Future M2.5 could add per-user rate limiting.
|
||||
- **Clock skew between client and server.** The client sends `at` timestamps; if the client's clock is wildly off, sessions could be assigned incorrectly (same user might see a "new session" for events that should extend the current one). Mitigation: server uses its own `now()` if `at` is omitted, and the events module always omits `at` (lets the server timestamp). The `at` field exists for the Subsonic compatibility path (Subsonic clients send `time=` and we honor it).
|
||||
- **Subsonic synthetic completed plays are too coarse.** A client that scrobbles after a 90% play is treated identically to a 100% play. Mitigation: Subsonic protocol doesn't expose finer info; this is a known limitation. M3's recommendation engine should weight Subsonic-sourced plays slightly lower if it ever matters in practice.
|
||||
- **`nowPlayingMap` deletion breaks an unknown caller.** Verified there are no consumers of the existing in-memory map outside `handleScrobble` itself; the constructor change is local. Migration is safe.
|
||||
- **JSONB `session_vector_at_play` ships nullable but is referenced in M3 plans.** No M3 task starts in this slice; the column is purely forward-compatibility. Documented in the schema comment.
|
||||
Reference in New Issue
Block a user