docs: seed server spec on dev

2026-04-18 17:56:28 +00:00
parent 68912e60f7
commit 5861c146f7
1 changed files with 340 additions and 0 deletions
@@ -0,0 +1,340 @@
 # Project Spec: Smart Music Server (working name)
 ## 1. Overview
 A self-hosted music server and streaming backend built to close a real gap in the self-hosted music ecosystem: **state and intelligence belong on the server, not the client.** Existing Subsonic-compatible servers (Navidrome, Gonic, Airsonic-Advanced) defer all of the "smart" behavior — shuffle logic, radio, play count semantics, like management — to clients. The result is fragmented experience across devices: a song liked on one client is unknown to another; shuffle quality depends on which app you opened; play counts diverge. This project centralizes those decisions on the server so every client sees the same canonical state.
 Primary differentiator: a **dual-like model** (general likes vs. contextual likes tied to the recent listening session) combined with **session-aware weighted shuffle** and **"smart radio" playlist generation** that considers user history, session context, and external similarity data from ListenBrainz/MusicBrainz.
 The system also integrates with Lidarr for library management and scrobbles to ListenBrainz.
 **Scope of this spec:** the backend server. A companion Flutter mobile client is part of the v1 product and is specified in a separate document (`smart-music-client-spec.md`). The two are co-designed: the extended API in this spec is shaped by what the client needs, and the client spec assumes the server features described here. Implementation proceeds in parallel after the server reaches Subsonic-compatible baseline.
 ## 2. Goals and non-goals
 ### Goals
 - Self-hosted music server running on Linux (primary target: Docker container on Ubuntu/CachyOS).
 - OpenSubsonic API compatibility so existing clients (Symfonium, Tempo, Feishin, etc.) work out of the box.
 - Server-side smart shuffle that beats random playback in a measurable way for the user.
 - Server-side "radio" feature: seed from a track, artist, or album and generate a contextually coherent queue.
 - Dual-like model (contextual and general) with retrievable contextual signal.
 - Session tracking with play/skip/seek/complete events persisted as first-class data.
 - ListenBrainz scrobbling (outbound) and similarity data pull (inbound).
 - Lidarr integration: search and add from the server UI; quarantine-based removal workflow (never auto-delete).
 - Web UI sufficient to manage the library, likes, Lidarr integration, and system settings.
 ### Non-goals (v1)
 - Multi-tenant / public-facing deployment. Single-household use assumed; multi-user within household is a goal but not hardened against adversarial users.
 - Audio fingerprinting / deduplication beyond what MusicBrainz IDs provide.
 - Local ML embedding generation. Use ListenBrainz/MusicBrainz similarity data instead.
 - Transcoding profiles beyond what OpenSubsonic expects.
 - Podcast support.
 - Video support.
 - Internet radio streaming.
 ### Explicit non-goals to push back on later
 If during implementation there is pressure to add: local audio feature extraction (Essentia/librosa), neural embeddings, or auto-delete-on-dislike — stop and re-scope. These are v2+ discussions.
 ## 3. Technology choices
 - **Language:** Go. Rationale: single-binary deployment, strong concurrency model for streaming + background workers, mature ecosystem for HTTP services, good fit for home lab containerized deployment.
 - **HTTP:** Go standard library `net/http` + `chi` or `echo` router. No heavyweight framework.
 - **Database:** PostgreSQL. Rationale: the session-vector similarity queries and event aggregations will benefit from real indexing, window functions, and JSONB. SQLite is tempting for single-user deployments but will bottleneck on the analytics queries central to this project's value.
 - **Database access:** `sqlc` for typed queries generated from SQL. Migrations via `goose` or `golang-migrate`.
 - **Audio/transcoding:** `ffmpeg` as an external binary invoked per stream. Do not bundle or reimplement.
 - **Tag reading:** `go-taglib` bindings or `dhowden/tag` for pure-Go tag parsing.
 - **MusicBrainz / ListenBrainz:** direct HTTP clients, no wrapper libraries unless a well-maintained one exists at implementation time.
 - **Lidarr:** direct HTTP client against Lidarr's v1 API.
 - **Web UI:** server-rendered HTML + HTMX for v1. Rationale: avoids a full SPA build pipeline for what is a minority surface. Reconsider for v2 if the UI grows.
 - **Auth:** token-based for API clients (OpenSubsonic uses its own auth scheme — support it). Session cookies for the web UI.
 - **Deployment:** Dockerfile producing a scratch or distroless image containing the server binary + ffmpeg. Docker Compose example in the repo.
 - **Config:** YAML config file with env var override. Sensible defaults.
 - **Logging:** structured logging via `log/slog`. Log levels configurable.
 ## 4. Architecture
 ### High-level components
 1. **Library scanner** — walks configured music folders, parses tags, writes canonical track/artist/album records, resolves MusicBrainz IDs where available.
 2. **Stream server** — OpenSubsonic endpoints for browsing, streaming, transcoding on demand.
 3. **Event ingestion** — records play/skip/seek/complete events from clients, groups them into sessions.
 4. **Session service** — maintains the notion of a "current session" per user, computes session vectors (the previous-N tracks and their aggregate features), handles session timeouts.
 5. **Recommendation engine** — the core value layer. Implements weighted shuffle, radio generation, and contextual-like retrieval. Pure functions operating on data from the DB and similarity service.
 6. **Similarity service** — wraps ListenBrainz similarity API calls with aggressive caching. Local DB stores similarity edges.
 7. **Scrobble worker** — batches completed plays and forwards to ListenBrainz.
 8. **Lidarr integration** — adapter for search, add, quarantine operations.
 9. **Web UI** — thin surface over the above.
 10. **Extended API** — non-Subsonic endpoints for features Subsonic clients cannot express (contextual likes, session inspection, Lidarr actions).
 ### Key architectural decisions
 - **OpenSubsonic compatibility first, extensions second.** All existing Subsonic features must work with third-party clients. The novel features live in a parallel API namespace (e.g. `/api/v1/`) that the future custom client will use.
 - **Events are append-only.** The events table is the source of truth for listening history. All aggregates (play counts, skip ratios, session vectors) are computed from it. This enables retroactive model changes without data loss.
 - **Session computation is online but session similarity is precomputed.** When a track is played, its "session vector at time of play" is computed and stored on the play event. This makes contextual-like retrieval a straightforward query rather than a replay of history.
 - **Recommendation is a pure function of DB state.** No in-memory state that would be lost on restart. Caches are explicit and rebuildable.
 ### Request lifecycle (streaming)
 Client → auth check → track lookup → (optional) transcode ffmpeg pipeline → byte range response. Play event recorded when client reports progress beyond a threshold (see §6).
 ### Request lifecycle (smart shuffle / radio)
 Client → auth → `getRandomSongs` (Subsonic) or `/api/v1/radio` (extended) → recommendation engine → query composer pulls candidate set → weighting applied → ranked list returned.
 ## 5. Data model
 The schema below is the v1 target. Prefer explicit, descriptive column names over abbreviations. All timestamps are `timestamptz`.
 ### Core library tables
 **`artists`** — id (uuid), name, sort_name, mbid (nullable, unique when present), created_at, updated_at.
 **`albums`** — id, title, sort_title, artist_id, release_date, mbid, cover_art_path, created_at, updated_at.
 **`tracks`** — id, title, album_id, artist_id, track_number, disc_number, duration_ms, file_path (unique), file_size, file_format, bitrate, mbid, genre (denormalized text for fast filtering), added_at, updated_at.
 **`users`** — id, username, password_hash, api_token, is_admin, created_at.
 ### Event and session tables
 **`play_events`** — id, user_id, track_id, started_at, ended_at (nullable until completion reported), duration_played_ms, completion_ratio (computed: duration_played_ms / track.duration_ms), was_skipped (bool), client_id (nullable), session_id, session_vector_at_play (jsonb — see §6), scrobbled_at (nullable).
 **`skip_events`** — id, user_id, track_id, skipped_at, position_ms, session_id. (Subset of play_events where was_skipped=true, but kept as a separate index-friendly table for skip analytics.)
 **`sessions`** — id, user_id, started_at, ended_at (nullable), last_event_at, track_count, client_id.
 ### Like tables
 **`general_likes`** — user_id, track_id, liked_at. Primary key (user_id, track_id). A single row per user-track pair. Unliking deletes the row.
 **`contextual_likes`** — id, user_id, track_id, liked_at, session_vector (jsonb, snapshot at time of like), session_id. Multiple rows per user-track pair are allowed and expected — each represents a like in a specific context.
 **`artist_preferences`** — user_id, artist_id, weight (float, -1.0 to 1.0), updated_at. Aggregate signal, computed from likes/skips but directly adjustable by user.
 ### Similarity tables
 **`track_similarity`** — track_a_id, track_b_id, score (float), source (enum: listenbrainz, musicbrainz_tag, user_cooccurrence), fetched_at. Primary key (track_a_id, track_b_id, source).
 **`artist_similarity`** — analogous to track_similarity.
 ### Integration tables
 **`lidarr_quarantine`** — id, track_id, requested_by_user_id, reason (enum: wrong_match, low_quality, disliked, other), requested_at, resolved_at (nullable), resolution (enum: deleted, restored, nullable).
 **`scrobble_queue`** — id, play_event_id, queued_at, attempts, last_attempt_at, last_error (nullable), submitted_at (nullable).
 ### Indexes (non-exhaustive)
 - `play_events (user_id, started_at DESC)` for session assembly.
 - `play_events (user_id, track_id)` for per-track history.
 - `contextual_likes USING gin (session_vector)` for vector similarity queries.
 - `track_similarity (track_a_id, score DESC)` for fast candidate generation.
 ## 6. Core algorithms
 ### Session definition
 A session is a contiguous sequence of plays by a single user with no gap longer than **30 minutes** between events. The timeout is configurable. A new play event more than 30 minutes after the last event in a session starts a new session.
 ### Session vector
 At the moment a track begins playing, compute a vector representing the session context. v1 definition:
 - Last N = 5 tracks played in the current session (if fewer than 5, use what exists; if session just started, vector is marked `seed` and excluded from contextual retrieval until session has ≥3 tracks).
 - For each of those N tracks, collect: artist_id, genre tags, MusicBrainz tags if available, approximate BPM/energy if available (optional, skip if not).
 - Aggregate into a JSONB structure:
  - `artists`: set of artist_ids
  - `tags`: bag-of-tags with counts
  - `recent_track_ids`: ordered list of the N track ids
 - Store this JSONB on the play event and on any contextual_like created during this track.
 ### Session similarity
 Two session vectors are compared via weighted Jaccard similarity on the `tags` and `artists` fields. The exact weights should be tunable via config (start with: tags 0.7, artists 0.3). This is simple and good enough for v1; upgrade paths include cosine similarity on tag vectors and eventually audio embeddings.
 ### Play vs. skip classification
 A play event is a **skip** if ended before 50% of track duration AND before 30 seconds of playback. Otherwise it's a **play**. A **complete play** is ≥ 90% of duration. Thresholds configurable.
 ### Weighted shuffle
 Given a candidate set (e.g. all tracks, or all tracks in an album/playlist/genre), compute a score per track:
 ```
 score = base_weight
      + (is_general_liked ? LIKE_BOOST : 0)
      + (contextual_match_score * CONTEXT_WEIGHT)
      + (recency_decay(last_played_at) * RECENCY_WEIGHT)
      - (skip_ratio * SKIP_PENALTY)
      - (recently_played_penalty)
      + small_random_jitter
 ```
 - `contextual_match_score` = max session similarity between the current session vector and any contextual_like session vector for this track, for this user.
 - `recency_decay` returns a value that is high when a track hasn't been played recently and low when it has, but not infinite for never-played tracks (cold-start handling).
 - `recently_played_penalty` hard-suppresses anything played in the last hour to prevent immediate repeats.
 Weights are in config. Ship with reasonable defaults and document how to tune them.
 ### Radio generation
 Given a seed (track, artist, or album):
 1. Build a candidate pool from: (a) ListenBrainz similar tracks to the seed, (b) tracks by similar artists, (c) tracks sharing MusicBrainz tags, (d) user's general likes matching those tags.
 2. Filter to the user's library (Lidarr-integrated libraries may choose to also propose additions — see §7).
 3. Apply weighted shuffle scoring on the pool.
 4. Return an ordered queue of configurable length (default 50).
 5. Refresh policy: regenerate when the queue is 80% consumed, biased against tracks already played in this radio session.
 ### Scrobbling
 A play event is eligible for scrobbling when:
 - completion_ratio ≥ 0.5, OR
 - duration_played_ms ≥ 4 minutes
 (These are ListenBrainz's recommended thresholds; confirm at implementation time.) Eligible events enter the scrobble_queue and are batched by the scrobble worker. Retries with exponential backoff on network or 5xx errors. Permanent failures (4xx, malformed) are logged and the row is marked failed.
 ### ListenBrainz similarity ingest
 On library scan completion, and periodically thereafter, for tracks with MBIDs:
 - Query ListenBrainz similarity endpoints in batches.
 - Populate `track_similarity` and `artist_similarity`.
 - Respect ListenBrainz rate limits. Cache aggressively; re-fetch no more than weekly per track.
 ## 7. Lidarr integration
 ### Search and add
 Expose an endpoint on the extended API that proxies Lidarr's search. Users can initiate an add from the web UI. The server calls Lidarr with the user's configured root folder and quality profile. After Lidarr reports the download complete, the next library scan picks up the file.
 ### Quarantine (never auto-delete)
 When a user flags a track as wrong/low-quality/disliked:
 1. A `lidarr_quarantine` row is created.
 2. The track is hidden from playback for that user (soft-hide; other users in the household still see it).
 3. The file is NOT deleted.
 4. An admin UI lists quarantined tracks. An admin can choose to delete (via Lidarr's delete endpoint) or restore.
 **Rationale:** a one-tap deletion of a 40MB FLAC rip during a user-hostile moment is unacceptable. Quarantine is cheap; deletion is destructive. This is the correct tradeoff even though it adds a UI step.
 ### Radio with add-suggestions
 When generating a radio queue, if the server has similarity data pointing to tracks not in the library AND Lidarr integration is enabled, surface these as "suggested additions" in the queue response (separate field, not interleaved with playable tracks). The user can one-tap add-to-Lidarr from the client.
 ## 8. API surface
 ### Subsonic / OpenSubsonic endpoints
 Implement the full OpenSubsonic spec as of implementation time. At minimum: `ping`, `getLicense`, `getMusicFolders`, `getIndexes`, `getArtists`, `getArtist`, `getAlbum`, `getAlbumList2`, `getSong`, `search3`, `stream`, `download`, `scrobble`, `star`, `unstar`, `getStarred2`, `getRandomSongs`, `getSimilarSongs2`, `createPlaylist`, `getPlaylists`, `getPlaylist`, `updatePlaylist`, `deletePlaylist`, `getCoverArt`.
 Map Subsonic concepts onto the extended model:
 - Subsonic `star` → creates a `general_likes` row.
 - Subsonic `scrobble` with `submission=true` → creates a play_event.
 - Subsonic `scrobble` with `submission=false` → updates now-playing state.
 - Subsonic `getRandomSongs` → invokes weighted shuffle.
 - Subsonic `getSimilarSongs2` → invokes a simplified radio generation.
 ### Extended API (`/api/v1/`)
 - `POST /api/v1/events/play` — richer play event reporting (includes seek events, client metadata).
 - `POST /api/v1/likes/contextual` — create a contextual like. Server computes session vector.
 - `DELETE /api/v1/likes/contextual/:id` — remove a specific contextual like.
 - `GET /api/v1/session/current` — inspect the caller's current session.
 - `GET /api/v1/radio` — generate radio queue with explicit parameters (seed type, pool size, weights override).
 - `POST /api/v1/lidarr/search` — proxy search.
 - `POST /api/v1/lidarr/add` — request add.
 - `POST /api/v1/quarantine/:track_id` — flag a track.
 - `POST /api/v1/quarantine/:id/resolve` — admin resolution.
 - `GET /api/v1/stats/user` — listening stats for the caller.
 All extended endpoints use JSON request/response, not Subsonic's XML/JSON hybrid.
 ## 9. Web UI
 Server-rendered HTML + HTMX. Pages needed:
 - Login.
 - Library overview (artist/album/track browse, mostly a fallback — real browsing happens in clients).
 - Now playing / current session view.
 - Likes management (both lists, with ability to see contextual likes grouped by detected "contexts").
 - Lidarr search and add.
 - Quarantine admin.
 - User settings (scrobble target, shuffle weight overrides).
 - Server settings (library paths, scan triggers, integration config) — admin only.
 - Listening stats.
 Design constraint: functional and legible over fancy. The custom client is where UX effort goes.
 ## 10. Configuration
 YAML config at `/etc/smartmusic/config.yaml` by default, overridable with `--config` flag and env vars (prefixed `SMARTMUSIC_`).
 Required config sections:
 - `server` (listen address, external URL)
 - `database` (postgres connection)
 - `library` (scan paths, scan schedule)
 - `transcoding` (ffmpeg path, default profiles)
 - `listenbrainz` (token, enabled)
 - `lidarr` (url, api key, enabled, default root folder, default quality profile)
 - `shuffle` (weight overrides)
 - `session` (timeout minutes, N for session vector)
 - `auth` (token lifetime, admin bootstrap)
 ## 11. Testing
 - Unit tests for the recommendation engine with fixed inputs / expected rankings.
 - Unit tests for session boundary logic.
 - Integration tests for the Subsonic endpoint compatibility using a known-good client's test suite if one exists, otherwise golden-response tests.
 - Integration test harness with a Postgres testcontainer.
 - Load test for the stream endpoint under concurrent clients.
 Target: ≥70% coverage on the recommendation engine and session code specifically. Do not chase coverage on the Subsonic handler code — those tests should be scenario-driven.
 ## 12. Operational concerns
 - **Library scan** should be incremental (by mtime) with a full-rescan option. Large libraries (>100k tracks) should scan in under 10 minutes incremental, under 2 hours full.
 - **Streaming** should support byte-range requests correctly for seek support.
 - **Transcoding** results should be cacheable on disk with LRU eviction, configurable cache size.
 - **Backup** guidance: the Postgres DB is the only stateful component other than the music files themselves. Document pg_dump-based backup in the README.
 - **Metrics:** expose `/metrics` in Prometheus format. Key metrics: active streams, scan duration, scrobble queue depth, recommendation latency.
 ## 13. Implementation order
 This is the suggested sequencing for the server. The mobile client is specified separately and its work runs in parallel starting from step 5, once the server has a verifiable Subsonic baseline.
 1. Project skeleton, config loading, logging, DB migrations framework.
 2. Core schema (library tables, users).
 3. Library scanner — walk, tag parse, write records.
 4. Basic Subsonic API subset: auth, browse, stream. Verify against a real client (Symfonium or Feishin) end-to-end. **Milestone: usable Subsonic server.**
 5. Event ingestion + session service. *(Client work can begin here against a stubbed API.)*
 6. Like tables + Subsonic star/unstar wired to general_likes.
 7. Recommendation engine v1: weighted shuffle without contextual signal yet.
 8. Contextual likes + session vectors + contextual signal in recommendation.
 9. ListenBrainz scrobble worker.
 10. ListenBrainz similarity ingest.
 11. Radio generation.
 12. Lidarr integration (search, add, quarantine).
 13. Web UI.
 14. Metrics, packaging, Docker image, documentation.
 Each step should produce a runnable, testable increment. Don't implement step N+1 before step N is verified working. Server API changes discovered during client development should feed back into this spec and be implemented as they come up, rather than batched.
 ## 14. Out of scope reminders
 If during development any of the following come up, stop and reconsider scope:
 - Local audio feature extraction
 - Neural network anything
 - Adversarial multi-user hardening
 - Podcast / video / internet radio
 - Custom audio format support beyond what ffmpeg handles natively
 ## 15. Open questions to resolve during implementation
 - Exact session vector JSONB schema — finalize after looking at real MusicBrainz tag data.
 - Whether to store the `session_vector_at_play` on every play event or only on events that generate contextual likes. Storage cost vs. query simplicity tradeoff.
 - Whether `getRandomSongs` from Subsonic should use the full weighted shuffle or a lighter version, since some clients call it on a loop.
 - Handling of tracks without MBIDs (a significant fraction of many libraries). They can't participate in similarity-based radio. Decide whether to fall back to artist/genre/tag similarity or exclude them from radio entirely.
 - Whether contextual likes should decay over time (old contextual likes become weaker signal).