From 5861c146f79e58aab31831e08bbadc53cd1ef127 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 17:56:28 +0000 Subject: [PATCH 01/26] docs: seed server spec on dev --- docs/smart-music-server-spec.md | 340 ++++++++++++++++++++++++++++++++ 1 file changed, 340 insertions(+) create mode 100644 docs/smart-music-server-spec.md diff --git a/docs/smart-music-server-spec.md b/docs/smart-music-server-spec.md new file mode 100644 index 00000000..def6d03f --- /dev/null +++ b/docs/smart-music-server-spec.md @@ -0,0 +1,340 @@ +# Project Spec: Smart Music Server (working name) + +## 1. Overview + +A self-hosted music server and streaming backend built to close a real gap in the self-hosted music ecosystem: **state and intelligence belong on the server, not the client.** Existing Subsonic-compatible servers (Navidrome, Gonic, Airsonic-Advanced) defer all of the "smart" behavior — shuffle logic, radio, play count semantics, like management — to clients. The result is fragmented experience across devices: a song liked on one client is unknown to another; shuffle quality depends on which app you opened; play counts diverge. This project centralizes those decisions on the server so every client sees the same canonical state. + +Primary differentiator: a **dual-like model** (general likes vs. contextual likes tied to the recent listening session) combined with **session-aware weighted shuffle** and **"smart radio" playlist generation** that considers user history, session context, and external similarity data from ListenBrainz/MusicBrainz. + +The system also integrates with Lidarr for library management and scrobbles to ListenBrainz. + +**Scope of this spec:** the backend server. A companion Flutter mobile client is part of the v1 product and is specified in a separate document (`smart-music-client-spec.md`). The two are co-designed: the extended API in this spec is shaped by what the client needs, and the client spec assumes the server features described here. Implementation proceeds in parallel after the server reaches Subsonic-compatible baseline. + +## 2. Goals and non-goals + +### Goals + +- Self-hosted music server running on Linux (primary target: Docker container on Ubuntu/CachyOS). +- OpenSubsonic API compatibility so existing clients (Symfonium, Tempo, Feishin, etc.) work out of the box. +- Server-side smart shuffle that beats random playback in a measurable way for the user. +- Server-side "radio" feature: seed from a track, artist, or album and generate a contextually coherent queue. +- Dual-like model (contextual and general) with retrievable contextual signal. +- Session tracking with play/skip/seek/complete events persisted as first-class data. +- ListenBrainz scrobbling (outbound) and similarity data pull (inbound). +- Lidarr integration: search and add from the server UI; quarantine-based removal workflow (never auto-delete). +- Web UI sufficient to manage the library, likes, Lidarr integration, and system settings. + +### Non-goals (v1) + +- Multi-tenant / public-facing deployment. Single-household use assumed; multi-user within household is a goal but not hardened against adversarial users. +- Audio fingerprinting / deduplication beyond what MusicBrainz IDs provide. +- Local ML embedding generation. Use ListenBrainz/MusicBrainz similarity data instead. +- Transcoding profiles beyond what OpenSubsonic expects. +- Podcast support. +- Video support. +- Internet radio streaming. + +### Explicit non-goals to push back on later + +If during implementation there is pressure to add: local audio feature extraction (Essentia/librosa), neural embeddings, or auto-delete-on-dislike — stop and re-scope. These are v2+ discussions. + +## 3. Technology choices + +- **Language:** Go. Rationale: single-binary deployment, strong concurrency model for streaming + background workers, mature ecosystem for HTTP services, good fit for home lab containerized deployment. +- **HTTP:** Go standard library `net/http` + `chi` or `echo` router. No heavyweight framework. +- **Database:** PostgreSQL. Rationale: the session-vector similarity queries and event aggregations will benefit from real indexing, window functions, and JSONB. SQLite is tempting for single-user deployments but will bottleneck on the analytics queries central to this project's value. +- **Database access:** `sqlc` for typed queries generated from SQL. Migrations via `goose` or `golang-migrate`. +- **Audio/transcoding:** `ffmpeg` as an external binary invoked per stream. Do not bundle or reimplement. +- **Tag reading:** `go-taglib` bindings or `dhowden/tag` for pure-Go tag parsing. +- **MusicBrainz / ListenBrainz:** direct HTTP clients, no wrapper libraries unless a well-maintained one exists at implementation time. +- **Lidarr:** direct HTTP client against Lidarr's v1 API. +- **Web UI:** server-rendered HTML + HTMX for v1. Rationale: avoids a full SPA build pipeline for what is a minority surface. Reconsider for v2 if the UI grows. +- **Auth:** token-based for API clients (OpenSubsonic uses its own auth scheme — support it). Session cookies for the web UI. +- **Deployment:** Dockerfile producing a scratch or distroless image containing the server binary + ffmpeg. Docker Compose example in the repo. +- **Config:** YAML config file with env var override. Sensible defaults. +- **Logging:** structured logging via `log/slog`. Log levels configurable. + +## 4. Architecture + +### High-level components + +1. **Library scanner** — walks configured music folders, parses tags, writes canonical track/artist/album records, resolves MusicBrainz IDs where available. +2. **Stream server** — OpenSubsonic endpoints for browsing, streaming, transcoding on demand. +3. **Event ingestion** — records play/skip/seek/complete events from clients, groups them into sessions. +4. **Session service** — maintains the notion of a "current session" per user, computes session vectors (the previous-N tracks and their aggregate features), handles session timeouts. +5. **Recommendation engine** — the core value layer. Implements weighted shuffle, radio generation, and contextual-like retrieval. Pure functions operating on data from the DB and similarity service. +6. **Similarity service** — wraps ListenBrainz similarity API calls with aggressive caching. Local DB stores similarity edges. +7. **Scrobble worker** — batches completed plays and forwards to ListenBrainz. +8. **Lidarr integration** — adapter for search, add, quarantine operations. +9. **Web UI** — thin surface over the above. +10. **Extended API** — non-Subsonic endpoints for features Subsonic clients cannot express (contextual likes, session inspection, Lidarr actions). + +### Key architectural decisions + +- **OpenSubsonic compatibility first, extensions second.** All existing Subsonic features must work with third-party clients. The novel features live in a parallel API namespace (e.g. `/api/v1/`) that the future custom client will use. +- **Events are append-only.** The events table is the source of truth for listening history. All aggregates (play counts, skip ratios, session vectors) are computed from it. This enables retroactive model changes without data loss. +- **Session computation is online but session similarity is precomputed.** When a track is played, its "session vector at time of play" is computed and stored on the play event. This makes contextual-like retrieval a straightforward query rather than a replay of history. +- **Recommendation is a pure function of DB state.** No in-memory state that would be lost on restart. Caches are explicit and rebuildable. + +### Request lifecycle (streaming) + +Client → auth check → track lookup → (optional) transcode ffmpeg pipeline → byte range response. Play event recorded when client reports progress beyond a threshold (see §6). + +### Request lifecycle (smart shuffle / radio) + +Client → auth → `getRandomSongs` (Subsonic) or `/api/v1/radio` (extended) → recommendation engine → query composer pulls candidate set → weighting applied → ranked list returned. + +## 5. Data model + +The schema below is the v1 target. Prefer explicit, descriptive column names over abbreviations. All timestamps are `timestamptz`. + +### Core library tables + +**`artists`** — id (uuid), name, sort_name, mbid (nullable, unique when present), created_at, updated_at. + +**`albums`** — id, title, sort_title, artist_id, release_date, mbid, cover_art_path, created_at, updated_at. + +**`tracks`** — id, title, album_id, artist_id, track_number, disc_number, duration_ms, file_path (unique), file_size, file_format, bitrate, mbid, genre (denormalized text for fast filtering), added_at, updated_at. + +**`users`** — id, username, password_hash, api_token, is_admin, created_at. + +### Event and session tables + +**`play_events`** — id, user_id, track_id, started_at, ended_at (nullable until completion reported), duration_played_ms, completion_ratio (computed: duration_played_ms / track.duration_ms), was_skipped (bool), client_id (nullable), session_id, session_vector_at_play (jsonb — see §6), scrobbled_at (nullable). + +**`skip_events`** — id, user_id, track_id, skipped_at, position_ms, session_id. (Subset of play_events where was_skipped=true, but kept as a separate index-friendly table for skip analytics.) + +**`sessions`** — id, user_id, started_at, ended_at (nullable), last_event_at, track_count, client_id. + +### Like tables + +**`general_likes`** — user_id, track_id, liked_at. Primary key (user_id, track_id). A single row per user-track pair. Unliking deletes the row. + +**`contextual_likes`** — id, user_id, track_id, liked_at, session_vector (jsonb, snapshot at time of like), session_id. Multiple rows per user-track pair are allowed and expected — each represents a like in a specific context. + +**`artist_preferences`** — user_id, artist_id, weight (float, -1.0 to 1.0), updated_at. Aggregate signal, computed from likes/skips but directly adjustable by user. + +### Similarity tables + +**`track_similarity`** — track_a_id, track_b_id, score (float), source (enum: listenbrainz, musicbrainz_tag, user_cooccurrence), fetched_at. Primary key (track_a_id, track_b_id, source). + +**`artist_similarity`** — analogous to track_similarity. + +### Integration tables + +**`lidarr_quarantine`** — id, track_id, requested_by_user_id, reason (enum: wrong_match, low_quality, disliked, other), requested_at, resolved_at (nullable), resolution (enum: deleted, restored, nullable). + +**`scrobble_queue`** — id, play_event_id, queued_at, attempts, last_attempt_at, last_error (nullable), submitted_at (nullable). + +### Indexes (non-exhaustive) + +- `play_events (user_id, started_at DESC)` for session assembly. +- `play_events (user_id, track_id)` for per-track history. +- `contextual_likes USING gin (session_vector)` for vector similarity queries. +- `track_similarity (track_a_id, score DESC)` for fast candidate generation. + +## 6. Core algorithms + +### Session definition + +A session is a contiguous sequence of plays by a single user with no gap longer than **30 minutes** between events. The timeout is configurable. A new play event more than 30 minutes after the last event in a session starts a new session. + +### Session vector + +At the moment a track begins playing, compute a vector representing the session context. v1 definition: + +- Last N = 5 tracks played in the current session (if fewer than 5, use what exists; if session just started, vector is marked `seed` and excluded from contextual retrieval until session has ≥3 tracks). +- For each of those N tracks, collect: artist_id, genre tags, MusicBrainz tags if available, approximate BPM/energy if available (optional, skip if not). +- Aggregate into a JSONB structure: + - `artists`: set of artist_ids + - `tags`: bag-of-tags with counts + - `recent_track_ids`: ordered list of the N track ids +- Store this JSONB on the play event and on any contextual_like created during this track. + +### Session similarity + +Two session vectors are compared via weighted Jaccard similarity on the `tags` and `artists` fields. The exact weights should be tunable via config (start with: tags 0.7, artists 0.3). This is simple and good enough for v1; upgrade paths include cosine similarity on tag vectors and eventually audio embeddings. + +### Play vs. skip classification + +A play event is a **skip** if ended before 50% of track duration AND before 30 seconds of playback. Otherwise it's a **play**. A **complete play** is ≥ 90% of duration. Thresholds configurable. + +### Weighted shuffle + +Given a candidate set (e.g. all tracks, or all tracks in an album/playlist/genre), compute a score per track: + +``` +score = base_weight + + (is_general_liked ? LIKE_BOOST : 0) + + (contextual_match_score * CONTEXT_WEIGHT) + + (recency_decay(last_played_at) * RECENCY_WEIGHT) + - (skip_ratio * SKIP_PENALTY) + - (recently_played_penalty) + + small_random_jitter +``` + +- `contextual_match_score` = max session similarity between the current session vector and any contextual_like session vector for this track, for this user. +- `recency_decay` returns a value that is high when a track hasn't been played recently and low when it has, but not infinite for never-played tracks (cold-start handling). +- `recently_played_penalty` hard-suppresses anything played in the last hour to prevent immediate repeats. + +Weights are in config. Ship with reasonable defaults and document how to tune them. + +### Radio generation + +Given a seed (track, artist, or album): + +1. Build a candidate pool from: (a) ListenBrainz similar tracks to the seed, (b) tracks by similar artists, (c) tracks sharing MusicBrainz tags, (d) user's general likes matching those tags. +2. Filter to the user's library (Lidarr-integrated libraries may choose to also propose additions — see §7). +3. Apply weighted shuffle scoring on the pool. +4. Return an ordered queue of configurable length (default 50). +5. Refresh policy: regenerate when the queue is 80% consumed, biased against tracks already played in this radio session. + +### Scrobbling + +A play event is eligible for scrobbling when: +- completion_ratio ≥ 0.5, OR +- duration_played_ms ≥ 4 minutes + +(These are ListenBrainz's recommended thresholds; confirm at implementation time.) Eligible events enter the scrobble_queue and are batched by the scrobble worker. Retries with exponential backoff on network or 5xx errors. Permanent failures (4xx, malformed) are logged and the row is marked failed. + +### ListenBrainz similarity ingest + +On library scan completion, and periodically thereafter, for tracks with MBIDs: +- Query ListenBrainz similarity endpoints in batches. +- Populate `track_similarity` and `artist_similarity`. +- Respect ListenBrainz rate limits. Cache aggressively; re-fetch no more than weekly per track. + +## 7. Lidarr integration + +### Search and add + +Expose an endpoint on the extended API that proxies Lidarr's search. Users can initiate an add from the web UI. The server calls Lidarr with the user's configured root folder and quality profile. After Lidarr reports the download complete, the next library scan picks up the file. + +### Quarantine (never auto-delete) + +When a user flags a track as wrong/low-quality/disliked: +1. A `lidarr_quarantine` row is created. +2. The track is hidden from playback for that user (soft-hide; other users in the household still see it). +3. The file is NOT deleted. +4. An admin UI lists quarantined tracks. An admin can choose to delete (via Lidarr's delete endpoint) or restore. + +**Rationale:** a one-tap deletion of a 40MB FLAC rip during a user-hostile moment is unacceptable. Quarantine is cheap; deletion is destructive. This is the correct tradeoff even though it adds a UI step. + +### Radio with add-suggestions + +When generating a radio queue, if the server has similarity data pointing to tracks not in the library AND Lidarr integration is enabled, surface these as "suggested additions" in the queue response (separate field, not interleaved with playable tracks). The user can one-tap add-to-Lidarr from the client. + +## 8. API surface + +### Subsonic / OpenSubsonic endpoints + +Implement the full OpenSubsonic spec as of implementation time. At minimum: `ping`, `getLicense`, `getMusicFolders`, `getIndexes`, `getArtists`, `getArtist`, `getAlbum`, `getAlbumList2`, `getSong`, `search3`, `stream`, `download`, `scrobble`, `star`, `unstar`, `getStarred2`, `getRandomSongs`, `getSimilarSongs2`, `createPlaylist`, `getPlaylists`, `getPlaylist`, `updatePlaylist`, `deletePlaylist`, `getCoverArt`. + +Map Subsonic concepts onto the extended model: +- Subsonic `star` → creates a `general_likes` row. +- Subsonic `scrobble` with `submission=true` → creates a play_event. +- Subsonic `scrobble` with `submission=false` → updates now-playing state. +- Subsonic `getRandomSongs` → invokes weighted shuffle. +- Subsonic `getSimilarSongs2` → invokes a simplified radio generation. + +### Extended API (`/api/v1/`) + +- `POST /api/v1/events/play` — richer play event reporting (includes seek events, client metadata). +- `POST /api/v1/likes/contextual` — create a contextual like. Server computes session vector. +- `DELETE /api/v1/likes/contextual/:id` — remove a specific contextual like. +- `GET /api/v1/session/current` — inspect the caller's current session. +- `GET /api/v1/radio` — generate radio queue with explicit parameters (seed type, pool size, weights override). +- `POST /api/v1/lidarr/search` — proxy search. +- `POST /api/v1/lidarr/add` — request add. +- `POST /api/v1/quarantine/:track_id` — flag a track. +- `POST /api/v1/quarantine/:id/resolve` — admin resolution. +- `GET /api/v1/stats/user` — listening stats for the caller. + +All extended endpoints use JSON request/response, not Subsonic's XML/JSON hybrid. + +## 9. Web UI + +Server-rendered HTML + HTMX. Pages needed: + +- Login. +- Library overview (artist/album/track browse, mostly a fallback — real browsing happens in clients). +- Now playing / current session view. +- Likes management (both lists, with ability to see contextual likes grouped by detected "contexts"). +- Lidarr search and add. +- Quarantine admin. +- User settings (scrobble target, shuffle weight overrides). +- Server settings (library paths, scan triggers, integration config) — admin only. +- Listening stats. + +Design constraint: functional and legible over fancy. The custom client is where UX effort goes. + +## 10. Configuration + +YAML config at `/etc/smartmusic/config.yaml` by default, overridable with `--config` flag and env vars (prefixed `SMARTMUSIC_`). + +Required config sections: +- `server` (listen address, external URL) +- `database` (postgres connection) +- `library` (scan paths, scan schedule) +- `transcoding` (ffmpeg path, default profiles) +- `listenbrainz` (token, enabled) +- `lidarr` (url, api key, enabled, default root folder, default quality profile) +- `shuffle` (weight overrides) +- `session` (timeout minutes, N for session vector) +- `auth` (token lifetime, admin bootstrap) + +## 11. Testing + +- Unit tests for the recommendation engine with fixed inputs / expected rankings. +- Unit tests for session boundary logic. +- Integration tests for the Subsonic endpoint compatibility using a known-good client's test suite if one exists, otherwise golden-response tests. +- Integration test harness with a Postgres testcontainer. +- Load test for the stream endpoint under concurrent clients. + +Target: ≥70% coverage on the recommendation engine and session code specifically. Do not chase coverage on the Subsonic handler code — those tests should be scenario-driven. + +## 12. Operational concerns + +- **Library scan** should be incremental (by mtime) with a full-rescan option. Large libraries (>100k tracks) should scan in under 10 minutes incremental, under 2 hours full. +- **Streaming** should support byte-range requests correctly for seek support. +- **Transcoding** results should be cacheable on disk with LRU eviction, configurable cache size. +- **Backup** guidance: the Postgres DB is the only stateful component other than the music files themselves. Document pg_dump-based backup in the README. +- **Metrics:** expose `/metrics` in Prometheus format. Key metrics: active streams, scan duration, scrobble queue depth, recommendation latency. + +## 13. Implementation order + +This is the suggested sequencing for the server. The mobile client is specified separately and its work runs in parallel starting from step 5, once the server has a verifiable Subsonic baseline. + +1. Project skeleton, config loading, logging, DB migrations framework. +2. Core schema (library tables, users). +3. Library scanner — walk, tag parse, write records. +4. Basic Subsonic API subset: auth, browse, stream. Verify against a real client (Symfonium or Feishin) end-to-end. **Milestone: usable Subsonic server.** +5. Event ingestion + session service. *(Client work can begin here against a stubbed API.)* +6. Like tables + Subsonic star/unstar wired to general_likes. +7. Recommendation engine v1: weighted shuffle without contextual signal yet. +8. Contextual likes + session vectors + contextual signal in recommendation. +9. ListenBrainz scrobble worker. +10. ListenBrainz similarity ingest. +11. Radio generation. +12. Lidarr integration (search, add, quarantine). +13. Web UI. +14. Metrics, packaging, Docker image, documentation. + +Each step should produce a runnable, testable increment. Don't implement step N+1 before step N is verified working. Server API changes discovered during client development should feed back into this spec and be implemented as they come up, rather than batched. + +## 14. Out of scope reminders + +If during development any of the following come up, stop and reconsider scope: +- Local audio feature extraction +- Neural network anything +- Adversarial multi-user hardening +- Podcast / video / internet radio +- Custom audio format support beyond what ffmpeg handles natively + +## 15. Open questions to resolve during implementation + +- Exact session vector JSONB schema — finalize after looking at real MusicBrainz tag data. +- Whether to store the `session_vector_at_play` on every play event or only on events that generate contextual likes. Storage cost vs. query simplicity tradeoff. +- Whether `getRandomSongs` from Subsonic should use the full weighted shuffle or a lighter version, since some clients call it on a loop. +- Handling of tracks without MBIDs (a significant fraction of many libraries). They can't participate in similarity-based radio. Decide whether to fall back to artist/genre/tag similarity or exclude them from radio entirely. +- Whether contextual likes should decay over time (old contextual likes become weaker signal). From 419c0b6f4c4d92cf0ebd11ab72795606725f3904 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 17:58:00 +0000 Subject: [PATCH 02/26] docs: seed client spec on dev --- docs/smart-music-client-spec.md | 243 ++++++++++++++++++++++++++++++++ 1 file changed, 243 insertions(+) create mode 100644 docs/smart-music-client-spec.md diff --git a/docs/smart-music-client-spec.md b/docs/smart-music-client-spec.md new file mode 100644 index 00000000..d74f8a86 --- /dev/null +++ b/docs/smart-music-client-spec.md @@ -0,0 +1,243 @@ +# Project Spec: Smart Music Client (companion app) + +## 1. Overview + +A mobile music player client built as the intentional companion to the Smart Music Server (see `smart-music-server-spec.md`). This is not a general-purpose Subsonic client — it is the client that knows how to consume the server's extended API and surface its novel features (contextual likes, session-aware radio, Lidarr integration, server-canonical state) as first-class UX. + +The thesis driving this project, restated: **state and intelligence belong on the server.** The client's job is to be a great playback surface and input device for the server's decisions, not to reimplement them. Compared to existing Subsonic clients (Symfonium, Tempo, Substreamer), this client is thinner in its "brains" and heavier in its integration with a specific server. + +**Scope of this spec:** v1 mobile client. Read the server spec first; this document assumes its terminology and data model. + +## 2. Goals and non-goals + +### Goals + +- Native-feeling mobile app on both iOS and Android from a single codebase. +- Full playback: background audio, lock screen controls, notification controls, Bluetooth/headset button handling, CarPlay/Android Auto *awareness* (not necessarily full integration in v1 — see non-goals). +- Offline cache with explicit user control. +- First-class UX for the dual-like model: clear, fast distinction between "like generally" and "like in this context." +- Real-time session awareness: the client knows what session it's in, shows the user what the server thinks the current context is, and surfaces contextually relevant actions. +- Radio flow that feels like a single gesture (seed → queue → playing) with clear affordances for "keep this vibe," "shift vibe," and "stop." +- Lidarr search and add directly from the player when listening to suggested additions. +- Multi-device consistency: state (likes, play counts, current queue if desired) reflects the canonical server state. Two devices signed in to the same user should see the same world. + +### Non-goals (v1) + +- CarPlay / Android Auto full integration. v1 should not *break* when these launch (lock screen controls must work, basic media session must be present) but dedicated CarPlay UI is deferred. +- Desktop client. The server's web UI covers desktop for v1. +- Cross-server support. This client talks to the Smart Music Server specifically and uses its extended API. Pointing it at a stock Navidrome is not a goal. +- Chromecast / AirPlay. Nice-to-have, not v1. +- Lyrics display. +- Equalizer / DSP. +- Playlist editing UI beyond basic add/remove (full playlist management lives in the server web UI). +- Social features (sharing, following, comments). + +## 3. Technology choices + +- **Framework:** Flutter. Rationale in the server spec conversation; the audio ecosystem (`just_audio`, `audio_service`) is the most mature cross-platform option for this use case. +- **Language:** Dart. +- **State management:** Riverpod. Rationale: testable, compile-time safe, well-suited to the amount of server state this app reflects. Avoid Bloc's boilerplate; avoid Provider's looseness. +- **HTTP:** `dio` with a typed client layer generated from the server's OpenAPI spec if the server publishes one; otherwise hand-written. +- **Local storage:** + - `sqflite` or `drift` for structured local data (cached library, offline play event queue). + - `path_provider` + direct filesystem for cached audio files. + - `flutter_secure_storage` for credentials. +- **Audio:** `just_audio` for playback, `just_audio_background` / `audio_service` for lock screen and background. +- **Navigation:** `go_router`. +- **Packaging:** standard Flutter build pipeline, published as APK/AAB for Android and TestFlight build for iOS. App store publication is out of scope for v1 (sideload / personal use). + +## 4. Architecture + +### Layers + +1. **Transport layer** — wraps the server's Subsonic and extended APIs. Handles auth, retries, connectivity changes. Exposes typed methods to higher layers. +2. **Repository layer** — merges local cache and remote data. Owns the decision of "when do I serve cached data vs. fetch fresh." Exposes streams that UI subscribes to. +3. **Playback engine** — wraps `just_audio`. Owns the queue, position, playback state. Emits events consumed by the session reporter. +4. **Session reporter** — background worker that translates playback engine events into server event API calls (play start, skip, seek, complete). Batches and retries when offline. +5. **Offline cache manager** — owns the disk cache of audio files and cover art. Implements eviction policy, explicit user-pinned downloads. +6. **UI layer** — Flutter widgets consuming Riverpod providers that wrap repositories. + +### Key architectural decisions + +- **Server is source of truth, local is cache.** The client does not hold authoritative state for likes, play counts, or queue membership. All user actions are fire-and-persist-locally, then sync to server. Conflict resolution favors server on write failures (with local retry). +- **Offline mode is first-class.** Every user action — liking, skipping, playing — must work offline and sync later. This means local event queue with retry, and UI that doesn't lie about server state it couldn't confirm. +- **No client-side recommendation logic.** Shuffle, radio, "play similar" all round-trip to the server. If offline, fall back to a simple random-from-cache with a UI indicator that smart features are degraded. +- **Optimistic UI.** Liking a song updates the UI immediately; sync happens in the background. Failed syncs surface as a non-blocking indicator, not a modal. + +### Threading model + +Flutter's main isolate handles UI. Playback and audio service run on their platform-native background threads (handled by `audio_service`). The session reporter and cache manager run as long-lived providers on the main isolate but do their work in async. Heavy operations (e.g. bulk cache cleanup) use `compute()` to offload to a worker isolate. + +## 5. Data flow examples + +### Starting a play session + +1. User taps a track. +2. UI calls playback engine: load queue, start playback. +3. Playback engine emits "track started" event. +4. Session reporter calls `POST /api/v1/events/play` with start metadata. +5. Server creates or updates session, computes session vector, returns session info. +6. Client updates a "current session" provider; UI can display session context (e.g. "tagged: rock, energetic"). +7. As playback progresses, session reporter periodically sends progress; on completion or skip, sends terminal event. + +### Liking a song + +Two buttons in the player UI: **Like** (general) and **Like this vibe** (contextual). Contextual like is only enabled when there is an active session of sufficient length (per server spec, ≥3 tracks in session). + +1. User taps Like this vibe. +2. UI optimistically marks the track as contextually liked in this session; shows a brief toast confirming. +3. Client calls `POST /api/v1/likes/contextual`. The server computes the session vector from its own records (the client does not send it — the server is authoritative). +4. On success, the like is confirmed. On failure, the action is queued for retry; UI shows a subtle indicator. + +### Starting radio + +1. User long-presses a track or taps a "Radio from here" affordance. +2. UI calls `GET /api/v1/radio?seed_track=`. +3. Server returns a queue and optional "suggested additions" (tracks not in library). +4. Playback engine replaces queue; UI renders the queue with suggested additions grouped separately at the bottom with Lidarr-add buttons. +5. As the queue drains, client requests a refresh at 80% consumption. + +### Offline playback + +1. User goes offline. +2. Client continues playback from cached files. +3. Play/skip events accumulate in the local event queue with their timestamps. +4. On reconnect, event queue is flushed to server in order. Server is expected to backfill events with their original timestamps, reconstruct sessions correctly even retroactively. +5. If a user liked a track offline, the like is queued the same way. + +## 6. UI structure + +### Screens + +- **Sign-in** — server URL, username, password. Remember server. Option to save multiple servers (for dev/prod, single household use). +- **Home** — suggested radios, recently played, recent likes, continue listening. All served by server endpoints. +- **Library** — artists, albums, tracks, playlists browse. Subsonic-standard. +- **Search** — searches server. Also offers "search Lidarr for this" affordance on empty results. +- **Now playing** — full-screen player. Primary affordances: play/pause/skip/seek, like general, like contextual, radio-from-here, add-to-playlist, queue view, session inspector. +- **Queue** — current queue with drag-to-reorder. Shows suggested additions below the real queue. +- **Session inspector** — shows the server's current session vector in human-readable form (recent tracks, detected tags, detected artists). Useful for debugging and for user trust in the "why is this playing" question. Accessible from the now-playing screen. +- **Downloads** — pinned/cached content, cache size, eviction controls. +- **Settings** — account, offline behavior, audio quality, cache size, server info, log out. + +### Now-playing affordances (design constraint) + +The now-playing screen is where the novel UX lives. It must make the following feel natural: + +- A single-tap way to like this song generally. +- A single-tap way to like this song in this context. Visually distinct from general like. +- Clear indication of whether either or both likes are active for the current track. +- An obvious but not-dominant "radio from here" action. +- When playback is inside a radio queue, a "keep this vibe" action that pins the current context, and a "shift" action that mutates it. (Exact semantics tunable; start with "keep" = re-seed radio from current track with current session vector, "shift" = re-seed ignoring session vector.) + +The exact visual design is deferred; the spec requires these *capabilities* be reachable without drilling into menus. Buried features won't get used and the contextual like is the whole point. + +### Design constraint: don't lie about server state + +If an action hasn't succeeded on the server yet, the UI may show the optimistic result but must not present it as fully confirmed. A subtle pending indicator on the like button, for instance. Errors that affect user perception of their library (e.g. a like that never synced) must be surfaceable, not silently dropped. + +## 7. Offline cache + +### Two kinds of cached content + +1. **Transient cache** — populated automatically during playback. LRU-evicted. Size-capped per user config (default 2 GB). +2. **Pinned downloads** — user-explicit "download this album / playlist." Not evicted automatically. Pinned content can be browsed in the Downloads screen. + +### Cache decisions + +- Cached files are stored at the audio quality the server was asked to deliver. Quality is chosen per user config (e.g. "always original," "transcode to 192k on cellular"). +- Cover art cached separately, smaller budget. +- Cache survives app reinstall if possible (Android: scoped external storage; iOS: app container — accepting that iOS reinstall loses it). +- User-visible cache size accounting must be accurate. + +### Offline UI + +An offline indicator appears in the top chrome when the client can't reach the server. All server-dependent actions show degraded affordances (e.g. smart shuffle shows "shuffling cached tracks only"). Pinned downloads remain fully usable. + +## 8. API usage + +### Endpoints consumed + +From the server's Subsonic surface: `ping`, `getMusicFolders`, `getIndexes`/`getArtists`, `getArtist`, `getAlbum`, `getAlbumList2`, `search3`, `stream`, `getCoverArt`, `getPlaylists`, `getPlaylist`, `createPlaylist`, `updatePlaylist`, `getRandomSongs`, `star`/`unstar` (for general likes — though the extended API's equivalent is preferred for future-proofing). + +From the server's extended API: all of `/api/v1/*` described in the server spec. Specifically depended on: +- `POST /api/v1/events/play` (with all event types the server supports) +- `POST /api/v1/likes/contextual` +- `DELETE /api/v1/likes/contextual/:id` +- `GET /api/v1/session/current` +- `GET /api/v1/radio` +- `POST /api/v1/lidarr/search` +- `POST /api/v1/lidarr/add` +- `POST /api/v1/quarantine/:track_id` +- `GET /api/v1/stats/user` + +### Auth + +Standard Subsonic auth for Subsonic endpoints (token + salt scheme). Bearer token for extended API. Client holds both; transport layer adds them transparently. + +### Versioning + +Client sends its version in a header on every extended API request. Server may respond with compatibility warnings. Major server API changes bump a version the client checks on startup. + +## 9. Reliability + +### Event queue durability + +Play/skip/like events are written to local SQL before any network attempt. The sync worker drains the queue FIFO with exponential backoff on failures. Events older than 30 days in the queue without successful sync are logged and dropped (with user visibility — a "some events failed to sync" notification). + +### Playback robustness + +- Network interruption during streaming: if the track is partially buffered, finish it from buffer; on exhaustion, pause and show offline state. +- Track not found (deleted between queue build and playback): skip forward, log, don't crash. +- Corrupt cache file: purge it, redownload or skip. + +### Battery and data + +- Don't prefetch aggressively on cellular unless user opts in. +- Audio session must release properly when playback ends; no background drain. + +## 10. Testing + +- Widget tests for the novel UI components (like buttons, session inspector, radio queue). +- Unit tests for the event queue and sync logic — this is where silent bugs lose user trust. +- Integration tests against a local server instance for the full happy-path flows. +- Manual test matrix documented: playback across lock/unlock, app backgrounded, headphone disconnect, network loss mid-stream, receive call mid-stream. + +## 11. Implementation order + +The client should not start before server step 4 (usable Subsonic baseline). Once that milestone is reached, client work proceeds in parallel: + +1. Project skeleton, Riverpod setup, routing, theme. +2. Auth + sign-in screen. Talk to `ping`. Store credentials securely. +3. Library browse: artists, albums, tracks. Read-only against Subsonic API. +4. Playback engine: `just_audio` integration, queue, background audio, lock screen. **Milestone: can play music from the server.** +5. Event ingestion against server (requires server step 5 complete): play start, complete, skip events wired to server. +6. Like buttons (general) wired to Subsonic `star`. +7. Contextual like button + session inspector (requires server step 8). +8. Radio flow (requires server step 11). +9. Lidarr search/add integration (requires server step 12). +10. Offline cache manager. +11. Pinned downloads. +12. Quarantine action from player. +13. Settings polish, stats screen. +14. Packaging, release pipeline. + +Steps 5, 7, 8, 9 gate on server features. Plan accordingly; don't block client work on those, sequence around them. + +## 12. Open questions to resolve during implementation + +- Whether queue state should itself be server-synced (so starting playback on one device resumes on another). Technically appealing but a significant complication — may defer to v2. +- Whether to offer a "this song is playing because of these contextual likes" explanation affordance. Requires server API to expose the reasoning, and it isn't currently in the server spec. Worth considering adding. +- Exact wording and iconography for general vs. contextual like. This needs user testing (even if the user is just you). +- Whether offline-created contextual likes are a good idea at all — the server is the authority on session vectors, and the client's view of a session may drift offline. Possibly contextual likes require connectivity while general likes do not. +- How aggressive to be about prefetching the radio queue in advance for offline continuation. + +## 13. Out of scope reminders + +If during development any of the following come up, stop and reconsider scope: +- CarPlay / Android Auto native UI +- Chromecast / AirPlay +- Lyrics +- EQ / DSP +- Cross-server support +- Desktop port +- Social / sharing From e1bfbaa3ea856e32b27cd3dd01f6a59a5ac55119 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 18:05:38 +0000 Subject: [PATCH 03/26] docs: expand README with spec links and dev workflow --- README.md | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 983032a7..0fec0d42 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,27 @@ -# minstrel +# Minstrel -Self-hosted music server with OpenSubsonic compatibility, server-side smart shuffle, dual-like model, session-aware radio, and Lidarr integration. Go + Postgres. \ No newline at end of file +Self-hosted music server with OpenSubsonic compatibility plus an extended API for server-side smart shuffle, a dual-like model (general + contextual), session-aware radio, ListenBrainz scrobble/similarity, and Lidarr integration. Go + Postgres, packaged as a Docker container. + +> State and intelligence belong on the server, not the client. + +## Specs + +Authoritative v1 scope lives under [`docs/`](./docs): + +- [Server spec](./docs/smart-music-server-spec.md) — current implementation focus. +- [Client spec](./docs/smart-music-client-spec.md) — Flutter companion app; work starts once the server reaches its Subsonic-compatible baseline. + +## Development workflow + +- Day-to-day work happens on the `dev` branch (or feature branches merged into `dev`). +- `main` is **protected** — changes land only via pull request from `dev`. +- Production builds are cut by tagging a release (`v*`) off `main`. + +## CI / container image + +Forgejo Actions handles: + +- Tests on push to `dev` and on PRs into `main`. +- Container image build + push to the Forgejo registry on merge to `main` (`:main`) and on `v*` tags (`:`). + +Task and milestone tracking: Fable (`Minstrel` project, id 12). From fba136653773afb13f2856ac1d432a12e95c801a Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 18:07:50 +0000 Subject: [PATCH 04/26] ci: add Forgejo Actions test workflow with Postgres service --- .forgejo/workflows/test.yml | 73 +++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100644 .forgejo/workflows/test.yml diff --git a/.forgejo/workflows/test.yml b/.forgejo/workflows/test.yml new file mode 100644 index 00000000..9e61b375 --- /dev/null +++ b/.forgejo/workflows/test.yml @@ -0,0 +1,73 @@ +name: test + +on: + push: + branches: [dev] + pull_request: + branches: [main] + +jobs: + test: + runs-on: ubuntu-latest + + services: + postgres: + image: postgres:16-alpine + env: + POSTGRES_USER: minstrel + POSTGRES_PASSWORD: minstrel + POSTGRES_DB: minstrel_test + ports: + - 5432:5432 + options: >- + --health-cmd "pg_isready -U minstrel" + --health-interval 10s + --health-timeout 5s + --health-retries 10 + + env: + MINSTREL_TEST_DATABASE_URL: postgres://minstrel:minstrel@localhost:5432/minstrel_test?sslmode=disable + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Detect Go project + id: gomod + shell: bash + run: | + if [ -f go.mod ]; then + echo "present=true" >> "$GITHUB_OUTPUT" + else + echo "present=false" >> "$GITHUB_OUTPUT" + echo "::notice::No go.mod yet — Go steps will be skipped until the skeleton lands" + fi + + - name: Set up Go + if: steps.gomod.outputs.present == 'true' + uses: actions/setup-go@v5 + with: + go-version-file: go.mod + cache: true + + - name: go vet + if: steps.gomod.outputs.present == 'true' + run: go vet ./... + + - name: golangci-lint + if: steps.gomod.outputs.present == 'true' + uses: golangci/golangci-lint-action@v6 + with: + version: latest + + - name: go test (race + coverage) + if: steps.gomod.outputs.present == 'true' + run: go test -race -coverprofile=coverage.out ./... + + - name: Upload coverage artifact + if: steps.gomod.outputs.present == 'true' + uses: actions/upload-artifact@v4 + with: + name: coverage + path: coverage.out + if-no-files-found: ignore From 078e6e989b78dd3587c6005bb70d8a3eab7b6a19 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 18:17:16 +0000 Subject: [PATCH 05/26] ci: retarget test workflow to go-ci runner label --- .forgejo/workflows/test.yml | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/.forgejo/workflows/test.yml b/.forgejo/workflows/test.yml index 9e61b375..f440c885 100644 --- a/.forgejo/workflows/test.yml +++ b/.forgejo/workflows/test.yml @@ -8,7 +8,7 @@ on: jobs: test: - runs-on: ubuntu-latest + runs-on: go-ci services: postgres: @@ -26,7 +26,7 @@ jobs: --health-retries 10 env: - MINSTREL_TEST_DATABASE_URL: postgres://minstrel:minstrel@localhost:5432/minstrel_test?sslmode=disable + MINSTREL_TEST_DATABASE_URL: postgres://minstrel:minstrel@postgres:5432/minstrel_test?sslmode=disable steps: - name: Checkout @@ -43,12 +43,11 @@ jobs: echo "::notice::No go.mod yet — Go steps will be skipped until the skeleton lands" fi - - name: Set up Go + - name: Toolchain versions if: steps.gomod.outputs.present == 'true' - uses: actions/setup-go@v5 - with: - go-version-file: go.mod - cache: true + run: | + go version + golangci-lint --version - name: go vet if: steps.gomod.outputs.present == 'true' @@ -56,9 +55,7 @@ jobs: - name: golangci-lint if: steps.gomod.outputs.present == 'true' - uses: golangci/golangci-lint-action@v6 - with: - version: latest + run: golangci-lint run ./... - name: go test (race + coverage) if: steps.gomod.outputs.present == 'true' From fa5a24140be2cc6a17b7b7666178db67f4e8c586 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 18:50:31 +0000 Subject: [PATCH 06/26] ci: slim test workflow to lint + unit tests, drop Postgres service --- .forgejo/workflows/test.yml | 30 ++---------------------------- 1 file changed, 2 insertions(+), 28 deletions(-) diff --git a/.forgejo/workflows/test.yml b/.forgejo/workflows/test.yml index f440c885..1a7a6f89 100644 --- a/.forgejo/workflows/test.yml +++ b/.forgejo/workflows/test.yml @@ -10,24 +10,6 @@ jobs: test: runs-on: go-ci - services: - postgres: - image: postgres:16-alpine - env: - POSTGRES_USER: minstrel - POSTGRES_PASSWORD: minstrel - POSTGRES_DB: minstrel_test - ports: - - 5432:5432 - options: >- - --health-cmd "pg_isready -U minstrel" - --health-interval 10s - --health-timeout 5s - --health-retries 10 - - env: - MINSTREL_TEST_DATABASE_URL: postgres://minstrel:minstrel@postgres:5432/minstrel_test?sslmode=disable - steps: - name: Checkout uses: actions/checkout@v4 @@ -57,14 +39,6 @@ jobs: if: steps.gomod.outputs.present == 'true' run: golangci-lint run ./... - - name: go test (race + coverage) + - name: go test (short, race) if: steps.gomod.outputs.present == 'true' - run: go test -race -coverprofile=coverage.out ./... - - - name: Upload coverage artifact - if: steps.gomod.outputs.present == 'true' - uses: actions/upload-artifact@v4 - with: - name: coverage - path: coverage.out - if-no-files-found: ignore + run: go test -short -race ./... From 83d71f670302a82075eb93d92076f5d7908bdf2e Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 19:07:30 +0000 Subject: [PATCH 07/26] ci: re-trigger test workflow against rebuilt go-ci image --- .forgejo/workflows/test.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.forgejo/workflows/test.yml b/.forgejo/workflows/test.yml index 1a7a6f89..fb057905 100644 --- a/.forgejo/workflows/test.yml +++ b/.forgejo/workflows/test.yml @@ -1,5 +1,8 @@ name: test +# Lint + short unit tests only. Integration tests needing Postgres/ffmpeg +# run locally via docker-compose; they should guard with testing.Short(). + on: push: branches: [dev] From 8b9f57128a133069fc37837bd012838ce9fb849b Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 19:21:38 +0000 Subject: [PATCH 08/26] ci: add release workflow for image build + push on main and v* tags --- .forgejo/workflows/release.yml | 60 ++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 .forgejo/workflows/release.yml diff --git a/.forgejo/workflows/release.yml b/.forgejo/workflows/release.yml new file mode 100644 index 00000000..66ec2587 --- /dev/null +++ b/.forgejo/workflows/release.yml @@ -0,0 +1,60 @@ +name: release + +# Builds and pushes the minstrel container image to the Forgejo registry. +# +# push to main → :main +# push tag vX.Y.Z → :vX.Y.Z and :latest +# +# No-ops cleanly until the server skeleton (go.mod + Dockerfile) lands. + +on: + push: + branches: [main] + tags: ['v*'] + +jobs: + release: + runs-on: go-ci + + env: + IMAGE: git.fabledsword.com/bvandeusen/minstrel + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Detect buildable project + id: guard + shell: bash + run: | + if [ -f Dockerfile ] && [ -f go.mod ]; then + echo "ready=true" >> "$GITHUB_OUTPUT" + else + echo "ready=false" >> "$GITHUB_OUTPUT" + echo "::notice::No Dockerfile + go.mod yet — release build skipped" + fi + + - name: Compute image tags + id: tags + if: steps.guard.outputs.ready == 'true' + shell: bash + run: | + if [[ "${GITHUB_REF}" == refs/tags/v* ]]; then + VERSION="${GITHUB_REF#refs/tags/}" + echo "args=-t ${IMAGE}:${VERSION} -t ${IMAGE}:latest" >> "$GITHUB_OUTPUT" + echo "::notice::Release build: ${VERSION} + latest" + else + echo "args=-t ${IMAGE}:main" >> "$GITHUB_OUTPUT" + echo "::notice::Main-branch build: :main" + fi + + - name: Registry login + if: steps.guard.outputs.ready == 'true' + shell: bash + run: | + echo "${{ secrets.GITHUB_TOKEN }}" \ + | docker login git.fabledsword.com -u "${{ github.actor }}" --password-stdin + + - name: Build and push + if: steps.guard.outputs.ready == 'true' + run: docker buildx build --push ${{ steps.tags.outputs.args }} . From 417e3d05dd4ee549b0779db510b86d084b2a9492 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:19:17 +0000 Subject: [PATCH 09/26] feat(skel): add go module definition Initial module at git.fabledsword.com/bvandeusen/minstrel pinned to go 1.23.0 to match the runner-base:go-ci image. Dependencies kept minimal (chi router, yaml.v3) so CI's first run is fast and contained. --- go.mod | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 go.mod diff --git a/go.mod b/go.mod new file mode 100644 index 00000000..ed209b7d --- /dev/null +++ b/go.mod @@ -0,0 +1,8 @@ +module git.fabledsword.com/bvandeusen/minstrel + +go 1.23.0 + +require ( + github.com/go-chi/chi/v5 v5.2.5 + gopkg.in/yaml.v3 v3.0.1 +) From f98b9f66aafe876934feb2e4fc7ecfef2d3d7b5c Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:19:26 +0000 Subject: [PATCH 10/26] feat(skel): add go.sum for initial dependency set --- go.sum | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 go.sum diff --git a/go.sum b/go.sum new file mode 100644 index 00000000..0c0854f2 --- /dev/null +++ b/go.sum @@ -0,0 +1,6 @@ +github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug= +github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= From be181f82453427041d760c1bb14b74166ebb8a34 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:19:39 +0000 Subject: [PATCH 11/26] feat(skel): add cmd/minstrel entrypoint Parses --config / SMARTMUSIC_CONFIG, loads YAML config with env overlay, initializes slog, starts the chi HTTP server, and shuts down cleanly on SIGINT/SIGTERM. --- cmd/minstrel/main.go | 81 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 81 insertions(+) create mode 100644 cmd/minstrel/main.go diff --git a/cmd/minstrel/main.go b/cmd/minstrel/main.go new file mode 100644 index 00000000..4de77cc6 --- /dev/null +++ b/cmd/minstrel/main.go @@ -0,0 +1,81 @@ +package main + +import ( + "context" + "errors" + "flag" + "fmt" + "net/http" + "os" + "os/signal" + "syscall" + "time" + + "git.fabledsword.com/bvandeusen/minstrel/internal/config" + "git.fabledsword.com/bvandeusen/minstrel/internal/logging" + "git.fabledsword.com/bvandeusen/minstrel/internal/server" +) + +func main() { + if err := run(); err != nil { + fmt.Fprintf(os.Stderr, "minstrel: %v\n", err) + os.Exit(1) + } +} + +func run() error { + configPath := flag.String("config", os.Getenv("SMARTMUSIC_CONFIG"), "path to YAML config file") + flag.Parse() + + cfg, err := config.Load(*configPath) + if err != nil { + return fmt.Errorf("load config: %w", err) + } + + logger, err := logging.New(os.Stdout, cfg.Log.Level, cfg.Log.Format) + if err != nil { + return fmt.Errorf("init logger: %w", err) + } + + logger.Info("minstrel starting", + "address", cfg.Server.Address, + "log_level", cfg.Log.Level, + "log_format", cfg.Log.Format, + ) + + srv := server.New(logger) + httpServer := &http.Server{ + Addr: cfg.Server.Address, + Handler: srv.Router(), + ReadHeaderTimeout: 10 * time.Second, + } + + ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) + defer stop() + + errCh := make(chan error, 1) + go func() { + logger.Info("listening", "address", cfg.Server.Address) + if err := httpServer.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) { + errCh <- err + } + close(errCh) + }() + + select { + case <-ctx.Done(): + logger.Info("shutdown signal received") + case err := <-errCh: + if err != nil { + return fmt.Errorf("server: %w", err) + } + } + + shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + if err := httpServer.Shutdown(shutdownCtx); err != nil { + return fmt.Errorf("shutdown: %w", err) + } + logger.Info("minstrel stopped") + return nil +} From d2d8612e06e1fcf14a7993da5b5573196fd43c7e Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:19:50 +0000 Subject: [PATCH 12/26] feat(skel): config loader with YAML + SMARTMUSIC_* env overlay --- internal/config/config.go | 67 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) create mode 100644 internal/config/config.go diff --git a/internal/config/config.go b/internal/config/config.go new file mode 100644 index 00000000..4f69369e --- /dev/null +++ b/internal/config/config.go @@ -0,0 +1,67 @@ +package config + +import ( + "fmt" + "os" + "strings" + + "gopkg.in/yaml.v3" +) + +type Config struct { + Server ServerConfig `yaml:"server"` + Database DatabaseConfig `yaml:"database"` + Log LogConfig `yaml:"log"` +} + +type ServerConfig struct { + Address string `yaml:"address"` +} + +type DatabaseConfig struct { + URL string `yaml:"url"` +} + +type LogConfig struct { + Level string `yaml:"level"` + Format string `yaml:"format"` +} + +func Default() Config { + return Config{ + Server: ServerConfig{Address: ":4533"}, + Database: DatabaseConfig{URL: ""}, + Log: LogConfig{Level: "info", Format: "text"}, + } +} + +func Load(path string) (Config, error) { + cfg := Default() + if path != "" { + data, err := os.ReadFile(path) + if err != nil { + if !os.IsNotExist(err) { + return cfg, fmt.Errorf("read config %q: %w", path, err) + } + } else if err := yaml.Unmarshal(data, &cfg); err != nil { + return cfg, fmt.Errorf("parse config %q: %w", path, err) + } + } + applyEnv(&cfg) + return cfg, nil +} + +func applyEnv(cfg *Config) { + if v, ok := os.LookupEnv("SMARTMUSIC_SERVER_ADDRESS"); ok { + cfg.Server.Address = v + } + if v, ok := os.LookupEnv("SMARTMUSIC_DATABASE_URL"); ok { + cfg.Database.URL = v + } + if v, ok := os.LookupEnv("SMARTMUSIC_LOG_LEVEL"); ok { + cfg.Log.Level = strings.ToLower(v) + } + if v, ok := os.LookupEnv("SMARTMUSIC_LOG_FORMAT"); ok { + cfg.Log.Format = strings.ToLower(v) + } +} From a29d876ac42e69b54bbe9eba23c3d88c684a2985 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:20:02 +0000 Subject: [PATCH 13/26] test(config): cover defaults, YAML load, missing file, env overrides --- internal/config/config_test.go | 80 ++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 internal/config/config_test.go diff --git a/internal/config/config_test.go b/internal/config/config_test.go new file mode 100644 index 00000000..2466f545 --- /dev/null +++ b/internal/config/config_test.go @@ -0,0 +1,80 @@ +package config + +import ( + "os" + "path/filepath" + "testing" +) + +func TestDefault(t *testing.T) { + cfg := Default() + if cfg.Server.Address != ":4533" { + t.Errorf("default server address = %q, want :4533", cfg.Server.Address) + } + if cfg.Log.Level != "info" { + t.Errorf("default log level = %q, want info", cfg.Log.Level) + } +} + +func TestLoadYAML(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "config.yaml") + content := []byte(`server: + address: ":9000" +database: + url: "postgres://example" +log: + level: "debug" + format: "json" +`) + if err := os.WriteFile(path, content, 0o644); err != nil { + t.Fatal(err) + } + cfg, err := Load(path) + if err != nil { + t.Fatalf("Load: %v", err) + } + if cfg.Server.Address != ":9000" { + t.Errorf("server.address = %q", cfg.Server.Address) + } + if cfg.Database.URL != "postgres://example" { + t.Errorf("database.url = %q", cfg.Database.URL) + } + if cfg.Log.Level != "debug" || cfg.Log.Format != "json" { + t.Errorf("log = %+v", cfg.Log) + } +} + +func TestLoadMissingFileReturnsDefaults(t *testing.T) { + cfg, err := Load(filepath.Join(t.TempDir(), "does-not-exist.yaml")) + if err != nil { + t.Fatalf("Load: %v", err) + } + if cfg.Server.Address != ":4533" { + t.Errorf("expected defaults, got %+v", cfg) + } +} + +func TestEnvOverrides(t *testing.T) { + t.Setenv("SMARTMUSIC_SERVER_ADDRESS", ":8080") + t.Setenv("SMARTMUSIC_DATABASE_URL", "postgres://env") + t.Setenv("SMARTMUSIC_LOG_LEVEL", "WARN") + t.Setenv("SMARTMUSIC_LOG_FORMAT", "JSON") + + cfg, err := Load("") + if err != nil { + t.Fatalf("Load: %v", err) + } + if cfg.Server.Address != ":8080" { + t.Errorf("server.address = %q", cfg.Server.Address) + } + if cfg.Database.URL != "postgres://env" { + t.Errorf("database.url = %q", cfg.Database.URL) + } + if cfg.Log.Level != "warn" { + t.Errorf("log.level = %q, want lowercased warn", cfg.Log.Level) + } + if cfg.Log.Format != "json" { + t.Errorf("log.format = %q, want lowercased json", cfg.Log.Format) + } +} From d206f9281f70c7017deaaab390a865f83e490bdb Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:20:10 +0000 Subject: [PATCH 14/26] feat(skel): slog logger factory with text|json format and level parsing --- internal/logging/logging.go | 42 +++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 internal/logging/logging.go diff --git a/internal/logging/logging.go b/internal/logging/logging.go new file mode 100644 index 00000000..d0561466 --- /dev/null +++ b/internal/logging/logging.go @@ -0,0 +1,42 @@ +package logging + +import ( + "fmt" + "io" + "log/slog" + "strings" +) + +func New(w io.Writer, level, format string) (*slog.Logger, error) { + lvl, err := parseLevel(level) + if err != nil { + return nil, err + } + opts := &slog.HandlerOptions{Level: lvl} + + var h slog.Handler + switch strings.ToLower(format) { + case "", "text": + h = slog.NewTextHandler(w, opts) + case "json": + h = slog.NewJSONHandler(w, opts) + default: + return nil, fmt.Errorf("unknown log format %q (want text|json)", format) + } + return slog.New(h), nil +} + +func parseLevel(s string) (slog.Level, error) { + switch strings.ToLower(strings.TrimSpace(s)) { + case "", "info": + return slog.LevelInfo, nil + case "debug": + return slog.LevelDebug, nil + case "warn", "warning": + return slog.LevelWarn, nil + case "error": + return slog.LevelError, nil + default: + return 0, fmt.Errorf("unknown log level %q", s) + } +} From 16f2fc2ce4cbd43800f858f831dd9ea0d1f8fdfe Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:20:17 +0000 Subject: [PATCH 15/26] feat(skel): chi router with RequestID/Recoverer middleware and /healthz --- internal/server/server.go | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 internal/server/server.go diff --git a/internal/server/server.go b/internal/server/server.go new file mode 100644 index 00000000..ff2aec2c --- /dev/null +++ b/internal/server/server.go @@ -0,0 +1,33 @@ +package server + +import ( + "encoding/json" + "log/slog" + "net/http" + + "github.com/go-chi/chi/v5" + "github.com/go-chi/chi/v5/middleware" +) + +type Server struct { + Logger *slog.Logger +} + +func New(logger *slog.Logger) *Server { + return &Server{Logger: logger} +} + +func (s *Server) Router() http.Handler { + r := chi.NewRouter() + r.Use(middleware.RequestID) + r.Use(middleware.Recoverer) + + r.Get("/healthz", s.handleHealthz) + return r +} + +func (s *Server) handleHealthz(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + _ = json.NewEncoder(w).Encode(map[string]string{"status": "ok"}) +} From 4dbf02d645b47ad4a35308e656202324f905e206 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:20:23 +0000 Subject: [PATCH 16/26] test(server): verify /healthz returns 200 with {"status":"ok"} --- internal/server/server_test.go | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 internal/server/server_test.go diff --git a/internal/server/server_test.go b/internal/server/server_test.go new file mode 100644 index 00000000..877b2a87 --- /dev/null +++ b/internal/server/server_test.go @@ -0,0 +1,33 @@ +package server + +import ( + "encoding/json" + "io" + "log/slog" + "net/http" + "net/http/httptest" + "testing" +) + +func TestHealthz(t *testing.T) { + s := New(slog.New(slog.NewTextHandler(io.Discard, nil))) + ts := httptest.NewServer(s.Router()) + defer ts.Close() + + resp, err := http.Get(ts.URL + "/healthz") + if err != nil { + t.Fatalf("GET /healthz: %v", err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + t.Errorf("status = %d, want 200", resp.StatusCode) + } + var body map[string]string + if err := json.NewDecoder(resp.Body).Decode(&body); err != nil { + t.Fatalf("decode: %v", err) + } + if body["status"] != "ok" { + t.Errorf("body = %v, want status=ok", body) + } +} From 58fa790b2b49f5b337a5f82f0e630628df3eb948 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:20:34 +0000 Subject: [PATCH 17/26] feat(skel): db package with embedded migrations stub MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Establishes the embed.FS and migrations/ layout up front. Migrate() returns ErrNotConfigured intentionally — the real goose runner lands in M2 alongside Postgres work, avoiding a toolchain bump to Go 1.24/1.25 through goose's transitive modernc.org/libc dependency chain. --- internal/db/db.go | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 internal/db/db.go diff --git a/internal/db/db.go b/internal/db/db.go new file mode 100644 index 00000000..587aab69 --- /dev/null +++ b/internal/db/db.go @@ -0,0 +1,33 @@ +package db + +import ( + "embed" + "errors" +) + +//go:embed migrations/*.sql +var migrationsFS embed.FS + +// MigrationsFS exposes the embedded migration files. The concrete migration +// runner (goose) will be wired in M2 when Postgres work begins; this package +// intentionally keeps the embed + a placeholder surface so the file layout +// and `go:embed` directive are established from day one. +func MigrationsFS() embed.FS { + return migrationsFS +} + +// ErrNotConfigured is returned by Migrate until the real runner is wired. +var ErrNotConfigured = errors.New("db: migrations not yet configured (wired in M2)") + +// Migrate is a placeholder. It validates that the embedded migrations load +// and then returns ErrNotConfigured so callers can detect the stub. +func Migrate() error { + entries, err := migrationsFS.ReadDir("migrations") + if err != nil { + return err + } + if len(entries) == 0 { + return errors.New("db: no migration files found") + } + return ErrNotConfigured +} From 9b9e0e40115769e0c0134de0c37699bc50f3d1d5 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:20:40 +0000 Subject: [PATCH 18/26] test(db): verify migrations stub loads embedded files and returns ErrNotConfigured --- internal/db/db_test.go | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 internal/db/db_test.go diff --git a/internal/db/db_test.go b/internal/db/db_test.go new file mode 100644 index 00000000..bac5ae34 --- /dev/null +++ b/internal/db/db_test.go @@ -0,0 +1,13 @@ +package db + +import ( + "errors" + "testing" +) + +func TestMigrateStubLoadsEmbeddedFiles(t *testing.T) { + err := Migrate() + if !errors.Is(err, ErrNotConfigured) { + t.Errorf("Migrate stub err = %v, want ErrNotConfigured", err) + } +} From ff7cd8d38b4823e1d83065eed2281e79db79adf9 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:20:45 +0000 Subject: [PATCH 19/26] feat(skel): placeholder migration so embed.FS has a file to index --- internal/db/migrations/00001_placeholder.sql | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 internal/db/migrations/00001_placeholder.sql diff --git a/internal/db/migrations/00001_placeholder.sql b/internal/db/migrations/00001_placeholder.sql new file mode 100644 index 00000000..28618351 --- /dev/null +++ b/internal/db/migrations/00001_placeholder.sql @@ -0,0 +1,9 @@ +-- +goose Up +-- +goose StatementBegin +SELECT 1; +-- +goose StatementEnd + +-- +goose Down +-- +goose StatementBegin +SELECT 1; +-- +goose StatementEnd From 70776141164c7a92328bbe3c1182c02b2d19839c Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:20:54 +0000 Subject: [PATCH 20/26] feat(skel): multi-stage Dockerfile for minstrel runtime MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Builder uses golang:1.23-bookworm (matches runner-base:go-ci). Runtime is debian:bookworm-slim with ffmpeg (server spec §3) plus a non-root minstrel user. Release workflow builds from this Dockerfile. --- Dockerfile | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 Dockerfile diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 00000000..ed61cdc7 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,25 @@ +# syntax=docker/dockerfile:1.6 + +FROM golang:1.23-bookworm AS builder +WORKDIR /src +COPY go.mod go.sum ./ +RUN go mod download +COPY . . +ENV CGO_ENABLED=0 +RUN go build -trimpath -ldflags="-s -w" -o /out/minstrel ./cmd/minstrel + +FROM debian:bookworm-slim +RUN apt-get update \ + && apt-get install -y --no-install-recommends ca-certificates ffmpeg \ + && rm -rf /var/lib/apt/lists/* + +RUN groupadd --system --gid 1000 minstrel \ + && useradd --system --uid 1000 --gid minstrel --shell /usr/sbin/nologin minstrel + +COPY --from=builder /out/minstrel /usr/local/bin/minstrel +COPY config.example.yaml /etc/smartmusic/config.yaml + +USER minstrel +EXPOSE 4533 +ENTRYPOINT ["/usr/local/bin/minstrel"] +CMD ["--config", "/etc/smartmusic/config.yaml"] From b40e379b84db0c372fba9b98332bd9f0ce764d46 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:21:01 +0000 Subject: [PATCH 21/26] feat(skel): docker-compose for local Postgres (integration-test stack) --- docker-compose.yml | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 docker-compose.yml diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 00000000..aaa796fb --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,26 @@ +version: "3.9" + +# Local development environment for Minstrel. +# Not used by CI — integration tests run against this compose stack manually: +# docker compose up -d postgres +# go test ./... # runs full tests (no -short flag) + +services: + postgres: + image: postgres:16-alpine + environment: + POSTGRES_USER: minstrel + POSTGRES_PASSWORD: minstrel + POSTGRES_DB: minstrel + ports: + - "5432:5432" + volumes: + - minstrel-pgdata:/var/lib/postgresql/data + healthcheck: + test: ["CMD-SHELL", "pg_isready -U minstrel -d minstrel"] + interval: 5s + timeout: 5s + retries: 10 + +volumes: + minstrel-pgdata: From 70c89bb7dfa78322b52c6107e3b6dcddd1bd4636 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:21:08 +0000 Subject: [PATCH 22/26] feat(skel): example YAML config mirroring Default() --- config.example.yaml | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 config.example.yaml diff --git a/config.example.yaml b/config.example.yaml new file mode 100644 index 00000000..2b21f2a3 --- /dev/null +++ b/config.example.yaml @@ -0,0 +1,15 @@ +# Minstrel example configuration. +# Every field is optional; defaults shown below match Default() in internal/config. +# Each setting can be overridden at runtime via SMARTMUSIC_* environment variables +# (see internal/config/config.go for the full list). + +server: + address: ":4533" + +database: + # Postgres connection string, e.g. postgres://minstrel:minstrel@localhost:5432/minstrel?sslmode=disable + url: "" + +log: + level: "info" # debug | info | warn | error + format: "text" # text | json From 9bce7fee1012845585d28462dc9092eee4766c9c Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:21:14 +0000 Subject: [PATCH 23/26] ci: add .golangci.yml config for CI lint step --- .golangci.yml | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 .golangci.yml diff --git a/.golangci.yml b/.golangci.yml new file mode 100644 index 00000000..7b87777e --- /dev/null +++ b/.golangci.yml @@ -0,0 +1,25 @@ +run: + timeout: 5m + tests: true + +linters: + disable-all: true + enable: + - errcheck + - govet + - ineffassign + - staticcheck + - unused + - gofmt + - goimports + - revive + +linters-settings: + revive: + rules: + - name: var-naming + - name: exported + arguments: ["checkPrivateReceivers", "sayRepetitiveInsteadOfStutters"] + +issues: + exclude-use-default: false From 743fbe9d5c151e4af3af7c010b3e31ff2f9b5f0c Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:47:40 +0000 Subject: [PATCH 24/26] ci(lint): drop revive `exported` rule; require no doc-comment boilerplate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Revive's `exported` check forced doc comments on every exported symbol — that conflicts with the project's policy of only writing comments when the "why" is non-obvious. Keep `var-naming`, `unused-parameter`, and `early-return` for signal that doesn't mandate prose. --- .golangci.yml | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/.golangci.yml b/.golangci.yml index 7b87777e..f89576e3 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -16,10 +16,13 @@ linters: linters-settings: revive: + # Intentionally narrow: we skip `exported` (no doc-comment requirement) per + # the project's no-boilerplate-comment policy. Re-enable if the public API + # surface grows to the point where documentation lives alongside it. rules: - name: var-naming - - name: exported - arguments: ["checkPrivateReceivers", "sayRepetitiveInsteadOfStutters"] + - name: unused-parameter + - name: early-return issues: exclude-use-default: false From 4c5b4d1790dbd73309fd1137c6c43f6f6f0f5c09 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:47:52 +0000 Subject: [PATCH 25/26] fix(server): rename unused request param to _ for revive/unused-parameter --- internal/server/server.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/internal/server/server.go b/internal/server/server.go index ff2aec2c..71bfa3eb 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -26,7 +26,7 @@ func (s *Server) Router() http.Handler { return r } -func (s *Server) handleHealthz(w http.ResponseWriter, r *http.Request) { +func (s *Server) handleHealthz(w http.ResponseWriter, _ *http.Request) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(http.StatusOK) _ = json.NewEncoder(w).Encode(map[string]string{"status": "ok"}) From 8fbf3553059d2e0a0ec01e28fbd24d248361dd79 Mon Sep 17 00:00:00 2001 From: bvandeusen Date: Sat, 18 Apr 2026 21:48:07 +0000 Subject: [PATCH 26/26] fix(server): silence errcheck on deferred Body.Close --- internal/server/server_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/internal/server/server_test.go b/internal/server/server_test.go index 877b2a87..3e49da77 100644 --- a/internal/server/server_test.go +++ b/internal/server/server_test.go @@ -18,7 +18,7 @@ func TestHealthz(t *testing.T) { if err != nil { t.Fatalf("GET /healthz: %v", err) } - defer resp.Body.Close() + defer func() { _ = resp.Body.Close() }() if resp.StatusCode != http.StatusOK { t.Errorf("status = %d, want 200", resp.StatusCode)