feat: M4a outbound ListenBrainz scrobble worker #26

2026-04-28T17:53:58-04:00

bvandeusen commented

2026-04-28 17:53:58 -04:00

(Migrated from git.fabledsword.com)

First M4 sub-plan (Fable #345). Sends qualifying user plays to ListenBrainz with batching + exponential-backoff retry. Per-user token, plaintext storage, pull-based 30s timer worker, patient backoff (1m → 5m → 30m → 2h → 6h, 5 attempts).

What this ships

Schema (migration `0008_scrobble`)

users.listenbrainz_token TEXT NULL + users.listenbrainz_enabled BOOLEAN DEFAULT FALSE — per-user config columns
scrobble_queue table: (id, user_id, play_event_id, status ∈ {pending, failed}, attempts, next_attempt_at, last_error, enqueued_at) with UNIQUE(play_event_id) for idempotency and a partial index WHERE status='pending' for the worker's hot path
Lifecycle: successful submit DELETEs the queue row + stamps play_events.scrobbled_at (canonical record). Only pending and failed states ever exist in the queue.

Backend pipeline (`internal/scrobble/`)

listenbrainz/client.go — pure HTTP client for /1/submit-listens. Typed errors ErrAuth (401), ErrPermanent (other 4xx), ErrTransient (5xx + network), *RetryAfterError (429 with parsed Retry-After header). 9 httptest-driven tests including payload shape, auth header, and listen_type=single|import switching.
threshold.go — Qualifies(durationPlayedMs, completionRatio) returning true when ≥240 seconds played OR ≥50% of track length. 6 boundary tests.
queue.go::MaybeEnqueue — gates on user config + threshold + idempotent INSERT (ON CONFLICT DO NOTHING). 5 integration tests.
worker.go — Worker struct with Run (30s ticker), tickOnce (drains up to 50 rows, groups per-user, batches POST), and processBatch (handles 6 outcome paths — success, 429-retry-after, 401-disable-user, ErrPermanent, ErrTransient-with-backoff, ErrTransient-give-up). Backoff schedule 1m → 5m → 30m → 2h → 6h, then permanent fail. 6 integration tests.

Hooks (`internal/playevents/writer.go`)

RecordPlayEnded and RecordSyntheticCompletedPlay (Subsonic submission=true) both gain a post-commit scrobble.MaybeEnqueue call. Best-effort: errors logged + swallowed (matches the M3 CaptureContextualLikeIfPlaying pattern). 2 new integration tests.

API (`internal/api/listenbrainz.go`)

GET /api/me/listenbrainz → {enabled, token_set, last_scrobbled_at}. Token write-only — token_set is the only signal back about whether one exists. last_scrobbled_at queried via MAX(play_events.scrobbled_at).
PUT /api/me/listenbrainz body {token?, enabled?}. Clearing token ({token: ""}) auto-forces enabled=false via SQL CASE. Enabling without a token returns 400.
4 integration tests.

Frontend (`web/src/routes/settings/+page.svelte`)

New minimal /settings page with a single ListenBrainz section. Token field is a password input with Save/Clear buttons (masked when set). Enabled checkbox disabled when no token. "Last scrobbled" timestamp renders when present. Disclaimer about plaintext storage.

web/src/lib/components/Shell.svelte gains Settings in the nav. Helper module web/src/lib/api/listenbrainz.ts exports query/mutation factory functions matching the createSearchQuery/createAlbumQuery pattern. api.put<T> added to client.ts. 5 vitest tests.

Boot wiring (`cmd/minstrel/main.go`)

scrobbleWorker := scrobble.NewWorker(pool, listenbrainz.NewClient(), logger.With("component", "scrobble"))
go scrobbleWorker.Run(ctx)

Test plan

go test -short -race ./... — all 15 packages pass (now including internal/scrobble + internal/scrobble/listenbrainz)
Full integration suite — all packages green; pre-existing TestScanner_Integration flake unchanged (CI runs -short which skips it)
golangci-lint run ./... — clean
internal/scrobble coverage: 68.0% (target was ≥80%; 18 integration tests cover all major paths — gap is in less-critical logging branches in processBatch)
internal/scrobble/listenbrainz coverage: 70.2%
cd web && npm run check — 0 errors / 0 warnings (319 files)
cd web && npm test — 180 tests across 30 files (was 175)
cd web && npm run build — succeeds
docker build -t minstrel:m4a-smoke . — succeeds
Worker boot smoke test (5s go run) — server starts, no panic
Manual: generate token → /settings → paste + enable → play track → check listenbrainz.org/ for the listen within ~30s

Decisions ledger

#	Decision	Rationale
1	Per-user token	Forward-compat to multi-user; cost is 2 nullable columns
2	Plaintext storage	Industry norm for self-hosted scrobble apps; key-mgmt is a separate scope
3	Threshold ≥240s OR ≥50% (separate from skip)	Matches LB recommendation; skip threshold serves a different purpose
4	Pull-based 30s timer	Survives restarts/outages; failure handling natural; latency invisible at single-user scale
5	Minimal `/settings` in M4a	User needs somewhere to put the token; scaffold for M6 to extend
6	Patient backoff (1m → 5m → 30m → 2h → 6h, ~9h window)	Survives overnight LB outages without unbounded queue growth
7	Delete sent rows; keep failed	Canonical record in `play_events.scrobbled_at`; queue is a work list

Out of scope (deferred)

now-playing endpoint (needs push architecture)
Listen import (LB profile backfill)
Encrypted-at-rest token storage
PlayerBar scrobble indicator
Multi-instance queue safety (single-process assumed)
Multiple scrobble services (LB-only by design)

Sub-plan progression (M4)

M4a (this) — outbound scrobble worker.
M4b — inbound LB similarity ingest (track_similarity table + weekly cache).
M4c — radio similarity-driven candidate pool + queue refresh at 80% (closes M4).

🤖 Generated with Claude Code

First M4 sub-plan (Fable #345). Sends qualifying user plays to ListenBrainz with batching + exponential-backoff retry. Per-user token, plaintext storage, pull-based 30s timer worker, patient backoff (1m → 5m → 30m → 2h → 6h, 5 attempts). ## What this ships ### Schema (migration `0008_scrobble`) - `users.listenbrainz_token TEXT NULL` + `users.listenbrainz_enabled BOOLEAN DEFAULT FALSE` — per-user config columns - `scrobble_queue` table: `(id, user_id, play_event_id, status ∈ {pending, failed}, attempts, next_attempt_at, last_error, enqueued_at)` with `UNIQUE(play_event_id)` for idempotency and a partial index `WHERE status='pending'` for the worker's hot path - Lifecycle: successful submit DELETEs the queue row + stamps `play_events.scrobbled_at` (canonical record). Only `pending` and `failed` states ever exist in the queue. ### Backend pipeline (`internal/scrobble/`) - **`listenbrainz/client.go`** — pure HTTP client for `/1/submit-listens`. Typed errors `ErrAuth` (401), `ErrPermanent` (other 4xx), `ErrTransient` (5xx + network), `*RetryAfterError` (429 with parsed `Retry-After` header). 9 httptest-driven tests including payload shape, auth header, and `listen_type=single|import` switching. - **`threshold.go`** — `Qualifies(durationPlayedMs, completionRatio)` returning true when ≥240 seconds played OR ≥50% of track length. 6 boundary tests. - **`queue.go::MaybeEnqueue`** — gates on user config + threshold + idempotent INSERT (`ON CONFLICT DO NOTHING`). 5 integration tests. - **`worker.go`** — `Worker` struct with `Run` (30s ticker), `tickOnce` (drains up to 50 rows, groups per-user, batches POST), and `processBatch` (handles 6 outcome paths — success, 429-retry-after, 401-disable-user, ErrPermanent, ErrTransient-with-backoff, ErrTransient-give-up). Backoff schedule `1m → 5m → 30m → 2h → 6h`, then permanent fail. 6 integration tests. ### Hooks (`internal/playevents/writer.go`) `RecordPlayEnded` and `RecordSyntheticCompletedPlay` (Subsonic submission=true) both gain a post-commit `scrobble.MaybeEnqueue` call. Best-effort: errors logged + swallowed (matches the M3 `CaptureContextualLikeIfPlaying` pattern). 2 new integration tests. ### API (`internal/api/listenbrainz.go`) - `GET /api/me/listenbrainz` → `{enabled, token_set, last_scrobbled_at}`. Token write-only — `token_set` is the only signal back about whether one exists. `last_scrobbled_at` queried via `MAX(play_events.scrobbled_at)`. - `PUT /api/me/listenbrainz` body `{token?, enabled?}`. Clearing token (`{token: ""}`) auto-forces `enabled=false` via SQL CASE. Enabling without a token returns 400. - 4 integration tests. ### Frontend (`web/src/routes/settings/+page.svelte`) New minimal `/settings` page with a single ListenBrainz section. Token field is a password input with Save/Clear buttons (masked when set). Enabled checkbox disabled when no token. "Last scrobbled" timestamp renders when present. Disclaimer about plaintext storage. `web/src/lib/components/Shell.svelte` gains `Settings` in the nav. Helper module `web/src/lib/api/listenbrainz.ts` exports query/mutation factory functions matching the `createSearchQuery`/`createAlbumQuery` pattern. `api.put<T>` added to `client.ts`. 5 vitest tests. ### Boot wiring (`cmd/minstrel/main.go`) ```go scrobbleWorker := scrobble.NewWorker(pool, listenbrainz.NewClient(), logger.With("component", "scrobble")) go scrobbleWorker.Run(ctx) ``` ## Test plan - [x] `go test -short -race ./...` — all 15 packages pass (now including `internal/scrobble` + `internal/scrobble/listenbrainz`) - [x] Full integration suite — all packages green; pre-existing `TestScanner_Integration` flake unchanged (CI runs `-short` which skips it) - [x] `golangci-lint run ./...` — clean - [x] `internal/scrobble` coverage: **68.0%** (target was ≥80%; 18 integration tests cover all major paths — gap is in less-critical logging branches in `processBatch`) - [x] `internal/scrobble/listenbrainz` coverage: **70.2%** - [x] `cd web && npm run check` — 0 errors / 0 warnings (319 files) - [x] `cd web && npm test` — **180 tests** across 30 files (was 175) - [x] `cd web && npm run build` — succeeds - [x] `docker build -t minstrel:m4a-smoke .` — succeeds - [x] Worker boot smoke test (5s `go run`) — server starts, no panic - [ ] Manual: generate token → /settings → paste + enable → play track → check listenbrainz.org/<user> for the listen within ~30s ## Decisions ledger | # | Decision | Rationale | |---|---|---| | 1 | Per-user token | Forward-compat to multi-user; cost is 2 nullable columns | | 2 | Plaintext storage | Industry norm for self-hosted scrobble apps; key-mgmt is a separate scope | | 3 | Threshold ≥240s OR ≥50% (separate from skip) | Matches LB recommendation; skip threshold serves a different purpose | | 4 | Pull-based 30s timer | Survives restarts/outages; failure handling natural; latency invisible at single-user scale | | 5 | Minimal `/settings` in M4a | User needs somewhere to put the token; scaffold for M6 to extend | | 6 | Patient backoff (1m → 5m → 30m → 2h → 6h, ~9h window) | Survives overnight LB outages without unbounded queue growth | | 7 | Delete sent rows; keep failed | Canonical record in `play_events.scrobbled_at`; queue is a work list | ## Out of scope (deferred) - `now-playing` endpoint (needs push architecture) - Listen import (LB profile backfill) - Encrypted-at-rest token storage - PlayerBar scrobble indicator - Multi-instance queue safety (single-process assumed) - Multiple scrobble services (LB-only by design) ## Sub-plan progression (M4) - **M4a (this) — outbound scrobble worker.** - M4b — inbound LB similarity ingest (`track_similarity` table + weekly cache). - M4c — radio similarity-driven candidate pool + queue refresh at 80% (closes M4). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen referenced this issue from a commit

2026-07-13 21:44:05 -04:00

feat(tags): folksonomy tag enricher — pluggable provider chain (#1490 Step 2)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/minstrel#26

feat: M4a outbound ListenBrainz scrobble worker #26

What this ships

Schema (migration 0008_scrobble)

Backend pipeline (internal/scrobble/)

Hooks (internal/playevents/writer.go)

API (internal/api/listenbrainz.go)

Frontend (web/src/routes/settings/+page.svelte)

Boot wiring (cmd/minstrel/main.go)