a90deab862
12-task TDD-shaped plan covering: migration 0008 (users LB columns + scrobble_queue table), sqlc query additions, pure ListenBrainz HTTP client with typed errors and httptest coverage, threshold function, MaybeEnqueue with idempotent insert + threshold gate, backoff schedule 1m→5m→30m→2h→6h, Worker with batched submit + retry/fail/auth handling, playevents.Writer post-commit best-effort hooks, GET/PUT /api/me/listenbrainz endpoints, /settings page in SvelteKit, full verification sequence.
2524 lines
82 KiB
Markdown
2524 lines
82 KiB
Markdown
# M4a — ListenBrainz Outbound Scrobble Worker Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Send the user's qualifying plays (≥240s OR ≥50% completion) to ListenBrainz with batching + exponential-backoff retry. Per-user token + enabled flag in the SPA's new `/settings` page. Backed by a `scrobble_queue` table that the worker drains every 30s.
|
|
|
|
**Architecture:** New `internal/scrobble/` package tree: `listenbrainz/` for the pure HTTP client (typed errors for auth/permanent/transient/retry-after), and the parent package for `Qualifies`, `MaybeEnqueue`, and the `Worker` goroutine. `playevents.Writer.RecordPlayEnded` and `RecordSyntheticCompletedPlay` gain a post-commit best-effort enqueue call. New API endpoints `GET/PUT /api/me/listenbrainz` and a minimal `/settings` page in the SPA.
|
|
|
|
**Tech Stack:** Go 1.23 + chi + sqlc + pgx/v5. SvelteKit 2 + Svelte 5 + TanStack Query. New SQL migration; no breaking changes to existing schema.
|
|
|
|
**Reference:** design spec at `docs/superpowers/specs/2026-04-28-m4a-scrobble-design.md`.
|
|
|
|
---
|
|
|
|
## File Structure
|
|
|
|
**New server files:**
|
|
|
|
| File | Responsibility |
|
|
|---|---|
|
|
| `internal/db/migrations/0008_scrobble.up.sql` + `.down.sql` | `users.listenbrainz_token`, `users.listenbrainz_enabled`, `scrobble_queue` table + partial index. |
|
|
| `internal/db/queries/users.sql` (modify) | Add `SetListenBrainzToken`, `SetListenBrainzEnabled`, `GetListenBrainzConfig`. |
|
|
| `internal/db/queries/scrobble.sql` | New: `EnqueueScrobble :execrows`, `ListPendingScrobbles`, `MarkScrobbleSent`, `MarkScrobbleFailed`, `RescheduleScrobble`, `MarkScrobbleRetryAfter`, `GetLastScrobbledForUser`. |
|
|
| `internal/db/dbq/scrobble.sql.go` | Generated bindings. |
|
|
| `internal/scrobble/listenbrainz/client.go` | Pure HTTP client. `Client`, `Listen`, `Track` types; `SubmitListens` method; typed errors `ErrAuth`, `ErrPermanent`, `ErrTransient`, `*RetryAfterError`. |
|
|
| `internal/scrobble/listenbrainz/client_test.go` | `httptest.Server`-driven tests for each error mapping + payload shape. |
|
|
| `internal/scrobble/threshold.go` | Pure `Qualifies(durationPlayedMs int, completionRatio float64) bool`. |
|
|
| `internal/scrobble/threshold_test.go` | Boundary tests. |
|
|
| `internal/scrobble/queue.go` | `MaybeEnqueue(ctx, q, playEventID)`. Reads user config + threshold; idempotent INSERT. |
|
|
| `internal/scrobble/queue_test.go` | Live-DB integration tests. |
|
|
| `internal/scrobble/worker.go` | `Worker` struct, `Run` loop, `tickOnce`, `backoffDelay` helper. |
|
|
| `internal/scrobble/worker_test.go` | Pure tests for `backoffDelay`. |
|
|
| `internal/scrobble/worker_integration_test.go` | Live-DB integration: full ticker pipeline against an httptest LB. |
|
|
| `internal/api/listenbrainz.go` | `handleGetListenBrainz`, `handlePutListenBrainz`. |
|
|
| `internal/api/listenbrainz_test.go` | Live-DB HTTP tests. |
|
|
|
|
**Modified server files:**
|
|
|
|
| File | Change |
|
|
|---|---|
|
|
| `internal/db/dbq/users.sql.go` + `models.go` | Generated additions for the new user columns. |
|
|
| `internal/playevents/writer.go::RecordPlayEnded` | After `pgx.BeginFunc` returns nil, call `scrobble.MaybeEnqueue(ctx, dbq.New(w.pool), playEventID)`; log + swallow errors. |
|
|
| `internal/playevents/writer.go::RecordSyntheticCompletedPlay` | Same pattern. |
|
|
| `internal/playevents/writer_test.go` | Three new tests covering enqueue: qualifying play → row inserted, sub-threshold → no row, user-disabled → no row. |
|
|
| `internal/api/api.go::Mount` | Add `authed.Get("/me/listenbrainz", h.handleGetListenBrainz)` and `authed.Put("/me/listenbrainz", h.handlePutListenBrainz)`. |
|
|
| `internal/api/auth_test.go` | testHandlers helper unchanged (no new struct fields). |
|
|
| `cmd/minstrel/main.go` | Construct `scrobble.Worker` after pool open; `go worker.Run(ctx)` after server starts. |
|
|
|
|
**Frontend files:**
|
|
|
|
| File | Responsibility |
|
|
|---|---|
|
|
| `web/src/lib/api/client.ts` (modify) | Add `api.put<T>(path, body)` helper. |
|
|
| `web/src/lib/api/listenbrainz.ts` | New: `getListenBrainzStatus()`, `setListenBrainzToken(token)`, `setListenBrainzEnabled(enabled)` + `LBStatus` type. |
|
|
| `web/src/routes/settings/+page.svelte` | The Settings page, single ListenBrainz section. |
|
|
| `web/src/routes/settings/settings.test.ts` | Vitest coverage for the page. |
|
|
| `web/src/lib/components/Shell.svelte` | Add `{ href: '/settings', label: 'Settings' }` to `navItems`. |
|
|
| `web/src/lib/components/Shell.test.ts` (modify) | Update nav assertions if the test enumerates items. |
|
|
|
|
---
|
|
|
|
## Task 1: Migration 0008 — schema
|
|
|
|
**Files:**
|
|
- Create: `internal/db/migrations/0008_scrobble.up.sql`
|
|
- Create: `internal/db/migrations/0008_scrobble.down.sql`
|
|
|
|
- [ ] **Step 1: Write the up migration**
|
|
|
|
Create `internal/db/migrations/0008_scrobble.up.sql`:
|
|
|
|
```sql
|
|
-- M4a: outbound ListenBrainz scrobble worker.
|
|
-- Per-user LB config (plaintext token; users rotate via /settings if leaked).
|
|
-- scrobble_queue is a work list, not a log: successfully-sent rows are
|
|
-- DELETEd by the worker (the canonical record stays in play_events.scrobbled_at
|
|
-- via M2's column). Only `pending` and `failed` states exist.
|
|
|
|
ALTER TABLE users
|
|
ADD COLUMN listenbrainz_token TEXT NULL,
|
|
ADD COLUMN listenbrainz_enabled BOOLEAN NOT NULL DEFAULT FALSE;
|
|
|
|
CREATE TABLE scrobble_queue (
|
|
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
|
play_event_id uuid NOT NULL REFERENCES play_events(id) ON DELETE CASCADE,
|
|
status TEXT NOT NULL CHECK (status IN ('pending', 'failed')),
|
|
attempts INTEGER NOT NULL DEFAULT 0,
|
|
next_attempt_at timestamptz NOT NULL DEFAULT now(),
|
|
last_error TEXT NULL,
|
|
enqueued_at timestamptz NOT NULL DEFAULT now(),
|
|
UNIQUE (play_event_id)
|
|
);
|
|
|
|
-- Hot path for the worker's per-tick query.
|
|
CREATE INDEX scrobble_queue_pending_idx
|
|
ON scrobble_queue (next_attempt_at)
|
|
WHERE status = 'pending';
|
|
```
|
|
|
|
- [ ] **Step 2: Write the down migration**
|
|
|
|
Create `internal/db/migrations/0008_scrobble.down.sql`:
|
|
|
|
```sql
|
|
DROP TABLE IF EXISTS scrobble_queue;
|
|
ALTER TABLE users DROP COLUMN IF EXISTS listenbrainz_enabled;
|
|
ALTER TABLE users DROP COLUMN IF EXISTS listenbrainz_token;
|
|
```
|
|
|
|
- [ ] **Step 3: Verify the migration applies cleanly**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && MINSTREL_TEST_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' go test -short -race ./internal/db/`
|
|
|
|
Expected: PASS. The migration test (which applies all migrations against a temp schema) finds version 8.
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add internal/db/migrations/0008_scrobble.up.sql internal/db/migrations/0008_scrobble.down.sql
|
|
git commit -m "feat(db): add migration 0008 for scrobble (users LB columns + scrobble_queue table)"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 2: sqlc queries — users LB config + scrobble_queue
|
|
|
|
**Files:**
|
|
- Modify: `internal/db/queries/users.sql`
|
|
- Create: `internal/db/queries/scrobble.sql`
|
|
- Generated: `internal/db/dbq/users.sql.go`, `internal/db/dbq/scrobble.sql.go`, `internal/db/dbq/models.go`
|
|
|
|
- [ ] **Step 1: Add user-LB-config queries**
|
|
|
|
Append to `internal/db/queries/users.sql`:
|
|
|
|
```sql
|
|
-- name: SetListenBrainzToken :exec
|
|
UPDATE users
|
|
SET listenbrainz_token = $2,
|
|
listenbrainz_enabled = CASE WHEN COALESCE($2, '') = '' THEN FALSE ELSE listenbrainz_enabled END
|
|
WHERE id = $1;
|
|
|
|
-- name: SetListenBrainzEnabled :exec
|
|
UPDATE users
|
|
SET listenbrainz_enabled = $2
|
|
WHERE id = $1;
|
|
|
|
-- name: GetListenBrainzConfig :one
|
|
-- Returns the user's LB token + enabled flag and the most recent
|
|
-- play_events.scrobbled_at for last-scrobbled-at status.
|
|
SELECT
|
|
u.listenbrainz_token,
|
|
u.listenbrainz_enabled,
|
|
(SELECT MAX(pe.scrobbled_at)
|
|
FROM play_events pe
|
|
WHERE pe.user_id = u.id) AS last_scrobbled_at
|
|
FROM users u
|
|
WHERE u.id = $1;
|
|
```
|
|
|
|
- [ ] **Step 2: Add scrobble_queue queries**
|
|
|
|
Create `internal/db/queries/scrobble.sql`:
|
|
|
|
```sql
|
|
-- name: EnqueueScrobble :execrows
|
|
-- Idempotent: if a row exists for this play_event_id, ON CONFLICT does nothing
|
|
-- and the affected-rows count is 0. Caller (MaybeEnqueue) doesn't treat 0 as
|
|
-- an error.
|
|
INSERT INTO scrobble_queue (user_id, play_event_id, status)
|
|
VALUES ($1, $2, 'pending')
|
|
ON CONFLICT (play_event_id) DO NOTHING;
|
|
|
|
-- name: ListPendingScrobbles :many
|
|
-- Worker pulls up to N pending rows whose next_attempt_at has passed.
|
|
-- Joins play_events + tracks/albums/artists + users so the worker can
|
|
-- assemble the LB Listen payload in one round-trip.
|
|
SELECT
|
|
sq.id AS queue_id,
|
|
sq.user_id AS user_id,
|
|
sq.play_event_id AS play_event_id,
|
|
sq.attempts AS attempts,
|
|
u.listenbrainz_token AS lb_token,
|
|
u.listenbrainz_enabled AS lb_enabled,
|
|
pe.started_at AS started_at,
|
|
t.title AS track_title,
|
|
t.duration_ms AS track_duration_ms,
|
|
t.mbid AS track_mbid,
|
|
al.title AS album_title,
|
|
al.mbid AS album_mbid,
|
|
ar.name AS artist_name,
|
|
ar.mbid AS artist_mbid
|
|
FROM scrobble_queue sq
|
|
JOIN users u ON u.id = sq.user_id
|
|
JOIN play_events pe ON pe.id = sq.play_event_id
|
|
JOIN tracks t ON t.id = pe.track_id
|
|
JOIN albums al ON al.id = t.album_id
|
|
JOIN artists ar ON ar.id = t.artist_id
|
|
WHERE sq.status = 'pending'
|
|
AND sq.next_attempt_at <= now()
|
|
ORDER BY sq.next_attempt_at
|
|
LIMIT $1;
|
|
|
|
-- name: MarkScrobbleSent :exec
|
|
-- Successful submit. Delete the queue row AND stamp play_events.scrobbled_at
|
|
-- in one statement via a CTE so partial failure is impossible.
|
|
WITH del AS (
|
|
DELETE FROM scrobble_queue
|
|
WHERE id = $1
|
|
RETURNING play_event_id
|
|
)
|
|
UPDATE play_events
|
|
SET scrobbled_at = now()
|
|
WHERE id = (SELECT play_event_id FROM del);
|
|
|
|
-- name: RescheduleScrobble :exec
|
|
-- After a transient failure: increment attempts, schedule next attempt,
|
|
-- record the error.
|
|
UPDATE scrobble_queue
|
|
SET attempts = attempts + 1,
|
|
next_attempt_at = $2,
|
|
last_error = $3
|
|
WHERE id = $1;
|
|
|
|
-- name: MarkScrobbleRetryAfter :exec
|
|
-- After a 429: schedule next attempt per LB's Retry-After header, but do NOT
|
|
-- increment attempts (the server told us to wait, not that we failed).
|
|
UPDATE scrobble_queue
|
|
SET next_attempt_at = $2
|
|
WHERE id = $1;
|
|
|
|
-- name: MarkScrobbleFailed :exec
|
|
-- After a permanent failure (4xx, exhausted retries, auth) — mark failed
|
|
-- and keep the row for diagnostics.
|
|
UPDATE scrobble_queue
|
|
SET status = 'failed',
|
|
last_error = $2
|
|
WHERE id = $1;
|
|
```
|
|
|
|
- [ ] **Step 3: Run sqlc generate**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && make generate`
|
|
|
|
Expected: docker pulls sqlc 1.27.0, regenerates `internal/db/dbq/`. New methods appear on `*Queries`:
|
|
- `SetListenBrainzToken`, `SetListenBrainzEnabled`, `GetListenBrainzConfig`
|
|
- `EnqueueScrobble`, `ListPendingScrobbles`, `MarkScrobbleSent`, `RescheduleScrobble`, `MarkScrobbleRetryAfter`, `MarkScrobbleFailed`
|
|
|
|
`models.go` gains the two new `User` fields: `ListenbrainzToken *string` and `ListenbrainzEnabled bool`.
|
|
|
|
- [ ] **Step 4: Verify compile**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && go build ./...`
|
|
|
|
Expected: clean. No callers yet — generated bindings just need to compile.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add internal/db/queries/ internal/db/dbq/
|
|
git commit -m "feat(db): add ListenBrainz user-config and scrobble_queue queries"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 3: ListenBrainz HTTP client (pure)
|
|
|
|
**Files:**
|
|
- Create: `internal/scrobble/listenbrainz/client.go`
|
|
- Create: `internal/scrobble/listenbrainz/client_test.go`
|
|
|
|
- [ ] **Step 1: Write the failing tests**
|
|
|
|
Create `internal/scrobble/listenbrainz/client_test.go`:
|
|
|
|
```go
|
|
package listenbrainz
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"strings"
|
|
"testing"
|
|
"time"
|
|
)
|
|
|
|
func newTestClient(handler http.HandlerFunc) (*Client, *httptest.Server) {
|
|
srv := httptest.NewServer(handler)
|
|
return &Client{BaseURL: srv.URL, HTTP: srv.Client()}, srv
|
|
}
|
|
|
|
func TestClient_SubmitListens_Success(t *testing.T) {
|
|
var seenPath, seenAuth, seenBody string
|
|
c, srv := newTestClient(func(w http.ResponseWriter, r *http.Request) {
|
|
seenPath = r.URL.Path
|
|
seenAuth = r.Header.Get("Authorization")
|
|
buf := make([]byte, 4096)
|
|
n, _ := r.Body.Read(buf)
|
|
seenBody = string(buf[:n])
|
|
w.WriteHeader(http.StatusOK)
|
|
_, _ = w.Write([]byte(`{"status":"ok"}`))
|
|
})
|
|
defer srv.Close()
|
|
|
|
listens := []Listen{{
|
|
ListenedAt: 1700000000,
|
|
Track: Track{
|
|
ArtistName: "Miles Davis",
|
|
TrackName: "So What",
|
|
ReleaseName: "Kind of Blue",
|
|
DurationMs: 545000,
|
|
},
|
|
}}
|
|
if err := c.SubmitListens(context.Background(), "tk", listens); err != nil {
|
|
t.Fatalf("err: %v", err)
|
|
}
|
|
if seenPath != "/1/submit-listens" {
|
|
t.Errorf("path = %q, want /1/submit-listens", seenPath)
|
|
}
|
|
if seenAuth != "Token tk" {
|
|
t.Errorf("auth = %q, want %q", seenAuth, "Token tk")
|
|
}
|
|
if !strings.Contains(seenBody, `"track_name":"So What"`) {
|
|
t.Errorf("body missing track_name: %s", seenBody)
|
|
}
|
|
if !strings.Contains(seenBody, `"artist_name":"Miles Davis"`) {
|
|
t.Errorf("body missing artist_name: %s", seenBody)
|
|
}
|
|
}
|
|
|
|
func TestClient_SubmitListens_401_ReturnsErrAuth(t *testing.T) {
|
|
c, srv := newTestClient(func(w http.ResponseWriter, r *http.Request) {
|
|
w.WriteHeader(http.StatusUnauthorized)
|
|
})
|
|
defer srv.Close()
|
|
err := c.SubmitListens(context.Background(), "bad", []Listen{{}})
|
|
if !errors.Is(err, ErrAuth) {
|
|
t.Errorf("err = %v, want ErrAuth", err)
|
|
}
|
|
}
|
|
|
|
func TestClient_SubmitListens_400_ReturnsErrPermanent(t *testing.T) {
|
|
c, srv := newTestClient(func(w http.ResponseWriter, r *http.Request) {
|
|
w.WriteHeader(http.StatusBadRequest)
|
|
})
|
|
defer srv.Close()
|
|
err := c.SubmitListens(context.Background(), "tk", []Listen{{}})
|
|
if !errors.Is(err, ErrPermanent) {
|
|
t.Errorf("err = %v, want ErrPermanent", err)
|
|
}
|
|
}
|
|
|
|
func TestClient_SubmitListens_503_ReturnsErrTransient(t *testing.T) {
|
|
c, srv := newTestClient(func(w http.ResponseWriter, r *http.Request) {
|
|
w.WriteHeader(http.StatusServiceUnavailable)
|
|
})
|
|
defer srv.Close()
|
|
err := c.SubmitListens(context.Background(), "tk", []Listen{{}})
|
|
if !errors.Is(err, ErrTransient) {
|
|
t.Errorf("err = %v, want ErrTransient", err)
|
|
}
|
|
}
|
|
|
|
func TestClient_SubmitListens_429_ReturnsRetryAfter(t *testing.T) {
|
|
c, srv := newTestClient(func(w http.ResponseWriter, r *http.Request) {
|
|
w.Header().Set("Retry-After", "60")
|
|
w.WriteHeader(http.StatusTooManyRequests)
|
|
})
|
|
defer srv.Close()
|
|
err := c.SubmitListens(context.Background(), "tk", []Listen{{}})
|
|
var ra *RetryAfterError
|
|
if !errors.As(err, &ra) {
|
|
t.Fatalf("err = %v, want *RetryAfterError", err)
|
|
}
|
|
if ra.Wait != 60*time.Second {
|
|
t.Errorf("Wait = %v, want 60s", ra.Wait)
|
|
}
|
|
}
|
|
|
|
func TestClient_SubmitListens_NetworkFailure_ReturnsErrTransient(t *testing.T) {
|
|
c := &Client{BaseURL: "http://127.0.0.1:1", HTTP: &http.Client{Timeout: 100 * time.Millisecond}}
|
|
err := c.SubmitListens(context.Background(), "tk", []Listen{{}})
|
|
if !errors.Is(err, ErrTransient) {
|
|
t.Errorf("err = %v, want ErrTransient", err)
|
|
}
|
|
}
|
|
|
|
func TestClient_SubmitListens_PayloadTypeImportForBatch(t *testing.T) {
|
|
var seenBody string
|
|
c, srv := newTestClient(func(w http.ResponseWriter, r *http.Request) {
|
|
buf := make([]byte, 4096)
|
|
n, _ := r.Body.Read(buf)
|
|
seenBody = string(buf[:n])
|
|
w.WriteHeader(http.StatusOK)
|
|
})
|
|
defer srv.Close()
|
|
_ = c.SubmitListens(context.Background(), "tk", []Listen{
|
|
{ListenedAt: 1, Track: Track{ArtistName: "A", TrackName: "X"}},
|
|
{ListenedAt: 2, Track: Track{ArtistName: "B", TrackName: "Y"}},
|
|
})
|
|
if !strings.Contains(seenBody, `"listen_type":"import"`) {
|
|
t.Errorf("batch body missing listen_type=import: %s", seenBody)
|
|
}
|
|
}
|
|
|
|
func TestClient_SubmitListens_PayloadTypeSingleForOne(t *testing.T) {
|
|
var seenBody string
|
|
c, srv := newTestClient(func(w http.ResponseWriter, r *http.Request) {
|
|
buf := make([]byte, 4096)
|
|
n, _ := r.Body.Read(buf)
|
|
seenBody = string(buf[:n])
|
|
w.WriteHeader(http.StatusOK)
|
|
})
|
|
defer srv.Close()
|
|
_ = c.SubmitListens(context.Background(), "tk", []Listen{
|
|
{ListenedAt: 1, Track: Track{ArtistName: "A", TrackName: "X"}},
|
|
})
|
|
if !strings.Contains(seenBody, `"listen_type":"single"`) {
|
|
t.Errorf("single body missing listen_type=single: %s", seenBody)
|
|
}
|
|
}
|
|
|
|
func TestClient_SubmitListens_OmitsEmptyMBIDs(t *testing.T) {
|
|
var seenBody string
|
|
c, srv := newTestClient(func(w http.ResponseWriter, r *http.Request) {
|
|
buf := make([]byte, 4096)
|
|
n, _ := r.Body.Read(buf)
|
|
seenBody = string(buf[:n])
|
|
w.WriteHeader(http.StatusOK)
|
|
})
|
|
defer srv.Close()
|
|
_ = c.SubmitListens(context.Background(), "tk", []Listen{
|
|
{ListenedAt: 1, Track: Track{ArtistName: "A", TrackName: "X"}},
|
|
})
|
|
if strings.Contains(seenBody, "recording_mbid") {
|
|
t.Errorf("body should omit empty recording_mbid: %s", seenBody)
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && go test ./internal/scrobble/listenbrainz/ -v`
|
|
|
|
Expected: FAIL with "no Go files" or "undefined: Client".
|
|
|
|
- [ ] **Step 3: Write the client**
|
|
|
|
Create `internal/scrobble/listenbrainz/client.go`:
|
|
|
|
```go
|
|
// Package listenbrainz is the pure HTTP client for the ListenBrainz
|
|
// submit-listens endpoint. No DB, no worker plumbing — just types and one
|
|
// method. Errors are typed (ErrAuth, ErrPermanent, ErrTransient,
|
|
// *RetryAfterError) so callers can branch on response semantics.
|
|
package listenbrainz
|
|
|
|
import (
|
|
"bytes"
|
|
"context"
|
|
"encoding/json"
|
|
"errors"
|
|
"fmt"
|
|
"net/http"
|
|
"strconv"
|
|
"time"
|
|
)
|
|
|
|
const defaultBaseURL = "https://api.listenbrainz.org"
|
|
|
|
// Listen is one row sent to ListenBrainz.
|
|
type Listen struct {
|
|
ListenedAt int64 // unix seconds
|
|
Track Track
|
|
}
|
|
|
|
// Track is the per-listen metadata. Optional fields are omitted from the
|
|
// payload when zero-valued.
|
|
type Track struct {
|
|
ArtistName string
|
|
TrackName string
|
|
ReleaseName string
|
|
DurationMs int
|
|
RecordingMBID string
|
|
ArtistMBIDs []string
|
|
ReleaseMBID string
|
|
}
|
|
|
|
// Client posts to the LB submit-listens endpoint.
|
|
type Client struct {
|
|
BaseURL string
|
|
HTTP *http.Client
|
|
}
|
|
|
|
// NewClient returns a default-configured client (LB production base URL,
|
|
// 30s timeout).
|
|
func NewClient() *Client {
|
|
return &Client{
|
|
BaseURL: defaultBaseURL,
|
|
HTTP: &http.Client{Timeout: 30 * time.Second},
|
|
}
|
|
}
|
|
|
|
// Sentinel errors. The worker branches on these to decide retry vs fail.
|
|
var (
|
|
ErrAuth = errors.New("listenbrainz: auth rejected")
|
|
ErrPermanent = errors.New("listenbrainz: permanent error")
|
|
ErrTransient = errors.New("listenbrainz: transient error")
|
|
)
|
|
|
|
// RetryAfterError carries the LB-supplied wait duration for 429 responses.
|
|
type RetryAfterError struct{ Wait time.Duration }
|
|
|
|
func (e *RetryAfterError) Error() string {
|
|
return fmt.Sprintf("listenbrainz: retry after %v", e.Wait)
|
|
}
|
|
|
|
// SubmitListens posts the given listens. payload_type is "single" for one
|
|
// listen, "import" for batches.
|
|
func (c *Client) SubmitListens(ctx context.Context, token string, listens []Listen) error {
|
|
if len(listens) == 0 {
|
|
return nil
|
|
}
|
|
|
|
listenType := "import"
|
|
if len(listens) == 1 {
|
|
listenType = "single"
|
|
}
|
|
|
|
body, err := json.Marshal(buildPayload(listenType, listens))
|
|
if err != nil {
|
|
return fmt.Errorf("listenbrainz: marshal: %w", err)
|
|
}
|
|
|
|
base := c.BaseURL
|
|
if base == "" {
|
|
base = defaultBaseURL
|
|
}
|
|
req, err := http.NewRequestWithContext(ctx, http.MethodPost, base+"/1/submit-listens", bytes.NewReader(body))
|
|
if err != nil {
|
|
return fmt.Errorf("listenbrainz: build request: %w", err)
|
|
}
|
|
req.Header.Set("Content-Type", "application/json")
|
|
req.Header.Set("Authorization", "Token "+token)
|
|
|
|
httpClient := c.HTTP
|
|
if httpClient == nil {
|
|
httpClient = http.DefaultClient
|
|
}
|
|
resp, err := httpClient.Do(req)
|
|
if err != nil {
|
|
return fmt.Errorf("%w: %v", ErrTransient, err)
|
|
}
|
|
defer func() { _ = resp.Body.Close() }()
|
|
|
|
switch {
|
|
case resp.StatusCode >= 200 && resp.StatusCode < 300:
|
|
return nil
|
|
case resp.StatusCode == http.StatusUnauthorized:
|
|
return ErrAuth
|
|
case resp.StatusCode == http.StatusTooManyRequests:
|
|
wait := parseRetryAfter(resp.Header.Get("Retry-After"))
|
|
return &RetryAfterError{Wait: wait}
|
|
case resp.StatusCode >= 500:
|
|
return fmt.Errorf("%w: status %d", ErrTransient, resp.StatusCode)
|
|
default:
|
|
return fmt.Errorf("%w: status %d", ErrPermanent, resp.StatusCode)
|
|
}
|
|
}
|
|
|
|
func parseRetryAfter(header string) time.Duration {
|
|
if header == "" {
|
|
return 60 * time.Second // sensible default
|
|
}
|
|
if secs, err := strconv.Atoi(header); err == nil {
|
|
return time.Duration(secs) * time.Second
|
|
}
|
|
if t, err := http.ParseTime(header); err == nil {
|
|
d := time.Until(t)
|
|
if d < 0 {
|
|
return 60 * time.Second
|
|
}
|
|
return d
|
|
}
|
|
return 60 * time.Second
|
|
}
|
|
|
|
// payload mirrors the LB submit-listens body. Optional MBID fields use
|
|
// `omitempty` so empty strings/slices don't appear in the JSON.
|
|
type payload struct {
|
|
ListenType string `json:"listen_type"`
|
|
Payload []payloadEntry `json:"payload"`
|
|
}
|
|
|
|
type payloadEntry struct {
|
|
ListenedAt int64 `json:"listened_at"`
|
|
TrackMetadata trackMetadata `json:"track_metadata"`
|
|
}
|
|
|
|
type trackMetadata struct {
|
|
ArtistName string `json:"artist_name"`
|
|
TrackName string `json:"track_name"`
|
|
ReleaseName string `json:"release_name,omitempty"`
|
|
AdditionalInfo additionalInfo `json:"additional_info,omitempty"`
|
|
}
|
|
|
|
type additionalInfo struct {
|
|
DurationMs int `json:"duration_ms,omitempty"`
|
|
RecordingMBID string `json:"recording_mbid,omitempty"`
|
|
ArtistMBIDs []string `json:"artist_mbids,omitempty"`
|
|
ReleaseMBID string `json:"release_mbid,omitempty"`
|
|
}
|
|
|
|
func buildPayload(listenType string, listens []Listen) payload {
|
|
entries := make([]payloadEntry, 0, len(listens))
|
|
for _, l := range listens {
|
|
entries = append(entries, payloadEntry{
|
|
ListenedAt: l.ListenedAt,
|
|
TrackMetadata: trackMetadata{
|
|
ArtistName: l.Track.ArtistName,
|
|
TrackName: l.Track.TrackName,
|
|
ReleaseName: l.Track.ReleaseName,
|
|
AdditionalInfo: additionalInfo{
|
|
DurationMs: l.Track.DurationMs,
|
|
RecordingMBID: l.Track.RecordingMBID,
|
|
ArtistMBIDs: l.Track.ArtistMBIDs,
|
|
ReleaseMBID: l.Track.ReleaseMBID,
|
|
},
|
|
},
|
|
})
|
|
}
|
|
return payload{ListenType: listenType, Payload: entries}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && go test ./internal/scrobble/listenbrainz/ -v`
|
|
|
|
Expected: PASS for all 9 tests.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add internal/scrobble/listenbrainz/
|
|
git commit -m "feat(scrobble): add pure ListenBrainz HTTP client with typed errors"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 4: Threshold (pure)
|
|
|
|
**Files:**
|
|
- Create: `internal/scrobble/threshold.go`
|
|
- Create: `internal/scrobble/threshold_test.go`
|
|
|
|
- [ ] **Step 1: Write the failing tests**
|
|
|
|
Create `internal/scrobble/threshold_test.go`:
|
|
|
|
```go
|
|
package scrobble
|
|
|
|
import "testing"
|
|
|
|
func TestQualifies_NeverPlayed(t *testing.T) {
|
|
if Qualifies(0, 0) {
|
|
t.Error("0/0 should not qualify")
|
|
}
|
|
}
|
|
|
|
func TestQualifies_AtLeast240Seconds(t *testing.T) {
|
|
if !Qualifies(240_000, 0.0) {
|
|
t.Error("240_000 ms played should qualify regardless of ratio")
|
|
}
|
|
}
|
|
|
|
func TestQualifies_AtLeastHalfRatio(t *testing.T) {
|
|
if !Qualifies(0, 0.5) {
|
|
t.Error("ratio=0.5 should qualify regardless of duration")
|
|
}
|
|
}
|
|
|
|
func TestQualifies_BelowBoth(t *testing.T) {
|
|
if Qualifies(120_000, 0.49) {
|
|
t.Error("120s + 49% should not qualify")
|
|
}
|
|
}
|
|
|
|
func TestQualifies_BoundaryJustUnder(t *testing.T) {
|
|
if Qualifies(239_999, 0.4999) {
|
|
t.Error("just-under both thresholds should not qualify")
|
|
}
|
|
}
|
|
|
|
func TestQualifies_BoundaryExactly(t *testing.T) {
|
|
if !Qualifies(240_000, 0.4999) {
|
|
t.Error("exactly 240s should qualify")
|
|
}
|
|
if !Qualifies(239_999, 0.5) {
|
|
t.Error("exactly 50% should qualify")
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && go test ./internal/scrobble/ -run Qualifies -v`
|
|
|
|
Expected: FAIL with "no Go files" or "undefined: Qualifies".
|
|
|
|
- [ ] **Step 3: Write the threshold function**
|
|
|
|
Create `internal/scrobble/threshold.go`:
|
|
|
|
```go
|
|
// Package scrobble owns the outbound-scrobble pipeline: threshold detection,
|
|
// the queue write path, and the worker goroutine. The HTTP client is in
|
|
// internal/scrobble/listenbrainz; this package consumes it.
|
|
package scrobble
|
|
|
|
// Qualifies returns true if a closed play_event meets ListenBrainz's
|
|
// recommended scrobble threshold: ≥240 seconds played OR ≥50% of the
|
|
// track length played.
|
|
//
|
|
// The two thresholds are deliberately separate from M2's skip-detection
|
|
// thresholds (which serve a different purpose — internal engine signal
|
|
// for the scoring formula's skip penalty).
|
|
func Qualifies(durationPlayedMs int, completionRatio float64) bool {
|
|
return durationPlayedMs >= 240_000 || completionRatio >= 0.5
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && go test ./internal/scrobble/ -run Qualifies -v`
|
|
|
|
Expected: PASS for all 6 tests.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add internal/scrobble/threshold.go internal/scrobble/threshold_test.go
|
|
git commit -m "feat(scrobble): add pure Qualifies threshold function"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 5: MaybeEnqueue (DB-backed)
|
|
|
|
**Files:**
|
|
- Create: `internal/scrobble/queue.go`
|
|
- Create: `internal/scrobble/queue_test.go`
|
|
|
|
- [ ] **Step 1: Write the failing integration tests**
|
|
|
|
Create `internal/scrobble/queue_test.go`:
|
|
|
|
```go
|
|
package scrobble
|
|
|
|
import (
|
|
"context"
|
|
"io"
|
|
"log/slog"
|
|
"os"
|
|
"testing"
|
|
|
|
"github.com/jackc/pgx/v5/pgtype"
|
|
"github.com/jackc/pgx/v5/pgxpool"
|
|
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/db"
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
|
|
)
|
|
|
|
func testPool(t *testing.T) (*pgxpool.Pool, *dbq.Queries) {
|
|
t.Helper()
|
|
if testing.Short() {
|
|
t.Skip("skipping in -short mode")
|
|
}
|
|
dsn := os.Getenv("MINSTREL_TEST_DATABASE_URL")
|
|
if dsn == "" {
|
|
t.Skip("MINSTREL_TEST_DATABASE_URL not set")
|
|
}
|
|
if err := db.Migrate(dsn, slog.New(slog.NewTextHandler(io.Discard, nil))); err != nil {
|
|
t.Fatalf("migrate: %v", err)
|
|
}
|
|
pool, err := pgxpool.New(context.Background(), dsn)
|
|
if err != nil {
|
|
t.Fatalf("pool: %v", err)
|
|
}
|
|
t.Cleanup(pool.Close)
|
|
if _, err := pool.Exec(context.Background(),
|
|
"TRUNCATE scrobble_queue, play_events, skip_events, play_sessions, sessions, users, tracks, albums, artists RESTART IDENTITY CASCADE"); err != nil {
|
|
t.Fatalf("truncate: %v", err)
|
|
}
|
|
return pool, dbq.New(pool)
|
|
}
|
|
|
|
type setup struct {
|
|
pool *pgxpool.Pool
|
|
q *dbq.Queries
|
|
user pgtype.UUID
|
|
playEvent pgtype.UUID
|
|
}
|
|
|
|
// seed creates: user (with optional LB token + enabled), one artist/album/track,
|
|
// one closed play_event with the given duration_played_ms and completion_ratio.
|
|
func seed(t *testing.T, opts seedOpts) setup {
|
|
t.Helper()
|
|
pool, q := testPool(t)
|
|
ctx := context.Background()
|
|
u, err := q.CreateUser(ctx, dbq.CreateUserParams{
|
|
Username: "tester", PasswordHash: "x", ApiToken: "x", IsAdmin: false,
|
|
})
|
|
if err != nil {
|
|
t.Fatalf("user: %v", err)
|
|
}
|
|
if opts.lbToken != "" {
|
|
if err := q.SetListenBrainzToken(ctx, dbq.SetListenBrainzTokenParams{ID: u.ID, ListenbrainzToken: &opts.lbToken}); err != nil {
|
|
t.Fatalf("token: %v", err)
|
|
}
|
|
}
|
|
if opts.lbEnabled {
|
|
if err := q.SetListenBrainzEnabled(ctx, dbq.SetListenBrainzEnabledParams{ID: u.ID, ListenbrainzEnabled: true}); err != nil {
|
|
t.Fatalf("enabled: %v", err)
|
|
}
|
|
}
|
|
a, _ := q.UpsertArtist(ctx, dbq.UpsertArtistParams{Name: "X", SortName: "X"})
|
|
al, _ := q.UpsertAlbum(ctx, dbq.UpsertAlbumParams{Title: "X", SortTitle: "X", ArtistID: a.ID})
|
|
tr, _ := q.UpsertTrack(ctx, dbq.UpsertTrackParams{
|
|
Title: "Y", AlbumID: al.ID, ArtistID: a.ID, FilePath: "/tmp/y.flac", DurationMs: 300_000,
|
|
})
|
|
// Insert play_session + play_event directly with the provided duration/ratio.
|
|
var sessionID pgtype.UUID
|
|
if err := pool.QueryRow(ctx,
|
|
`INSERT INTO play_sessions (user_id, started_at, last_event_at, client_id)
|
|
VALUES ($1, now() - interval '5 minutes', now(), 'test') RETURNING id`,
|
|
u.ID).Scan(&sessionID); err != nil {
|
|
t.Fatalf("session: %v", err)
|
|
}
|
|
var peID pgtype.UUID
|
|
if err := pool.QueryRow(ctx,
|
|
`INSERT INTO play_events (user_id, track_id, session_id, started_at, ended_at, duration_played_ms, completion_ratio, was_skipped)
|
|
VALUES ($1, $2, $3, now() - interval '1 minute', now(), $4, $5, false) RETURNING id`,
|
|
u.ID, tr.ID, sessionID, opts.durationMs, opts.ratio).Scan(&peID); err != nil {
|
|
t.Fatalf("play_event: %v", err)
|
|
}
|
|
return setup{pool: pool, q: q, user: u.ID, playEvent: peID}
|
|
}
|
|
|
|
type seedOpts struct {
|
|
lbToken string
|
|
lbEnabled bool
|
|
durationMs int32
|
|
ratio float64
|
|
}
|
|
|
|
func countQueueRows(t *testing.T, s setup) int {
|
|
t.Helper()
|
|
var n int
|
|
if err := s.pool.QueryRow(context.Background(),
|
|
`SELECT count(*) FROM scrobble_queue WHERE play_event_id = $1`, s.playEvent).Scan(&n); err != nil {
|
|
t.Fatalf("count: %v", err)
|
|
}
|
|
return n
|
|
}
|
|
|
|
func TestMaybeEnqueue_QualifyingPlayWithEnabledUser_InsertsRow(t *testing.T) {
|
|
s := seed(t, seedOpts{lbToken: "tk", lbEnabled: true, durationMs: 250_000, ratio: 0.83})
|
|
if err := MaybeEnqueue(context.Background(), s.q, s.playEvent); err != nil {
|
|
t.Fatalf("MaybeEnqueue: %v", err)
|
|
}
|
|
if got := countQueueRows(t, s); got != 1 {
|
|
t.Errorf("rows = %d, want 1", got)
|
|
}
|
|
}
|
|
|
|
func TestMaybeEnqueue_SubThreshold_NoRow(t *testing.T) {
|
|
// 100s, 33% — neither threshold met.
|
|
s := seed(t, seedOpts{lbToken: "tk", lbEnabled: true, durationMs: 100_000, ratio: 0.33})
|
|
if err := MaybeEnqueue(context.Background(), s.q, s.playEvent); err != nil {
|
|
t.Fatalf("MaybeEnqueue: %v", err)
|
|
}
|
|
if got := countQueueRows(t, s); got != 0 {
|
|
t.Errorf("rows = %d, want 0 (sub-threshold)", got)
|
|
}
|
|
}
|
|
|
|
func TestMaybeEnqueue_DisabledUser_NoRow(t *testing.T) {
|
|
s := seed(t, seedOpts{lbToken: "tk", lbEnabled: false, durationMs: 250_000, ratio: 0.83})
|
|
if err := MaybeEnqueue(context.Background(), s.q, s.playEvent); err != nil {
|
|
t.Fatalf("MaybeEnqueue: %v", err)
|
|
}
|
|
if got := countQueueRows(t, s); got != 0 {
|
|
t.Errorf("rows = %d, want 0 (disabled)", got)
|
|
}
|
|
}
|
|
|
|
func TestMaybeEnqueue_NoToken_NoRow(t *testing.T) {
|
|
// Even with enabled=true, an empty token shouldn't enqueue.
|
|
s := seed(t, seedOpts{lbToken: "", lbEnabled: true, durationMs: 250_000, ratio: 0.83})
|
|
if err := MaybeEnqueue(context.Background(), s.q, s.playEvent); err != nil {
|
|
t.Fatalf("MaybeEnqueue: %v", err)
|
|
}
|
|
if got := countQueueRows(t, s); got != 0 {
|
|
t.Errorf("rows = %d, want 0 (no token)", got)
|
|
}
|
|
}
|
|
|
|
func TestMaybeEnqueue_Idempotent_TwoCallsOneRow(t *testing.T) {
|
|
s := seed(t, seedOpts{lbToken: "tk", lbEnabled: true, durationMs: 250_000, ratio: 0.83})
|
|
for i := 0; i < 2; i++ {
|
|
if err := MaybeEnqueue(context.Background(), s.q, s.playEvent); err != nil {
|
|
t.Fatalf("MaybeEnqueue #%d: %v", i, err)
|
|
}
|
|
}
|
|
if got := countQueueRows(t, s); got != 1 {
|
|
t.Errorf("rows = %d, want 1 (idempotent)", got)
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && MINSTREL_TEST_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' go test ./internal/scrobble/ -run MaybeEnqueue -v`
|
|
|
|
Expected: FAIL with "undefined: MaybeEnqueue".
|
|
|
|
- [ ] **Step 3: Write the implementation**
|
|
|
|
Create `internal/scrobble/queue.go`:
|
|
|
|
```go
|
|
package scrobble
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
|
|
"github.com/jackc/pgx/v5"
|
|
"github.com/jackc/pgx/v5/pgtype"
|
|
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
|
|
)
|
|
|
|
// MaybeEnqueue inserts a scrobble_queue row for the given play_event if:
|
|
// 1. The user has listenbrainz_enabled = true with a non-empty token.
|
|
// 2. The play_event passes Qualifies().
|
|
// Idempotent via UNIQUE(play_event_id) — re-runs are no-ops.
|
|
//
|
|
// Best-effort: callers (the playevents.Writer hooks) should log returned
|
|
// errors but not propagate them, since scrobble delivery is enrichment
|
|
// and must not block the canonical play history.
|
|
func MaybeEnqueue(ctx context.Context, q *dbq.Queries, playEventID pgtype.UUID) error {
|
|
pe, err := q.GetPlayEventByID(ctx, playEventID)
|
|
if err != nil {
|
|
if errors.Is(err, pgx.ErrNoRows) {
|
|
return nil
|
|
}
|
|
return err
|
|
}
|
|
// Only closed events qualify (DurationPlayedMs/CompletionRatio populated).
|
|
if pe.DurationPlayedMs == nil || pe.CompletionRatio == nil {
|
|
return nil
|
|
}
|
|
if !Qualifies(int(*pe.DurationPlayedMs), *pe.CompletionRatio) {
|
|
return nil
|
|
}
|
|
cfg, err := q.GetListenBrainzConfig(ctx, pe.UserID)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if !cfg.ListenbrainzEnabled || cfg.ListenbrainzToken == nil || *cfg.ListenbrainzToken == "" {
|
|
return nil
|
|
}
|
|
_, err = q.EnqueueScrobble(ctx, dbq.EnqueueScrobbleParams{
|
|
UserID: pe.UserID,
|
|
PlayEventID: pe.ID,
|
|
})
|
|
return err
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && MINSTREL_TEST_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' go test ./internal/scrobble/ -run MaybeEnqueue -v`
|
|
|
|
Expected: PASS for all 5 tests.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add internal/scrobble/queue.go internal/scrobble/queue_test.go
|
|
git commit -m "feat(scrobble): add MaybeEnqueue with idempotent insert + threshold gate"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 6: Backoff schedule (pure)
|
|
|
|
**Files:**
|
|
- Create: `internal/scrobble/worker.go` (skeleton with backoffDelay only)
|
|
- Create: `internal/scrobble/worker_test.go`
|
|
|
|
- [ ] **Step 1: Write the failing tests**
|
|
|
|
Create `internal/scrobble/worker_test.go`:
|
|
|
|
```go
|
|
package scrobble
|
|
|
|
import (
|
|
"testing"
|
|
"time"
|
|
)
|
|
|
|
func TestBackoffDelay(t *testing.T) {
|
|
cases := []struct {
|
|
attempts int
|
|
want time.Duration
|
|
ok bool
|
|
}{
|
|
{1, 1 * time.Minute, true},
|
|
{2, 5 * time.Minute, true},
|
|
{3, 30 * time.Minute, true},
|
|
{4, 2 * time.Hour, true},
|
|
{5, 6 * time.Hour, true},
|
|
{6, 0, false}, // exhausted
|
|
{99, 0, false}, // way past max
|
|
}
|
|
for _, c := range cases {
|
|
got, ok := backoffDelay(c.attempts)
|
|
if ok != c.ok {
|
|
t.Errorf("backoffDelay(%d) ok = %v, want %v", c.attempts, ok, c.ok)
|
|
}
|
|
if got != c.want {
|
|
t.Errorf("backoffDelay(%d) = %v, want %v", c.attempts, got, c.want)
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && go test ./internal/scrobble/ -run BackoffDelay -v`
|
|
|
|
Expected: FAIL with "undefined: backoffDelay".
|
|
|
|
- [ ] **Step 3: Write the worker skeleton with backoffDelay**
|
|
|
|
Create `internal/scrobble/worker.go`:
|
|
|
|
```go
|
|
package scrobble
|
|
|
|
import (
|
|
"time"
|
|
)
|
|
|
|
// backoffSchedule maps the failure count (1-indexed: 1 = first failure
|
|
// just happened) to the delay before the next attempt. Per spec §9.6:
|
|
// 1m → 5m → 30m → 2h → 6h, then give up.
|
|
var backoffSchedule = []time.Duration{
|
|
1 * time.Minute,
|
|
5 * time.Minute,
|
|
30 * time.Minute,
|
|
2 * time.Hour,
|
|
6 * time.Hour,
|
|
}
|
|
|
|
const maxAttempts = 5
|
|
|
|
// backoffDelay returns the delay before the next attempt given the value
|
|
// of the `attempts` column AFTER the failure has been recorded (1-indexed).
|
|
// Returns (_, false) when attempts > maxAttempts (give up — the row should
|
|
// be marked failed).
|
|
func backoffDelay(attempts int) (time.Duration, bool) {
|
|
if attempts < 1 || attempts > maxAttempts {
|
|
return 0, false
|
|
}
|
|
return backoffSchedule[attempts-1], true
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && go test ./internal/scrobble/ -run BackoffDelay -v`
|
|
|
|
Expected: PASS for all cases.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add internal/scrobble/worker.go internal/scrobble/worker_test.go
|
|
git commit -m "feat(scrobble): add backoffDelay schedule (1m, 5m, 30m, 2h, 6h, give up)"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 7: Worker tickOnce (DB-backed)
|
|
|
|
**Files:**
|
|
- Modify: `internal/scrobble/worker.go` (add Worker struct, NewWorker, tickOnce, Run)
|
|
- Create: `internal/scrobble/worker_integration_test.go`
|
|
|
|
- [ ] **Step 1: Write the failing integration tests**
|
|
|
|
Create `internal/scrobble/worker_integration_test.go`:
|
|
|
|
```go
|
|
package scrobble
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"strings"
|
|
"sync/atomic"
|
|
"testing"
|
|
"time"
|
|
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble/listenbrainz"
|
|
)
|
|
|
|
// helperEnqueue inserts a queue row directly so we don't go through the threshold path.
|
|
func helperEnqueue(t *testing.T, s setup) {
|
|
t.Helper()
|
|
if _, err := s.pool.Exec(context.Background(),
|
|
`INSERT INTO scrobble_queue (user_id, play_event_id, status) VALUES ($1, $2, 'pending')`,
|
|
s.user, s.playEvent); err != nil {
|
|
t.Fatalf("enqueue: %v", err)
|
|
}
|
|
}
|
|
|
|
func helperPendingCount(t *testing.T, s setup) int {
|
|
t.Helper()
|
|
var n int
|
|
if err := s.pool.QueryRow(context.Background(),
|
|
`SELECT count(*) FROM scrobble_queue WHERE status = 'pending' AND play_event_id = $1`, s.playEvent).Scan(&n); err != nil {
|
|
t.Fatalf("pending: %v", err)
|
|
}
|
|
return n
|
|
}
|
|
|
|
func helperFailedCount(t *testing.T, s setup) int {
|
|
t.Helper()
|
|
var n int
|
|
if err := s.pool.QueryRow(context.Background(),
|
|
`SELECT count(*) FROM scrobble_queue WHERE status = 'failed' AND play_event_id = $1`, s.playEvent).Scan(&n); err != nil {
|
|
t.Fatalf("failed: %v", err)
|
|
}
|
|
return n
|
|
}
|
|
|
|
func helperPlayEventScrobbledAt(t *testing.T, s setup) bool {
|
|
t.Helper()
|
|
var v *time.Time
|
|
if err := s.pool.QueryRow(context.Background(),
|
|
`SELECT scrobbled_at FROM play_events WHERE id = $1`, s.playEvent).Scan(&v); err != nil {
|
|
t.Fatalf("scrobbled_at: %v", err)
|
|
}
|
|
return v != nil
|
|
}
|
|
|
|
func newTestWorker(s setup, lb *listenbrainz.Client) *Worker {
|
|
return NewWorker(s.pool, lb, helperLogger())
|
|
}
|
|
|
|
func helperLogger() *slog.Logger { return slog.New(slog.NewTextHandler(io.Discard, nil)) }
|
|
|
|
func TestWorker_Success_DeletesQueueRowAndStampsScrobbledAt(t *testing.T) {
|
|
s := seed(t, seedOpts{lbToken: "tk", lbEnabled: true, durationMs: 250_000, ratio: 0.83})
|
|
helperEnqueue(t, s)
|
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
w.WriteHeader(http.StatusOK)
|
|
}))
|
|
defer srv.Close()
|
|
lb := &listenbrainz.Client{BaseURL: srv.URL, HTTP: srv.Client()}
|
|
w := newTestWorker(s, lb)
|
|
|
|
if err := w.tickOnce(context.Background()); err != nil {
|
|
t.Fatalf("tickOnce: %v", err)
|
|
}
|
|
if helperPendingCount(t, s) != 0 {
|
|
t.Error("queue row should be deleted on success")
|
|
}
|
|
if helperFailedCount(t, s) != 0 {
|
|
t.Error("no failed row expected on success")
|
|
}
|
|
if !helperPlayEventScrobbledAt(t, s) {
|
|
t.Error("play_events.scrobbled_at should be populated on success")
|
|
}
|
|
}
|
|
|
|
func TestWorker_503Once_RowRemainsAttempts1(t *testing.T) {
|
|
s := seed(t, seedOpts{lbToken: "tk", lbEnabled: true, durationMs: 250_000, ratio: 0.83})
|
|
helperEnqueue(t, s)
|
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
w.WriteHeader(http.StatusServiceUnavailable)
|
|
}))
|
|
defer srv.Close()
|
|
lb := &listenbrainz.Client{BaseURL: srv.URL, HTTP: srv.Client()}
|
|
w := newTestWorker(s, lb)
|
|
_ = w.tickOnce(context.Background())
|
|
|
|
var attempts int
|
|
if err := s.pool.QueryRow(context.Background(),
|
|
`SELECT attempts FROM scrobble_queue WHERE play_event_id = $1`, s.playEvent).Scan(&attempts); err != nil {
|
|
t.Fatalf("attempts: %v", err)
|
|
}
|
|
if attempts != 1 {
|
|
t.Errorf("attempts = %d, want 1 after 503", attempts)
|
|
}
|
|
if helperPendingCount(t, s) != 1 {
|
|
t.Error("row should remain pending after transient failure")
|
|
}
|
|
}
|
|
|
|
func TestWorker_503FiveTimes_MarksFailed(t *testing.T) {
|
|
s := seed(t, seedOpts{lbToken: "tk", lbEnabled: true, durationMs: 250_000, ratio: 0.83})
|
|
helperEnqueue(t, s)
|
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
w.WriteHeader(http.StatusServiceUnavailable)
|
|
}))
|
|
defer srv.Close()
|
|
lb := &listenbrainz.Client{BaseURL: srv.URL, HTTP: srv.Client()}
|
|
w := newTestWorker(s, lb)
|
|
// Five failures: each tick marches next_attempt_at forward, so we need to
|
|
// reset it between ticks to simulate the time passing.
|
|
for i := 0; i < 5; i++ {
|
|
if _, err := s.pool.Exec(context.Background(),
|
|
`UPDATE scrobble_queue SET next_attempt_at = now() WHERE play_event_id = $1`, s.playEvent); err != nil {
|
|
t.Fatalf("reset: %v", err)
|
|
}
|
|
_ = w.tickOnce(context.Background())
|
|
}
|
|
if helperFailedCount(t, s) != 1 {
|
|
t.Errorf("expected row marked failed after 5 attempts; pending=%d failed=%d",
|
|
helperPendingCount(t, s), helperFailedCount(t, s))
|
|
}
|
|
var attempts int
|
|
var lastErr *string
|
|
_ = s.pool.QueryRow(context.Background(),
|
|
`SELECT attempts, last_error FROM scrobble_queue WHERE play_event_id = $1`, s.playEvent).Scan(&attempts, &lastErr)
|
|
if attempts != 5 {
|
|
t.Errorf("attempts = %d, want 5", attempts)
|
|
}
|
|
if lastErr == nil || *lastErr == "" {
|
|
t.Error("last_error should be populated")
|
|
}
|
|
}
|
|
|
|
func TestWorker_401_MarksFailedAndDisablesUser(t *testing.T) {
|
|
s := seed(t, seedOpts{lbToken: "tk", lbEnabled: true, durationMs: 250_000, ratio: 0.83})
|
|
helperEnqueue(t, s)
|
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
w.WriteHeader(http.StatusUnauthorized)
|
|
}))
|
|
defer srv.Close()
|
|
lb := &listenbrainz.Client{BaseURL: srv.URL, HTTP: srv.Client()}
|
|
w := newTestWorker(s, lb)
|
|
_ = w.tickOnce(context.Background())
|
|
|
|
if helperFailedCount(t, s) != 1 {
|
|
t.Error("401 should mark failed immediately")
|
|
}
|
|
cfg, _ := s.q.GetListenBrainzConfig(context.Background(), s.user)
|
|
if cfg.ListenbrainzEnabled {
|
|
t.Error("user listenbrainz_enabled should be set to false on auth failure")
|
|
}
|
|
}
|
|
|
|
func TestWorker_429_RespectsRetryAfterWithoutIncrementingAttempts(t *testing.T) {
|
|
s := seed(t, seedOpts{lbToken: "tk", lbEnabled: true, durationMs: 250_000, ratio: 0.83})
|
|
helperEnqueue(t, s)
|
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
w.Header().Set("Retry-After", "300")
|
|
w.WriteHeader(http.StatusTooManyRequests)
|
|
}))
|
|
defer srv.Close()
|
|
lb := &listenbrainz.Client{BaseURL: srv.URL, HTTP: srv.Client()}
|
|
w := newTestWorker(s, lb)
|
|
_ = w.tickOnce(context.Background())
|
|
|
|
var attempts int
|
|
var nextAttemptAt time.Time
|
|
_ = s.pool.QueryRow(context.Background(),
|
|
`SELECT attempts, next_attempt_at FROM scrobble_queue WHERE play_event_id = $1`, s.playEvent).
|
|
Scan(&attempts, &nextAttemptAt)
|
|
if attempts != 0 {
|
|
t.Errorf("attempts = %d, want 0 (429 should not increment)", attempts)
|
|
}
|
|
if d := time.Until(nextAttemptAt); d < 250*time.Second || d > 350*time.Second {
|
|
t.Errorf("next_attempt_at = +%v, want ~+5m (Retry-After 300)", d)
|
|
}
|
|
}
|
|
|
|
func TestWorker_BatchOfTen_OnePost(t *testing.T) {
|
|
s := seed(t, seedOpts{lbToken: "tk", lbEnabled: true, durationMs: 250_000, ratio: 0.83})
|
|
// Enqueue 10 distinct play_events to test batching.
|
|
for i := 0; i < 10; i++ {
|
|
var newPE pgtype.UUID
|
|
if err := s.pool.QueryRow(context.Background(),
|
|
`INSERT INTO play_events (user_id, track_id, session_id, started_at, ended_at, duration_played_ms, completion_ratio, was_skipped)
|
|
SELECT user_id, track_id, session_id, now(), now(), duration_played_ms, completion_ratio, was_skipped FROM play_events WHERE id = $1
|
|
RETURNING id`, s.playEvent).Scan(&newPE); err != nil {
|
|
t.Fatalf("dup play_event: %v", err)
|
|
}
|
|
if _, err := s.pool.Exec(context.Background(),
|
|
`INSERT INTO scrobble_queue (user_id, play_event_id, status) VALUES ($1, $2, 'pending')`,
|
|
s.user, newPE); err != nil {
|
|
t.Fatalf("enqueue: %v", err)
|
|
}
|
|
}
|
|
var posts atomic.Int32
|
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
posts.Add(1)
|
|
w.WriteHeader(http.StatusOK)
|
|
}))
|
|
defer srv.Close()
|
|
lb := &listenbrainz.Client{BaseURL: srv.URL, HTTP: srv.Client()}
|
|
w := newTestWorker(s, lb)
|
|
_ = w.tickOnce(context.Background())
|
|
|
|
if posts.Load() != 1 {
|
|
t.Errorf("LB POSTs = %d, want 1 (batch)", posts.Load())
|
|
}
|
|
}
|
|
```
|
|
|
|
Note: this test file imports `io`, `log/slog`, `pgtype` — make sure to add those imports at the top of the file. The exact import block:
|
|
|
|
```go
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"io"
|
|
"log/slog"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"strings"
|
|
"sync/atomic"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/jackc/pgx/v5/pgtype"
|
|
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble/listenbrainz"
|
|
)
|
|
```
|
|
|
|
(The `errors` and `strings` imports may be unused — remove if Go compiler complains.)
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && MINSTREL_TEST_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' go test ./internal/scrobble/ -run Worker -v`
|
|
|
|
Expected: FAIL with "undefined: NewWorker" and "undefined: tickOnce".
|
|
|
|
- [ ] **Step 3: Implement Worker, NewWorker, tickOnce, Run**
|
|
|
|
Replace `internal/scrobble/worker.go` contents:
|
|
|
|
```go
|
|
package scrobble
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"log/slog"
|
|
"time"
|
|
|
|
"github.com/jackc/pgx/v5/pgtype"
|
|
"github.com/jackc/pgx/v5/pgxpool"
|
|
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble/listenbrainz"
|
|
)
|
|
|
|
// backoffSchedule maps the failure count (1-indexed) to the delay before
|
|
// the next attempt. Per spec §9.6: 1m → 5m → 30m → 2h → 6h, then give up.
|
|
var backoffSchedule = []time.Duration{
|
|
1 * time.Minute,
|
|
5 * time.Minute,
|
|
30 * time.Minute,
|
|
2 * time.Hour,
|
|
6 * time.Hour,
|
|
}
|
|
|
|
const maxAttempts = 5
|
|
|
|
// backoffDelay returns the delay before the next attempt given the value
|
|
// of the `attempts` column AFTER the failure has been recorded (1-indexed).
|
|
func backoffDelay(attempts int) (time.Duration, bool) {
|
|
if attempts < 1 || attempts > maxAttempts {
|
|
return 0, false
|
|
}
|
|
return backoffSchedule[attempts-1], true
|
|
}
|
|
|
|
// Worker drains scrobble_queue rows and POSTs them to ListenBrainz.
|
|
type Worker struct {
|
|
pool *pgxpool.Pool
|
|
client *listenbrainz.Client
|
|
logger *slog.Logger
|
|
tick time.Duration
|
|
batch int32
|
|
}
|
|
|
|
// NewWorker constructs a worker with production defaults: 30s tick, 50-row
|
|
// batch.
|
|
func NewWorker(pool *pgxpool.Pool, client *listenbrainz.Client, logger *slog.Logger) *Worker {
|
|
return &Worker{pool: pool, client: client, logger: logger, tick: 30 * time.Second, batch: 50}
|
|
}
|
|
|
|
// Run blocks until ctx is cancelled, ticking every w.tick.
|
|
func (w *Worker) Run(ctx context.Context) {
|
|
t := time.NewTicker(w.tick)
|
|
defer t.Stop()
|
|
for {
|
|
select {
|
|
case <-ctx.Done():
|
|
return
|
|
case <-t.C:
|
|
if err := w.tickOnce(ctx); err != nil {
|
|
w.logger.Error("scrobble: tick failed", "err", err)
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
// tickOnce drains up to w.batch pending rows. Returns the first error
|
|
// encountered loading rows; per-row errors are logged and don't abort
|
|
// the batch.
|
|
func (w *Worker) tickOnce(ctx context.Context) error {
|
|
q := dbq.New(w.pool)
|
|
rows, err := q.ListPendingScrobbles(ctx, w.batch)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if len(rows) == 0 {
|
|
return nil
|
|
}
|
|
|
|
// Group rows by user (each user has their own token + may produce its
|
|
// own batch).
|
|
type userBatch struct {
|
|
token string
|
|
queueIDs []pgtype.UUID
|
|
listens []listenbrainz.Listen
|
|
userID pgtype.UUID
|
|
playIDs []pgtype.UUID
|
|
}
|
|
batches := map[string]*userBatch{}
|
|
for _, r := range rows {
|
|
if r.LbToken == nil || *r.LbToken == "" || !r.LbEnabled {
|
|
// Defensive: row got into queue but user since disabled. Skip.
|
|
continue
|
|
}
|
|
key := r.UserID.String()
|
|
ub := batches[key]
|
|
if ub == nil {
|
|
ub = &userBatch{token: *r.LbToken, userID: r.UserID}
|
|
batches[key] = ub
|
|
}
|
|
ub.queueIDs = append(ub.queueIDs, r.QueueID)
|
|
ub.playIDs = append(ub.playIDs, r.PlayEventID)
|
|
l := listenbrainz.Listen{
|
|
ListenedAt: r.StartedAt.Time.Unix(),
|
|
Track: listenbrainz.Track{
|
|
ArtistName: r.ArtistName,
|
|
TrackName: r.TrackTitle,
|
|
ReleaseName: r.AlbumTitle,
|
|
DurationMs: int(r.TrackDurationMs),
|
|
},
|
|
}
|
|
if r.TrackMbid != nil {
|
|
l.Track.RecordingMBID = *r.TrackMbid
|
|
}
|
|
if r.AlbumMbid != nil {
|
|
l.Track.ReleaseMBID = *r.AlbumMbid
|
|
}
|
|
if r.ArtistMbid != nil {
|
|
l.Track.ArtistMBIDs = []string{*r.ArtistMbid}
|
|
}
|
|
ub.listens = append(ub.listens, l)
|
|
}
|
|
|
|
for _, ub := range batches {
|
|
w.processBatch(ctx, q, ub.userID, ub.token, ub.queueIDs, ub.listens)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// processBatch submits one user's pending batch and updates queue rows
|
|
// according to the response.
|
|
func (w *Worker) processBatch(
|
|
ctx context.Context,
|
|
q *dbq.Queries,
|
|
userID pgtype.UUID,
|
|
token string,
|
|
queueIDs []pgtype.UUID,
|
|
listens []listenbrainz.Listen,
|
|
) {
|
|
err := w.client.SubmitListens(ctx, token, listens)
|
|
now := time.Now().UTC()
|
|
|
|
if err == nil {
|
|
for _, id := range queueIDs {
|
|
if uerr := q.MarkScrobbleSent(ctx, id); uerr != nil {
|
|
w.logger.Error("scrobble: MarkScrobbleSent", "queue_id", id, "err", uerr)
|
|
}
|
|
}
|
|
return
|
|
}
|
|
|
|
// 429: schedule retry per Retry-After, do NOT increment attempts.
|
|
var ra *listenbrainz.RetryAfterError
|
|
if errors.As(err, &ra) {
|
|
next := pgtype.Timestamptz{Time: now.Add(ra.Wait), Valid: true}
|
|
for _, id := range queueIDs {
|
|
if uerr := q.MarkScrobbleRetryAfter(ctx, dbq.MarkScrobbleRetryAfterParams{
|
|
ID: id, NextAttemptAt: next,
|
|
}); uerr != nil {
|
|
w.logger.Error("scrobble: MarkScrobbleRetryAfter", "err", uerr)
|
|
}
|
|
}
|
|
return
|
|
}
|
|
|
|
// 401: mark failed AND disable user (defensive — bad token shouldn't keep
|
|
// retrying or feed future rows).
|
|
if errors.Is(err, listenbrainz.ErrAuth) {
|
|
w.logger.Warn("scrobble: auth rejected; disabling user listenbrainz", "user_id", userID)
|
|
if uerr := q.SetListenBrainzEnabled(ctx, dbq.SetListenBrainzEnabledParams{
|
|
ID: userID, ListenbrainzEnabled: false,
|
|
}); uerr != nil {
|
|
w.logger.Error("scrobble: SetListenBrainzEnabled", "err", uerr)
|
|
}
|
|
errStr := err.Error()
|
|
for _, id := range queueIDs {
|
|
if uerr := q.MarkScrobbleFailed(ctx, dbq.MarkScrobbleFailedParams{
|
|
ID: id, LastError: &errStr,
|
|
}); uerr != nil {
|
|
w.logger.Error("scrobble: MarkScrobbleFailed", "err", uerr)
|
|
}
|
|
}
|
|
return
|
|
}
|
|
|
|
// 4xx: permanent. Mark failed, no retry.
|
|
if errors.Is(err, listenbrainz.ErrPermanent) {
|
|
errStr := err.Error()
|
|
for _, id := range queueIDs {
|
|
if uerr := q.MarkScrobbleFailed(ctx, dbq.MarkScrobbleFailedParams{
|
|
ID: id, LastError: &errStr,
|
|
}); uerr != nil {
|
|
w.logger.Error("scrobble: MarkScrobbleFailed", "err", uerr)
|
|
}
|
|
}
|
|
return
|
|
}
|
|
|
|
// Transient (5xx, network): increment attempts, schedule next or give up.
|
|
// Each row in the batch shares the same fate since they were submitted
|
|
// together — we update each independently using its current attempts+1.
|
|
errStr := err.Error()
|
|
for _, id := range queueIDs {
|
|
// Read current attempts to compute next state.
|
|
var attempts int32
|
|
if rerr := w.pool.QueryRow(ctx,
|
|
`SELECT attempts FROM scrobble_queue WHERE id = $1`, id).Scan(&attempts); rerr != nil {
|
|
w.logger.Error("scrobble: read attempts", "err", rerr)
|
|
continue
|
|
}
|
|
next := attempts + 1
|
|
if delay, ok := backoffDelay(int(next)); ok {
|
|
nextAt := pgtype.Timestamptz{Time: now.Add(delay), Valid: true}
|
|
if uerr := q.RescheduleScrobble(ctx, dbq.RescheduleScrobbleParams{
|
|
ID: id, NextAttemptAt: nextAt, LastError: &errStr,
|
|
}); uerr != nil {
|
|
w.logger.Error("scrobble: RescheduleScrobble", "err", uerr)
|
|
}
|
|
} else {
|
|
if uerr := q.MarkScrobbleFailed(ctx, dbq.MarkScrobbleFailedParams{
|
|
ID: id, LastError: &errStr,
|
|
}); uerr != nil {
|
|
w.logger.Error("scrobble: MarkScrobbleFailed", "err", uerr)
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run all scrobble tests**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && MINSTREL_TEST_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' go test ./internal/scrobble/... -v`
|
|
|
|
Expected: PASS for all worker tests + previously-passing tests in `listenbrainz/`, `threshold_test.go`, `queue_test.go`, `worker_test.go`.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add internal/scrobble/worker.go internal/scrobble/worker_integration_test.go
|
|
git commit -m "feat(scrobble): add Worker with tickOnce, batched submit, retry/fail/auth handling"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 8: Hook MaybeEnqueue into playevents.Writer
|
|
|
|
**Files:**
|
|
- Modify: `internal/playevents/writer.go`
|
|
- Modify: `internal/playevents/writer_test.go` (or a sibling file, depending on existing structure)
|
|
|
|
- [ ] **Step 1: Write the failing test**
|
|
|
|
Append to `internal/playevents/writer_test.go` (or create one if missing):
|
|
|
|
```go
|
|
func TestRecordPlayEnded_QualifyingPlay_EnqueuesScrobble(t *testing.T) {
|
|
if testing.Short() {
|
|
t.Skip("integration test")
|
|
}
|
|
dsn := os.Getenv("MINSTREL_TEST_DATABASE_URL")
|
|
if dsn == "" {
|
|
t.Skip("MINSTREL_TEST_DATABASE_URL not set")
|
|
}
|
|
pool, q := newTestPool(t, dsn) // assume helper from existing writer_test.go
|
|
defer pool.Close()
|
|
logger := slog.New(slog.NewTextHandler(io.Discard, nil))
|
|
w := NewWriter(pool, logger, 30*time.Minute, 0.5, 30000)
|
|
|
|
user, track := seedUserAndTrack(t, q, "tester", 300_000) // helper, set duration=300s
|
|
|
|
// Enable LB.
|
|
tk := "tk"
|
|
if err := q.SetListenBrainzToken(context.Background(), dbq.SetListenBrainzTokenParams{ID: user.ID, ListenbrainzToken: &tk}); err != nil {
|
|
t.Fatalf("token: %v", err)
|
|
}
|
|
if err := q.SetListenBrainzEnabled(context.Background(), dbq.SetListenBrainzEnabledParams{ID: user.ID, ListenbrainzEnabled: true}); err != nil {
|
|
t.Fatalf("enable: %v", err)
|
|
}
|
|
|
|
// Start + end with 250s played (qualifies).
|
|
res, err := w.RecordPlayStarted(context.Background(), user.ID, track.ID, "test", time.Now().Add(-1*time.Minute))
|
|
if err != nil {
|
|
t.Fatalf("start: %v", err)
|
|
}
|
|
if err := w.RecordPlayEnded(context.Background(), res.PlayEventID, 250_000, time.Now()); err != nil {
|
|
t.Fatalf("end: %v", err)
|
|
}
|
|
|
|
// Allow the post-commit goroutine (if any) to settle. Currently MaybeEnqueue
|
|
// is called synchronously after BeginFunc returns, so a sleep isn't needed —
|
|
// but if we ever switch to a goroutine, this gives it time.
|
|
time.Sleep(50 * time.Millisecond)
|
|
|
|
var n int
|
|
if err := pool.QueryRow(context.Background(),
|
|
`SELECT count(*) FROM scrobble_queue WHERE play_event_id = $1`, res.PlayEventID).Scan(&n); err != nil {
|
|
t.Fatalf("count: %v", err)
|
|
}
|
|
if n != 1 {
|
|
t.Errorf("scrobble_queue rows = %d, want 1", n)
|
|
}
|
|
}
|
|
|
|
func TestRecordPlayEnded_SubThreshold_NoEnqueue(t *testing.T) {
|
|
if testing.Short() {
|
|
t.Skip("integration test")
|
|
}
|
|
dsn := os.Getenv("MINSTREL_TEST_DATABASE_URL")
|
|
if dsn == "" {
|
|
t.Skip("MINSTREL_TEST_DATABASE_URL not set")
|
|
}
|
|
pool, q := newTestPool(t, dsn)
|
|
defer pool.Close()
|
|
logger := slog.New(slog.NewTextHandler(io.Discard, nil))
|
|
w := NewWriter(pool, logger, 30*time.Minute, 0.5, 30000)
|
|
user, track := seedUserAndTrack(t, q, "tester", 300_000)
|
|
|
|
tk := "tk"
|
|
_ = q.SetListenBrainzToken(context.Background(), dbq.SetListenBrainzTokenParams{ID: user.ID, ListenbrainzToken: &tk})
|
|
_ = q.SetListenBrainzEnabled(context.Background(), dbq.SetListenBrainzEnabledParams{ID: user.ID, ListenbrainzEnabled: true})
|
|
|
|
res, _ := w.RecordPlayStarted(context.Background(), user.ID, track.ID, "test", time.Now().Add(-1*time.Minute))
|
|
_ = w.RecordPlayEnded(context.Background(), res.PlayEventID, 60_000, time.Now()) // 20%, well below threshold
|
|
|
|
var n int
|
|
_ = pool.QueryRow(context.Background(),
|
|
`SELECT count(*) FROM scrobble_queue WHERE play_event_id = $1`, res.PlayEventID).Scan(&n)
|
|
if n != 0 {
|
|
t.Errorf("expected 0 queue rows for sub-threshold play, got %d", n)
|
|
}
|
|
}
|
|
```
|
|
|
|
If `seedUserAndTrack` and `newTestPool` aren't already in `writer_test.go`, look for the existing fixture functions (the existing tests must use something equivalent) and reuse those. If they're truly absent, write minimal versions following the pattern from `internal/scrobble/queue_test.go::seed`.
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && MINSTREL_TEST_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' go test ./internal/playevents/ -run RecordPlayEnded.Scrobble -v`
|
|
|
|
Expected: tests fail because no enqueue hook exists yet.
|
|
|
|
- [ ] **Step 3: Add the post-commit enqueue hook**
|
|
|
|
In `internal/playevents/writer.go`, modify `RecordPlayEnded` to add the post-commit `MaybeEnqueue` call:
|
|
|
|
```go
|
|
import (
|
|
// existing imports …
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble"
|
|
)
|
|
|
|
func (w *Writer) RecordPlayEnded(
|
|
ctx context.Context,
|
|
playEventID pgtype.UUID,
|
|
durationPlayedMs int32,
|
|
at time.Time,
|
|
) error {
|
|
if err := pgx.BeginFunc(ctx, w.pool, func(tx pgx.Tx) error {
|
|
// … existing close logic …
|
|
}); err != nil {
|
|
return err
|
|
}
|
|
|
|
// Post-commit best-effort scrobble enqueue. Errors logged + swallowed:
|
|
// scrobble delivery is enrichment, not a precondition for canonical play
|
|
// history.
|
|
if err := scrobble.MaybeEnqueue(ctx, dbq.New(w.pool), playEventID); err != nil {
|
|
w.logger.Warn("playevents: scrobble enqueue failed", "play_event_id", playEventID, "err", err)
|
|
}
|
|
return nil
|
|
}
|
|
```
|
|
|
|
Apply the **same pattern** to `RecordSyntheticCompletedPlay`: keep the existing `pgx.BeginFunc`, capture the inserted play_event ID, and after the commit call `scrobble.MaybeEnqueue(ctx, dbq.New(w.pool), evID)`. The existing function already creates and ends the play_event in one transaction; you'll need to surface the ID up out of the closure into a local variable before the post-commit call.
|
|
|
|
The exact diff for `RecordSyntheticCompletedPlay`:
|
|
|
|
```go
|
|
func (w *Writer) RecordSyntheticCompletedPlay(
|
|
ctx context.Context,
|
|
userID, trackID pgtype.UUID,
|
|
clientID string,
|
|
at time.Time,
|
|
) error {
|
|
var playEventID pgtype.UUID
|
|
if err := pgx.BeginFunc(ctx, w.pool, func(tx pgx.Tx) error {
|
|
// … existing logic, but assign ev.ID to playEventID before returning …
|
|
// e.g. after `ev, err := q.InsertPlayEvent(...)` succeeds:
|
|
// playEventID = ev.ID
|
|
}); err != nil {
|
|
return err
|
|
}
|
|
if err := scrobble.MaybeEnqueue(ctx, dbq.New(w.pool), playEventID); err != nil {
|
|
w.logger.Warn("playevents: scrobble enqueue failed", "play_event_id", playEventID, "err", err)
|
|
}
|
|
return nil
|
|
}
|
|
```
|
|
|
|
(Inspect the current `RecordSyntheticCompletedPlay` body and add `playEventID = ev.ID` immediately after `q.InsertPlayEvent` returns. The function ends the play_event in the same transaction; a single playEventID variable captured pre-commit is correct.)
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && MINSTREL_TEST_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' go test ./internal/playevents/ -v`
|
|
|
|
Expected: PASS for all existing tests + 2 new scrobble-enqueue tests.
|
|
|
|
- [ ] **Step 5: Verify Subsonic scrobble path also passes**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && MINSTREL_TEST_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' go test ./internal/subsonic/ -v`
|
|
|
|
Expected: PASS. Pre-existing scrobble integration tests still pass; the post-commit enqueue is a no-op for users without LB enabled.
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add internal/playevents/writer.go internal/playevents/writer_test.go
|
|
git commit -m "feat(playevents): post-commit MaybeEnqueue in RecordPlayEnded + Synthetic"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 9: API endpoints — GET/PUT /api/me/listenbrainz
|
|
|
|
**Files:**
|
|
- Create: `internal/api/listenbrainz.go`
|
|
- Create: `internal/api/listenbrainz_test.go`
|
|
- Modify: `internal/api/api.go::Mount`
|
|
|
|
- [ ] **Step 1: Write the failing tests**
|
|
|
|
Create `internal/api/listenbrainz_test.go`:
|
|
|
|
```go
|
|
package api
|
|
|
|
import (
|
|
"bytes"
|
|
"context"
|
|
"encoding/json"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"testing"
|
|
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
|
|
)
|
|
|
|
type lbStatusJSON struct {
|
|
Enabled bool `json:"enabled"`
|
|
TokenSet bool `json:"token_set"`
|
|
LastScrobbledAt *string `json:"last_scrobbled_at"`
|
|
}
|
|
|
|
func callGetLB(h *handlers, user dbq.User) *httptest.ResponseRecorder {
|
|
req := httptest.NewRequest(http.MethodGet, "/api/me/listenbrainz", nil)
|
|
req = req.WithContext(auth.WithUser(req.Context(), user))
|
|
w := httptest.NewRecorder()
|
|
h.handleGetListenBrainz(w, req)
|
|
return w
|
|
}
|
|
|
|
func callPutLB(h *handlers, user dbq.User, body any) *httptest.ResponseRecorder {
|
|
buf, _ := json.Marshal(body)
|
|
req := httptest.NewRequest(http.MethodPut, "/api/me/listenbrainz", bytes.NewReader(buf))
|
|
req.Header.Set("Content-Type", "application/json")
|
|
req = req.WithContext(auth.WithUser(req.Context(), user))
|
|
w := httptest.NewRecorder()
|
|
h.handlePutListenBrainz(w, req)
|
|
return w
|
|
}
|
|
|
|
func TestHandleGetListenBrainz_NoTokenInitial(t *testing.T) {
|
|
h, pool := testHandlers(t)
|
|
truncateLibrary(t, pool)
|
|
u := seedUser(t, pool, "alice", "x", false)
|
|
w := callGetLB(h, u)
|
|
if w.Code != http.StatusOK {
|
|
t.Fatalf("status = %d", w.Code)
|
|
}
|
|
var resp lbStatusJSON
|
|
_ = json.Unmarshal(w.Body.Bytes(), &resp)
|
|
if resp.Enabled || resp.TokenSet || resp.LastScrobbledAt != nil {
|
|
t.Errorf("initial state wrong: %+v", resp)
|
|
}
|
|
}
|
|
|
|
func TestHandlePutListenBrainz_SetTokenThenGet(t *testing.T) {
|
|
h, pool := testHandlers(t)
|
|
truncateLibrary(t, pool)
|
|
u := seedUser(t, pool, "alice", "x", false)
|
|
|
|
put := callPutLB(h, u, map[string]any{"token": "abc"})
|
|
if put.Code != http.StatusOK {
|
|
t.Fatalf("PUT status = %d body = %s", put.Code, put.Body.String())
|
|
}
|
|
|
|
get := callGetLB(h, u)
|
|
var resp lbStatusJSON
|
|
_ = json.Unmarshal(get.Body.Bytes(), &resp)
|
|
if !resp.TokenSet {
|
|
t.Error("TokenSet = false after PUT, want true")
|
|
}
|
|
}
|
|
|
|
func TestHandlePutListenBrainz_ClearTokenForcesEnabledFalse(t *testing.T) {
|
|
h, pool := testHandlers(t)
|
|
truncateLibrary(t, pool)
|
|
u := seedUser(t, pool, "alice", "x", false)
|
|
|
|
_ = callPutLB(h, u, map[string]any{"token": "abc"})
|
|
_ = callPutLB(h, u, map[string]any{"enabled": true})
|
|
clear := callPutLB(h, u, map[string]any{"token": ""})
|
|
if clear.Code != http.StatusOK {
|
|
t.Fatalf("clear status = %d", clear.Code)
|
|
}
|
|
|
|
get := callGetLB(h, u)
|
|
var resp lbStatusJSON
|
|
_ = json.Unmarshal(get.Body.Bytes(), &resp)
|
|
if resp.TokenSet {
|
|
t.Error("TokenSet = true after clearing")
|
|
}
|
|
if resp.Enabled {
|
|
t.Error("Enabled = true after clearing token, want false (force-cleared)")
|
|
}
|
|
}
|
|
|
|
func TestHandlePutListenBrainz_EnableWithoutToken_400(t *testing.T) {
|
|
h, pool := testHandlers(t)
|
|
truncateLibrary(t, pool)
|
|
u := seedUser(t, pool, "alice", "x", false)
|
|
resp := callPutLB(h, u, map[string]any{"enabled": true})
|
|
if resp.Code != http.StatusBadRequest {
|
|
t.Errorf("status = %d, want 400 (cannot enable without token)", resp.Code)
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && MINSTREL_TEST_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' go test ./internal/api/ -run ListenBrainz -v`
|
|
|
|
Expected: FAIL — handlers undefined.
|
|
|
|
- [ ] **Step 3: Implement the handlers**
|
|
|
|
Create `internal/api/listenbrainz.go`:
|
|
|
|
```go
|
|
package api
|
|
|
|
import (
|
|
"encoding/json"
|
|
"errors"
|
|
"net/http"
|
|
|
|
"github.com/jackc/pgx/v5"
|
|
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
|
|
)
|
|
|
|
// LBStatus is the GET response body and a subset of the PUT response.
|
|
type LBStatus struct {
|
|
Enabled bool `json:"enabled"`
|
|
TokenSet bool `json:"token_set"`
|
|
LastScrobbledAt *string `json:"last_scrobbled_at"`
|
|
}
|
|
|
|
func (h *handlers) handleGetListenBrainz(w http.ResponseWriter, r *http.Request) {
|
|
user, ok := auth.UserFromContext(r.Context())
|
|
if !ok {
|
|
writeErr(w, http.StatusUnauthorized, "unauthorized", "authentication required")
|
|
return
|
|
}
|
|
resp, err := buildLBStatus(r, h, user.ID)
|
|
if err != nil {
|
|
h.logger.Error("api: get listenbrainz", "err", err)
|
|
writeErr(w, http.StatusInternalServerError, "server_error", "lookup failed")
|
|
return
|
|
}
|
|
writeJSON(w, http.StatusOK, resp)
|
|
}
|
|
|
|
type putLBBody struct {
|
|
Token *string `json:"token,omitempty"`
|
|
Enabled *bool `json:"enabled,omitempty"`
|
|
}
|
|
|
|
func (h *handlers) handlePutListenBrainz(w http.ResponseWriter, r *http.Request) {
|
|
user, ok := auth.UserFromContext(r.Context())
|
|
if !ok {
|
|
writeErr(w, http.StatusUnauthorized, "unauthorized", "authentication required")
|
|
return
|
|
}
|
|
var body putLBBody
|
|
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
|
writeErr(w, http.StatusBadRequest, "bad_request", "invalid json")
|
|
return
|
|
}
|
|
|
|
q := dbq.New(h.pool)
|
|
if body.Token != nil {
|
|
t := *body.Token
|
|
var tokenPtr *string
|
|
if t != "" {
|
|
tokenPtr = &t
|
|
} // empty string clears via SetListenBrainzToken's COALESCE handling
|
|
if err := q.SetListenBrainzToken(r.Context(), dbq.SetListenBrainzTokenParams{
|
|
ID: user.ID, ListenbrainzToken: tokenPtr,
|
|
}); err != nil {
|
|
h.logger.Error("api: set listenbrainz token", "err", err)
|
|
writeErr(w, http.StatusInternalServerError, "server_error", "update failed")
|
|
return
|
|
}
|
|
}
|
|
if body.Enabled != nil && *body.Enabled {
|
|
// Enabling requires a non-empty token.
|
|
cfg, err := q.GetListenBrainzConfig(r.Context(), user.ID)
|
|
if err != nil {
|
|
h.logger.Error("api: get listenbrainz config", "err", err)
|
|
writeErr(w, http.StatusInternalServerError, "server_error", "lookup failed")
|
|
return
|
|
}
|
|
if cfg.ListenbrainzToken == nil || *cfg.ListenbrainzToken == "" {
|
|
writeErr(w, http.StatusBadRequest, "bad_request", "cannot enable without a token")
|
|
return
|
|
}
|
|
}
|
|
if body.Enabled != nil {
|
|
if err := q.SetListenBrainzEnabled(r.Context(), dbq.SetListenBrainzEnabledParams{
|
|
ID: user.ID, ListenbrainzEnabled: *body.Enabled,
|
|
}); err != nil {
|
|
h.logger.Error("api: set listenbrainz enabled", "err", err)
|
|
writeErr(w, http.StatusInternalServerError, "server_error", "update failed")
|
|
return
|
|
}
|
|
}
|
|
|
|
resp, err := buildLBStatus(r, h, user.ID)
|
|
if err != nil {
|
|
h.logger.Error("api: put listenbrainz refresh status", "err", err)
|
|
writeErr(w, http.StatusInternalServerError, "server_error", "lookup failed")
|
|
return
|
|
}
|
|
writeJSON(w, http.StatusOK, resp)
|
|
}
|
|
|
|
func buildLBStatus(r *http.Request, h *handlers, userID pgtypeUUID) (LBStatus, error) {
|
|
q := dbq.New(h.pool)
|
|
cfg, err := q.GetListenBrainzConfig(r.Context(), userID)
|
|
if err != nil {
|
|
if errors.Is(err, pgx.ErrNoRows) {
|
|
return LBStatus{}, nil
|
|
}
|
|
return LBStatus{}, err
|
|
}
|
|
out := LBStatus{
|
|
Enabled: cfg.ListenbrainzEnabled,
|
|
TokenSet: cfg.ListenbrainzToken != nil && *cfg.ListenbrainzToken != "",
|
|
}
|
|
if cfg.LastScrobbledAt.Valid {
|
|
s := cfg.LastScrobbledAt.Time.UTC().Format(time.RFC3339)
|
|
out.LastScrobbledAt = &s
|
|
}
|
|
return out, nil
|
|
}
|
|
```
|
|
|
|
Note: `pgtypeUUID` is shorthand here — replace with the actual `pgtype.UUID` import. Add `"github.com/jackc/pgx/v5/pgtype"` and `"time"` to the imports. Update the function signature: `func buildLBStatus(r *http.Request, h *handlers, userID pgtype.UUID) (LBStatus, error)`.
|
|
|
|
- [ ] **Step 4: Mount the routes**
|
|
|
|
In `internal/api/api.go::Mount`, add inside the `authed.Group(...)` block (after the existing `/me` route):
|
|
|
|
```go
|
|
authed.Get("/me/listenbrainz", h.handleGetListenBrainz)
|
|
authed.Put("/me/listenbrainz", h.handlePutListenBrainz)
|
|
```
|
|
|
|
- [ ] **Step 5: Run tests to verify they pass**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && MINSTREL_TEST_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' go test ./internal/api/ -run ListenBrainz -v`
|
|
|
|
Expected: PASS for all 4 tests.
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add internal/api/listenbrainz.go internal/api/listenbrainz_test.go internal/api/api.go
|
|
git commit -m "feat(api): add GET/PUT /api/me/listenbrainz endpoints"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 10: Boot the worker in main.go
|
|
|
|
**Files:**
|
|
- Modify: `cmd/minstrel/main.go`
|
|
|
|
- [ ] **Step 1: Wire the worker startup**
|
|
|
|
In `cmd/minstrel/main.go`, add the imports:
|
|
|
|
```go
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble"
|
|
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble/listenbrainz"
|
|
```
|
|
|
|
After the `pool` is opened (after `pool, err := db.Open(...)` and the `defer pool.Close()`), and before the HTTP server starts, add:
|
|
|
|
```go
|
|
// Start the ListenBrainz scrobble worker. Per spec §M4a, runs every 30s,
|
|
// drains up to 50 pending rows per tick. Per-user gating happens inside the
|
|
// worker (rows from disabled users are skipped).
|
|
scrobbleWorker := scrobble.NewWorker(pool, listenbrainz.NewClient(), logger.With("component", "scrobble"))
|
|
go scrobbleWorker.Run(ctx)
|
|
```
|
|
|
|
- [ ] **Step 2: Verify compile**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && go build ./...`
|
|
|
|
Expected: clean.
|
|
|
|
- [ ] **Step 3: Verify the worker actually starts**
|
|
|
|
Run the binary briefly and check the logs for the scrobble worker:
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && SMARTMUSIC_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' SMARTMUSIC_LIBRARY_SCAN_PATHS='' SMARTMUSIC_LIBRARY_SCAN_ON_STARTUP=false timeout 5 go run ./cmd/minstrel/ 2>&1 | head -20`
|
|
|
|
Expected: server starts, no panic, worker tick logs (or absence of error logs) demonstrate Run is alive.
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add cmd/minstrel/main.go
|
|
git commit -m "feat(cmd): start scrobble worker alongside HTTP server"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 11: Frontend — apiPut + listenbrainz API helpers + /settings page
|
|
|
|
**Files:**
|
|
- Modify: `web/src/lib/api/client.ts`
|
|
- Create: `web/src/lib/api/listenbrainz.ts`
|
|
- Create: `web/src/routes/settings/+page.svelte`
|
|
- Create: `web/src/routes/settings/settings.test.ts`
|
|
- Modify: `web/src/lib/components/Shell.svelte`
|
|
|
|
- [ ] **Step 1: Add `api.put` helper**
|
|
|
|
In `web/src/lib/api/client.ts`, add a `put` method to the `api` object. The existing object looks like:
|
|
|
|
```ts
|
|
export const api = {
|
|
get: <T>(path: string): Promise<T> => apiFetch(path) as Promise<T>,
|
|
post: <T>(path: string, body: unknown): Promise<T> =>
|
|
apiFetch(path, { method: 'POST', body: JSON.stringify(body) }) as Promise<T>,
|
|
del: (path: string): Promise<null> =>
|
|
apiFetch(path, { method: 'DELETE' }) as Promise<null>
|
|
};
|
|
```
|
|
|
|
Replace with:
|
|
|
|
```ts
|
|
export const api = {
|
|
get: <T>(path: string): Promise<T> => apiFetch(path) as Promise<T>,
|
|
post: <T>(path: string, body: unknown): Promise<T> =>
|
|
apiFetch(path, { method: 'POST', body: JSON.stringify(body) }) as Promise<T>,
|
|
put: <T>(path: string, body: unknown): Promise<T> =>
|
|
apiFetch(path, { method: 'PUT', body: JSON.stringify(body) }) as Promise<T>,
|
|
del: (path: string): Promise<null> =>
|
|
apiFetch(path, { method: 'DELETE' }) as Promise<null>
|
|
};
|
|
```
|
|
|
|
- [ ] **Step 2: Add ListenBrainz API helpers**
|
|
|
|
Create `web/src/lib/api/listenbrainz.ts`:
|
|
|
|
```ts
|
|
import { api } from './client';
|
|
|
|
export type LBStatus = {
|
|
enabled: boolean;
|
|
token_set: boolean;
|
|
last_scrobbled_at: string | null;
|
|
};
|
|
|
|
export function getListenBrainzStatus(): Promise<LBStatus> {
|
|
return api.get<LBStatus>('/api/me/listenbrainz');
|
|
}
|
|
|
|
export function setListenBrainzToken(token: string): Promise<LBStatus> {
|
|
return api.put<LBStatus>('/api/me/listenbrainz', { token });
|
|
}
|
|
|
|
export function setListenBrainzEnabled(enabled: boolean): Promise<LBStatus> {
|
|
return api.put<LBStatus>('/api/me/listenbrainz', { enabled });
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Create the Settings page**
|
|
|
|
Create `web/src/routes/settings/+page.svelte`:
|
|
|
|
```svelte
|
|
<script lang="ts">
|
|
import { useQueryClient, createQuery, createMutation } from '@tanstack/svelte-query';
|
|
import {
|
|
getListenBrainzStatus,
|
|
setListenBrainzToken,
|
|
setListenBrainzEnabled,
|
|
type LBStatus
|
|
} from '$lib/api/listenbrainz';
|
|
|
|
const queryClient = useQueryClient();
|
|
|
|
const status = createQuery<LBStatus>(() => ({
|
|
queryKey: ['settings', 'listenbrainz'],
|
|
queryFn: getListenBrainzStatus
|
|
}));
|
|
|
|
const tokenMutation = createMutation(() => ({
|
|
mutationFn: (token: string) => setListenBrainzToken(token),
|
|
onSuccess: () => queryClient.invalidateQueries({ queryKey: ['settings', 'listenbrainz'] })
|
|
}));
|
|
|
|
const enabledMutation = createMutation(() => ({
|
|
mutationFn: (enabled: boolean) => setListenBrainzEnabled(enabled),
|
|
onSuccess: () => queryClient.invalidateQueries({ queryKey: ['settings', 'listenbrainz'] })
|
|
}));
|
|
|
|
let tokenInput = $state('');
|
|
|
|
function saveToken() {
|
|
const t = tokenInput.trim();
|
|
if (!t) return;
|
|
$tokenMutation.mutate(t, { onSuccess: () => { tokenInput = ''; } });
|
|
}
|
|
|
|
function clearToken() {
|
|
$tokenMutation.mutate('');
|
|
}
|
|
|
|
function toggleEnabled() {
|
|
if (!$status.data) return;
|
|
$enabledMutation.mutate(!$status.data.enabled);
|
|
}
|
|
</script>
|
|
|
|
<div class="space-y-6">
|
|
<h1 class="text-2xl font-semibold">Settings</h1>
|
|
|
|
<section class="space-y-4 rounded border border-border bg-surface p-4">
|
|
<h2 class="text-lg font-semibold">ListenBrainz</h2>
|
|
|
|
{#if $status.isPending}
|
|
<p class="text-text-secondary">Loading…</p>
|
|
{:else if $status.isError}
|
|
<p class="text-text-secondary">Couldn't load status.</p>
|
|
{:else if $status.data}
|
|
<div class="space-y-3">
|
|
<label class="block text-sm">
|
|
<span class="block text-text-secondary mb-1">User token</span>
|
|
{#if $status.data.token_set}
|
|
<div class="flex items-center gap-2">
|
|
<span class="text-text-primary">••••••••••• (set)</span>
|
|
<button
|
|
type="button"
|
|
onclick={clearToken}
|
|
class="text-sm text-accent hover:underline"
|
|
disabled={$tokenMutation.isPending}
|
|
>clear</button>
|
|
</div>
|
|
{:else}
|
|
<div class="flex gap-2">
|
|
<input
|
|
type="password"
|
|
bind:value={tokenInput}
|
|
placeholder="paste your LB token"
|
|
class="flex-1 rounded border border-border bg-background px-2 py-1 text-sm"
|
|
/>
|
|
<button
|
|
type="button"
|
|
onclick={saveToken}
|
|
disabled={!tokenInput.trim() || $tokenMutation.isPending}
|
|
class="rounded bg-accent px-3 py-1 text-sm text-background disabled:opacity-50"
|
|
>Save</button>
|
|
</div>
|
|
{/if}
|
|
</label>
|
|
|
|
<label class="flex items-center gap-2 text-sm">
|
|
<input
|
|
type="checkbox"
|
|
checked={$status.data.enabled}
|
|
disabled={!$status.data.token_set || $enabledMutation.isPending}
|
|
onchange={toggleEnabled}
|
|
/>
|
|
<span>Send my plays to ListenBrainz</span>
|
|
</label>
|
|
|
|
{#if $status.data.last_scrobbled_at}
|
|
<p class="text-sm text-text-secondary">
|
|
Last scrobbled: {new Date($status.data.last_scrobbled_at).toLocaleString()}
|
|
</p>
|
|
{/if}
|
|
|
|
<p class="text-xs text-text-secondary">
|
|
Get a token at <a href="https://listenbrainz.org/profile/" target="_blank" rel="noopener" class="text-accent hover:underline">listenbrainz.org/profile</a>.
|
|
Tokens are stored unencrypted in this server's database — treat as sensitive.
|
|
</p>
|
|
</div>
|
|
{/if}
|
|
</section>
|
|
</div>
|
|
```
|
|
|
|
- [ ] **Step 4: Add Settings to the nav**
|
|
|
|
In `web/src/lib/components/Shell.svelte`, in the `navItems` array:
|
|
|
|
```ts
|
|
const navItems = [
|
|
{ href: '/', label: 'Library' },
|
|
{ href: '/library/liked', label: 'Liked' },
|
|
{ href: '/search', label: 'Search' },
|
|
{ href: '/playlists', label: 'Playlists' },
|
|
{ href: '/settings', label: 'Settings' }
|
|
];
|
|
```
|
|
|
|
- [ ] **Step 5: Write the failing settings tests**
|
|
|
|
Create `web/src/routes/settings/settings.test.ts`:
|
|
|
|
```ts
|
|
import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';
|
|
import { render, screen, fireEvent, waitFor } from '@testing-library/svelte';
|
|
import { QueryClient, QueryClientProvider } from '@tanstack/svelte-query';
|
|
import { createRawSnippet } from 'svelte';
|
|
|
|
vi.mock('$lib/api/listenbrainz', () => ({
|
|
getListenBrainzStatus: vi.fn(),
|
|
setListenBrainzToken: vi.fn(),
|
|
setListenBrainzEnabled: vi.fn()
|
|
}));
|
|
|
|
import SettingsPage from './+page.svelte';
|
|
import {
|
|
getListenBrainzStatus,
|
|
setListenBrainzToken,
|
|
setListenBrainzEnabled
|
|
} from '$lib/api/listenbrainz';
|
|
|
|
function renderPage() {
|
|
const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } });
|
|
return render(QueryClientProvider, {
|
|
props: {
|
|
client: qc,
|
|
children: createRawSnippet(() => ({ render: () => '<div data-testid="root"></div>' }))
|
|
}
|
|
});
|
|
// NOTE: The above is a sketch; actual integration into your QueryClientProvider
|
|
// wrapping should match the pattern used in /library/liked tests. See
|
|
// web/src/routes/library/liked/liked.test.ts for the canonical example.
|
|
}
|
|
```
|
|
|
|
In practice, follow the pattern used by `web/src/routes/library/liked/liked.test.ts` (which already wraps a route with `QueryClientProvider`) — copy that boilerplate. Replace the test cases with the SearchBox tests below:
|
|
|
|
```ts
|
|
beforeEach(() => {
|
|
vi.clearAllMocks();
|
|
});
|
|
|
|
describe('Settings page — ListenBrainz', () => {
|
|
test('token-not-set state renders input + Save button', async () => {
|
|
(getListenBrainzStatus as any).mockResolvedValue({
|
|
enabled: false, token_set: false, last_scrobbled_at: null
|
|
});
|
|
renderPage();
|
|
await waitFor(() => expect(screen.getByPlaceholderText(/paste your lb token/i)).toBeInTheDocument());
|
|
expect(screen.getByRole('button', { name: /save/i })).toBeInTheDocument();
|
|
});
|
|
|
|
test('token-set state renders masked + Clear button', async () => {
|
|
(getListenBrainzStatus as any).mockResolvedValue({
|
|
enabled: false, token_set: true, last_scrobbled_at: null
|
|
});
|
|
renderPage();
|
|
await waitFor(() => expect(screen.getByText(/\(set\)/)).toBeInTheDocument());
|
|
expect(screen.getByRole('button', { name: /clear/i })).toBeInTheDocument();
|
|
});
|
|
|
|
test('Save button calls setListenBrainzToken with typed value', async () => {
|
|
(getListenBrainzStatus as any).mockResolvedValue({
|
|
enabled: false, token_set: false, last_scrobbled_at: null
|
|
});
|
|
(setListenBrainzToken as any).mockResolvedValue({ enabled: false, token_set: true, last_scrobbled_at: null });
|
|
renderPage();
|
|
const input = await screen.findByPlaceholderText(/paste your lb token/i);
|
|
await fireEvent.input(input, { target: { value: 'mytoken' } });
|
|
await fireEvent.click(screen.getByRole('button', { name: /save/i }));
|
|
await waitFor(() => expect(setListenBrainzToken).toHaveBeenCalledWith('mytoken'));
|
|
});
|
|
|
|
test('Enabled checkbox is disabled when no token', async () => {
|
|
(getListenBrainzStatus as any).mockResolvedValue({
|
|
enabled: false, token_set: false, last_scrobbled_at: null
|
|
});
|
|
renderPage();
|
|
const checkbox = await screen.findByRole('checkbox');
|
|
expect(checkbox).toBeDisabled();
|
|
});
|
|
|
|
test('Last-scrobbled-at renders when present', async () => {
|
|
(getListenBrainzStatus as any).mockResolvedValue({
|
|
enabled: true,
|
|
token_set: true,
|
|
last_scrobbled_at: '2026-04-28T03:00:00Z'
|
|
});
|
|
renderPage();
|
|
await waitFor(() => expect(screen.getByText(/last scrobbled:/i)).toBeInTheDocument());
|
|
});
|
|
});
|
|
```
|
|
|
|
- [ ] **Step 6: Run web tests + check + build**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel/web && npm run check 2>&1 | tail -3 && npm test -- --run src/routes/settings 2>&1 | tail -10`
|
|
|
|
Expected: type-check clean; 5 settings tests pass.
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel/web && npm test -- --run 2>&1 | tail -5`
|
|
|
|
Expected: full vitest suite passes (175 + 5 new = 180).
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel/web && npm run build 2>&1 | tail -5`
|
|
|
|
Expected: adapter-static emits `web/build/`.
|
|
|
|
- [ ] **Step 7: Commit**
|
|
|
|
```bash
|
|
git add web/
|
|
git commit -m "feat(web): add /settings page with ListenBrainz section"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 12: Final verification + branch finish
|
|
|
|
**Files:** none (verification only)
|
|
|
|
- [ ] **Step 1: Full Go test suite (-short -race)**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && go test -short -race ./...`
|
|
|
|
Expected: PASS across all 14 packages (now including `internal/scrobble` and `internal/scrobble/listenbrainz`).
|
|
|
|
- [ ] **Step 2: Full integration suite**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && MINSTREL_TEST_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' go test -race -p 1 ./...`
|
|
|
|
Expected: PASS apart from the pre-existing `TestScanner_Integration` flake (CI runs `-short` which skips it; not introduced by this PR).
|
|
|
|
- [ ] **Step 3: Lint**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && golangci-lint run ./...`
|
|
|
|
Expected: clean.
|
|
|
|
- [ ] **Step 4: Coverage check on new package**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && MINSTREL_TEST_DATABASE_URL='postgres://minstrel:minstrel@172.22.0.2:5432/minstrel?sslmode=disable' go test -p 1 -coverprofile=/tmp/cover-m4a.out ./internal/scrobble/... && go tool cover -func=/tmp/cover-m4a.out | tail -5`
|
|
|
|
Expected: `internal/scrobble` and `internal/scrobble/listenbrainz` combined coverage ≥ 80%.
|
|
|
|
- [ ] **Step 5: Web verification**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel/web && npm run check && npm test -- --run && npm run build`
|
|
|
|
Expected: svelte-check 0/0; 180 vitest tests pass (was 175); build succeeds.
|
|
|
|
- [ ] **Step 6: Docker smoke**
|
|
|
|
Run: `cd /home/bvandeusen/Nextcloud/Projects/Minstrel/minstrel && docker build -t minstrel:m4a-smoke .`
|
|
|
|
Expected: container builds.
|
|
|
|
- [ ] **Step 7: Manual end-to-end verification**
|
|
|
|
Set up:
|
|
1. `docker compose up --build -d minstrel` — pick up new code
|
|
2. Generate a token at https://listenbrainz.org/profile/
|
|
3. /settings → paste token → Save
|
|
4. /settings → toggle "Send my plays to ListenBrainz"
|
|
5. Play a track for ≥240 seconds OR finish it
|
|
6. Within 30 seconds, check listenbrainz.org/<your-username> — listen should appear
|
|
7. /settings → "Last scrobbled" timestamp populates
|
|
|
|
Sad path:
|
|
8. /settings → enter a deliberately bad token → toggle enabled → play a track
|
|
9. Within 30s, check `psql -c "SELECT status, last_error FROM scrobble_queue"` — row marked `failed`, error mentions auth
|
|
10. /settings refresh → enabled toggle has been auto-cleared
|
|
|
|
- [ ] **Step 8: Update Fable task #345**
|
|
|
|
After PR opens, mark `in_progress`. After PR merges, mark `done` with summary of what shipped.
|
|
|
|
- [ ] **Step 9: Finishing the branch**
|
|
|
|
**REQUIRED SUB-SKILL:** Use `superpowers:finishing-a-development-branch` to verify tests, present completion options (merge locally / push + PR / keep / discard), and execute the user's choice.
|
|
|
|
Per established cadence: this slice will land as a single-purpose PR. After merge, M4a closes; M4b (similarity ingest) is next.
|
|
|
|
---
|
|
|
|
## Self-Review
|
|
|
|
**Spec coverage:**
|
|
- §3.1 (new packages) → Tasks 3, 4, 5, 6, 7 ✓
|
|
- §3.2 (existing-code hooks) → Task 8 (Writer hooks), Task 9 (api.go Mount), Task 10 (main.go) ✓
|
|
- §4 (database schema) → Tasks 1, 2 ✓
|
|
- §5.1 (LB client) → Task 3 ✓
|
|
- §5.2 (threshold) → Task 4 ✓
|
|
- §5.3 (MaybeEnqueue) → Task 5 ✓
|
|
- §5.4 (Worker, Run, tickOnce, backoffDelay) → Tasks 6, 7 ✓
|
|
- §5.5 (API endpoints) → Task 9 ✓
|
|
- §6 (frontend) → Task 11 ✓
|
|
- §7.1-7.3 (test plan) → embedded across Tasks 3-11 ✓
|
|
- §7.4 (coverage target) → Task 12 step 4 ✓
|
|
- §7.5 (manual verification) → Task 12 step 7 ✓
|
|
- §8 (backwards compat) → preserved by zero-default user columns + new package not affecting existing code ✓
|
|
|
|
No gaps.
|
|
|
|
**Placeholder scan:** No "TBD"/"TODO" content. All steps have explicit code or commands.
|
|
|
|
**Type consistency:**
|
|
- `LBStatus` shape (Go) matches `LBStatus` shape (TypeScript) — `enabled`, `token_set`, `last_scrobbled_at`.
|
|
- `MaybeEnqueue(ctx, q, playEventID)` signature consistent across Tasks 5 and 8.
|
|
- `backoffDelay(attempts int) (time.Duration, bool)` — same in Task 6 and Task 7.
|
|
- `Worker.tickOnce(ctx) error` — same in Task 7 and Task 12.
|
|
- sqlc query names consistent across Task 2 (definitions) and Tasks 5/7/9 (callers).
|