Files
minstrel/internal/api/api.go
T
bvandeusenandClaude Opus 5 721154847e
test-go / test (push) Successful in 1m5s
test-go / integration (push) Failing after 3m39s
release / Build signed APK (releases and dev) (push) Successful in 4m55s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 1m38s
fix(recommendation): size the candidate pool to the library
Operator, 2026-09-10: "is the pool that we draw from somehow scaled to the
amount of music in the library... my earlier understanding of the tuning and
work may have been skewed by what was in my library."

It was not. DefaultCandidateSourceLimits returns what its own comment calls
"the v1 hardcoded constants per spec" — ~170 candidates for a 500-track
library and a 100,000-track one alike. The pool therefore samples a
shrinking FRACTION of a growing collection: 17% of 1,000 tracks, 1.7% of
10,000, 0.17% of 100,000. RandomFill, whose whole job is exploration,
becomes a thinner and noisier slice at exactly the moment a library gets
more diverse — which is the "starting to feel weird" being reported.

    1,000 tracks -> pool 170     (unchanged)
    5,000        -> pool 170     (unchanged)
   20,000        -> pool 280
   80,000        -> pool 500     (ceiling)

THE SCALING IS PER-ARM, and that is the substance rather than a refinement.
A limit only matters if there are rows for it to cut off, so what an arm is
BOUNDED BY decides whether library size can help it. LBSimilar,
SimilarArtist, TagOverlap and RandomFill grow: they are bounded by
similarity/tag data and by the library itself. LikesOverlap, UserCoplay and
TasteOverlap do not: they are bounded by the user's likes, the instance's
co-play graph and the taste profile, none of which grow when the library
does. Raising those would sample more of a set that did not change — churn,
not reach. It also keeps this from inflating the sim_score-0 share, since
TasteOverlap is one of the two zero-similarity arms.

sqrt, not linear: linear would put a 100,000-track library at a
3,400-candidate pool, long past where more candidates improve the answer.
A 4x ceiling bounds it at ~500.

Never shrinks an arm. The base limits are a floor, and #3889 makes that
load-bearing rather than tidy — shrinking an arm ordered by unseeded
random() changes pool membership between same-day rebuilds.

Library size comes from a TTL-cached count reusing CountTracksMatching with
an empty pattern (rule 28 — a new query would need sqlc regeneration, which
is blocked). The ILIKE defeats every index, so it is a full scan and must
not run per request. It degrades rather than fails: an error keeps the last
known value, a never-counted cache returns 0, and 0 scales to the base
limits — today's behaviour exactly. Nothing about sizing a pool justifies
failing the request it is sizing. Bounded by a 3s deadline (rule 156), and
a failed refresh does not stamp the clock, so a blip cannot pin a stale
value for the whole TTL.

THE REFERENCE IS ASSUMED, NOT MEASURED. libraryScaleReference = 5000 is
where growth starts, and the size the v1 constants were really tuned against
is unrecorded. #3879 should replace it; until then that constant is the one
thing to change. Deliberately conservative: below it nothing scales at all,
so no existing install changes behaviour.

Falsification caught a weak guard: the sqrt-vs-linear assertion was written
at SIXTEEN times the reference, where linear has already been clamped by the
ceiling and both curves land on 4x. It proved nothing. Moved to four times
the reference, below the ceiling for both, where sqrt gives 2x and linear
would give 4x.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 23:23:04 -04:00

314 lines
16 KiB
Go

// Package api implements Minstrel's native JSON surface under /api. It is
// consumed by the built-in web SPA and (eventually) the Flutter client.
// Subsonic-compatible endpoints under /rest are intentionally separate —
// see internal/subsonic — and the two packages must not depend on each other.
package api
import (
"log/slog"
"math/rand"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
"git.fabledsword.com/bvandeusen/minstrel/internal/config"
"git.fabledsword.com/bvandeusen/minstrel/internal/coverart"
"git.fabledsword.com/bvandeusen/minstrel/internal/eventbus"
"git.fabledsword.com/bvandeusen/minstrel/internal/library"
"git.fabledsword.com/bvandeusen/minstrel/internal/lidarrconfig"
"git.fabledsword.com/bvandeusen/minstrel/internal/lidarrquarantine"
"git.fabledsword.com/bvandeusen/minstrel/internal/lidarrrequests"
"git.fabledsword.com/bvandeusen/minstrel/internal/mailer"
"git.fabledsword.com/bvandeusen/minstrel/internal/netsettings"
"git.fabledsword.com/bvandeusen/minstrel/internal/playevents"
"git.fabledsword.com/bvandeusen/minstrel/internal/playlists"
"git.fabledsword.com/bvandeusen/minstrel/internal/reacquisition"
"git.fabledsword.com/bvandeusen/minstrel/internal/recommendation"
"git.fabledsword.com/bvandeusen/minstrel/internal/recsettings"
"git.fabledsword.com/bvandeusen/minstrel/internal/tags"
"git.fabledsword.com/bvandeusen/minstrel/internal/tracks"
)
// Mount attaches /api/* handlers to r. Public endpoints (login) are outside
// RequireUser; everything else is gated by the middleware. The events writer
// is shared with the Subsonic mount so /rest/scrobble feeds the same store.
func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playevents.Writer, recCfg config.RecommendationConfig, recSettings *recsettings.Service, lidarrCfg *lidarrconfig.Service, lidarrReqs *lidarrrequests.Service, lidarrQuar *lidarrquarantine.Service, tracksSvc *tracks.Service, playlistsSvc *playlists.Service, coverEnricher *coverart.Enricher, coverSettings *coverart.SettingsService, tagSettings *tags.SettingsService, scanner *library.Scanner, scanCfg library.RunScanConfig, dataDir string, sender mailer.Sender, bus *eventbus.Bus, playlistScheduler *playlists.Scheduler, streamSecret []byte, netSettings *netsettings.Service, reacqSettings *reacquisition.SettingsService) {
rng := rand.New(rand.NewSource(rand.Int63()))
h := &handlers{
pool: pool, logger: logger, events: events, recCfg: recCfg,
recSettings: recSettings,
rng: rng.Float64,
lidarrCfg: lidarrCfg,
lidarrRequests: lidarrReqs,
lidarrQuarantine: lidarrQuar,
tracks: tracksSvc,
playlists: playlistsSvc,
coverart: coverEnricher,
coverSettings: coverSettings,
tagSettings: tagSettings,
scanner: scanner,
scanCfg: scanCfg,
dataDir: dataDir,
mailer: sender,
eventbus: bus,
playlistScheduler: playlistScheduler,
streamSecret: streamSecret,
netSettings: netSettings,
reacqSettings: reacqSettings,
librarySize: recommendation.NewLibrarySize(nil),
}
r.Route("/api", func(api chi.Router) {
api.Post("/auth/login", h.handleLogin)
api.Post("/auth/register", h.handleRegister)
api.Post("/auth/forgot-password", h.handleForgotPassword)
api.Post("/auth/reset-password", h.handleResetPassword)
// Stream lives outside authed.Group so it can accept EITHER a
// session (resolved by the OptionalUser middleware) OR a signed
// query token (UPnP / Sonos path; see streamAuthOk). The
// middleware attaches user to context when a valid cookie /
// bearer is present but does NOT 401 on absence; the handler's
// own streamAuthOk performs the actual auth check. See the
// design at
// docs/superpowers/specs/2026-06-03-android-output-picker-upnp-design.md.
api.With(auth.OptionalUser(pool, logger)).Get("/tracks/{id}/stream", h.handleGetStream)
// Extension-bearing alias so Sonos's URL probe can identify the
// audio format from the path. The {ext} param is consumed by chi
// and ignored by the handler (which keys off {id}). See task #610.
api.With(auth.OptionalUser(pool, logger)).Get("/tracks/{id}/stream.{ext}", h.handleGetStream)
api.Group(func(authed chi.Router) {
authed.Use(auth.RequireUser(pool, netSettings.Hops))
authed.Post("/auth/logout", h.handleLogout)
authed.Get("/me", h.handleGetMe)
authed.Get("/me/system-playlists-status", h.handleGetSystemPlaylistsStatus)
authed.Get("/me/recommendation-metrics", h.handleGetRecommendationMetrics)
authed.Get("/me/listenbrainz", h.handleGetListenBrainz)
authed.Put("/me/listenbrainz", h.handlePutListenBrainz)
authed.Get("/me/history", h.handleGetMyHistory)
authed.Put("/me/password", h.handleChangePassword)
authed.Put("/me/profile", h.handleUpdateMyProfile)
authed.Put("/me/timezone", h.handlePutTimezone)
authed.Get("/me/api-token", h.handleGetMyAPIToken)
authed.Post("/me/api-token", h.handleRegenerateMyAPIToken)
authed.Get("/me/sessions", h.handleListMySessions)
authed.Delete("/me/sessions/{id}", h.handleRevokeMySession)
authed.Post("/me/sessions/logout-others", h.handleRevokeMyOtherSessions)
authed.Get("/artists", h.handleListArtists)
authed.Get("/artists/{id}", h.handleGetArtist)
authed.Get("/artists/{id}/tracks", h.handleGetArtistTracks)
authed.Get("/artists/{id}/similar", h.handleGetSimilarArtists)
authed.Get("/artists/{id}/top-tracks", h.handleGetArtistTopTracks)
authed.Get("/albums/{id}", h.handleGetAlbum)
authed.Get("/albums/{id}/cover", h.handleGetCover)
authed.Get("/library/shuffle", h.handleLibraryShuffle)
authed.Get("/library/albums", h.handleListLibraryAlbums)
// Browse indexes (#367). Genre filtering rides
// /library/albums?genre= rather than a path segment, because raw
// ID3 genres contain slashes ("Rock/Pop") that a path can't carry.
authed.Get("/library/genres", h.handleListGenres)
authed.Get("/library/years", h.handleListAlbumYears)
authed.Get("/library/sync", h.handleLibrarySync)
authed.Get("/tracks/{id}", h.handleGetTrack)
// /tracks/{id}/stream is mounted above with OptionalUser so
// it can accept either a session or a signed token.
authed.Get("/search", h.handleSearch)
authed.Get("/radio", h.handleRadio)
authed.Get("/discover/suggestions", h.handleListSuggestions)
// Snooze = "not right now", time-boxed and self-expiring
// (#2374). Not a dislike — see the migration for why.
authed.Post("/discover/suggestions/{mbid}/snooze", h.handleSnoozeSuggestion)
authed.Delete("/discover/suggestions/{mbid}/snooze", h.handleUnsnoozeSuggestion)
authed.Get("/discover/snoozes", h.handleListSuggestionSnoozes)
authed.Get("/home", h.handleGetHome)
authed.Get("/home/index", h.handleGetHomeIndex)
authed.Post("/events", h.handleEvents)
authed.Get("/events/stream", h.handleEventsStream)
// UPnP / Sonos cast slice: issue a short-lived HMAC stream URL
// the speaker can fetch without the user's session. See the
// design at
// docs/superpowers/specs/2026-06-03-android-output-picker-upnp-design.md.
authed.Post("/cast/stream-token", h.handleCastStreamToken)
authed.Post("/likes/tracks/{id}", h.handleLikeTrack)
authed.Delete("/likes/tracks/{id}", h.handleUnlikeTrack)
authed.Post("/likes/albums/{id}", h.handleLikeAlbum)
authed.Delete("/likes/albums/{id}", h.handleUnlikeAlbum)
authed.Post("/likes/artists/{id}", h.handleLikeArtist)
authed.Delete("/likes/artists/{id}", h.handleUnlikeArtist)
authed.Get("/likes/tracks", h.handleListLikedTracks)
authed.Get("/likes/albums", h.handleListLikedAlbums)
authed.Get("/likes/artists", h.handleListLikedArtists)
authed.Get("/likes/ids", h.handleGetLikedIDs)
authed.Get("/lidarr/search", h.handleLidarrSearch)
authed.Post("/requests", h.handleCreateRequest)
authed.Get("/requests", h.handleListRequests)
authed.Get("/requests/{id}", h.handleGetRequest)
authed.Delete("/requests/{id}", h.handleCancelRequest)
authed.Post("/quarantine", h.handleFlag)
authed.Delete("/quarantine/{track_id}", h.handleUnflag)
authed.Get("/quarantine/mine", h.handleListMyQuarantine)
// Client-reported playback errors (zero-duration tracks,
// load failures). Admin-only inbox; any user can report.
authed.Post("/playback-errors", h.handleReportPlaybackError)
// Device diagnostics ingest (M9). Any signed-in user can
// POST a batch, but events are only stored when the
// account's debug_mode_enabled flag is on (handler no-ops
// otherwise). Admin views live under /admin/diagnostics.
authed.Post("/diagnostics", h.handleReportDiagnostics)
// Self-hosted in-app update channel (#397). Auth-gated to
// prevent anonymous bandwidth abuse on the APK stream;
// /apk additionally per-user rate-limited.
authed.Get("/client/version", h.handleClientVersion)
authed.Get("/client/apk", h.handleClientAPK)
authed.Route("/admin", func(admin chi.Router) {
admin.Use(auth.RequireAdmin())
admin.Get("/lidarr/config", h.handleGetLidarrConfig)
admin.Put("/lidarr/config", h.handlePutLidarrConfig)
admin.Post("/lidarr/test", h.handleTestLidarrConnection)
admin.Get("/lidarr/quality-profiles", h.handleListQualityProfiles)
admin.Get("/lidarr/metadata-profiles", h.handleListMetadataProfiles)
admin.Get("/lidarr/root-folders", h.handleListRootFolders)
admin.Get("/requests", h.handleListAdminRequests)
admin.Post("/requests/{id}/approve", h.handleApproveRequest)
admin.Post("/requests/{id}/reject", h.handleRejectRequest)
admin.Get("/quarantine", h.handleListAdminQuarantine)
admin.Post("/quarantine/{track_id}/resolve", h.handleResolveQuarantine)
admin.Get("/playback-errors", h.handleListAdminPlaybackErrors)
admin.Post("/playback-errors/{id}/resolve", h.handleResolvePlaybackError)
admin.Post("/quarantine/{track_id}/delete-file", h.handleDeleteQuarantineFile)
admin.Post("/quarantine/{track_id}/delete-via-lidarr", h.handleDeleteQuarantineViaLidarr)
admin.Get("/quarantine/actions", h.handleListQuarantineActions)
admin.Delete("/tracks/{id}", h.handleRemoveTrack)
admin.Post("/albums/{id}/cover/refetch", h.handleAdminAlbumRefetchCover)
admin.Post("/covers/refetch-missing", h.handleAdminBulkRefetchCovers)
admin.Get("/network-settings", h.handleGetNetworkSettings)
admin.Put("/network-settings", h.handleUpdateNetworkSettings)
// Policy for turning a missing file back into a Lidarr
// request (#290). Sits beside the missing-files list it
// governs rather than under /lidarr, because the operator
// meets it on the missing-files surface.
admin.Get("/library/reacquisition", h.handleGetReacquisitionSettings)
admin.Put("/library/reacquisition", h.handleUpdateReacquisitionSettings)
admin.Get("/scan/status", h.handleGetScanStatus)
admin.Post("/scan/run", h.handleTriggerScan)
// Sits under /library rather than /tracks because what it
// reports is a property of the library's relationship to disk,
// and because the destructive /tracks/{id} route above must
// not be mistaken for it (#2527).
admin.Get("/library/missing", h.handleListMissingTracks)
admin.Get("/library/coverage", h.handleGetLibraryCoverage)
admin.Get("/invites", h.handleListInvites)
admin.Post("/invites", h.handleCreateInvite)
admin.Delete("/invites/{token}", h.handleDeleteInvite)
admin.Get("/users", h.handleAdminListUsers)
admin.Put("/users/{id}/admin", h.handleUpdateUserAdmin)
admin.Post("/users", h.handleAdminCreateUser)
admin.Delete("/users/{id}", h.handleAdminDeleteUser)
admin.Post("/users/{id}/reset-password", h.handleAdminResetPassword)
admin.Put("/users/{id}/auto-approve", h.handleAdminAutoApproveToggle)
admin.Put("/users/{id}/debug-mode", h.handleAdminDebugModeToggle)
// Device diagnostics timeline + device overview (M9).
admin.Get("/diagnostics", h.handleListAdminDiagnostics)
admin.Get("/diagnostics/devices", h.handleListAdminDiagnosticDevices)
admin.Get("/cover-sources", h.handleListCoverSources)
admin.Patch("/cover-sources/{provider_id}", h.handleUpdateCoverSource)
admin.Post("/cover-sources/{provider_id}/test", h.handleTestCoverSource)
admin.Post("/cover-sources/research", h.handleResearchMissingArt)
admin.Get("/tag-sources", h.handleListTagSources)
admin.Patch("/tag-sources/{provider_id}", h.handleUpdateTagSource)
admin.Post("/tag-sources/{provider_id}/test", h.handleTestTagSource)
admin.Post("/tag-sources/research", h.handleResearchTags)
admin.Get("/smtp-config", h.handleGetSMTPConfig)
admin.Put("/smtp-config", h.handleUpdateSMTPConfig)
admin.Post("/smtp-config/test", h.handleTestSMTPConfig)
// Recommendation tuning lab (#1250): scoring-weight
// profiles + taste-build knobs, DB-backed, live effect.
admin.Get("/recommendation-tuning", h.handleGetRecommendationTuning)
admin.Patch("/recommendation-tuning/{scope}", h.handlePatchRecommendationTuning)
admin.Post("/recommendation-tuning/{scope}/reset", h.handleResetRecommendationTuning)
// Weekly outcome trends + knob-turn markers (#1251).
admin.Get("/recommendation-trends", h.handleGetRecommendationTrends)
})
authed.Get("/playlists", h.handleListPlaylists)
authed.Post("/playlists", h.handleCreatePlaylist)
authed.Get("/playlists/{id}", h.handleGetPlaylist)
authed.Patch("/playlists/{id}", h.handleUpdatePlaylist)
authed.Delete("/playlists/{id}", h.handleDeletePlaylist)
authed.Post("/playlists/{id}/tracks", h.handleAppendTracks)
authed.Delete("/playlists/{id}/tracks/{position}", h.handleRemovePlaylistTrack)
authed.Put("/playlists/{id}/tracks", h.handleReorderPlaylist)
authed.Get("/playlists/{id}/cover", h.handleGetPlaylistCover)
authed.Post("/playlists/system/{kind}/refresh", h.handleSystemPlaylistRefresh)
authed.Get("/playlists/system/{kind}/shuffle", h.handleSystemPlaylistShuffle)
})
})
}
type handlers struct {
pool *pgxpool.Pool
logger *slog.Logger
events *playevents.Writer
recCfg config.RecommendationConfig
recSettings *recsettings.Service
rng func() float64
// librarySize memoises the track count that sizes the candidate pool
// (#3880). Held here rather than counted per request: the count is a
// full table scan, and library size only moves when a scan runs.
librarySize *recommendation.LibrarySize
lidarrCfg *lidarrconfig.Service
lidarrRequests *lidarrrequests.Service
lidarrQuarantine *lidarrquarantine.Service
tracks *tracks.Service
playlists *playlists.Service
coverart *coverart.Enricher
coverSettings *coverart.SettingsService
tagSettings *tags.SettingsService
scanner *library.Scanner
scanCfg library.RunScanConfig
dataDir string
mailer mailer.Sender
eventbus *eventbus.Bus
playlistScheduler *playlists.Scheduler
// reacqSettings is the DB-backed policy for auto re-acquisition of
// missing files (milestone #290) — grace window, backoff, attempt caps.
// Cached in the service, so the admin card reads it without a query.
reacqSettings *reacquisition.SettingsService
// netSettings caches the trusted reverse-proxy depth read by the auth
// middleware on every request and edited from the admin network card.
netSettings *netsettings.Service
// streamSecret is the HMAC key used by SignStreamToken /
// VerifyStreamToken to authenticate the UPnP-speaker stream path
// (see internal/api/stream_token.go and the design at
// docs/superpowers/specs/2026-06-03-android-output-picker-upnp-design.md).
// nil in slice 1; slice 2 wires the env-var-with-app_preferences-
// fallback loader. A nil secret leaves the cookie path intact and
// makes the token path unreachable (HMAC of empty key won't match
// anything a client mints), which is the desired slice-1 default.
streamSecret []byte
}