feat: Minstrel stops acquiring music and drops Discover
release / go (push) Successful in 1m54s
release / web (push) Failing after 5s
release / integration (push) Failing after 3m44s
release / govulncheck (push) Successful in 20s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / Build signed APK (push) Canceled after 19s
release / android (push) Canceled after 2m37s

Minstrel is now a viewer of the library it is pointed at. Finding,
downloading and filing music moves out of it.

Removed from the server:
- the request flow (requests, approval, auto-approve) and the wanted list,
  follows, release profiles and release blocks;
- searching indexers, the download clients, the importer, the album
  lifecycle and missing-file re-acquisition;
- the catalogue's discography, search and art (the MusicBrainz release and
  track-list cache stays, for the duplicate resolver);
- Discover: its suggestions of artists not in the library, snoozes, the
  candidate-artist tag enrichment and its tuning scope;
- writes to acquisition tables from moving the library folder and from
  merging duplicates, the want-list join on Suspect sources, the sealing of
  acquisition credentials, and the release-group lookup only
  re-acquisition used.

Settling a wrong-slot duplicate no longer sets the release aside: the
copies go and their slots show as missing until a good copy is in the
library.

Removed from the web: Admin > Acquisition and Requests, /requests,
/discover, Follow on the artist page, the auto-approve toggle, and the
Collection's wanted-list tabs. The Collection is now the library's health:
an overview, Reports, Duplicates, Suspect sources and Missing files. The
README and docs no longer describe getting music.

The tables stay for now; a later migration drops them. The request
notification kinds and the Android screens are still to go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-21 20:51:14 -05:00
co-authored by Claude Opus 5.5
parent dabe1e052d
commit 55c7497ca7
277 changed files with 521 additions and 40021 deletions
+7 -14
View File
@@ -4,7 +4,7 @@
<p align="center">A self-hosted music server that picks what to play next.</p>
Minstrel serves your music library to the web, to its own Android app, and to any Subsonic client. Shuffle, radio, likes and play history are worked out on the server, so every device gets the same mixes and the same history. It can also find, download and file the albums you want, through a search service and a download client you run yourself. It is for anyone who keeps their own music collection and wants it to sound like more than a random shuffle.
Minstrel serves your music library to the web, to its own Android app, and to any Subsonic client. Shuffle, radio, likes and play history are worked out on the server, so every device gets the same mixes and the same history. It is for anyone who keeps their own music collection and wants it to sound like more than a random shuffle.
<a href="docs/screenshots/home.png"><img src="docs/screenshots/home.png" width="820" alt="Minstrel home — your library at a glance"></a>
@@ -49,7 +49,7 @@ Open `http://<host>:4533/register` and create the first account with that setup
1. **Let the first scan finish.** Minstrel scans the library on startup. Watch it, or scan again, on the **Admin** page (`/admin`). Later scans only pick up what changed.
2. **Install the Android app.** In **Settings** (`/settings`), the *Install the Android app* card downloads the app this server ships. It updates itself from your server after that.
3. **Invite the household.** In **Admin › Users** (`/admin/users`), make an invite token or turn on open registration. Each person gets their own likes, history and recommendations.
4. **Optional: connect services.** Add a search service and a download client in **Admin › Acquisition** (`/admin/acquisition`). Each person adds their own ListenBrainz token in **Settings › Connections**.
4. **Optional: connect ListenBrainz.** Each person adds their own ListenBrainz token in **Settings › Connections**.
<a href="docs/screenshots/library-scan.png"><img src="docs/screenshots/library-scan.png" width="820" alt="The Admin page, where the library scan runs and reports progress"></a>
@@ -64,11 +64,11 @@ The quick start sets everything that's required. These are the settings people m
| `MINSTREL_BRANDING_APP_NAME` | The name shown in the header, the browser tab and link previews. |
| `MINSTREL_LOG_LEVEL` | `debug`, `info`, `warn` or `error`. |
Search services, download clients, ListenBrainz, email and recommendation tuning are set in the web UI and take effect without a restart. The full file-based surface is in [`config.example.yaml`](./config.example.yaml); to use it, mount your copy at `/etc/smartmusic/config.yaml`.
ListenBrainz, email, metadata sources and recommendation tuning are set in the web UI and take effect without a restart. The full file-based surface is in [`config.example.yaml`](./config.example.yaml); to use it, mount your copy at `/etc/smartmusic/config.yaml`.
## Setup pitfalls
- **The library mount has to be writable by uid 1000.** Minstrel files the albums it downloads into the library, and it deletes a file when an admin asks it to. It never moves, renames or retags files that were already there. Mount the library `:ro` to forbid writes: those actions then refuse and say why.
- **The library mount has to be writable by uid 1000.** Minstrel deletes a file when an admin asks it to, and when the duplicate resolver removes a copy. It never moves, renames or retags your files. Mount the library `:ro` to forbid writes: those actions then refuse and say why.
- **`MINSTREL_LIBRARY_SCAN_PATHS` names the path inside the container** (`/music`), not the folder on your host.
- **On the internet, put it behind HTTPS.** Plain `http://` is fine on a network you trust. For a public server, bind the port to `127.0.0.1` and put a reverse proxy in front: see [docs/hosting.md](docs/hosting.md).
@@ -103,11 +103,6 @@ Every image carries a signed Android app, all with the same key, so a phone can
- Duplicates, missing files and suspect copies are found and listed under **Collection** for an admin to fix.
- Missing album art and MusicBrainz ids are filled in automatically.
**Getting music**
- A wanted list: requests from the household, artists you follow, and albums whose files went missing.
- Minstrel searches the indexers you configure, hands the best release to your download client, checks what arrives against MusicBrainz, tags it, and files it. A better copy replaces a poor one.
- Off until you add a search service and a download client.
**Clients and accounts**
- A full web app, built into the server.
- A native Android app that ships inside the server image, keeps playing what it has cached when the server is out of reach, and updates itself from your server.
@@ -116,13 +111,11 @@ Every image carries a signed Android app, all with the same key, so a phone can
## Scope and responsible use
Minstrel indexes the files you point it at and streams them to your own clients. It ships no indexers, no trackers, no torrent, Usenet or NZB engine of its own, and no DRM circumvention.
Minstrel indexes the files you point it at and streams them to your own clients. It does not find or download music, and it does no DRM circumvention.
Getting music is optional and does nothing until you configure it. It searches only the search service (Prowlarr or a Torznab indexer) and the download client (qBittorrent) you add yourself.
What you put in your library is your responsibility. Copyright law applies to your collection as it does to any other software that plays a file; please respect it, and the terms of any service you connect.
What you put in your library, and the sources you configure, are your responsibility. Copyright law applies to your collection as it does to any other software that plays a file; please respect it, and the terms of any service you connect.
Minstrel is not affiliated with or endorsed by ListenBrainz, MusicBrainz, Prowlarr, qBittorrent or Subsonic.
Minstrel is not affiliated with or endorsed by ListenBrainz, MusicBrainz or Subsonic.
## Development
+5 -58
View File
@@ -25,16 +25,12 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/db"
"git.fabledsword.com/bvandeusen/minstrel/internal/eventbus"
"git.fabledsword.com/bvandeusen/minstrel/internal/gc"
"git.fabledsword.com/bvandeusen/minstrel/internal/importer"
"git.fabledsword.com/bvandeusen/minstrel/internal/library"
"git.fabledsword.com/bvandeusen/minstrel/internal/lifecycle"
"git.fabledsword.com/bvandeusen/minstrel/internal/logging"
"git.fabledsword.com/bvandeusen/minstrel/internal/mailer"
"git.fabledsword.com/bvandeusen/minstrel/internal/notifications"
"git.fabledsword.com/bvandeusen/minstrel/internal/playlists"
"git.fabledsword.com/bvandeusen/minstrel/internal/reacquisition"
"git.fabledsword.com/bvandeusen/minstrel/internal/recsettings"
"git.fabledsword.com/bvandeusen/minstrel/internal/requests"
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble"
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble/listenbrainz"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
@@ -43,7 +39,6 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/subsonic"
syncpkg "git.fabledsword.com/bvandeusen/minstrel/internal/sync"
"git.fabledsword.com/bvandeusen/minstrel/internal/tags"
"git.fabledsword.com/bvandeusen/minstrel/internal/wanted"
)
func main() {
@@ -203,8 +198,8 @@ func run() error {
if acoustIDErr != nil {
logger.Warn("acoustid settings: using defaults (lookup off)", "err", acoustIDErr)
}
// One AcoustID client, so the lookup worker and the import's check share
// its pace (AcoustID allows three requests a second per key).
// One AcoustID client, so every lookup shares its pace (AcoustID allows
// three requests a second per key).
acoustIDClient := acoustid.New(acoustid.DefaultBaseURL)
acoustIDLookup := library.NewAcoustIDLookupWorker(pool, logger.With("component", "acoustid_lookup"),
acoustIDSettings, acoustIDClient)
@@ -355,44 +350,12 @@ func run() error {
// same pool and bus.
notifier := notifications.New(pool, bus, logger.With("component", "notifications"))
// The request reconciler completes approved requests once the library
// holds what they asked for.
requestReconciler := requests.NewReconciler(pool, logger.With("component", "requests"), bus)
requestReconciler.SetNotifier(notifier)
library.SetNotifier(notifier)
go requestReconciler.Run(ctx)
// Release catalogue: re-reads discographies older than a
// week from MusicBrainz, a batch an hour, through the process-wide
// MusicBrainz pacer it shares with tag enrichment.
// Release track lists from MusicBrainz, read on demand by the duplicate
// resolver through the process-wide MusicBrainz pacer it shares with tag
// enrichment.
cat := catalogue.New(pool, nil, logger.With("component", "catalogue"))
go cat.Run(ctx)
// The wanted list: reads followed artists' discographies
// daily for new releases and marks wants held once the library has them.
go wanted.New(pool, cat, logger.With("component", "wanted")).Run(ctx)
// Search and grab: every ten minutes, searches the
// configured indexers for wants due, paced by each source's
// searches-per-hour, and hands the best release to the download client
// when auto-grab is on.
go wanted.NewSearcher(pool, cat, logger.With("component", "search")).Run(ctx)
// Import with verification: every minute, finished
// downloads are checked against their album (AcoustID, lengths, tags)
// and filed into the library with MusicBrainz tags, or refused.
var printer importer.Printer
if acoustIDSettings != nil {
printer = &importer.AcoustIDPrinter{Settings: acoustIDSettings, Client: acoustIDClient}
}
imp := importer.New(pool, scanner, cat, printer, scanner.Roots, logger.With("component", "import"))
imp.DataDir = cfg.Storage.DataDir
go imp.Run(ctx)
// Download lifecycle: every ten minutes, a download that
// failed or that nobody shares is retried, then given up and its album
// searched again; an imported one seeds, then is removed.
go lifecycle.New(pool, logger.With("component", "lifecycle")).Run(ctx)
// Duplicate resolver: classifies the sweep's groups and, with auto-resolve
// on, merges the copies that are safe to merge.
@@ -404,19 +367,6 @@ func run() error {
TrackLists: cat.RefreshReleaseGroup, Identify: acoustIDLookup.IdentifyRecordings,
}).Run(ctx)
// Missing-file re-acquisition: wants back albums whose files have been
// gone longer than the grace window, for the search schedule to find.
// Hourly, because the grace window is measured in hours.
//
// A settings-load failure is logged, not fatal: NewSettingsService always
// returns a usable service holding the defaults.
reacqSettings, reacqErr := reacquisition.NewSettingsService(
ctx, pool, logger.With("component", "reacquisition"))
if reacqErr != nil {
logger.Warn("reacquisition: using default settings", "err", reacqErr)
}
go reacquisition.NewSweeper(pool, reacqSettings, logger.With("component", "reacquisition")).Run(ctx)
// library_changes compactor: daily, deletes
// rows older than the configured retention so the change-log table
// doesn't grow unbounded. Clients that drop offline longer than
@@ -495,9 +445,6 @@ func run() error {
srv.FingerprintSettings = fpSettings
srv.LoudnessSettings = loudSettings
srv.AcoustIDLookup = acoustIDLookup
// The sweeper above holds this same instance, so a save from the admin
// card changes what it does on its next tick.
srv.ReacqSettings = reacqSettings
// The server's own backup: the loop runs the nightly schedule, and the
// admin page starts, tests and configures the same instance.
backups := backup.NewService(pool, logger.With("component", "backup"), buildinfo.Version)
+1 -1
View File
@@ -30,7 +30,7 @@ library:
branding:
app_name: "Minstrel" # MINSTREL_BRANDING_APP_NAME
description: "Self-hosted music server with Subsonic compatibility, smart shuffle, and music acquisition." # MINSTREL_BRANDING_DESCRIPTION
description: "Self-hosted music server with Subsonic compatibility and smart shuffle." # MINSTREL_BRANDING_DESCRIPTION
subsonic:
# Accept the plaintext `p=` password on /rest/*. Leave off unless a client
+2 -2
View File
@@ -86,7 +86,7 @@ changes on every restart until an account exists.
## Backups
Minstrel can back itself up once a night: accounts, likes, playlists,
listening history, requests and settings, in one file. It is off until an
listening history and settings, in one file. It is off until an
admin switches it on under **Admin › Backups**. Anything Minstrel can fetch
or work out again (artwork, fingerprints, MusicBrainz data) is left out, and
so are the keys and passwords Minstrel keeps for other services, which are
@@ -145,7 +145,7 @@ the place of what the scan found.
The library scan then finds each file, and the artwork, tags, fingerprints
and loudness are fetched or worked out again over the following hours. Keys
for other services (ListenBrainz, AcoustID, artwork and tag providers,
email, download clients) and the backup's own destination are entered again
email) and the backup's own destination are entered again
under **Admin**. An app that synced with the old server starts its sync over.
## Android app
-1
View File
@@ -53,7 +53,6 @@ they aren't checked.
Some credentials can't be hashed, because Minstrel has to send or compare the
value itself:
- the API keys and passwords for indexers, Prowlarr and download clients;
- the AcoustID key, and the cover-art and tag provider keys;
- the SMTP password;
- the secret key of the S3 bucket backups go to;
-180
View File
@@ -1,180 +0,0 @@
package acquire
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"path/filepath"
"strings"
"testing"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/downloader"
"git.fabledsword.com/bvandeusen/minstrel/internal/exchange"
"git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
"git.fabledsword.com/bvandeusen/minstrel/internal/prowlarr"
)
func validUpdate() Update {
return Update{Label: "Prowlarr", BaseURL: "http://prowlarr:9696", TimeoutMs: 30000, LocalPath: "/downloads", SearchesPerHour: 120}
}
func TestUpdateValidate(t *testing.T) {
if err := validUpdate().validate(); err != nil {
t.Fatalf("a valid form: %v", err)
}
for name, tc := range map[string]struct {
edit func(*Update)
field string
}{
"no name": {func(u *Update) { u.Label = "" }, "label"},
"switched on with no address": {func(u *Update) { u.Enabled, u.BaseURL = true, "" }, "base_url"},
"an address with no scheme": {func(u *Update) { u.BaseURL = "prowlarr:9696" }, "base_url"},
"a timeout too short": {func(u *Update) { u.TimeoutMs = 10 }, "timeout_ms"},
"a relative downloads folder": {func(u *Update) { u.LocalPath = "downloads" }, "local_path"},
"a relative client folder": {func(u *Update) { u.RemotePath = "data" }, "remote_path"},
"too many searches": {func(u *Update) { u.SearchesPerHour = 4000 }, "searches_per_hour"},
} {
u := validUpdate()
tc.edit(&u)
var invalid *InvalidError
if err := u.validate(); !errors.As(err, &invalid) || invalid.Field != tc.field {
t.Errorf("%s: err = %v, want an InvalidError on %s", name, err, tc.field)
}
}
// Switched off, an empty address is a row still being set up.
u := validUpdate()
u.BaseURL = ""
if err := u.validate(); err != nil {
t.Errorf("switched off with no address: %v", err)
}
}
func TestSourceFor_BuildsTheClientTheProtocolNames(t *testing.T) {
for protocol, check := range map[string]func(Source) bool{
ProtocolProwlarr: func(s Source) bool { _, ok := s.Search.(*prowlarr.Client); return ok && s.Fetch != nil },
ProtocolTorznab: func(s Source) bool { _, ok := s.Search.(*indexer.Torznab); return ok && s.Fetch != nil },
// Exchange's results are all magnets: nothing to fetch through it.
ProtocolExchange: func(s Source) bool { _, ok := s.Search.(*exchange.Client); return ok && s.Fetch == nil },
} {
src := SourceFor(dbq.AcquisitionService{Code: protocol, Protocol: protocol, SearchesPerHour: 7})
if !check(src) || src.Code != protocol || src.PerHour != 7 {
t.Errorf("%s: source %+v", protocol, src)
}
}
}
func TestJobs_AnExchangeDoesWhatItsSwitchesAllow(t *testing.T) {
ex := dbq.AcquisitionService{Kind: KindDownload, Protocol: ProtocolExchange, ServesSearch: true}
if !Searches(ex) || Fetches(ex) {
t.Error("an exchange with only search on should search and not fetch")
}
// A qBittorrent's switches mean nothing: it fetches, whatever they say.
qb := dbq.AcquisitionService{Kind: KindDownload, Protocol: ProtocolQBittorrent, ServesSearch: true}
if Searches(qb) || !Fetches(qb) {
t.Error("a qBittorrent fetches and never searches")
}
}
func TestTest_NoAddressIsNothingToReach(t *testing.T) {
r := Test(context.Background(), dbq.AcquisitionService{Protocol: ProtocolProwlarr, Kind: KindSearch})
if r.OK || !strings.Contains(r.Detail, "No address") {
t.Fatalf("result = %+v", r)
}
}
func TestTest_SearchReportsResultsAndARefusedKey(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("X-Api-Key") != "good" {
w.WriteHeader(http.StatusUnauthorized)
return
}
_, _ = w.Write([]byte(`[{"title": "Radiohead - OK Computer"}]`))
}))
defer srv.Close()
row := dbq.AcquisitionService{Kind: KindSearch, Protocol: ProtocolProwlarr, BaseUrl: srv.URL, ApiKey: "good", TimeoutMs: 5000}
if r := Test(context.Background(), row); !r.OK || !strings.Contains(r.Detail, "1 results") {
t.Errorf("a working key: %+v", r)
}
row.ApiKey = "bad"
if r := Test(context.Background(), row); r.OK || !strings.Contains(r.Detail, "refused") {
t.Errorf("a refused key: %+v", r)
}
}
// fakeExchange answers the three native APIs.
func fakeExchange(t *testing.T) *httptest.Server {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case r.URL.Path == "/api/downloads":
_, _ = w.Write([]byte(`{"enabled": true, "running": true, "inbound_conns": 3, "downloads": []}`))
case r.URL.Path == "/api/search":
_, _ = w.Write([]byte(`{"total": 0, "results": []}`))
case strings.HasPrefix(r.URL.Path, "/api/torrents/"):
w.WriteHeader(http.StatusNotFound)
_, _ = w.Write([]byte(`{"error": "unknown torrent"}`))
default:
w.WriteHeader(http.StatusNotFound)
}
}))
t.Cleanup(srv.Close)
return srv
}
func TestTest_AnExchangeReportsEachJobSwitchedOn(t *testing.T) {
srv := fakeExchange(t)
row := dbq.AcquisitionService{
Kind: KindDownload, Protocol: ProtocolExchange, BaseUrl: srv.URL, TimeoutMs: 5000,
ServesDownloads: true, ServesSearch: true, ServesEnrichment: true, LocalPath: t.TempDir(),
}
r := Test(context.Background(), row)
if !r.OK {
t.Fatalf("all three jobs: %+v", r)
}
for _, job := range []string{"Downloads:", "Search:", "Release details:"} {
if !strings.Contains(r.Detail, job) {
t.Errorf("detail %q does not report %s", r.Detail, job)
}
}
row.ServesDownloads, row.ServesEnrichment = false, false
if r := Test(context.Background(), row); strings.Contains(r.Detail, "Downloads:") || !strings.Contains(r.Detail, "Search:") {
t.Errorf("only search on: %+v", r)
}
row.ServesSearch = false
if r := Test(context.Background(), row); r.OK || !strings.Contains(r.Detail, "switched off") {
t.Errorf("every job off: %+v", r)
}
}
func TestTest_ADownloadClientNeedsAFolderMinstrelCanSee(t *testing.T) {
srv := fakeExchange(t)
row := dbq.AcquisitionService{
Kind: KindDownload, Protocol: ProtocolExchange, BaseUrl: srv.URL, TimeoutMs: 5000,
ServesDownloads: true, Category: DefaultCategory,
LocalPath: filepath.Join(t.TempDir(), "not-mounted"),
}
r := Test(context.Background(), row)
if r.OK || !strings.Contains(r.Detail, "cannot see its downloads folder") {
t.Fatalf("a missing folder: %+v", r)
}
row.LocalPath = t.TempDir()
if r := Test(context.Background(), row); !r.OK || !strings.Contains(r.Detail, `"minstrel" category`) {
t.Errorf("a visible, writable folder: %+v", r)
}
}
func TestDownloadClient_WrapsThePathMapping(t *testing.T) {
d, err := DownloadClient(dbq.AcquisitionService{Protocol: ProtocolQBittorrent, LocalPath: "/downloads", RemotePath: "/data"})
if err != nil {
t.Fatal(err)
}
if _, plain := d.(*downloader.Client); plain {
t.Error("a row naming two folders should come back wrapped in its mapping")
}
if _, err := DownloadClient(dbq.AcquisitionService{Protocol: ProtocolProwlarr}); err == nil {
t.Error("a search protocol is not a download client")
}
}
-45
View File
@@ -1,45 +0,0 @@
package acquire
import (
"context"
"fmt"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/release"
)
// Build makes a Service from the settings as they stand: every enabled
// search source behind one paced Fanout, the first enabled download client,
// and the quality profile. Built afresh for each pass and each request, so a
// changed setting takes effect on the next one.
func Build(ctx context.Context, pool *pgxpool.Pool) (*Service, dbq.AcquisitionSetting, error) {
q := dbq.New(pool)
settings, err := q.GetAcquisitionSettings(ctx)
if err != nil {
return nil, settings, fmt.Errorf("acquire: read settings: %w", err)
}
sources, err := Sources(ctx, pool)
if err != nil {
return nil, settings, fmt.Errorf("acquire: read search sources: %w", err)
}
clients, err := usableServices(ctx, q, JobDownload)
if err != nil {
return nil, settings, fmt.Errorf("acquire: read download clients: %w", err)
}
s := &Service{Profile: release.NewProfile(settings.AllowedQualities), Memory: NewMemory(pool)}
fan := &Fanout{Sources: sources, Pace: SearchPace}
if len(sources) > 0 {
s.Search = fan
}
if len(clients) > 0 {
row := clients[0]
client, err := DownloadClient(row)
if err != nil {
return nil, settings, fmt.Errorf("acquire: download client %s: %w", row.Code, err)
}
s.Grabber = &Downloader{Client: client, Code: row.Code, Landing: row.LocalPath, Fetch: fan}
}
return s, settings, nil
}
-164
View File
@@ -1,164 +0,0 @@
package acquire
import (
"context"
"errors"
"fmt"
"strings"
"sync"
"git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
)
// Searching every source at once. A source that is down narrows a search rather
// than failing it: the results come back with the list of who did not answer.
// SourceFailure is one source that could not be reached.
type SourceFailure struct {
Code string
Label string
Err error
}
// PartialError is a search that reached some sources and not others. It is
// returned alongside the results, not instead of them.
type PartialError struct {
Failures []SourceFailure
// Answered is how many sources did answer. Zero means the search failed.
Answered int
}
func (e *PartialError) Error() string {
if e == nil || len(e.Failures) == 0 {
return "some sources did not answer"
}
parts := make([]string, 0, len(e.Failures))
for _, f := range e.Failures {
parts = append(parts, fmt.Sprintf("%s: %s", f.Label, indexer.FirstLine(f.Err.Error())))
}
return "some sources did not answer — " + strings.Join(parts, "; ")
}
// Unwrap lets errors.Is find indexer.ErrUnreachable through a partial failure.
func (e *PartialError) Unwrap() []error {
if e == nil {
return nil
}
out := make([]error, 0, len(e.Failures))
for _, f := range e.Failures {
out = append(out, f.Err)
}
return out
}
// Fanout searches every source and merges what comes back. It is both the
// Searcher and the Fetcher, routing a .torrent fetch to the source that
// offered the result.
type Fanout struct {
// Sources in code order: the fold keeps a duplicate's first arrival, so
// the order decides which source a grab is fetched through.
Sources []Source
// Pace spaces each source's searches. Nil paces nothing.
Pace *Pacer
}
// Search asks every source concurrently. Each client bounds its own call by
// its configured timeout, so there is no shared deadline here.
func (f *Fanout) Search(ctx context.Context, query indexer.Query) ([]indexer.Result, error) {
if f == nil || len(f.Sources) == 0 {
return nil, indexer.ErrUnconfigured
}
found := make([][]indexer.Result, len(f.Sources))
errs := make([]error, len(f.Sources))
var wg sync.WaitGroup
for i, src := range f.Sources {
if src.Search == nil {
continue
}
f.Pace.spend(src)
wg.Add(1)
go func(i int, src Source) {
defer wg.Done()
found[i], errs[i] = src.Search.Search(ctx, query)
}(i, src)
}
wg.Wait()
out := make([]indexer.Result, 0, 32)
var failures []SourceFailure
answered, unconfigured := 0, 0
for i, src := range f.Sources {
if src.Search == nil {
continue
}
if errors.Is(errs[i], indexer.ErrUnconfigured) {
unconfigured++
continue
}
if rl, ok := indexer.AsRateLimited(errs[i]); ok {
f.Pace.hold(src.Code, rl)
}
if errs[i] != nil {
failures = append(failures, SourceFailure{Code: src.Code, Label: src.Label, Err: errs[i]})
continue
}
answered++
for j := range found[i] {
if found[i][j].Source == "" {
found[i][j].Source = src.Code
}
}
out = append(out, found[i]...)
}
switch {
case answered == 0 && len(failures) > 0:
return nil, &PartialError{Failures: failures}
case len(failures) > 0:
return out, &PartialError{Failures: failures, Answered: answered}
case answered == 0 && unconfigured > 0:
return nil, indexer.ErrUnconfigured
}
return out, nil
}
// Download fetches a result's .torrent from the source that offered it: a
// Prowlarr link carries Prowlarr's key and only Prowlarr can redeem it.
func (f *Fanout) Download(ctx context.Context, r indexer.Result) ([]byte, error) {
if f == nil {
return nil, indexer.ErrUnconfigured
}
for _, src := range f.Sources {
if src.Code != r.Source {
continue
}
if src.Fetch == nil {
return nil, fmt.Errorf("%w: %s cannot fetch torrent files", indexer.ErrUnconfigured, src.Label)
}
return src.Fetch.Download(ctx, r)
}
if r.Source == "" {
return nil, fmt.Errorf("%w: this result does not say which source it came from", indexer.ErrUnconfigured)
}
return nil, fmt.Errorf("%w: %q is no longer a configured source", indexer.ErrUnconfigured, r.Source)
}
// Narrowing is one source that did not answer, as a page shows it.
type Narrowing struct {
Source string `json:"source"`
Label string `json:"label"`
Reason string `json:"reason"`
}
// narrowings reads a PartialError out of err, and whether any source
// answered. Nil when err is not a partial failure.
func narrowings(err error) ([]Narrowing, bool) {
var partial *PartialError
if !errors.As(err, &partial) || len(partial.Failures) == 0 {
return nil, false
}
out := make([]Narrowing, 0, len(partial.Failures))
for _, f := range partial.Failures {
out = append(out, Narrowing{Source: f.Code, Label: f.Label, Reason: indexer.FirstLine(f.Err.Error())})
}
return out, partial.Answered > 0
}
-338
View File
@@ -1,338 +0,0 @@
package acquire
import (
"context"
"errors"
"path"
"strings"
"sync"
"time"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/downloader"
"git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
"git.fabledsword.com/bvandeusen/minstrel/internal/release"
)
// Taking a release. One live grab per want: a second is tie-broken against
// the download already running.
// Tag goes on every torrent Minstrel adds, so they can be told apart from
// anything else in the client.
const Tag = "minstrel"
// MissingClient is Taken.Missing when no download client is set up.
const MissingClient = "download_client"
// nearlyDone is how far along a running download stays whatever would
// replace it.
const nearlyDone = 0.9
// Downloader is the grab half of a Service.
type Downloader struct {
Client downloader.Driver
// Code is the client's acquisition_services row.
Code string
// Landing is the client's downloads folder as Minstrel sees it; each
// grab lands in a folder of its own under it.
Landing string
// Fetch gets a result's .torrent from the source that offered it.
Fetch Fetcher
}
// Origin says who took a release.
type Origin struct {
By string // OriginSchedule or OriginOperator
User pgtype.UUID
}
// Contest is how a grab fared against the download already running.
type Contest struct {
// Kept: the running one stays and this grab was not made. Replaced:
// this grab took its place.
Kept, Replaced bool
Reason string
Rival dbq.Grab
}
// Taken is what came of a grab.
type Taken struct {
Chosen *Candidate
// Added is the only field that means a download started.
Added bool
SavePath string
Grab dbq.Grab
// Gone: the search no longer offers the release.
Gone bool
// Refused: not taken; the reason is on Chosen.
Refused bool
// Unconfigured: Missing names what is not set up ("" for search).
Unconfigured bool
Missing string
Contest *Contest
Error string
}
// Take searches again and grabs the candidate named by key: a person's
// grab. Searching again keeps the indexer's download link (which can carry
// its key) on the server, and finds a release gone since it was shown.
// force takes what the verdict would not.
func (s *Service) Take(ctx context.Context, want Want, key string, force bool, origin Origin) Taken {
if s.Grabber == nil {
return Taken{Unconfigured: true, Missing: MissingClient}
}
found := s.For(ctx, want)
switch {
case found.Unconfigured:
return Taken{Unconfigured: true}
case found.Error != "":
return Taken{Error: found.Error}
}
chosen := pick(found.Candidates, key)
if chosen == nil {
return Taken{Gone: true}
}
return s.Grab(ctx, want, chosen, found.Candidates, force, origin)
}
// Grab hands a judged candidate to the download client. The schedule calls
// it with the candidate it has just judged; Take calls it after searching
// again.
func (s *Service) Grab(ctx context.Context, want Want, chosen *Candidate, all []Candidate, force bool, origin Origin) Taken {
out := Taken{Chosen: chosen}
if s.Grabber == nil || s.Grabber.Client == nil {
out.Unconfigured, out.Missing = true, MissingClient
return out
}
if chosen.Verdict == Blocked || (chosen.Verdict != Grab && !force) {
out.Refused = true
return out
}
unlock := lockWant(want.ID)
defer unlock()
contest := s.contest(ctx, want, chosen, all, origin)
if contest != nil && contest.Kept {
out.Contest = contest
return out
}
add := downloader.Add{Magnet: chosen.Source.Magnet, Tags: []string{Tag}}
hash := chosen.InfoHash
if add.Magnet == "" {
if s.Grabber.Fetch == nil {
out.Error = "This release is a .torrent link, and nothing is set up to fetch it."
return out
}
body, err := s.Grabber.Fetch.Download(ctx, chosen.Source)
if magnet, ok := indexer.AsMagnet(err); ok {
add.Magnet, err = magnet, nil
}
if err != nil {
out.Error = indexer.FirstLine(err.Error())
return out
}
if add.Magnet == "" {
add.Torrent = body
if meta, err := release.ParseTorrent(body); err == nil {
if hash == "" {
hash = meta.InfoHash
}
if refused := s.judgeFiles(ctx, want, chosen, hash, meta.Files, force); refused {
out.Refused = true
return out
}
}
}
}
if hash == "" && add.Magnet != "" {
hash = indexer.MagnetHash(add.Magnet)
}
add.SavePath = landingFor(s.Grabber.Landing, hash, chosen.Name)
err := s.Grabber.Client.AddTorrent(ctx, add)
if errors.Is(err, downloader.ErrDuplicate) && s.heldElsewhere(ctx, hash) {
why := "Already in the download client outside Minstrel's category, where Minstrel cannot follow it (" +
strings.TrimPrefix(indexer.FirstLine(err.Error()), downloader.ErrDuplicate.Error()+": ") + ")"
if force {
out.Error = why + "."
return out
}
set(chosen, Blocked, Hard, CodeBlocked, why+".")
if s.Memory != nil && hash != "" {
_ = s.Memory.Block(ctx, dbq.BlockReleaseParams{
InfoHash: hash, Name: chosen.Name, Reason: BlockGone, Detail: why + ".", WantID: want.ID,
ExpiresAt: pgtype.Timestamptz{Time: time.Now().Add(GoneBlock), Valid: true},
})
}
out.Refused = true
return out
}
if err != nil && !errors.Is(err, downloader.ErrDuplicate) {
if errors.Is(err, downloader.ErrUnconfigured) {
out.Unconfigured, out.Missing = true, MissingClient
} else {
out.Error = indexer.FirstLine(err.Error())
}
return out
}
out.Added, out.SavePath = true, add.SavePath
if s.Memory == nil {
return out
}
var replaced pgtype.UUID
why := ""
if contest != nil && contest.Replaced {
out.Contest = contest
replaced, why = contest.Rival.ID, "Replaced by "+chosen.Name+": "+contest.Reason+"."
}
g, err := s.Memory.Grabbed(ctx, dbq.InsertGrabParams{
WantID: want.ID, InfoHash: hash, Name: chosen.Name, Key: Key(*chosen), Indexer: chosen.Indexer,
Source: chosen.Source.Source, Client: s.Grabber.Code, SavePath: add.SavePath,
Quality: string(chosen.Tier), Origin: origin.By, Forced: force, GrabbedBy: origin.User,
}, replaced, why)
if err != nil {
out.Error = "The download started, but could not be recorded: " + indexer.FirstLine(err.Error())
return out
}
out.Grab = g
// Removed only after the new download was added and recorded, so a
// failure never leaves the want with nothing running.
if replaced.Valid && contest.Rival.InfoHash != "" {
if err := s.Grabber.Client.Delete(ctx, true, contest.Rival.InfoHash); err != nil {
contest.Reason += ". The download it replaced could not be removed from the client: " + indexer.FirstLine(err.Error())
}
}
return out
}
// heldElsewhere reports whether a release the client says it already holds is
// one it does not list for Minstrel: held under another category. Recorded
// as a grab, it would read as gone half an hour later and be taken again,
// every pass. A client that
// cannot answer now is given the benefit of the doubt, as the add was.
func (s *Service) heldElsewhere(ctx context.Context, hash string) bool {
if hash == "" {
return false
}
listed, err := s.Grabber.Client.Torrents(ctx, indexer.NormalizeHash(hash))
return err == nil && len(listed) == 0
}
// judgeFiles judges a chosen release again on its .torrent's file list, and
// sets it aside when the files say it is not the album, so no later search
// offers it again. force still takes it.
func (s *Service) judgeFiles(ctx context.Context, want Want, chosen *Candidate, hash string, files []release.File, force bool) bool {
chosen.Files = files
if chosen.FilesCount == 0 {
chosen.FilesCount = len(files)
}
veto, reason, _ := release.JudgeFiles(want.Want, files)
if veto == "" || force {
return false
}
set(chosen, Rejected, Hard, veto, reason)
if s.Memory != nil && hash != "" {
_ = s.Memory.Block(ctx, dbq.BlockReleaseParams{
InfoHash: hash, Name: chosen.Name, Reason: BlockFiles, Detail: reason, WantID: want.ID,
})
}
return true
}
var wantLocks sync.Map // want id → *sync.Mutex
// lockWant settles two grabs for one want one after the other. The
// one-live-grab index is the backstop for anything outside this process.
func lockWant(id pgtype.UUID) func() {
if !id.Valid {
return func() {}
}
m, _ := wantLocks.LoadOrStore(id.Bytes, &sync.Mutex{})
mu := m.(*sync.Mutex)
mu.Lock()
return mu.Unlock
}
// contest settles a grab against the download running for its want: a
// person's grab beats the schedule's; between two of a kind the better
// candidate wins; a tie, a download nearly done, or one that cannot be
// removed stays. Nil when nothing is running.
func (s *Service) contest(ctx context.Context, want Want, chosen *Candidate, all []Candidate, origin Origin) *Contest {
if s.Memory == nil || !want.ID.Valid {
return nil
}
running, ok, err := s.Memory.LiveGrab(ctx, want.ID)
if err != nil || !ok {
return nil
}
c := &Contest{Rival: running}
if running.InfoHash != "" && running.InfoHash == chosen.InfoHash {
c.Kept, c.Reason = true, "That release is already downloading for this album."
return c
}
switch {
case origin.By == OriginOperator && running.Origin == OriginSchedule:
c.Reason = "you chose it over the one the schedule started"
case origin.By == OriginSchedule && running.Origin == OriginOperator:
c.Kept, c.Reason = true, "A download you started for this album is already running."
return c
default:
var theirs *Candidate
for i := range all {
if (running.InfoHash != "" && all[i].InfoHash == running.InfoHash) || Key(all[i]) == running.Key {
theirs = &all[i]
break
}
}
if theirs == nil || !better(*chosen, *theirs) {
c.Kept, c.Reason = true, "A download for this album is already running, and this release is no better."
return c
}
c.Reason = "it ranks above the one that was running"
}
if running.InfoHash == "" {
c.Kept, c.Reason = true, "A download for this album is already running, and it cannot be identified to replace it."
return c
}
if ts, err := s.Grabber.Client.Torrents(ctx, running.InfoHash); err == nil && len(ts) > 0 && ts[0].Progress >= nearlyDone {
c.Kept, c.Reason = true, "A download for this album is already running and nearly done."
return c
}
c.Replaced = true
return c
}
// landingFor is the folder a download lands in: one per torrent, named by
// its infohash, so two grabs never unpack into the same place.
func landingFor(root, hash, name string) string {
dir := hash
if dir == "" {
dir = slug(name)
}
if dir == "" {
dir = "unnamed"
}
return path.Join(root, dir)
}
// slug keeps a folder name to characters every filesystem in the chain
// takes.
func slug(s string) string {
var b strings.Builder
for _, r := range strings.ToLower(strings.TrimSpace(s)) {
if b.Len() >= 80 {
break
}
switch {
case r >= 'a' && r <= 'z', r >= '0' && r <= '9':
b.WriteRune(r)
case r == '-', r == '_', r == ' ', r == '.':
b.WriteRune('-')
}
}
return strings.Trim(b.String(), "-")
}
-199
View File
@@ -1,199 +0,0 @@
package acquire
import (
"context"
"encoding/json"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgtype"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
"git.fabledsword.com/bvandeusen/minstrel/internal/release"
)
// Block reasons, matching release_blocks_reason_check.
const (
BlockOperator = "operator"
BlockFiles = "files"
// BlockImport: the import found its files were not the album.
BlockImport = "import"
// BlockStalled: given up on because it would not finish; held
// back until the block's expires_at.
BlockStalled = "stalled"
// BlockGone: its download client would not let Minstrel follow it, held
// outside Minstrel's category or leaving the client twice; held
// back for GoneBlock.
BlockGone = "gone"
)
// GoneBlock is how long a release its client lost track of is held back:
// long enough that a release which keeps vanishing is tried once a day rather
// than on every pass, short enough that it is taken once the client lets go.
const GoneBlock = 24 * time.Hour
// Grab origins, matching grabs_origin_check.
const (
OriginSchedule = "schedule"
OriginOperator = "operator"
)
// Memory is what searches and grabs leave behind: every candidate of a
// want's last search, the grabs, and the releases set aside.
type Memory struct {
pool *pgxpool.Pool
}
// NewMemory returns a Memory over pool.
func NewMemory(pool *pgxpool.Pool) *Memory { return &Memory{pool: pool} }
// Keep replaces a want's candidates with this search's, in order.
func (m *Memory) Keep(ctx context.Context, wantID pgtype.UUID, cands []Candidate) error {
return pgx.BeginFunc(ctx, m.pool, func(tx pgx.Tx) error {
q := dbq.New(tx)
if err := q.DeleteWantCandidates(ctx, wantID); err != nil {
return err
}
for i, c := range cands {
reasons, err := json.Marshal(c.Reasons)
if err != nil {
return err
}
if c.Reasons == nil {
reasons = []byte("[]")
}
folded := c.FoldedFrom
if folded == nil {
folded = []string{}
}
if err := q.InsertWantCandidate(ctx, dbq.InsertWantCandidateParams{
WantID: wantID, Key: Key(c), Name: c.Name, InfoHash: c.InfoHash, Indexer: c.Indexer,
Source: c.Source.Source, SizeBytes: c.SizeBytes, Seeders: int32(c.Seeders),
FilesCount: int32(c.FilesCount), Format: release.Describe(c.Release), Quality: string(c.Tier),
Year: int32(c.Year), Verdict: string(c.Verdict), Reason: c.Reason, Reasons: reasons,
FoldedFrom: folded, Rank: int32(i),
}); err != nil {
return err
}
}
return nil
})
}
// blocked is every release set aside, by infohash, with the sentence a
// candidate shows. A failed read blocks nothing: offering a release set
// aside costs one grab a person can undo, while blocking everything on a
// database blip would hide every result.
func (m *Memory) blocked(ctx context.Context) map[string]string {
if m == nil {
return nil
}
rows, err := dbq.New(m.pool).ListReleaseBlocks(ctx)
if err != nil {
return nil
}
out := make(map[string]string, len(rows))
for _, r := range rows {
out[indexer.NormalizeHash(r.InfoHash)] = BlockSentence(r.Reason, r.Detail)
}
return out
}
// BlockSentence is what a blocked candidate says.
func BlockSentence(reason, detail string) string {
switch reason {
case BlockFiles:
if detail != "" {
return "Set aside for what its files hold: " + detail
}
return "Set aside for what its files hold."
case BlockImport:
if detail != "" {
return "Refused on import: " + detail
}
return "Refused on import: its files were not the album."
case BlockGone:
if detail != "" {
return "Held back for now: " + detail
}
return "Held back for now: its download client lost track of it."
case BlockStalled:
if detail != "" {
return "Given up on for now: " + detail
}
return "Given up on for now: it would not finish."
}
return "You set this release aside."
}
// Block sets a release aside and marks every kept candidate of it blocked.
func (m *Memory) Block(ctx context.Context, p dbq.BlockReleaseParams) error {
p.InfoHash = indexer.NormalizeHash(p.InfoHash)
if p.InfoHash == "" {
return errors.New("a release without an infohash cannot be set aside")
}
return pgx.BeginFunc(ctx, m.pool, func(tx pgx.Tx) error {
q := dbq.New(tx)
if err := q.BlockRelease(ctx, p); err != nil {
return err
}
return q.MarkCandidatesBlocked(ctx, dbq.MarkCandidatesBlockedParams{
InfoHash: p.InfoHash, Reason: BlockSentence(p.Reason, p.Detail),
})
})
}
// Unblock lets a release be taken again. Its kept candidates wait for a
// person until the next search judges them afresh.
func (m *Memory) Unblock(ctx context.Context, hash string) (bool, error) {
hash = indexer.NormalizeHash(hash)
var n int64
err := pgx.BeginFunc(ctx, m.pool, func(tx pgx.Tx) error {
q := dbq.New(tx)
var err error
if n, err = q.UnblockRelease(ctx, hash); err != nil {
return err
}
return q.MarkCandidatesUnblocked(ctx, dbq.MarkCandidatesUnblockedParams{
InfoHash: hash, Reason: "Allowed again. The next search judges it afresh.",
})
})
return n > 0, err
}
// LiveGrab is the download running for a want, if there is one.
func (m *Memory) LiveGrab(ctx context.Context, wantID pgtype.UUID) (dbq.Grab, bool, error) {
g, err := dbq.New(m.pool).GetLiveGrab(ctx, wantID)
if errors.Is(err, pgx.ErrNoRows) {
return g, false, nil
}
if err != nil {
return g, false, err
}
return g, true, nil
}
// Grabbed records a download started, ending the grab it replaced first so
// the one-live-grab index holds.
func (m *Memory) Grabbed(ctx context.Context, p dbq.InsertGrabParams, replaced pgtype.UUID, why string) (dbq.Grab, error) {
var g dbq.Grab
err := pgx.BeginFunc(ctx, m.pool, func(tx pgx.Tx) error {
q := dbq.New(tx)
if replaced.Valid {
if err := q.EndGrab(ctx, dbq.EndGrabParams{ID: replaced, EndReason: "superseded", EndDetail: why}); err != nil {
return err
}
}
var err error
g, err = q.InsertGrab(ctx, p)
return err
})
if err != nil {
return g, fmt.Errorf("record grab: %w", err)
}
return g, nil
}
-145
View File
@@ -1,145 +0,0 @@
package acquire
import (
"context"
"sync"
"time"
"git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
)
// SearchPace is the one Pacer every search shares. Process-wide because a
// Fanout is built fresh for each search; a Pacer per Fanout would start empty
// and pace nothing.
var SearchPace = &Pacer{}
// holdWithoutRetryAfter is how long a source that answered 429 without saying
// for how long is left alone: one schedule tick.
const holdWithoutRetryAfter = 10 * time.Minute
// Pacer spaces the searches sent to each source by its searches_per_hour: a
// spacing, not a burst bucket, because Prowlarr answers 429 to a few requests
// in quick succession. Every search spends, the schedule's and a
// person's alike; only the schedule waits.
type Pacer struct {
// Now and Sleep are the clock, replaceable in tests.
Now func() time.Time
Sleep func(context.Context, time.Duration) error
mu sync.Mutex
next map[string]time.Time
}
func (p *Pacer) now() time.Time {
if p.Now != nil {
return p.Now()
}
return time.Now()
}
func (p *Pacer) sleep(ctx context.Context, d time.Duration) error {
if p.Sleep != nil {
return p.Sleep(ctx, d)
}
t := time.NewTimer(d)
defer t.Stop()
select {
case <-t.C:
return nil
case <-ctx.Done():
return ctx.Err()
}
}
// gap is the spacing a source's number asks for; zero is unpaced.
func gap(perHour int) time.Duration {
if perHour <= 0 {
return 0
}
return time.Hour / time.Duration(perHour)
}
// readyAt is when every source may next be asked.
func (p *Pacer) readyAt(sources []Source) time.Time {
p.mu.Lock()
defer p.mu.Unlock()
var at time.Time
for _, s := range sources {
if s.Search == nil {
continue
}
if n := p.next[s.Code]; n.After(at) {
at = n
}
}
return at
}
// Wait holds until every source may be asked, and answers false, without
// waiting, when that is after the deadline or ctx ends first. All of them,
// not any: a search while one is held back would be narrowed, and the want
// recorded as looked for.
func (p *Pacer) Wait(ctx context.Context, sources []Source, deadline time.Time) bool {
if p == nil {
return true
}
at := p.readyAt(sources)
if !deadline.IsZero() && at.After(deadline) {
return false
}
if d := at.Sub(p.now()); d > 0 {
return p.sleep(ctx, d) == nil
}
return ctx.Err() == nil
}
// spend takes one search's slot from a source.
func (p *Pacer) spend(s Source) {
g := gap(s.PerHour)
if p == nil || g == 0 {
return
}
p.mu.Lock()
defer p.mu.Unlock()
if p.next == nil {
p.next = map[string]time.Time{}
}
at := p.next[s.Code]
if t := p.now(); at.Before(t) {
at = t
}
p.next[s.Code] = at.Add(g)
}
// hold leaves a source alone after it answered 429, for as long as it asked,
// whatever its searches_per_hour says.
func (p *Pacer) hold(code string, rl *indexer.RateLimited) {
if p == nil {
return
}
d := rl.RetryAfter
if d <= 0 {
d = holdWithoutRetryAfter
}
p.mu.Lock()
defer p.mu.Unlock()
if p.next == nil {
p.next = map[string]time.Time{}
}
if until := p.now().Add(d); until.After(p.next[code]) {
p.next[code] = until
}
}
// Ready holds until every source this service searches may be asked again,
// and answers false when that is past the deadline. The schedule calls it
// before each want; a person's search never does.
func (s *Service) Ready(ctx context.Context, deadline time.Time) bool {
if s == nil {
return ctx.Err() == nil
}
if f, ok := s.Search.(*Fanout); ok && f != nil {
return f.Pace.Wait(ctx, f.Sources, deadline)
}
return ctx.Err() == nil
}
-408
View File
@@ -1,408 +0,0 @@
package acquire
import (
"context"
"errors"
"sort"
"strings"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
"git.fabledsword.com/bvandeusen/minstrel/internal/release"
)
// One search for one want, and what could be done about each result.
// internal/release
// says whether a name is the album and what quality it is; this package
// asks in the right order and says why in words a person can act on.
// Verdict is what can be done about one result.
type Verdict string
const (
// Grab: the album, in a quality the profile takes.
Grab Verdict = "grab"
// Review: maybe the album; a person decides.
Review Verdict = "review"
// Refused: the album, in a quality the profile does not take. The only
// verdict that names a setting.
Refused Verdict = "refused"
// Rejected: not the album.
Rejected Verdict = "rejected"
// Blocked: a release set aside, by a person or by its files.
Blocked Verdict = "blocked"
)
// order sorts verdicts: what can be taken first, what was given up on last.
var order = map[Verdict]int{Grab: 0, Review: 1, Refused: 2, Rejected: 3, Blocked: 4}
// Reason kinds: a hard reason is a fact about the bytes, a soft one about the
// moment or a setting.
const (
Hard = "hard"
Soft = "soft"
)
// Reason codes beyond internal/release's vetoes.
const (
CodeBlocked = "blocked"
CodeProfile = "profile"
CodeNoSeeders = "no_seeders"
CodeDoubt = "doubt"
// CodeNotBetter: the album, but no better than the copy the library
// holds, for a want that is an upgrade.
CodeNotBetter = "not_better"
)
// Reason is one thing a verdict rests on.
type Reason struct {
Kind string `json:"kind"`
Code string `json:"code"`
Text string `json:"text"`
}
// Want is the album searched for: its wants row and how the catalogue
// describes it.
type Want struct {
ID pgtype.UUID
release.Want
// Floor is the tier of the copy the library holds, for an album wanted
// for a better one: a release must rank above it. "" for an
// album not held, and for a video rip, which any clean copy beats.
Floor release.Tier
// Held says the floor in words: "MP3, about 192 kbps".
Held string
}
// Candidate is one result with everything decided about it.
type Candidate struct {
release.Release
// Source is the indexer's own result. It never leaves the server: its
// DownloadURL can carry the indexer's key.
Source indexer.Result
InfoHash string
Indexer string
SizeBytes int64
Seeders int
SwarmKnown bool
Category int
FilesCount int
// Files is the torrent's file list once its .torrent has been read.
Files []release.File
FoldedFrom []string
Tier release.Tier
YearAgrees bool
Verdict Verdict
Reason string
Reasons []Reason
}
// Key is how a candidate is named across a search and a grab: its infohash,
// or its name when the source gave none.
func Key(c Candidate) string {
if c.InfoHash != "" {
return c.InfoHash
}
return c.Name
}
// Result is one search.
type Result struct {
Query string
// Unconfigured: no search source is set up.
Unconfigured bool
// Error is a search that failed outright.
Error string
// Narrowed names sources that did not answer on a search that otherwise
// did.
Narrowed []Narrowing
Considered int
Folded int
// Looked is how many .torrent files were read for their file lists.
Looked int
Candidates []Candidate
// KeepError: the candidates could not be stored.
KeepError string
}
// Grabbable is what may be taken, best first.
func (r Result) Grabbable() []Candidate {
var out []Candidate
for _, c := range r.Candidates {
if c.Verdict == Grab {
out = append(out, c)
}
}
return out
}
// ForAPerson says whether anything found is worth a person's look: a
// release that may be the album, whatever its quality.
func (r Result) ForAPerson() bool {
for _, c := range r.Candidates {
if c.Verdict == Grab || c.Verdict == Review || c.Verdict == Refused {
return true
}
}
return false
}
// Service searches and grabs.
type Service struct {
// Search is nil when no source is configured.
Search Searcher
// Grabber is nil when no download client is configured.
Grabber *Downloader
Profile release.Profile
// Memory stores candidates, grabs and blocks. Nil keeps nothing.
Memory *Memory
}
// Query is the text sent to the indexers: the artist and the title, without
// a trailing bracketed part the release names usually leave off.
func Query(w release.Want) string {
title := w.Title
if i := strings.IndexAny(title, "(["); i > 0 {
title = title[:i]
}
return strings.Join(strings.Fields(strings.TrimSpace(w.Artist)+" "+strings.TrimSpace(title)), " ")
}
// For searches for one want and judges everything that comes back. It never
// returns an error: a search that could not be made is a Result saying so.
func (s *Service) For(ctx context.Context, want Want) Result {
r := Result{Query: Query(want.Want)}
if s == nil || s.Search == nil {
r.Unconfigured = true
return r
}
found, err := s.Search.Search(ctx, indexer.Query{Text: r.Query, Categories: indexer.MusicCategories})
// A partial search arrives as results and an error together.
if narrowed, answered := narrowings(err); narrowed != nil && answered {
r.Narrowed, err = narrowed, nil
}
switch {
case errors.Is(err, indexer.ErrUnconfigured):
r.Unconfigured = true
return r
case err != nil:
r.Error = indexer.FirstLine(err.Error())
return r
}
r.Considered = len(found)
cands, folded := fold(found)
r.Folded = folded
blocked := s.Memory.blocked(ctx)
for i := range cands {
decide(&cands[i], want, s.Profile, blocked)
}
sortCandidates(cands)
for _, i := range s.peek(ctx, cands) {
decide(&cands[i], want, s.Profile, blocked)
r.Looked++
}
sortCandidates(cands)
r.Candidates = cands
if s.Memory != nil && want.ID.Valid {
if err := s.Memory.Keep(ctx, want.ID, cands); err != nil {
r.KeepError = indexer.FirstLine(err.Error())
}
}
return r
}
// decide works out one candidate's verdict, in order: given up on, then
// whether it is the album, then whether the profile takes its quality, then
// whether anything leaves it to a person. A quality refusal said about the
// wrong album would be noise, so identity comes first.
func decide(c *Candidate, want Want, profile release.Profile, blocked map[string]string) {
w := want.Want
c.Verdict, c.Reason, c.Reasons = "", "", nil
if why, ok := blocked[c.InfoHash]; ok && c.InfoHash != "" {
set(c, Blocked, Hard, CodeBlocked, why)
return
}
m := release.Judge(w, c.Release)
c.YearAgrees = m.YearAgrees
if m.Veto != "" {
set(c, Rejected, Hard, m.Veto, m.Reason)
return
}
doubt := m.Doubt
if len(c.Files) > 0 {
if in := release.Inspect(c.Files); in.Format != "" {
c.Format = in.Format
}
veto, reason, fileDoubt := release.JudgeFiles(w, c.Files)
if veto != "" {
set(c, Rejected, Hard, veto, reason)
return
}
if doubt == "" {
doubt = fileDoubt
}
}
c.Tier = release.TierOf(c.Release, c.Category)
if !profile.Allows(c.Tier) {
set(c, Refused, Soft, CodeProfile, "Your quality settings don’t take "+strings.ToLower(release.Label(c.Tier))+".")
return
}
if want.Floor != "" && release.Rank(c.Tier) <= release.Rank(want.Floor) {
held := want.Held
if held == "" {
held = strings.ToLower(release.Label(want.Floor))
}
set(c, Refused, Soft, CodeNotBetter, "No better than the copy you have ("+held+").")
return
}
if doubt != "" {
set(c, Review, Soft, CodeDoubt, doubt)
return
}
if c.Seeders == 0 {
set(c, Review, Soft, CodeNoSeeders, "Nobody is sharing it right now.")
return
}
c.Verdict = Grab
c.Reason = "The album, " + strings.ToLower(release.Label(c.Tier)) + "."
if d := release.Describe(c.Release); d != "" {
c.Reason = "The album, " + d + "."
}
}
func set(c *Candidate, v Verdict, kind, code, text string) {
c.Verdict, c.Reason = v, text
c.Reasons = append(c.Reasons, Reason{Kind: kind, Code: code, Text: text})
}
// sortCandidates puts the best first, stably so the same search proposes the
// same grab twice.
func sortCandidates(cands []Candidate) {
sort.SliceStable(cands, func(a, b int) bool { return better(cands[a], cands[b]) })
}
// better says whether x ranks above y: verdict, then quality, then how many
// share it, then whether its year agrees.
func better(x, y Candidate) bool {
if order[x.Verdict] != order[y.Verdict] {
return order[x.Verdict] < order[y.Verdict]
}
if rx, ry := release.Rank(x.Tier), release.Rank(y.Tier); rx != ry {
return rx > ry
}
if x.Seeders != y.Seeders {
return x.Seeders > y.Seeders
}
return x.YearAgrees && !y.YearAgrees
}
// fold merges results that are the same torrent, by infohash. Prowlarr
// aggregates, so one release arrives once per tracker that lists it. A
// result without a hash is never folded: an absent hash is not a shared one.
func fold(found []indexer.Result) ([]Candidate, int) {
out := make([]Candidate, 0, len(found))
at := make(map[string]int, len(found))
folded := 0
for _, f := range found {
if strings.TrimSpace(f.Title) == "" {
continue
}
hash := indexer.NormalizeHash(f.InfoHash)
if hash == "" && f.Magnet != "" {
hash = indexer.MagnetHash(f.Magnet)
}
if hash != "" {
if i, seen := at[hash]; seen {
folded++
if f.Indexer != "" {
out[i].FoldedFrom = append(out[i].FoldedFrom, f.Indexer)
}
// A measured swarm beats a reported one; among like, the
// highest count.
switch {
case f.SwarmKnown && !out[i].SwarmKnown:
out[i].Seeders, out[i].SwarmKnown = f.Seeders, true
case f.SwarmKnown == out[i].SwarmKnown && f.Seeders > out[i].Seeders:
out[i].Seeders = f.Seeders
}
if f.FilesCount > 0 && out[i].FilesCount == 0 {
out[i].FilesCount = f.FilesCount
}
continue
}
at[hash] = len(out)
}
out = append(out, Candidate{
Release: release.Parse(f.Title), Source: f, InfoHash: hash, Indexer: f.Indexer,
SizeBytes: f.SizeBytes, Seeders: f.Seeders, SwarmKnown: f.SwarmKnown,
Category: f.Category, FilesCount: f.FilesCount,
})
}
return out, folded
}
// maxPeeks is how many .torrent files one search reads. Few: each goes
// through Prowlarr to a tracker, and Prowlarr answers 429 to a burst.
const maxPeeks = 3
// peek reads the .torrent of candidates that are the album but whose name
// states no format, so their files can say what they are; it returns which
// now carry a file list. A failed fetch leaves the candidate as it was.
func (s *Service) peek(ctx context.Context, cands []Candidate) []int {
f, ok := s.Search.(Fetcher)
if !ok {
return nil
}
var looked []int
for i := range cands {
if len(looked) >= maxPeeks || ctx.Err() != nil {
break
}
c := &cands[i]
if c.Tier != release.TierUnknown || (c.Verdict != Grab && c.Verdict != Refused) ||
c.Files != nil || c.Source.DownloadURL == "" || c.Source.Magnet != "" {
continue
}
body, err := f.Download(ctx, c.Source)
if err != nil {
continue
}
meta, err := release.ParseTorrent(body)
if err != nil {
continue
}
c.Files = meta.Files
if c.InfoHash == "" {
c.InfoHash = meta.InfoHash
}
if c.FilesCount == 0 {
c.FilesCount = len(meta.Files)
}
looked = append(looked, i)
}
return looked
}
// pick finds a candidate by key: the infohash first, the exact name for a
// result that has none.
func pick(cands []Candidate, key string) *Candidate {
key = strings.TrimSpace(key)
if key == "" {
return nil
}
hash := indexer.NormalizeHash(key)
for i := range cands {
if cands[i].InfoHash != "" && cands[i].InfoHash == hash {
return &cands[i]
}
}
for i := range cands {
if cands[i].InfoHash == "" && cands[i].Name == key {
return &cands[i]
}
}
return nil
}
-291
View File
@@ -1,291 +0,0 @@
package acquire
import (
"context"
"errors"
"fmt"
"strings"
"testing"
"time"
"git.fabledsword.com/bvandeusen/minstrel/internal/downloader"
"git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
"git.fabledsword.com/bvandeusen/minstrel/internal/release"
)
type fakeSearch struct {
results []indexer.Result
err error
asked []indexer.Query
}
func (f *fakeSearch) Search(_ context.Context, q indexer.Query) ([]indexer.Result, error) {
f.asked = append(f.asked, q)
return f.results, f.err
}
var ram = Want{Want: release.Want{
Artist: "Daft Punk", Title: "Random Access Memories", Year: 2013, PrimaryType: "Album",
MinTracks: 13, MaxTracks: 15,
}}
var anyQuality = release.NewProfile([]string{
"lossless_hires", "lossless", "lossy_high", "lossy_standard", "lossy_low", "lossy_poor", "lossy_trash",
})
func result(title, hash string, seeders int) indexer.Result {
return indexer.Result{Title: title, InfoHash: hash, Seeders: seeders, Indexer: "Knaben", Source: "prowlarr",
Magnet: "magnet:?xt=urn:btih:" + hash}
}
func TestFor_JudgesSortsAndFolds(t *testing.T) {
search := &fakeSearch{results: []indexer.Result{
result("Daft Punk - Random Access Memories (2013) [MP3 320]", "aaaa", 40),
result("Daft Punk Random Access Memories (Drumless Edition 2023) [Flac 24-88]", "bbbb", 90),
result("Daft Punk - Random Access Memories (2013) [FLAC]", "cccc", 12),
result("Daft Punk - Random Access Memories (2013) [FLAC]", "CCCC", 30), // the same torrent from another tracker
result("Daft Punk - Discovery (2001) [FLAC]", "dddd", 50),
}}
s := &Service{Search: search, Profile: anyQuality}
r := s.For(context.Background(), ram)
if len(search.asked) != 1 || search.asked[0].Text != "Daft Punk Random Access Memories" {
t.Fatalf("asked %+v", search.asked)
}
if r.Considered != 5 || r.Folded != 1 || len(r.Candidates) != 4 {
t.Fatalf("considered %d folded %d candidates %d", r.Considered, r.Folded, len(r.Candidates))
}
// Lossless before high-quality lossy among what can be taken; the
// folded copy keeps the higher seeder count.
first, second := r.Candidates[0], r.Candidates[1]
if first.InfoHash != "cccc" || first.Verdict != Grab || first.Seeders != 30 || first.Tier != release.TierLossless {
t.Errorf("first = %s %s %d %s", first.Name, first.Verdict, first.Seeders, first.Tier)
}
if second.InfoHash != "aaaa" || second.Verdict != Grab {
t.Errorf("second = %s %s", second.Name, second.Verdict)
}
for _, c := range r.Candidates[2:] {
if c.Verdict != Rejected || len(c.Reasons) != 1 || c.Reasons[0].Kind != Hard {
t.Errorf("%s: %s %+v", c.Name, c.Verdict, c.Reasons)
}
}
if len(r.Grabbable()) != 2 || !r.ForAPerson() {
t.Errorf("grabbable %d", len(r.Grabbable()))
}
}
func TestFor_ProfileSeedersAndDoubt(t *testing.T) {
search := &fakeSearch{results: []indexer.Result{
result("Daft Punk - Random Access Memories [MP3 128]", "aaaa", 5),
result("Daft Punk - Random Access Memories [FLAC]", "bbbb", 0),
result("Daft Punk - Random Access Memories", "cccc", 9),
}}
s := &Service{Search: search, Profile: release.NewProfile([]string{"lossless", "lossy_high"})}
r := s.For(context.Background(), ram)
verdicts := map[string]Verdict{}
for _, c := range r.Candidates {
verdicts[c.InfoHash] = c.Verdict
}
if verdicts["aaaa"] != Refused || verdicts["bbbb"] != Review || verdicts["cccc"] != Refused {
t.Errorf("verdicts = %v", verdicts)
}
if len(r.Grabbable()) != 0 || !r.ForAPerson() {
t.Error("nothing grabbable, but something for a person")
}
}
func TestFor_BlockedReleasesAreSetAside(t *testing.T) {
c := Candidate{Release: release.Parse("Daft Punk - Random Access Memories [FLAC]"), InfoHash: "aaaa", Seeders: 3}
decide(&c, ram, anyQuality, map[string]string{"aaaa": "You set this release aside."})
if c.Verdict != Blocked || c.Reason != "You set this release aside." {
t.Errorf("verdict %s %q", c.Verdict, c.Reason)
}
}
func TestFor_UnconfiguredAndFailedSearches(t *testing.T) {
if r := (&Service{}).For(context.Background(), ram); !r.Unconfigured {
t.Error("no source is unconfigured")
}
r := (&Service{Search: &fakeSearch{err: indexer.ErrUnconfigured}}).For(context.Background(), ram)
if !r.Unconfigured {
t.Error("ErrUnconfigured is unconfigured")
}
r = (&Service{Search: &fakeSearch{err: errors.New("dial tcp: refused")}}).For(context.Background(), ram)
if r.Error == "" {
t.Error("a failed search says so")
}
}
func TestFanout_ANarrowedSearchKeepsWhatAnswered(t *testing.T) {
up := &fakeSearch{results: []indexer.Result{result("Daft Punk - Random Access Memories [FLAC]", "aaaa", 3)}}
down := &fakeSearch{err: indexer.ErrUnreachable}
f := &Fanout{Sources: []Source{{Code: "prowlarr", Label: "Prowlarr", Search: up}, {Code: "torznab", Label: "Feed", Search: down}}}
r := (&Service{Search: f, Profile: anyQuality}).For(context.Background(), ram)
if r.Error != "" || len(r.Candidates) != 1 || len(r.Narrowed) != 1 || r.Narrowed[0].Source != "torznab" {
t.Errorf("narrowed search: error %q, %d candidates, narrowed %+v", r.Error, len(r.Candidates), r.Narrowed)
}
f.Sources = f.Sources[1:]
if r := (&Service{Search: f}).For(context.Background(), ram); r.Error == "" {
t.Error("every source down is a failed search")
}
}
func TestPacer_SpacesSearchesAndHoldsAfterA429(t *testing.T) {
now := time.Unix(0, 0)
var slept time.Duration
p := &Pacer{Now: func() time.Time { return now }, Sleep: func(_ context.Context, d time.Duration) error { slept += d; return nil }}
src := []Source{{Code: "prowlarr", PerHour: 120, Search: &fakeSearch{}}}
p.spend(src[0])
if !p.Wait(context.Background(), src, now.Add(time.Hour)) || slept != 30*time.Second {
t.Errorf("slept %s, want 30s", slept)
}
if p.Wait(context.Background(), src, now.Add(time.Second)) {
t.Error("waited past the deadline")
}
p.hold("prowlarr", &indexer.RateLimited{RetryAfter: 5 * time.Minute})
if at := p.readyAt(src); at.Sub(now) != 5*time.Minute {
t.Errorf("held until %s", at.Sub(now))
}
}
type fakeClient struct {
downloader.Driver
added []downloader.Add
addErr error
deleted []string
progress float64
// elsewhere: the client holds it but does not list it for Minstrel.
elsewhere bool
}
func (c *fakeClient) AddTorrent(_ context.Context, a downloader.Add) error {
c.added = append(c.added, a)
return c.addErr
}
func (c *fakeClient) Torrents(_ context.Context, hashes ...string) ([]downloader.Torrent, error) {
if c.elsewhere {
return nil, nil
}
return []downloader.Torrent{{Hash: hashes[0], Progress: c.progress}}, nil
}
func (c *fakeClient) Delete(_ context.Context, _ bool, hashes ...string) error {
c.deleted = append(c.deleted, hashes...)
return nil
}
func TestGrab_AddsAMagnetToItsOwnFolder(t *testing.T) {
client := &fakeClient{}
s := &Service{Search: &fakeSearch{results: []indexer.Result{
result("Daft Punk - Random Access Memories (2013) [FLAC]", "ABCD", 8),
}}, Profile: anyQuality, Grabber: &Downloader{Client: client, Code: "qbittorrent", Landing: "/downloads"}}
r := s.For(context.Background(), ram)
took := s.Grab(context.Background(), ram, &r.Candidates[0], r.Candidates, false, Origin{By: OriginSchedule})
if !took.Added || took.SavePath != "/downloads/abcd" {
t.Fatalf("took %+v", took)
}
if len(client.added) != 1 || !strings.Contains(strings.ToLower(client.added[0].Magnet), "abcd") || client.added[0].Tags[0] != Tag {
t.Errorf("added %+v", client.added)
}
}
func TestGrab_RefusesWhatTheVerdictWouldNotTake(t *testing.T) {
client := &fakeClient{}
s := &Service{Grabber: &Downloader{Client: client, Landing: "/downloads"}}
review := Candidate{Verdict: Review, InfoHash: "aaaa", Source: indexer.Result{Magnet: "magnet:?xt=urn:btih:aaaa"}}
if took := s.Grab(context.Background(), ram, &review, nil, false, Origin{By: OriginSchedule}); !took.Refused {
t.Error("a review was taken unforced")
}
if took := s.Grab(context.Background(), ram, &review, nil, true, Origin{By: OriginOperator}); !took.Added {
t.Errorf("a forced review was not taken: %+v", took)
}
blocked := Candidate{Verdict: Blocked, InfoHash: "bbbb"}
if took := s.Grab(context.Background(), ram, &blocked, nil, true, Origin{By: OriginOperator}); !took.Refused {
t.Error("forcing took a release set aside")
}
if took := (&Service{}).Grab(context.Background(), ram, &review, nil, true, Origin{}); !took.Unconfigured || took.Missing != MissingClient {
t.Errorf("no client: %+v", took)
}
}
func TestGrab_AClientThatAlreadyHasItCounts(t *testing.T) {
client := &fakeClient{addErr: downloader.ErrDuplicate}
s := &Service{Grabber: &Downloader{Client: client, Landing: "/downloads"}}
c := Candidate{Verdict: Grab, InfoHash: "aaaa", Source: indexer.Result{Magnet: "magnet:?xt=urn:btih:aaaa"}}
if took := s.Grab(context.Background(), ram, &c, nil, false, Origin{By: OriginSchedule}); !took.Added {
t.Errorf("a duplicate: %+v", took)
}
}
// A duplicate the client does not list for Minstrel sits under another
// category, where it would read as gone half an hour later. It is set
// aside rather than taken; forced, it is refused with the client's reason.
func TestGrab_ADuplicateHeldOutsideMinstrelIsNotTaken(t *testing.T) {
client := &fakeClient{addErr: fmt.Errorf("%w: already downloading Ohio Players (lidarr)", downloader.ErrDuplicate), elsewhere: true}
s := &Service{Grabber: &Downloader{Client: client, Landing: "/downloads"}}
c := Candidate{Verdict: Grab, InfoHash: "aaaa", Source: indexer.Result{Magnet: "magnet:?xt=urn:btih:aaaa"}}
took := s.Grab(context.Background(), ram, &c, nil, false, Origin{By: OriginSchedule})
if took.Added || !took.Refused {
t.Fatalf("took %+v", took)
}
if c.Verdict != Blocked || !strings.Contains(c.Reason, "(already downloading Ohio Players (lidarr))") {
t.Errorf("candidate %s: %q", c.Verdict, c.Reason)
}
forced := Candidate{Verdict: Grab, InfoHash: "aaaa", Source: indexer.Result{Magnet: "magnet:?xt=urn:btih:aaaa"}}
took = s.Grab(context.Background(), ram, &forced, nil, true, Origin{By: OriginOperator})
if took.Added || !strings.Contains(took.Error, "outside Minstrel's category") {
t.Errorf("forced: %+v", took)
}
}
func TestBetter_QualityThenSeedersThenYear(t *testing.T) {
flac := Candidate{Verdict: Grab, Tier: release.TierLossless, Seeders: 2}
mp3 := Candidate{Verdict: Grab, Tier: release.TierLossyHigh, Seeders: 90}
if !better(flac, mp3) || better(mp3, flac) {
t.Error("quality first among grabs")
}
busy := Candidate{Verdict: Grab, Tier: release.TierLossless, Seeders: 20}
if !better(busy, flac) {
t.Error("then seeders")
}
dated := Candidate{Verdict: Grab, Tier: release.TierLossless, Seeders: 2, YearAgrees: true}
if !better(dated, flac) {
t.Error("then the year")
}
if better(Candidate{Verdict: Review}, flac) || !better(flac, Candidate{Verdict: Review}) {
t.Error("a grab ranks above a review")
}
}
func TestLandingFor_OneFolderPerTorrent(t *testing.T) {
if got := landingFor("/downloads", "abcd", "x"); got != "/downloads/abcd" {
t.Error(got)
}
if got := landingFor("/downloads", "", "Daft Punk: R.A.M. [FLAC]"); got != "/downloads/daft-punk-r-a-m--flac" {
t.Error(got)
}
}
func TestDecide_AnUpgradeTakesOnlyABetterCopy(t *testing.T) {
up := ram
up.Floor, up.Held = release.TierLossyLow, "MP3, about 192 kbps"
same := Candidate{Release: release.Parse("Daft Punk - Random Access Memories [MP3 192]"), InfoHash: "aaaa", Seeders: 3}
decide(&same, up, anyQuality, nil)
if same.Verdict != Refused || same.Reasons[0].Code != CodeNotBetter ||
same.Reason != "No better than the copy you have (MP3, about 192 kbps)." {
t.Errorf("same quality: %s %q", same.Verdict, same.Reason)
}
flac := Candidate{Release: release.Parse("Daft Punk - Random Access Memories [FLAC]"), InfoHash: "bbbb", Seeders: 3}
decide(&flac, up, anyQuality, nil)
if flac.Verdict != Grab {
t.Errorf("a FLAC over a 192 MP3: %s %q", flac.Verdict, flac.Reason)
}
// A video rip has no floor: any clean copy is better.
rip := ram
decide(&same, rip, anyQuality, nil)
if same.Verdict != Grab {
t.Errorf("no floor: %s %q", same.Verdict, same.Reason)
}
}
-215
View File
@@ -1,215 +0,0 @@
// Package acquire turns the configured indexers and download clients into
// working clients.
//
// ONE BUILDER PER KIND. SourceFor makes a search source from a row and
// DownloadClient makes a download client, path mapping included. The grab,
// the importer, the stall watchdog and the settings page's Test all build
// through them, so a Test that passes vouches for the path real traffic takes.
// Clients are rebuilt from the row on every use, so a changed address or a new
// key takes effect on the next call.
package acquire
import (
"context"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/downloader"
"git.fabledsword.com/bvandeusen/minstrel/internal/exchange"
"git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
"git.fabledsword.com/bvandeusen/minstrel/internal/prowlarr"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
)
// Kinds: which list a row appears in.
const (
KindSearch = "search"
KindDownload = "download"
)
// Jobs a row can do. An exchange row does each its switch allows; every
// other row does the one its kind names.
const (
JobSearch = "search"
JobDownload = "download"
JobEnrichment = "enrichment"
)
// Protocols: which client a row builds.
const (
ProtocolProwlarr = "prowlarr"
ProtocolTorznab = "torznab"
ProtocolQBittorrent = "qbittorrent"
ProtocolExchange = "exchange"
)
// DefaultCategory files everything Minstrel adds to a download client.
const DefaultCategory = "minstrel"
// addable is what a new row of a protocol starts as.
type addable struct {
kind string
label string
category string
timeoutMs int32
perHour int32
}
// catalogue is every protocol an admin can add. Exchange starts unpaced
// (it searches its own database); the others at 120 searches an hour, which
// keeps a Prowlarr's indexers under the rate most trackers allow.
var catalogue = map[string]addable{
ProtocolProwlarr: {kind: KindSearch, label: "Prowlarr", timeoutMs: 60000, perHour: 120},
ProtocolTorznab: {kind: KindSearch, label: "Torznab feed", timeoutMs: 30000, perHour: 120},
ProtocolQBittorrent: {kind: KindDownload, label: "qBittorrent", category: DefaultCategory, timeoutMs: 15000, perHour: 0},
ProtocolExchange: {kind: KindDownload, label: "FabledExchange", category: DefaultCategory, timeoutMs: 30000, perHour: 0},
}
// Addable says whether protocol is one an admin can add.
func Addable(protocol string) bool {
_, ok := catalogue[protocol]
return ok
}
// Searches says whether a row is a search source.
func Searches(row dbq.AcquisitionService) bool {
if row.Protocol == ProtocolExchange {
return row.ServesSearch
}
return row.Kind == KindSearch
}
// Fetches says whether a row is a download client.
func Fetches(row dbq.AcquisitionService) bool {
if row.Protocol == ProtocolExchange {
return row.ServesDownloads
}
return row.Kind == KindDownload
}
// Searcher is a search source.
type Searcher interface {
Search(ctx context.Context, q indexer.Query) ([]indexer.Result, error)
}
// Fetcher fetches a result's .torrent through the source that offered it.
type Fetcher interface {
Download(ctx context.Context, r indexer.Result) ([]byte, error)
}
// Source is one configured search source.
type Source struct {
Code string
Label string
// PerHour is the most searches the schedule sends it an hour; 0 = unpaced.
PerHour int
Search Searcher
// Fetch is nil for a source whose results are all magnets.
Fetch Fetcher
}
// SourceFor builds the search source a row describes.
func SourceFor(row dbq.AcquisitionService) Source {
src := Source{Code: row.Code, Label: row.Label, PerHour: int(row.SearchesPerHour)}
timeout := msDuration(row.TimeoutMs)
switch row.Protocol {
case ProtocolProwlarr:
c := &prowlarr.Client{Config: prowlarr.Config{
Code: row.Code, BaseURL: row.BaseUrl, APIKey: row.ApiKey, Timeout: timeout,
}}
src.Search, src.Fetch = c, c
case ProtocolExchange:
src.Search = ExchangeFor(row)
default:
c := &indexer.Torznab{Config: indexer.TorznabConfig{
Code: row.Code, FeedURL: row.BaseUrl, APIKey: row.ApiKey, Timeout: timeout,
}}
src.Search, src.Fetch = c, c
}
return src
}
// ExchangeFor builds the search and enrichment client for an exchange row.
func ExchangeFor(row dbq.AcquisitionService) *exchange.Client {
return &exchange.Client{
Code: row.Code,
Config: exchange.Config{
BaseURL: row.BaseUrl, APIKey: row.ApiKey, Timeout: msDuration(row.TimeoutMs),
},
}
}
// DownloadClient builds the download client a row describes, wrapped in its
// path mapping.
func DownloadClient(row dbq.AcquisitionService) (downloader.Driver, error) {
d, err := downloader.For(row.Protocol, downloader.Config{
BaseURL: row.BaseUrl, Username: row.Username, Password: row.Password,
APIKey: row.ApiKey, Category: row.Category, Timeout: msDuration(row.TimeoutMs),
})
if err != nil {
return nil, err
}
return downloader.Mapped(d, downloader.PathMap{Local: row.LocalPath, Remote: row.RemotePath}), nil
}
// DownloadClientFor builds the download client a grab went to, by the code
// of its acquisition_services row.
func DownloadClientFor(ctx context.Context, pool *pgxpool.Pool, code string) (downloader.Driver, error) {
row, err := getService(ctx, dbq.New(pool), code)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, fmt.Errorf("the download client %q is no longer set up", code)
}
return nil, err
}
return DownloadClient(row)
}
// Sources builds every enabled search source.
func Sources(ctx context.Context, pool *pgxpool.Pool) ([]Source, error) {
rows, err := usableServices(ctx, dbq.New(pool), JobSearch)
if err != nil {
return nil, err
}
out := make([]Source, len(rows))
for i, r := range rows {
out[i] = SourceFor(r)
}
return out, nil
}
// getService and usableServices are the only reads of a full
// acquisition_services row. They hand back the credentials opened, so the
// clients built from a row are given the key and password themselves.
func getService(ctx context.Context, q *dbq.Queries, code string) (dbq.AcquisitionService, error) {
row, err := q.GetAcquisitionService(ctx, code)
if err != nil {
return row, err
}
return revealed(row), nil
}
func usableServices(ctx context.Context, q *dbq.Queries, job string) ([]dbq.AcquisitionService, error) {
rows, err := q.ListUsableAcquisitionServices(ctx, job)
if err != nil {
return nil, err
}
for i := range rows {
rows[i] = revealed(rows[i])
}
return rows, nil
}
// revealed is row with its sealed credentials opened.
func revealed(row dbq.AcquisitionService) dbq.AcquisitionService {
row.ApiKey = secret.Reveal(secret.AcquisitionAPIKey.Purpose(), row.Code, row.ApiKey)
row.Password = secret.Reveal(secret.AcquisitionPassword.Purpose(), row.Code, row.Password)
return row
}
func msDuration(ms int32) time.Duration { return time.Duration(ms) * time.Millisecond }
-200
View File
@@ -1,200 +0,0 @@
package acquire
import (
"context"
"errors"
"fmt"
"net/url"
"strings"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
)
// ErrNotFound is a code with no row.
var ErrNotFound = errors.New("no such service")
// ErrTooMany is a twenty-first row of one protocol.
var ErrTooMany = errors.New("there are already twenty of these")
// InvalidError is a setting an admin has to change, in words for the
// settings card.
type InvalidError struct {
Field string
Message string
}
func (e *InvalidError) Error() string { return e.Message }
func invalid(field, msg string) error { return &InvalidError{Field: field, Message: msg} }
// maxPerProtocol bounds the codes tried for a new row. Nobody runs twenty
// Prowlarrs; the bound keeps a loop over a full table finite.
const maxPerProtocol = 20
// Store reads and writes acquisition_services.
type Store struct {
pool *pgxpool.Pool
}
// NewStore returns a Store over pool.
func NewStore(pool *pgxpool.Pool) *Store { return &Store{pool: pool} }
// List returns every row without its secrets.
func (s *Store) List(ctx context.Context) ([]dbq.ListAcquisitionServicesRow, error) {
return dbq.New(s.pool).ListAcquisitionServices(ctx)
}
// Get returns one row, secrets included: for building a client only.
func (s *Store) Get(ctx context.Context, code string) (dbq.AcquisitionService, error) {
row, err := getService(ctx, dbq.New(s.pool), code)
if errors.Is(err, pgx.ErrNoRows) {
return row, ErrNotFound
}
return row, err
}
// Create adds a row of protocol, switched off and with no address. Its code
// is the protocol for the first, then protocol-2, protocol-3.
func (s *Store) Create(ctx context.Context, protocol string) (dbq.AcquisitionService, error) {
a, ok := catalogue[protocol]
if !ok {
return dbq.AcquisitionService{}, invalid("protocol", "One of prowlarr, torznab, qbittorrent, exchange.")
}
q := dbq.New(s.pool)
for n := 1; n <= maxPerProtocol; n++ {
code, label := protocol, a.label
if n > 1 {
code, label = fmt.Sprintf("%s-%d", protocol, n), fmt.Sprintf("%s %d", a.label, n)
}
row, err := q.CreateAcquisitionService(ctx, dbq.CreateAcquisitionServiceParams{
Code: code, Kind: a.kind, Protocol: protocol, Label: label,
Category: a.category, TimeoutMs: a.timeoutMs, SearchesPerHour: a.perHour,
})
if errors.Is(err, pgx.ErrNoRows) {
continue
}
return row, err
}
return dbq.AcquisitionService{}, ErrTooMany
}
// Update is everything on the settings form except the credentials.
type Update struct {
Label string
Enabled bool
BaseURL string
Username string
Category string
TimeoutMs int32
ServesSearch bool
ServesDownloads bool
ServesEnrichment bool
LocalPath string
RemotePath string
SearchesPerHour int32
}
// Save validates u and writes it to code's row.
func (s *Store) Save(ctx context.Context, code string, u Update) (dbq.AcquisitionService, error) {
u.Label = strings.TrimSpace(u.Label)
u.BaseURL = strings.TrimSpace(u.BaseURL)
u.Username = strings.TrimSpace(u.Username)
u.Category = strings.TrimSpace(u.Category)
u.LocalPath = strings.TrimSpace(u.LocalPath)
u.RemotePath = strings.TrimSpace(u.RemotePath)
if err := u.validate(); err != nil {
return dbq.AcquisitionService{}, err
}
row, err := dbq.New(s.pool).UpdateAcquisitionService(ctx, dbq.UpdateAcquisitionServiceParams{
Code: code, Label: u.Label, Enabled: u.Enabled, BaseUrl: u.BaseURL,
Username: u.Username, Category: u.Category, TimeoutMs: u.TimeoutMs,
ServesSearch: u.ServesSearch, ServesDownloads: u.ServesDownloads,
ServesEnrichment: u.ServesEnrichment, LocalPath: u.LocalPath,
RemotePath: u.RemotePath, SearchesPerHour: u.SearchesPerHour,
})
if errors.Is(err, pgx.ErrNoRows) {
return row, ErrNotFound
}
return row, err
}
// validate mirrors the table's CHECKs, so a bad value is named in the card's
// own terms before Postgres refuses it.
func (u Update) validate() error {
if u.Label == "" {
return invalid("label", "Name: give it a name.")
}
if u.Enabled && u.BaseURL == "" {
return invalid("base_url", "Address: give the address before switching this on.")
}
if u.BaseURL != "" {
parsed, err := url.Parse(u.BaseURL)
if err != nil || (parsed.Scheme != "http" && parsed.Scheme != "https") || parsed.Host == "" {
return invalid("base_url", "Address: start with http:// or https://.")
}
}
if u.TimeoutMs < 1000 || u.TimeoutMs > 300000 {
return invalid("timeout_ms", "Give up after: between 1000 and 300000 milliseconds.")
}
if !strings.HasPrefix(u.LocalPath, "/") {
return invalid("local_path", "Downloads folder as Minstrel sees it: a full path, starting with /.")
}
if u.RemotePath != "" && !strings.HasPrefix(u.RemotePath, "/") {
return invalid("remote_path", "Downloads folder as the client sees it: a full path starting with /, or empty when it is the same path.")
}
if u.SearchesPerHour < 0 || u.SearchesPerHour > 3600 {
return invalid("searches_per_hour", "Searches per hour: between 0 and 3600. 0 means no limit.")
}
return nil
}
// Delete removes code's row.
func (s *Store) Delete(ctx context.Context, code string) error {
n, err := dbq.New(s.pool).DeleteAcquisitionService(ctx, code)
if err != nil {
return err
}
if n == 0 {
return ErrNotFound
}
return nil
}
// SetAPIKey stores a key, trimmed and sealed; empty clears it.
func (s *Store) SetAPIKey(ctx context.Context, code, key string) error {
sealed, err := secret.Seal(secret.AcquisitionAPIKey.Purpose(), code, strings.TrimSpace(key))
if err != nil {
return err
}
n, err := dbq.New(s.pool).SetAcquisitionServiceAPIKey(ctx, dbq.SetAcquisitionServiceAPIKeyParams{
Code: code, ApiKey: sealed,
})
return affected(n, err)
}
// SetPassword stores a password as typed (spaces are part of a password),
// sealed; empty clears it.
func (s *Store) SetPassword(ctx context.Context, code, password string) error {
sealed, err := secret.Seal(secret.AcquisitionPassword.Purpose(), code, password)
if err != nil {
return err
}
n, err := dbq.New(s.pool).SetAcquisitionServicePassword(ctx, dbq.SetAcquisitionServicePasswordParams{
Code: code, Password: sealed,
})
return affected(n, err)
}
func affected(n int64, err error) error {
if err != nil {
return err
}
if n == 0 {
return ErrNotFound
}
return nil
}
-118
View File
@@ -1,118 +0,0 @@
package acquire
import (
"context"
"errors"
"io"
"log/slog"
"os"
"testing"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/db"
"git.fabledsword.com/bvandeusen/minstrel/internal/dbtest"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
)
func newPool(t *testing.T) *pgxpool.Pool {
t.Helper()
if testing.Short() {
t.Skip("skipping integration test in -short mode")
}
dsn := os.Getenv("MINSTREL_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("MINSTREL_TEST_DATABASE_URL not set")
}
if err := db.Migrate(dsn, slog.New(slog.NewTextHandler(io.Discard, nil))); err != nil {
t.Fatalf("migrate: %v", err)
}
pool, err := pgxpool.New(context.Background(), dsn)
if err != nil {
t.Fatalf("pool: %v", err)
}
t.Cleanup(pool.Close)
dbtest.ResetDB(t, pool)
return pool
}
func TestStore_AddsConfiguresAndRemoves_Integration(t *testing.T) {
ctx := context.Background()
s := NewStore(newPool(t))
first, err := s.Create(ctx, ProtocolExchange)
if err != nil {
t.Fatalf("create: %v", err)
}
second, err := s.Create(ctx, ProtocolExchange)
if err != nil {
t.Fatalf("create a second: %v", err)
}
if first.Code != "exchange" || second.Code != "exchange-2" || second.Label != "FabledExchange 2" {
t.Errorf("codes = %q, %q (%q); want exchange, exchange-2", first.Code, second.Code, second.Label)
}
// A new exchange is off, does every job, files under minstrel and is
// unpaced.
if first.Enabled || first.Kind != KindDownload || !first.ServesSearch || !first.ServesDownloads ||
!first.ServesEnrichment || first.Category != DefaultCategory || first.SearchesPerHour != 0 || first.LocalPath != "/downloads" {
t.Errorf("a new exchange = %+v", first)
}
if _, err := s.Create(ctx, "transmission"); !errors.As(err, new(*InvalidError)) {
t.Errorf("an unknown protocol: err = %v", err)
}
u := Update{
Label: "Exchange", Enabled: true, BaseURL: "http://exchange:8080", Category: "minstrel",
TimeoutMs: 20000, ServesSearch: true, LocalPath: "/downloads", RemotePath: "/data/torrents",
}
saved, err := s.Save(ctx, first.Code, u)
if err != nil {
t.Fatalf("save: %v", err)
}
if !saved.Enabled || saved.ServesDownloads || saved.RemotePath != "/data/torrents" {
t.Errorf("saved = %+v", saved)
}
if _, err := s.Save(ctx, "nope", u); !errors.Is(err, ErrNotFound) {
t.Errorf("saving a missing row: err = %v", err)
}
// Secrets are write-only: the list says whether one is set, never what.
if err := s.SetAPIKey(ctx, first.Code, " key "); err != nil {
t.Fatalf("set key: %v", err)
}
full, err := s.Get(ctx, first.Code)
if err != nil || full.ApiKey != "key" {
t.Errorf("stored key = %q (err %v), want it trimmed", full.ApiKey, err)
}
// The column holds the key sealed, never the key.
var raw string
if err := s.pool.QueryRow(ctx,
"SELECT api_key FROM acquisition_services WHERE code = $1", first.Code).Scan(&raw); err != nil {
t.Fatalf("read api_key: %v", err)
}
if raw == "key" || !secret.IsSealed(raw) {
t.Errorf("api_key column = %q, want it sealed", raw)
}
rows, err := s.List(ctx)
if err != nil || len(rows) != 2 {
t.Fatalf("list: %d rows, err %v", len(rows), err)
}
for _, r := range rows {
if r.Code == first.Code && !r.HasApiKey {
t.Error("the list should say a key is set")
}
}
// Only the exchange with search switched on is a search source.
srcs, err := Sources(ctx, s.pool)
if err != nil || len(srcs) != 1 || srcs[0].Code != first.Code {
t.Errorf("sources = %+v, err %v; want only the enabled exchange", srcs, err)
}
if err := s.Delete(ctx, second.Code); err != nil {
t.Fatalf("delete: %v", err)
}
if err := s.Delete(ctx, second.Code); !errors.Is(err, ErrNotFound) {
t.Errorf("deleting twice: err = %v", err)
}
}
-134
View File
@@ -1,134 +0,0 @@
package acquire
import (
"context"
"errors"
"fmt"
"os"
"strings"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/exchange"
"git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
)
// TestResult is what the settings card says after Test.
type TestResult struct {
OK bool `json:"ok"`
Detail string `json:"detail"`
}
// testQuery is a search every music indexer should answer.
const testQuery = "Radiohead"
// Test reaches the service a row describes, through the same builders real
// traffic uses. It runs whether or not the row is switched on: paste the
// address, test it, then switch it on.
func Test(ctx context.Context, row dbq.AcquisitionService) TestResult {
if strings.TrimSpace(row.BaseUrl) == "" {
return TestResult{Detail: "No address is set, so there is nothing to reach yet."}
}
if row.Protocol == ProtocolExchange {
return testExchange(ctx, row)
}
if row.Kind == KindSearch {
return testSearch(ctx, row)
}
return testDownload(ctx, row)
}
// testExchange runs each job the row has switched on and reports them all.
func testExchange(ctx context.Context, row dbq.AcquisitionService) TestResult {
var parts []string
ok := true
run := func(name string, on bool, test func() TestResult) {
if !on {
return
}
r := test()
ok = ok && r.OK
parts = append(parts, name+": "+r.Detail)
}
run("Downloads", row.ServesDownloads, func() TestResult { return testDownload(ctx, row) })
run("Search", row.ServesSearch, func() TestResult { return testSearch(ctx, row) })
run("Release details", row.ServesEnrichment, func() TestResult { return testEnrichment(ctx, row) })
if len(parts) == 0 {
return TestResult{Detail: "Every job of this FabledExchange is switched off, so there is nothing to test."}
}
return TestResult{OK: ok, Detail: strings.Join(parts, " ")}
}
// testSearch runs a real search.
func testSearch(ctx context.Context, row dbq.AcquisitionService) TestResult {
src := SourceFor(row)
results, err := src.Search.Search(ctx, indexer.Query{Text: testQuery, Categories: indexer.MusicCategories})
switch {
case errors.Is(err, indexer.ErrUnconfigured):
return TestResult{Detail: "Add this service's address, and the API key it shows on its own settings page."}
case errors.Is(err, indexer.ErrAuth):
return TestResult{Detail: indexer.FirstLine(err.Error()) + ". Check it on the service's own settings page."}
case err != nil:
return TestResult{Detail: indexer.FirstLine(err.Error())}
case len(results) == 0:
return TestResult{OK: true, Detail: "Connected, but a test search for music found nothing. Check that it has an indexer with music categories."}
}
return TestResult{OK: true, Detail: fmt.Sprintf("Connected. %d results for a test search.", len(results))}
}
// testDownload lists the client's torrents, which proves the session and not
// only the login, then checks the downloads folder from Minstrel's side.
func testDownload(ctx context.Context, row dbq.AcquisitionService) TestResult {
c, err := DownloadClient(row)
if err != nil {
return TestResult{Detail: err.Error()}
}
ts, err := c.Torrents(ctx)
if err != nil {
return TestResult{Detail: indexer.FirstLine(err.Error())}
}
detail := fmt.Sprintf("Connected. %d torrents.", len(ts))
if row.Category != "" {
detail = fmt.Sprintf("Connected. %d torrents in the %q category.", len(ts), row.Category)
}
if up, err := c.Connected(ctx); err == nil && !up {
detail += " The client reports it is not connected to any peers, so downloads will stall."
}
if problem := landingProblem(row.LocalPath); problem != "" {
return TestResult{Detail: detail + " " + problem}
}
return TestResult{OK: true, Detail: detail}
}
// landingProblem checks Minstrel can see and write the downloads folder: it
// moves finished downloads out of it. Only the local path can be checked;
// the client's own name for it cannot be seen from here.
func landingProblem(dir string) string {
info, err := os.Stat(dir)
switch {
case err != nil:
return fmt.Sprintf("But Minstrel cannot see its downloads folder at %s. Mount the download client's downloads folder there, or change where Minstrel looks for it.", dir)
case !info.IsDir():
return fmt.Sprintf("But %s is a file, not the downloads folder.", dir)
}
probe, err := os.CreateTemp(dir, ".minstrel-probe-*")
if err != nil {
return fmt.Sprintf("But Minstrel cannot write to %s, so it could not move a finished download out of it. Check the mount is not read-only and Minstrel's user can write there.", dir)
}
name := probe.Name()
_ = probe.Close()
_ = os.Remove(name)
return ""
}
// testEnrichment asks about a hash no torrent has. A JSON "not crawled" is a
// healthy answer; a 404 from something that is not Exchange is not.
func testEnrichment(ctx context.Context, row dbq.AcquisitionService) TestResult {
_, err := ExchangeFor(row).Torrent(ctx, strings.Repeat("0", 40))
switch {
case err == nil, errors.Is(err, exchange.ErrUnknown):
return TestResult{OK: true, Detail: "Connected. The index answered."}
case errors.Is(err, exchange.ErrUnconfigured):
return TestResult{Detail: "No address is set, so there is nothing to reach yet."}
}
return TestResult{Detail: indexer.FirstLine(err.Error())}
}
-210
View File
@@ -1,210 +0,0 @@
package api
import (
"encoding/json"
"errors"
"io"
"net/http"
"github.com/go-chi/chi/v5"
"git.fabledsword.com/bvandeusen/minstrel/internal/acquire"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
// Indexers and download clients, admin side. Rows an admin
// adds and configures; nothing is seeded. Secrets are write-only: no response
// carries one, only whether it is set.
// acquisitionBodyLimit caps a settings body. The largest is a full form of
// short strings.
const acquisitionBodyLimit = 64 << 10
type acquisitionServiceResp struct {
Code string `json:"code"`
Kind string `json:"kind"`
Protocol string `json:"protocol"`
Label string `json:"label"`
Enabled bool `json:"enabled"`
BaseURL string `json:"base_url"`
HasAPIKey bool `json:"has_api_key"`
HasPassword bool `json:"has_password"`
Username string `json:"username"`
Category string `json:"category"`
TimeoutMs int32 `json:"timeout_ms"`
ServesSearch bool `json:"serves_search"`
ServesDownloads bool `json:"serves_downloads"`
ServesEnrichment bool `json:"serves_enrichment"`
LocalPath string `json:"local_path"`
RemotePath string `json:"remote_path"`
SearchesPerHour int32 `json:"searches_per_hour"`
// Usable is switched on and addressed: the state the jobs read.
Usable bool `json:"usable"`
}
func acquisitionRespOf(r dbq.AcquisitionService) acquisitionServiceResp {
return acquisitionServiceResp{
Code: r.Code, Kind: r.Kind, Protocol: r.Protocol, Label: r.Label,
Enabled: r.Enabled, BaseURL: r.BaseUrl,
HasAPIKey: r.ApiKey != "", HasPassword: r.Password != "",
Username: r.Username, Category: r.Category, TimeoutMs: r.TimeoutMs,
ServesSearch: r.ServesSearch, ServesDownloads: r.ServesDownloads,
ServesEnrichment: r.ServesEnrichment,
LocalPath: r.LocalPath, RemotePath: r.RemotePath,
SearchesPerHour: r.SearchesPerHour,
Usable: r.Enabled && r.BaseUrl != "",
}
}
// handleListAcquisitionServices implements GET /api/admin/acquisition/services.
func (h *handlers) handleListAcquisitionServices(w http.ResponseWriter, r *http.Request) {
rows, err := h.acquisition.List(r.Context())
if err != nil {
writeErrWithLog(w, h.logger, "admin: list acquisition services", apierror.Internal(err))
return
}
out := make([]acquisitionServiceResp, len(rows))
for i, row := range rows {
out[i] = acquisitionRespOf(dbq.AcquisitionService{
Code: row.Code, Kind: row.Kind, Protocol: row.Protocol, Label: row.Label,
Enabled: row.Enabled, BaseUrl: row.BaseUrl, Username: row.Username,
Category: row.Category, TimeoutMs: row.TimeoutMs,
ServesSearch: row.ServesSearch, ServesDownloads: row.ServesDownloads,
ServesEnrichment: row.ServesEnrichment, LocalPath: row.LocalPath,
RemotePath: row.RemotePath, SearchesPerHour: row.SearchesPerHour,
})
out[i].HasAPIKey, out[i].HasPassword = row.HasApiKey, row.HasPassword
}
writeJSON(w, http.StatusOK, map[string]any{"services": out})
}
// handleCreateAcquisitionService implements POST
// /api/admin/acquisition/services with {"protocol":
// "prowlarr"|"torznab"|"qbittorrent"|"exchange"}. The row starts switched off,
// with no address.
func (h *handlers) handleCreateAcquisitionService(w http.ResponseWriter, r *http.Request) {
var body struct {
Protocol string `json:"protocol"`
}
if !decodeAcquisitionBody(w, r, &body) {
return
}
row, err := h.acquisition.Create(r.Context(), body.Protocol)
if err != nil {
h.writeAcquisitionErr(w, "admin: create acquisition service", err)
return
}
writeJSON(w, http.StatusCreated, acquisitionRespOf(row))
}
type updateAcquisitionBody struct {
Label string `json:"label"`
Enabled bool `json:"enabled"`
BaseURL string `json:"base_url"`
Username string `json:"username"`
Category string `json:"category"`
TimeoutMs int32 `json:"timeout_ms"`
ServesSearch bool `json:"serves_search"`
ServesDownloads bool `json:"serves_downloads"`
ServesEnrichment bool `json:"serves_enrichment"`
LocalPath string `json:"local_path"`
RemotePath string `json:"remote_path"`
SearchesPerHour int32 `json:"searches_per_hour"`
}
// handleUpdateAcquisitionService implements PUT
// /api/admin/acquisition/services/{code}: the whole form except credentials.
func (h *handlers) handleUpdateAcquisitionService(w http.ResponseWriter, r *http.Request) {
var body updateAcquisitionBody
if !decodeAcquisitionBody(w, r, &body) {
return
}
row, err := h.acquisition.Save(r.Context(), chi.URLParam(r, "code"), acquire.Update(body))
if err != nil {
h.writeAcquisitionErr(w, "admin: update acquisition service", err)
return
}
writeJSON(w, http.StatusOK, acquisitionRespOf(row))
}
// handleDeleteAcquisitionService implements DELETE
// /api/admin/acquisition/services/{code}.
func (h *handlers) handleDeleteAcquisitionService(w http.ResponseWriter, r *http.Request) {
if err := h.acquisition.Delete(r.Context(), chi.URLParam(r, "code")); err != nil {
h.writeAcquisitionErr(w, "admin: delete acquisition service", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleTestAcquisitionService implements POST
// /api/admin/acquisition/services/{code}/test. POST because it reaches
// another machine, which a prefetch must not trigger. Always 200 with
// {ok, detail}: a failed connection is the answer, not an error.
func (h *handlers) handleTestAcquisitionService(w http.ResponseWriter, r *http.Request) {
row, err := h.acquisition.Get(r.Context(), chi.URLParam(r, "code"))
if err != nil {
h.writeAcquisitionErr(w, "admin: test acquisition service", err)
return
}
writeJSON(w, http.StatusOK, acquire.Test(r.Context(), row))
}
// handleSetAcquisitionSecret implements PUT and DELETE on
// /api/admin/acquisition/services/{code}/api-key and .../password. PUT takes
// {"value": "..."}; DELETE clears it. 204 with no body either way: echoing
// anything here is how a secret ends up in a response.
func (h *handlers) handleSetAcquisitionSecret(field string) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
value := ""
if r.Method == http.MethodPut {
var body struct {
Value string `json:"value"`
}
if !decodeAcquisitionBody(w, r, &body) {
return
}
if body.Value == "" {
writeErr(w, apierror.BadRequest("invalid_setting", "Paste a value, or use Remove to clear the one that is set."))
return
}
value = body.Value
}
code := chi.URLParam(r, "code")
var err error
if field == "password" {
err = h.acquisition.SetPassword(r.Context(), code, value)
} else {
err = h.acquisition.SetAPIKey(r.Context(), code, value)
}
if err != nil {
h.writeAcquisitionErr(w, "admin: set acquisition "+field, err)
return
}
w.WriteHeader(http.StatusNoContent)
}
}
func decodeAcquisitionBody(w http.ResponseWriter, r *http.Request, into any) bool {
err := json.NewDecoder(http.MaxBytesReader(w, r.Body, acquisitionBodyLimit)).Decode(into)
if err != nil && !errors.Is(err, io.EOF) {
writeErr(w, apierror.BadRequest("invalid_body", "malformed JSON"))
return false
}
return true
}
func (h *handlers) writeAcquisitionErr(w http.ResponseWriter, msg string, err error) {
var invalid *acquire.InvalidError
switch {
case errors.As(err, &invalid):
writeErr(w, apierror.BadRequest("invalid_setting", invalid.Message))
case errors.Is(err, acquire.ErrNotFound):
writeErr(w, apierror.NotFound("service"))
case errors.Is(err, acquire.ErrTooMany):
writeErr(w, apierror.Conflict("too_many_services", "There are already twenty of these. Remove one you no longer use first."))
default:
writeErrWithLog(w, h.logger, msg, apierror.Internal(err))
}
}
+5 -10
View File
@@ -459,21 +459,19 @@ type refetchDuplicateRequest struct {
KeepTrackID string `json:"keep_track_id"`
}
// refetchDuplicateResponse reports what went. Blocked names the releases set
// aside; empty when the album came from before Minstrel downloaded anything.
// refetchDuplicateResponse reports what went.
type refetchDuplicateResponse struct {
KeptTrackID *string `json:"kept_track_id"`
RemovedPaths []string `json:"removed_paths"`
Blocked []string `json:"blocked"`
}
// handleRefetchDuplicateGroup implements POST
// /api/admin/library/duplicates/{id}/refetch: settle a group of one
// recording in several slots of an album the way the resolver would, with an
// admin's answer. The copies in slots that are not the recording's are
// removed and marked missing, the release they came from is set aside, and the
// album is wanted again. Merge is the wrong action there: it keeps one title
// whether or not it is the audio's, and fetches nothing.
// removed and marked missing, so the slot shows as missing until a good copy
// arrives in the library. Merge is the wrong action there: it keeps one title
// whether or not it is the audio's.
//
// Errors:
// - 409 library_not_writable / 500 file_delete_failed when a file could not be
@@ -525,10 +523,7 @@ func (h *handlers) handleRefetchDuplicateGroup(w http.ResponseWriter, r *http.Re
return
}
resp := refetchDuplicateResponse{RemovedPaths: make([]string, 0, len(res.Removed)), Blocked: res.Blocked}
if resp.Blocked == nil {
resp.Blocked = []string{}
}
resp := refetchDuplicateResponse{RemovedPaths: make([]string, 0, len(res.Removed))}
if res.Kept != nil {
id := uuidToString(res.Kept.TrackID)
resp.KeptTrackID = &id
-285
View File
@@ -1,285 +0,0 @@
package api
import (
"encoding/json"
"errors"
"net/http"
"github.com/jackc/pgx/v5"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/importer"
"git.fabledsword.com/bvandeusen/minstrel/internal/lifecycle"
)
// Downloads and their imports: the Wanted view's Activity tab,
// a person's "import anyway", "refuse" and "retry", and the Blocked tab.
// downloadView is one grab and what its import made of it.
type downloadView struct {
ID string `json:"id"`
WantID *string `json:"want_id"`
WantTitle string `json:"want_title"`
WantArtist string `json:"want_artist"`
Name string `json:"name"`
Indexer string `json:"indexer"`
Client string `json:"client"`
Quality string `json:"quality"`
Origin string `json:"origin"`
GrabbedAt string `json:"grabbed_at"`
EndedAt *string `json:"ended_at"`
EndReason string `json:"end_reason"`
Progress float32 `json:"progress"`
ImportState string `json:"import_state"`
ImportReason string `json:"import_reason"`
ImportedAt *string `json:"imported_at"`
ImportedFiles int32 `json:"imported_files"`
Destination string `json:"destination"`
Edition string `json:"edition,omitempty"`
Evidence *importer.Evidence `json:"evidence,omitempty"`
Files []string `json:"files"`
Skipped int `json:"skipped"`
RehomedFrom string `json:"rehomed_from,omitempty"`
// EndDetail says why an ended download ended.
EndDetail string `json:"end_detail"`
// Seeding: imported and still sharing; Ratio its share ratio as last
// read; SeedReleasedAt when it was removed once it had shared enough.
Seeding bool `json:"seeding"`
Ratio float32 `json:"ratio"`
SeedReleasedAt *string `json:"seed_released_at"`
// Condition is what is wrong with it now, if anything.
Condition *conditionView `json:"condition"`
}
// conditionView is an open row of the intervention ledger: what was seen,
// since when, and how many times it was retried.
type conditionView struct {
Trigger string `json:"trigger"`
Detail string `json:"detail"`
Retries int32 `json:"retries"`
Since string `json:"since"`
}
func downloadFrom(g dbq.ListGrabsRow) downloadView {
v := downloadView{
ID: uuidToString(g.ID), Name: g.Name, Indexer: g.Indexer, Client: g.Client, Quality: g.Quality,
Origin: g.Origin, GrabbedAt: formatTimestamp(g.GrabbedAt), EndReason: g.EndReason,
Progress: g.Progress, ImportState: g.ImportState, ImportReason: g.ImportReason,
ImportedFiles: g.ImportedFiles, Destination: g.Destination, Files: []string{},
EndDetail: g.EndDetail, Seeding: g.Seeding, Ratio: g.Ratio,
}
if g.SeedReleasedAt.Valid {
at := formatTimestamp(g.SeedReleasedAt)
v.SeedReleasedAt = &at
}
if g.Condition != nil {
v.Condition = &conditionView{Trigger: *g.Condition, Detail: deref(g.ConditionDetail),
Since: formatTimestamp(g.ConditionSince)}
if g.ConditionRetries != nil {
v.Condition.Retries = *g.ConditionRetries
}
}
if g.WantID.Valid {
id := uuidToString(g.WantID)
v.WantID = &id
}
if g.WantTitle != nil {
v.WantTitle = *g.WantTitle
}
if g.WantArtist != nil {
v.WantArtist = *g.WantArtist
}
if g.EndedAt.Valid {
at := formatTimestamp(g.EndedAt)
v.EndedAt = &at
}
if g.ImportedAt.Valid {
at := formatTimestamp(g.ImportedAt)
v.ImportedAt = &at
}
var det importer.Detail
if len(g.ImportDetail) > 0 && json.Unmarshal(g.ImportDetail, &det) == nil && det.Verdict.Outcome != "" {
ev := det.Verdict.Evidence
v.Evidence, v.Edition, v.Skipped, v.RehomedFrom = &ev, det.Edition, det.Skipped, det.RehomedFrom
if det.Files != nil {
v.Files = det.Files
}
}
return v
}
type downloadsPage struct {
Items []downloadView `json:"items"`
Limit int `json:"limit"`
Offset int `json:"offset"`
// ClientsDown are the download clients not answering now.
ClientsDown []clientDownView `json:"clients_down"`
}
type clientDownView struct {
Client string `json:"client"`
Detail string `json:"detail"`
Since string `json:"since"`
}
// handleListDownloads implements GET /api/admin/downloads?offset=: running
// downloads first, then the rest newest first.
func (h *handlers) handleListDownloads(w http.ResponseWriter, r *http.Request) {
limit, offset, err := parsePaging(r.URL.Query())
if err != nil {
writeErr(w, apierror.BadRequest("invalid_paging", "limit and offset must be numbers"))
return
}
rows, err := dbq.New(h.pool).ListGrabs(r.Context(), dbq.ListGrabsParams{MaxRows: int32(limit), Skip: int32(offset)})
if err != nil {
writeErrWithLog(w, h.logger, "admin downloads: list", apierror.Internal(err))
return
}
down, err := dbq.New(h.pool).ListClientsDown(r.Context())
if err != nil {
writeErrWithLog(w, h.logger, "admin downloads: clients down", apierror.Internal(err))
return
}
out := downloadsPage{Items: make([]downloadView, 0, len(rows)), Limit: limit, Offset: offset,
ClientsDown: make([]clientDownView, 0, len(down))}
for _, g := range rows {
out.Items = append(out.Items, downloadFrom(g))
}
for _, c := range down {
out.ClientsDown = append(out.ClientsDown, clientDownView{Client: c.Client, Detail: c.Detail,
Since: formatTimestamp(c.FirstSeen)})
}
writeJSON(w, http.StatusOK, out)
}
// handleAbandonDownload implements POST /api/admin/downloads/{id}/abandon: a
// running download is given up, its release set aside, the download removed
// and the album searched again.
func (h *handlers) handleAbandonDownload(w http.ResponseWriter, r *http.Request) {
admin, ok := requireUser(w, r)
if !ok {
return
}
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
if _, err := h.lifecycle.Abandon(r.Context(), id, admin.ID); err != nil {
switch {
case errors.Is(err, pgx.ErrNoRows), errors.Is(err, lifecycle.ErrNotAbandonable):
writeErr(w, apierror.Conflict("not_running", "that download is no longer running"))
default:
writeErrWithLog(w, h.logger, "admin downloads: abandon", apierror.Internal(err))
}
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleImportAnyway implements POST /api/admin/downloads/{id}/import: a
// download the check held for a person (or one that failed) is imported on
// the next pass whatever the check says.
func (h *handlers) handleImportAnyway(w http.ResponseWriter, r *http.Request) {
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
if _, err := dbq.New(h.pool).ForceGrabImport(r.Context(), id); err != nil {
h.writeImportErr(w, "import anyway", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleRetryImport implements POST /api/admin/downloads/{id}/retry.
func (h *handlers) handleRetryImport(w http.ResponseWriter, r *http.Request) {
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
if _, err := dbq.New(h.pool).RetryGrabImport(r.Context(), id); err != nil {
h.writeImportErr(w, "retry", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleRefuseDownload implements POST /api/admin/downloads/{id}/refuse: the
// release is blocked, the download removed and the album searched again.
func (h *handlers) handleRefuseDownload(w http.ResponseWriter, r *http.Request) {
admin, ok := requireUser(w, r)
if !ok {
return
}
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
if _, err := h.importer.Refuse(r.Context(), id, admin.ID); err != nil {
h.writeImportErr(w, "refuse", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
func (h *handlers) writeImportErr(w http.ResponseWriter, what string, err error) {
switch {
case errors.Is(err, pgx.ErrNoRows), errors.Is(err, importer.ErrNotDecidable):
writeErr(w, apierror.Conflict("not_decidable", "that download is not waiting on a decision"))
default:
writeErrWithLog(w, h.logger, "admin downloads: "+what, apierror.Internal(err))
}
}
type releaseBlockView struct {
InfoHash string `json:"info_hash"`
Name string `json:"name"`
Reason string `json:"reason"`
Detail string `json:"detail"`
BlockedAt string `json:"blocked_at"`
// ExpiresAt is when a release given up on may be taken again; null is
// for good.
ExpiresAt *string `json:"expires_at"`
WantTitle string `json:"want_title"`
WantArtist string `json:"want_artist"`
}
type releaseBlocksPage struct {
Items []releaseBlockView `json:"items"`
Limit int `json:"limit"`
Offset int `json:"offset"`
}
// handleListReleaseBlocks implements GET /api/admin/release-blocks?offset=:
// every release set aside, by a person or by a check, newest first.
func (h *handlers) handleListReleaseBlocks(w http.ResponseWriter, r *http.Request) {
limit, offset, err := parsePaging(r.URL.Query())
if err != nil {
writeErr(w, apierror.BadRequest("invalid_paging", "limit and offset must be numbers"))
return
}
rows, err := dbq.New(h.pool).ListReleaseBlocksPage(r.Context(), dbq.ListReleaseBlocksPageParams{
MaxRows: int32(limit), Skip: int32(offset)})
if err != nil {
writeErrWithLog(w, h.logger, "admin release blocks: list", apierror.Internal(err))
return
}
out := releaseBlocksPage{Items: make([]releaseBlockView, 0, len(rows)), Limit: limit, Offset: offset}
for _, b := range rows {
v := releaseBlockView{InfoHash: b.InfoHash, Name: b.Name, Reason: b.Reason, Detail: b.Detail,
BlockedAt: formatTimestamp(b.BlockedAt)}
if b.ExpiresAt.Valid {
at := formatTimestamp(b.ExpiresAt)
v.ExpiresAt = &at
}
if b.WantTitle != nil {
v.WantTitle = *b.WantTitle
}
if b.WantArtist != nil {
v.WantArtist = *b.WantArtist
}
out.Items = append(out.Items, v)
}
writeJSON(w, http.StatusOK, out)
}
+1 -127
View File
@@ -2,9 +2,6 @@ package api
import (
"net/http"
"time"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
@@ -15,7 +12,7 @@ import (
// the point: the recording is still a known thing with a title, an artist and
// a play history — only its bytes are absent. LastPlayedAt is nullable because
// plenty of missing files were never played, and that is exactly the signal an
// operator wants when deciding whether to bother re-acquiring one.
// operator wants when deciding whether a lost file is worth replacing.
type missingTrackView struct {
TrackID string `json:"track_id"`
Title string `json:"title"`
@@ -29,18 +26,6 @@ type missingTrackView struct {
LastPlayedAt *string `json:"last_played_at"`
}
// reacquisitionStateView is what the wanted list says about one album: when
// it was wanted back, and when the library
// held it again. A missing album with no want yet is still inside the grace
// window, or MusicBrainz cannot name it. Want is that want as the wanted list
// sends it, so the Missing page can show its search and download and act on
// it.
type reacquisitionStateView struct {
WantedAt *string `json:"wanted_at"`
HeldAt *string `json:"held_at"`
Want *wantView `json:"want"`
}
// missingGroupView is a directory's worth of missing tracks.
//
// Grouping is the whole ergonomic argument for this surface: a reorganised
@@ -52,10 +37,6 @@ type missingGroupView struct {
Directory string `json:"directory"`
MissingSince string `json:"missing_since"`
Tracks []missingTrackView `json:"tracks"`
// Nil when nothing has been attempted for this group's album — the
// common case for a folder that just went missing, and distinct from
// an attempts:0 record, which cannot occur.
Reacquisition *reacquisitionStateView `json:"reacquisition"`
}
// adminMissingResponse is the paged envelope. Total counts TRACKS, not
@@ -99,7 +80,6 @@ func (h *handlers) handleListMissingTracks(w http.ResponseWriter, r *http.Reques
}
groups := groupMissingByDirectory(rows)
h.attachReacquisitionState(r, q, rows, groups)
out := adminMissingResponse{
Total: total,
@@ -154,109 +134,3 @@ func groupMissingByDirectory(rows []dbq.ListMissingTracksRow) []missingGroupView
}
return groups
}
// attachReacquisitionState decorates each group with its album's want.
//
// Best-effort: this is context on a list whose primary job is showing what is
// missing, so a failure here leaves the groups bare rather than failing the
// page. One batched query for the whole page, not one per group.
//
// A group is keyed by directory while a want is keyed by album, and those
// line up in practice (an album's files live in one folder) but are not
// guaranteed to — a directory holding two albums takes the first album's
// state, which is the same album its first track belongs to.
func (h *handlers) attachReacquisitionState(
r *http.Request,
q *dbq.Queries,
rows []dbq.ListMissingTracksRow,
groups []missingGroupView,
) {
if len(groups) == 0 {
return
}
// Directory -> the album its first row belongs to, matching the order
// groupMissingByDirectory folded them in.
dirAlbum := make(map[string]pgtype.UUID, len(groups))
ids := make([]pgtype.UUID, 0, len(groups))
// The latest loss in each folder: a want held before it is the album
// as it was before its files went, not the library holding it again.
dirLost := make(map[string]time.Time, len(groups))
for _, row := range rows {
if row.MissingSince.Valid && row.MissingSince.Time.After(dirLost[row.Directory]) {
dirLost[row.Directory] = row.MissingSince.Time
}
if _, seen := dirAlbum[row.Directory]; seen {
continue
}
dirAlbum[row.Directory] = row.AlbumID
ids = append(ids, row.AlbumID)
}
wants, err := q.GetWantsForAlbums(r.Context(), ids)
if err != nil {
h.logger.Warn("admin missing: want lookup failed", "err", err)
return
}
// Newest want first, so the first seen per album is its current one.
byAlbum := make(map[string]dbq.GetWantsForAlbumsRow, len(wants))
for _, w := range wants {
if _, seen := byAlbum[uuidToString(w.AlbumID)]; !seen {
byAlbum[uuidToString(w.AlbumID)] = w
}
}
full := h.wantViewsByID(r, q, byAlbum)
for i := range groups {
albumID, ok := dirAlbum[groups[i].Directory]
if !ok {
continue
}
w, ok := byAlbum[uuidToString(albumID)]
if !ok {
continue
}
// Held before its files went (every library album is on the list,
// held): not wanted back yet, which the sweep does once
// the grace window has passed.
if w.FulfilledAt.Valid && !w.FulfilledAt.Time.After(dirLost[groups[i].Directory]) {
continue
}
view := &reacquisitionStateView{}
wantedAt := formatTimestamp(w.WantedAt)
view.WantedAt = &wantedAt
if w.FulfilledAt.Valid {
heldAt := formatTimestamp(w.FulfilledAt)
view.HeldAt = &heldAt
}
if v, ok := full[uuidToString(w.WantID)]; ok {
view.Want = &v
}
groups[i].Reacquisition = view
}
}
// wantViewsByID reads each album's current want in full, keyed by want id.
// Best-effort like the state it decorates: a failure leaves the groups with
// their dates and no want.
func (h *handlers) wantViewsByID(
r *http.Request,
q *dbq.Queries,
byAlbum map[string]dbq.GetWantsForAlbumsRow,
) map[string]wantView {
ids := make([]pgtype.UUID, 0, len(byAlbum))
for _, w := range byAlbum {
ids = append(ids, w.WantID)
}
out := make(map[string]wantView, len(ids))
if len(ids) == 0 {
return out
}
rows, err := q.ListWants(r.Context(), dbq.ListWantsParams{State: "all", WantIds: ids, MaxRows: int32(len(ids))})
if err != nil {
h.logger.Warn("admin missing: want detail lookup failed", "err", err)
return out
}
for _, row := range rows {
out[uuidToString(row.ID)] = wantViewOf(row)
}
return out
}
@@ -1,84 +0,0 @@
package api
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/jackc/pgx/v5/pgtype"
)
// seedMissingWant marks a track's file missing for two days and puts its
// album on the wanted list, held at heldDaysAgo (0 for an open want) and
// last searched with nothing found.
func seedMissingWant(t *testing.T, h *handlers, trackID, albumID pgtype.UUID, group string, heldDaysAgo int) string {
t.Helper()
ctx := context.Background()
if _, err := h.pool.Exec(ctx,
`UPDATE tracks SET missing_since = now() - interval '2 days' WHERE id = $1`, trackID); err != nil {
t.Fatalf("mark missing: %v", err)
}
if _, err := h.pool.Exec(ctx, `DELETE FROM wants WHERE release_group_mbid = $1`, group); err != nil {
t.Fatalf("clear want: %v", err)
}
var id pgtype.UUID
if err := h.pool.QueryRow(ctx,
`INSERT INTO wants (release_group_mbid, artist_mbid, artist_name, title, reason, album_id,
wanted_at, fulfilled_at, searched_at, search_outcome)
VALUES ($1, 'artist-'||$1, 'Artist', 'Album '||$1, 'library', $2,
now() - interval '30 days',
CASE WHEN $3::int > 0 THEN now() - make_interval(days => $3::int) END,
now() - interval '1 hour', 'nothing')
RETURNING id`, group, albumID, heldDaysAgo).Scan(&id); err != nil {
t.Fatalf("insert want: %v", err)
}
return uuidToString(id)
}
// A folder's album that was held before its files went is not wanted back
// yet (every library album is on the list, held); an open want comes with
// its search, so the Missing page can show it and act on it.
func TestListMissingTracks_WantState(t *testing.T) {
h, pool := testHandlers(t)
truncateLibrary(t, pool)
admin := seedUser(t, pool, "admin", "pw", true)
heldTrack, _ := seedTrackOnDisk(t, h, "missheld")
openTrack, _ := seedTrackOnDisk(t, h, "missopen")
seedMissingWant(t, h, heldTrack.ID, heldTrack.AlbumID, "rg-missing-held", 10)
openID := seedMissingWant(t, h, openTrack.ID, openTrack.AlbumID, "rg-missing-open", 0)
req := withUser(httptest.NewRequest(http.MethodGet, "/api/admin/library/missing", nil), admin)
w := httptest.NewRecorder()
h.handleListMissingTracks(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status %d: %s", w.Code, w.Body.String())
}
var resp adminMissingResponse
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v", err)
}
if len(resp.Groups) != 2 {
t.Fatalf("groups = %d, want 2", len(resp.Groups))
}
byTitle := map[string]missingGroupView{}
for _, g := range resp.Groups {
byTitle[g.Tracks[0].Title] = g
}
if r := byTitle["AdmQ Track missheld"].Reacquisition; r != nil {
t.Errorf("held before the loss: reacquisition = %+v, want none", r)
}
r := byTitle["AdmQ Track missopen"].Reacquisition
if r == nil || r.WantedAt == nil || r.HeldAt != nil {
t.Fatalf("open want: reacquisition = %+v, want wanted and not held", r)
}
if r.Want == nil || r.Want.ID != openID {
t.Fatalf("open want: want = %+v, want id %s", r.Want, openID)
}
if r.Want.Search == nil || r.Want.Search.Outcome != "nothing" {
t.Errorf("open want: search = %+v, want outcome nothing", r.Want.Search)
}
}
-78
View File
@@ -1,78 +0,0 @@
package api
import (
"encoding/json"
"errors"
"net/http"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/reacquisition"
)
// reacquisitionSettingsResp is the admin card's payload.
//
// Carries more than the stored settings: [UnnameableAlbums] is the count of
// albums with missing files that can never be wanted automatically because
// neither they nor their artist has an MBID. The card states that number
// rather than leaving an admin to wonder why some rows never get wanted —
// a feature that silently does nothing for part of its input reads as broken.
type reacquisitionSettingsResp struct {
Enabled bool `json:"enabled"`
GraceHours int32 `json:"grace_hours"`
MaxPerPass int32 `json:"max_per_pass"`
UnnameableAlbums int64 `json:"unnameable_albums"`
}
// handleGetReacquisitionSettings implements GET
// /api/admin/library/reacquisition.
func (h *handlers) handleGetReacquisitionSettings(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, h.reacquisitionPayload(r))
}
// handleUpdateReacquisitionSettings implements PUT
// /api/admin/library/reacquisition.
func (h *handlers) handleUpdateReacquisitionSettings(w http.ResponseWriter, r *http.Request) {
var req reacquisitionSettingsResp
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, apierror.BadRequest("invalid_body", "malformed JSON"))
return
}
_, err := h.reacqSettings.Set(r.Context(), reacquisition.Settings{
Enabled: req.Enabled,
GraceHours: req.GraceHours,
MaxPerPass: req.MaxPerPass,
})
if err != nil {
// The Go-side validation mirrors migration 0056's CHECKs so the
// operator gets a readable message naming the field, rather than a
// constraint-violation string leaking through as a 500.
if errors.Is(err, reacquisition.ErrOutOfRange) {
writeErr(w, apierror.BadRequest("invalid_setting", err.Error()))
return
}
writeErrWithLog(w, h.logger, "admin reacquisition: update failed", apierror.Internal(err))
return
}
// Echo the payload recomputed under the new values so the card reflects
// what it just did without a reload.
writeJSON(w, http.StatusOK, h.reacquisitionPayload(r))
}
func (h *handlers) reacquisitionPayload(r *http.Request) reacquisitionSettingsResp {
cur := h.reacqSettings.Get()
out := reacquisitionSettingsResp{
Enabled: cur.Enabled,
GraceHours: cur.GraceHours,
MaxPerPass: cur.MaxPerPass,
}
// Best-effort: the settings are the point of this endpoint, and failing
// the whole card because a count query hiccuped would be the wrong trade.
if n, err := dbq.New(h.pool).CountAlbumsMissingWithoutMbid(r.Context()); err == nil {
out.UnnameableAlbums = n
} else {
h.logger.Warn("admin reacquisition: unnameable count failed", "err", err)
}
return out
}
@@ -67,30 +67,15 @@ func tasteRespFrom(t recsettings.TasteTuning) tasteTuningResp {
}
}
// discoverTuningResp is the Discover scope on the wire.
type discoverTuningResp struct {
TagOverlapWeight float64 `json:"tag_overlap_weight"`
SnoozeDays float64 `json:"snooze_days"`
}
func discoverRespFrom(d recsettings.DiscoverTuning) discoverTuningResp {
return discoverTuningResp{
TagOverlapWeight: d.TagOverlapWeight,
SnoozeDays: d.SnoozeDays,
}
}
// tuningSnapshot is both the GET response and the post-mutation echo:
// current values alongside shipped defaults so the card can mark
// which knobs deviate.
type tuningSnapshot struct {
Profiles map[string]weightsResp `json:"profiles"`
Taste tasteTuningResp `json:"taste"`
Discover discoverTuningResp `json:"discover"`
Shipped struct {
Profiles map[string]weightsResp `json:"profiles"`
Taste tasteTuningResp `json:"taste"`
Discover discoverTuningResp `json:"discover"`
} `json:"shipped"`
}
@@ -102,14 +87,12 @@ func (h *handlers) tuningSnapshot() tuningSnapshot {
recsettings.ScopeSongsLike: weightsRespFrom(h.recSettings.Weights(recsettings.ScopeSongsLike)),
}
out.Taste = tasteRespFrom(h.recSettings.Taste())
out.Discover = discoverRespFrom(h.recSettings.Discover())
out.Shipped.Profiles = map[string]weightsResp{
recsettings.ScopeRadio: weightsRespFrom(recsettings.ShippedRadioWeights()),
recsettings.ScopeDailyMix: weightsRespFrom(recsettings.ShippedDailyMixWeights()),
recsettings.ScopeSongsLike: weightsRespFrom(recsettings.ShippedSongsLikeWeights()),
}
out.Shipped.Taste = tasteRespFrom(recsettings.ShippedTasteTuning())
out.Shipped.Discover = discoverRespFrom(recsettings.ShippedDiscoverTuning())
return out
}
@@ -143,8 +126,6 @@ func (h *handlers) handlePatchRecommendationTuning(w http.ResponseWriter, r *htt
switch scope {
case recsettings.ScopeTaste:
err = h.recSettings.UpdateTaste(r.Context(), body.Values)
case recsettings.ScopeDiscover:
err = h.recSettings.UpdateDiscover(r.Context(), body.Values)
default:
err = h.recSettings.UpdateProfile(r.Context(), scope, body.Values)
}
-190
View File
@@ -1,190 +0,0 @@
package api
import (
"context"
"encoding/json"
"errors"
"net/http"
"strconv"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/notifications"
"git.fabledsword.com/bvandeusen/minstrel/internal/requests"
"git.fabledsword.com/bvandeusen/minstrel/internal/wanted"
)
// validRequestStatuses is the set of allowed values for the ?status= param.
var validRequestStatuses = map[string]bool{
"pending": true,
"approved": true,
"rejected": true,
"completed": true,
"failed": true,
}
// handleListAdminRequests implements GET /api/admin/requests.
// ?status= defaults to "pending"; ?limit= defaults to 50, max 200.
func (h *handlers) handleListAdminRequests(w http.ResponseWriter, r *http.Request) {
status := r.URL.Query().Get("status")
if status == "" {
status = "pending"
}
if !validRequestStatuses[status] {
writeAdminJSONErr(w, http.StatusBadRequest, "invalid_status")
return
}
limitStr := r.URL.Query().Get("limit")
limit := 50
if limitStr != "" {
v, err := strconv.Atoi(limitStr)
if err != nil || v <= 0 {
writeAdminJSONErr(w, http.StatusBadRequest, "invalid_limit")
return
}
if v > 200 {
v = 200
}
limit = v
}
rows, err := h.requests.ListByStatus(r.Context(), status, int32(limit))
if err != nil {
h.logger.Error("admin: list requests", "err", err)
writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
return
}
q := dbq.New(h.pool)
names := requesterNames(r.Context(), q, rows)
out := make([]requestView, 0, len(rows))
for _, row := range rows {
v := requestViewFrom(row)
v.Username = names[row.UserID]
fillProgress(r.Context(), q, row, &v)
out = append(out, v)
}
writeJSON(w, http.StatusOK, out)
}
// requesterNames maps each request's user id to that user's username, in one
// query for the whole page. Like fillProgress, a failed lookup is swallowed:
// the names are a reporting nicety, and the page falls back to the id.
func requesterNames(ctx context.Context, q *dbq.Queries, rows []dbq.Request) map[pgtype.UUID]string {
ids := make([]pgtype.UUID, 0, len(rows))
for _, row := range rows {
ids = append(ids, row.UserID)
}
names := make(map[pgtype.UUID]string, len(rows))
users, err := q.ListUsernamesByIDs(ctx, ids)
if err != nil {
return names
}
for _, u := range users {
names[u.ID] = u.Username
}
return names
}
// approveRequestBody is the optional JSON body for POST
// /api/admin/requests/:id/approve. profile_id picks the release-type profile an
// artist request is followed on; omitted, the default profile.
type approveRequestBody struct {
ProfileID string `json:"profile_id"`
}
// handleApproveRequest implements POST /api/admin/requests/:id/approve. The
// request goes on the wanted list: an album or track request
// wants its album, an artist request follows the artist.
func (h *handlers) handleApproveRequest(w http.ResponseWriter, r *http.Request) {
admin, ok := auth.UserFromContext(r.Context())
if !ok {
writeAdminJSONErr(w, http.StatusUnauthorized, "unauthorized")
return
}
id, ok := parseUUID(chi.URLParam(r, "id"))
if !ok {
writeAdminJSONErr(w, http.StatusBadRequest, "invalid_id")
return
}
var body approveRequestBody
// Body is entirely optional; ignore decode errors.
_ = json.NewDecoder(r.Body).Decode(&body)
var profile pgtype.UUID
if body.ProfileID != "" {
if profile, ok = parseUUID(body.ProfileID); !ok {
writeAdminJSONErr(w, http.StatusBadRequest, "invalid_profile")
return
}
}
row, err := h.requests.Approve(r.Context(), id, admin.ID, profile)
if err != nil {
switch {
case errors.Is(err, requests.ErrNotFound):
writeAdminJSONErr(w, http.StatusNotFound, "request_not_found")
case errors.Is(err, requests.ErrNotPending):
writeAdminJSONErr(w, http.StatusConflict, "request_not_pending")
case errors.Is(err, wanted.ErrProfileNotFound):
writeAdminJSONErr(w, http.StatusBadRequest, "profile_not_found")
case errors.Is(err, wanted.ErrNotFollowable):
writeAdminJSONErr(w, http.StatusBadRequest, "not_followable")
default:
h.logger.Error("admin: approve request", "err", err)
writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
}
return
}
h.notifyRequestDecided(r.Context(), notifications.KindRequestApproved, admin, row, "")
h.publishRequestStatusChanged(row)
writeJSON(w, http.StatusOK, requestViewFrom(row))
}
// rejectRequestBody is the optional JSON body for POST
// /api/admin/requests/:id/reject.
type rejectRequestBody struct {
Notes string `json:"notes"`
}
// handleRejectRequest implements POST /api/admin/requests/:id/reject.
func (h *handlers) handleRejectRequest(w http.ResponseWriter, r *http.Request) {
admin, ok := auth.UserFromContext(r.Context())
if !ok {
writeAdminJSONErr(w, http.StatusUnauthorized, "unauthorized")
return
}
id, ok := parseUUID(chi.URLParam(r, "id"))
if !ok {
writeAdminJSONErr(w, http.StatusBadRequest, "invalid_id")
return
}
var body rejectRequestBody
_ = json.NewDecoder(r.Body).Decode(&body)
row, err := h.requests.Reject(r.Context(), id, admin.ID, body.Notes)
if err != nil {
switch {
case errors.Is(err, requests.ErrNotFound):
writeAdminJSONErr(w, http.StatusNotFound, "request_not_found")
case errors.Is(err, requests.ErrNotPending):
writeAdminJSONErr(w, http.StatusConflict, "request_not_pending")
default:
h.logger.Error("admin: reject request", "err", err)
writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
}
return
}
h.notifyRequestDecided(r.Context(), notifications.KindRequestRejected, admin, row, body.Notes)
h.publishRequestStatusChanged(row)
writeJSON(w, http.StatusOK, requestViewFrom(row))
}
-335
View File
@@ -1,335 +0,0 @@
package api
import (
"bytes"
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
// newAdminRequestsRouter builds a test chi router with the three admin/requests
// endpoints. RequireAdmin middleware is applied. Tests inject a user into
// context directly (skipping RequireUser).
func newAdminRequestsRouter(h *handlers) chi.Router {
r := chi.NewRouter()
r.Route("/api/admin", func(admin chi.Router) {
admin.Use(auth.RequireAdmin())
admin.Get("/requests", h.handleListAdminRequests)
admin.Post("/requests/{id}/approve", h.handleApproveRequest)
admin.Post("/requests/{id}/reject", h.handleRejectRequest)
})
return r
}
// doAdminRequestReq fires an HTTP request at the admin requests router with
// the given user injected into context (bypassing RequireUser).
func doAdminRequestReq(t *testing.T, h *handlers, method, path string, body []byte, user dbq.User) *httptest.ResponseRecorder {
t.Helper()
var reqBody *bytes.Buffer
if body != nil {
reqBody = bytes.NewBuffer(body)
} else {
reqBody = bytes.NewBuffer(nil)
}
req := httptest.NewRequest(method, path, reqBody)
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
req = withUser(req, user)
w := httptest.NewRecorder()
newAdminRequestsRouter(h).ServeHTTP(w, req)
return w
}
// seedPendingArtistRequest creates a pending artist-kind request row
// for the given user and returns its parsed requestView.
func seedPendingArtistRequest(t *testing.T, h *handlers, user dbq.User, artistMBID, artistName string) requestView {
t.Helper()
return createArtistRequest(t, h, user, artistMBID, artistName)
}
// TestHandleListAdminRequests_DefaultStatus_Pending seeds 2 pending + 1
// rejected and verifies that GET without ?status returns only the 2 pending
// rows.
func TestHandleListAdminRequests_DefaultStatus_Pending(t *testing.T) {
h, _ := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, h.pool, "alice", "pw", false)
admin := seedUser(t, h.pool, "admin", "pw", true)
createArtistRequest(t, h, alice, "mbid-pending-1", "Pending Artist 1")
createArtistRequest(t, h, alice, "mbid-pending-2", "Pending Artist 2")
// Reject one request directly via service.
rv3 := createArtistRequest(t, h, alice, "mbid-reject-1", "Reject Me")
_, err := h.requests.Reject(context.Background(), rv3.ID, admin.ID, "test")
if err != nil {
t.Fatalf("reject seed: %v", err)
}
w := doAdminRequestReq(t, h, http.MethodGet, "/api/admin/requests", nil, admin)
if w.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
}
var results []requestView
if err := json.Unmarshal(w.Body.Bytes(), &results); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if len(results) != 2 {
t.Fatalf("len(results) = %d, want 2 (pending only)", len(results))
}
for _, rv := range results {
if rv.Status != "pending" {
t.Errorf("result status = %q, want pending", rv.Status)
}
if rv.Username != alice.Username {
t.Errorf("result username = %q, want %q (the requester)", rv.Username, alice.Username)
}
}
}
// TestHandleListAdminRequests_StatusFilter verifies GET ?status=rejected
// returns only rejected rows.
func TestHandleListAdminRequests_StatusFilter(t *testing.T) {
h, _ := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, h.pool, "alice", "pw", false)
admin := seedUser(t, h.pool, "admin", "pw", true)
createArtistRequest(t, h, alice, "mbid-pending-f1", "Pending Filter 1")
rv2 := createArtistRequest(t, h, alice, "mbid-rejected-f1", "Rejected Filter 1")
_, err := h.requests.Reject(context.Background(), rv2.ID, admin.ID, "")
if err != nil {
t.Fatalf("reject seed: %v", err)
}
w := doAdminRequestReq(t, h, http.MethodGet, "/api/admin/requests?status=rejected", nil, admin)
if w.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
}
var results []requestView
if err := json.Unmarshal(w.Body.Bytes(), &results); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if len(results) != 1 {
t.Fatalf("len(results) = %d, want 1 rejected row", len(results))
}
if results[0].Status != "rejected" {
t.Errorf("status = %q, want rejected", results[0].Status)
}
}
// TestHandleListAdminRequests_BadStatus_400 verifies GET with an invalid
// ?status returns 400.
func TestHandleListAdminRequests_BadStatus_400(t *testing.T) {
h, _ := testHandlers(t)
resetRequests(t, h)
admin := seedUser(t, h.pool, "admin", "pw", true)
w := doAdminRequestReq(t, h, http.MethodGet, "/api/admin/requests?status=garbage", nil, admin)
if w.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400; body = %s", w.Code, w.Body.String())
}
var resp map[string]string
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if resp["error"] != "invalid_status" {
t.Errorf("error = %q, want invalid_status", resp["error"])
}
}
// TestHandleApproveRequest_NotPending_409 pre-rejects a row and verifies
// POST /approve → 409 request_not_pending.
func TestHandleApproveRequest_NotPending_409(t *testing.T) {
h, _ := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, h.pool, "alice", "pw", false)
admin := seedUser(t, h.pool, "admin", "pw", true)
rv := seedPendingArtistRequest(t, h, alice, "artist-mbid-notpending", "Not Pending")
// Pre-reject the row.
_, err := h.requests.Reject(context.Background(), rv.ID, admin.ID, "pre-rejected")
if err != nil {
t.Fatalf("pre-reject: %v", err)
}
w := doAdminRequestReq(t, h, http.MethodPost, "/api/admin/requests/"+uuidToString(rv.ID)+"/approve", nil, admin)
if w.Code != http.StatusConflict {
t.Fatalf("status = %d, want 409; body = %s", w.Code, w.Body.String())
}
var resp map[string]string
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if resp["error"] != "request_not_pending" {
t.Errorf("error = %q, want request_not_pending", resp["error"])
}
}
// TestHandleRejectRequest_HappyPath verifies POST /reject with notes returns
// 200 and the row transitions to rejected with notes saved.
func TestHandleRejectRequest_HappyPath(t *testing.T) {
h, _ := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, h.pool, "alice", "pw", false)
admin := seedUser(t, h.pool, "admin", "pw", true)
rv := seedPendingArtistRequest(t, h, alice, "artist-mbid-reject", "Reject Me")
body := []byte(`{"notes":"low-quality release; will look for the remaster"}`)
w := doAdminRequestReq(t, h, http.MethodPost, "/api/admin/requests/"+uuidToString(rv.ID)+"/reject", body, admin)
if w.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
}
var got requestView
if err := json.Unmarshal(w.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if got.Status != "rejected" {
t.Errorf("status = %q, want rejected", got.Status)
}
if got.Notes == nil || *got.Notes != "low-quality release; will look for the remaster" {
t.Errorf("notes = %v, want expected string", got.Notes)
}
}
// TestHandleRejectRequest_NoNotes_OK verifies POST /reject with empty body
// returns 200 (notes are optional).
func TestHandleRejectRequest_NoNotes_OK(t *testing.T) {
h, _ := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, h.pool, "alice", "pw", false)
admin := seedUser(t, h.pool, "admin", "pw", true)
rv := seedPendingArtistRequest(t, h, alice, "artist-mbid-reject-nonotes", "Reject No Notes")
w := doAdminRequestReq(t, h, http.MethodPost, "/api/admin/requests/"+uuidToString(rv.ID)+"/reject", nil, admin)
if w.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
}
var got requestView
if err := json.Unmarshal(w.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if got.Status != "rejected" {
t.Errorf("status = %q, want rejected", got.Status)
}
}
// TestAdminRequestsEndpoints_NonAdmin403 is a table-driven test verifying that
// all 3 admin request endpoints reject non-admin users with 403.
func TestAdminRequestsEndpoints_NonAdmin403(t *testing.T) {
h, _ := testHandlers(t)
resetRequests(t, h)
nonAdmin := seedUser(t, h.pool, "regular", "pw", false)
// Use a placeholder UUID for the ID endpoints.
fakeID := "00000000-0000-0000-0000-000000000001"
cases := []struct {
method string
path string
body []byte
}{
{http.MethodGet, "/api/admin/requests", nil},
{http.MethodPost, "/api/admin/requests/" + fakeID + "/approve", nil},
{http.MethodPost, "/api/admin/requests/" + fakeID + "/reject", nil},
}
for _, tc := range cases {
tc := tc
t.Run(tc.method+" "+tc.path, func(t *testing.T) {
w := doAdminRequestReq(t, h, tc.method, tc.path, tc.body, nonAdmin)
if w.Code != http.StatusForbidden {
t.Fatalf("status = %d, want 403; body = %s", w.Code, w.Body.String())
}
var resp map[string]string
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if resp["error"] != "not_authorized" {
t.Errorf("error = %q, want not_authorized", resp["error"])
}
})
}
}
// TestHandleApproveRequest_AlbumBecomesAWant: approving an album request puts
// its release group on the wanted list, as the requester's.
func TestHandleApproveRequest_AlbumBecomesAWant(t *testing.T) {
h, _ := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, h.pool, "alice", "pw", false)
admin := seedUser(t, h.pool, "admin", "pw", true)
w := doCreateRequest(h, alice, `{"kind":"album","artist_mbid":"artist-x","artist_name":"Gorillaz","album_mbid":"group-x","album_title":"Demon Days"}`)
if w.Code != http.StatusCreated {
t.Fatalf("create: %d %s", w.Code, w.Body.String())
}
var rv requestView
_ = json.Unmarshal(w.Body.Bytes(), &rv)
resp := doAdminRequestReq(t, h, http.MethodPost, "/api/admin/requests/"+uuidToString(rv.ID)+"/approve", nil, admin)
if resp.Code != http.StatusOK {
t.Fatalf("approve: %d %s", resp.Code, resp.Body.String())
}
want, err := dbq.New(h.pool).GetWantByReleaseGroup(context.Background(), "group-x")
if err != nil {
t.Fatalf("no want recorded: %v", err)
}
if want.Title != "Demon Days" || want.CreatedBy != alice.ID || want.Reason != "request" {
t.Errorf("want = %+v", want)
}
}
// TestHandleApproveRequest_ArtistOnTheChosenProfile: an artist request is
// followed on the profile the admin picked; an unknown profile is a 400.
func TestHandleApproveRequest_ArtistOnTheChosenProfile(t *testing.T) {
h, _ := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, h.pool, "alice", "pw", false)
admin := seedUser(t, h.pool, "admin", "pw", true)
ctx := context.Background()
profile, err := dbq.New(h.pool).CreateReleaseTypeProfile(ctx, dbq.CreateReleaseTypeProfileParams{
Name: "Albums only", PrimaryTypes: []string{"Album"}, SecondaryTypes: []string{},
})
if err != nil {
t.Fatal(err)
}
rv := seedPendingArtistRequest(t, h, alice, "artist-profiled", "Profiled")
path := "/api/admin/requests/" + uuidToString(rv.ID) + "/approve"
if resp := doAdminRequestReq(t, h, http.MethodPost, path, []byte(`{"profile_id":"00000000-0000-4000-8000-000000000000"}`), admin); resp.Code != http.StatusBadRequest {
t.Fatalf("unknown profile: %d %s", resp.Code, resp.Body.String())
}
body := []byte(`{"profile_id":"` + uuidToString(profile.ID) + `"}`)
if resp := doAdminRequestReq(t, h, http.MethodPost, path, body, admin); resp.Code != http.StatusOK {
t.Fatalf("approve: %d %s", resp.Code, resp.Body.String())
}
f, err := dbq.New(h.pool).GetFollow(ctx, "artist-profiled")
if err != nil || f.ProfileID != profile.ID || !f.WantExisting {
t.Errorf("follow = %+v, %v: on Albums only, wanting what exists", f, err)
}
}
-340
View File
@@ -1,340 +0,0 @@
package api
import (
"context"
"encoding/json"
"errors"
"net/http"
"slices"
"time"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5"
"git.fabledsword.com/bvandeusen/minstrel/internal/acquire"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/release"
"git.fabledsword.com/bvandeusen/minstrel/internal/wanted"
)
// Searching for wants and grabbing releases: the schedule's
// settings, each want's candidates, and a person's Search, Grab and "never
// this one".
// searchTimeout bounds a search a person waits on: a Prowlarr fanning out to
// many trackers can take most of a minute.
const searchTimeout = 2 * time.Minute
type searchSettingsView struct {
AutoGrab bool `json:"auto_grab"`
SearchIntervalHours int32 `json:"search_interval_hours"`
AllowedQualities []string `json:"allowed_qualities"`
Tiers []release.TierInfo `json:"tiers"`
LastPassAt *string `json:"last_pass_at"`
LastPass json.RawMessage `json:"last_pass"`
// ImportRoot is the library folder new music is filed into; ""
// is the first of ScanPaths, the library's folders.
ImportRoot string `json:"import_root"`
ScanPaths []string `json:"scan_paths"`
// The download lifecycle: seed an imported download to this
// ratio or for this many days; give up on one nobody shares after this
// many hours. 0 is: remove once imported, no time limit, never.
SeedRatio float32 `json:"seed_ratio"`
SeedDays int32 `json:"seed_days"`
StallHours int32 `json:"stall_hours"`
// UpgradeUntil is the tier a held album is upgraded to; "" is
// no quality upgrades.
UpgradeUntil string `json:"upgrade_until"`
}
func searchSettings(s dbq.AcquisitionSetting, roots []string) searchSettingsView {
v := searchSettingsView{
AutoGrab: s.AutoGrab, SearchIntervalHours: s.SearchIntervalHours,
AllowedQualities: s.AllowedQualities, Tiers: release.Tiers, LastPass: s.LastPass,
ImportRoot: s.ImportRoot, ScanPaths: roots,
SeedRatio: s.SeedRatio, SeedDays: s.SeedDays, StallHours: s.StallHours,
UpgradeUntil: s.UpgradeUntil,
}
if v.ScanPaths == nil {
v.ScanPaths = []string{}
}
if v.AllowedQualities == nil {
v.AllowedQualities = []string{}
}
if len(v.LastPass) == 0 {
v.LastPass = json.RawMessage("{}")
}
if s.LastPassAt.Valid {
at := formatTimestamp(s.LastPassAt)
v.LastPassAt = &at
}
return v
}
// handleGetSearchSettings implements GET /api/admin/acquisition/settings.
func (h *handlers) handleGetSearchSettings(w http.ResponseWriter, r *http.Request) {
s, err := dbq.New(h.pool).GetAcquisitionSettings(r.Context())
if err != nil {
writeErrWithLog(w, h.logger, "admin search settings: get", apierror.Internal(err))
return
}
writeJSON(w, http.StatusOK, searchSettings(s, h.importer.Roots()))
}
type putSearchSettingsBody struct {
AutoGrab bool `json:"auto_grab"`
SearchIntervalHours int32 `json:"search_interval_hours"`
AllowedQualities []string `json:"allowed_qualities"`
ImportRoot string `json:"import_root"`
SeedRatio float32 `json:"seed_ratio"`
SeedDays int32 `json:"seed_days"`
StallHours int32 `json:"stall_hours"`
UpgradeUntil string `json:"upgrade_until"`
}
// handlePutSearchSettings implements PUT /api/admin/acquisition/settings.
func (h *handlers) handlePutSearchSettings(w http.ResponseWriter, r *http.Request) {
var body putSearchSettingsBody
if !decodeBody(w, r, &body) {
return
}
if body.SearchIntervalHours < 0 || body.SearchIntervalHours > 720 {
writeErr(w, apierror.BadRequest("bad_interval", "search_interval_hours must be between 0 and 720"))
return
}
if len(body.AllowedQualities) == 0 {
writeErr(w, apierror.BadRequest("bad_quality", "allow at least one quality"))
return
}
for _, t := range body.AllowedQualities {
if !release.KnownTier(t) {
writeErr(w, apierror.BadRequest("bad_quality", "unknown quality "+t))
return
}
}
if body.UpgradeUntil != "" && !release.KnownTier(body.UpgradeUntil) {
writeErr(w, apierror.BadRequest("bad_upgrade_until", "unknown quality "+body.UpgradeUntil))
return
}
if body.ImportRoot != "" && !slices.Contains(h.importer.Roots(), body.ImportRoot) {
writeErr(w, apierror.BadRequest("bad_import_root", "file new music into one of the library's folders"))
return
}
if body.SeedRatio < 0 || body.SeedRatio > 100 || body.SeedDays < 0 || body.SeedDays > 365 ||
body.StallHours < 0 || body.StallHours > 720 {
writeErr(w, apierror.BadRequest("bad_lifecycle",
"seed ratio must be 0 to 100, seed days 0 to 365 and stall hours 0 to 720"))
return
}
s, err := dbq.New(h.pool).UpdateAcquisitionSettings(r.Context(), dbq.UpdateAcquisitionSettingsParams{
AutoGrab: body.AutoGrab, SearchIntervalHours: body.SearchIntervalHours, AllowedQualities: body.AllowedQualities,
ImportRoot: body.ImportRoot, SeedRatio: body.SeedRatio, SeedDays: body.SeedDays, StallHours: body.StallHours,
UpgradeUntil: body.UpgradeUntil,
})
if err != nil {
writeErrWithLog(w, h.logger, "admin search settings: put", apierror.Internal(err))
return
}
writeJSON(w, http.StatusOK, searchSettings(s, h.importer.Roots()))
}
// candidateView is one release a want's last search found.
type candidateView struct {
Key string `json:"key"`
Name string `json:"name"`
InfoHash string `json:"info_hash"`
Indexer string `json:"indexer"`
SizeBytes int64 `json:"size_bytes"`
Seeders int32 `json:"seeders"`
FilesCount int32 `json:"files_count"`
Format string `json:"format"`
Quality string `json:"quality"`
Year int32 `json:"year"`
Verdict string `json:"verdict"`
Reason string `json:"reason"`
Reasons json.RawMessage `json:"reasons"`
FoldedFrom []string `json:"folded_from"`
SearchedAt string `json:"searched_at"`
Downloading bool `json:"downloading"`
}
// handleListCandidates implements GET /api/admin/wanted/{id}/candidates.
func (h *handlers) handleListCandidates(w http.ResponseWriter, r *http.Request) {
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
q := dbq.New(h.pool)
rows, err := q.ListWantCandidates(r.Context(), id)
if err != nil {
writeErrWithLog(w, h.logger, "admin wanted: candidates", apierror.Internal(err))
return
}
live, err := q.GetLiveGrab(r.Context(), id)
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
writeErrWithLog(w, h.logger, "admin wanted: live grab", apierror.Internal(err))
return
}
out := make([]candidateView, 0, len(rows))
for _, c := range rows {
v := candidateView{
Key: c.Key, Name: c.Name, InfoHash: c.InfoHash, Indexer: c.Indexer, SizeBytes: c.SizeBytes,
Seeders: c.Seeders, FilesCount: c.FilesCount, Format: c.Format, Quality: c.Quality, Year: c.Year,
Verdict: c.Verdict, Reason: c.Reason, Reasons: c.Reasons, FoldedFrom: c.FoldedFrom,
SearchedAt: formatTimestamp(c.SearchedAt),
Downloading: live.ID.Valid && live.Key == c.Key,
}
if v.FoldedFrom == nil {
v.FoldedFrom = []string{}
}
if len(v.Reasons) == 0 {
v.Reasons = json.RawMessage("[]")
}
out = append(out, v)
}
writeJSON(w, http.StatusOK, map[string]any{"items": out})
}
// handleSearchWant implements POST /api/admin/wanted/{id}/search: search
// now, as the schedule would, taking the best release when auto-grab is on.
func (h *handlers) handleSearchWant(w http.ResponseWriter, r *http.Request) {
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
ctx, cancel := context.WithTimeout(r.Context(), searchTimeout)
defer cancel()
res, err := h.searcher.SearchNow(ctx, id)
if h.writeSearchErr(w, err) {
return
}
writeJSON(w, http.StatusOK, res)
}
type grabBody struct {
Key string `json:"key"`
Force bool `json:"force"`
}
// grabView is what came of a person's grab.
type grabView struct {
Added bool `json:"added"`
Refused bool `json:"refused"`
Gone bool `json:"gone"`
Unconfigured bool `json:"unconfigured"`
Missing string `json:"missing,omitempty"`
Kept bool `json:"kept"`
Replaced bool `json:"replaced"`
Verdict string `json:"verdict,omitempty"`
Reason string `json:"reason,omitempty"`
Error string `json:"error,omitempty"`
}
// handleGrabCandidate implements POST /api/admin/wanted/{id}/grab: take the
// release named by key. The search runs again first, so the indexer's link
// never leaves the server; force takes what the verdict would not.
func (h *handlers) handleGrabCandidate(w http.ResponseWriter, r *http.Request) {
admin, ok := requireUser(w, r)
if !ok {
return
}
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
var body grabBody
if !decodeBody(w, r, &body) {
return
}
ctx, cancel := context.WithTimeout(r.Context(), searchTimeout)
defer cancel()
took, err := h.searcher.Take(ctx, id, body.Key, body.Force, admin.ID)
if h.writeSearchErr(w, err) {
return
}
v := grabView{
Added: took.Added, Refused: took.Refused, Gone: took.Gone, Unconfigured: took.Unconfigured,
Missing: took.Missing, Error: took.Error,
}
if took.Chosen != nil {
v.Verdict, v.Reason = string(took.Chosen.Verdict), took.Chosen.Reason
}
if took.Contest != nil {
v.Kept, v.Replaced = took.Contest.Kept, took.Contest.Replaced
if took.Contest.Kept {
v.Reason = took.Contest.Reason
}
}
writeJSON(w, http.StatusOK, v)
}
type blockBody struct {
Key string `json:"key"`
}
// handleBlockCandidate implements POST /api/admin/wanted/{id}/block: never
// take this release again, for any want.
func (h *handlers) handleBlockCandidate(w http.ResponseWriter, r *http.Request) {
admin, ok := requireUser(w, r)
if !ok {
return
}
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
var body blockBody
if !decodeBody(w, r, &body) {
return
}
c, err := dbq.New(h.pool).GetWantCandidate(r.Context(), dbq.GetWantCandidateParams{WantID: id, Key: body.Key})
if errors.Is(err, pgx.ErrNoRows) {
writeErr(w, apierror.NotFound("candidate"))
return
}
if err != nil {
writeErrWithLog(w, h.logger, "admin wanted: block", apierror.Internal(err))
return
}
if c.InfoHash == "" {
writeErr(w, apierror.BadRequest("no_infohash", "this release has no infohash to set aside by"))
return
}
if err := acquire.NewMemory(h.pool).Block(r.Context(), dbq.BlockReleaseParams{
InfoHash: c.InfoHash, Name: c.Name, Reason: acquire.BlockOperator, WantID: id, BlockedBy: admin.ID,
}); err != nil {
writeErrWithLog(w, h.logger, "admin wanted: block", apierror.Internal(err))
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleUnblockRelease implements DELETE /api/admin/release-blocks/{hash}.
func (h *handlers) handleUnblockRelease(w http.ResponseWriter, r *http.Request) {
removed, err := acquire.NewMemory(h.pool).Unblock(r.Context(), chi.URLParam(r, "hash"))
if err != nil {
writeErrWithLog(w, h.logger, "admin wanted: unblock", apierror.Internal(err))
return
}
if !removed {
writeErr(w, apierror.NotFound("block"))
return
}
w.WriteHeader(http.StatusNoContent)
}
// writeSearchErr answers a Searcher error, and says whether there was one.
func (h *handlers) writeSearchErr(w http.ResponseWriter, err error) bool {
switch {
case err == nil:
return false
case errors.Is(err, pgx.ErrNoRows):
writeErr(w, apierror.NotFound("want"))
case errors.Is(err, wanted.ErrWantHeld):
writeErr(w, apierror.Conflict("want_held", "the library already holds this album, and it isn't wanted for a better copy"))
default:
writeErrWithLog(w, h.logger, "admin wanted: search", apierror.Internal(err))
}
return true
}
+3 -58
View File
@@ -11,7 +11,7 @@ import (
)
// The admin Overview's status sections: after the verdict, who is
// listening, what is downloading, what the library holds and how much room
// listening, what the library holds and how much room
// it has, and how the server itself is doing. One read for the page, polled
// while it is open.
@@ -36,17 +36,6 @@ type listeningStatusView struct {
LastPlayAt *string `json:"last_play_at"`
}
type downloadsStatusView struct {
Running int64 `json:"running"`
NeedsYou int64 `json:"needs_you"`
NeedsYouSince *string `json:"needs_you_since"`
ImportedDay int64 `json:"imported_day"`
GivenUpDay int64 `json:"given_up_day"`
// Live is the downloads not yet ended, the ones needing a person first.
Live []downloadView `json:"live"`
ClientsDown []clientDownView `json:"clients_down"`
}
// diskView is the filesystem one library folder, or the data folder, sits
// on.
type diskView struct {
@@ -91,15 +80,10 @@ type serverStatusView struct {
type adminStatusView struct {
Listening listeningStatusView `json:"listening"`
Downloads downloadsStatusView `json:"downloads"`
Library libraryStatusView `json:"library"`
Server serverStatusView `json:"server"`
}
// overviewLiveDownloads is how many running downloads the Overview lists;
// Activity has the rest.
const overviewLiveDownloads = 5
// overviewErrors is how many recent errors the Overview lists.
const overviewErrors = 10
@@ -136,12 +120,6 @@ func (h *handlers) handleGetAdminStatus(w http.ResponseWriter, r *http.Request)
out.Listening.LastPlayAt = &at
}
if err := h.downloadsStatus(r, q, &out.Downloads); err != nil {
h.logger.Error("admin status: downloads", "err", err)
writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
return
}
lib, err := q.LibraryStatus(ctx)
if err != nil {
h.logger.Error("admin status: library", "err", err)
@@ -150,8 +128,8 @@ func (h *handlers) handleGetAdminStatus(w http.ResponseWriter, r *http.Request)
}
out.Library = libraryStatusView{Tracks: lib.Tracks, Albums: lib.Albums, Artists: lib.Artists,
Bytes: lib.Bytes, Disks: []diskView{}}
if h.importer != nil {
for _, root := range h.importer.Roots() {
if h.scanner != nil {
for _, root := range h.scanner.Roots() {
out.Library.Disks = append(out.Library.Disks, diskOf(root))
}
}
@@ -170,39 +148,6 @@ func (h *handlers) handleGetAdminStatus(w http.ResponseWriter, r *http.Request)
writeJSON(w, http.StatusOK, out)
}
func (h *handlers) downloadsStatus(r *http.Request, q *dbq.Queries, out *downloadsStatusView) error {
st, err := q.DownloadsStatus(r.Context())
if err != nil {
return err
}
*out = downloadsStatusView{Running: st.Running, NeedsYou: st.NeedsYou,
ImportedDay: st.ImportedDay, GivenUpDay: st.GivenUpDay,
Live: []downloadView{}, ClientsDown: []clientDownView{}}
if st.NeedsYou > 0 && st.NeedsYouSince.Valid {
since := formatTimestamp(st.NeedsYouSince)
out.NeedsYouSince = &since
}
// ListGrabs puts the ones needing a person first, then the running ones.
rows, err := q.ListGrabs(r.Context(), dbq.ListGrabsParams{MaxRows: overviewLiveDownloads})
if err != nil {
return err
}
for _, g := range rows {
if !g.EndedAt.Valid {
out.Live = append(out.Live, downloadFrom(g))
}
}
down, err := q.ListClientsDown(r.Context())
if err != nil {
return err
}
for _, c := range down {
out.ClientsDown = append(out.ClientsDown, clientDownView{Client: c.Client, Detail: c.Detail,
Since: formatTimestamp(c.FirstSeen)})
}
return nil
}
// diskOf reads the size and free space of the filesystem path is on.
func diskOf(path string) diskView {
var st syscall.Statfs_t
+1 -1
View File
@@ -76,7 +76,7 @@ func TestAdminStatus_ListeningAndServer(t *testing.T) {
if got.Library.Tracks != 2 || got.Library.Albums != 2 || got.Library.Bytes != 10 {
t.Errorf("library = %+v, want 2 tracks, 2 albums, 10 bytes", got.Library)
}
if got.Downloads.Live == nil || got.Downloads.ClientsDown == nil || got.Library.Disks == nil {
if got.Library.Disks == nil {
t.Error("lists must be [], never null")
}
-2
View File
@@ -19,7 +19,6 @@ type adminQueueView struct {
// every queue an admin acts on, in one response, so each admin page pays one
// cheap read for its rail.
type adminSummaryView struct {
Requests adminQueueView `json:"requests"`
Reports adminQueueView `json:"reports"`
Duplicates adminQueueView `json:"duplicates"`
Suspect adminQueueView `json:"suspect"`
@@ -37,7 +36,6 @@ func queueView(count int64, since pgtype.Timestamptz) adminQueueView {
func adminSummaryFrom(row dbq.AdminSummaryRow) adminSummaryView {
return adminSummaryView{
Requests: queueView(row.Requests, row.RequestsSince),
Reports: queueView(row.Reports, row.ReportsSince),
Duplicates: queueView(row.Duplicates, row.DuplicatesSince),
Suspect: queueView(row.Suspect, row.SuspectSince),
+6 -7
View File
@@ -35,6 +35,7 @@ func getAdminSummary(t *testing.T, h *handlers, user dbq.User) (int, adminSummar
// oldest item has waited; an empty queue has no since.
func TestAdminSummary_CountsTheQueuesAndSinceWhen(t *testing.T) {
h, _ := testHandlers(t)
truncateLibrary(t, h.pool)
admin := seedUser(t, h.pool, "admin", "pw", true)
listener := seedUser(t, h.pool, "listener", "pw", false)
ctx := context.Background()
@@ -43,19 +44,17 @@ func TestAdminSummary_CountsTheQueuesAndSinceWhen(t *testing.T) {
if code != http.StatusOK {
t.Fatalf("status = %d", code)
}
if empty.Requests.Count != 0 || empty.Requests.Since != nil || empty.Missing.Count != 0 {
if empty.Missing.Count != 0 || empty.Missing.Since != nil {
t.Errorf("empty summary = %+v", empty)
}
if _, err := dbq.New(h.pool).CreateRequest(ctx, dbq.CreateRequestParams{
UserID: listener.ID, Kind: dbq.RequestKindArtist, ArtistMbid: "a0000000-0000-4000-8000-000000000001",
ArtistName: "Annadale",
}); err != nil {
track := seedQuarantineTrack(t, h, "sum-missing")
if _, err := h.pool.Exec(ctx, `UPDATE tracks SET missing_since = now() WHERE id = $1`, track.ID); err != nil {
t.Fatal(err)
}
_, got := getAdminSummary(t, h, admin)
if got.Requests.Count != 1 || got.Requests.Since == nil {
t.Errorf("requests = %+v, want 1 with a since", got.Requests)
if got.Missing.Count != 1 || got.Missing.Since == nil {
t.Errorf("missing = %+v, want 1 with a since", got.Missing)
}
if code, _ := getAdminSummary(t, h, listener); code != http.StatusForbidden {
-9
View File
@@ -31,10 +31,6 @@ type suspectTrackView struct {
// first pass. VerdictAt is when it was last set.
Verdict *string `json:"verdict"`
VerdictAt *string `json:"verdict_at"`
// Replacing: the album is on the wanted list for a clean copy;
// SearchedAt is when one was last searched for, null before the first.
Replacing bool `json:"replacing"`
SearchedAt *string `json:"searched_at"`
}
// suspectGroupView is a folder's worth of flagged tracks. As on the
@@ -116,11 +112,6 @@ func groupSuspectByDirectory(rows []dbq.ListSuspectSourceTracksRow) []suspectGro
at := row.SourceVerdictAt.Time.UTC().Format(time.RFC3339)
t.VerdictAt = &at
}
t.Replacing = row.WantUpgrade == "rip"
if t.Replacing && row.SearchedAt.Valid {
at := row.SearchedAt.Time.UTC().Format(time.RFC3339)
t.SearchedAt = &at
}
if n := len(groups); n > 0 && groups[n-1].Directory == row.Directory {
groups[n-1].Tracks = append(groups[n-1].Tracks, t)
continue
+24 -76
View File
@@ -17,13 +17,12 @@ import (
)
type adminUserView struct {
ID string `json:"id"`
Username string `json:"username"`
DisplayName *string `json:"display_name"`
IsAdmin bool `json:"is_admin"`
AutoApproveRequests bool `json:"auto_approve_requests"`
DebugModeEnabled bool `json:"debug_mode_enabled"`
CreatedAt string `json:"created_at"`
ID string `json:"id"`
Username string `json:"username"`
DisplayName *string `json:"display_name"`
IsAdmin bool `json:"is_admin"`
DebugModeEnabled bool `json:"debug_mode_enabled"`
CreatedAt string `json:"created_at"`
}
type listUsersResp struct {
@@ -40,13 +39,12 @@ func (h *handlers) handleAdminListUsers(w http.ResponseWriter, r *http.Request)
out := make([]adminUserView, 0, len(rows))
for _, row := range rows {
v := adminUserView{
ID: uuidToString(row.ID),
Username: row.Username,
DisplayName: row.DisplayName,
IsAdmin: row.IsAdmin,
AutoApproveRequests: row.AutoApproveRequests,
DebugModeEnabled: row.DebugModeEnabled,
CreatedAt: row.CreatedAt.Time.UTC().Format(time.RFC3339),
ID: uuidToString(row.ID),
Username: row.Username,
DisplayName: row.DisplayName,
IsAdmin: row.IsAdmin,
DebugModeEnabled: row.DebugModeEnabled,
CreatedAt: row.CreatedAt.Time.UTC().Format(time.RFC3339),
}
out = append(out, v)
}
@@ -117,13 +115,12 @@ func (h *handlers) handleUpdateUserAdmin(w http.ResponseWriter, r *http.Request)
audit.WriteOrLog(r.Context(), h.pool, h.logger, caller.ID, targetID, action, nil)
writeJSON(w, http.StatusOK, adminUserView{
ID: uuidToString(updated.ID),
Username: updated.Username,
DisplayName: updated.DisplayName,
IsAdmin: updated.IsAdmin,
AutoApproveRequests: updated.AutoApproveRequests,
DebugModeEnabled: updated.DebugModeEnabled,
CreatedAt: updated.CreatedAt.Time.UTC().Format(time.RFC3339),
ID: uuidToString(updated.ID),
Username: updated.Username,
DisplayName: updated.DisplayName,
IsAdmin: updated.IsAdmin,
DebugModeEnabled: updated.DebugModeEnabled,
CreatedAt: updated.CreatedAt.Time.UTC().Format(time.RFC3339),
})
}
@@ -187,13 +184,12 @@ func (h *handlers) handleAdminCreateUser(w http.ResponseWriter, r *http.Request)
map[string]any{"is_admin": req.IsAdmin})
writeJSON(w, http.StatusOK, adminUserView{
ID: uuidToString(user.ID),
Username: user.Username,
DisplayName: user.DisplayName,
IsAdmin: user.IsAdmin,
AutoApproveRequests: user.AutoApproveRequests,
DebugModeEnabled: user.DebugModeEnabled,
CreatedAt: user.CreatedAt.Time.UTC().Format(time.RFC3339),
ID: uuidToString(user.ID),
Username: user.Username,
DisplayName: user.DisplayName,
IsAdmin: user.IsAdmin,
DebugModeEnabled: user.DebugModeEnabled,
CreatedAt: user.CreatedAt.Time.UTC().Format(time.RFC3339),
})
}
@@ -302,51 +298,3 @@ func (h *handlers) handleAdminResetPassword(w http.ResponseWriter, r *http.Reque
w.WriteHeader(http.StatusNoContent)
}
// --- Admin auto-approve toggle ---------------------------------------------
type adminAutoApproveReq struct {
AutoApprove bool `json:"auto_approve"`
}
func (h *handlers) handleAdminAutoApproveToggle(w http.ResponseWriter, r *http.Request) {
caller, ok := requireUser(w, r)
if !ok {
return
}
targetID, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
var req adminAutoApproveReq
if !decodeBody(w, r, &req) {
return
}
q := dbq.New(h.pool)
updated, err := q.UpdateUserAutoApprove(r.Context(), dbq.UpdateUserAutoApproveParams{
ID: targetID,
AutoApproveRequests: req.AutoApprove,
})
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
writeErr(w, &apierror.Error{Status: http.StatusNotFound, Code: "not_found", Message: ""})
return
}
writeErrWithLog(w, h.logger, "admin auto-approve: update failed", apierror.Internal(err))
return
}
audit.WriteOrLog(r.Context(), h.pool, h.logger, caller.ID, targetID, audit.ActionAutoApproveToggle,
map[string]any{"auto_approve": req.AutoApprove})
writeJSON(w, http.StatusOK, adminUserView{
ID: uuidToString(updated.ID),
Username: updated.Username,
DisplayName: updated.DisplayName,
IsAdmin: updated.IsAdmin,
AutoApproveRequests: updated.AutoApproveRequests,
DebugModeEnabled: updated.DebugModeEnabled,
CreatedAt: updated.CreatedAt.Time.UTC().Format(time.RFC3339),
})
}
-38
View File
@@ -25,7 +25,6 @@ func newAdminUsersRouter(h *handlers) chi.Router {
admin.Post("/users", h.handleAdminCreateUser)
admin.Delete("/users/{id}", h.handleAdminDeleteUser)
admin.Post("/users/{id}/reset-password", h.handleAdminResetPassword)
admin.Put("/users/{id}/auto-approve", h.handleAdminAutoApproveToggle)
})
return r
}
@@ -271,40 +270,3 @@ func TestAdminResetPassword_TooShort_400(t *testing.T) {
t.Errorf("status = %d, want 400", rec.Code)
}
}
func TestAdminAutoApproveToggle(t *testing.T) {
if os.Getenv("MINSTREL_TEST_DATABASE_URL") == "" {
t.Skip("MINSTREL_TEST_DATABASE_URL not set")
}
h, pool := testHandlers(t)
admin := seedUser(t, pool, "aacaller", "pw", true)
target := seedUser(t, pool, "aatarget", "pw", false)
body := `{"auto_approve":true}`
req := httptest.NewRequest(http.MethodPut, "/api/admin/users/"+uuidToString(target.ID)+"/auto-approve",
bytes.NewReader([]byte(body)))
req = withUser(req, admin)
rec := httptest.NewRecorder()
newAdminUsersRouter(h).ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
var resp adminUserView
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v", err)
}
if !resp.AutoApproveRequests {
t.Errorf("auto_approve_requests = false on response; expected true")
}
// Verify DB.
var aa bool
if err := pool.QueryRow(context.Background(),
"SELECT auto_approve_requests FROM users WHERE id = $1", target.ID).Scan(&aa); err != nil {
t.Fatalf("verify: %v", err)
}
if !aa {
t.Errorf("DB row not updated")
}
}
-821
View File
@@ -1,821 +0,0 @@
package api
import (
"context"
"encoding/json"
"errors"
"net/http"
"strconv"
"strings"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/catalogue"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/musicbrainz"
"git.fabledsword.com/bvandeusen/minstrel/internal/wanted"
)
// The wanted list's admin surface: wants, follows and
// release-type profiles.
// wantView is one row of GET /api/admin/wanted.
type wantView struct {
ID string `json:"id"`
ReleaseGroupMBID string `json:"release_group_mbid"`
Title string `json:"title"`
ArtistMBID string `json:"artist_mbid"`
ArtistName string `json:"artist_name"`
ArtistID string `json:"artist_id,omitempty"`
PrimaryType string `json:"primary_type"`
SecondaryTypes []string `json:"secondary_types"`
Year string `json:"year"`
Reason string `json:"reason"`
RequestedBy string `json:"requested_by,omitempty"`
AlbumID string `json:"album_id,omitempty"`
WantedAt string `json:"wanted_at"`
HeldAt *string `json:"held_at"`
ImageURL string `json:"image_url"`
// Upgrade: why a held album is wanted again, quality or rip, and
// HeldQuality what the library's copy is.
Upgrade string `json:"upgrade,omitempty"`
HeldQuality string `json:"held_quality,omitempty"`
// Search is the schedule's last look; Grab the download running.
Search *wantSearchView `json:"search"`
Grab *wantGrabView `json:"grab"`
}
type wantSearchView struct {
SearchedAt string `json:"searched_at"`
Outcome string `json:"outcome"`
Considered int32 `json:"considered"`
BestRelease string `json:"best_release"`
BestVerdict string `json:"best_verdict"`
Error string `json:"error"`
}
// wantGrabView is the want's latest download, running or ended, and what
// its import made of it.
type wantGrabView struct {
ID string `json:"id"`
Name string `json:"name"`
GrabbedAt string `json:"grabbed_at"`
Origin string `json:"origin"`
Quality string `json:"quality"`
Live bool `json:"live"`
EndReason string `json:"end_reason"`
Progress float32 `json:"progress"`
ImportState string `json:"import_state"`
ImportReason string `json:"import_reason"`
EndDetail string `json:"end_detail"`
ImportedAt *string `json:"imported_at"`
ImportedFiles int32 `json:"imported_files"`
}
// wantCounts sizes each tab of the Wanted page. Artists: how many artists
// the list holds albums of, its main page.
type wantCounts struct {
Artists int64 `json:"artists"`
// Unmatched: library albums waiting to be matched by hand.
Unmatched int64 `json:"unmatched"`
Wanted int64 `json:"wanted"`
Missing int64 `json:"missing"`
Held int64 `json:"held"`
// Upgrades: held albums wanted for a better copy.
Upgrades int64 `json:"upgrades"`
Following int64 `json:"following"`
// Activity: downloads running; Attention: of those, the ones waiting on
// a person; Blocked: releases set aside.
Activity int64 `json:"activity"`
Attention int64 `json:"attention"`
Blocked int64 `json:"blocked"`
}
type wantsPage struct {
Items []wantView `json:"items"`
Total int64 `json:"total"`
Limit int `json:"limit"`
Offset int `json:"offset"`
Counts wantCounts `json:"counts"`
// Library is the one-time listing of the library's albums.
Library libraryListingView `json:"library"`
// Artist heads one artist's page, when the list is narrowed to them.
Artist *wantArtistView `json:"artist,omitempty"`
}
// wantArtistView is one artist on the Wanted view's main page: how
// many of their albums the list holds, of how many.
type wantArtistView struct {
ArtistMBID string `json:"artist_mbid"`
ArtistName string `json:"artist_name"`
SortName string `json:"sort_name"`
ArtistID string `json:"artist_id,omitempty"`
ImageURL string `json:"image_url"`
Albums int32 `json:"albums"`
Held int32 `json:"held"`
Wanted int32 `json:"wanted"`
Missing int32 `json:"missing"`
Downloading int32 `json:"downloading"`
Following bool `json:"following"`
}
type wantArtistsPage struct {
Items []wantArtistView `json:"items"`
Total int64 `json:"total"`
Limit int `json:"limit"`
Offset int `json:"offset"`
Counts wantCounts `json:"counts"`
Library libraryListingView `json:"library"`
}
// libraryListingView: whether the library's albums are all on the list
// yet, and how many could not be matched to MusicBrainz; those are
// listed in the Collection's Unmatched section.
type libraryListingView struct {
ListedAt *string `json:"listed_at"`
Unlisted int32 `json:"unlisted"`
}
// unmatchedAlbumView is one album in the Collection's Unmatched section: why
// no MusicBrainz album matched it, or that a person left it off.
type unmatchedAlbumView struct {
AlbumID string `json:"album_id"`
Title string `json:"title"`
Artist string `json:"artist"`
Detail string `json:"detail"`
Tracks int32 `json:"tracks"`
LeftOff bool `json:"left_off"`
DecidedAt string `json:"decided_at"`
}
// unmatchedShown bounds the Unmatched section: a library a listing could
// not match more of than this has a bigger problem than one page shows.
const unmatchedShown = 500
const wantsPageSize = 50
// wantArtistsPageSize fills a few rows of the artist grid a page.
const wantArtistsPageSize = 96
// wantsHeader is what every Collection page carries: each section's count,
// and how the one-time library listing went.
func (h *handlers) wantsHeader(r *http.Request, q *dbq.Queries) (wantCounts, libraryListingView, error) {
counts, err := q.CountWants(r.Context())
if err != nil {
return wantCounts{}, libraryListingView{}, err
}
listing, err := q.GetLibraryListing(r.Context())
if err != nil {
return wantCounts{}, libraryListingView{}, err
}
lib := libraryListingView{Unlisted: listing.LibraryUnlisted}
if listing.LibraryListedAt.Valid {
at := formatTimestamp(listing.LibraryListedAt)
lib.ListedAt = &at
}
return wantCounts{Artists: counts.Artists, Unmatched: counts.Unmatched, Wanted: counts.Wanted, Missing: counts.Missing, Held: counts.Held,
Upgrades: counts.Upgrades, Following: counts.Following, Activity: counts.Activity, Attention: counts.Attention, Blocked: counts.Blocked}, lib, nil
}
func offsetParam(r *http.Request) int {
offset, _ := strconv.Atoi(r.URL.Query().Get("offset"))
return max(offset, 0)
}
func wantArtistViewOf(row dbq.ListWantArtistsRow) wantArtistView {
v := wantArtistView{
ArtistMBID: row.ArtistMbid, ArtistName: row.ArtistName, SortName: row.SortName,
ImageURL: artistArtURL(row.ArtistMbid, row.ArtistName),
Albums: row.Albums, Held: row.Held, Wanted: row.Wanted, Missing: row.Missing,
Downloading: row.Downloading, Following: row.Following,
}
if row.LibraryArtistID.Valid {
v.ArtistID = uuidToString(row.LibraryArtistID)
}
return v
}
// handleListWantArtists implements GET /api/admin/wanted/artists?q=&offset=:
// the Wanted view's main page, every artist on the list A–Z.
func (h *handlers) handleListWantArtists(w http.ResponseWriter, r *http.Request) {
offset := offsetParam(r)
q := dbq.New(h.pool)
rows, err := q.ListWantArtists(r.Context(), dbq.ListWantArtistsParams{
Q: strings.TrimSpace(r.URL.Query().Get("q")), Skip: int32(offset), MaxRows: wantArtistsPageSize})
if err != nil {
writeErrWithLog(w, h.logger, "admin wanted: artists", apierror.Internal(err))
return
}
counts, lib, err := h.wantsHeader(r, q)
if err != nil {
writeErrWithLog(w, h.logger, "admin wanted: counts", apierror.Internal(err))
return
}
out := wantArtistsPage{Items: make([]wantArtistView, 0, len(rows)), Limit: wantArtistsPageSize, Offset: offset,
Counts: counts, Library: lib}
for _, row := range rows {
out.Total = row.Total
out.Items = append(out.Items, wantArtistViewOf(row))
}
writeJSON(w, http.StatusOK, out)
}
// handleListWants implements GET
// /api/admin/wanted?state=wanted|missing|upgrade|held|all&artist=&q=&offset=.
// wanted is everything open but missing files; upgrade the held albums wanted
// for a better copy. Each list runs artist by artist, A–Z, like the library; q
// narrows it to an artist or album, and artist to one artist's page, which it
// then heads.
func (h *handlers) handleListWants(w http.ResponseWriter, r *http.Request) {
state := r.URL.Query().Get("state")
if state == "" {
state = "wanted"
}
if state != "wanted" && state != "missing" && state != "upgrade" && state != "held" && state != "all" {
writeErr(w, apierror.BadRequest("bad_state", "state must be wanted, missing, upgrade, held or all"))
return
}
offset := offsetParam(r)
artist := strings.TrimSpace(r.URL.Query().Get("artist"))
q := dbq.New(h.pool)
rows, err := q.ListWants(r.Context(), dbq.ListWantsParams{State: state, ArtistMbid: artist,
Q: strings.TrimSpace(r.URL.Query().Get("q")), Skip: int32(offset), MaxRows: wantsPageSize})
if err != nil {
writeErrWithLog(w, h.logger, "admin wanted: list", apierror.Internal(err))
return
}
counts, lib, err := h.wantsHeader(r, q)
if err != nil {
writeErrWithLog(w, h.logger, "admin wanted: counts", apierror.Internal(err))
return
}
out := wantsPage{Library: lib, Counts: counts,
Items: make([]wantView, 0, len(rows)), Limit: wantsPageSize, Offset: offset}
if artist != "" {
heads, err := q.ListWantArtists(r.Context(), dbq.ListWantArtistsParams{ArtistMbid: artist, MaxRows: 1})
if err != nil {
writeErrWithLog(w, h.logger, "admin wanted: artist", apierror.Internal(err))
return
}
if len(heads) == 0 {
writeErr(w, apierror.NotFound("artist"))
return
}
v := wantArtistViewOf(heads[0])
out.Artist = &v
}
for _, row := range rows {
out.Total = row.Total
out.Items = append(out.Items, wantViewOf(row))
}
writeJSON(w, http.StatusOK, out)
}
// wantViewOf is one ListWants row as the wanted list and the Missing page
// send it.
func wantViewOf(row dbq.ListWantsRow) wantView {
v := wantView{
ID: uuidToString(row.ID), ReleaseGroupMBID: row.ReleaseGroupMbid, Title: row.Title,
ArtistMBID: row.ArtistMbid, ArtistName: row.ArtistName,
PrimaryType: row.PrimaryType, SecondaryTypes: row.SecondaryTypes,
Year: first4(row.FirstReleaseDate), Reason: row.Reason,
WantedAt: formatTimestamp(row.WantedAt), Upgrade: row.Upgrade, HeldQuality: row.HeldQuality,
ImageURL: groupArtURL(musicbrainz.ReleaseGroup{MBID: row.ReleaseGroupMbid, Title: row.Title, ArtistCredit: row.ArtistName}),
}
if row.LibraryArtistID.Valid {
v.ArtistID = uuidToString(row.LibraryArtistID)
}
if row.AlbumID.Valid {
v.AlbumID = uuidToString(row.AlbumID)
}
if row.RequestedBy != nil {
v.RequestedBy = *row.RequestedBy
}
if row.FulfilledAt.Valid {
held := formatTimestamp(row.FulfilledAt)
v.HeldAt = &held
}
if v.SecondaryTypes == nil {
v.SecondaryTypes = []string{}
}
if row.SearchedAt.Valid {
v.Search = &wantSearchView{
SearchedAt: formatTimestamp(row.SearchedAt), Outcome: row.SearchOutcome,
Considered: row.SearchConsidered, BestRelease: row.BestRelease,
BestVerdict: row.BestVerdict, Error: row.SearchError,
}
}
if row.GrabName != nil {
v.Grab = &wantGrabView{ID: uuidToString(row.GrabID), Name: *row.GrabName,
GrabbedAt: formatTimestamp(row.GrabbedAt), Live: !row.GrabEndedAt.Valid,
Origin: deref(row.GrabOrigin), Quality: deref(row.GrabQuality), EndReason: deref(row.GrabEndReason),
ImportState: deref(row.GrabImportState), ImportReason: deref(row.GrabImportReason),
EndDetail: deref(row.GrabEndDetail)}
if row.GrabProgress != nil {
v.Grab.Progress = *row.GrabProgress
}
if row.GrabImportedFiles != nil {
v.Grab.ImportedFiles = *row.GrabImportedFiles
}
if row.GrabImportedAt.Valid {
at := formatTimestamp(row.GrabImportedAt)
v.Grab.ImportedAt = &at
}
}
return v
}
// wantTracksView is an album's track list for the Wanted view: the
// library's own tracks for an album it holds or held, else MusicBrainz's
// standard edition of what is wanted.
type wantTracksView struct {
// library, musicbrainz, or none when MusicBrainz lists no edition.
Source string `json:"source"`
// Edition names the MusicBrainz release the list is from.
Edition string `json:"edition,omitempty"`
Tracks []wantTrackView `json:"tracks"`
}
type wantTrackView struct {
Disc int32 `json:"disc"`
Number int32 `json:"number"`
Title string `json:"title"`
LengthMs int32 `json:"length_ms"`
// Missing: the library's file for it is gone.
Missing bool `json:"missing"`
}
// handleWantTracks implements GET /api/admin/wanted/{id}/tracks.
func (h *handlers) handleWantTracks(w http.ResponseWriter, r *http.Request) {
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
q := dbq.New(h.pool)
want, err := q.GetWantAlbum(r.Context(), id)
if errors.Is(err, pgx.ErrNoRows) {
writeErr(w, apierror.NotFound("want"))
return
}
if err != nil {
writeErrWithLog(w, h.logger, "admin wanted: tracks", apierror.Internal(err))
return
}
out := wantTracksView{Tracks: []wantTrackView{}}
if want.AlbumID.Valid {
rows, err := q.ListLibraryAlbumTracks(r.Context(), want.AlbumID)
if err != nil {
writeErrWithLog(w, h.logger, "admin wanted: library tracks", apierror.Internal(err))
return
}
out.Source = "library"
for _, t := range rows {
v := wantTrackView{Title: t.Title, LengthMs: t.DurationMs, Missing: t.Missing}
if t.DiscNumber != nil {
v.Disc = *t.DiscNumber
}
if t.TrackNumber != nil {
v.Number = *t.TrackNumber
}
out.Tracks = append(out.Tracks, v)
}
writeJSON(w, http.StatusOK, out)
return
}
eds, err := h.catalogue.Releases(r.Context(), want.ReleaseGroupMbid)
switch {
case errors.Is(err, catalogue.ErrNotFound):
out.Source = "none"
writeJSON(w, http.StatusOK, out)
return
case errors.Is(err, musicbrainz.ErrUnreachable), errors.Is(err, musicbrainz.ErrBadResponse):
writeErr(w, &apierror.Error{Status: 503, Code: "musicbrainz_unreachable", Message: "MusicBrainz can't be reached", Cause: err})
return
case err != nil:
writeErrWithLog(w, h.logger, "admin wanted: editions", apierror.Internal(err))
return
}
e, ok := standardEdition(eds.Editions)
if !ok {
out.Source = "none"
writeJSON(w, http.StatusOK, out)
return
}
out.Source = "musicbrainz"
out.Edition = editionLabel(e.Release)
for _, t := range e.Tracks {
out.Tracks = append(out.Tracks, wantTrackView{Disc: t.MediumPosition, Number: t.Position, Title: t.Title, LengthMs: t.LengthMs})
}
writeJSON(w, http.StatusOK, out)
}
// standardEdition is the edition a person would call the album: an official
// one with the fewest tracks, else any with the fewest. The catalogue lists
// official editions first by date, so a tie goes to the earliest.
func standardEdition(eds []catalogue.Edition) (catalogue.Edition, bool) {
best := -1
for i, e := range eds {
if len(e.Tracks) == 0 {
continue
}
if best < 0 {
best = i
continue
}
b := eds[best]
bo, eo := b.Release.Status == "Official", e.Release.Status == "Official"
if (eo && !bo) || (eo == bo && len(e.Tracks) < len(b.Tracks)) {
best = i
}
}
if best < 0 {
return catalogue.Edition{}, false
}
return eds[best], true
}
// editionLabel names a release for a person: "2023 · CD · GB".
func editionLabel(r dbq.MbRelease) string {
var parts []string
for _, p := range []string{first4(r.ReleaseDate), r.Formats, r.Country} {
if p != "" {
parts = append(parts, p)
}
}
return strings.Join(parts, " · ")
}
func first4(date string) string {
if len(date) < 4 {
return ""
}
return date[:4]
}
// handleRemoveWant implements DELETE /api/admin/wanted/{id}.
func (h *handlers) handleRemoveWant(w http.ResponseWriter, r *http.Request) {
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
removed, err := h.wanted.RemoveWant(r.Context(), id)
if err != nil {
writeErrWithLog(w, h.logger, "admin wanted: remove", apierror.Internal(err))
return
}
if !removed {
writeErr(w, apierror.NotFound("want"))
return
}
w.WriteHeader(http.StatusNoContent)
}
// followView is a followed artist.
type followView struct {
ArtistMBID string `json:"artist_mbid"`
ArtistName string `json:"artist_name"`
ArtistID string `json:"artist_id,omitempty"`
ProfileID string `json:"profile_id"`
ProfileName string `json:"profile_name"`
FollowedAt string `json:"followed_at"`
CheckedAt *string `json:"checked_at"`
OpenWants int64 `json:"open_wants"`
ImageURL string `json:"image_url"`
}
// handleListFollows implements GET /api/admin/follows.
func (h *handlers) handleListFollows(w http.ResponseWriter, r *http.Request) {
rows, err := dbq.New(h.pool).ListFollows(r.Context())
if err != nil {
writeErrWithLog(w, h.logger, "admin follows: list", apierror.Internal(err))
return
}
out := make([]followView, 0, len(rows))
for _, f := range rows {
v := followView{
ArtistMBID: f.ArtistMbid, ArtistName: f.ArtistName,
ProfileID: uuidToString(f.ProfileID), ProfileName: f.ProfileName,
FollowedAt: formatTimestamp(f.FollowedAt), OpenWants: f.OpenWants,
ImageURL: artistArtURL(f.ArtistMbid, f.ArtistName),
}
if f.ArtistID.Valid {
v.ArtistID = uuidToString(f.ArtistID)
}
if f.CheckedAt.Valid {
checked := formatTimestamp(f.CheckedAt)
v.CheckedAt = &checked
}
out = append(out, v)
}
writeJSON(w, http.StatusOK, out)
}
// followStateView answers whether one artist is followed, for the control on
// an artist page.
type followStateView struct {
Following bool `json:"following"`
ProfileID string `json:"profile_id,omitempty"`
ProfileName string `json:"profile_name,omitempty"`
}
// handleGetFollow implements GET /api/admin/follows/{mbid}.
func (h *handlers) handleGetFollow(w http.ResponseWriter, r *http.Request) {
f, err := dbq.New(h.pool).GetFollow(r.Context(), chi.URLParam(r, "mbid"))
if errors.Is(err, pgx.ErrNoRows) {
writeJSON(w, http.StatusOK, followStateView{})
return
}
if err != nil {
writeErrWithLog(w, h.logger, "admin follows: get", apierror.Internal(err))
return
}
writeJSON(w, http.StatusOK, followStateView{
Following: true, ProfileID: uuidToString(f.ProfileID), ProfileName: f.ProfileName,
})
}
type putFollowBody struct {
ArtistName string `json:"artist_name"`
ProfileID string `json:"profile_id"`
WantExisting bool `json:"want_existing"`
}
// handlePutFollow implements PUT /api/admin/follows/{mbid}: follow an artist,
// or change the profile it is followed on. want_existing also wants what the
// artist has already released that the profile passes and isn't held.
func (h *handlers) handlePutFollow(w http.ResponseWriter, r *http.Request) {
admin, ok := requireUser(w, r)
if !ok {
return
}
var body putFollowBody
if !decodeBody(w, r, &body) {
return
}
var profile pgtype.UUID
if body.ProfileID != "" {
if profile, ok = parseUUID(body.ProfileID); !ok {
writeErr(w, apierror.BadRequest("bad_profile", "profile_id is not an id"))
return
}
}
mbid := chi.URLParam(r, "mbid")
if _, err := h.wanted.Follow(r.Context(), wanted.FollowParams{
ArtistMBID: mbid, ArtistName: body.ArtistName, ProfileID: profile,
By: admin.ID, WantExisting: body.WantExisting,
}); err != nil {
switch {
case errors.Is(err, wanted.ErrNotFollowable):
writeErr(w, apierror.BadRequest("not_followable", "Various Artists and MusicBrainz placeholders can't be followed"))
case errors.Is(err, wanted.ErrProfileNotFound):
writeErr(w, apierror.BadRequest("profile_not_found", "no such profile"))
default:
writeErrWithLog(w, h.logger, "admin follows: put", apierror.Internal(err))
}
return
}
h.handleGetFollow(w, r)
}
// handleDeleteFollow implements DELETE /api/admin/follows/{mbid}. What the
// follow already wanted stays on the list.
func (h *handlers) handleDeleteFollow(w http.ResponseWriter, r *http.Request) {
removed, err := h.wanted.Unfollow(r.Context(), chi.URLParam(r, "mbid"))
if err != nil {
writeErrWithLog(w, h.logger, "admin follows: delete", apierror.Internal(err))
return
}
if !removed {
writeErr(w, apierror.NotFound("follow"))
return
}
w.WriteHeader(http.StatusNoContent)
}
// releaseProfileView is a release-type profile.
type releaseProfileView struct {
ID string `json:"id"`
Name string `json:"name"`
PrimaryTypes []string `json:"primary_types"`
SecondaryTypes []string `json:"secondary_types"`
IsDefault bool `json:"is_default"`
FollowCount int64 `json:"follow_count"`
}
// releaseProfilesResponse also carries MusicBrainz's types, so the editor
// offers exactly what a profile may hold.
type releaseProfilesResponse struct {
Profiles []releaseProfileView `json:"profiles"`
PrimaryTypes []string `json:"primary_types"`
SecondaryTypes []string `json:"secondary_types"`
}
// handleListReleaseProfiles implements GET /api/admin/release-profiles.
func (h *handlers) handleListReleaseProfiles(w http.ResponseWriter, r *http.Request) {
rows, err := dbq.New(h.pool).ListReleaseTypeProfiles(r.Context())
if err != nil {
writeErrWithLog(w, h.logger, "admin release profiles: list", apierror.Internal(err))
return
}
out := releaseProfilesResponse{
Profiles: make([]releaseProfileView, 0, len(rows)),
PrimaryTypes: wanted.PrimaryTypes, SecondaryTypes: wanted.SecondaryTypes,
}
for _, p := range rows {
out.Profiles = append(out.Profiles, releaseProfileView{
ID: uuidToString(p.ID), Name: p.Name, PrimaryTypes: p.PrimaryTypes,
SecondaryTypes: p.SecondaryTypes, IsDefault: p.IsDefault, FollowCount: p.FollowCount,
})
}
writeJSON(w, http.StatusOK, out)
}
type releaseProfileBody struct {
Name string `json:"name"`
PrimaryTypes []string `json:"primary_types"`
SecondaryTypes []string `json:"secondary_types"`
IsDefault bool `json:"is_default"`
}
func (b releaseProfileBody) input() wanted.ProfileInput {
return wanted.ProfileInput{
Name: b.Name, PrimaryTypes: b.PrimaryTypes, SecondaryTypes: b.SecondaryTypes, IsDefault: b.IsDefault,
}
}
// handleCreateReleaseProfile implements POST /api/admin/release-profiles.
func (h *handlers) handleCreateReleaseProfile(w http.ResponseWriter, r *http.Request) {
var body releaseProfileBody
if !decodeBody(w, r, &body) {
return
}
p, err := h.wanted.CreateProfile(r.Context(), body.input())
if err != nil {
h.writeProfileErr(w, err)
return
}
writeJSON(w, http.StatusCreated, releaseProfileView{
ID: uuidToString(p.ID), Name: p.Name, PrimaryTypes: p.PrimaryTypes,
SecondaryTypes: p.SecondaryTypes, IsDefault: p.IsDefault,
})
}
// handleUpdateReleaseProfile implements PUT /api/admin/release-profiles/{id}.
// A follow on the profile judges new releases by the new types from its next
// check; releases it has already judged are not judged again.
func (h *handlers) handleUpdateReleaseProfile(w http.ResponseWriter, r *http.Request) {
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
var body releaseProfileBody
if !decodeBody(w, r, &body) {
return
}
p, err := h.wanted.UpdateProfile(r.Context(), id, body.input())
if err != nil {
h.writeProfileErr(w, err)
return
}
writeJSON(w, http.StatusOK, releaseProfileView{
ID: uuidToString(p.ID), Name: p.Name, PrimaryTypes: p.PrimaryTypes,
SecondaryTypes: p.SecondaryTypes, IsDefault: p.IsDefault,
})
}
// handleDeleteReleaseProfile implements DELETE
// /api/admin/release-profiles/{id}.
func (h *handlers) handleDeleteReleaseProfile(w http.ResponseWriter, r *http.Request) {
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
if err := h.wanted.DeleteProfile(r.Context(), id); err != nil {
h.writeProfileErr(w, err)
return
}
w.WriteHeader(http.StatusNoContent)
}
func (h *handlers) writeProfileErr(w http.ResponseWriter, err error) {
switch {
case errors.Is(err, wanted.ErrProfileInvalid):
writeErr(w, apierror.BadRequest("profile_invalid", err.Error()))
case errors.Is(err, wanted.ErrProfileNotFound):
writeErr(w, apierror.NotFound("profile"))
case errors.Is(err, wanted.ErrProfileNameUsed):
writeErr(w, apierror.Conflict("profile_name_taken", "a profile already has that name"))
case errors.Is(err, wanted.ErrProfileInUse):
writeErr(w, apierror.Conflict("profile_in_use", "artists are followed on this profile; move them first"))
case errors.Is(err, wanted.ErrProfileDefault):
writeErr(w, apierror.Conflict("profile_is_default", "the default profile can't be deleted"))
default:
writeErrWithLog(w, h.logger, "admin release profiles", apierror.Internal(err))
}
}
// deref is a nullable column's text, or "".
func deref(s *string) string {
if s == nil {
return ""
}
return *s
}
// handleListUnmatched implements GET /api/admin/wanted/unmatched: the
// Collection's Unmatched section, the albums no MusicBrainz album
// matched and, after them, the ones a person left off.
func (h *handlers) handleListUnmatched(w http.ResponseWriter, r *http.Request) {
rows, err := dbq.New(h.pool).ListUnmatchedLibraryAlbums(r.Context(), unmatchedShown)
if err != nil {
writeErrWithLog(w, h.logger, "admin wanted: unmatched", apierror.Internal(err))
return
}
out := struct {
Items []unmatchedAlbumView `json:"items"`
}{Items: make([]unmatchedAlbumView, 0, len(rows))}
for _, u := range rows {
out.Items = append(out.Items, unmatchedAlbumView{
AlbumID: uuidToString(u.AlbumID), Title: u.Title, Artist: u.ArtistName, Detail: u.Detail,
Tracks: u.Tracks, LeftOff: u.Outcome == "dismissed", DecidedAt: formatTimestamp(u.DecidedAt),
})
}
writeJSON(w, http.StatusOK, out)
}
type manualMatchView struct {
ReleaseGroupMBID string `json:"release_group_mbid"`
Title string `json:"title"`
Artist string `json:"artist"`
Agree int `json:"agree"`
Tracks int `json:"tracks"`
}
// handleMatchUnmatched implements POST /api/admin/wanted/unmatched/{id}/match
// {"ref": "<MusicBrainz link or id>"}: the album goes on the list as held,
// as that MusicBrainz album.
func (h *handlers) handleMatchUnmatched(w http.ResponseWriter, r *http.Request) {
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
var body struct {
Ref string `json:"ref"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeErr(w, apierror.BadRequest("bad_body", "expected {\"ref\": \"…\"}"))
return
}
m, err := h.wanted.MatchAlbumTo(r.Context(), id, body.Ref)
switch {
case errors.Is(err, wanted.ErrMatchRef):
writeErr(w, apierror.BadRequest("bad_ref", "That isn't a MusicBrainz album link or id."))
return
case errors.Is(err, wanted.ErrMatchNoGroup):
writeErr(w, &apierror.Error{Status: http.StatusUnprocessableEntity, Code: "no_such_album",
Message: "MusicBrainz has no album at that link.", Cause: err})
return
case errors.Is(err, wanted.ErrAlbumNotUnmatched):
writeErr(w, apierror.NotFound("unmatched album"))
return
case errors.Is(err, musicbrainz.ErrUnreachable), errors.Is(err, musicbrainz.ErrBadResponse):
writeErr(w, &apierror.Error{Status: http.StatusServiceUnavailable, Code: "musicbrainz_unreachable",
Message: "MusicBrainz can't be reached", Cause: err})
return
case err != nil:
writeErrWithLog(w, h.logger, "admin wanted: match by hand", apierror.Internal(err))
return
}
writeJSON(w, http.StatusOK, manualMatchView{ReleaseGroupMBID: m.Group, Title: m.Title, Artist: m.Artist,
Agree: m.Agree, Tracks: m.Tracks})
}
// handleLeaveUnmatchedOff implements POST
// /api/admin/wanted/unmatched/{id}/leave-off, and handlePutUnmatchedBack its
// undoing, POST …/{id}/put-back.
func (h *handlers) handleLeaveUnmatchedOff(w http.ResponseWriter, r *http.Request) {
h.setUnmatched(w, r, h.wanted.LeaveAlbumOff)
}
func (h *handlers) handlePutUnmatchedBack(w http.ResponseWriter, r *http.Request) {
h.setUnmatched(w, r, h.wanted.PutAlbumBack)
}
func (h *handlers) setUnmatched(w http.ResponseWriter, r *http.Request, set func(context.Context, pgtype.UUID) error) {
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
err := set(r.Context(), id)
switch {
case errors.Is(err, wanted.ErrAlbumNotUnmatched):
writeErr(w, apierror.NotFound("unmatched album"))
case err != nil:
writeErrWithLog(w, h.logger, "admin wanted: unmatched", apierror.Internal(err))
default:
w.WriteHeader(http.StatusNoContent)
}
}
-371
View File
@@ -1,371 +0,0 @@
package api
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"github.com/go-chi/chi/v5"
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
"git.fabledsword.com/bvandeusen/minstrel/internal/catalogue"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/musicbrainz"
"git.fabledsword.com/bvandeusen/minstrel/internal/wanted"
)
func doWantedReq(t *testing.T, h *handlers, path string, user dbq.User) *httptest.ResponseRecorder {
t.Helper()
return doWanted(t, h, http.MethodGet, path, "", user)
}
func doWanted(t *testing.T, h *handlers, method, path, body string, user dbq.User) *httptest.ResponseRecorder {
t.Helper()
r := chi.NewRouter()
r.Route("/api/admin", func(admin chi.Router) {
admin.Use(auth.RequireAdmin())
admin.Get("/wanted", h.handleListWants)
admin.Get("/wanted/artists", h.handleListWantArtists)
admin.Get("/wanted/unmatched", h.handleListUnmatched)
admin.Post("/wanted/unmatched/{id}/match", h.handleMatchUnmatched)
admin.Post("/wanted/unmatched/{id}/leave-off", h.handleLeaveUnmatchedOff)
admin.Post("/wanted/unmatched/{id}/put-back", h.handlePutUnmatchedBack)
admin.Get("/wanted/{id}/tracks", h.handleWantTracks)
})
w := httptest.NewRecorder()
r.ServeHTTP(w, withUser(httptest.NewRequest(method, path, strings.NewReader(body)), user))
return w
}
func seedWant(t *testing.T, h *handlers, group, artistMBID, artist, title, date string) dbq.Want {
t.Helper()
w, err := wanted.WantWith(context.Background(), dbq.New(h.pool), wanted.WantParams{
GroupMBID: group, ArtistMBID: artistMBID, ArtistName: artist, Title: title,
PrimaryType: "Album", FirstReleaseDate: date, Reason: wanted.ReasonFollow,
})
if err != nil {
t.Fatal(err)
}
return w
}
func listWanted(t *testing.T, h *handlers, user dbq.User, q string) wantsPage {
t.Helper()
w := doWantedReq(t, h, "/api/admin/wanted?"+url.Values{"q": {q}}.Encode(), user)
if w.Code != http.StatusOK {
t.Fatalf("status = %d; body = %s", w.Code, w.Body.String())
}
var page wantsPage
if err := json.Unmarshal(w.Body.Bytes(), &page); err != nil {
t.Fatal(err)
}
return page
}
// The list runs like the library: artists A–Z by the library's sort
// name where the library holds the artist, each artist's albums oldest
// first; q narrows it to an artist or an album.
func TestListWants_ArtistsAToZThenTheirAlbumsByDate(t *testing.T) {
h, _ := testHandlers(t)
admin := seedUser(t, h.pool, "admin", "pw", true)
ctx := context.Background()
beatles := "a0000000-0000-4000-8000-000000000002"
if _, err := dbq.New(h.pool).UpsertArtist(ctx, dbq.UpsertArtistParams{Name: "The Beatles", SortName: "Beatles, The", Mbid: &beatles}); err != nil {
t.Fatal(err)
}
seedWant(t, h, "g0000000-0000-4000-8000-000000000001", "a0000000-0000-4000-8000-000000000003", "Zeds", "Zed Album", "2001")
seedWant(t, h, "g0000000-0000-4000-8000-000000000002", beatles, "The Beatles", "Abbey Road", "1969-09-26")
seedWant(t, h, "g0000000-0000-4000-8000-000000000003", beatles, "The Beatles", "Please Please Me", "1963-03-22")
seedWant(t, h, "g0000000-0000-4000-8000-000000000004", "a0000000-0000-4000-8000-000000000001", "Camel", "Moonmadness", "1976")
page := listWanted(t, h, admin, "")
var got []string
for _, it := range page.Items {
got = append(got, it.ArtistName+"/"+it.Title)
}
want := "[The Beatles/Please Please Me The Beatles/Abbey Road Camel/Moonmadness Zeds/Zed Album]"
if fmt.Sprint(got) != want {
t.Errorf("order = %v, want %s", got, want)
}
if page.Items[0].ArtistID == "" || page.Items[2].ArtistID != "" {
t.Errorf("artist ids = %q, %q: only an artist the library holds has one", page.Items[0].ArtistID, page.Items[2].ArtistID)
}
page = listWanted(t, h, admin, "BEATLES")
if len(page.Items) != 2 || page.Total != 2 {
t.Errorf("q=BEATLES: %d items, total %d; want the two Beatles albums", len(page.Items), page.Total)
}
page = listWanted(t, h, admin, "moon")
if len(page.Items) != 1 || page.Items[0].Title != "Moonmadness" || page.Total != 1 {
t.Errorf("q=moon: %+v, total %d", page.Items, page.Total)
}
if page = listWanted(t, h, admin, "nothing like it"); len(page.Items) != 0 || page.Total != 0 {
t.Errorf("no match: %d items, total %d", len(page.Items), page.Total)
}
}
// The Collection's catalogue: every artist on the list A–Z, each
// with how many of their albums are held; an artist's page lists every album
// of theirs, held or not, under that artist.
func TestListWantArtists_CatalogueAndAnArtistsPage(t *testing.T) {
h, _ := testHandlers(t)
admin := seedUser(t, h.pool, "admin", "pw", true)
ctx := context.Background()
q := dbq.New(h.pool)
beatles := "a0000000-0000-4000-8000-000000000002"
camel := "a0000000-0000-4000-8000-000000000001"
if _, err := q.UpsertArtist(ctx, dbq.UpsertArtistParams{Name: "The Beatles", SortName: "Beatles, The", Mbid: &beatles}); err != nil {
t.Fatal(err)
}
seedWant(t, h, "g0000000-0000-4000-8000-000000000002", beatles, "The Beatles", "Abbey Road", "1969")
held := seedWant(t, h, "g0000000-0000-4000-8000-000000000003", beatles, "The Beatles", "Please Please Me", "1963")
if _, err := h.pool.Exec(ctx, `UPDATE wants SET fulfilled_at = now() WHERE id = $1`, held.ID); err != nil {
t.Fatal(err)
}
seedWant(t, h, "g0000000-0000-4000-8000-000000000004", camel, "Camel", "Moonmadness", "1976")
if _, err := h.pool.Exec(ctx, `INSERT INTO follows (artist_mbid, artist_name, profile_id)
SELECT $1, 'Camel', id FROM release_type_profiles WHERE is_default`, camel); err != nil {
t.Fatal(err)
}
w := doWantedReq(t, h, "/api/admin/wanted/artists", admin)
if w.Code != http.StatusOK {
t.Fatalf("status = %d; body = %s", w.Code, w.Body.String())
}
var page wantArtistsPage
if err := json.Unmarshal(w.Body.Bytes(), &page); err != nil {
t.Fatal(err)
}
if len(page.Items) != 2 || page.Total != 2 || page.Counts.Artists != 2 {
t.Fatalf("catalogue = %+v, total %d, artists %d", page.Items, page.Total, page.Counts.Artists)
}
b, c := page.Items[0], page.Items[1]
if b.ArtistName != "The Beatles" || b.SortName != "Beatles, The" || b.Albums != 2 || b.Held != 1 || b.Wanted != 1 ||
b.ArtistID == "" || b.Following || b.ImageURL == "" {
t.Errorf("Beatles = %+v", b)
}
if c.ArtistName != "Camel" || c.Albums != 1 || c.Held != 0 || !c.Following || c.ArtistID != "" {
t.Errorf("Camel = %+v", c)
}
// q finds an artist by an album's title.
w = doWantedReq(t, h, "/api/admin/wanted/artists?q=moon", admin)
page = wantArtistsPage{}
if err := json.Unmarshal(w.Body.Bytes(), &page); err != nil {
t.Fatal(err)
}
if len(page.Items) != 1 || page.Items[0].ArtistName != "Camel" {
t.Errorf("q=moon: %+v", page.Items)
}
// The Beatles' page: both albums, held or not, headed by the artist.
w = doWantedReq(t, h, "/api/admin/wanted?state=all&artist="+beatles, admin)
var albums wantsPage
if err := json.Unmarshal(w.Body.Bytes(), &albums); err != nil {
t.Fatal(err)
}
if len(albums.Items) != 2 || albums.Items[0].Title != "Please Please Me" || albums.Items[0].HeldAt == nil ||
albums.Artist == nil || albums.Artist.Held != 1 {
t.Errorf("artist page = %+v, head %+v", albums.Items, albums.Artist)
}
if w := doWantedReq(t, h, "/api/admin/wanted?state=all&artist=nobody", admin); w.Code != http.StatusNotFound {
t.Errorf("an artist with nothing on the list: status %d, want 404", w.Code)
}
}
// fakeEditions is a MusicBrainz holding one group with two editions, the
// shorter official one the standard, or 503 when down is set.
func fakeEditions(t *testing.T, group string, down bool) *httptest.Server {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if down {
w.WriteHeader(http.StatusServiceUnavailable)
return
}
switch {
case r.URL.Path == "/release-group/"+group:
_, _ = fmt.Fprintf(w, `{"id": %q, "title": "Moonmadness", "primary-type": "Album", "first-release-date": "1976"}`, group)
case r.URL.Path == "/release" && r.URL.Query().Get("release-group") == group:
_, _ = fmt.Fprint(w, `{"release-count": 2, "releases": [
{"id": "r-deluxe", "title": "Moonmadness", "status": "Official", "date": "2009", "country": "GB",
"media": [{"position": 1, "format": "CD", "tracks": [
{"position": 1, "number": "1", "title": "Aristillus", "length": 120000, "recording": {"id": "rec-1"}},
{"position": 2, "number": "2", "title": "Song Within a Song", "length": 433000, "recording": {"id": "rec-2"}},
{"position": 3, "number": "3", "title": "Bonus", "length": 200000, "recording": {"id": "rec-3"}}]}]},
{"id": "r-orig", "title": "Moonmadness", "status": "Official", "date": "1976-03", "country": "GB",
"media": [{"position": 1, "format": "12\" Vinyl", "tracks": [
{"position": 1, "number": "A1", "title": "Aristillus", "length": 120000, "recording": {"id": "rec-1"}},
{"position": 2, "number": "A2", "title": "Song Within a Song", "length": 433000, "recording": {"id": "rec-2"}}]}]}]}`)
default:
w.WriteHeader(http.StatusNotFound)
}
}))
t.Cleanup(srv.Close)
return srv
}
func wantTracks(t *testing.T, h *handlers, user dbq.User, id string) (int, wantTracksView) {
t.Helper()
w := doWantedReq(t, h, "/api/admin/wanted/"+id+"/tracks", user)
var out wantTracksView
if w.Code == http.StatusOK {
if err := json.Unmarshal(w.Body.Bytes(), &out); err != nil {
t.Fatal(err)
}
}
return w.Code, out
}
// An album not yet held shows MusicBrainz's standard edition: the shortest
// official one.
func TestWantTracks_StandardEditionFromMusicBrainz(t *testing.T) {
const group = "g0000000-0000-4000-8000-000000000010"
for _, down := range []bool{false, true} {
h, pool := testHandlers(t)
admin := seedUser(t, h.pool, "admin", "pw", true)
h.catalogue = catalogue.New(pool, &musicbrainz.Client{BaseURL: fakeEditions(t, group, down).URL,
Pace: &musicbrainz.Pacer{}, Retries: -1}, h.logger)
w := seedWant(t, h, group, "a0000000-0000-4000-8000-000000000001", "Camel", "Moonmadness", "1976")
code, out := wantTracks(t, h, admin, uuidToString(w.ID))
if down {
if code != http.StatusServiceUnavailable {
t.Errorf("MusicBrainz down: status %d, want 503", code)
}
continue
}
if code != http.StatusOK || out.Source != "musicbrainz" || len(out.Tracks) != 2 {
t.Fatalf("status %d, %+v: want the two-track original", code, out)
}
if out.Edition != "1976 · 12\" Vinyl · GB" {
t.Errorf("edition = %q", out.Edition)
}
if tr := out.Tracks[1]; tr.Disc != 1 || tr.Number != 2 || tr.Title != "Song Within a Song" || tr.LengthMs != 433000 {
t.Errorf("track 2 = %+v", tr)
}
}
}
// A held album shows the library's own tracks, a lost file marked.
func TestWantTracks_HeldAlbumShowsTheLibrarysTracks(t *testing.T) {
h, _ := testHandlers(t)
admin := seedUser(t, h.pool, "admin", "pw", true)
ctx := context.Background()
q := dbq.New(h.pool)
mbid := "a0000000-0000-4000-8000-000000000001"
artist, err := q.UpsertArtist(ctx, dbq.UpsertArtistParams{Name: "Camel", SortName: "Camel", Mbid: &mbid})
if err != nil {
t.Fatal(err)
}
album, err := q.UpsertAlbum(ctx, dbq.UpsertAlbumParams{Title: "Moonmadness", SortTitle: "Moonmadness", ArtistID: artist.ID})
if err != nil {
t.Fatal(err)
}
for i, title := range []string{"Aristillus", "Song Within a Song"} {
num := int32(i + 1)
if _, err := q.UpsertTrack(ctx, dbq.UpsertTrackParams{
Title: title, AlbumID: album.ID, ArtistID: artist.ID, TrackNumber: &num, DurationMs: 120000,
FilePath: fmt.Sprintf("/music/camel/%02d.flac", i+1), FileSize: 1, FileFormat: "flac",
}); err != nil {
t.Fatal(err)
}
}
if _, err := h.pool.Exec(ctx, `UPDATE tracks SET missing_since = now() WHERE title = 'Aristillus'`); err != nil {
t.Fatal(err)
}
w := seedWant(t, h, "g0000000-0000-4000-8000-000000000011", mbid, "Camel", "Moonmadness", "1976")
if err := q.FulfillWant(ctx, dbq.FulfillWantParams{ID: w.ID, AlbumID: album.ID}); err != nil {
t.Fatal(err)
}
code, out := wantTracks(t, h, admin, uuidToString(w.ID))
if code != http.StatusOK || out.Source != "library" || len(out.Tracks) != 2 {
t.Fatalf("status %d, %+v", code, out)
}
if !out.Tracks[0].Missing || out.Tracks[1].Missing || out.Tracks[1].Number != 2 {
t.Errorf("tracks = %+v: the first is missing, the second on disk", out.Tracks)
}
}
// The Unmatched section: the albums matching could not place, a
// local band's album left off and put back, and a link that names no
// MusicBrainz album refused before MusicBrainz is asked.
func TestUnmatched_ListLeaveOffPutBack(t *testing.T) {
h, _ := testHandlers(t)
admin := seedUser(t, h.pool, "admin", "pw", true)
ctx := context.Background()
q := dbq.New(h.pool)
artist, err := q.UpsertArtist(ctx, dbq.UpsertArtistParams{Name: "Annadale", SortName: "Annadale"})
if err != nil {
t.Fatal(err)
}
album, err := q.UpsertAlbum(ctx, dbq.UpsertAlbumParams{Title: "Dial Tone", SortTitle: "Dial Tone", ArtistID: artist.ID})
if err != nil {
t.Fatal(err)
}
if _, err := q.UpsertTrack(ctx, dbq.UpsertTrackParams{
Title: "Intro", AlbumID: album.ID, ArtistID: artist.ID, DurationMs: 120000,
FilePath: "/music/annadale/01.flac", FileSize: 1, FileFormat: "flac",
}); err != nil {
t.Fatal(err)
}
if err := q.RecordAlbumMatch(ctx, dbq.RecordAlbumMatchParams{
AlbumID: album.ID, Outcome: "unmatched", Detail: "MusicBrainz has no album by that name", Matcher: 1}); err != nil {
t.Fatal(err)
}
id := uuidToString(album.ID)
list := func() []unmatchedAlbumView {
t.Helper()
w := doWantedReq(t, h, "/api/admin/wanted/unmatched", admin)
if w.Code != http.StatusOK {
t.Fatalf("status = %d; body = %s", w.Code, w.Body.String())
}
var out struct {
Items []unmatchedAlbumView `json:"items"`
}
if err := json.Unmarshal(w.Body.Bytes(), &out); err != nil {
t.Fatal(err)
}
return out.Items
}
if got := list(); len(got) != 1 || got[0].AlbumID != id || got[0].Artist != "Annadale" || got[0].Tracks != 1 ||
got[0].LeftOff || got[0].Detail == "" || got[0].DecidedAt == "" {
t.Fatalf("unmatched = %+v", got)
}
for _, c := range []struct {
path, body string
code int
}{
{"/api/admin/wanted/unmatched/" + id + "/match", `{"ref":"Dial Tone"}`, http.StatusBadRequest},
{"/api/admin/wanted/unmatched/" + id + "/match", `nope`, http.StatusBadRequest},
{"/api/admin/wanted/unmatched/not-a-uuid/leave-off", "", http.StatusBadRequest},
{"/api/admin/wanted/unmatched/" + id + "/put-back", "", http.StatusNotFound},
{"/api/admin/wanted/unmatched/" + id + "/leave-off", "", http.StatusNoContent},
{"/api/admin/wanted/unmatched/" + id + "/leave-off", "", http.StatusNotFound},
} {
if w := doWanted(t, h, http.MethodPost, c.path, c.body, admin); w.Code != c.code {
t.Errorf("POST %s %s = %d, want %d; body = %s", c.path, c.body, w.Code, c.code, w.Body.String())
}
}
if got := list(); len(got) != 1 || !got[0].LeftOff {
t.Fatalf("left off = %+v", got)
}
if w := doWanted(t, h, http.MethodPost, "/api/admin/wanted/unmatched/"+id+"/put-back", "", admin); w.Code != http.StatusNoContent {
t.Fatalf("put back = %d; body = %s", w.Code, w.Body.String())
}
if got := list(); len(got) != 1 || got[0].LeftOff {
t.Errorf("put back = %+v", got)
}
user := seedUser(t, h.pool, "listener", "pw", false)
if w := doWantedReq(t, h, "/api/admin/wanted/unmatched", user); w.Code != http.StatusForbidden {
t.Errorf("a listener reads the unmatched section: %d", w.Code)
}
}
+1 -136
View File
@@ -13,35 +13,28 @@ import (
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/acquire"
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
"git.fabledsword.com/bvandeusen/minstrel/internal/backup"
"git.fabledsword.com/bvandeusen/minstrel/internal/catalogue"
"git.fabledsword.com/bvandeusen/minstrel/internal/config"
"git.fabledsword.com/bvandeusen/minstrel/internal/coverart"
"git.fabledsword.com/bvandeusen/minstrel/internal/eventbus"
"git.fabledsword.com/bvandeusen/minstrel/internal/importer"
"git.fabledsword.com/bvandeusen/minstrel/internal/library"
"git.fabledsword.com/bvandeusen/minstrel/internal/lifecycle"
"git.fabledsword.com/bvandeusen/minstrel/internal/mailer"
"git.fabledsword.com/bvandeusen/minstrel/internal/netsettings"
"git.fabledsword.com/bvandeusen/minstrel/internal/notifications"
"git.fabledsword.com/bvandeusen/minstrel/internal/playevents"
"git.fabledsword.com/bvandeusen/minstrel/internal/playlists"
"git.fabledsword.com/bvandeusen/minstrel/internal/quarantine"
"git.fabledsword.com/bvandeusen/minstrel/internal/reacquisition"
"git.fabledsword.com/bvandeusen/minstrel/internal/recommendation"
"git.fabledsword.com/bvandeusen/minstrel/internal/recsettings"
"git.fabledsword.com/bvandeusen/minstrel/internal/requests"
"git.fabledsword.com/bvandeusen/minstrel/internal/tags"
"git.fabledsword.com/bvandeusen/minstrel/internal/tracks"
"git.fabledsword.com/bvandeusen/minstrel/internal/wanted"
)
// Mount attaches /api/* handlers to r. Public endpoints (login) are outside
// RequireUser; everything else is gated by the middleware. The events writer
// is shared with the Subsonic mount so /rest/scrobble feeds the same store.
func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playevents.Writer, recCfg config.RecommendationConfig, recSettings *recsettings.Service, requestSvc *requests.Service, quarantineSvc *quarantine.Service, tracksSvc *tracks.Service, playlistsSvc *playlists.Service, coverEnricher *coverart.Enricher, coverSettings *coverart.SettingsService, tagSettings *tags.SettingsService, scanner *library.Scanner, scanCfg library.RunScanConfig, dataDir string, sender mailer.Sender, bus *eventbus.Bus, playlistScheduler *playlists.Scheduler, streamSecret []byte, netSettings *netsettings.Service, reacqSettings *reacquisition.SettingsService, fpSettings *library.FingerprintSettingsService, loudSettings *library.LoudnessSettingsService, acoustIDLookup *library.AcoustIDLookupWorker, backups *backup.Service, restart func()) {
func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playevents.Writer, recCfg config.RecommendationConfig, recSettings *recsettings.Service, quarantineSvc *quarantine.Service, tracksSvc *tracks.Service, playlistsSvc *playlists.Service, coverEnricher *coverart.Enricher, coverSettings *coverart.SettingsService, tagSettings *tags.SettingsService, scanner *library.Scanner, scanCfg library.RunScanConfig, dataDir string, sender mailer.Sender, bus *eventbus.Bus, playlistScheduler *playlists.Scheduler, streamSecret []byte, netSettings *netsettings.Service, fpSettings *library.FingerprintSettingsService, loudSettings *library.LoudnessSettingsService, acoustIDLookup *library.AcoustIDLookupWorker, backups *backup.Service, restart func()) {
rng := rand.New(rand.NewSource(rand.Int63()))
setupToken, err := auth.NewSetupToken()
if err != nil {
@@ -50,12 +43,10 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
panic("api: mint setup token: " + err.Error())
}
logSetupTokenIfNeeded(pool, logger, setupToken)
cat := catalogue.New(pool, nil, logger)
h := &handlers{
pool: pool, logger: logger, events: events, recCfg: recCfg,
recSettings: recSettings,
rng: rng.Float64,
requests: requestSvc,
quarantine: quarantineSvc,
tracks: tracksSvc,
playlists: playlistsSvc,
@@ -71,20 +62,11 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
playlistScheduler: playlistScheduler,
streamSecret: streamSecret,
netSettings: netSettings,
reacqSettings: reacqSettings,
fingerprintSettings: fpSettings,
loudnessSettings: loudSettings,
acoustIDLookup: acoustIDLookup,
backups: backups,
restart: restart,
acquisition: acquire.NewStore(pool),
catalogue: cat,
wanted: wanted.New(pool, cat, logger),
searcher: wanted.NewSearcher(pool, cat, logger),
importer: importer.New(pool, scanner, cat, nil, scannerRoots(scanner), logger),
lifecycle: lifecycle.New(pool, logger),
catalogueSearches: newSearchCache(),
catalogueArt: catalogue.NewArt(dataDir, artSources(coverSettings)),
leveled: newLeveledRenderer(dataDir, loudSettings, logger),
librarySize: recommendation.NewLibrarySize(nil),
loginGuard: auth.NewLoginGuard(),
@@ -170,12 +152,6 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
// it can accept either a session or a signed token.
authed.Get("/search", h.handleSearch)
authed.Get("/radio", h.handleRadio)
authed.Get("/discover/suggestions", h.handleListSuggestions)
// Snooze = "not right now", time-boxed and self-expiring.
// Not a dislike — see the migration for why.
authed.Post("/discover/suggestions/{mbid}/snooze", h.handleSnoozeSuggestion)
authed.Delete("/discover/suggestions/{mbid}/snooze", h.handleUnsnoozeSuggestion)
authed.Get("/discover/snoozes", h.handleListSuggestionSnoozes)
authed.Get("/home", h.handleGetHome)
authed.Get("/home/index", h.handleGetHomeIndex)
authed.Post("/events", h.handleEvents)
@@ -194,17 +170,6 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
authed.Get("/likes/artists", h.handleListLikedArtists)
authed.Get("/likes/ids", h.handleGetLikedIDs)
// The release catalogue: request search asks
// MusicBrainz, and art for what isn't held comes through
// Minstrel from the enabled cover-art sources.
authed.Get("/catalogue/search", h.handleCatalogueSearch)
authed.Get("/catalogue/art/{kind}/{mbid}", h.handleCatalogueArt)
authed.Post("/requests", h.handleCreateRequest)
authed.Get("/requests", h.handleListRequests)
authed.Get("/requests/{id}", h.handleGetRequest)
authed.Delete("/requests/{id}", h.handleCancelRequest)
authed.Post("/quarantine", h.handleFlag)
authed.Delete("/quarantine/{track_id}", h.handleUnflag)
authed.Get("/quarantine/mine", h.handleListMyQuarantine)
@@ -227,57 +192,10 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
authed.Route("/admin", func(admin chi.Router) {
admin.Use(auth.RequireAdmin())
admin.Get("/acquisition/services", h.handleListAcquisitionServices)
admin.Post("/acquisition/services", h.handleCreateAcquisitionService)
admin.Put("/acquisition/services/{code}", h.handleUpdateAcquisitionService)
admin.Delete("/acquisition/services/{code}", h.handleDeleteAcquisitionService)
admin.Post("/acquisition/services/{code}/test", h.handleTestAcquisitionService)
admin.Put("/acquisition/services/{code}/api-key", h.handleSetAcquisitionSecret("api_key"))
admin.Delete("/acquisition/services/{code}/api-key", h.handleSetAcquisitionSecret("api_key"))
admin.Put("/acquisition/services/{code}/password", h.handleSetAcquisitionSecret("password"))
admin.Delete("/acquisition/services/{code}/password", h.handleSetAcquisitionSecret("password"))
// The queues that wait on an admin, for the rail and the
// Overview.
admin.Get("/summary", h.handleGetAdminSummary)
admin.Get("/status", h.handleGetAdminStatus)
admin.Get("/requests", h.handleListAdminRequests)
admin.Post("/requests/{id}/approve", h.handleApproveRequest)
admin.Post("/requests/{id}/reject", h.handleRejectRequest)
// The wanted list: what Minstrel is looking
// for, the artists it follows, and the release-type
// profiles that decide which of their releases count.
admin.Get("/wanted", h.handleListWants)
admin.Get("/wanted/artists", h.handleListWantArtists)
admin.Get("/wanted/unmatched", h.handleListUnmatched)
admin.Post("/wanted/unmatched/{id}/match", h.handleMatchUnmatched)
admin.Post("/wanted/unmatched/{id}/leave-off", h.handleLeaveUnmatchedOff)
admin.Post("/wanted/unmatched/{id}/put-back", h.handlePutUnmatchedBack)
admin.Delete("/wanted/{id}", h.handleRemoveWant)
// Searching and grabbing.
admin.Get("/acquisition/settings", h.handleGetSearchSettings)
admin.Put("/acquisition/settings", h.handlePutSearchSettings)
admin.Get("/wanted/{id}/candidates", h.handleListCandidates)
admin.Get("/wanted/{id}/tracks", h.handleWantTracks)
admin.Post("/wanted/{id}/search", h.handleSearchWant)
admin.Post("/wanted/{id}/grab", h.handleGrabCandidate)
admin.Post("/wanted/{id}/block", h.handleBlockCandidate)
admin.Delete("/release-blocks/{hash}", h.handleUnblockRelease)
admin.Get("/release-blocks", h.handleListReleaseBlocks)
admin.Get("/downloads", h.handleListDownloads)
admin.Post("/downloads/{id}/import", h.handleImportAnyway)
admin.Post("/downloads/{id}/retry", h.handleRetryImport)
admin.Post("/downloads/{id}/refuse", h.handleRefuseDownload)
admin.Post("/downloads/{id}/abandon", h.handleAbandonDownload)
admin.Get("/follows", h.handleListFollows)
admin.Get("/follows/{mbid}", h.handleGetFollow)
admin.Put("/follows/{mbid}", h.handlePutFollow)
admin.Delete("/follows/{mbid}", h.handleDeleteFollow)
admin.Get("/release-profiles", h.handleListReleaseProfiles)
admin.Post("/release-profiles", h.handleCreateReleaseProfile)
admin.Put("/release-profiles/{id}", h.handleUpdateReleaseProfile)
admin.Delete("/release-profiles/{id}", h.handleDeleteReleaseProfile)
admin.Get("/quarantine", h.handleListAdminQuarantine)
admin.Post("/quarantine/{track_id}/resolve", h.handleResolveQuarantine)
@@ -294,12 +212,6 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
admin.Get("/network-settings", h.handleGetNetworkSettings)
admin.Put("/network-settings", h.handleUpdateNetworkSettings)
// Policy for wanting back an album whose files went missing
// Sits beside the missing-files list it governs, because an
// admin meets it on that surface.
admin.Get("/library/reacquisition", h.handleGetReacquisitionSettings)
admin.Put("/library/reacquisition", h.handleUpdateReacquisitionSettings)
// Where the library lives. A changed folder moves its
// tracks rather than re-adding them.
admin.Get("/library/folders", h.handleGetLibraryFolders)
@@ -345,7 +257,6 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
admin.Post("/users", h.handleAdminCreateUser)
admin.Delete("/users/{id}", h.handleAdminDeleteUser)
admin.Post("/users/{id}/reset-password", h.handleAdminResetPassword)
admin.Put("/users/{id}/auto-approve", h.handleAdminAutoApproveToggle)
admin.Put("/users/{id}/debug-mode", h.handleAdminDebugModeToggle)
// Device diagnostics timeline + device overview.
@@ -408,7 +319,6 @@ type handlers struct {
// Held here rather than counted per request: the count is a
// full table scan, and library size only moves when a scan runs.
librarySize *recommendation.LibrarySize
requests *requests.Service
quarantine *quarantine.Service
tracks *tracks.Service
playlists *playlists.Service
@@ -424,10 +334,6 @@ type handlers struct {
// notifier records nothing, which is what most handler tests want.
notifier *notifications.Notifier
playlistScheduler *playlists.Scheduler
// reacqSettings is the DB-backed policy for auto re-acquisition of
// missing files — grace window, backoff, attempt caps.
// Cached in the service, so the admin card reads it without a query.
reacqSettings *reacquisition.SettingsService
// fingerprintSettings is the fingerprinting policy, the same
// instance the scanner and the fingerprint workers read, so a save from the
// admin card reaches them without a restart. Nil serves the defaults.
@@ -443,26 +349,6 @@ type handlers struct {
// restart stops the server and starts it again in place, so a restored
// backup's settings are read at boot. Nil in tests.
restart func()
// acquisition is the configured indexers and download clients.
acquisition *acquire.Store
// catalogue is the release catalogue read from MusicBrainz, and
// catalogueArt the images for what isn't held.
catalogue *catalogue.Service
catalogueArt *catalogue.Art
// wanted is the wanted list: follows, release-type profiles and wants.
// Its worker runs in main; the API reads and edits.
wanted *wanted.Service
// searcher searches for wants and grabs releases for them.
searcher *wanted.Searcher
// importer refuses a download at a person's word. The import
// loop itself runs in main; this one only acts on a request.
importer *importer.Importer
// lifecycle gives up on a download at a person's word; its loop
// runs in main.
lifecycle *lifecycle.Service
// catalogueSearches reuses recent MusicBrainz answers; nil asks every
// time, which is what handler tests get.
catalogueSearches *searchCache
// leveled renders the leveled streams handed to Sonos/UPnP speakers. Nil
// when its cache directory cannot be made: a level request then gets the
// plain stream.
@@ -502,24 +388,3 @@ func newLeveledRenderer(dataDir string, settings *library.LoudnessSettingsServic
}
return r
}
// artSources hands the art cache the cover-art settings, or nothing when
// there are none: a nil *SettingsService inside the interface would not
// compare equal to nil and would be called.
func artSources(s *coverart.SettingsService) catalogue.ArtSources {
if s == nil {
return nil
}
return s
}
// scannerRoots reads the library folders, or none when the API runs without
// a scanner (tests).
func scannerRoots(s *library.Scanner) func() []string {
return func() []string {
if s == nil {
return nil
}
return s.Roots()
}
}
+7 -6
View File
@@ -18,7 +18,6 @@ import (
"golang.org/x/crypto/bcrypt"
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
"git.fabledsword.com/bvandeusen/minstrel/internal/catalogue"
"git.fabledsword.com/bvandeusen/minstrel/internal/config"
"git.fabledsword.com/bvandeusen/minstrel/internal/db"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
@@ -30,9 +29,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/playlists"
"git.fabledsword.com/bvandeusen/minstrel/internal/quarantine"
"git.fabledsword.com/bvandeusen/minstrel/internal/recsettings"
"git.fabledsword.com/bvandeusen/minstrel/internal/requests"
"git.fabledsword.com/bvandeusen/minstrel/internal/tracks"
"git.fabledsword.com/bvandeusen/minstrel/internal/wanted"
)
// testHandlers spins up a handlers instance against MINSTREL_TEST_DATABASE_URL.
@@ -65,13 +62,11 @@ func testHandlers(t *testing.T) (*handlers, *pgxpool.Pool) {
if err != nil {
t.Fatalf("recsettings: %v", err)
}
requestSvc := requests.NewService(pool)
cat := catalogue.New(pool, nil, logger)
quarantineSvc := quarantine.NewService(pool, "")
dataDir := t.TempDir()
tracksSvc := tracks.NewService(pool, logger, dataDir)
playlistsSvc := playlists.NewService(pool, logger, dataDir)
h := &handlers{pool: pool, logger: logger, events: w, recCfg: recCfg, recSettings: recSettings, rng: func() float64 { return 0.5 }, requests: requestSvc, catalogue: cat, wanted: wanted.New(pool, cat, logger), quarantine: quarantineSvc, tracks: tracksSvc, playlists: playlistsSvc, dataDir: dataDir, scanner: nil, scanCfg: library.RunScanConfig{}, mailer: &mailer.FakeSender{}, notifier: notifications.New(pool, nil, nil)}
h := &handlers{pool: pool, logger: logger, events: w, recCfg: recCfg, recSettings: recSettings, rng: func() float64 { return 0.5 }, quarantine: quarantineSvc, tracks: tracksSvc, playlists: playlistsSvc, dataDir: dataDir, scanner: nil, scanCfg: library.RunScanConfig{}, mailer: &mailer.FakeSender{}, notifier: notifications.New(pool, nil, nil)}
return h, pool
}
@@ -223,6 +218,12 @@ func TestHandleLogin_MalformedBodyReturns400(t *testing.T) {
func userCtxKeyForTest() any { return auth.UserCtxKeyForTest() }
// withUser injects a user into the request context the same way RequireUser
// does.
func withUser(req *http.Request, user dbq.User) *http.Request {
return req.WithContext(context.WithValue(req.Context(), userCtxKeyForTest(), user))
}
func TestHandleLogout_DeletesSessionAndClearsCookie(t *testing.T) {
h, pool := testHandlers(t)
user := seedUser(t, pool, "alice", "hunter2", false)
-284
View File
@@ -1,284 +0,0 @@
package api
import (
"context"
"errors"
"net/http"
"net/url"
"slices"
"strings"
"sync"
"time"
"github.com/go-chi/chi/v5"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/catalogue"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/musicbrainz"
)
// catalogueSearchLimit is how many results one search shows.
const catalogueSearchLimit = 25
// catalogueSearchResult is one row of GET /api/catalogue/search, read by the
// Discover page and request rows. artist_mbid and album_mbid are the parents a
// request needs: empty on an artist, the artist on an album, both on a track.
// An album is a MusicBrainz release group.
type catalogueSearchResult struct {
MBID string `json:"mbid"`
Name string `json:"name"`
SecondaryText string `json:"secondary_text"`
ImageURL string `json:"image_url"`
ArtistMBID string `json:"artist_mbid"`
AlbumMBID string `json:"album_mbid"`
InLibrary bool `json:"in_library"`
Requested bool `json:"requested"`
}
// handleCatalogueSearch implements GET
// /api/catalogue/search?q=&kind=artist|album|track asking MusicBrainz directly.
// An unreachable MusicBrainz is a 503 the page shows as such, not an empty
// result.
func (h *handlers) handleCatalogueSearch(w http.ResponseWriter, r *http.Request) {
kind := strings.TrimSpace(r.URL.Query().Get("kind"))
switch kind {
case "artist", "album", "track":
default:
writeErr(w, apierror.BadRequest("bad_kind", "kind must be artist, album, or track"))
return
}
q := strings.TrimSpace(r.URL.Query().Get("q"))
if q == "" {
writeErr(w, apierror.BadRequest("missing_query", "q is required"))
return
}
out, err := h.catalogueSearches.get(r.Context(), kind, q, h.searchCatalogue)
if err != nil {
switch {
case errors.Is(err, musicbrainz.ErrUnreachable), errors.Is(err, musicbrainz.ErrBadResponse):
writeErr(w, &apierror.Error{Status: 503, Code: "musicbrainz_unreachable", Message: "MusicBrainz can't be reached", Cause: err})
case r.Context().Err() != nil:
return
default:
h.logger.Error("api: catalogue search", "kind", kind, "err", err)
writeErr(w, apierror.InternalMsg("search failed", err))
}
return
}
dbQ := dbq.New(h.pool)
for i := range out {
inLib, err := h.catalogueInLibrary(r.Context(), kind, out[i].MBID)
if err != nil {
writeErr(w, apierror.InternalMsg("library check failed", err))
return
}
requested, err := dbQ.HasNonTerminalRequestForMBID(r.Context(), out[i].MBID)
if err != nil {
writeErr(w, apierror.InternalMsg("request check failed", err))
return
}
out[i].InLibrary, out[i].Requested = inLib, requested
}
writeJSON(w, http.StatusOK, out)
}
func (h *handlers) searchCatalogue(ctx context.Context, kind, q string) ([]catalogueSearchResult, error) {
mb := h.catalogue.MusicBrainz()
out := []catalogueSearchResult{}
switch kind {
case "artist":
artists, err := mb.SearchArtists(ctx, q, catalogueSearchLimit)
if err != nil {
return nil, err
}
for _, a := range artists {
out = append(out, catalogueSearchResult{
MBID: a.MBID, Name: a.Name, SecondaryText: artistLine(a),
ImageURL: artistArtURL(a.MBID, a.Name),
})
}
case "album":
groups, err := mb.SearchReleaseGroups(ctx, q, catalogueSearchLimit)
if err != nil {
return nil, err
}
for _, g := range groups {
out = append(out, catalogueSearchResult{
MBID: g.MBID, Name: g.Title, SecondaryText: joinNonEmpty(g.ArtistCredit, g.Year(), groupType(g)),
ImageURL: groupArtURL(g), ArtistMBID: first(g.ArtistMBIDs),
})
}
case "track":
recs, err := mb.SearchRecordings(ctx, q, catalogueSearchLimit)
if err != nil {
return nil, err
}
for _, rec := range recs {
res := catalogueSearchResult{
MBID: rec.MBID, Name: rec.Title,
SecondaryText: joinNonEmpty(rec.ReleaseGroup.Title, rec.ArtistCredit),
ArtistMBID: first(rec.ArtistMBIDs), AlbumMBID: rec.ReleaseGroup.MBID,
}
if rec.ReleaseGroup.MBID != "" {
res.ImageURL = groupArtURL(rec.ReleaseGroup)
}
out = append(out, res)
}
}
return out, nil
}
// catalogueSearchTTL is how long a search's MusicBrainz answer is reused.
// MusicBrainz allows a request a second, and Discover searches as the user
// types, so the same words are often asked again within moments.
const catalogueSearchTTL = 10 * time.Minute
// catalogueSearchCacheMax bounds the cache; past it, the cache starts over.
const catalogueSearchCacheMax = 500
// searchCache keeps recent MusicBrainz answers by kind and query. A nil
// cache asks every time. Whether a result is held or requested is not
// cached: the handler works that out on every request.
type searchCache struct {
mu sync.Mutex
entries map[string]searchEntry
now func() time.Time
}
type searchEntry struct {
at time.Time
results []catalogueSearchResult
}
func newSearchCache() *searchCache {
return &searchCache{entries: map[string]searchEntry{}, now: time.Now}
}
func (c *searchCache) get(ctx context.Context, kind, q string,
ask func(context.Context, string, string) ([]catalogueSearchResult, error),
) ([]catalogueSearchResult, error) {
if c == nil {
return ask(ctx, kind, q)
}
key := kind + "\x00" + strings.ToLower(q)
c.mu.Lock()
e, ok := c.entries[key]
c.mu.Unlock()
if ok && c.now().Sub(e.at) < catalogueSearchTTL {
return slices.Clone(e.results), nil
}
results, err := ask(ctx, kind, q)
if err != nil {
return nil, err
}
c.mu.Lock()
if len(c.entries) >= catalogueSearchCacheMax {
c.entries = map[string]searchEntry{}
}
c.entries[key] = searchEntry{at: c.now(), results: slices.Clone(results)}
c.mu.Unlock()
return results, nil
}
// artistLine is what tells two artists of one name apart: MusicBrainz's own
// disambiguation when it has one, else the kind of artist and the country.
func artistLine(a musicbrainz.Artist) string {
if a.Disambiguation != "" {
return a.Disambiguation
}
return joinNonEmpty(a.Type, a.Country)
}
// groupType names an album's kind the way people do: "Album", "EP",
// "Album · Live", "Single · Remix".
func groupType(g musicbrainz.ReleaseGroup) string {
return joinNonEmpty(append([]string{g.PrimaryType}, g.SecondaryTypes...)...)
}
func joinNonEmpty(parts ...string) string {
kept := parts[:0:0]
for _, p := range parts {
if p != "" {
kept = append(kept, p)
}
}
return strings.Join(kept, " · ")
}
func first(s []string) string {
if len(s) == 0 {
return ""
}
return s[0]
}
// artistArtURL and groupArtURL point at Minstrel's own art endpoints. The
// names ride along for the sources that look art up by name.
func artistArtURL(mbid, name string) string {
if mbid == "" {
return ""
}
return "/api/catalogue/art/artist/" + mbid + "?" + url.Values{"name": {name}}.Encode()
}
func groupArtURL(g musicbrainz.ReleaseGroup) string {
return "/api/catalogue/art/release-group/" + g.MBID + "?" +
url.Values{"artist": {g.ArtistCredit}, "title": {g.Title}}.Encode()
}
// catalogueInLibrary reports whether a result is already held: an artist by
// its MBID, an album by its release group (or a release id that happens to be
// the same), a track by its recording.
func (h *handlers) catalogueInLibrary(ctx context.Context, kind, mbid string) (bool, error) {
var query string
switch kind {
case "artist":
query = `SELECT EXISTS(SELECT 1 FROM artists WHERE mbid = $1)`
case "album":
query = `SELECT EXISTS(SELECT 1 FROM albums WHERE release_group_mbid = $1 OR mbid = $1)`
case "track":
query = `SELECT EXISTS(SELECT 1 FROM tracks WHERE mbid = $1)`
default:
return false, nil
}
var exists bool
err := h.pool.QueryRow(ctx, query, mbid).Scan(&exists)
return exists, err
}
// handleCatalogueArt implements GET /api/catalogue/art/{kind}/{mbid} for an
// artist or a release group Minstrel doesn't hold. The image comes from the
// enabled cover-art sources and is kept on disk, so the browser
// never fetches a third-party host. A 404 shows the placeholder.
func (h *handlers) handleCatalogueArt(w http.ResponseWriter, r *http.Request) {
kind, mbid := chi.URLParam(r, "kind"), chi.URLParam(r, "mbid")
if !catalogue.ValidMBID(mbid) {
writeErr(w, apierror.NotFound("art"))
return
}
var (
body []byte
err error
)
switch kind {
case "artist":
body, err = h.catalogueArt.Artist(r.Context(), mbid, r.URL.Query().Get("name"))
case "release-group":
body, err = h.catalogueArt.ReleaseGroup(r.Context(), mbid, r.URL.Query().Get("artist"), r.URL.Query().Get("title"))
default:
writeErr(w, apierror.NotFound("art"))
return
}
if err != nil {
if r.Context().Err() == nil {
writeErr(w, apierror.NotFound("art"))
}
return
}
w.Header().Set("Content-Type", http.DetectContentType(body))
w.Header().Set("Cache-Control", "private, max-age=86400")
_, _ = w.Write(body)
}
-254
View File
@@ -1,254 +0,0 @@
package api
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"net/url"
"testing"
"time"
"github.com/go-chi/chi/v5"
"git.fabledsword.com/bvandeusen/minstrel/internal/catalogue"
"git.fabledsword.com/bvandeusen/minstrel/internal/coverart"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/dbtest"
"git.fabledsword.com/bvandeusen/minstrel/internal/musicbrainz"
)
const (
searchArtistMBID = "a74b1b7f-71a5-4011-9441-d0b5e4122711"
searchGroupMBID = "b1392450-e666-3926-a536-22c65f834433"
searchRecMBID = "c1a0e1b2-0000-4000-8000-000000000001"
)
// fakeMusicBrainz answers the three searches with one Radiohead result each,
// or 503 when down is set.
func fakeMusicBrainz(t *testing.T, down bool) *httptest.Server {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if down {
w.WriteHeader(http.StatusServiceUnavailable)
return
}
credit := `[{"name": "Radiohead", "artist": {"id": "` + searchArtistMBID + `"}}]`
group := `{"id": "` + searchGroupMBID + `", "title": "OK Computer", "primary-type": "Album",
"first-release-date": "1997-05-21", "artist-credit": ` + credit + `}`
switch r.URL.Path {
case "/artist":
_, _ = w.Write([]byte(`{"artists": [{"id": "` + searchArtistMBID + `", "name": "Radiohead", "type": "Group", "country": "GB"}]}`))
case "/release-group":
_, _ = w.Write([]byte(`{"release-groups": [` + group + `]}`))
case "/recording":
_, _ = w.Write([]byte(`{"recordings": [{"id": "` + searchRecMBID + `", "title": "Karma Police",
"artist-credit": ` + credit + `,
"releases": [{"id": "r1", "status": "Official", "release-group": ` + group + `}]}]}`))
default:
w.WriteHeader(http.StatusNotFound)
}
}))
t.Cleanup(srv.Close)
return srv
}
func catalogueHandlers(t *testing.T, down bool) *handlers {
t.Helper()
h, pool := testHandlers(t)
resetRequests(t, h)
mb := &musicbrainz.Client{BaseURL: fakeMusicBrainz(t, down).URL, Pace: &musicbrainz.Pacer{}, Retries: -1}
h.catalogue = catalogue.New(pool, mb, h.logger)
return h
}
func doCatalogueSearch(t *testing.T, h *handlers, q, kind string) *httptest.ResponseRecorder {
t.Helper()
req := httptest.NewRequest(http.MethodGet, "/api/catalogue/search?"+url.Values{"q": {q}, "kind": {kind}}.Encode(), nil)
w := httptest.NewRecorder()
h.handleCatalogueSearch(w, req)
return w
}
func decodeResults(t *testing.T, w *httptest.ResponseRecorder) []catalogueSearchResult {
t.Helper()
if w.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", w.Code, w.Body.String())
}
var out []catalogueSearchResult
if err := json.Unmarshal(w.Body.Bytes(), &out); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
return out
}
func errorCode(t *testing.T, w *httptest.ResponseRecorder) string {
t.Helper()
var resp struct {
Error struct {
Code string `json:"code"`
} `json:"error"`
}
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
return resp.Error.Code
}
func TestCatalogueSearch_RefusesABadKindOrNoQuery(t *testing.T) {
h := catalogueHandlers(t, false)
if w := doCatalogueSearch(t, h, "x", "playlist"); w.Code != http.StatusBadRequest || errorCode(t, w) != "bad_kind" {
t.Errorf("bad kind: %d %s", w.Code, w.Body.String())
}
if w := doCatalogueSearch(t, h, " ", "artist"); w.Code != http.StatusBadRequest || errorCode(t, w) != "missing_query" {
t.Errorf("no query: %d %s", w.Code, w.Body.String())
}
}
func TestCatalogueSearch_EachKindAsksMusicBrainz(t *testing.T) {
h := catalogueHandlers(t, false)
artists := decodeResults(t, doCatalogueSearch(t, h, "radiohead", "artist"))
if len(artists) != 1 || artists[0].MBID != searchArtistMBID || artists[0].SecondaryText != "Group · GB" {
t.Fatalf("artists = %+v", artists)
}
if artists[0].ImageURL != "/api/catalogue/art/artist/"+searchArtistMBID+"?name=Radiohead" {
t.Errorf("artist art = %q: it must be Minstrel's own endpoint", artists[0].ImageURL)
}
albums := decodeResults(t, doCatalogueSearch(t, h, "ok computer", "album"))
if len(albums) != 1 || albums[0].MBID != searchGroupMBID || albums[0].ArtistMBID != searchArtistMBID ||
albums[0].SecondaryText != "Radiohead · 1997 · Album" {
t.Fatalf("albums = %+v", albums)
}
tracks := decodeResults(t, doCatalogueSearch(t, h, "karma police", "track"))
if len(tracks) != 1 || tracks[0].AlbumMBID != searchGroupMBID || tracks[0].ArtistMBID != searchArtistMBID ||
tracks[0].SecondaryText != "OK Computer · Radiohead" {
t.Fatalf("tracks = %+v", tracks)
}
}
func TestCatalogueSearch_MarksWhatIsHeldAndWhatIsRequested(t *testing.T) {
h := catalogueHandlers(t, false)
ctx := context.Background()
q := dbq.New(h.pool)
artistMBID := searchArtistMBID
artist, err := q.UpsertArtist(ctx, dbq.UpsertArtistParams{Name: dbtest.TestUserPrefix + "Radiohead", SortName: "Radiohead", Mbid: &artistMBID})
if err != nil {
t.Fatal(err)
}
// The album is held under a release id; its release group is what matches.
if _, err := h.pool.Exec(ctx,
`INSERT INTO albums (title, sort_title, artist_id, mbid, release_group_mbid) VALUES ('OK Computer', 'OK Computer', $1, 'some-release-id', $2)`,
artist.ID, searchGroupMBID); err != nil {
t.Fatal(err)
}
if got := decodeResults(t, doCatalogueSearch(t, h, "radiohead", "artist")); !got[0].InLibrary {
t.Error("the artist is held")
}
if got := decodeResults(t, doCatalogueSearch(t, h, "ok computer", "album")); !got[0].InLibrary {
t.Error("the album is held, matched by release group")
}
user := seedUser(t, h.pool, "requester", "pw", false)
track := searchRecMBID
if _, err := q.CreateRequest(ctx, dbq.CreateRequestParams{
UserID: user.ID, Kind: dbq.RequestKindTrack, ArtistMbid: searchArtistMBID,
TrackMbid: &track, ArtistName: "Radiohead",
}); err != nil {
t.Fatal(err)
}
got := decodeResults(t, doCatalogueSearch(t, h, "karma police", "track"))
if got[0].InLibrary || !got[0].Requested {
t.Errorf("track = %+v, want requested and not held", got[0])
}
}
func TestCatalogueSearch_UnreachableMusicBrainzIs503(t *testing.T) {
h := catalogueHandlers(t, true)
w := doCatalogueSearch(t, h, "radiohead", "artist")
if w.Code != http.StatusServiceUnavailable || errorCode(t, w) != "musicbrainz_unreachable" {
t.Fatalf("status %d, body %s", w.Code, w.Body.String())
}
}
// artSource is an artist-art source with one image.
type artSource struct{ apiTestAlbumProvider }
func (artSource) FetchArtistArt(_ context.Context, _ coverart.ArtistRef) ([]byte, []byte, error) {
return []byte("\x89PNG\r\n\x1a\nrest"), nil, nil
}
type fakeArtSources struct{ artists []coverart.ArtistArtProvider }
func (s fakeArtSources) EnabledArtistProviders() []coverart.ArtistArtProvider { return s.artists }
func (fakeArtSources) EnabledAlbumProviders() []coverart.AlbumCoverProvider { return nil }
func TestCatalogueArt_ServesTheImageOrAPlaceholder404(t *testing.T) {
h := &handlers{catalogueArt: catalogue.NewArt(t.TempDir(), fakeArtSources{artists: []coverart.ArtistArtProvider{&artSource{}}})}
r := chi.NewRouter()
r.Get("/api/catalogue/art/{kind}/{mbid}", h.handleCatalogueArt)
get := func(path string) *httptest.ResponseRecorder {
w := httptest.NewRecorder()
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, path, nil))
return w
}
if w := get("/api/catalogue/art/artist/" + searchArtistMBID + "?name=Radiohead"); w.Code != http.StatusOK || w.Header().Get("Content-Type") != "image/png" {
t.Errorf("artist: %d %q", w.Code, w.Header().Get("Content-Type"))
}
// No album source is enabled: a 404, which the page shows as its
// placeholder.
if w := get("/api/catalogue/art/release-group/" + searchGroupMBID); w.Code != http.StatusNotFound {
t.Errorf("release group with no source: %d", w.Code)
}
for _, bad := range []string{"/api/catalogue/art/artist/not-an-id", "/api/catalogue/art/label/" + searchArtistMBID} {
if w := get(bad); w.Code != http.StatusNotFound {
t.Errorf("%s: %d", bad, w.Code)
}
}
}
func TestSuggestionArt_PointsAtMinstrelsOwnEndpoint(t *testing.T) {
views := []suggestionView{{MBID: searchArtistMBID, Name: "Sigur Rós"}, {Name: "No MBID"}}
resolveSuggestionArt(views)
if views[0].ImageURL != "/api/catalogue/art/artist/"+searchArtistMBID+"?name=Sigur+R%C3%B3s" {
t.Errorf("image_url = %q", views[0].ImageURL)
}
if views[1].ImageURL != "" {
t.Errorf("no MBID, no image: %q", views[1].ImageURL)
}
}
func TestSearchCache_ReusesAnAnswerButNotTheFlags(t *testing.T) {
c := newSearchCache()
now := time.Now()
c.now = func() time.Time { return now }
asks := 0
ask := func(context.Context, string, string) ([]catalogueSearchResult, error) {
asks++
return []catalogueSearchResult{{MBID: "m"}}, nil
}
first, _ := c.get(context.Background(), "artist", "Radiohead", ask)
first[0].InLibrary = true // the handler sets flags on what it is given
again, _ := c.get(context.Background(), "artist", "radiohead", ask)
if asks != 1 {
t.Errorf("asks = %d: the same words in another case are the same search", asks)
}
if again[0].InLibrary {
t.Error("a flag set on one answer leaked into the cache")
}
if _, _ = c.get(context.Background(), "album", "radiohead", ask); asks != 2 {
t.Error("another kind is another search")
}
now = now.Add(catalogueSearchTTL + time.Second)
if _, _ = c.get(context.Background(), "artist", "radiohead", ask); asks != 3 {
t.Error("an old answer is asked again")
}
var none *searchCache
if _, _ = none.get(context.Background(), "artist", "x", ask); asks != 4 {
t.Error("a nil cache asks every time")
}
}
+8
View File
@@ -194,3 +194,11 @@ func nonNilStrings(in []string) []string {
}
return in
}
// deref is a nullable column's text, or "".
func deref(s *string) string {
if s == nil {
return ""
}
return *s
}
+6 -7
View File
@@ -280,13 +280,12 @@ func (h *handlers) handleAdminDebugModeToggle(w http.ResponseWriter, r *http.Req
h.logger.Info("admin: debug mode toggled",
"target_user", uuidToString(targetID), "enabled", req.Enabled)
writeJSON(w, http.StatusOK, adminUserView{
ID: uuidToString(updated.ID),
Username: updated.Username,
DisplayName: updated.DisplayName,
IsAdmin: updated.IsAdmin,
AutoApproveRequests: updated.AutoApproveRequests,
DebugModeEnabled: updated.DebugModeEnabled,
CreatedAt: updated.CreatedAt.Time.UTC().Format(time.RFC3339),
ID: uuidToString(updated.ID),
Username: updated.Username,
DisplayName: updated.DisplayName,
IsAdmin: updated.IsAdmin,
DebugModeEnabled: updated.DebugModeEnabled,
CreatedAt: updated.CreatedAt.Time.UTC().Format(time.RFC3339),
})
}
-21
View File
@@ -6,7 +6,6 @@
package api
import (
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/eventbus"
"github.com/jackc/pgx/v5/pgtype"
)
@@ -102,23 +101,3 @@ func (h *handlers) publishSystemRebuilt(userID pgtype.UUID) {
Data: map[string]any{},
})
}
// publishRequestStatusChanged broadcasts a request status flip to
// the request's original requester so their /requests page reflects the
// new state without manual refresh. Admin actors (approve / reject) still
// route to the requester's user_id, not the admin's — the original user
// is who needs to see the update.
func (h *handlers) publishRequestStatusChanged(row dbq.Request) {
if h.eventbus == nil {
return
}
h.eventbus.Publish(eventbus.Event{
Kind: "request.status_changed",
UserID: uuidToString(row.UserID),
Data: map[string]any{
"request_id": uuidToString(row.ID),
"status": string(row.Status),
"kind": string(row.Kind),
},
})
}
+1 -1
View File
@@ -1,7 +1,7 @@
// Live event stream — Server-Sent Events at GET /api/events/stream.
//
// The web app and the Android app subscribe to be notified of state changes
// elsewhere in the system (likes, request status flips, quarantine
// elsewhere in the system (likes, quarantine
// resolutions, scan progress, playlist mutations) so static screens
// can invalidate cached data without explicit user action.
//
+1 -2
View File
@@ -468,7 +468,7 @@ func TestRoutesRegisteredInMount(t *testing.T) {
r := chi.NewRouter()
w := playevents.NewWriter(h.pool, slog.New(slog.NewTextHandler(io.Discard, nil)),
30*time.Minute, 0.5, 30000)
Mount(r, h.pool, h.logger, w, config.RecommendationConfig{RadioSize: 50, RadioSizeMax: 200, RecentlyPlayedHours: 1}, h.recSettings, h.requests, h.quarantine, h.tracks, h.playlists, h.coverart, h.coverSettings, h.tagSettings, h.scanner, h.scanCfg, h.dataDir, nil, eventbus.New(), nil, nil, h.netSettings, nil, nil, nil, nil, backup.NewService(h.pool, h.logger, "test"), nil)
Mount(r, h.pool, h.logger, w, config.RecommendationConfig{RadioSize: 50, RadioSizeMax: 200, RecentlyPlayedHours: 1}, h.recSettings, h.quarantine, h.tracks, h.playlists, h.coverart, h.coverSettings, h.tagSettings, h.scanner, h.scanCfg, h.dataDir, nil, eventbus.New(), nil, nil, h.netSettings, nil, nil, nil, backup.NewService(h.pool, h.logger, "test"), nil)
paths := []string{
"/api/artists",
@@ -486,7 +486,6 @@ func TestRoutesRegisteredInMount(t *testing.T) {
// admin group, so reaching the auth check is what proves they are
// wired.
"/api/admin/library/missing",
"/api/admin/library/reacquisition",
"/api/admin/library/fingerprints",
"/api/admin/library/fingerprint-settings",
"/api/admin/library/duplicates",
-20
View File
@@ -8,7 +8,6 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/library"
"git.fabledsword.com/bvandeusen/minstrel/internal/notifications"
"git.fabledsword.com/bvandeusen/minstrel/internal/requests"
)
// Helpers for the handlers that produce notifications. Every call is
@@ -23,25 +22,6 @@ func userLabel(u dbq.User) string {
return u.Username
}
func requestPayload(row dbq.Request) notifications.Payload {
return notifications.Payload{
RequestID: uuidToString(row.ID),
RequestKind: string(row.Kind),
Name: requests.DisplayName(row),
}
}
// notifyRequestDecided tells the requester an admin approved or declined
// their request. An admin deciding their own request needs no notice.
func (h *handlers) notifyRequestDecided(ctx context.Context, kind notifications.Kind, admin dbq.User, row dbq.Request, reason string) {
if row.UserID == admin.ID {
return
}
p := requestPayload(row)
p.Reason = reason
h.notifier.NotifyLogged(ctx, kind, notifications.ToUser(row.UserID), p.Map())
}
// quarantineReasonLabel is a flag reason as the flag popover words it, in
// lower case to sit inside a sentence ("flagged X: bad rip").
func quarantineReasonLabel(reason string) string {
-93
View File
@@ -53,99 +53,6 @@ func wantInbox(t *testing.T, who string, got []inboxItem, kinds ...notifications
}
}
func TestNotify_NewRequestReachesOtherAdminsOnce(t *testing.T) {
h, pool := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, pool, "np-alice", "pw", false)
admin := seedUser(t, pool, "np-admin", "pw", true)
selfAdmin := seedUser(t, pool, "np-self", "pw", true)
rv := createArtistRequest(t, h, alice, "np-mbid-1", "Pending Band")
// The same request again dedups into the first and is not announced twice.
createArtistRequest(t, h, alice, "np-mbid-1", "Pending Band")
got := inboxOf(t, pool, admin)
wantInbox(t, "admin", got, notifications.KindRequestPending)
if got[0].Payload.RequestID != uuidToString(rv.ID) || got[0].Payload.Name != "Pending Band" || got[0].Payload.Actor != alice.Username {
t.Errorf("pending payload = %+v", got[0].Payload)
}
wantInbox(t, "requester", inboxOf(t, pool, alice))
// An admin's own request goes to the other admins, never back to them.
createArtistRequest(t, h, selfAdmin, "np-mbid-2", "Own Band")
wantInbox(t, "self-admin", inboxOf(t, pool, selfAdmin), notifications.KindRequestPending)
if n := len(inboxOf(t, pool, admin)); n != 2 {
t.Errorf("admin inbox = %d, want 2 (alice's and the self-admin's)", n)
}
}
func TestNotify_AutoApprovedRequestWaitsOnNobody(t *testing.T) {
h, _ := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, h.pool, "np-auto", "pw", false)
admin := seedUser(t, h.pool, "np-auto-admin", "pw", true)
if _, err := h.pool.Exec(context.Background(),
"UPDATE users SET auto_approve_requests = true WHERE id = $1", alice.ID); err != nil {
t.Fatalf("set auto_approve: %v", err)
}
alice, err := dbq.New(h.pool).GetUserByID(context.Background(), alice.ID)
if err != nil {
t.Fatalf("reload user: %v", err)
}
rv := createArtistRequest(t, h, alice, "np-auto-mbid", "Auto Band")
if rv.Status != "approved" {
t.Fatalf("status = %q, want approved — the stub should accept the add", rv.Status)
}
wantInbox(t, "admin", inboxOf(t, h.pool, admin))
wantInbox(t, "requester", inboxOf(t, h.pool, alice))
}
func TestNotify_DecisionsReachTheRequester(t *testing.T) {
h, _ := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, h.pool, "np-dec-alice", "pw", false)
admin := seedUser(t, h.pool, "np-dec-admin", "pw", true)
approved := seedPendingArtistRequest(t, h, alice, "np-dec-1", "Yes Band")
rejected := seedPendingArtistRequest(t, h, alice, "np-dec-2", "No Band")
own := seedPendingArtistRequest(t, h, admin, "np-dec-3", "Own Band")
for _, c := range []struct {
id string
verb string
body []byte
}{
{uuidToString(approved.ID), "approve", nil},
{uuidToString(rejected.ID), "reject", []byte(`{"notes":"Only a live bootleg exists"}`)},
{uuidToString(own.ID), "approve", nil},
} {
w := doAdminRequestReq(t, h, http.MethodPost, "/api/admin/requests/"+c.id+"/"+c.verb, c.body, admin)
if w.Code != http.StatusOK {
t.Fatalf("%s %s: status = %d; body = %s", c.verb, c.id, w.Code, w.Body.String())
}
}
got := inboxOf(t, h.pool, alice)
wantInbox(t, "requester", got, notifications.KindRequestRejected, notifications.KindRequestApproved)
if got[0].Payload.Name != "No Band" || got[0].Payload.Reason != "Only a live bootleg exists" {
t.Errorf("rejected payload = %+v", got[0].Payload)
}
if got[1].Payload.Name != "Yes Band" || got[1].Payload.Reason != "" {
t.Errorf("approved payload = %+v", got[1].Payload)
}
// The admin decided their own request: no notice. alice's two pending
// notices are all the admin holds.
for _, it := range inboxOf(t, h.pool, admin) {
if it.Kind != string(notifications.KindRequestPending) {
t.Errorf("admin inbox holds %q, want only request_pending", it.Kind)
}
}
}
func TestNotify_FlagReachesAdminsButNotTheFlagger(t *testing.T) {
h, pool := testHandlers(t)
truncateLibrary(t, pool)
-264
View File
@@ -1,264 +0,0 @@
package api
import (
"context"
"errors"
"net/http"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/notifications"
"git.fabledsword.com/bvandeusen/minstrel/internal/requests"
)
// requestView is the JSON shape returned by all /api/requests handlers.
// Field names follow the lowercase snake_case convention for /api/* responses.
type requestView struct {
ID pgtype.UUID `json:"id"`
UserID pgtype.UUID `json:"user_id"`
// Username names the requester. Only the admin list fills it; a user's
// own /requests page has no need to name them back to themselves.
Username string `json:"username,omitempty"`
Status string `json:"status"`
Kind string `json:"kind"`
ArtistMBID string `json:"artist_mbid"`
AlbumMBID *string `json:"album_mbid,omitempty"`
TrackMBID *string `json:"track_mbid,omitempty"`
ArtistName string `json:"artist_name"`
AlbumTitle *string `json:"album_title,omitempty"`
TrackTitle *string `json:"track_title,omitempty"`
DecidedAt pgtype.Timestamptz `json:"decided_at,omitempty"`
DecidedBy pgtype.UUID `json:"decided_by,omitempty"`
Notes *string `json:"notes,omitempty"`
CompletedAt pgtype.Timestamptz `json:"completed_at,omitempty"`
MatchedTrackID pgtype.UUID `json:"matched_track_id,omitempty"`
MatchedAlbumID pgtype.UUID `json:"matched_album_id,omitempty"`
MatchedArtistID pgtype.UUID `json:"matched_artist_id,omitempty"`
RequestedAt pgtype.Timestamptz `json:"requested_at"`
UpdatedAt pgtype.Timestamptz `json:"updated_at"`
// Progress counters, computed at response time from the matched
// entity's children. 0 when nothing has been ingested yet (or when
// the request has no matched entity yet). For artist requests both
// counters fill; for album/track requests only ImportedTrackCount.
ImportedAlbumCount int `json:"imported_album_count"`
ImportedTrackCount int `json:"imported_track_count"`
}
func requestViewFrom(row dbq.Request) requestView {
return requestView{
ID: row.ID,
UserID: row.UserID,
Status: string(row.Status),
Kind: string(row.Kind),
ArtistMBID: row.ArtistMbid,
AlbumMBID: row.AlbumMbid,
TrackMBID: row.TrackMbid,
ArtistName: row.ArtistName,
AlbumTitle: row.AlbumTitle,
TrackTitle: row.TrackTitle,
DecidedAt: row.DecidedAt,
DecidedBy: row.DecidedBy,
Notes: row.Notes,
CompletedAt: row.CompletedAt,
MatchedTrackID: row.MatchedTrackID,
MatchedAlbumID: row.MatchedAlbumID,
MatchedArtistID: row.MatchedArtistID,
RequestedAt: row.RequestedAt,
UpdatedAt: row.UpdatedAt,
}
}
// fillProgress fills ImportedAlbumCount + ImportedTrackCount on a view by
// querying the count of children for the request's matched entity.
//
// kind=artist → counts all albums + tracks under the matched artist.
// kind=album → counts tracks under the matched album.
// kind=track → ImportedTrackCount = 1 once matched_track_id is set.
//
// N+1 against the request list: acceptable at admin scale (dozens of
// in-flight requests at most). Errors are swallowed — progress is a
// reporting nicety, not a correctness gate.
func fillProgress(ctx context.Context, q *dbq.Queries, row dbq.Request, view *requestView) {
switch row.Kind {
case dbq.RequestKindArtist:
if !row.MatchedArtistID.Valid {
return
}
if n, err := q.CountAlbumsByArtist(ctx, row.MatchedArtistID); err == nil {
view.ImportedAlbumCount = int(n)
}
if n, err := q.CountTracksByArtist(ctx, row.MatchedArtistID); err == nil {
view.ImportedTrackCount = int(n)
}
case dbq.RequestKindAlbum:
if !row.MatchedAlbumID.Valid {
return
}
view.ImportedAlbumCount = 1
if n, err := q.CountTracksByAlbum(ctx, row.MatchedAlbumID); err == nil {
view.ImportedTrackCount = int(n)
}
case dbq.RequestKindTrack:
if row.MatchedTrackID.Valid {
view.ImportedTrackCount = 1
}
}
}
// createRequestBody is the decoded JSON for POST /api/requests. album_mbid
// is the album's MusicBrainz release group, as catalogue search returns it.
type createRequestBody struct {
Kind string `json:"kind"`
ArtistMBID string `json:"artist_mbid"`
AlbumMBID string `json:"album_mbid"`
TrackMBID string `json:"track_mbid"`
ArtistName string `json:"artist_name"`
AlbumTitle string `json:"album_title"`
TrackTitle string `json:"track_title"`
}
// handleCreateRequest implements POST /api/requests.
func (h *handlers) handleCreateRequest(w http.ResponseWriter, r *http.Request) {
user, ok := requireUser(w, r)
if !ok {
return
}
var body createRequestBody
if !decodeBody(w, r, &body) {
return
}
row, created, err := h.requests.CreateTracked(r.Context(), user.ID, requests.CreateParams{
Kind: body.Kind,
ArtistMBID: body.ArtistMBID,
AlbumMBID: body.AlbumMBID,
TrackMBID: body.TrackMBID,
ArtistName: body.ArtistName,
AlbumTitle: body.AlbumTitle,
TrackTitle: body.TrackTitle,
})
if err != nil {
if errors.Is(err, requests.ErrInvalidKindFields) {
writeErr(w, apierror.BadRequest("mbid_required", err.Error()))
return
}
h.logger.Error("api: create request", "err", err)
writeErr(w, apierror.InternalMsg("create failed", err))
return
}
// Auto-approve: when an admin has set the user's
// auto_approve_requests flag, the new request goes straight onto the
// wanted list as if approved. A failure leaves it pending for an admin,
// the same as a manual approval hitting the same error. The approver is
// the user themselves, acting under the privilege the admin granted; the
// grant is in audit_log (ActionAutoApproveToggle).
if user.AutoApproveRequests {
approved, aerr := h.requests.Approve(r.Context(), row.ID, user.ID, pgtype.UUID{})
if aerr == nil {
row = approved
} else {
h.logger.Warn("api: auto-approve failed; request stays pending",
"user_id", uuidToString(user.ID),
"request_id", uuidToString(row.ID),
"err", aerr)
}
}
// A new request still pending after any auto-approval waits on an admin.
// One that deduped into a request already in flight was announced when it
// was first made, and the requester, if they are an admin themselves,
// needs no notice of their own request.
if created && row.Status == dbq.RequestStatusPending {
p := requestPayload(row)
p.Actor = userLabel(user)
h.notifier.NotifyLogged(r.Context(), notifications.KindRequestPending, notifications.ToAdmins(user.ID), p.Map())
}
h.publishRequestStatusChanged(row)
writeJSON(w, http.StatusCreated, requestViewFrom(row))
}
// handleListRequests implements GET /api/requests — returns caller's own
// requests.
func (h *handlers) handleListRequests(w http.ResponseWriter, r *http.Request) {
user, ok := requireUser(w, r)
if !ok {
return
}
rows, err := h.requests.ListForUser(r.Context(), user.ID, 100)
if err != nil {
h.logger.Error("api: list requests", "err", err)
writeErr(w, apierror.InternalMsg("list failed", err))
return
}
q := dbq.New(h.pool)
out := make([]requestView, 0, len(rows))
for _, row := range rows {
v := requestViewFrom(row)
fillProgress(r.Context(), q, row, &v)
out = append(out, v)
}
writeJSON(w, http.StatusOK, out)
}
// handleGetRequest implements GET /api/requests/:id.
// The caller must own the request, or be an admin.
func (h *handlers) handleGetRequest(w http.ResponseWriter, r *http.Request) {
user, ok := requireUser(w, r)
if !ok {
return
}
row, apiErr := resolveByID(r, "id", dbq.New(h.pool).GetRequestByID, "request")
if apiErr != nil {
writeErr(w, apiErr)
return
}
// Allow if caller owns the request or is an admin.
if row.UserID != user.ID && !user.IsAdmin {
writeErr(w, apierror.NotFound("request"))
return
}
v := requestViewFrom(row)
fillProgress(r.Context(), dbq.New(h.pool), row, &v)
writeJSON(w, http.StatusOK, v)
}
// handleCancelRequest implements DELETE /api/requests/:id.
// Cancels the caller's own pending request.
func (h *handlers) handleCancelRequest(w http.ResponseWriter, r *http.Request) {
user, ok := requireUser(w, r)
if !ok {
return
}
id, ok := requireURLUUID(w, r, "id")
if !ok {
return
}
row, err := h.requests.Cancel(r.Context(), id, user.ID)
if err != nil {
switch {
case errors.Is(err, requests.ErrNotPending):
writeErr(w, apierror.Conflict("request_not_pending", "request is not pending"))
case errors.Is(err, requests.ErrNotFound):
writeErr(w, apierror.NotFound("request"))
default:
h.logger.Error("api: cancel request", "err", err)
writeErr(w, apierror.InternalMsg("cancel failed", err))
}
return
}
h.publishRequestStatusChanged(row)
writeJSON(w, http.StatusOK, requestViewFrom(row))
}
-396
View File
@@ -1,396 +0,0 @@
package api
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
// newRequestsRouter builds a test chi router wiring the /api/requests handlers
// with chi URL params but without RequireUser middleware. Tests inject a user
// into the context manually.
func newRequestsRouter(h *handlers) chi.Router {
r := chi.NewRouter()
r.Post("/api/requests", h.handleCreateRequest)
r.Get("/api/requests", h.handleListRequests)
r.Get("/api/requests/{id}", h.handleGetRequest)
r.Delete("/api/requests/{id}", h.handleCancelRequest)
return r
}
// withUser injects a user into the request context the same way RequireUser
// does.
func withUser(req *http.Request, user dbq.User) *http.Request {
return req.WithContext(context.WithValue(req.Context(), userCtxKeyForTest(), user))
}
// doCreateRequest fires POST /api/requests with the given JSON body as the
// given user.
func doCreateRequest(h *handlers, user dbq.User, body string) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodPost, "/api/requests", bytes.NewBufferString(body))
req.Header.Set("Content-Type", "application/json")
req = withUser(req, user)
w := httptest.NewRecorder()
newRequestsRouter(h).ServeHTTP(w, req)
return w
}
// doListRequests fires GET /api/requests as the given user.
func doListRequests(h *handlers, user dbq.User) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodGet, "/api/requests", nil)
req = withUser(req, user)
w := httptest.NewRecorder()
newRequestsRouter(h).ServeHTTP(w, req)
return w
}
// doGetRequest fires GET /api/requests/:id as the given user.
func doGetRequest(h *handlers, user dbq.User, id pgtype.UUID) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodGet, "/api/requests/"+uuidToString(id), nil)
req = withUser(req, user)
w := httptest.NewRecorder()
newRequestsRouter(h).ServeHTTP(w, req)
return w
}
// doCancelRequest fires DELETE /api/requests/:id as the given user.
func doCancelRequest(h *handlers, user dbq.User, id pgtype.UUID) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodDelete, "/api/requests/"+uuidToString(id), nil)
req = withUser(req, user)
w := httptest.NewRecorder()
newRequestsRouter(h).ServeHTTP(w, req)
return w
}
// createArtistRequest is a convenience wrapper that seeds a valid artist
// request via the handler and returns the parsed response.
func createArtistRequest(t *testing.T, h *handlers, user dbq.User, artistMBID, artistName string) requestView {
t.Helper()
body := fmt.Sprintf(`{"kind":"artist","artist_mbid":%q,"artist_name":%q}`, artistMBID, artistName)
w := doCreateRequest(h, user, body)
if w.Code != http.StatusCreated {
t.Fatalf("createArtistRequest: status = %d, want 201; body = %s", w.Code, w.Body.String())
}
var rv requestView
if err := json.Unmarshal(w.Body.Bytes(), &rv); err != nil {
t.Fatalf("createArtistRequest: decode: %v; body = %s", err, w.Body.String())
}
return rv
}
// TestHandleCreateRequest_AutoApprove_GoesStraightOnTheWantedList verifies
// that a user with auto_approve_requests=true gets their request approved on
// creation: an artist request follows the artist at once.
func TestHandleCreateRequest_AutoApprove_GoesStraightOnTheWantedList(t *testing.T) {
h, pool := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, pool, "alice", "pw", false)
if _, err := pool.Exec(context.Background(),
"UPDATE users SET auto_approve_requests = true WHERE id = $1", alice.ID); err != nil {
t.Fatalf("set auto_approve: %v", err)
}
// Re-fetch so the User row carried into the request context has the flag.
refreshed, err := dbq.New(pool).GetUserByID(context.Background(), alice.ID)
if err != nil {
t.Fatalf("reload user: %v", err)
}
body := `{"kind":"artist","artist_mbid":"auto-mbid-1","artist_name":"Autoband"}`
w := doCreateRequest(h, refreshed, body)
if w.Code != http.StatusCreated {
t.Fatalf("status = %d, want 201; body = %s", w.Code, w.Body.String())
}
var rv requestView
if err := json.Unmarshal(w.Body.Bytes(), &rv); err != nil {
t.Fatalf("decode: %v", err)
}
if rv.Status != "approved" {
t.Errorf("status = %q, want approved", rv.Status)
}
if _, err := dbq.New(pool).GetFollow(context.Background(), "auto-mbid-1"); err != nil {
t.Errorf("the artist should be followed: %v", err)
}
}
// TestHandleCreateRequest_Artist_HappyPath verifies a valid artist request
// returns 201 with kind=artist and status=pending.
func TestHandleCreateRequest_Artist_HappyPath(t *testing.T) {
h, pool := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, pool, "alice", "pw", false)
body := `{"kind":"artist","artist_mbid":"artist-mbid-1","artist_name":"The Beatles"}`
w := doCreateRequest(h, alice, body)
if w.Code != http.StatusCreated {
t.Fatalf("status = %d, want 201; body = %s", w.Code, w.Body.String())
}
var rv requestView
if err := json.Unmarshal(w.Body.Bytes(), &rv); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if string(rv.Kind) != "artist" {
t.Errorf("kind = %q, want artist", rv.Kind)
}
if rv.Status != "pending" {
t.Errorf("status = %q, want pending", rv.Status)
}
if rv.ArtistMBID != "artist-mbid-1" {
t.Errorf("artist_mbid = %q", rv.ArtistMBID)
}
if rv.ArtistName != "The Beatles" {
t.Errorf("artist_name = %q", rv.ArtistName)
}
if !rv.ID.Valid {
t.Error("id not set")
}
}
// TestHandleCreateRequest_AlbumWithoutAlbumMBID_400 verifies that sending
// kind=album without album_mbid returns 400 with mbid_required.
func TestHandleCreateRequest_AlbumWithoutAlbumMBID_400(t *testing.T) {
h, pool := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, pool, "alice", "pw", false)
// Missing album_mbid and album_title.
body := `{"kind":"album","artist_mbid":"artist-mbid-1","artist_name":"The Beatles"}`
w := doCreateRequest(h, alice, body)
if w.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400; body = %s", w.Code, w.Body.String())
}
var resp struct {
Error struct {
Code string `json:"code"`
} `json:"error"`
}
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if resp.Error.Code != "mbid_required" {
t.Errorf("error.code = %q, want mbid_required", resp.Error.Code)
}
}
// TestHandleCreateRequest_TrackWithoutTrackFields_400 verifies that kind=track
// with missing track_mbid/track_title returns 400 with mbid_required.
func TestHandleCreateRequest_TrackWithoutTrackFields_400(t *testing.T) {
h, pool := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, pool, "alice", "pw", false)
// Provides album fields but misses track_mbid and track_title.
body := `{"kind":"track","artist_mbid":"artist-mbid-1","artist_name":"Beatles","album_mbid":"album-mbid-1","album_title":"Abbey Road"}`
w := doCreateRequest(h, alice, body)
if w.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400; body = %s", w.Code, w.Body.String())
}
var resp struct {
Error struct {
Code string `json:"code"`
} `json:"error"`
}
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if resp.Error.Code != "mbid_required" {
t.Errorf("error.code = %q, want mbid_required", resp.Error.Code)
}
}
// TestHandleListRequests_OnlyOwnRequests verifies alice sees only her own
// requests, not bob's.
func TestHandleListRequests_OnlyOwnRequests(t *testing.T) {
h, pool := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, pool, "alice", "pw", false)
bob := seedUser(t, pool, "bob", "pw", false)
createArtistRequest(t, h, alice, "alice-mbid-1", "Alice Artist 1")
createArtistRequest(t, h, alice, "alice-mbid-2", "Alice Artist 2")
createArtistRequest(t, h, bob, "bob-mbid-1", "Bob Artist 1")
w := doListRequests(h, alice)
if w.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
}
var results []requestView
if err := json.Unmarshal(w.Body.Bytes(), &results); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if len(results) != 2 {
t.Fatalf("len(results) = %d, want 2", len(results))
}
for _, rv := range results {
if rv.UserID != alice.ID {
t.Errorf("result belongs to wrong user: %v", rv.UserID)
}
}
}
// TestHandleGetRequest_OwnReturns200 verifies a caller can fetch their own
// request.
func TestHandleGetRequest_OwnReturns200(t *testing.T) {
h, pool := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, pool, "alice", "pw", false)
rv := createArtistRequest(t, h, alice, "artist-mbid-get", "Get Artist")
w := doGetRequest(h, alice, rv.ID)
if w.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
}
var got requestView
if err := json.Unmarshal(w.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if got.ID != rv.ID {
t.Errorf("id mismatch")
}
}
// TestHandleGetRequest_OtherUserNotAdmin_404 verifies a non-admin caller
// cannot see another user's request.
func TestHandleGetRequest_OtherUserNotAdmin_404(t *testing.T) {
h, pool := testHandlers(t)
resetRequests(t, h)
bob := seedUser(t, pool, "bob", "pw", false)
charlie := seedUser(t, pool, "charlie", "pw", false)
rv := createArtistRequest(t, h, bob, "bob-artist-mbid", "Bob's Artist")
// Charlie is not an admin, tries to fetch Bob's request.
w := doGetRequest(h, charlie, rv.ID)
if w.Code != http.StatusNotFound {
t.Fatalf("status = %d, want 404; body = %s", w.Code, w.Body.String())
}
var resp struct {
Error struct {
Code string `json:"code"`
} `json:"error"`
}
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if resp.Error.Code != "request_not_found" {
t.Errorf("error.code = %q, want request_not_found", resp.Error.Code)
}
}
// TestHandleGetRequest_OtherUserAsAdmin_200 verifies an admin can fetch any
// user's request.
func TestHandleGetRequest_OtherUserAsAdmin_200(t *testing.T) {
h, pool := testHandlers(t)
resetRequests(t, h)
bob := seedUser(t, pool, "bob", "pw", false)
admin := seedUser(t, pool, "admin", "pw", true)
rv := createArtistRequest(t, h, bob, "bob-admin-mbid", "Bob's Artist For Admin")
// Admin fetches Bob's request — should succeed.
w := doGetRequest(h, admin, rv.ID)
if w.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
}
var got requestView
if err := json.Unmarshal(w.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if got.ID != rv.ID {
t.Errorf("id mismatch")
}
}
// TestHandleCancelRequest_PendingHappyPath verifies that cancelling a pending
// request returns 200 with status=rejected.
func TestHandleCancelRequest_PendingHappyPath(t *testing.T) {
h, pool := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, pool, "alice", "pw", false)
rv := createArtistRequest(t, h, alice, "cancel-mbid-1", "Cancel Me")
w := doCancelRequest(h, alice, rv.ID)
if w.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
}
var got requestView
if err := json.Unmarshal(w.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v; body = %s", err, w.Body.String())
}
if got.Status != "rejected" {
t.Errorf("status = %q, want rejected", got.Status)
}
}
// TestHandleCancelRequest_NonPending_409 verifies that cancelling an already
// rejected request returns 409 with request_not_pending.
func TestHandleCancelRequest_NonPending_409(t *testing.T) {
h, pool := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, pool, "alice", "pw", false)
rv := createArtistRequest(t, h, alice, "cancel-mbid-2", "Cancel Twice")
// First cancel transitions to rejected.
w1 := doCancelRequest(h, alice, rv.ID)
if w1.Code != http.StatusOK {
t.Fatalf("first cancel: status = %d, want 200; body = %s", w1.Code, w1.Body.String())
}
// Second cancel on a now-rejected row → ErrNotPending → 409. The cancel SQL
// checks user_id = $2 AND status = 'pending'; zero rows → ErrNotPending.
// For this test we use dbq directly to reject via admin so Cancel returns
// ErrNotPending rather than the ownership miss. We already did first cancel
// so row is rejected.
w2 := doCancelRequest(h, alice, rv.ID)
if w2.Code != http.StatusConflict {
t.Fatalf("second cancel: status = %d, want 409; body = %s", w2.Code, w2.Body.String())
}
var resp struct {
Error struct {
Code string `json:"code"`
} `json:"error"`
}
if err := json.Unmarshal(w2.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v; body = %s", err, w2.Body.String())
}
if resp.Error.Code != "request_not_pending" {
t.Errorf("error.code = %q, want request_not_pending", resp.Error.Code)
}
}
// TestHandleCancelRequest_WrongUser_409 verifies that attempting to cancel
// another user's request returns 409 (not_pending from the ownership miss).
func TestHandleCancelRequest_WrongUser_409(t *testing.T) {
h, pool := testHandlers(t)
resetRequests(t, h)
alice := seedUser(t, pool, "alice-cancel-owner", "pw", false)
bob := seedUser(t, pool, "bob-cancel-other", "pw", false)
rv := createArtistRequest(t, h, alice, "cancel-wrong-user-mbid", "Alice's Artist")
// Bob tries to cancel Alice's request: SQL WHERE user_id=bob AND
// status=pending → 0 rows.
w := doCancelRequest(h, bob, rv.ID)
if w.Code != http.StatusConflict {
t.Fatalf("status = %d, want 409; body = %s", w.Code, w.Body.String())
}
}
-261
View File
@@ -1,261 +0,0 @@
package api
import (
"encoding/json"
"errors"
"io"
"net/http"
"strconv"
"strings"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/recommendation"
)
// suggestionView is the wire shape returned by GET /api/discover/suggestions.
type suggestionView struct {
MBID string `json:"mbid"`
Name string `json:"name"`
Score float64 `json:"score"`
Attribution []seedContributionView `json:"attribution"`
// MatchedTags are the candidate's tags that overlap the user's taste
// profile, strongest first — the "matches: shoegaze, melancholic"
// line. Omitted when empty, which is common: tag coverage for
// out-of-library artists is permanently partial, and the card falls back
// to the seed attribution.
MatchedTags []string `json:"matched_tags,omitempty"`
// ImageURL is Minstrel's catalogue art endpoint for the candidate
// (out-of-library artists have no local art row). It
// answers 404 when no enabled source has an image, and the client
// falls back to a placeholder.
ImageURL string `json:"image_url,omitempty"`
}
type seedContributionView struct {
ArtistID pgtype.UUID `json:"artist_id"`
Name string `json:"name"`
Contribution float64 `json:"contribution"`
IsLiked bool `json:"is_liked"`
PlayCount int64 `json:"play_count"`
}
// handleListSuggestions implements GET /api/discover/suggestions.
//
// Query params:
// - limit (default 12, capped at 50)
// - half_life_days (default 30)
//
// Returns 200 with a JSON array (possibly empty). Read-only; no admin gate.
func (h *handlers) handleListSuggestions(w http.ResponseWriter, r *http.Request) {
user, ok := requireUser(w, r)
if !ok {
return
}
limit := 12
if v := r.URL.Query().Get("limit"); v != "" {
n, err := strconv.Atoi(v)
if err != nil || n < 1 {
writeErr(w, apierror.BadRequest("bad_request", "invalid limit"))
return
}
limit = n
}
halfLife := 30.0
if v := r.URL.Query().Get("half_life_days"); v != "" {
f, err := strconv.ParseFloat(v, 64)
if err != nil || f <= 0 {
writeErr(w, apierror.BadRequest("bad_request", "invalid half_life_days"))
return
}
halfLife = f
}
// Read the tuned weight per request so an admin change takes effect on the
// next refresh, no restart.
tagWeight := h.recSettings.Discover().TagOverlapWeight
suggestions, err := recommendation.SuggestArtists(
r.Context(), h.pool, user.ID, halfLife, limit, tagWeight)
if err != nil {
h.logger.Error("api: list suggestions", "err", err)
writeErr(w, apierror.InternalMsg("failed to load suggestions", err))
return
}
out := make([]suggestionView, 0, len(suggestions))
for _, s := range suggestions {
attr := make([]seedContributionView, 0, len(s.Attribution))
for _, a := range s.Attribution {
attr = append(attr, seedContributionView{
ArtistID: a.ArtistID,
Name: a.Name,
Contribution: a.Contribution,
IsLiked: a.IsLiked,
PlayCount: a.PlayCount,
})
}
out = append(out, suggestionView{
MBID: s.MBID, Name: s.Name, Score: s.Score, Attribution: attr,
MatchedTags: s.MatchedTags,
})
}
resolveSuggestionArt(out)
writeJSON(w, http.StatusOK, out)
}
// maxSnoozeDays caps a client-supplied duration. The DEFAULT is not here: it's
// a DB-backed knob on the admin tuning card, read per request via
// recSettings.Discover().SnoozeDays.
const maxSnoozeDays = 365.0
// snoozeRequest is the POST body. Both fields are optional in the JSON sense
// (an absent body snoozes for the default), but Name is required in practice:
// candidates are out-of-library, so the server has no artists row to resolve a
// display name from and the un-snooze list would have nothing to show. The
// client always has it — it just rendered the card.
type snoozeRequest struct {
Name string `json:"name"`
Days float64 `json:"days"`
}
// snoozeView is one row of GET /api/discover/snoozes.
type snoozeView struct {
MBID string `json:"mbid"`
Name string `json:"name"`
SnoozedUntil pgtype.Timestamptz `json:"snoozed_until"`
CreatedAt pgtype.Timestamptz `json:"created_at"`
}
// handleSnoozeSuggestion implements
// POST /api/discover/suggestions/{mbid}/snooze.
//
// Parks a candidate for `days` (default 90, capped at 365). Idempotent:
// snoozing an already-snoozed candidate extends it rather than conflicting.
//
// This is NOT negative feedback. It records no verdict on the artist and is
// never read by internal/taste — see 0049_suggestion_snoozes.up.sql for why.
// Returns 204.
func (h *handlers) handleSnoozeSuggestion(w http.ResponseWriter, r *http.Request) {
user, ok := requireUser(w, r)
if !ok {
return
}
mbid := strings.TrimSpace(chi.URLParam(r, "mbid"))
if mbid == "" {
writeErr(w, apierror.BadRequest("invalid_id", "missing mbid"))
return
}
// An empty body is a valid "snooze this for the default period", so EOF
// is not an error here — decodeBody would reject it as a malformed body.
var body snoozeRequest
if err := json.NewDecoder(r.Body).Decode(&body); err != nil && !errors.Is(err, io.EOF) {
writeErr(w, apierror.BadRequest("invalid_body", ""))
return
}
name := strings.TrimSpace(body.Name)
if name == "" {
writeErr(w, apierror.BadRequest("invalid_body", "name is required"))
return
}
days := body.Days
if days <= 0 {
days = h.recSettings.Discover().SnoozeDays
}
if days > maxSnoozeDays {
// Clamp rather than reject: a client asking for longer than we allow
// still means "park this", and failing the write would leave the card
// sitting there as if the tap did nothing.
days = maxSnoozeDays
}
q := dbq.New(h.pool)
if err := q.SnoozeSuggestion(r.Context(), dbq.SnoozeSuggestionParams{
UserID: user.ID,
CandidateMbid: mbid,
CandidateName: name,
Column4: days,
}); err != nil {
h.logger.Error("api: snooze suggestion", "err", err)
writeErr(w, apierror.InternalMsg("failed to snooze suggestion", err))
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleUnsnoozeSuggestion implements
// DELETE /api/discover/suggestions/{mbid}/snooze.
//
// Brings a parked candidate back immediately. 404s an MBID this user never
// snoozed, so the client can tell "undone" from "there was nothing there".
func (h *handlers) handleUnsnoozeSuggestion(w http.ResponseWriter, r *http.Request) {
user, ok := requireUser(w, r)
if !ok {
return
}
mbid := strings.TrimSpace(chi.URLParam(r, "mbid"))
if mbid == "" {
writeErr(w, apierror.BadRequest("invalid_id", "missing mbid"))
return
}
q := dbq.New(h.pool)
rows, err := q.UnsnoozeSuggestion(r.Context(), dbq.UnsnoozeSuggestionParams{
UserID: user.ID,
CandidateMbid: mbid,
})
if err != nil {
h.logger.Error("api: unsnooze suggestion", "err", err)
writeErr(w, apierror.InternalMsg("failed to unsnooze suggestion", err))
return
}
if rows == 0 {
writeErr(w, apierror.NotFound("snooze"))
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleListSuggestionSnoozes implements GET /api/discover/snoozes.
//
// The un-snooze surface needs this: a parked candidate is by definition
// absent from the suggestion deck, so without a list there is no way to
// reach the DELETE above. Scoped to the caller. Expired rows are
// already filtered by the query — the hourly gc sweep only reclaims space.
func (h *handlers) handleListSuggestionSnoozes(w http.ResponseWriter, r *http.Request) {
user, ok := requireUser(w, r)
if !ok {
return
}
rows, err := dbq.New(h.pool).ListActiveSuggestionSnoozes(r.Context(), user.ID)
if err != nil {
h.logger.Error("api: list suggestion snoozes", "err", err)
writeErr(w, apierror.InternalMsg("failed to load snoozes", err))
return
}
out := make([]snoozeView, 0, len(rows))
for _, row := range rows {
out = append(out, snoozeView{
MBID: row.CandidateMbid,
Name: row.CandidateName,
SnoozedUntil: row.SnoozedUntil,
CreatedAt: row.CreatedAt,
})
}
writeJSON(w, http.StatusOK, out)
}
// resolveSuggestionArt points each suggestion's ImageURL at Minstrel's
// catalogue art endpoint. Nothing is fetched here: the image is
// looked up from the enabled cover-art sources when the browser asks for it,
// and kept, so listing suggestions makes no outbound call. A candidate with
// no MBID keeps an empty ImageURL and the client renders its placeholder.
func resolveSuggestionArt(views []suggestionView) {
for i := range views {
if views[i].MBID != "" {
views[i].ImageURL = artistArtURL(views[i].MBID, views[i].Name)
}
}
}
-104
View File
@@ -1,104 +0,0 @@
package api
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
func newSuggestionsRouter(h *handlers) chi.Router {
r := chi.NewRouter()
r.Get("/api/discover/suggestions", h.handleListSuggestions)
return r
}
func doListSuggestions(h *handlers, user dbq.User, query string) *httptest.ResponseRecorder {
url := "/api/discover/suggestions"
if query != "" {
url += "?" + query
}
req := httptest.NewRequest(http.MethodGet, url, nil)
req = withUser(req, user)
w := httptest.NewRecorder()
newSuggestionsRouter(h).ServeHTTP(w, req)
return w
}
func TestSuggestions_HappyPath(t *testing.T) {
h, pool := testHandlers(t)
user := seedUser(t, pool, "alice", "pw", false)
q := dbq.New(pool)
seed, err := q.UpsertArtist(context.Background(), dbq.UpsertArtistParams{
Name: "Seed", SortName: "Seed",
})
if err != nil {
t.Fatalf("UpsertArtist: %v", err)
}
if _, err := pool.Exec(context.Background(),
`INSERT INTO general_likes_artists (user_id, artist_id) VALUES ($1, $2)`,
user.ID, seed.ID,
); err != nil {
t.Fatalf("like artist: %v", err)
}
if err := q.UpsertArtistSimilarityUnmatched(context.Background(), dbq.UpsertArtistSimilarityUnmatchedParams{
SeedArtistID: seed.ID,
CandidateMbid: "out-mbid",
CandidateName: "Outsider",
Score: 0.9,
Source: "listenbrainz",
}); err != nil {
t.Fatalf("UpsertArtistSimilarityUnmatched: %v", err)
}
w := doListSuggestions(h, user, "")
if w.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
}
var got []suggestionView
if err := json.Unmarshal(w.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v", err)
}
if len(got) != 1 {
t.Fatalf("len = %d, want 1; body = %s", len(got), w.Body.String())
}
if got[0].MBID != "out-mbid" || got[0].Name != "Outsider" {
t.Errorf("got = %+v", got[0])
}
if len(got[0].Attribution) != 1 || got[0].Attribution[0].Name != "Seed" {
t.Errorf("attribution = %+v, want one entry named Seed", got[0].Attribution)
}
if !got[0].Attribution[0].IsLiked {
t.Errorf("attribution.IsLiked = false, want true")
}
}
func TestSuggestions_EmptyForNewUser(t *testing.T) {
h, pool := testHandlers(t)
user := seedUser(t, pool, "newbie", "pw", false)
w := doListSuggestions(h, user, "")
if w.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
}
body := w.Body.String()
if body != "[]" && body != "[]\n" {
t.Errorf("body = %q, want []", body)
}
}
func TestSuggestions_BadLimit(t *testing.T) {
h, pool := testHandlers(t)
user := seedUser(t, pool, "alice", "pw", false)
w := doListSuggestions(h, user, "limit=not-a-number")
if w.Code != http.StatusBadRequest {
t.Errorf("status = %d, want 400; body = %s", w.Code, w.Body.String())
}
}
+1 -1
View File
@@ -58,7 +58,7 @@ const (
// The resolver settling a group without a merge:
// dismissing two versions of a song that are not duplicates, and removing
// copies that filled another song's track slot (their tracks are kept,
// marked missing, and fetched again).
// marked missing, until a good copy is in the library).
ActionDuplicateDismiss Action = "duplicate_dismiss"
ActionDuplicateWrongFile Action = "duplicate_wrong_file"
-214
View File
@@ -1,214 +0,0 @@
package catalogue
import (
"context"
"errors"
"fmt"
"os"
"path/filepath"
"regexp"
"sync"
"time"
"git.fabledsword.com/bvandeusen/minstrel/internal/coverart"
)
// NoArtRetry is how long "no source has art for this" is believed before
// the sources are asked again.
const NoArtRetry = 7 * 24 * time.Hour
// fetchTimeout bounds one look-up across every source.
const fetchTimeout = 30 * time.Second
// ErrNoArt: no enabled source has an image, or the sources could not be
// reached. The caller shows its placeholder.
var ErrNoArt = errors.New("catalogue: no art")
var mbidPattern = regexp.MustCompile(`^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$`)
// ValidMBID reports whether s is shaped like a MusicBrainz id. It is the
// guard that keeps a request path out of the cache directory's parents.
func ValidMBID(s string) bool { return mbidPattern.MatchString(s) }
// Art serves images for artists and release groups that are not in the
// library (search results, Discover suggestions, wanted albums), from the
// enabled cover-art sources. Each image is fetched once and kept
// under the data directory, so the browser only ever talks to Minstrel.
type Art struct {
dir string
artistSources func() []coverart.ArtistArtProvider
albumSources func() []coverart.AlbumCoverProvider
now func() time.Time
mu sync.Mutex
inflight map[string]*artCall
}
type artCall struct {
done chan struct{}
body []byte
err error
}
// ArtSources lists the image sources to ask, in order. The cover-art
// settings service is one: it answers with the sources an admin enabled.
type ArtSources interface {
EnabledArtistProviders() []coverart.ArtistArtProvider
EnabledAlbumProviders() []coverart.AlbumCoverProvider
}
// NewArt keeps images under dataDir/catalogue-art and asks the given
// sources. Nil sources ask nothing, so every image is the placeholder.
func NewArt(dataDir string, sources ArtSources) *Art {
a := &Art{dir: filepath.Join(dataDir, "catalogue-art"), now: time.Now, inflight: map[string]*artCall{}}
if sources != nil {
a.artistSources = sources.EnabledArtistProviders
a.albumSources = sources.EnabledAlbumProviders
}
return a
}
// Artist returns an image of the artist: the first enabled artist source
// with one wins, its thumbnail preferred over its backdrop.
func (a *Art) Artist(ctx context.Context, mbid, name string) ([]byte, error) {
return a.cached(ctx, "artist", mbid, func(ctx context.Context) ([]byte, error) {
if a.artistSources == nil {
return nil, coverart.ErrNotFound
}
ref := coverart.ArtistRef{MBID: mbid, Name: name}
return firstOf(a.artistSources(), func(p coverart.ArtistArtProvider) ([]byte, error) {
thumb, fanart, err := p.FetchArtistArt(ctx, ref)
if err == nil && len(thumb) == 0 {
thumb = fanart
}
return thumb, err
})
})
}
// ReleaseGroup returns a cover for the release group: a source that can
// answer by group (the Cover Art Archive) is asked by its id, the rest by
// artist and title.
func (a *Art) ReleaseGroup(ctx context.Context, mbid, artist, title string) ([]byte, error) {
return a.cached(ctx, "release-group", mbid, func(ctx context.Context) ([]byte, error) {
if a.albumSources == nil {
return nil, coverart.ErrNotFound
}
byName := coverart.AlbumRef{ArtistName: artist, AlbumTitle: title}
return firstOf(a.albumSources(), func(p coverart.AlbumCoverProvider) ([]byte, error) {
if g, ok := p.(coverart.ReleaseGroupCoverProvider); ok {
return g.FetchReleaseGroupCover(ctx, mbid)
}
if artist == "" || title == "" {
return nil, coverart.ErrNotFound
}
return p.FetchAlbumCover(ctx, byName)
})
})
}
// firstOf asks each source in turn. It reports ErrNotFound only when every
// source said so; if any could not be reached, the miss is transient.
func firstOf[P any](sources []P, ask func(P) ([]byte, error)) ([]byte, error) {
transient := false
for _, p := range sources {
body, err := ask(p)
if err == nil && len(body) > 0 {
return body, nil
}
if err != nil && !errors.Is(err, coverart.ErrNotFound) {
transient = true
}
}
if transient {
return nil, coverart.ErrTransient
}
return nil, coverart.ErrNotFound
}
// cached serves kind/mbid from disk, or fetches it once however many
// requests ask at the same moment. A definite "nobody has one" is written
// down and believed for NoArtRetry; an unreachable source is not, so the
// next request asks again.
func (a *Art) cached(ctx context.Context, kind, mbid string, fetch func(context.Context) ([]byte, error)) ([]byte, error) {
if !ValidMBID(mbid) {
return nil, ErrNoArt
}
dir := filepath.Join(a.dir, kind)
imgPath := filepath.Join(dir, mbid)
nonePath := imgPath + ".none"
if body, err := os.ReadFile(imgPath); err == nil && len(body) > 0 {
return body, nil
}
if fi, err := os.Stat(nonePath); err == nil && a.now().Sub(fi.ModTime()) < NoArtRetry {
return nil, ErrNoArt
}
key := kind + "/" + mbid
a.mu.Lock()
if call, ok := a.inflight[key]; ok {
a.mu.Unlock()
select {
case <-call.done:
return call.body, call.err
case <-ctx.Done():
return nil, ctx.Err()
}
}
call := &artCall{done: make(chan struct{})}
a.inflight[key] = call
a.mu.Unlock()
defer func() {
a.mu.Lock()
delete(a.inflight, key)
a.mu.Unlock()
close(call.done)
}()
// The fetch outlives a browser that navigates away: a second request is
// likely, and the image is worth keeping.
fctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), fetchTimeout)
defer cancel()
body, err := fetch(fctx)
switch {
case err == nil && len(body) > 0:
if werr := writeFile(dir, imgPath, body); werr != nil {
// Serving it uncached is still better than a placeholder.
call.body = body
return body, nil
}
_ = os.Remove(nonePath)
call.body = body
return body, nil
case err == nil || errors.Is(err, coverart.ErrNotFound):
_ = writeFile(dir, nonePath, nil)
}
call.err = ErrNoArt
return nil, ErrNoArt
}
func writeFile(dir, path string, body []byte) error {
if err := os.MkdirAll(dir, 0o755); err != nil {
return fmt.Errorf("catalogue art: %w", err)
}
tmp, err := os.CreateTemp(dir, ".part-*")
if err != nil {
return fmt.Errorf("catalogue art: %w", err)
}
if _, err := tmp.Write(body); err != nil {
_ = tmp.Close()
_ = os.Remove(tmp.Name())
return fmt.Errorf("catalogue art: %w", err)
}
if err := tmp.Close(); err != nil {
_ = os.Remove(tmp.Name())
return fmt.Errorf("catalogue art: %w", err)
}
if err := os.Rename(tmp.Name(), path); err != nil {
_ = os.Remove(tmp.Name())
return fmt.Errorf("catalogue art: %w", err)
}
return nil
}
-165
View File
@@ -1,165 +0,0 @@
package catalogue
import (
"context"
"errors"
"os"
"path/filepath"
"sync"
"sync/atomic"
"testing"
"time"
"git.fabledsword.com/bvandeusen/minstrel/internal/coverart"
)
const someMBID = "e21857d5-3256-4547-afb3-4b6ded592596"
// fakeSource is an artist and album source answering with fixed results.
type fakeSource struct {
thumb, fanart, cover []byte
err error
calls atomic.Int32
gate chan struct{} // when set, each fetch waits for it
lastRef coverart.AlbumRef
mu sync.Mutex
}
func (f *fakeSource) ID() string { return "fake" }
func (f *fakeSource) DisplayName() string { return "Fake" }
func (f *fakeSource) RequiresAPIKey() bool { return false }
func (f *fakeSource) DefaultEnabled() bool { return true }
func (f *fakeSource) Configure(_ coverart.ProviderSettings) error { return nil }
func (f *fakeSource) wait() {
f.calls.Add(1)
if f.gate != nil {
<-f.gate
}
}
func (f *fakeSource) FetchArtistArt(_ context.Context, _ coverart.ArtistRef) ([]byte, []byte, error) {
f.wait()
return f.thumb, f.fanart, f.err
}
func (f *fakeSource) FetchAlbumCover(_ context.Context, ref coverart.AlbumRef) ([]byte, error) {
f.wait()
f.mu.Lock()
f.lastRef = ref
f.mu.Unlock()
return f.cover, f.err
}
// groupSource can also answer by release group, as the Cover Art Archive does.
type groupSource struct{ *fakeSource }
func (g groupSource) FetchReleaseGroupCover(_ context.Context, _ string) ([]byte, error) {
g.wait()
return []byte("by-group"), nil
}
type sources struct {
artists []coverart.ArtistArtProvider
albums []coverart.AlbumCoverProvider
}
func (s sources) EnabledArtistProviders() []coverart.ArtistArtProvider { return s.artists }
func (s sources) EnabledAlbumProviders() []coverart.AlbumCoverProvider { return s.albums }
func TestArt_FetchesOnceAndKeepsIt(t *testing.T) {
src := &fakeSource{fanart: []byte("backdrop")}
a := NewArt(t.TempDir(), sources{artists: []coverart.ArtistArtProvider{src}})
body, err := a.Artist(context.Background(), someMBID, "Gorillaz")
if err != nil || string(body) != "backdrop" {
t.Fatalf("with only a backdrop, use it: %q, %v", body, err)
}
if body, _ := a.Artist(context.Background(), someMBID, "Gorillaz"); string(body) != "backdrop" || src.calls.Load() != 1 {
t.Errorf("the second ask should come from disk: %q after %d calls", body, src.calls.Load())
}
}
func TestArt_NobodyHasOneIsBelievedForAWeek(t *testing.T) {
src := &fakeSource{err: coverart.ErrNotFound}
a := NewArt(t.TempDir(), sources{artists: []coverart.ArtistArtProvider{src}})
if _, err := a.Artist(context.Background(), someMBID, "x"); !errors.Is(err, ErrNoArt) {
t.Fatalf("err = %v, want ErrNoArt", err)
}
if _, err := a.Artist(context.Background(), someMBID, "x"); !errors.Is(err, ErrNoArt) || src.calls.Load() != 1 {
t.Errorf("a definite miss is remembered: %d calls", src.calls.Load())
}
a.now = func() time.Time { return time.Now().Add(NoArtRetry + time.Hour) }
src.err, src.thumb = nil, []byte("thumb")
if body, err := a.Artist(context.Background(), someMBID, "x"); err != nil || string(body) != "thumb" {
t.Errorf("after a week the sources are asked again: %q, %v", body, err)
}
}
func TestArt_AnUnreachableSourceIsAskedAgain(t *testing.T) {
src := &fakeSource{err: coverart.ErrTransient}
a := NewArt(t.TempDir(), sources{artists: []coverart.ArtistArtProvider{src}})
for range 2 {
if _, err := a.Artist(context.Background(), someMBID, "x"); !errors.Is(err, ErrNoArt) {
t.Fatalf("err = %v", err)
}
}
if src.calls.Load() != 2 {
t.Errorf("calls = %d: an outage must not be written down as 'no art'", src.calls.Load())
}
}
func TestArt_ReleaseGroupAsksByGroupThenByName(t *testing.T) {
byName := &fakeSource{cover: []byte("by-name")}
caa := groupSource{&fakeSource{}}
a := NewArt(t.TempDir(), sources{albums: []coverart.AlbumCoverProvider{caa, byName}})
if body, _ := a.ReleaseGroup(context.Background(), someMBID, "Gorillaz", "Demon Days"); string(body) != "by-group" {
t.Errorf("a source that answers by group is asked by group: %q", body)
}
other := "0c4d5b5f-2f3c-3d31-9c4e-4b4f6b4c1a01"
a = NewArt(t.TempDir(), sources{albums: []coverart.AlbumCoverProvider{byName}})
if body, _ := a.ReleaseGroup(context.Background(), other, "Gorillaz", "Demon Days"); string(body) != "by-name" {
t.Errorf("by name: %q", body)
}
if byName.lastRef.MBID != "" || byName.lastRef.AlbumTitle != "Demon Days" {
t.Errorf("a release-group id is not a release id, so it is not passed as one: %+v", byName.lastRef)
}
}
func TestArt_ConcurrentAsksShareOneFetch(t *testing.T) {
src := &fakeSource{thumb: []byte("t"), gate: make(chan struct{})}
a := NewArt(t.TempDir(), sources{artists: []coverart.ArtistArtProvider{src}})
var wg sync.WaitGroup
for range 5 {
wg.Add(1)
go func() {
defer wg.Done()
if body, err := a.Artist(context.Background(), someMBID, "x"); err != nil || string(body) != "t" {
t.Errorf("%q, %v", body, err)
}
}()
}
// Let the fetch through once every caller has had time to queue on it.
time.Sleep(50 * time.Millisecond)
close(src.gate)
wg.Wait()
if src.calls.Load() != 1 {
t.Errorf("calls = %d, want one fetch for five asks", src.calls.Load())
}
}
func TestArt_RefusesWhatIsNotAnMBID(t *testing.T) {
dir := t.TempDir()
a := NewArt(dir, sources{artists: []coverart.ArtistArtProvider{&fakeSource{thumb: []byte("t")}}})
if _, err := a.Artist(context.Background(), "../../etc/passwd", "x"); !errors.Is(err, ErrNoArt) {
t.Fatalf("err = %v", err)
}
if _, err := os.Stat(filepath.Join(dir, "catalogue-art")); !os.IsNotExist(err) {
t.Error("nothing should be written for a refused id")
}
if _, err := NewArt(dir, nil).Artist(context.Background(), someMBID, "x"); !errors.Is(err, ErrNoArt) {
t.Errorf("no sources: err = %v", err)
}
}
+6 -211
View File
@@ -1,7 +1,6 @@
// Package catalogue is Minstrel's own record of what artists have released,
// read from MusicBrainz and kept in the database. The wanted
// list, search and import read it; MusicBrainz is asked only when what is
// held is missing or old.
// Package catalogue keeps MusicBrainz's releases and track lists for the
// albums the library holds. The duplicate resolver reads them to tell a song
// filed in the wrong slot from one legitimately repeated.
package catalogue
import (
@@ -9,34 +8,22 @@ import (
"errors"
"fmt"
"log/slog"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgtype"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/musicbrainz"
)
const (
// DiscographyMaxAge is how long an artist's list of release groups is
// trusted before it is read again. New releases appear on this cadence.
DiscographyMaxAge = 7 * 24 * time.Hour
// ReleasesMaxAge is how long a release group's editions and track lists
// are trusted. They change rarely once a group is out.
ReleasesMaxAge = 30 * 24 * time.Hour
)
// ErrNotFound: MusicBrainz has no artist or release group under that id.
// ErrNotFound: MusicBrainz has no release group under that id.
var ErrNotFound = errors.New("catalogue: not in MusicBrainz")
// Service reads and refreshes the catalogue.
// Service reads release groups from MusicBrainz into the catalogue.
type Service struct {
pool *pgxpool.Pool
mb *musicbrainz.Client
logger *slog.Logger
now func() time.Time
}
// New makes a Service. A nil client talks to the public MusicBrainz.
@@ -47,91 +34,7 @@ func New(pool *pgxpool.Pool, mb *musicbrainz.Client, logger *slog.Logger) *Servi
if logger == nil {
logger = slog.Default()
}
return &Service{pool: pool, mb: mb, logger: logger, now: time.Now}
}
// MusicBrainz is the client the service asks, for callers that search.
func (s *Service) MusicBrainz() *musicbrainz.Client { return s.mb }
// Discography is an artist and the release groups credited to them.
type Discography struct {
Artist dbq.MbArtist
Groups []dbq.MbReleaseGroup
// Stale: MusicBrainz could not be asked, so this is the last copy read.
Stale bool
}
// Discography returns the artist's release groups, reading them from
// MusicBrainz first when they have never been read or are older than
// DiscographyMaxAge. When MusicBrainz is unreachable, a copy already held is
// returned marked Stale; with nothing held, the error is returned.
func (s *Service) Discography(ctx context.Context, artistMBID string) (Discography, error) {
return s.DiscographyWithin(ctx, artistMBID, DiscographyMaxAge)
}
// DiscographyWithin is Discography with the age past which the held copy is
// read again: a followed artist is checked for new releases more often than
// one the catalogue merely knows.
func (s *Service) DiscographyWithin(ctx context.Context, artistMBID string, maxAge time.Duration) (Discography, error) {
q := dbq.New(s.pool)
artist, err := q.GetMBArtist(ctx, artistMBID)
held := err == nil && artist.DiscographyRefreshedAt.Valid
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
return Discography{}, fmt.Errorf("catalogue: read artist: %w", err)
}
stale := false
if !held || s.now().Sub(artist.DiscographyRefreshedAt.Time) > maxAge {
if rerr := s.RefreshArtist(ctx, artistMBID); rerr != nil {
if !held || !errors.Is(rerr, musicbrainz.ErrUnreachable) {
return Discography{}, rerr
}
stale = true
}
if artist, err = q.GetMBArtist(ctx, artistMBID); err != nil {
return Discography{}, fmt.Errorf("catalogue: read artist: %w", err)
}
}
groups, err := q.ListMBArtistReleaseGroups(ctx, artistMBID)
if err != nil {
return Discography{}, fmt.Errorf("catalogue: read discography: %w", err)
}
return Discography{Artist: artist, Groups: groups, Stale: stale}, nil
}
// RefreshArtist reads the artist and every release group credited to them,
// and replaces the held discography in one transaction.
func (s *Service) RefreshArtist(ctx context.Context, artistMBID string) error {
artist, err := s.mb.Artist(ctx, artistMBID)
if err != nil {
return mbErr(err)
}
groups, err := s.mb.ArtistReleaseGroups(ctx, artistMBID)
if err != nil {
return mbErr(err)
}
return pgx.BeginFunc(ctx, s.pool, func(tx pgx.Tx) error {
q := dbq.New(tx)
if err := q.UpsertMBArtist(ctx, dbq.UpsertMBArtistParams{
Mbid: artist.MBID, Name: artist.Name, SortName: artist.SortName,
Disambiguation: artist.Disambiguation, ArtistType: artist.Type, Country: artist.Country,
}); err != nil {
return fmt.Errorf("catalogue: save artist: %w", err)
}
if err := q.DeleteMBArtistReleaseGroups(ctx, artist.MBID); err != nil {
return fmt.Errorf("catalogue: clear discography: %w", err)
}
for _, g := range groups {
if err := upsertGroup(ctx, q, g); err != nil {
return err
}
if err := q.AddMBArtistReleaseGroup(ctx, dbq.AddMBArtistReleaseGroupParams{
ArtistMbid: artist.MBID, ReleaseGroupMbid: g.MBID,
}); err != nil {
return fmt.Errorf("catalogue: link release group: %w", err)
}
}
return q.MarkMBArtistDiscographyRefreshed(ctx, artist.MBID)
})
return &Service{pool: pool, mb: mb, logger: logger}
}
func upsertGroup(ctx context.Context, q *dbq.Queries, g musicbrainz.ReleaseGroup) error {
@@ -148,60 +51,6 @@ func upsertGroup(ctx context.Context, q *dbq.Queries, g musicbrainz.ReleaseGroup
return nil
}
// Edition is one release of a group with its track list.
type Edition struct {
Release dbq.MbRelease
Tracks []dbq.MbReleaseTrack
}
// Editions is a release group and its releases.
type Editions struct {
Group dbq.MbReleaseGroup
Editions []Edition
Stale bool
}
// Releases returns a release group's releases with their track lists, read
// from MusicBrainz first when never read or older than ReleasesMaxAge. Like
// Discography, an unreachable MusicBrainz serves the held copy as Stale.
func (s *Service) Releases(ctx context.Context, groupMBID string) (Editions, error) {
q := dbq.New(s.pool)
group, err := q.GetMBReleaseGroup(ctx, groupMBID)
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
return Editions{}, fmt.Errorf("catalogue: read release group: %w", err)
}
held := err == nil && group.ReleasesRefreshedAt.Valid
stale := false
if !held || s.now().Sub(group.ReleasesRefreshedAt.Time) > ReleasesMaxAge {
if rerr := s.RefreshReleaseGroup(ctx, groupMBID); rerr != nil {
if !held || !errors.Is(rerr, musicbrainz.ErrUnreachable) {
return Editions{}, rerr
}
stale = true
}
if group, err = q.GetMBReleaseGroup(ctx, groupMBID); err != nil {
return Editions{}, fmt.Errorf("catalogue: read release group: %w", err)
}
}
releases, err := q.ListMBReleasesForGroup(ctx, groupMBID)
if err != nil {
return Editions{}, fmt.Errorf("catalogue: read releases: %w", err)
}
tracks, err := q.ListMBReleaseTracksForGroup(ctx, groupMBID)
if err != nil {
return Editions{}, fmt.Errorf("catalogue: read track lists: %w", err)
}
byRelease := make(map[string][]dbq.MbReleaseTrack, len(releases))
for _, t := range tracks {
byRelease[t.ReleaseMbid] = append(byRelease[t.ReleaseMbid], t)
}
out := Editions{Group: group, Stale: stale}
for _, r := range releases {
out.Editions = append(out.Editions, Edition{Release: r, Tracks: byRelease[r.Mbid]})
}
return out, nil
}
// RefreshReleaseGroup reads a group's releases and track lists and replaces
// the held copy in one transaction. A group not yet held is looked up first.
func (s *Service) RefreshReleaseGroup(ctx context.Context, groupMBID string) error {
@@ -254,60 +103,6 @@ func (s *Service) RefreshReleaseGroup(ctx context.Context, groupMBID string) err
})
}
// RefreshDue re-reads up to limit discographies that are older than
// DiscographyMaxAge, never-read first. It stops at the first unreachable
// MusicBrainz, since the rest would fail the same way.
func (s *Service) RefreshDue(ctx context.Context, limit int) (refreshed int, err error) {
due, err := dbq.New(s.pool).ListMBArtistsDueForRefresh(ctx, dbq.ListMBArtistsDueForRefreshParams{
Before: pgtype.Timestamptz{Time: s.now().Add(-DiscographyMaxAge), Valid: true},
MaxRows: int32(limit),
})
if err != nil {
return 0, fmt.Errorf("catalogue: list due artists: %w", err)
}
for _, mbid := range due {
if err := s.RefreshArtist(ctx, mbid); err != nil {
if errors.Is(err, musicbrainz.ErrUnreachable) || ctx.Err() != nil {
return refreshed, err
}
// One artist MusicBrainz cannot describe (merged away, say)
// must not stop the rest.
s.logger.Warn("catalogue: refresh artist", "mbid", mbid, "err", err)
continue
}
refreshed++
}
return refreshed, nil
}
// RefreshInterval is how often Run looks for discographies due a re-read,
// and RefreshBatch how many it reads per look: 20 artists is about a minute
// of MusicBrainz's budget.
const (
RefreshInterval = time.Hour
RefreshBatch = 20
)
// Run refreshes due discographies every RefreshInterval until ctx ends.
func (s *Service) Run(ctx context.Context) {
ticker := time.NewTicker(RefreshInterval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
}
n, err := s.RefreshDue(ctx, RefreshBatch)
switch {
case err != nil && ctx.Err() == nil:
s.logger.Warn("catalogue: refresh pass stopped", "refreshed", n, "err", err)
case n > 0:
s.logger.Info("catalogue: refresh pass complete", "refreshed", n)
}
}
}
// mbErr maps MusicBrainz's not-found to the catalogue's; anything else passes
// through so callers can tell an unreachable MusicBrainz apart.
func mbErr(err error) error {
+32 -128
View File
@@ -9,14 +9,13 @@ import (
"net/http"
"net/http/httptest"
"os"
"strings"
"sync/atomic"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/db"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/dbtest"
"git.fabledsword.com/bvandeusen/minstrel/internal/musicbrainz"
)
@@ -44,15 +43,13 @@ func newPool(t *testing.T) *pgxpool.Pool {
const (
gorillaz = "e21857d5-3256-4547-afb3-4b6ded592596"
deLaSoul = "4f2a6b9e-9ae6-4f34-9a41-3e9f2f7c4c1d"
demonDays = "0c4d5b5f-2f3c-3d31-9c4e-4b4f6b4c1a01"
feelGood = "0c4d5b5f-2f3c-3d31-9c4e-4b4f6b4c1a02" // credited to both
demonDaysR = "1a2b3c4d-0000-4000-8000-000000000001"
demonDaysJ = "1a2b3c4d-0000-4000-8000-000000000002"
)
// fakeMB is a MusicBrainz with two artists, a collaboration between them,
// and one album with two editions. down makes every request a 503.
// fakeMB is a MusicBrainz with one album in two editions. down makes every
// request a 503.
type fakeMB struct {
srv *httptest.Server
calls atomic.Int32
@@ -70,21 +67,6 @@ func newFakeMB(t *testing.T) *fakeMB {
}
q := r.URL.Query()
switch {
case r.URL.Path == "/artist/"+gorillaz:
_, _ = fmt.Fprintf(w, `{"id": %q, "name": "Gorillaz", "sort-name": "Gorillaz", "type": "Group", "country": "GB"}`, gorillaz)
case r.URL.Path == "/artist/"+deLaSoul:
_, _ = fmt.Fprintf(w, `{"id": %q, "name": "De La Soul", "type": "Group", "country": "US"}`, deLaSoul)
case r.URL.Path == "/release-group" && q.Get("artist") == gorillaz:
_, _ = fmt.Fprintf(w, `{"release-group-count": 2, "release-groups": [
{"id": %q, "title": "Demon Days", "primary-type": "Album", "first-release-date": "2005-05-11",
"artist-credit": [{"name": "Gorillaz", "artist": {"id": %q}}]},
{"id": %q, "title": "Feel Good Inc.", "primary-type": "Single", "secondary-types": [], "first-release-date": "2005-05-09",
"artist-credit": [{"name": "Gorillaz", "joinphrase": " feat. ", "artist": {"id": %q}}, {"name": "De La Soul", "artist": {"id": %q}}]}
]}`, demonDays, gorillaz, feelGood, gorillaz, deLaSoul)
case r.URL.Path == "/release-group" && q.Get("artist") == deLaSoul:
_, _ = fmt.Fprintf(w, `{"release-group-count": 1, "release-groups": [
{"id": %q, "title": "Feel Good Inc.", "primary-type": "Single", "first-release-date": "2005-05-09"}
]}`, feelGood)
case r.URL.Path == "/release-group/"+demonDays:
_, _ = fmt.Fprintf(w, `{"id": %q, "title": "Demon Days", "primary-type": "Album", "first-release-date": "2005-05-11",
"artist-credit": [{"name": "Gorillaz", "artist": {"id": %q}}]}`, demonDays, gorillaz)
@@ -110,138 +92,60 @@ func (f *fakeMB) client() *musicbrainz.Client {
return &musicbrainz.Client{BaseURL: f.srv.URL, Pace: &musicbrainz.Pacer{}, Retries: -1}
}
func TestDiscography_ReadsOnceThenServesTheHeldCopy_Integration(t *testing.T) {
ctx := context.Background()
mb := newFakeMB(t)
s := New(newPool(t), mb.client(), nil)
d, err := s.Discography(ctx, gorillaz)
if err != nil {
t.Fatal(err)
}
if d.Artist.Name != "Gorillaz" || d.Artist.Country != "GB" || len(d.Groups) != 2 || d.Stale {
t.Fatalf("discography = %+v", d)
}
// Oldest first: the single came out two days before the album.
if d.Groups[0].Mbid != feelGood || d.Groups[1].PrimaryType != "Album" {
t.Errorf("order = %s, %s", d.Groups[0].Title, d.Groups[1].Title)
}
if d.Groups[0].ArtistCredit != "Gorillaz feat. De La Soul" {
t.Errorf("credit = %q", d.Groups[0].ArtistCredit)
}
before := mb.calls.Load()
if _, err := s.Discography(ctx, gorillaz); err != nil {
t.Fatal(err)
}
if mb.calls.Load() != before {
t.Error("a fresh discography should be served without asking MusicBrainz")
}
// The collaboration belongs to both artists.
dls, err := s.Discography(ctx, deLaSoul)
if err != nil {
t.Fatal(err)
}
if len(dls.Groups) != 1 || dls.Groups[0].Mbid != feelGood {
t.Errorf("De La Soul's discography = %+v", dls.Groups)
}
if again, _ := s.Discography(ctx, gorillaz); len(again.Groups) != 2 {
t.Error("refreshing one artist must not take a shared group from the other")
}
}
func TestDiscography_UnreachableServesStaleOrFails_Integration(t *testing.T) {
func TestRefreshReleaseGroup_ReadsEditionsAndTrackLists_Integration(t *testing.T) {
ctx := context.Background()
mb := newFakeMB(t)
pool := newPool(t)
s := New(pool, mb.client(), nil)
mb.down.Store(true)
if _, err := s.Discography(ctx, gorillaz); !errors.Is(err, musicbrainz.ErrUnreachable) {
t.Fatalf("nothing held and MusicBrainz down: err = %v, want ErrUnreachable", err)
}
mb.down.Store(false)
if _, err := s.Discography(ctx, gorillaz); err != nil {
t.Fatal(err)
}
// A week and a day later, MusicBrainz is down: the held copy is served.
s.now = func() time.Time { return time.Now().Add(DiscographyMaxAge + 24*time.Hour) }
mb.down.Store(true)
d, err := s.Discography(ctx, gorillaz)
if err != nil || !d.Stale || len(d.Groups) != 2 {
t.Fatalf("stale copy: %+v, %v", d, err)
}
}
func TestDiscography_UnknownArtistIsNotFound_Integration(t *testing.T) {
s := New(newPool(t), newFakeMB(t).client(), nil)
if _, err := s.Discography(context.Background(), "00000000-0000-4000-8000-000000000000"); !errors.Is(err, ErrNotFound) {
t.Fatalf("err = %v, want ErrNotFound", err)
}
}
func TestReleases_ReadsEditionsAndTrackLists_Integration(t *testing.T) {
ctx := context.Background()
mb := newFakeMB(t)
s := New(newPool(t), mb.client(), nil)
q := dbq.New(pool)
// The group isn't held yet: it is looked up first.
e, err := s.Releases(ctx, demonDays)
if err := s.RefreshReleaseGroup(ctx, demonDays); err != nil {
t.Fatal(err)
}
group, err := q.GetMBReleaseGroup(ctx, demonDays)
if err != nil {
t.Fatal(err)
}
if e.Group.Title != "Demon Days" || len(e.Editions) != 2 {
t.Fatalf("editions = %+v", e)
if group.Title != "Demon Days" || !group.ReleasesRefreshedAt.Valid {
t.Fatalf("group = %+v", group)
}
official := e.Editions[0]
if official.Release.Mbid != demonDaysR || official.Release.TrackCount != 2 || official.Release.Formats != "CD" {
t.Errorf("the official edition should come first: %+v", official.Release)
}
if len(official.Tracks) != 2 || official.Tracks[1].RecordingMbid != "rec-lls" || official.Tracks[1].LengthMs != 190000 {
t.Errorf("tracks = %+v", official.Tracks)
}
before := mb.calls.Load()
if _, err := s.Releases(ctx, demonDays); err != nil {
tracks, err := q.ListMBReleaseTracksForGroup(ctx, demonDays)
if err != nil {
t.Fatal(err)
}
if mb.calls.Load() != before {
t.Error("fresh editions should be served without asking MusicBrainz")
if len(tracks) != 3 {
t.Fatalf("want 3 tracks across both editions, got %+v", tracks)
}
var lls *dbq.MbReleaseTrack
for i := range tracks {
if tracks[i].RecordingMbid == "rec-lls" {
lls = &tracks[i]
}
}
if lls == nil || lls.ReleaseMbid != demonDaysR || lls.LengthMs != 190000 {
t.Errorf("Last Living Souls = %+v", lls)
}
// Refreshing replaces rather than adds.
if err := s.RefreshReleaseGroup(ctx, demonDays); err != nil {
t.Fatal(err)
}
if again, _ := s.Releases(ctx, demonDays); len(again.Editions) != 2 || len(again.Editions[0].Tracks) != 2 {
t.Errorf("after a refresh: %+v", again)
if again, _ := q.ListMBReleaseTracksForGroup(ctx, demonDays); len(again) != 3 {
t.Errorf("after a refresh: %d tracks, want 3", len(again))
}
}
func TestRefreshDue_ReadsWhatIsOldAndStopsWhenMusicBrainzIsDown_Integration(t *testing.T) {
func TestRefreshReleaseGroup_UnknownAndUnreachable_Integration(t *testing.T) {
ctx := context.Background()
mb := newFakeMB(t)
s := New(newPool(t), mb.client(), nil)
for _, a := range []string{gorillaz, deLaSoul} {
if err := s.RefreshArtist(ctx, a); err != nil {
t.Fatal(err)
}
}
if n, err := s.RefreshDue(ctx, 10); err != nil || n != 0 {
t.Fatalf("nothing is due yet: %d, %v", n, err)
}
s.now = func() time.Time { return time.Now().Add(DiscographyMaxAge + time.Hour) }
if n, err := s.RefreshDue(ctx, 1); err != nil || n != 1 {
t.Errorf("one due, limit one: %d, %v", n, err)
if err := s.RefreshReleaseGroup(ctx, "00000000-0000-4000-8000-000000000000"); !errors.Is(err, ErrNotFound) {
t.Errorf("unknown group: err = %v, want ErrNotFound", err)
}
mb.down.Store(true)
n, err := s.RefreshDue(ctx, 10)
if !errors.Is(err, musicbrainz.ErrUnreachable) || n != 0 {
t.Errorf("MusicBrainz down: %d, %v", n, err)
}
if !strings.Contains(err.Error(), "503") {
t.Errorf("the error should say what MusicBrainz answered: %v", err)
if err := s.RefreshReleaseGroup(ctx, demonDays); !errors.Is(err, musicbrainz.ErrUnreachable) {
t.Errorf("MusicBrainz down: err = %v, want ErrUnreachable", err)
}
}
+1 -1
View File
@@ -99,7 +99,7 @@ func Default() Config {
Log: LogConfig{Level: "info", Format: "text"},
Branding: BrandingConfig{
AppName: "Minstrel",
Description: "Self-hosted music server with Subsonic compatibility, smart shuffle, and music acquisition.",
Description: "Self-hosted music server with Subsonic compatibility and smart shuffle.",
},
Events: EventsConfig{
SessionTimeoutMinutes: 30,
-362
View File
@@ -1,362 +0,0 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.31.1
// source: acquisition.sql
package dbq
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const createAcquisitionService = `-- name: CreateAcquisitionService :one
INSERT INTO acquisition_services
(code, kind, protocol, label, category, timeout_ms, searches_per_hour)
VALUES
($1, $2, $3, $4,
$5, $6, $7)
ON CONFLICT (code) DO NOTHING
RETURNING code, kind, protocol, label, enabled, base_url, api_key, username, password, category, timeout_ms, serves_search, serves_downloads, serves_enrichment, local_path, remote_path, searches_per_hour, created_at, updated_at
`
type CreateAcquisitionServiceParams struct {
Code string
Kind string
Protocol string
Label string
Category string
TimeoutMs int32
SearchesPerHour int32
}
// ON CONFLICT so the caller can try the next code when this one is taken.
func (q *Queries) CreateAcquisitionService(ctx context.Context, arg CreateAcquisitionServiceParams) (AcquisitionService, error) {
row := q.db.QueryRow(ctx, createAcquisitionService,
arg.Code,
arg.Kind,
arg.Protocol,
arg.Label,
arg.Category,
arg.TimeoutMs,
arg.SearchesPerHour,
)
var i AcquisitionService
err := row.Scan(
&i.Code,
&i.Kind,
&i.Protocol,
&i.Label,
&i.Enabled,
&i.BaseUrl,
&i.ApiKey,
&i.Username,
&i.Password,
&i.Category,
&i.TimeoutMs,
&i.ServesSearch,
&i.ServesDownloads,
&i.ServesEnrichment,
&i.LocalPath,
&i.RemotePath,
&i.SearchesPerHour,
&i.CreatedAt,
&i.UpdatedAt,
)
return i, err
}
const deleteAcquisitionService = `-- name: DeleteAcquisitionService :execrows
DELETE FROM acquisition_services WHERE code = $1
`
func (q *Queries) DeleteAcquisitionService(ctx context.Context, code string) (int64, error) {
result, err := q.db.Exec(ctx, deleteAcquisitionService, code)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const getAcquisitionService = `-- name: GetAcquisitionService :one
SELECT code, kind, protocol, label, enabled, base_url, api_key, username, password, category, timeout_ms, serves_search, serves_downloads, serves_enrichment, local_path, remote_path, searches_per_hour, created_at, updated_at FROM acquisition_services WHERE code = $1
`
// The full row, credentials included. For building a client, never for a
// response.
func (q *Queries) GetAcquisitionService(ctx context.Context, code string) (AcquisitionService, error) {
row := q.db.QueryRow(ctx, getAcquisitionService, code)
var i AcquisitionService
err := row.Scan(
&i.Code,
&i.Kind,
&i.Protocol,
&i.Label,
&i.Enabled,
&i.BaseUrl,
&i.ApiKey,
&i.Username,
&i.Password,
&i.Category,
&i.TimeoutMs,
&i.ServesSearch,
&i.ServesDownloads,
&i.ServesEnrichment,
&i.LocalPath,
&i.RemotePath,
&i.SearchesPerHour,
&i.CreatedAt,
&i.UpdatedAt,
)
return i, err
}
const listAcquisitionServices = `-- name: ListAcquisitionServices :many
SELECT code, kind, protocol, label, enabled, base_url,
(api_key <> '')::boolean AS has_api_key,
(password <> '')::boolean AS has_password,
username, category, timeout_ms,
serves_search, serves_downloads, serves_enrichment,
local_path, remote_path, searches_per_hour,
created_at, updated_at
FROM acquisition_services
ORDER BY kind, label, code
`
type ListAcquisitionServicesRow struct {
Code string
Kind string
Protocol string
Label string
Enabled bool
BaseUrl string
HasApiKey bool
HasPassword bool
Username string
Category string
TimeoutMs int32
ServesSearch bool
ServesDownloads bool
ServesEnrichment bool
LocalPath string
RemotePath string
SearchesPerHour int32
CreatedAt pgtype.Timestamptz
UpdatedAt pgtype.Timestamptz
}
// Indexers and download clients.
// Returns no secret, only whether one is set: this feeds the settings page.
func (q *Queries) ListAcquisitionServices(ctx context.Context) ([]ListAcquisitionServicesRow, error) {
rows, err := q.db.Query(ctx, listAcquisitionServices)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListAcquisitionServicesRow
for rows.Next() {
var i ListAcquisitionServicesRow
if err := rows.Scan(
&i.Code,
&i.Kind,
&i.Protocol,
&i.Label,
&i.Enabled,
&i.BaseUrl,
&i.HasApiKey,
&i.HasPassword,
&i.Username,
&i.Category,
&i.TimeoutMs,
&i.ServesSearch,
&i.ServesDownloads,
&i.ServesEnrichment,
&i.LocalPath,
&i.RemotePath,
&i.SearchesPerHour,
&i.CreatedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listUsableAcquisitionServices = `-- name: ListUsableAcquisitionServices :many
SELECT code, kind, protocol, label, enabled, base_url, api_key, username, password, category, timeout_ms, serves_search, serves_downloads, serves_enrichment, local_path, remote_path, searches_per_hour, created_at, updated_at FROM acquisition_services
WHERE enabled AND base_url <> ''
AND CASE WHEN protocol = 'exchange' THEN
CASE $1::text
WHEN 'search' THEN serves_search
WHEN 'download' THEN serves_downloads
WHEN 'enrichment' THEN serves_enrichment
ELSE false
END
ELSE kind = $1::text
END
ORDER BY code
`
// Every enabled, addressed service doing one job. An exchange row does a job
// when its switch for it is on; any other row does the job its kind names.
// Ordered by code so a fan-out is deterministic.
func (q *Queries) ListUsableAcquisitionServices(ctx context.Context, job string) ([]AcquisitionService, error) {
rows, err := q.db.Query(ctx, listUsableAcquisitionServices, job)
if err != nil {
return nil, err
}
defer rows.Close()
var items []AcquisitionService
for rows.Next() {
var i AcquisitionService
if err := rows.Scan(
&i.Code,
&i.Kind,
&i.Protocol,
&i.Label,
&i.Enabled,
&i.BaseUrl,
&i.ApiKey,
&i.Username,
&i.Password,
&i.Category,
&i.TimeoutMs,
&i.ServesSearch,
&i.ServesDownloads,
&i.ServesEnrichment,
&i.LocalPath,
&i.RemotePath,
&i.SearchesPerHour,
&i.CreatedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const setAcquisitionServiceAPIKey = `-- name: SetAcquisitionServiceAPIKey :execrows
UPDATE acquisition_services SET api_key = $1, updated_at = now()
WHERE code = $2
`
type SetAcquisitionServiceAPIKeyParams struct {
ApiKey string
Code string
}
func (q *Queries) SetAcquisitionServiceAPIKey(ctx context.Context, arg SetAcquisitionServiceAPIKeyParams) (int64, error) {
result, err := q.db.Exec(ctx, setAcquisitionServiceAPIKey, arg.ApiKey, arg.Code)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const setAcquisitionServicePassword = `-- name: SetAcquisitionServicePassword :execrows
UPDATE acquisition_services SET password = $1, updated_at = now()
WHERE code = $2
`
type SetAcquisitionServicePasswordParams struct {
Password string
Code string
}
func (q *Queries) SetAcquisitionServicePassword(ctx context.Context, arg SetAcquisitionServicePasswordParams) (int64, error) {
result, err := q.db.Exec(ctx, setAcquisitionServicePassword, arg.Password, arg.Code)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const updateAcquisitionService = `-- name: UpdateAcquisitionService :one
UPDATE acquisition_services SET
label = $1,
enabled = $2,
base_url = $3,
username = $4,
category = $5,
timeout_ms = $6,
serves_search = $7,
serves_downloads = $8,
serves_enrichment = $9,
local_path = $10,
remote_path = $11,
searches_per_hour = $12,
updated_at = now()
WHERE code = $13
RETURNING code, kind, protocol, label, enabled, base_url, api_key, username, password, category, timeout_ms, serves_search, serves_downloads, serves_enrichment, local_path, remote_path, searches_per_hour, created_at, updated_at
`
type UpdateAcquisitionServiceParams struct {
Label string
Enabled bool
BaseUrl string
Username string
Category string
TimeoutMs int32
ServesSearch bool
ServesDownloads bool
ServesEnrichment bool
LocalPath string
RemotePath string
SearchesPerHour int32
Code string
}
// Everything on the settings form except the credentials, which have their
// own write-only calls.
func (q *Queries) UpdateAcquisitionService(ctx context.Context, arg UpdateAcquisitionServiceParams) (AcquisitionService, error) {
row := q.db.QueryRow(ctx, updateAcquisitionService,
arg.Label,
arg.Enabled,
arg.BaseUrl,
arg.Username,
arg.Category,
arg.TimeoutMs,
arg.ServesSearch,
arg.ServesDownloads,
arg.ServesEnrichment,
arg.LocalPath,
arg.RemotePath,
arg.SearchesPerHour,
arg.Code,
)
var i AcquisitionService
err := row.Scan(
&i.Code,
&i.Kind,
&i.Protocol,
&i.Label,
&i.Enabled,
&i.BaseUrl,
&i.ApiKey,
&i.Username,
&i.Password,
&i.Category,
&i.TimeoutMs,
&i.ServesSearch,
&i.ServesDownloads,
&i.ServesEnrichment,
&i.LocalPath,
&i.RemotePath,
&i.SearchesPerHour,
&i.CreatedAt,
&i.UpdatedAt,
)
return i, err
}
+2 -38
View File
@@ -11,42 +11,6 @@ import (
"github.com/jackc/pgx/v5/pgtype"
)
const downloadsStatus = `-- name: DownloadsStatus :one
SELECT COUNT(*) FILTER (WHERE ended_at IS NULL AND import_state IN ('', 'downloading', 'waiting', 'importing'))::bigint AS running,
COUNT(*) FILTER (WHERE ended_at IS NULL AND import_state IN ('review', 'failed'))::bigint AS needs_you,
MIN(COALESCE(import_updated_at, grabbed_at)) FILTER (WHERE ended_at IS NULL AND import_state IN ('review', 'failed'))::timestamptz AS needs_you_since,
COUNT(*) FILTER (WHERE imported_at > now() - INTERVAL '24 hours')::bigint AS imported_day,
COUNT(*) FILTER (WHERE ended_at > now() - INTERVAL '24 hours'
AND end_reason IN ('stalled', 'failed', 'gone', 'abandoned'))::bigint AS given_up_day
FROM grabs
WHERE ended_at IS NULL
OR ended_at > now() - INTERVAL '24 hours'
OR imported_at > now() - INTERVAL '24 hours'
`
type DownloadsStatusRow struct {
Running int64
NeedsYou int64
NeedsYouSince pgtype.Timestamptz
ImportedDay int64
GivenUpDay int64
}
// Downloads at a glance: running, waiting on a person (and since when), and
// the last day's imports and give-ups.
func (q *Queries) DownloadsStatus(ctx context.Context) (DownloadsStatusRow, error) {
row := q.db.QueryRow(ctx, downloadsStatus)
var i DownloadsStatusRow
err := row.Scan(
&i.Running,
&i.NeedsYou,
&i.NeedsYouSince,
&i.ImportedDay,
&i.GivenUpDay,
)
return i, err
}
const libraryStatus = `-- name: LibraryStatus :one
SELECT COUNT(*)::bigint AS tracks,
COUNT(DISTINCT album_id)::bigint AS albums,
@@ -109,8 +73,8 @@ type ListListeningNowRow struct {
AlbumTitle string
}
// The admin Overview's status sections: who is listening, what is
// downloading, what the library holds. Each is one cheap read.
// The admin Overview's status sections: who is listening and what the
// library holds. Each is one cheap read.
// Plays in progress: open play events still inside their track's length,
// with a few minutes' grace for a pause. A client that never says a play
// ended would otherwise read as listening until the 24-hour sweep closes it.
-6
View File
@@ -13,8 +13,6 @@ import (
const adminSummary = `-- name: AdminSummary :one
SELECT
(SELECT count(*) FROM requests WHERE status = 'pending')::bigint AS requests,
(SELECT min(requested_at) FROM requests WHERE status = 'pending')::timestamptz AS requests_since,
-- A broken track is one report however it was found: a listener flagged
-- it, or it failed to play. Counted by track, as the Reports
-- page lists them.
@@ -41,8 +39,6 @@ SELECT
`
type AdminSummaryRow struct {
Requests int64
RequestsSince pgtype.Timestamptz
Reports int64
ReportsSince pgtype.Timestamptz
Duplicates int64
@@ -62,8 +58,6 @@ func (q *Queries) AdminSummary(ctx context.Context) (AdminSummaryRow, error) {
row := q.db.QueryRow(ctx, adminSummary)
var i AdminSummaryRow
err := row.Scan(
&i.Requests,
&i.RequestsSince,
&i.Reports,
&i.ReportsSince,
&i.Duplicates,
-35
View File
@@ -267,22 +267,6 @@ func (q *Queries) GetAlbumsByIDs(ctx context.Context, dollar_1 []pgtype.UUID) ([
return items, nil
}
const getReleaseGroupForReleaseMbid = `-- name: GetReleaseGroupForReleaseMbid :one
SELECT release_group_mbid::text
FROM albums
WHERE mbid = $1 AND release_group_mbid IS NOT NULL
LIMIT 1
`
// The release group the library already knows for a release id, so a
// request carrying a release id can be repaired without asking MusicBrainz.
func (q *Queries) GetReleaseGroupForReleaseMbid(ctx context.Context, mbid *string) (string, error) {
row := q.db.QueryRow(ctx, getReleaseGroupForReleaseMbid, mbid)
var release_group_mbid string
err := row.Scan(&release_group_mbid)
return release_group_mbid, err
}
const listAlbumsAlphaByArtist = `-- name: ListAlbumsAlphaByArtist :many
SELECT albums.id, albums.title, albums.sort_title, albums.artist_id, albums.release_date, albums.mbid, albums.cover_art_path, albums.created_at, albums.updated_at, albums.cover_art_source, albums.cover_art_sources_version, albums.release_group_mbid, artists.sort_name AS artist_sort_name
FROM albums
@@ -1015,25 +999,6 @@ func (q *Queries) SetAlbumReleaseGroupMbidIfNull(ctx context.Context, arg SetAlb
return err
}
const setReleaseGroupForReleaseMbidIfNull = `-- name: SetReleaseGroupForReleaseMbidIfNull :exec
UPDATE albums
SET release_group_mbid = $1::text,
updated_at = now()
WHERE mbid = $2::text AND release_group_mbid IS NULL
`
type SetReleaseGroupForReleaseMbidIfNullParams struct {
ReleaseGroupMbid string
ReleaseMbid string
}
// Cache a MusicBrainz-resolved release group onto the album row that
// holds the release, filling only a NULL.
func (q *Queries) SetReleaseGroupForReleaseMbidIfNull(ctx context.Context, arg SetReleaseGroupForReleaseMbidIfNullParams) error {
_, err := q.db.Exec(ctx, setReleaseGroupForReleaseMbidIfNull, arg.ReleaseGroupMbid, arg.ReleaseMbid)
return err
}
const upsertAlbum = `-- name: UpsertAlbum :one
INSERT INTO albums (title, sort_title, artist_id, release_date, mbid, cover_art_path, release_group_mbid)
VALUES ($1, $2, $3, $4, $5, $6, $7)
@@ -1,228 +0,0 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.31.1
// source: candidate_artist_tags.sql
package dbq
import (
"context"
)
const countCandidateArtistTagCoverage = `-- name: CountCandidateArtistTagCoverage :one
SELECT count(*)::bigint AS processed,
count(*) FILTER (WHERE tag_source <> 'none')::bigint AS with_tags
FROM candidate_artist_tag_state
`
type CountCandidateArtistTagCoverageRow struct {
Processed int64
WithTags int64
}
// Operator-facing coverage: how many distinct candidates have been processed,
// and how many of those actually yielded tags. The gap is the honest ceiling
// from the task — obscure artists with no MBID presence or no upstream tags
// stay thin no matter how long the worker runs, and that is worth being able
// to see rather than inferring from a silent surface.
func (q *Queries) CountCandidateArtistTagCoverage(ctx context.Context) (CountCandidateArtistTagCoverageRow, error) {
row := q.db.QueryRow(ctx, countCandidateArtistTagCoverage)
var i CountCandidateArtistTagCoverageRow
err := row.Scan(&i.Processed, &i.WithTags)
return i, err
}
const deleteCandidateArtistTags = `-- name: DeleteCandidateArtistTags :exec
DELETE FROM candidate_artist_tags WHERE candidate_mbid = $1
`
// Clear a candidate's cached tags before rewriting (atomic replace by the
// caller, same shape as DeleteTrackTags).
func (q *Queries) DeleteCandidateArtistTags(ctx context.Context, candidateMbid string) error {
_, err := q.db.Exec(ctx, deleteCandidateArtistTags, candidateMbid)
return err
}
const gcDeleteOrphanedCandidateArtistTagState = `-- name: GcDeleteOrphanedCandidateArtistTagState :execrows
DELETE FROM candidate_artist_tag_state s
WHERE NOT EXISTS (
SELECT 1 FROM artist_similarity_unmatched u
WHERE u.candidate_mbid = s.candidate_mbid
)
OR EXISTS (SELECT 1 FROM artists a WHERE a.mbid = s.candidate_mbid)
`
// Same sweep for the bookkeeping rows. Kept as a separate statement rather
// than a cascade: the two tables are independent by design (a 'none' outcome
// has state but no tags), so neither can be the parent of the other.
func (q *Queries) GcDeleteOrphanedCandidateArtistTagState(ctx context.Context) (int64, error) {
result, err := q.db.Exec(ctx, gcDeleteOrphanedCandidateArtistTagState)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const gcDeleteOrphanedCandidateArtistTags = `-- name: GcDeleteOrphanedCandidateArtistTags :execrows
DELETE FROM candidate_artist_tags t
WHERE NOT EXISTS (
SELECT 1 FROM artist_similarity_unmatched u
WHERE u.candidate_mbid = t.candidate_mbid
)
OR EXISTS (SELECT 1 FROM artists a WHERE a.mbid = t.candidate_mbid)
`
// Drops cached tags for candidates that no longer appear in the similarity
// feed, or that have since been added to the library (their tags now live in
// track_tags). The feed is refetched periodically and churns, so without this
// the cache only ever grows.
func (q *Queries) GcDeleteOrphanedCandidateArtistTags(ctx context.Context) (int64, error) {
result, err := q.db.Exec(ctx, gcDeleteOrphanedCandidateArtistTags)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const insertCandidateArtistTag = `-- name: InsertCandidateArtistTag :exec
INSERT INTO candidate_artist_tags (candidate_mbid, tag, weight)
VALUES ($1, $2, $3)
ON CONFLICT (candidate_mbid, tag)
DO UPDATE SET weight = GREATEST(candidate_artist_tags.weight, EXCLUDED.weight)
`
type InsertCandidateArtistTagParams struct {
CandidateMbid string
Tag string
Weight float64
}
// Upsert one (candidate, tag); keep the stronger weight when two providers
// agree on a tag with different folksonomy strengths.
func (q *Queries) InsertCandidateArtistTag(ctx context.Context, arg InsertCandidateArtistTagParams) error {
_, err := q.db.Exec(ctx, insertCandidateArtistTag, arg.CandidateMbid, arg.Tag, arg.Weight)
return err
}
const listCandidateArtistTagsForMbids = `-- name: ListCandidateArtistTagsForMbids :many
SELECT candidate_mbid, tag, weight
FROM candidate_artist_tags
WHERE candidate_mbid = ANY($1::text[])
`
// Cached tags for a set of candidates, for the taste-overlap ranking.
// One row per (candidate, tag).
func (q *Queries) ListCandidateArtistTagsForMbids(ctx context.Context, dollar_1 []string) ([]CandidateArtistTag, error) {
rows, err := q.db.Query(ctx, listCandidateArtistTagsForMbids, dollar_1)
if err != nil {
return nil, err
}
defer rows.Close()
var items []CandidateArtistTag
for rows.Next() {
var i CandidateArtistTag
if err := rows.Scan(&i.CandidateMbid, &i.Tag, &i.Weight); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listCandidateArtistsMissingTags = `-- name: ListCandidateArtistsMissingTags :many
SELECT u.candidate_mbid,
coalesce(max(u.candidate_name), '')::text AS candidate_name,
sum(u.score)::float8 AS total_score
FROM artist_similarity_unmatched u
LEFT JOIN candidate_artist_tag_state s ON s.candidate_mbid = u.candidate_mbid
WHERE NOT EXISTS (SELECT 1 FROM artists a WHERE a.mbid = u.candidate_mbid)
AND (
s.candidate_mbid IS NULL
OR (s.tag_source = 'none' AND s.tag_sources_version < $1)
)
GROUP BY u.candidate_mbid
ORDER BY total_score DESC, u.candidate_mbid
LIMIT $2
`
type ListCandidateArtistsMissingTagsParams struct {
TagSourcesVersion int32
Limit int32
}
type ListCandidateArtistsMissingTagsRow struct {
CandidateMbid string
CandidateName string
TotalScore float64
}
// Folksonomy tags for out-of-library Discover candidates. Parallel to
// track_tags.sql, but keyed by MBID because the artist has no local row. See
// 0050_candidate_artist_tags.up.sql for why the bookkeeping is its own table.
// Candidates eligible for tag enrichment: never processed (no state row) or
// settled 'none' under an older provider version.
//
// artist_similarity_unmatched holds one row per (seed, candidate, source), so
// this GROUPs to one row per candidate — enriching the same MBID once per seed
// that pointed at it would multiply the API calls for no gain.
//
// ORDER BY summed similarity DESC is the load-bearing part. The candidate pool
// is O(library artists x neighbours per artist) — thousands — and MusicBrainz
// allows ~1 req/s, so it can NEVER be fully enriched in one pass. Draining in
// strength order means the candidates most likely to actually reach a user's
// deck get tags first, and the long tail fills in over subsequent ticks
// instead of starving behind it.
//
// Already-in-library candidates are skipped: they have an artists row, so
// their tags belong in track_tags, and the suggestion query filters them out
// anyway. $1 = current tag_sources_version, $2 = limit.
// candidate_name is coalesced to the empty string so it lands non-nullable in
// Go: the name is only a Last.fm lookup key, and empty simply means "MBID-keyed
// providers only", which the provider chain already handles. max() is an
// arbitrary-but-deterministic pick when several seeds spell one MBID
// differently.
func (q *Queries) ListCandidateArtistsMissingTags(ctx context.Context, arg ListCandidateArtistsMissingTagsParams) ([]ListCandidateArtistsMissingTagsRow, error) {
rows, err := q.db.Query(ctx, listCandidateArtistsMissingTags, arg.TagSourcesVersion, arg.Limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListCandidateArtistsMissingTagsRow
for rows.Next() {
var i ListCandidateArtistsMissingTagsRow
if err := rows.Scan(&i.CandidateMbid, &i.CandidateName, &i.TotalScore); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const setCandidateArtistTagState = `-- name: SetCandidateArtistTagState :exec
INSERT INTO candidate_artist_tag_state (candidate_mbid, tag_source, tag_sources_version)
VALUES ($1, $2, $3)
ON CONFLICT (candidate_mbid) DO UPDATE
SET tag_source = EXCLUDED.tag_source,
tag_sources_version = EXCLUDED.tag_sources_version,
updated_at = now()
`
type SetCandidateArtistTagStateParams struct {
CandidateMbid string
TagSource string
TagSourcesVersion int32
}
// Stamp the enrichment outcome so the drainer skips settled candidates.
// $2 = 'musicbrainz' | 'lastfm' | 'mixed' | 'none', $3 = current version.
func (q *Queries) SetCandidateArtistTagState(ctx context.Context, arg SetCandidateArtistTagStateParams) error {
_, err := q.db.Exec(ctx, setCandidateArtistTagState, arg.CandidateMbid, arg.TagSource, arg.TagSourcesVersion)
return err
}
+4 -205
View File
@@ -7,35 +7,8 @@ package dbq
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const addMBArtistReleaseGroup = `-- name: AddMBArtistReleaseGroup :exec
INSERT INTO mb_artist_release_groups (artist_mbid, release_group_mbid)
VALUES ($1, $2)
ON CONFLICT DO NOTHING
`
type AddMBArtistReleaseGroupParams struct {
ArtistMbid string
ReleaseGroupMbid string
}
func (q *Queries) AddMBArtistReleaseGroup(ctx context.Context, arg AddMBArtistReleaseGroupParams) error {
_, err := q.db.Exec(ctx, addMBArtistReleaseGroup, arg.ArtistMbid, arg.ReleaseGroupMbid)
return err
}
const deleteMBArtistReleaseGroups = `-- name: DeleteMBArtistReleaseGroups :exec
DELETE FROM mb_artist_release_groups WHERE artist_mbid = $1
`
func (q *Queries) DeleteMBArtistReleaseGroups(ctx context.Context, artistMbid string) error {
_, err := q.db.Exec(ctx, deleteMBArtistReleaseGroups, artistMbid)
return err
}
const deleteMBReleasesForGroup = `-- name: DeleteMBReleasesForGroup :exec
DELETE FROM mb_releases WHERE release_group_mbid = $1
`
@@ -46,27 +19,6 @@ func (q *Queries) DeleteMBReleasesForGroup(ctx context.Context, releaseGroupMbid
return err
}
const getMBArtist = `-- name: GetMBArtist :one
SELECT mbid, name, sort_name, disambiguation, artist_type, country, discography_refreshed_at, created_at, updated_at FROM mb_artists WHERE mbid = $1
`
func (q *Queries) GetMBArtist(ctx context.Context, mbid string) (MbArtist, error) {
row := q.db.QueryRow(ctx, getMBArtist, mbid)
var i MbArtist
err := row.Scan(
&i.Mbid,
&i.Name,
&i.SortName,
&i.Disambiguation,
&i.ArtistType,
&i.Country,
&i.DiscographyRefreshedAt,
&i.CreatedAt,
&i.UpdatedAt,
)
return i, err
}
const getMBReleaseGroup = `-- name: GetMBReleaseGroup :one
SELECT mbid, title, primary_type, secondary_types, first_release_date, artist_credit, releases_refreshed_at, created_at, updated_at FROM mb_release_groups WHERE mbid = $1
`
@@ -165,78 +117,6 @@ func (q *Queries) InsertMBReleaseTrack(ctx context.Context, arg InsertMBReleaseT
return err
}
const listMBArtistReleaseGroups = `-- name: ListMBArtistReleaseGroups :many
SELECT g.mbid, g.title, g.primary_type, g.secondary_types, g.first_release_date, g.artist_credit, g.releases_refreshed_at, g.created_at, g.updated_at FROM mb_release_groups g
JOIN mb_artist_release_groups ag ON ag.release_group_mbid = g.mbid
WHERE ag.artist_mbid = $1
ORDER BY g.first_release_date = '', g.first_release_date, g.title, g.mbid
`
// An artist's discography, oldest first; undated groups last.
func (q *Queries) ListMBArtistReleaseGroups(ctx context.Context, artistMbid string) ([]MbReleaseGroup, error) {
rows, err := q.db.Query(ctx, listMBArtistReleaseGroups, artistMbid)
if err != nil {
return nil, err
}
defer rows.Close()
var items []MbReleaseGroup
for rows.Next() {
var i MbReleaseGroup
if err := rows.Scan(
&i.Mbid,
&i.Title,
&i.PrimaryType,
&i.SecondaryTypes,
&i.FirstReleaseDate,
&i.ArtistCredit,
&i.ReleasesRefreshedAt,
&i.CreatedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listMBArtistsDueForRefresh = `-- name: ListMBArtistsDueForRefresh :many
SELECT mbid FROM mb_artists
WHERE discography_refreshed_at IS NULL OR discography_refreshed_at < $1
ORDER BY discography_refreshed_at NULLS FIRST, mbid
LIMIT $2
`
type ListMBArtistsDueForRefreshParams struct {
Before pgtype.Timestamptz
MaxRows int32
}
// Artists whose discography has never been read, or was read before the
// cutoff: never-read first, then oldest first.
func (q *Queries) ListMBArtistsDueForRefresh(ctx context.Context, arg ListMBArtistsDueForRefreshParams) ([]string, error) {
rows, err := q.db.Query(ctx, listMBArtistsDueForRefresh, arg.Before, arg.MaxRows)
if err != nil {
return nil, err
}
defer rows.Close()
var items []string
for rows.Next() {
var mbid string
if err := rows.Scan(&mbid); err != nil {
return nil, err
}
items = append(items, mbid)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listMBReleaseTracksForGroup = `-- name: ListMBReleaseTracksForGroup :many
SELECT t.release_mbid, t.medium_position, t.position, t.medium_format, t.number, t.title, t.length_ms, t.recording_mbid FROM mb_release_tracks t
JOIN mb_releases r ON r.mbid = t.release_mbid
@@ -273,53 +153,6 @@ func (q *Queries) ListMBReleaseTracksForGroup(ctx context.Context, releaseGroupM
return items, nil
}
const listMBReleasesForGroup = `-- name: ListMBReleasesForGroup :many
SELECT mbid, release_group_mbid, title, status, release_date, country, disambiguation, formats, track_count, created_at FROM mb_releases
WHERE release_group_mbid = $1
ORDER BY status <> 'Official', release_date = '', release_date, title, mbid
`
// Official editions first, then by date; undated last.
func (q *Queries) ListMBReleasesForGroup(ctx context.Context, releaseGroupMbid string) ([]MbRelease, error) {
rows, err := q.db.Query(ctx, listMBReleasesForGroup, releaseGroupMbid)
if err != nil {
return nil, err
}
defer rows.Close()
var items []MbRelease
for rows.Next() {
var i MbRelease
if err := rows.Scan(
&i.Mbid,
&i.ReleaseGroupMbid,
&i.Title,
&i.Status,
&i.ReleaseDate,
&i.Country,
&i.Disambiguation,
&i.Formats,
&i.TrackCount,
&i.CreatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const markMBArtistDiscographyRefreshed = `-- name: MarkMBArtistDiscographyRefreshed :exec
UPDATE mb_artists SET discography_refreshed_at = now() WHERE mbid = $1
`
func (q *Queries) MarkMBArtistDiscographyRefreshed(ctx context.Context, mbid string) error {
_, err := q.db.Exec(ctx, markMBArtistDiscographyRefreshed, mbid)
return err
}
const markMBReleaseGroupReleasesRefreshed = `-- name: MarkMBReleaseGroupReleasesRefreshed :exec
UPDATE mb_release_groups SET releases_refreshed_at = now() WHERE mbid = $1
`
@@ -329,43 +162,8 @@ func (q *Queries) MarkMBReleaseGroupReleasesRefreshed(ctx context.Context, mbid
return err
}
const upsertMBArtist = `-- name: UpsertMBArtist :exec
INSERT INTO mb_artists (mbid, name, sort_name, disambiguation, artist_type, country)
VALUES ($1, $2, $3, $4,
$5, $6)
ON CONFLICT (mbid) DO UPDATE SET
name = EXCLUDED.name,
sort_name = EXCLUDED.sort_name,
disambiguation = EXCLUDED.disambiguation,
artist_type = EXCLUDED.artist_type,
country = EXCLUDED.country,
updated_at = now()
`
type UpsertMBArtistParams struct {
Mbid string
Name string
SortName string
Disambiguation string
ArtistType string
Country string
}
// The release catalogue read from MusicBrainz.
func (q *Queries) UpsertMBArtist(ctx context.Context, arg UpsertMBArtistParams) error {
_, err := q.db.Exec(ctx, upsertMBArtist,
arg.Mbid,
arg.Name,
arg.SortName,
arg.Disambiguation,
arg.ArtistType,
arg.Country,
)
return err
}
const upsertMBReleaseGroup = `-- name: UpsertMBReleaseGroup :exec
INSERT INTO mb_release_groups (mbid, title, primary_type, secondary_types, first_release_date, artist_credit)
VALUES ($1, $2, $3, $4::text[],
$5, $6)
@@ -387,8 +185,9 @@ type UpsertMBReleaseGroupParams struct {
ArtistCredit string
}
// Leaves releases_refreshed_at alone: re-reading a discography says nothing
// about whether the group's track lists are current.
// The release catalogue read from MusicBrainz.
// Leaves releases_refreshed_at alone: only a full read of the group's
// releases and track lists marks it.
func (q *Queries) UpsertMBReleaseGroup(ctx context.Context, arg UpsertMBReleaseGroupParams) error {
_, err := q.db.Exec(ctx, upsertMBReleaseGroup,
arg.Mbid,
-902
View File
@@ -1,902 +0,0 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.31.1
// source: imports.sql
package dbq
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const claimGrabImport = `-- name: ClaimGrabImport :one
UPDATE grabs
SET import_state = 'importing', import_attempts = import_attempts + 1,
import_updated_at = now(), progress = 1
WHERE id = $1 AND ended_at IS NULL AND import_state IN ('', 'downloading', 'waiting')
RETURNING id, want_id, info_hash, name, key, indexer, source, client, save_path, quality, origin, forced, grabbed_by, grabbed_at, ended_at, end_reason, end_detail, progress, import_state, import_reason, import_detail, import_forced, import_attempts, import_updated_at, imported_at, imported_files, destination, seeding, ratio, seed_released_at
`
// Takes a finished download for import. The update is the lock: a second
// pass finds nothing to claim.
func (q *Queries) ClaimGrabImport(ctx context.Context, id pgtype.UUID) (Grab, error) {
row := q.db.QueryRow(ctx, claimGrabImport, id)
var i Grab
err := row.Scan(
&i.ID,
&i.WantID,
&i.InfoHash,
&i.Name,
&i.Key,
&i.Indexer,
&i.Source,
&i.Client,
&i.SavePath,
&i.Quality,
&i.Origin,
&i.Forced,
&i.GrabbedBy,
&i.GrabbedAt,
&i.EndedAt,
&i.EndReason,
&i.EndDetail,
&i.Progress,
&i.ImportState,
&i.ImportReason,
&i.ImportDetail,
&i.ImportForced,
&i.ImportAttempts,
&i.ImportUpdatedAt,
&i.ImportedAt,
&i.ImportedFiles,
&i.Destination,
&i.Seeding,
&i.Ratio,
&i.SeedReleasedAt,
)
return i, err
}
const clearTrackSourceVerdict = `-- name: ClearTrackSourceVerdict :exec
UPDATE tracks SET source_verdict = NULL, source_verdict_at = NULL
WHERE id = $1 AND source_verdict = 'suspect'
`
// A track whose file was replaced in place is no longer the rip it was.
func (q *Queries) ClearTrackSourceVerdict(ctx context.Context, id pgtype.UUID) error {
_, err := q.db.Exec(ctx, clearTrackSourceVerdict, id)
return err
}
const finishUpgrade = `-- name: FinishUpgrade :exec
UPDATE wants
SET album_id = COALESCE($1, album_id),
upgrade = '',
upgrade_checked_at = NULL,
updated_at = now()
WHERE id = $2
`
type FinishUpgradeParams struct {
AlbumID pgtype.UUID
ID pgtype.UUID
}
// An upgrade was imported: the want holds the album the new tracks are in,
// and is judged again by the next upgrade check.
func (q *Queries) FinishUpgrade(ctx context.Context, arg FinishUpgradeParams) error {
_, err := q.db.Exec(ctx, finishUpgrade, arg.AlbumID, arg.ID)
return err
}
const forceGrabImport = `-- name: ForceGrabImport :one
UPDATE grabs
SET import_state = '', import_forced = true, import_reason = '', import_updated_at = now()
WHERE id = $1 AND ended_at IS NULL AND import_state IN ('review', 'failed')
RETURNING id, want_id, info_hash, name, key, indexer, source, client, save_path, quality, origin, forced, grabbed_by, grabbed_at, ended_at, end_reason, end_detail, progress, import_state, import_reason, import_detail, import_forced, import_attempts, import_updated_at, imported_at, imported_files, destination, seeding, ratio, seed_released_at
`
// "Import anyway": the next pass imports the download whatever the check said.
func (q *Queries) ForceGrabImport(ctx context.Context, id pgtype.UUID) (Grab, error) {
row := q.db.QueryRow(ctx, forceGrabImport, id)
var i Grab
err := row.Scan(
&i.ID,
&i.WantID,
&i.InfoHash,
&i.Name,
&i.Key,
&i.Indexer,
&i.Source,
&i.Client,
&i.SavePath,
&i.Quality,
&i.Origin,
&i.Forced,
&i.GrabbedBy,
&i.GrabbedAt,
&i.EndedAt,
&i.EndReason,
&i.EndDetail,
&i.Progress,
&i.ImportState,
&i.ImportReason,
&i.ImportDetail,
&i.ImportForced,
&i.ImportAttempts,
&i.ImportUpdatedAt,
&i.ImportedAt,
&i.ImportedFiles,
&i.Destination,
&i.Seeding,
&i.Ratio,
&i.SeedReleasedAt,
)
return i, err
}
const getGrab = `-- name: GetGrab :one
SELECT id, want_id, info_hash, name, key, indexer, source, client, save_path, quality, origin, forced, grabbed_by, grabbed_at, ended_at, end_reason, end_detail, progress, import_state, import_reason, import_detail, import_forced, import_attempts, import_updated_at, imported_at, imported_files, destination, seeding, ratio, seed_released_at FROM grabs WHERE id = $1
`
func (q *Queries) GetGrab(ctx context.Context, id pgtype.UUID) (Grab, error) {
row := q.db.QueryRow(ctx, getGrab, id)
var i Grab
err := row.Scan(
&i.ID,
&i.WantID,
&i.InfoHash,
&i.Name,
&i.Key,
&i.Indexer,
&i.Source,
&i.Client,
&i.SavePath,
&i.Quality,
&i.Origin,
&i.Forced,
&i.GrabbedBy,
&i.GrabbedAt,
&i.EndedAt,
&i.EndReason,
&i.EndDetail,
&i.Progress,
&i.ImportState,
&i.ImportReason,
&i.ImportDetail,
&i.ImportForced,
&i.ImportAttempts,
&i.ImportUpdatedAt,
&i.ImportedAt,
&i.ImportedFiles,
&i.Destination,
&i.Seeding,
&i.Ratio,
&i.SeedReleasedAt,
)
return i, err
}
const heldPositions = `-- name: HeldPositions :many
SELECT DISTINCT COALESCE(t.disc_number, 1)::int AS disc, t.track_number::int AS track
FROM tracks t
JOIN albums a ON a.id = t.album_id
WHERE a.release_group_mbid = $1
AND t.missing_since IS NULL AND t.track_number IS NOT NULL
`
type HeldPositionsRow struct {
Disc int32
Track int32
}
// The disc and track positions the library holds for an album of this group.
func (q *Queries) HeldPositions(ctx context.Context, releaseGroupMbid *string) ([]HeldPositionsRow, error) {
rows, err := q.db.Query(ctx, heldPositions, releaseGroupMbid)
if err != nil {
return nil, err
}
defer rows.Close()
var items []HeldPositionsRow
for rows.Next() {
var i HeldPositionsRow
if err := rows.Scan(&i.Disc, &i.Track); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const heldRecordings = `-- name: HeldRecordings :many
SELECT DISTINCT mbid::text AS mbid FROM tracks
WHERE mbid = ANY($1::text[]) AND missing_since IS NULL
`
// Which of these recordings the library holds a file for.
func (q *Queries) HeldRecordings(ctx context.Context, recordings []string) ([]string, error) {
rows, err := q.db.Query(ctx, heldRecordings, recordings)
if err != nil {
return nil, err
}
defer rows.Close()
var items []string
for rows.Next() {
var mbid string
if err := rows.Scan(&mbid); err != nil {
return nil, err
}
items = append(items, mbid)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listGrabs = `-- name: ListGrabs :many
SELECT g.id, g.want_id, g.name, g.indexer, g.client, g.quality, g.origin, g.forced,
g.grabbed_at, g.ended_at, g.end_reason, g.end_detail, g.progress, g.import_state,
g.import_reason, g.import_detail, g.import_updated_at, g.imported_at, g.imported_files,
g.destination, g.seeding, g.ratio, g.seed_released_at,
w.title AS want_title, w.artist_name AS want_artist,
di.trigger AS condition, di.detail AS condition_detail, di.remediations AS condition_retries,
di.first_seen AS condition_since
FROM grabs g
LEFT JOIN wants w ON w.id = g.want_id
LEFT JOIN download_interventions di
ON di.id = (SELECT x.id FROM download_interventions x
WHERE x.grab_id = g.id AND x.outcome = 'open'
ORDER BY x.last_seen DESC LIMIT 1)
ORDER BY (g.ended_at IS NULL AND g.import_state IN ('review', 'failed')) DESC,
g.ended_at IS NULL DESC, g.grabbed_at DESC, g.id
LIMIT $2 OFFSET $1
`
type ListGrabsParams struct {
Skip int32
MaxRows int32
}
type ListGrabsRow struct {
ID pgtype.UUID
WantID pgtype.UUID
Name string
Indexer string
Client string
Quality string
Origin string
Forced bool
GrabbedAt pgtype.Timestamptz
EndedAt pgtype.Timestamptz
EndReason string
EndDetail string
Progress float32
ImportState string
ImportReason string
ImportDetail []byte
ImportUpdatedAt pgtype.Timestamptz
ImportedAt pgtype.Timestamptz
ImportedFiles int32
Destination string
Seeding bool
Ratio float32
SeedReleasedAt pgtype.Timestamptz
WantTitle *string
WantArtist *string
Condition *string
ConditionDetail *string
ConditionRetries *int32
ConditionSince pgtype.Timestamptz
}
// The Wanted view's Activity tab: downloads waiting on a person first, then
// the rest still running, then the ended ones newest first. Each carries
// what is wrong with it now, if anything.
func (q *Queries) ListGrabs(ctx context.Context, arg ListGrabsParams) ([]ListGrabsRow, error) {
rows, err := q.db.Query(ctx, listGrabs, arg.Skip, arg.MaxRows)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListGrabsRow
for rows.Next() {
var i ListGrabsRow
if err := rows.Scan(
&i.ID,
&i.WantID,
&i.Name,
&i.Indexer,
&i.Client,
&i.Quality,
&i.Origin,
&i.Forced,
&i.GrabbedAt,
&i.EndedAt,
&i.EndReason,
&i.EndDetail,
&i.Progress,
&i.ImportState,
&i.ImportReason,
&i.ImportDetail,
&i.ImportUpdatedAt,
&i.ImportedAt,
&i.ImportedFiles,
&i.Destination,
&i.Seeding,
&i.Ratio,
&i.SeedReleasedAt,
&i.WantTitle,
&i.WantArtist,
&i.Condition,
&i.ConditionDetail,
&i.ConditionRetries,
&i.ConditionSince,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listImportedGrabsHolding = `-- name: ListImportedGrabsHolding :many
SELECT id, info_hash, name, want_id
FROM grabs
WHERE import_state = 'imported'
AND info_hash <> ''
AND destination <> ''
AND starts_with($1::text, rtrim(destination, '/') || '/')
`
type ListImportedGrabsHoldingRow struct {
ID pgtype.UUID
InfoHash string
Name string
WantID pgtype.UUID
}
// The downloads Minstrel imported into the folder holding path: the releases a
// file there came from, to block when the files proved not to be the album
// . A file imported before Minstrel downloaded anything has none.
func (q *Queries) ListImportedGrabsHolding(ctx context.Context, path string) ([]ListImportedGrabsHoldingRow, error) {
rows, err := q.db.Query(ctx, listImportedGrabsHolding, path)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListImportedGrabsHoldingRow
for rows.Next() {
var i ListImportedGrabsHoldingRow
if err := rows.Scan(
&i.ID,
&i.InfoHash,
&i.Name,
&i.WantID,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listLiveGrabs = `-- name: ListLiveGrabs :many
SELECT g.id, g.want_id, g.info_hash, g.name, g.key, g.indexer, g.source, g.client, g.save_path, g.quality, g.origin, g.forced, g.grabbed_by, g.grabbed_at, g.ended_at, g.end_reason, g.end_detail, g.progress, g.import_state, g.import_reason, g.import_detail, g.import_forced, g.import_attempts, g.import_updated_at, g.imported_at, g.imported_files, g.destination, g.seeding, g.ratio, g.seed_released_at, w.release_group_mbid, w.artist_mbid, w.artist_name, w.title AS want_title,
w.reason AS want_reason, w.album_id AS want_album_id
FROM grabs g
LEFT JOIN wants w ON w.id = g.want_id
WHERE g.ended_at IS NULL
ORDER BY g.grabbed_at, g.id
`
type ListLiveGrabsRow struct {
ID pgtype.UUID
WantID pgtype.UUID
InfoHash string
Name string
Key string
Indexer string
Source string
Client string
SavePath string
Quality string
Origin string
Forced bool
GrabbedBy pgtype.UUID
GrabbedAt pgtype.Timestamptz
EndedAt pgtype.Timestamptz
EndReason string
EndDetail string
Progress float32
ImportState string
ImportReason string
ImportDetail []byte
ImportForced bool
ImportAttempts int32
ImportUpdatedAt pgtype.Timestamptz
ImportedAt pgtype.Timestamptz
ImportedFiles int32
Destination string
Seeding bool
Ratio float32
SeedReleasedAt pgtype.Timestamptz
ReleaseGroupMbid *string
ArtistMbid *string
ArtistName *string
WantTitle *string
WantReason *string
WantAlbumID pgtype.UUID
}
// Import with verification.
// Every download still running or waiting on its import, oldest first.
func (q *Queries) ListLiveGrabs(ctx context.Context) ([]ListLiveGrabsRow, error) {
rows, err := q.db.Query(ctx, listLiveGrabs)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListLiveGrabsRow
for rows.Next() {
var i ListLiveGrabsRow
if err := rows.Scan(
&i.ID,
&i.WantID,
&i.InfoHash,
&i.Name,
&i.Key,
&i.Indexer,
&i.Source,
&i.Client,
&i.SavePath,
&i.Quality,
&i.Origin,
&i.Forced,
&i.GrabbedBy,
&i.GrabbedAt,
&i.EndedAt,
&i.EndReason,
&i.EndDetail,
&i.Progress,
&i.ImportState,
&i.ImportReason,
&i.ImportDetail,
&i.ImportForced,
&i.ImportAttempts,
&i.ImportUpdatedAt,
&i.ImportedAt,
&i.ImportedFiles,
&i.Destination,
&i.Seeding,
&i.Ratio,
&i.SeedReleasedAt,
&i.ReleaseGroupMbid,
&i.ArtistMbid,
&i.ArtistName,
&i.WantTitle,
&i.WantReason,
&i.WantAlbumID,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listReleaseBlocksPage = `-- name: ListReleaseBlocksPage :many
SELECT b.info_hash, b.name, b.reason, b.detail, b.blocked_at, b.expires_at, w.title AS want_title,
w.artist_name AS want_artist
FROM release_blocks b
LEFT JOIN wants w ON w.id = b.want_id
WHERE b.expires_at IS NULL OR b.expires_at > now()
ORDER BY b.blocked_at DESC, b.info_hash
LIMIT $2 OFFSET $1
`
type ListReleaseBlocksPageParams struct {
Skip int32
MaxRows int32
}
type ListReleaseBlocksPageRow struct {
InfoHash string
Name string
Reason string
Detail string
BlockedAt pgtype.Timestamptz
ExpiresAt pgtype.Timestamptz
WantTitle *string
WantArtist *string
}
func (q *Queries) ListReleaseBlocksPage(ctx context.Context, arg ListReleaseBlocksPageParams) ([]ListReleaseBlocksPageRow, error) {
rows, err := q.db.Query(ctx, listReleaseBlocksPage, arg.Skip, arg.MaxRows)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListReleaseBlocksPageRow
for rows.Next() {
var i ListReleaseBlocksPageRow
if err := rows.Scan(
&i.InfoHash,
&i.Name,
&i.Reason,
&i.Detail,
&i.BlockedAt,
&i.ExpiresAt,
&i.WantTitle,
&i.WantArtist,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listUpgradeTracks = `-- name: ListUpgradeTracks :many
SELECT id, file_path, mbid, COALESCE(disc_number, 1)::int AS disc, track_number,
file_format, file_size, duration_ms, source_verdict
FROM tracks
WHERE album_id = $1 AND missing_since IS NULL
`
type ListUpgradeTracksRow struct {
ID pgtype.UUID
FilePath string
Mbid *string
Disc int32
TrackNumber *int32
FileFormat string
FileSize int64
DurationMs int32
SourceVerdict *string
}
// The tracks an upgrade may replace: the present tracks of the album
// the want holds.
func (q *Queries) ListUpgradeTracks(ctx context.Context, albumID pgtype.UUID) ([]ListUpgradeTracksRow, error) {
rows, err := q.db.Query(ctx, listUpgradeTracks, albumID)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListUpgradeTracksRow
for rows.Next() {
var i ListUpgradeTracksRow
if err := rows.Scan(
&i.ID,
&i.FilePath,
&i.Mbid,
&i.Disc,
&i.TrackNumber,
&i.FileFormat,
&i.FileSize,
&i.DurationMs,
&i.SourceVerdict,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const markWantSearched = `-- name: MarkWantSearched :exec
UPDATE wants SET searched_at = now() WHERE id = $1
`
// Imported: nothing to search for until the wanted pass settles the want.
func (q *Queries) MarkWantSearched(ctx context.Context, id pgtype.UUID) error {
_, err := q.db.Exec(ctx, markWantSearched, id)
return err
}
const openWantsForGroups = `-- name: OpenWantsForGroups :many
SELECT w.id, w.release_group_mbid, w.artist_mbid, w.artist_name, w.title, w.primary_type, w.first_release_date, w.reason, w.album_id, w.created_by, w.wanted_at, w.fulfilled_at, w.created_at, w.updated_at, w.searched_at, w.search_outcome, w.search_considered, w.best_release, w.best_verdict, w.search_error, w.upgrade, w.held_tier, w.held_quality, w.upgrade_checked_at FROM wants w
WHERE w.fulfilled_at IS NULL AND w.release_group_mbid = ANY($1::text[])
AND NOT EXISTS (SELECT 1 FROM grabs g WHERE g.want_id = w.id AND g.ended_at IS NULL)
`
// Open wants for these release groups with no download running.
func (q *Queries) OpenWantsForGroups(ctx context.Context, groups []string) ([]Want, error) {
rows, err := q.db.Query(ctx, openWantsForGroups, groups)
if err != nil {
return nil, err
}
defer rows.Close()
var items []Want
for rows.Next() {
var i Want
if err := rows.Scan(
&i.ID,
&i.ReleaseGroupMbid,
&i.ArtistMbid,
&i.ArtistName,
&i.Title,
&i.PrimaryType,
&i.FirstReleaseDate,
&i.Reason,
&i.AlbumID,
&i.CreatedBy,
&i.WantedAt,
&i.FulfilledAt,
&i.CreatedAt,
&i.UpdatedAt,
&i.SearchedAt,
&i.SearchOutcome,
&i.SearchConsidered,
&i.BestRelease,
&i.BestVerdict,
&i.SearchError,
&i.Upgrade,
&i.HeldTier,
&i.HeldQuality,
&i.UpgradeCheckedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const reclaimStaleImports = `-- name: ReclaimStaleImports :execrows
UPDATE grabs
SET import_state = 'waiting',
import_reason = 'The last import was cut short, so it is being tried again.'
WHERE ended_at IS NULL AND import_state = 'importing'
AND import_updated_at < now() - make_interval(mins => $1::int)
`
// An import cut short (a restart mid-copy) is tried again.
func (q *Queries) ReclaimStaleImports(ctx context.Context, minutes int32) (int64, error) {
result, err := q.db.Exec(ctx, reclaimStaleImports, minutes)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const recordImport = `-- name: RecordImport :one
UPDATE grabs
SET import_state = $1,
import_reason = $2,
import_detail = $3,
import_updated_at = now(),
import_forced = false,
imported_at = CASE WHEN $1::text = 'imported' THEN now() ELSE imported_at END,
-- The files were copied, so an imported download goes on sharing
-- until the lifecycle removes it.
seeding = $1::text = 'imported',
imported_files = $4,
destination = $5,
ended_at = CASE WHEN $6::text <> '' THEN now() ELSE ended_at END,
end_reason = $6,
end_detail = CASE WHEN $6::text <> '' THEN $2::text ELSE end_detail END
WHERE id = $7
RETURNING id, want_id, info_hash, name, key, indexer, source, client, save_path, quality, origin, forced, grabbed_by, grabbed_at, ended_at, end_reason, end_detail, progress, import_state, import_reason, import_detail, import_forced, import_attempts, import_updated_at, imported_at, imported_files, destination, seeding, ratio, seed_released_at
`
type RecordImportParams struct {
ImportState string
ImportReason string
ImportDetail []byte
ImportedFiles int32
Destination string
EndReason string
ID pgtype.UUID
}
// The import's outcome. An end reason ends the grab: imported, refused or
// held. Without one the grab stays live: review and failed wait for a person.
func (q *Queries) RecordImport(ctx context.Context, arg RecordImportParams) (Grab, error) {
row := q.db.QueryRow(ctx, recordImport,
arg.ImportState,
arg.ImportReason,
arg.ImportDetail,
arg.ImportedFiles,
arg.Destination,
arg.EndReason,
arg.ID,
)
var i Grab
err := row.Scan(
&i.ID,
&i.WantID,
&i.InfoHash,
&i.Name,
&i.Key,
&i.Indexer,
&i.Source,
&i.Client,
&i.SavePath,
&i.Quality,
&i.Origin,
&i.Forced,
&i.GrabbedBy,
&i.GrabbedAt,
&i.EndedAt,
&i.EndReason,
&i.EndDetail,
&i.Progress,
&i.ImportState,
&i.ImportReason,
&i.ImportDetail,
&i.ImportForced,
&i.ImportAttempts,
&i.ImportUpdatedAt,
&i.ImportedAt,
&i.ImportedFiles,
&i.Destination,
&i.Seeding,
&i.Ratio,
&i.SeedReleasedAt,
)
return i, err
}
const rehomeGrab = `-- name: RehomeGrab :one
UPDATE grabs g
SET want_id = $1
WHERE g.id = $2 AND g.ended_at IS NULL
AND NOT EXISTS (SELECT 1 FROM grabs o WHERE o.want_id = $1 AND o.ended_at IS NULL)
RETURNING id, want_id, info_hash, name, key, indexer, source, client, save_path, quality, origin, forced, grabbed_by, grabbed_at, ended_at, end_reason, end_detail, progress, import_state, import_reason, import_detail, import_forced, import_attempts, import_updated_at, imported_at, imported_files, destination, seeding, ratio, seed_released_at
`
type RehomeGrabParams struct {
WantID pgtype.UUID
ID pgtype.UUID
}
// A download that turned out to be another wanted album is that want's grab,
// when the want has none running.
func (q *Queries) RehomeGrab(ctx context.Context, arg RehomeGrabParams) (Grab, error) {
row := q.db.QueryRow(ctx, rehomeGrab, arg.WantID, arg.ID)
var i Grab
err := row.Scan(
&i.ID,
&i.WantID,
&i.InfoHash,
&i.Name,
&i.Key,
&i.Indexer,
&i.Source,
&i.Client,
&i.SavePath,
&i.Quality,
&i.Origin,
&i.Forced,
&i.GrabbedBy,
&i.GrabbedAt,
&i.EndedAt,
&i.EndReason,
&i.EndDetail,
&i.Progress,
&i.ImportState,
&i.ImportReason,
&i.ImportDetail,
&i.ImportForced,
&i.ImportAttempts,
&i.ImportUpdatedAt,
&i.ImportedAt,
&i.ImportedFiles,
&i.Destination,
&i.Seeding,
&i.Ratio,
&i.SeedReleasedAt,
)
return i, err
}
const retryGrabImport = `-- name: RetryGrabImport :one
UPDATE grabs
SET import_state = '', import_reason = '', import_updated_at = now()
WHERE id = $1 AND ended_at IS NULL AND import_state = 'failed'
RETURNING id, want_id, info_hash, name, key, indexer, source, client, save_path, quality, origin, forced, grabbed_by, grabbed_at, ended_at, end_reason, end_detail, progress, import_state, import_reason, import_detail, import_forced, import_attempts, import_updated_at, imported_at, imported_files, destination, seeding, ratio, seed_released_at
`
func (q *Queries) RetryGrabImport(ctx context.Context, id pgtype.UUID) (Grab, error) {
row := q.db.QueryRow(ctx, retryGrabImport, id)
var i Grab
err := row.Scan(
&i.ID,
&i.WantID,
&i.InfoHash,
&i.Name,
&i.Key,
&i.Indexer,
&i.Source,
&i.Client,
&i.SavePath,
&i.Quality,
&i.Origin,
&i.Forced,
&i.GrabbedBy,
&i.GrabbedAt,
&i.EndedAt,
&i.EndReason,
&i.EndDetail,
&i.Progress,
&i.ImportState,
&i.ImportReason,
&i.ImportDetail,
&i.ImportForced,
&i.ImportAttempts,
&i.ImportUpdatedAt,
&i.ImportedAt,
&i.ImportedFiles,
&i.Destination,
&i.Seeding,
&i.Ratio,
&i.SeedReleasedAt,
)
return i, err
}
const searchWantAgain = `-- name: SearchWantAgain :exec
UPDATE wants SET searched_at = NULL, search_outcome = '' WHERE id = $1
`
// Refused: search again on the next pass.
func (q *Queries) SearchWantAgain(ctx context.Context, id pgtype.UUID) error {
_, err := q.db.Exec(ctx, searchWantAgain, id)
return err
}
const setGrabProgress = `-- name: SetGrabProgress :exec
UPDATE grabs
SET progress = $1,
import_state = $2,
import_reason = $3,
import_updated_at = now()
WHERE id = $4 AND ended_at IS NULL
AND import_state IN ('', 'downloading', 'waiting')
`
type SetGrabProgressParams struct {
Progress float32
ImportState string
ImportReason string
ID pgtype.UUID
}
// What the download client says of a download not yet finished, or finished
// and waiting for what the import needs. Leaves a claimed, decided or failed
// import alone.
func (q *Queries) SetGrabProgress(ctx context.Context, arg SetGrabProgressParams) error {
_, err := q.db.Exec(ctx, setGrabProgress,
arg.Progress,
arg.ImportState,
arg.ImportReason,
arg.ID,
)
return err
}
-39
View File
@@ -50,45 +50,6 @@ func (q *Queries) RerootAlbumCoverPaths(ctx context.Context, arg RerootAlbumCove
return result.RowsAffected(), nil
}
const rerootGrabDestinations = `-- name: RerootGrabDestinations :execrows
UPDATE grabs
SET destination = $1::text || substr(destination, length($2::text) + 1)
WHERE destination = $2::text
OR starts_with(destination, $2::text || '/')
`
type RerootGrabDestinationsParams struct {
NewRoot string
OldRoot string
}
func (q *Queries) RerootGrabDestinations(ctx context.Context, arg RerootGrabDestinationsParams) (int64, error) {
result, err := q.db.Exec(ctx, rerootGrabDestinations, arg.NewRoot, arg.OldRoot)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const rerootImportRoot = `-- name: RerootImportRoot :execrows
UPDATE acquisition_settings
SET import_root = $1::text
WHERE import_root = $2::text
`
type RerootImportRootParams struct {
NewRoot string
OldRoot string
}
func (q *Queries) RerootImportRoot(ctx context.Context, arg RerootImportRootParams) (int64, error) {
result, err := q.db.Exec(ctx, rerootImportRoot, arg.NewRoot, arg.OldRoot)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const rerootTrackPaths = `-- name: RerootTrackPaths :execrows
UPDATE tracks
-408
View File
@@ -1,408 +0,0 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.31.1
// source: lifecycle.sql
package dbq
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const countGoneGrabs = `-- name: CountGoneGrabs :one
SELECT count(*) FROM grabs
WHERE info_hash = $1 AND end_reason = 'gone' AND ended_at > $2
`
type CountGoneGrabsParams struct {
InfoHash string
Since pgtype.Timestamptz
}
// How many of a release's grabs ended gone since a time: a release that keeps
// leaving its client is held back rather than taken on every pass.
func (q *Queries) CountGoneGrabs(ctx context.Context, arg CountGoneGrabsParams) (int64, error) {
row := q.db.QueryRow(ctx, countGoneGrabs, arg.InfoHash, arg.Since)
var count int64
err := row.Scan(&count)
return count, err
}
const countRemediation = `-- name: CountRemediation :one
UPDATE download_interventions
SET remediations = remediations + 1,
action = $1,
last_seen = now(),
outcome = CASE WHEN $1::text IN ('given_up', 'removed') THEN 'resolved' ELSE outcome END,
settled_at = CASE WHEN $1::text IN ('given_up', 'removed') THEN now() ELSE settled_at END
WHERE client = $2 AND info_hash = $3
AND trigger = $4 AND outcome = 'open'
RETURNING remediations
`
type CountRemediationParams struct {
Action string
Client string
InfoHash string
Trigger string
}
// Minstrel acted on an open condition. A retry climbs the ladder; giving up
// or removing ends the condition, since what it was about is gone. Written
// before the act, so a retry that happened is never uncounted.
func (q *Queries) CountRemediation(ctx context.Context, arg CountRemediationParams) (int32, error) {
row := q.db.QueryRow(ctx, countRemediation,
arg.Action,
arg.Client,
arg.InfoHash,
arg.Trigger,
)
var remediations int32
err := row.Scan(&remediations)
return remediations, err
}
const deleteExpiredBlocks = `-- name: DeleteExpiredBlocks :execrows
DELETE FROM release_blocks WHERE expires_at IS NOT NULL AND expires_at <= now()
`
// A release given up on is held back for a while; after that the search may
// take it again.
func (q *Queries) DeleteExpiredBlocks(ctx context.Context) (int64, error) {
result, err := q.db.Exec(ctx, deleteExpiredBlocks)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const giveUpGrab = `-- name: GiveUpGrab :one
UPDATE grabs
SET ended_at = now(), end_reason = $1, end_detail = $2,
import_updated_at = now()
WHERE id = $3 AND ended_at IS NULL AND import_state = ANY($4::text[])
RETURNING id, want_id, info_hash, name, key, indexer, source, client, save_path, quality, origin, forced, grabbed_by, grabbed_at, ended_at, end_reason, end_detail, progress, import_state, import_reason, import_detail, import_forced, import_attempts, import_updated_at, imported_at, imported_files, destination, seeding, ratio, seed_released_at
`
type GiveUpGrabParams struct {
EndReason string
EndDetail string
ID pgtype.UUID
States []string
}
// Ends a running grab, given up on. Only from the states named: the update
// is the lock against an import that claimed the download meanwhile.
func (q *Queries) GiveUpGrab(ctx context.Context, arg GiveUpGrabParams) (Grab, error) {
row := q.db.QueryRow(ctx, giveUpGrab,
arg.EndReason,
arg.EndDetail,
arg.ID,
arg.States,
)
var i Grab
err := row.Scan(
&i.ID,
&i.WantID,
&i.InfoHash,
&i.Name,
&i.Key,
&i.Indexer,
&i.Source,
&i.Client,
&i.SavePath,
&i.Quality,
&i.Origin,
&i.Forced,
&i.GrabbedBy,
&i.GrabbedAt,
&i.EndedAt,
&i.EndReason,
&i.EndDetail,
&i.Progress,
&i.ImportState,
&i.ImportReason,
&i.ImportDetail,
&i.ImportForced,
&i.ImportAttempts,
&i.ImportUpdatedAt,
&i.ImportedAt,
&i.ImportedFiles,
&i.Destination,
&i.Seeding,
&i.Ratio,
&i.SeedReleasedAt,
)
return i, err
}
const listClientsDown = `-- name: ListClientsDown :many
SELECT client, detail, first_seen FROM download_interventions
WHERE trigger = 'client_unreachable' AND outcome = 'open'
ORDER BY client
`
type ListClientsDownRow struct {
Client string
Detail string
FirstSeen pgtype.Timestamptz
}
// Download clients not answering now, for the Activity tab.
func (q *Queries) ListClientsDown(ctx context.Context) ([]ListClientsDownRow, error) {
rows, err := q.db.Query(ctx, listClientsDown)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListClientsDownRow
for rows.Next() {
var i ListClientsDownRow
if err := rows.Scan(&i.Client, &i.Detail, &i.FirstSeen); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listLeftovers = `-- name: ListLeftovers :many
SELECT id, client, info_hash, grab_id, name, trigger, action, outcome, detail, observations, remediations, first_seen, last_seen, settled_at FROM download_interventions
WHERE trigger = 'not_removed' AND outcome = 'open' AND info_hash <> ''
ORDER BY first_seen
`
// Downloads given up on that the client would not remove; each pass asks
// again.
func (q *Queries) ListLeftovers(ctx context.Context) ([]DownloadIntervention, error) {
rows, err := q.db.Query(ctx, listLeftovers)
if err != nil {
return nil, err
}
defer rows.Close()
var items []DownloadIntervention
for rows.Next() {
var i DownloadIntervention
if err := rows.Scan(
&i.ID,
&i.Client,
&i.InfoHash,
&i.GrabID,
&i.Name,
&i.Trigger,
&i.Action,
&i.Outcome,
&i.Detail,
&i.Observations,
&i.Remediations,
&i.FirstSeen,
&i.LastSeen,
&i.SettledAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listSeedingGrabs = `-- name: ListSeedingGrabs :many
SELECT id, client, info_hash, name, imported_at FROM grabs
WHERE seeding AND seed_released_at IS NULL
ORDER BY imported_at, id
`
type ListSeedingGrabsRow struct {
ID pgtype.UUID
Client string
InfoHash string
Name string
ImportedAt pgtype.Timestamptz
}
// Imported downloads still sharing, oldest import first.
func (q *Queries) ListSeedingGrabs(ctx context.Context) ([]ListSeedingGrabsRow, error) {
rows, err := q.db.Query(ctx, listSeedingGrabs)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListSeedingGrabsRow
for rows.Next() {
var i ListSeedingGrabsRow
if err := rows.Scan(
&i.ID,
&i.Client,
&i.InfoHash,
&i.Name,
&i.ImportedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const liveGrabOnHash = `-- name: LiveGrabOnHash :one
SELECT EXISTS (SELECT 1 FROM grabs WHERE info_hash = $1 AND ended_at IS NULL)::bool
`
func (q *Queries) LiveGrabOnHash(ctx context.Context, infoHash string) (bool, error) {
row := q.db.QueryRow(ctx, liveGrabOnHash, infoHash)
var column_1 bool
err := row.Scan(&column_1)
return column_1, err
}
const raiseIntervention = `-- name: RaiseIntervention :one
INSERT INTO download_interventions (client, info_hash, grab_id, name, trigger, detail)
VALUES ($1, $2, $3, $4,
$5, $6)
ON CONFLICT (client, info_hash, trigger) WHERE outcome = 'open' DO UPDATE SET
observations = download_interventions.observations + 1,
last_seen = now(),
detail = EXCLUDED.detail,
grab_id = COALESCE(EXCLUDED.grab_id, download_interventions.grab_id),
name = CASE WHEN EXCLUDED.name <> '' THEN EXCLUDED.name ELSE download_interventions.name END
RETURNING id, client, info_hash, grab_id, name, trigger, action, outcome, detail, observations, remediations, first_seen, last_seen, settled_at
`
type RaiseInterventionParams struct {
Client string
InfoHash string
GrabID pgtype.UUID
Name string
Trigger string
Detail string
}
// The download lifecycle: the intervention ledger, giving a
// download up, and seeding after import.
// A condition seen, or seen again: a second pass lands on the open row, so
// a download stuck for a week is one row that says how long. The name only
// improves, and the detail is the latest word.
func (q *Queries) RaiseIntervention(ctx context.Context, arg RaiseInterventionParams) (DownloadIntervention, error) {
row := q.db.QueryRow(ctx, raiseIntervention,
arg.Client,
arg.InfoHash,
arg.GrabID,
arg.Name,
arg.Trigger,
arg.Detail,
)
var i DownloadIntervention
err := row.Scan(
&i.ID,
&i.Client,
&i.InfoHash,
&i.GrabID,
&i.Name,
&i.Trigger,
&i.Action,
&i.Outcome,
&i.Detail,
&i.Observations,
&i.Remediations,
&i.FirstSeen,
&i.LastSeen,
&i.SettledAt,
)
return i, err
}
const releaseSeed = `-- name: ReleaseSeed :exec
UPDATE grabs SET seed_released_at = now() WHERE id = $1 AND seed_released_at IS NULL
`
// The seeding download was removed, or found gone.
func (q *Queries) ReleaseSeed(ctx context.Context, id pgtype.UUID) error {
_, err := q.db.Exec(ctx, releaseSeed, id)
return err
}
const setGrabRatio = `-- name: SetGrabRatio :exec
UPDATE grabs SET ratio = $1 WHERE id = $2
`
type SetGrabRatioParams struct {
Ratio float32
ID pgtype.UUID
}
func (q *Queries) SetGrabRatio(ctx context.Context, arg SetGrabRatioParams) error {
_, err := q.db.Exec(ctx, setGrabRatio, arg.Ratio, arg.ID)
return err
}
const settleEndedGrabs = `-- name: SettleEndedGrabs :execrows
UPDATE download_interventions di
SET outcome = 'resolved', settled_at = now(), last_seen = now()
FROM grabs g
WHERE g.id = di.grab_id AND g.ended_at IS NOT NULL
AND di.outcome = 'open' AND di.trigger <> 'not_removed'
`
// What was wrong with a download whose grab has ended (imported, refused,
// given up) is over, except a download left in the client to remove.
func (q *Queries) SettleEndedGrabs(ctx context.Context) (int64, error) {
result, err := q.db.Exec(ctx, settleEndedGrabs)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const settleIntervention = `-- name: SettleIntervention :execrows
UPDATE download_interventions
SET outcome = 'resolved', settled_at = now(), last_seen = now()
WHERE client = $1 AND info_hash = $2
AND trigger = $3 AND outcome = 'open'
`
type SettleInterventionParams struct {
Client string
InfoHash string
Trigger string
}
// The condition cleared.
func (q *Queries) SettleIntervention(ctx context.Context, arg SettleInterventionParams) (int64, error) {
result, err := q.db.Exec(ctx, settleIntervention, arg.Client, arg.InfoHash, arg.Trigger)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const settleInterventionsFor = `-- name: SettleInterventionsFor :execrows
UPDATE download_interventions
SET outcome = 'resolved', settled_at = now(), last_seen = now()
WHERE client = $1 AND info_hash = $2 AND outcome = 'open'
`
type SettleInterventionsForParams struct {
Client string
InfoHash string
}
// Everything open about one download is over: it is gone from the client.
func (q *Queries) SettleInterventionsFor(ctx context.Context, arg SettleInterventionsForParams) (int64, error) {
result, err := q.db.Exec(ctx, settleInterventionsFor, arg.Client, arg.InfoHash)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
-18
View File
@@ -297,24 +297,6 @@ func (q *Queries) MergeRepointPlaylistTracks(ctx context.Context, arg MergeRepoi
return items, nil
}
const mergeRepointRequests = `-- name: MergeRepointRequests :execrows
UPDATE requests SET matched_track_id = $1::uuid
WHERE matched_track_id = $2::uuid
`
type MergeRepointRequestsParams struct {
SurvivorID pgtype.UUID
LoserID pgtype.UUID
}
func (q *Queries) MergeRepointRequests(ctx context.Context, arg MergeRepointRequestsParams) (int64, error) {
result, err := q.db.Exec(ctx, mergeRepointRequests, arg.SurvivorID, arg.LoserID)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const mergeRepointSkipEvents = `-- name: MergeRepointSkipEvents :execrows
UPDATE skip_events SET track_id = $1::uuid WHERE track_id = $2::uuid
`
-272
View File
@@ -1,272 +0,0 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.31.1
// source: reacquisition.sql
package dbq
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const countAlbumsMissingWithoutMbid = `-- name: CountAlbumsMissingWithoutMbid :one
SELECT COUNT(DISTINCT albums.id)::bigint
FROM albums
JOIN artists ON artists.id = albums.artist_id
JOIN tracks ON tracks.album_id = albums.id
WHERE tracks.missing_since IS NOT NULL
AND ((albums.mbid IS NULL AND albums.release_group_mbid IS NULL) OR artists.mbid IS NULL)
`
// Albums with missing files that can never be wanted automatically because
// nothing identifies them to MusicBrainz. Surfaced on the admin card so the
// gap is visible: silently doing nothing for these would read as broken.
func (q *Queries) CountAlbumsMissingWithoutMbid(ctx context.Context) (int64, error) {
row := q.db.QueryRow(ctx, countAlbumsMissingWithoutMbid)
var column_1 int64
err := row.Scan(&column_1)
return column_1, err
}
const getReacquisitionSettings = `-- name: GetReacquisitionSettings :one
SELECT id, enabled, grace_hours, max_per_pass FROM reacquisition_settings WHERE id = true
`
// Re-acquisition of missing files.
// The unit is the album: an album whose files went missing becomes a want for
// its release group, and the search schedule looks for it.
func (q *Queries) GetReacquisitionSettings(ctx context.Context) (ReacquisitionSetting, error) {
row := q.db.QueryRow(ctx, getReacquisitionSettings)
var i ReacquisitionSetting
err := row.Scan(
&i.ID,
&i.Enabled,
&i.GraceHours,
&i.MaxPerPass,
)
return i, err
}
const getWantsForAlbums = `-- name: GetWantsForAlbums :many
SELECT a.id AS album_id, w.id AS want_id, w.wanted_at, w.fulfilled_at
FROM albums a
JOIN wants w ON w.album_id = a.id OR w.release_group_mbid = a.release_group_mbid
WHERE a.id = ANY($1::uuid[])
ORDER BY w.wanted_at DESC
`
type GetWantsForAlbumsRow struct {
AlbumID pgtype.UUID
WantID pgtype.UUID
WantedAt pgtype.Timestamptz
FulfilledAt pgtype.Timestamptz
}
// What the wanted list says about each album on the missing-files page.
func (q *Queries) GetWantsForAlbums(ctx context.Context, albumIds []pgtype.UUID) ([]GetWantsForAlbumsRow, error) {
rows, err := q.db.Query(ctx, getWantsForAlbums, albumIds)
if err != nil {
return nil, err
}
defer rows.Close()
var items []GetWantsForAlbumsRow
for rows.Next() {
var i GetWantsForAlbumsRow
if err := rows.Scan(
&i.AlbumID,
&i.WantID,
&i.WantedAt,
&i.FulfilledAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listAlbumsDueReacquisition = `-- name: ListAlbumsDueReacquisition :many
SELECT albums.id AS album_id,
albums.title AS album_title,
albums.mbid AS album_mbid,
albums.release_group_mbid AS album_release_group_mbid,
artists.name AS artist_name,
artists.mbid AS artist_mbid,
COUNT(tracks.id)::bigint AS missing_track_count,
MAX(tracks.missing_since)::timestamptz AS last_missing_since
FROM albums
JOIN artists ON artists.id = albums.artist_id
JOIN tracks ON tracks.album_id = albums.id
WHERE tracks.missing_since IS NOT NULL
AND tracks.missing_since <= now() - make_interval(hours => $1::int)
AND (albums.mbid IS NOT NULL OR albums.release_group_mbid IS NOT NULL)
AND artists.mbid IS NOT NULL
GROUP BY albums.id, albums.title, albums.mbid, albums.release_group_mbid,
artists.name, artists.mbid
HAVING EXISTS (
SELECT 1 FROM wants w
WHERE w.album_id = albums.id OR w.release_group_mbid = albums.release_group_mbid)
AND NOT EXISTS (
SELECT 1 FROM wants w
WHERE (w.album_id = albums.id OR w.release_group_mbid = albums.release_group_mbid)
AND (w.fulfilled_at IS NULL OR w.fulfilled_at > MAX(tracks.missing_since)))
ORDER BY MIN(tracks.missing_since), albums.sort_title
LIMIT $2
`
type ListAlbumsDueReacquisitionParams struct {
GraceHours int32
PageLimit int32
}
type ListAlbumsDueReacquisitionRow struct {
AlbumID pgtype.UUID
AlbumTitle string
AlbumMbid *string
AlbumReleaseGroupMbid *string
ArtistName string
ArtistMbid *string
MissingTrackCount int64
LastMissingSince pgtype.Timestamptz
}
// The sweeper's selection. An album qualifies when:
//
// - it still has at least one track whose file has been missing longer than
// the grace window. Measured on missing_since, which reconcile never
// re-stamps, so it is a genuine "gone since" clock;
// - MusicBrainz can name it: the artist has an MBID, and the album a
// release MBID or a release group. The release MBID is only ever used to
// find the group, and an album matched to a group may carry no release
// id. Albums failing this are counted separately
// (CountAlbumsMissingWithoutMbid);
// - it is on the wanted list: an album taken off the list, or
// dropped into the library by hand, is not fetched back;
// - no want already covers it: none open, and none held since its files
// went missing. A want held after the loss means the library has a full
// copy despite the missing rows, so asking again would loop.
func (q *Queries) ListAlbumsDueReacquisition(ctx context.Context, arg ListAlbumsDueReacquisitionParams) ([]ListAlbumsDueReacquisitionRow, error) {
rows, err := q.db.Query(ctx, listAlbumsDueReacquisition, arg.GraceHours, arg.PageLimit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListAlbumsDueReacquisitionRow
for rows.Next() {
var i ListAlbumsDueReacquisitionRow
if err := rows.Scan(
&i.AlbumID,
&i.AlbumTitle,
&i.AlbumMbid,
&i.AlbumReleaseGroupMbid,
&i.ArtistName,
&i.ArtistMbid,
&i.MissingTrackCount,
&i.LastMissingSince,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const updateReacquisitionSettings = `-- name: UpdateReacquisitionSettings :one
UPDATE reacquisition_settings
SET enabled = $1,
grace_hours = $2,
max_per_pass = $3
WHERE id = true
RETURNING id, enabled, grace_hours, max_per_pass
`
type UpdateReacquisitionSettingsParams struct {
Enabled bool
GraceHours int32
MaxPerPass int32
}
// Whole-row write from the admin card; the CHECKs in migration 0056 are the
// validation, so a bad value fails loudly rather than being clamped silently.
func (q *Queries) UpdateReacquisitionSettings(ctx context.Context, arg UpdateReacquisitionSettingsParams) (ReacquisitionSetting, error) {
row := q.db.QueryRow(ctx, updateReacquisitionSettings, arg.Enabled, arg.GraceHours, arg.MaxPerPass)
var i ReacquisitionSetting
err := row.Scan(
&i.ID,
&i.Enabled,
&i.GraceHours,
&i.MaxPerPass,
)
return i, err
}
const wantMissingAlbum = `-- name: WantMissingAlbum :one
INSERT INTO wants (release_group_mbid, artist_mbid, artist_name, title, reason, album_id)
VALUES ($1, $2, $3,
$4, 'missing', $5)
ON CONFLICT (release_group_mbid) DO UPDATE
SET reason = 'missing',
album_id = EXCLUDED.album_id,
wanted_at = now(),
fulfilled_at = NULL,
updated_at = now()
WHERE wants.fulfilled_at IS NOT NULL
RETURNING id, release_group_mbid, artist_mbid, artist_name, title, primary_type, first_release_date, reason, album_id, created_by, wanted_at, fulfilled_at, created_at, updated_at, searched_at, search_outcome, search_considered, best_release, best_verdict, search_error, upgrade, held_tier, held_quality, upgrade_checked_at
`
type WantMissingAlbumParams struct {
ReleaseGroupMbid string
ArtistMbid string
ArtistName string
Title string
AlbumID pgtype.UUID
}
// Records an album with missing files as wanted. A held want for the same
// group is reopened; an open one is left as it is, and no row comes back.
func (q *Queries) WantMissingAlbum(ctx context.Context, arg WantMissingAlbumParams) (Want, error) {
row := q.db.QueryRow(ctx, wantMissingAlbum,
arg.ReleaseGroupMbid,
arg.ArtistMbid,
arg.ArtistName,
arg.Title,
arg.AlbumID,
)
var i Want
err := row.Scan(
&i.ID,
&i.ReleaseGroupMbid,
&i.ArtistMbid,
&i.ArtistName,
&i.Title,
&i.PrimaryType,
&i.FirstReleaseDate,
&i.Reason,
&i.AlbumID,
&i.CreatedBy,
&i.WantedAt,
&i.FulfilledAt,
&i.CreatedAt,
&i.UpdatedAt,
&i.SearchedAt,
&i.SearchOutcome,
&i.SearchConsidered,
&i.BestRelease,
&i.BestVerdict,
&i.SearchError,
&i.Upgrade,
&i.HeldTier,
&i.HeldQuality,
&i.UpgradeCheckedAt,
)
return i, err
}
-149
View File
@@ -1112,152 +1112,3 @@ func (q *Queries) LoadRadioCandidatesV2(ctx context.Context, arg LoadRadioCandid
}
return items, nil
}
const suggestArtistsForUser = `-- name: SuggestArtistsForUser :many
WITH artist_plays AS (
-- Completed plays only: counting every play_event would make skipping an
-- artist repeatedly INCREASE its signal and push more of its neighbours
-- at the user.
SELECT t.artist_id, count(*)::bigint AS play_count
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1 AND pe.was_skipped = false
GROUP BY t.artist_id
),
profile_seeds AS (
SELECT tpa.artist_id, tpa.weight AS raw_signal
FROM taste_profile_artists tpa
WHERE tpa.user_id = $1 AND tpa.weight > 0
),
fallback_seeds AS (
SELECT a.id AS artist_id,
5.0 * (CASE WHEN gla.artist_id IS NOT NULL THEN 1 ELSE 0 END)
+ COALESCE(SUM(EXP(- EXTRACT(epoch FROM now() - pe.started_at) / ($2::float8 * 86400.0))), 0)
AS raw_signal
FROM artists a
LEFT JOIN general_likes_artists gla ON gla.artist_id = a.id AND gla.user_id = $1
LEFT JOIN tracks t ON t.artist_id = a.id
LEFT JOIN play_events pe
ON pe.track_id = t.id AND pe.user_id = $1 AND pe.was_skipped = false
WHERE (gla.artist_id IS NOT NULL OR pe.id IS NOT NULL)
AND NOT EXISTS (SELECT 1 FROM profile_seeds)
GROUP BY a.id, gla.artist_id
),
seeds AS (
SELECT s.artist_id,
ln(1.0 + s.raw_signal) AS signal,
(gla.artist_id IS NOT NULL) AS is_liked,
COALESCE(ap.play_count, 0)::bigint AS play_count
FROM (
SELECT artist_id, raw_signal FROM profile_seeds
UNION ALL
SELECT artist_id, raw_signal FROM fallback_seeds
) s
LEFT JOIN general_likes_artists gla
ON gla.artist_id = s.artist_id AND gla.user_id = $1
LEFT JOIN artist_plays ap ON ap.artist_id = s.artist_id
WHERE s.raw_signal > 0
),
contributions AS (
SELECT u.candidate_mbid,
u.candidate_name,
seeds.artist_id AS seed_id,
seeds.is_liked,
seeds.play_count,
seeds.signal * u.score AS contribution
FROM artist_similarity_unmatched u
JOIN seeds ON seeds.artist_id = u.seed_artist_id
WHERE NOT EXISTS (SELECT 1 FROM artists WHERE mbid = u.candidate_mbid)
AND NOT EXISTS (
SELECT 1 FROM requests r
WHERE r.user_id = $1
AND r.artist_mbid = u.candidate_mbid
AND r.status NOT IN ('rejected', 'failed')
)
-- Snoozed by this user and not yet expired. Time-boxed and
-- per-user: the candidate returns on its own once snoozed_until
-- passes, and stays visible to everyone else meanwhile. Deliberately
-- filtered at the candidate stage, NOT folded into the score — a
-- snooze carries no opinion about the music, so it must not become a
-- ranking signal.
AND NOT EXISTS (
SELECT 1 FROM suggestion_snoozes s
WHERE s.user_id = $1
AND s.candidate_mbid = u.candidate_mbid
AND s.snoozed_until > now()
)
)
SELECT candidate_mbid,
candidate_name,
SUM(contribution)::float8 AS total_score,
((array_agg(seed_id ORDER BY contribution DESC))[1:3])::uuid[] AS top_seed_ids,
((array_agg(contribution ORDER BY contribution DESC))[1:3])::float8[] AS top_contributions,
((array_agg(is_liked ORDER BY contribution DESC))[1:3])::boolean[] AS top_is_liked,
((array_agg(play_count ORDER BY contribution DESC))[1:3])::bigint[] AS top_play_counts
FROM contributions
GROUP BY candidate_mbid, candidate_name
ORDER BY total_score DESC
LIMIT $3
`
type SuggestArtistsForUserParams struct {
UserID pgtype.UUID
Column2 float64
Limit int32
}
type SuggestArtistsForUserRow struct {
CandidateMbid string
CandidateName string
TotalScore float64
TopSeedIds []pgtype.UUID
TopContributions []float64
TopIsLiked []bool
TopPlayCounts []int64
}
// Per-user artist suggestions ranked by taste signal x similarity, projected
// through artist_similarity_unmatched (out-of-library candidates only).
//
// Seeds are TIERED so the surface never empties:
//
// tier 1 - taste_profile_artists.weight: engagement-graded, time-decayed and
// SIGNED by internal/taste, so an artist the user has drifted away
// from stops contributing instead of accumulating forever.
// tier 2 - likes + completed plays, used ONLY when the profile has no rows
// (new account, or before the first daily recompute).
//
// The signal is log-damped: contribution is signal x similarity, and an
// undamped sum would let one heavily-played artist's neighbours take every
// slot -- entrenching harder the MORE the user listened.
//
// Candidates already in the library, already requested and not terminal, or
// snoozed by this user, are excluded. $1=user_id, $2=half_life_days (tier 2
// decay), $3=limit.
func (q *Queries) SuggestArtistsForUser(ctx context.Context, arg SuggestArtistsForUserParams) ([]SuggestArtistsForUserRow, error) {
rows, err := q.db.Query(ctx, suggestArtistsForUser, arg.UserID, arg.Column2, arg.Limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []SuggestArtistsForUserRow
for rows.Next() {
var i SuggestArtistsForUserRow
if err := rows.Scan(
&i.CandidateMbid,
&i.CandidateName,
&i.TotalScore,
&i.TopSeedIds,
&i.TopContributions,
&i.TopIsLiked,
&i.TopPlayCounts,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
@@ -9,22 +9,6 @@ import (
"context"
)
const getDiscoverTuning = `-- name: GetDiscoverTuning :one
SELECT singleton, tag_overlap_weight, snooze_days, updated_at FROM discover_tuning WHERE singleton = true
`
func (q *Queries) GetDiscoverTuning(ctx context.Context) (DiscoverTuning, error) {
row := q.db.QueryRow(ctx, getDiscoverTuning)
var i DiscoverTuning
err := row.Scan(
&i.Singleton,
&i.TagOverlapWeight,
&i.SnoozeDays,
&i.UpdatedAt,
)
return i, err
}
const getTasteTuning = `-- name: GetTasteTuning :one
SELECT singleton, half_life_days, engagement_hard_skip, engagement_neutral, engagement_full, updated_at, enriched_tag_scale, era_scale, mood_scale FROM taste_tuning WHERE singleton = true
`
@@ -134,32 +118,6 @@ func (q *Queries) ListWeightProfiles(ctx context.Context) ([]RecommendationWeigh
return items, nil
}
const updateDiscoverTuning = `-- name: UpdateDiscoverTuning :one
UPDATE discover_tuning
SET tag_overlap_weight = $1,
snooze_days = $2,
updated_at = now()
WHERE singleton = true
RETURNING singleton, tag_overlap_weight, snooze_days, updated_at
`
type UpdateDiscoverTuningParams struct {
TagOverlapWeight float64
SnoozeDays float64
}
func (q *Queries) UpdateDiscoverTuning(ctx context.Context, arg UpdateDiscoverTuningParams) (DiscoverTuning, error) {
row := q.db.QueryRow(ctx, updateDiscoverTuning, arg.TagOverlapWeight, arg.SnoozeDays)
var i DiscoverTuning
err := row.Scan(
&i.Singleton,
&i.TagOverlapWeight,
&i.SnoozeDays,
&i.UpdatedAt,
)
return i, err
}
const updateTasteTuning = `-- name: UpdateTasteTuning :one
UPDATE taste_tuning
SET half_life_days = $1,
@@ -268,24 +226,6 @@ func (q *Queries) UpdateWeightProfile(ctx context.Context, arg UpdateWeightProfi
return i, err
}
const upsertDiscoverTuningDefaults = `-- name: UpsertDiscoverTuningDefaults :exec
INSERT INTO discover_tuning (singleton, tag_overlap_weight, snooze_days)
VALUES (true, $1, $2)
ON CONFLICT (singleton) DO NOTHING
`
type UpsertDiscoverTuningDefaultsParams struct {
TagOverlapWeight float64
SnoozeDays float64
}
// Boot reconcile for the Discover scope. Never overwrites
// admin-tuned values, same contract as the other two.
func (q *Queries) UpsertDiscoverTuningDefaults(ctx context.Context, arg UpsertDiscoverTuningDefaultsParams) error {
_, err := q.db.Exec(ctx, upsertDiscoverTuningDefaults, arg.TagOverlapWeight, arg.SnoozeDays)
return err
}
const upsertTasteTuningDefaults = `-- name: UpsertTasteTuningDefaults :exec
INSERT INTO taste_tuning (
singleton, half_life_days, engagement_hard_skip,
-528
View File
@@ -1,528 +0,0 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.31.1
// source: requests.sql
package dbq
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const approveRequest = `-- name: ApproveRequest :one
UPDATE requests
SET status = 'approved',
decided_at = now(),
decided_by = $2,
updated_at = now()
WHERE id = $1 AND status = 'pending'
RETURNING id, user_id, status, kind, artist_mbid, album_mbid, track_mbid, artist_name, album_title, track_title, decided_at, decided_by, notes, completed_at, matched_track_id, matched_album_id, matched_artist_id, requested_at, updated_at
`
type ApproveRequestParams struct {
ID pgtype.UUID
DecidedBy pgtype.UUID
}
func (q *Queries) ApproveRequest(ctx context.Context, arg ApproveRequestParams) (Request, error) {
row := q.db.QueryRow(ctx, approveRequest, arg.ID, arg.DecidedBy)
var i Request
err := row.Scan(
&i.ID,
&i.UserID,
&i.Status,
&i.Kind,
&i.ArtistMbid,
&i.AlbumMbid,
&i.TrackMbid,
&i.ArtistName,
&i.AlbumTitle,
&i.TrackTitle,
&i.DecidedAt,
&i.DecidedBy,
&i.Notes,
&i.CompletedAt,
&i.MatchedTrackID,
&i.MatchedAlbumID,
&i.MatchedArtistID,
&i.RequestedAt,
&i.UpdatedAt,
)
return i, err
}
const cancelRequest = `-- name: CancelRequest :one
UPDATE requests
SET status = 'rejected',
notes = 'cancelled by user',
decided_at = now(),
decided_by = $2,
updated_at = now()
WHERE id = $1 AND user_id = $2 AND status = 'pending'
RETURNING id, user_id, status, kind, artist_mbid, album_mbid, track_mbid, artist_name, album_title, track_title, decided_at, decided_by, notes, completed_at, matched_track_id, matched_album_id, matched_artist_id, requested_at, updated_at
`
type CancelRequestParams struct {
ID pgtype.UUID
DecidedBy pgtype.UUID
}
// $2 serves as both the ownership guard (user_id = $2) and the
// decided_by audit field. The generated parameter struct names it
// DecidedBy — callers must pass the cancelling user's UUID, and
// a wrong UUID will silently no-op (zero rows updated) rather than
// erroring at the SQL layer.
func (q *Queries) CancelRequest(ctx context.Context, arg CancelRequestParams) (Request, error) {
row := q.db.QueryRow(ctx, cancelRequest, arg.ID, arg.DecidedBy)
var i Request
err := row.Scan(
&i.ID,
&i.UserID,
&i.Status,
&i.Kind,
&i.ArtistMbid,
&i.AlbumMbid,
&i.TrackMbid,
&i.ArtistName,
&i.AlbumTitle,
&i.TrackTitle,
&i.DecidedAt,
&i.DecidedBy,
&i.Notes,
&i.CompletedAt,
&i.MatchedTrackID,
&i.MatchedAlbumID,
&i.MatchedArtistID,
&i.RequestedAt,
&i.UpdatedAt,
)
return i, err
}
const completeRequest = `-- name: CompleteRequest :one
UPDATE requests
SET status = 'completed',
matched_track_id = $2,
matched_album_id = $3,
matched_artist_id = $4,
completed_at = now(),
updated_at = now()
WHERE id = $1 AND status = 'approved'
RETURNING id, user_id, status, kind, artist_mbid, album_mbid, track_mbid, artist_name, album_title, track_title, decided_at, decided_by, notes, completed_at, matched_track_id, matched_album_id, matched_artist_id, requested_at, updated_at
`
type CompleteRequestParams struct {
ID pgtype.UUID
MatchedTrackID pgtype.UUID
MatchedAlbumID pgtype.UUID
MatchedArtistID pgtype.UUID
}
// Reconciler transitions an approved request to completed when its
// target track/album/artist has appeared in the library. Callers
// should set ONLY the matched_*_id corresponding to the request's
// kind and pass pgtype.UUID{} (Valid: false) for the others —
// per-kind one-of-three is the contract, NULL for the others
// preserves auditability if the matched row is later evicted.
func (q *Queries) CompleteRequest(ctx context.Context, arg CompleteRequestParams) (Request, error) {
row := q.db.QueryRow(ctx, completeRequest,
arg.ID,
arg.MatchedTrackID,
arg.MatchedAlbumID,
arg.MatchedArtistID,
)
var i Request
err := row.Scan(
&i.ID,
&i.UserID,
&i.Status,
&i.Kind,
&i.ArtistMbid,
&i.AlbumMbid,
&i.TrackMbid,
&i.ArtistName,
&i.AlbumTitle,
&i.TrackTitle,
&i.DecidedAt,
&i.DecidedBy,
&i.Notes,
&i.CompletedAt,
&i.MatchedTrackID,
&i.MatchedAlbumID,
&i.MatchedArtistID,
&i.RequestedAt,
&i.UpdatedAt,
)
return i, err
}
const createRequest = `-- name: CreateRequest :one
INSERT INTO requests (
user_id, kind,
artist_mbid, album_mbid, track_mbid,
artist_name, album_title, track_title
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
RETURNING id, user_id, status, kind, artist_mbid, album_mbid, track_mbid, artist_name, album_title, track_title, decided_at, decided_by, notes, completed_at, matched_track_id, matched_album_id, matched_artist_id, requested_at, updated_at
`
type CreateRequestParams struct {
UserID pgtype.UUID
Kind RequestKind
ArtistMbid string
AlbumMbid *string
TrackMbid *string
ArtistName string
AlbumTitle *string
TrackTitle *string
}
func (q *Queries) CreateRequest(ctx context.Context, arg CreateRequestParams) (Request, error) {
row := q.db.QueryRow(ctx, createRequest,
arg.UserID,
arg.Kind,
arg.ArtistMbid,
arg.AlbumMbid,
arg.TrackMbid,
arg.ArtistName,
arg.AlbumTitle,
arg.TrackTitle,
)
var i Request
err := row.Scan(
&i.ID,
&i.UserID,
&i.Status,
&i.Kind,
&i.ArtistMbid,
&i.AlbumMbid,
&i.TrackMbid,
&i.ArtistName,
&i.AlbumTitle,
&i.TrackTitle,
&i.DecidedAt,
&i.DecidedBy,
&i.Notes,
&i.CompletedAt,
&i.MatchedTrackID,
&i.MatchedAlbumID,
&i.MatchedArtistID,
&i.RequestedAt,
&i.UpdatedAt,
)
return i, err
}
const getNonTerminalRequestForMBID = `-- name: GetNonTerminalRequestForMBID :one
SELECT id, user_id, status, kind, artist_mbid, album_mbid, track_mbid, artist_name, album_title, track_title, decided_at, decided_by, notes, completed_at, matched_track_id, matched_album_id, matched_artist_id, requested_at, updated_at FROM requests
WHERE status IN ('pending', 'approved', 'completed')
AND ((kind = 'artist' AND artist_mbid = $1)
OR (kind = 'album' AND album_mbid = $1)
OR (kind = 'track' AND track_mbid = $1))
ORDER BY requested_at ASC
LIMIT 1
`
// Returns the oldest non-terminal (pending/approved/completed) request
// whose own-kind MBID matches @mbid, or pgx.ErrNoRows when none. Create()
// uses this to dedupe: a re-request for something already in flight
// returns the existing row instead of inserting a duplicate.
func (q *Queries) GetNonTerminalRequestForMBID(ctx context.Context, mbid string) (Request, error) {
row := q.db.QueryRow(ctx, getNonTerminalRequestForMBID, mbid)
var i Request
err := row.Scan(
&i.ID,
&i.UserID,
&i.Status,
&i.Kind,
&i.ArtistMbid,
&i.AlbumMbid,
&i.TrackMbid,
&i.ArtistName,
&i.AlbumTitle,
&i.TrackTitle,
&i.DecidedAt,
&i.DecidedBy,
&i.Notes,
&i.CompletedAt,
&i.MatchedTrackID,
&i.MatchedAlbumID,
&i.MatchedArtistID,
&i.RequestedAt,
&i.UpdatedAt,
)
return i, err
}
const getRequestByID = `-- name: GetRequestByID :one
SELECT id, user_id, status, kind, artist_mbid, album_mbid, track_mbid, artist_name, album_title, track_title, decided_at, decided_by, notes, completed_at, matched_track_id, matched_album_id, matched_artist_id, requested_at, updated_at FROM requests WHERE id = $1
`
func (q *Queries) GetRequestByID(ctx context.Context, id pgtype.UUID) (Request, error) {
row := q.db.QueryRow(ctx, getRequestByID, id)
var i Request
err := row.Scan(
&i.ID,
&i.UserID,
&i.Status,
&i.Kind,
&i.ArtistMbid,
&i.AlbumMbid,
&i.TrackMbid,
&i.ArtistName,
&i.AlbumTitle,
&i.TrackTitle,
&i.DecidedAt,
&i.DecidedBy,
&i.Notes,
&i.CompletedAt,
&i.MatchedTrackID,
&i.MatchedAlbumID,
&i.MatchedArtistID,
&i.RequestedAt,
&i.UpdatedAt,
)
return i, err
}
const hasNonTerminalRequestForMBID = `-- name: HasNonTerminalRequestForMBID :one
SELECT (EXISTS (
SELECT 1 FROM requests r
WHERE r.status IN ('pending', 'approved', 'completed')
AND ((r.kind = 'artist' AND r.artist_mbid = $1)
OR (r.kind = 'album' AND r.album_mbid = $1)
OR (r.kind = 'track' AND r.track_mbid = $1))
) OR EXISTS (
SELECT 1 FROM wants w WHERE w.release_group_mbid = $1 AND w.fulfilled_at IS NULL
) OR EXISTS (
SELECT 1 FROM follows f WHERE f.artist_mbid = $1
))::bool AS requested
`
// Whether something is already on its way: a pending, approved or completed
// request at the given level, an open want for the release group, or a
// follow of the artist. Sets the `requested` flag on catalogue search.
// Terminal-status (rejected, failed) requests do not count. MBIDs are unique
// per entity type, so checking one id against every column cannot
// false-match.
func (q *Queries) HasNonTerminalRequestForMBID(ctx context.Context, mbid string) (bool, error) {
row := q.db.QueryRow(ctx, hasNonTerminalRequestForMBID, mbid)
var requested bool
err := row.Scan(&requested)
return requested, err
}
const listApprovedRequestsForReconcile = `-- name: ListApprovedRequestsForReconcile :many
SELECT id, user_id, status, kind, artist_mbid, album_mbid, track_mbid, artist_name, album_title, track_title, decided_at, decided_by, notes, completed_at, matched_track_id, matched_album_id, matched_artist_id, requested_at, updated_at FROM requests
WHERE status = 'approved'
ORDER BY decided_at ASC
LIMIT $1
`
func (q *Queries) ListApprovedRequestsForReconcile(ctx context.Context, limit int32) ([]Request, error) {
rows, err := q.db.Query(ctx, listApprovedRequestsForReconcile, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []Request
for rows.Next() {
var i Request
if err := rows.Scan(
&i.ID,
&i.UserID,
&i.Status,
&i.Kind,
&i.ArtistMbid,
&i.AlbumMbid,
&i.TrackMbid,
&i.ArtistName,
&i.AlbumTitle,
&i.TrackTitle,
&i.DecidedAt,
&i.DecidedBy,
&i.Notes,
&i.CompletedAt,
&i.MatchedTrackID,
&i.MatchedAlbumID,
&i.MatchedArtistID,
&i.RequestedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listRequestsByStatus = `-- name: ListRequestsByStatus :many
SELECT id, user_id, status, kind, artist_mbid, album_mbid, track_mbid, artist_name, album_title, track_title, decided_at, decided_by, notes, completed_at, matched_track_id, matched_album_id, matched_artist_id, requested_at, updated_at FROM requests
WHERE status = $1
ORDER BY requested_at DESC
LIMIT $2
`
type ListRequestsByStatusParams struct {
Status RequestStatus
Limit int32
}
func (q *Queries) ListRequestsByStatus(ctx context.Context, arg ListRequestsByStatusParams) ([]Request, error) {
rows, err := q.db.Query(ctx, listRequestsByStatus, arg.Status, arg.Limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []Request
for rows.Next() {
var i Request
if err := rows.Scan(
&i.ID,
&i.UserID,
&i.Status,
&i.Kind,
&i.ArtistMbid,
&i.AlbumMbid,
&i.TrackMbid,
&i.ArtistName,
&i.AlbumTitle,
&i.TrackTitle,
&i.DecidedAt,
&i.DecidedBy,
&i.Notes,
&i.CompletedAt,
&i.MatchedTrackID,
&i.MatchedAlbumID,
&i.MatchedArtistID,
&i.RequestedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listRequestsForUser = `-- name: ListRequestsForUser :many
SELECT id, user_id, status, kind, artist_mbid, album_mbid, track_mbid, artist_name, album_title, track_title, decided_at, decided_by, notes, completed_at, matched_track_id, matched_album_id, matched_artist_id, requested_at, updated_at FROM requests
WHERE user_id = $1
ORDER BY requested_at DESC
LIMIT $2
`
type ListRequestsForUserParams struct {
UserID pgtype.UUID
Limit int32
}
func (q *Queries) ListRequestsForUser(ctx context.Context, arg ListRequestsForUserParams) ([]Request, error) {
rows, err := q.db.Query(ctx, listRequestsForUser, arg.UserID, arg.Limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []Request
for rows.Next() {
var i Request
if err := rows.Scan(
&i.ID,
&i.UserID,
&i.Status,
&i.Kind,
&i.ArtistMbid,
&i.AlbumMbid,
&i.TrackMbid,
&i.ArtistName,
&i.AlbumTitle,
&i.TrackTitle,
&i.DecidedAt,
&i.DecidedBy,
&i.Notes,
&i.CompletedAt,
&i.MatchedTrackID,
&i.MatchedAlbumID,
&i.MatchedArtistID,
&i.RequestedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const rejectRequest = `-- name: RejectRequest :one
UPDATE requests
SET status = 'rejected',
notes = $2,
decided_at = now(),
decided_by = $3,
updated_at = now()
WHERE id = $1 AND status = 'pending'
RETURNING id, user_id, status, kind, artist_mbid, album_mbid, track_mbid, artist_name, album_title, track_title, decided_at, decided_by, notes, completed_at, matched_track_id, matched_album_id, matched_artist_id, requested_at, updated_at
`
type RejectRequestParams struct {
ID pgtype.UUID
Notes *string
DecidedBy pgtype.UUID
}
func (q *Queries) RejectRequest(ctx context.Context, arg RejectRequestParams) (Request, error) {
row := q.db.QueryRow(ctx, rejectRequest, arg.ID, arg.Notes, arg.DecidedBy)
var i Request
err := row.Scan(
&i.ID,
&i.UserID,
&i.Status,
&i.Kind,
&i.ArtistMbid,
&i.AlbumMbid,
&i.TrackMbid,
&i.ArtistName,
&i.AlbumTitle,
&i.TrackTitle,
&i.DecidedAt,
&i.DecidedBy,
&i.Notes,
&i.CompletedAt,
&i.MatchedTrackID,
&i.MatchedAlbumID,
&i.MatchedArtistID,
&i.RequestedAt,
&i.UpdatedAt,
)
return i, err
}
const repointRequestsAlbumMbid = `-- name: RepointRequestsAlbumMbid :exec
UPDATE requests
SET album_mbid = $1,
updated_at = now()
WHERE album_mbid = $2
`
type RepointRequestsAlbumMbidParams struct {
NewMbid *string
OldMbid *string
}
// A want moved from a release id to its group takes the requests
// that name it along, so the reconciler finds it.
func (q *Queries) RepointRequestsAlbumMbid(ctx context.Context, arg RepointRequestsAlbumMbidParams) error {
_, err := q.db.Exec(ctx, repointRequestsAlbumMbid, arg.NewMbid, arg.OldMbid)
return err
}
-680
View File
@@ -1,680 +0,0 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.31.1
// source: search.sql
package dbq
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const blockRelease = `-- name: BlockRelease :exec
INSERT INTO release_blocks (info_hash, name, reason, detail, want_id, blocked_by, expires_at)
VALUES ($1, $2, $3, $4,
$5, $6, $7)
ON CONFLICT (info_hash) DO UPDATE
SET reason = EXCLUDED.reason, detail = EXCLUDED.detail, blocked_at = now(),
expires_at = EXCLUDED.expires_at, blocked_by = EXCLUDED.blocked_by
WHERE release_blocks.expires_at IS NOT NULL OR EXCLUDED.expires_at IS NULL
`
type BlockReleaseParams struct {
InfoHash string
Name string
Reason string
Detail string
WantID pgtype.UUID
BlockedBy pgtype.UUID
ExpiresAt pgtype.Timestamptz
}
// A release set aside. expires_at NULL is for good. A block for good is never
// shortened by a timed one: only a timed block, or another for good, is
// replaced.
func (q *Queries) BlockRelease(ctx context.Context, arg BlockReleaseParams) error {
_, err := q.db.Exec(ctx, blockRelease,
arg.InfoHash,
arg.Name,
arg.Reason,
arg.Detail,
arg.WantID,
arg.BlockedBy,
arg.ExpiresAt,
)
return err
}
const deleteWantCandidates = `-- name: DeleteWantCandidates :exec
DELETE FROM want_candidates WHERE want_id = $1
`
func (q *Queries) DeleteWantCandidates(ctx context.Context, wantID pgtype.UUID) error {
_, err := q.db.Exec(ctx, deleteWantCandidates, wantID)
return err
}
const editionTrackSpan = `-- name: EditionTrackSpan :one
SELECT COALESCE(MIN(track_count) FILTER (WHERE status = 'Official'), MIN(track_count), 0)::int AS min_tracks,
COALESCE(MAX(track_count), 0)::int AS max_tracks
FROM mb_releases
WHERE release_group_mbid = $1 AND track_count > 0
`
type EditionTrackSpanRow struct {
MinTracks int32
MaxTracks int32
}
// The fewest tracks an official edition of the group has (any edition when
// none is official), and the most any has. Zero when none is catalogued.
func (q *Queries) EditionTrackSpan(ctx context.Context, releaseGroupMbid string) (EditionTrackSpanRow, error) {
row := q.db.QueryRow(ctx, editionTrackSpan, releaseGroupMbid)
var i EditionTrackSpanRow
err := row.Scan(&i.MinTracks, &i.MaxTracks)
return i, err
}
const endGrab = `-- name: EndGrab :exec
UPDATE grabs
SET ended_at = now(), end_reason = $1, end_detail = $2
WHERE id = $3 AND ended_at IS NULL
`
type EndGrabParams struct {
EndReason string
EndDetail string
ID pgtype.UUID
}
func (q *Queries) EndGrab(ctx context.Context, arg EndGrabParams) error {
_, err := q.db.Exec(ctx, endGrab, arg.EndReason, arg.EndDetail, arg.ID)
return err
}
const getAcquisitionSettings = `-- name: GetAcquisitionSettings :one
SELECT id, auto_grab, search_interval_hours, allowed_qualities, last_pass_at, last_pass, updated_at, import_root, seed_ratio, seed_days, stall_hours, library_listing_started_at, library_listed_at, library_unlisted, upgrade_until FROM acquisition_settings WHERE id = true
`
// Searching for wants and grabbing releases.
func (q *Queries) GetAcquisitionSettings(ctx context.Context) (AcquisitionSetting, error) {
row := q.db.QueryRow(ctx, getAcquisitionSettings)
var i AcquisitionSetting
err := row.Scan(
&i.ID,
&i.AutoGrab,
&i.SearchIntervalHours,
&i.AllowedQualities,
&i.LastPassAt,
&i.LastPass,
&i.UpdatedAt,
&i.ImportRoot,
&i.SeedRatio,
&i.SeedDays,
&i.StallHours,
&i.LibraryListingStartedAt,
&i.LibraryListedAt,
&i.LibraryUnlisted,
&i.UpgradeUntil,
)
return i, err
}
const getLiveGrab = `-- name: GetLiveGrab :one
SELECT id, want_id, info_hash, name, key, indexer, source, client, save_path, quality, origin, forced, grabbed_by, grabbed_at, ended_at, end_reason, end_detail, progress, import_state, import_reason, import_detail, import_forced, import_attempts, import_updated_at, imported_at, imported_files, destination, seeding, ratio, seed_released_at FROM grabs WHERE want_id = $1 AND ended_at IS NULL
`
func (q *Queries) GetLiveGrab(ctx context.Context, wantID pgtype.UUID) (Grab, error) {
row := q.db.QueryRow(ctx, getLiveGrab, wantID)
var i Grab
err := row.Scan(
&i.ID,
&i.WantID,
&i.InfoHash,
&i.Name,
&i.Key,
&i.Indexer,
&i.Source,
&i.Client,
&i.SavePath,
&i.Quality,
&i.Origin,
&i.Forced,
&i.GrabbedBy,
&i.GrabbedAt,
&i.EndedAt,
&i.EndReason,
&i.EndDetail,
&i.Progress,
&i.ImportState,
&i.ImportReason,
&i.ImportDetail,
&i.ImportForced,
&i.ImportAttempts,
&i.ImportUpdatedAt,
&i.ImportedAt,
&i.ImportedFiles,
&i.Destination,
&i.Seeding,
&i.Ratio,
&i.SeedReleasedAt,
)
return i, err
}
const getWantCandidate = `-- name: GetWantCandidate :one
SELECT want_id, key, name, info_hash, indexer, source, size_bytes, seeders, files_count, format, quality, year, verdict, reason, reasons, folded_from, rank, searched_at FROM want_candidates WHERE want_id = $1 AND key = $2
`
type GetWantCandidateParams struct {
WantID pgtype.UUID
Key string
}
func (q *Queries) GetWantCandidate(ctx context.Context, arg GetWantCandidateParams) (WantCandidate, error) {
row := q.db.QueryRow(ctx, getWantCandidate, arg.WantID, arg.Key)
var i WantCandidate
err := row.Scan(
&i.WantID,
&i.Key,
&i.Name,
&i.InfoHash,
&i.Indexer,
&i.Source,
&i.SizeBytes,
&i.Seeders,
&i.FilesCount,
&i.Format,
&i.Quality,
&i.Year,
&i.Verdict,
&i.Reason,
&i.Reasons,
&i.FoldedFrom,
&i.Rank,
&i.SearchedAt,
)
return i, err
}
const getWantForSearch = `-- name: GetWantForSearch :one
SELECT w.id, w.release_group_mbid, w.artist_mbid, w.artist_name, w.title, w.reason, w.fulfilled_at,
w.upgrade, w.held_tier, w.held_quality, w.album_id,
COALESCE(NULLIF(w.primary_type, ''), g.primary_type, '')::text AS primary_type,
COALESCE(g.secondary_types, '{}')::text[] AS secondary_types,
COALESCE(NULLIF(w.first_release_date, ''), g.first_release_date, '')::text AS first_release_date
FROM wants w
LEFT JOIN mb_release_groups g ON g.mbid = w.release_group_mbid
WHERE w.id = $1
`
type GetWantForSearchRow struct {
ID pgtype.UUID
ReleaseGroupMbid string
ArtistMbid string
ArtistName string
Title string
Reason string
FulfilledAt pgtype.Timestamptz
Upgrade string
HeldTier string
HeldQuality string
AlbumID pgtype.UUID
PrimaryType string
SecondaryTypes []string
FirstReleaseDate string
}
func (q *Queries) GetWantForSearch(ctx context.Context, id pgtype.UUID) (GetWantForSearchRow, error) {
row := q.db.QueryRow(ctx, getWantForSearch, id)
var i GetWantForSearchRow
err := row.Scan(
&i.ID,
&i.ReleaseGroupMbid,
&i.ArtistMbid,
&i.ArtistName,
&i.Title,
&i.Reason,
&i.FulfilledAt,
&i.Upgrade,
&i.HeldTier,
&i.HeldQuality,
&i.AlbumID,
&i.PrimaryType,
&i.SecondaryTypes,
&i.FirstReleaseDate,
)
return i, err
}
const insertGrab = `-- name: InsertGrab :one
INSERT INTO grabs (want_id, info_hash, name, key, indexer, source, client, save_path, quality,
origin, forced, grabbed_by)
VALUES ($1, $2, $3, $4, $5,
$6, $7, $8, $9,
$10, $11, $12)
RETURNING id, want_id, info_hash, name, key, indexer, source, client, save_path, quality, origin, forced, grabbed_by, grabbed_at, ended_at, end_reason, end_detail, progress, import_state, import_reason, import_detail, import_forced, import_attempts, import_updated_at, imported_at, imported_files, destination, seeding, ratio, seed_released_at
`
type InsertGrabParams struct {
WantID pgtype.UUID
InfoHash string
Name string
Key string
Indexer string
Source string
Client string
SavePath string
Quality string
Origin string
Forced bool
GrabbedBy pgtype.UUID
}
func (q *Queries) InsertGrab(ctx context.Context, arg InsertGrabParams) (Grab, error) {
row := q.db.QueryRow(ctx, insertGrab,
arg.WantID,
arg.InfoHash,
arg.Name,
arg.Key,
arg.Indexer,
arg.Source,
arg.Client,
arg.SavePath,
arg.Quality,
arg.Origin,
arg.Forced,
arg.GrabbedBy,
)
var i Grab
err := row.Scan(
&i.ID,
&i.WantID,
&i.InfoHash,
&i.Name,
&i.Key,
&i.Indexer,
&i.Source,
&i.Client,
&i.SavePath,
&i.Quality,
&i.Origin,
&i.Forced,
&i.GrabbedBy,
&i.GrabbedAt,
&i.EndedAt,
&i.EndReason,
&i.EndDetail,
&i.Progress,
&i.ImportState,
&i.ImportReason,
&i.ImportDetail,
&i.ImportForced,
&i.ImportAttempts,
&i.ImportUpdatedAt,
&i.ImportedAt,
&i.ImportedFiles,
&i.Destination,
&i.Seeding,
&i.Ratio,
&i.SeedReleasedAt,
)
return i, err
}
const insertWantCandidate = `-- name: InsertWantCandidate :exec
INSERT INTO want_candidates (want_id, key, name, info_hash, indexer, source, size_bytes, seeders,
files_count, format, quality, year, verdict, reason, reasons, folded_from, rank)
VALUES ($1, $2, $3, $4, $5,
$6, $7, $8, $9, $10,
$11, $12, $13, $14, $15,
$16, $17)
ON CONFLICT (want_id, key) DO NOTHING
`
type InsertWantCandidateParams struct {
WantID pgtype.UUID
Key string
Name string
InfoHash string
Indexer string
Source string
SizeBytes int64
Seeders int32
FilesCount int32
Format string
Quality string
Year int32
Verdict string
Reason string
Reasons []byte
FoldedFrom []string
Rank int32
}
func (q *Queries) InsertWantCandidate(ctx context.Context, arg InsertWantCandidateParams) error {
_, err := q.db.Exec(ctx, insertWantCandidate,
arg.WantID,
arg.Key,
arg.Name,
arg.InfoHash,
arg.Indexer,
arg.Source,
arg.SizeBytes,
arg.Seeders,
arg.FilesCount,
arg.Format,
arg.Quality,
arg.Year,
arg.Verdict,
arg.Reason,
arg.Reasons,
arg.FoldedFrom,
arg.Rank,
)
return err
}
const listReleaseBlocks = `-- name: ListReleaseBlocks :many
SELECT info_hash, name, reason, detail, want_id, blocked_by, blocked_at, expires_at FROM release_blocks
WHERE expires_at IS NULL OR expires_at > now()
ORDER BY blocked_at DESC
`
// Every release set aside and still held back: a stalled one lifts by itself
// at expires_at.
func (q *Queries) ListReleaseBlocks(ctx context.Context) ([]ReleaseBlock, error) {
rows, err := q.db.Query(ctx, listReleaseBlocks)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ReleaseBlock
for rows.Next() {
var i ReleaseBlock
if err := rows.Scan(
&i.InfoHash,
&i.Name,
&i.Reason,
&i.Detail,
&i.WantID,
&i.BlockedBy,
&i.BlockedAt,
&i.ExpiresAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listWantCandidates = `-- name: ListWantCandidates :many
SELECT want_id, key, name, info_hash, indexer, source, size_bytes, seeders, files_count, format, quality, year, verdict, reason, reasons, folded_from, rank, searched_at FROM want_candidates WHERE want_id = $1 ORDER BY rank, key
`
func (q *Queries) ListWantCandidates(ctx context.Context, wantID pgtype.UUID) ([]WantCandidate, error) {
rows, err := q.db.Query(ctx, listWantCandidates, wantID)
if err != nil {
return nil, err
}
defer rows.Close()
var items []WantCandidate
for rows.Next() {
var i WantCandidate
if err := rows.Scan(
&i.WantID,
&i.Key,
&i.Name,
&i.InfoHash,
&i.Indexer,
&i.Source,
&i.SizeBytes,
&i.Seeders,
&i.FilesCount,
&i.Format,
&i.Quality,
&i.Year,
&i.Verdict,
&i.Reason,
&i.Reasons,
&i.FoldedFrom,
&i.Rank,
&i.SearchedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listWantsDueForSearch = `-- name: ListWantsDueForSearch :many
SELECT w.id, w.release_group_mbid, w.artist_name, w.title, w.reason, w.upgrade, w.held_tier, w.held_quality,
COALESCE(NULLIF(w.primary_type, ''), g.primary_type, '')::text AS primary_type,
COALESCE(g.secondary_types, '{}')::text[] AS secondary_types,
COALESCE(NULLIF(w.first_release_date, ''), g.first_release_date, '')::text AS first_release_date
FROM wants w
LEFT JOIN mb_release_groups g ON g.mbid = w.release_group_mbid
WHERE (w.fulfilled_at IS NULL OR w.upgrade <> '')
AND NOT EXISTS (SELECT 1 FROM grabs gr WHERE gr.want_id = w.id AND gr.ended_at IS NULL)
AND (w.searched_at IS NULL
OR w.searched_at < now() - make_interval(hours => CASE WHEN w.fulfilled_at IS NULL
THEN $1::int
ELSE GREATEST($1::int, 168) END))
ORDER BY w.fulfilled_at IS NOT NULL,
w.searched_at IS NOT NULL,
CASE w.reason WHEN 'request' THEN 0 WHEN 'missing' THEN 1 ELSE 2 END,
w.searched_at NULLS FIRST, w.wanted_at, w.id
LIMIT $2
`
type ListWantsDueForSearchParams struct {
Hours int32
MaxRows int32
}
type ListWantsDueForSearchRow struct {
ID pgtype.UUID
ReleaseGroupMbid string
ArtistName string
Title string
Reason string
Upgrade string
HeldTier string
HeldQuality string
PrimaryType string
SecondaryTypes []string
FirstReleaseDate string
}
// Wants with no download running that were never searched or not since
// @hours ago: open wants first (never-searched, then requests, then missing
// files, then followed artists' releases), then held albums wanted for an
// upgrade, which are searched at most weekly.
func (q *Queries) ListWantsDueForSearch(ctx context.Context, arg ListWantsDueForSearchParams) ([]ListWantsDueForSearchRow, error) {
rows, err := q.db.Query(ctx, listWantsDueForSearch, arg.Hours, arg.MaxRows)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListWantsDueForSearchRow
for rows.Next() {
var i ListWantsDueForSearchRow
if err := rows.Scan(
&i.ID,
&i.ReleaseGroupMbid,
&i.ArtistName,
&i.Title,
&i.Reason,
&i.Upgrade,
&i.HeldTier,
&i.HeldQuality,
&i.PrimaryType,
&i.SecondaryTypes,
&i.FirstReleaseDate,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const markCandidatesBlocked = `-- name: MarkCandidatesBlocked :exec
UPDATE want_candidates
SET verdict = 'blocked', reason = $1,
reasons = jsonb_build_array(jsonb_build_object('kind', 'hard', 'code', 'blocked', 'text', $1::text))
WHERE info_hash = $2
`
type MarkCandidatesBlockedParams struct {
Reason string
InfoHash string
}
func (q *Queries) MarkCandidatesBlocked(ctx context.Context, arg MarkCandidatesBlockedParams) error {
_, err := q.db.Exec(ctx, markCandidatesBlocked, arg.Reason, arg.InfoHash)
return err
}
const markCandidatesUnblocked = `-- name: MarkCandidatesUnblocked :exec
UPDATE want_candidates
SET verdict = 'review', reason = $1,
reasons = jsonb_build_array(jsonb_build_object('kind', 'soft', 'code', 'unblocked', 'text', $1::text))
WHERE info_hash = $2 AND verdict = 'blocked'
`
type MarkCandidatesUnblockedParams struct {
Reason string
InfoHash string
}
func (q *Queries) MarkCandidatesUnblocked(ctx context.Context, arg MarkCandidatesUnblockedParams) error {
_, err := q.db.Exec(ctx, markCandidatesUnblocked, arg.Reason, arg.InfoHash)
return err
}
const recordSearchPass = `-- name: RecordSearchPass :exec
UPDATE acquisition_settings SET last_pass_at = now(), last_pass = $1 WHERE id = true
`
func (q *Queries) RecordSearchPass(ctx context.Context, report []byte) error {
_, err := q.db.Exec(ctx, recordSearchPass, report)
return err
}
const recordWantSearch = `-- name: RecordWantSearch :exec
UPDATE wants
SET searched_at = now(),
search_outcome = $1,
search_considered = $2,
best_release = $3,
best_verdict = $4,
search_error = $5,
updated_at = now()
WHERE id = $6
`
type RecordWantSearchParams struct {
Outcome string
Considered int32
BestRelease string
BestVerdict string
SearchError string
ID pgtype.UUID
}
func (q *Queries) RecordWantSearch(ctx context.Context, arg RecordWantSearchParams) error {
_, err := q.db.Exec(ctx, recordWantSearch,
arg.Outcome,
arg.Considered,
arg.BestRelease,
arg.BestVerdict,
arg.SearchError,
arg.ID,
)
return err
}
const unblockRelease = `-- name: UnblockRelease :execrows
DELETE FROM release_blocks WHERE info_hash = $1
`
func (q *Queries) UnblockRelease(ctx context.Context, infoHash string) (int64, error) {
result, err := q.db.Exec(ctx, unblockRelease, infoHash)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const updateAcquisitionSettings = `-- name: UpdateAcquisitionSettings :one
UPDATE acquisition_settings
SET auto_grab = $1,
search_interval_hours = $2,
allowed_qualities = $3,
import_root = $4,
seed_ratio = $5,
seed_days = $6,
stall_hours = $7,
upgrade_until = $8,
updated_at = now()
WHERE id = true
RETURNING id, auto_grab, search_interval_hours, allowed_qualities, last_pass_at, last_pass, updated_at, import_root, seed_ratio, seed_days, stall_hours, library_listing_started_at, library_listed_at, library_unlisted, upgrade_until
`
type UpdateAcquisitionSettingsParams struct {
AutoGrab bool
SearchIntervalHours int32
AllowedQualities []string
ImportRoot string
SeedRatio float32
SeedDays int32
StallHours int32
UpgradeUntil string
}
func (q *Queries) UpdateAcquisitionSettings(ctx context.Context, arg UpdateAcquisitionSettingsParams) (AcquisitionSetting, error) {
row := q.db.QueryRow(ctx, updateAcquisitionSettings,
arg.AutoGrab,
arg.SearchIntervalHours,
arg.AllowedQualities,
arg.ImportRoot,
arg.SeedRatio,
arg.SeedDays,
arg.StallHours,
arg.UpgradeUntil,
)
var i AcquisitionSetting
err := row.Scan(
&i.ID,
&i.AutoGrab,
&i.SearchIntervalHours,
&i.AllowedQualities,
&i.LastPassAt,
&i.LastPass,
&i.UpdatedAt,
&i.ImportRoot,
&i.SeedRatio,
&i.SeedDays,
&i.StallHours,
&i.LibraryListingStartedAt,
&i.LibraryListedAt,
&i.LibraryUnlisted,
&i.UpgradeUntil,
)
return i, err
}
-32
View File
@@ -340,35 +340,3 @@ func (q *Queries) UpsertArtistSimilarity(ctx context.Context, arg UpsertArtistSi
_, err := q.db.Exec(ctx, upsertArtistSimilarity, arg.ArtistAID, arg.ArtistBID, arg.Score)
return err
}
const upsertArtistSimilarityUnmatched = `-- name: UpsertArtistSimilarityUnmatched :exec
INSERT INTO artist_similarity_unmatched (
seed_artist_id, candidate_mbid, candidate_name, score, source
) VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (seed_artist_id, candidate_mbid, source) DO UPDATE SET
candidate_name = EXCLUDED.candidate_name,
score = EXCLUDED.score,
fetched_at = now()
`
type UpsertArtistSimilarityUnmatchedParams struct {
SeedArtistID pgtype.UUID
CandidateMbid string
CandidateName string
Score float64
Source string
}
// Persists an out-of-library similar-artist MBID. Idempotent on
// (seed_artist_id, candidate_mbid, source) — re-fetches refresh the
// name/score and bump fetched_at.
func (q *Queries) UpsertArtistSimilarityUnmatched(ctx context.Context, arg UpsertArtistSimilarityUnmatchedParams) error {
_, err := q.db.Exec(ctx, upsertArtistSimilarityUnmatched,
arg.SeedArtistID,
arg.CandidateMbid,
arg.CandidateName,
arg.Score,
arg.Source,
)
return err
}
-126
View File
@@ -1,126 +0,0 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.31.1
// source: suggestion_snoozes.sql
package dbq
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const gcDeleteExpiredSuggestionSnoozes = `-- name: GcDeleteExpiredSuggestionSnoozes :execrows
DELETE FROM suggestion_snoozes WHERE snoozed_until < now()
`
// Keeps the table from growing without bound. Every read already filters on
// snoozed_until > now(), so deleting an expired row changes no behaviour —
// this is purely reclamation.
func (q *Queries) GcDeleteExpiredSuggestionSnoozes(ctx context.Context) (int64, error) {
result, err := q.db.Exec(ctx, gcDeleteExpiredSuggestionSnoozes)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const listActiveSuggestionSnoozes = `-- name: ListActiveSuggestionSnoozes :many
SELECT candidate_mbid, candidate_name, snoozed_until, created_at
FROM suggestion_snoozes
WHERE user_id = $1 AND snoozed_until > now()
ORDER BY snoozed_until, candidate_mbid
`
type ListActiveSuggestionSnoozesRow struct {
CandidateMbid string
CandidateName string
SnoozedUntil pgtype.Timestamptz
CreatedAt pgtype.Timestamptz
}
// Backs the manage / un-snooze surface. Expired rows are filtered HERE
// rather than left to the sweeper: gc runs on an hourly tick, so a row can
// outlive its expiry by up to a tick and must not read as still-snoozed in
// the meantime.
func (q *Queries) ListActiveSuggestionSnoozes(ctx context.Context, userID pgtype.UUID) ([]ListActiveSuggestionSnoozesRow, error) {
rows, err := q.db.Query(ctx, listActiveSuggestionSnoozes, userID)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListActiveSuggestionSnoozesRow
for rows.Next() {
var i ListActiveSuggestionSnoozesRow
if err := rows.Scan(
&i.CandidateMbid,
&i.CandidateName,
&i.SnoozedUntil,
&i.CreatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const snoozeSuggestion = `-- name: SnoozeSuggestion :exec
INSERT INTO suggestion_snoozes (user_id, candidate_mbid, candidate_name, snoozed_until)
VALUES ($1, $2, $3, now() + ($4::float8 * INTERVAL '1 day'))
ON CONFLICT (user_id, candidate_mbid) DO UPDATE
SET snoozed_until = EXCLUDED.snoozed_until,
candidate_name = EXCLUDED.candidate_name
`
type SnoozeSuggestionParams struct {
UserID pgtype.UUID
CandidateMbid string
CandidateName string
Column4 float64
}
// Time-boxed "not right now" on a Discover artist suggestion.
//
// See 0049_suggestion_snoozes.up.sql for why this is a snooze and not a
// dismissal: it records no verdict on the music, expires on its own, and
// must never reach the taste profile. Nothing in internal/taste may read
// this table.
// Upsert, so re-snoozing an already-snoozed candidate EXTENDS it instead of
// erroring on the PK. The name is refreshed too — a later suggestion may
// carry a corrected spelling from the similarity feed.
// $1=user_id, $2=candidate_mbid, $3=candidate_name, $4=duration in days.
func (q *Queries) SnoozeSuggestion(ctx context.Context, arg SnoozeSuggestionParams) error {
_, err := q.db.Exec(ctx, snoozeSuggestion,
arg.UserID,
arg.CandidateMbid,
arg.CandidateName,
arg.Column4,
)
return err
}
const unsnoozeSuggestion = `-- name: UnsnoozeSuggestion :execrows
DELETE FROM suggestion_snoozes
WHERE user_id = $1 AND candidate_mbid = $2
`
type UnsnoozeSuggestionParams struct {
UserID pgtype.UUID
CandidateMbid string
}
// Row count is returned so the handler can 404 an MBID that was never
// snoozed rather than reporting success for a no-op.
func (q *Queries) UnsnoozeSuggestion(ctx context.Context, arg UnsnoozeSuggestionParams) (int64, error) {
result, err := q.db.Exec(ctx, unsnoozeSuggestion, arg.UserID, arg.CandidateMbid)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
+1 -13
View File
@@ -716,18 +716,10 @@ SELECT t.id,
artists.id AS artist_id,
artists.name AS artist_name,
t.source_verdict,
t.source_verdict_at,
-- The album's place on the wanted list: wanted for a clean copy, and
-- when that was last searched for.
COALESCE(w.upgrade, '')::text AS want_upgrade,
w.searched_at AS searched_at
t.source_verdict_at
FROM tracks t
JOIN albums ON albums.id = t.album_id
JOIN artists ON artists.id = t.artist_id
LEFT JOIN LATERAL (
SELECT x.upgrade, x.searched_at FROM wants x
WHERE x.album_id = albums.id AND x.fulfilled_at IS NOT NULL LIMIT 1
) w ON true
WHERE t.missing_since IS NULL
AND regexp_replace(t.file_path, '^.*/', '') ~* $1::text
ORDER BY directory, t.disc_number NULLS FIRST, t.track_number NULLS FIRST, t.title
@@ -754,8 +746,6 @@ type ListSuspectSourceTracksRow struct {
ArtistName string
SourceVerdict *string
SourceVerdictAt pgtype.Timestamptz
WantUpgrade string
SearchedAt pgtype.Timestamptz
}
// The admin report of present tracks whose filename looks like a video rip
@@ -791,8 +781,6 @@ func (q *Queries) ListSuspectSourceTracks(ctx context.Context, arg ListSuspectSo
&i.ArtistName,
&i.SourceVerdict,
&i.SourceVerdictAt,
&i.WantUpgrade,
&i.SearchedAt,
); err != nil {
return nil, err
}
+1 -38
View File
@@ -397,8 +397,7 @@ type ListUsernamesByIDsRow struct {
Username string
}
// Names a list's people in one round trip, e.g. who made each request on
// /admin/requests.
// Names a list's people in one round trip.
func (q *Queries) ListUsernamesByIDs(ctx context.Context, dollar_1 []pgtype.UUID) ([]ListUsernamesByIDsRow, error) {
rows, err := q.db.Query(ctx, listUsernamesByIDs, dollar_1)
if err != nil {
@@ -627,42 +626,6 @@ func (q *Queries) UpdateUserAdmin(ctx context.Context, arg UpdateUserAdminParams
return i, err
}
const updateUserAutoApprove = `-- name: UpdateUserAutoApprove :one
UPDATE users
SET auto_approve_requests = $2
WHERE id = $1
RETURNING id, username, password_hash, is_admin, created_at, subsonic_password, listenbrainz_token, listenbrainz_enabled, display_name, auto_approve_requests, email, timezone, timezone_updated_at, debug_mode_enabled, api_token_hash
`
type UpdateUserAutoApproveParams struct {
ID pgtype.UUID
AutoApproveRequests bool
}
// Toggle the per-user auto_approve_requests flag.
func (q *Queries) UpdateUserAutoApprove(ctx context.Context, arg UpdateUserAutoApproveParams) (User, error) {
row := q.db.QueryRow(ctx, updateUserAutoApprove, arg.ID, arg.AutoApproveRequests)
var i User
err := row.Scan(
&i.ID,
&i.Username,
&i.PasswordHash,
&i.IsAdmin,
&i.CreatedAt,
&i.SubsonicPassword,
&i.ListenbrainzToken,
&i.ListenbrainzEnabled,
&i.DisplayName,
&i.AutoApproveRequests,
&i.Email,
&i.Timezone,
&i.TimezoneUpdatedAt,
&i.DebugModeEnabled,
&i.ApiTokenHash,
)
return i, err
}
const updateUserProfile = `-- name: UpdateUserProfile :one
UPDATE users
SET display_name = $2,
File diff suppressed because it is too large Load Diff
-76
View File
@@ -1,76 +0,0 @@
-- Indexers and download clients.
-- name: ListAcquisitionServices :many
-- Returns no secret, only whether one is set: this feeds the settings page.
SELECT code, kind, protocol, label, enabled, base_url,
(api_key <> '')::boolean AS has_api_key,
(password <> '')::boolean AS has_password,
username, category, timeout_ms,
serves_search, serves_downloads, serves_enrichment,
local_path, remote_path, searches_per_hour,
created_at, updated_at
FROM acquisition_services
ORDER BY kind, label, code;
-- name: GetAcquisitionService :one
-- The full row, credentials included. For building a client, never for a
-- response.
SELECT * FROM acquisition_services WHERE code = sqlc.arg(code);
-- name: ListUsableAcquisitionServices :many
-- Every enabled, addressed service doing one job. An exchange row does a job
-- when its switch for it is on; any other row does the job its kind names.
-- Ordered by code so a fan-out is deterministic.
SELECT * FROM acquisition_services
WHERE enabled AND base_url <> ''
AND CASE WHEN protocol = 'exchange' THEN
CASE sqlc.arg(job)::text
WHEN 'search' THEN serves_search
WHEN 'download' THEN serves_downloads
WHEN 'enrichment' THEN serves_enrichment
ELSE false
END
ELSE kind = sqlc.arg(job)::text
END
ORDER BY code;
-- name: CreateAcquisitionService :one
-- ON CONFLICT so the caller can try the next code when this one is taken.
INSERT INTO acquisition_services
(code, kind, protocol, label, category, timeout_ms, searches_per_hour)
VALUES
(sqlc.arg(code), sqlc.arg(kind), sqlc.arg(protocol), sqlc.arg(label),
sqlc.arg(category), sqlc.arg(timeout_ms), sqlc.arg(searches_per_hour))
ON CONFLICT (code) DO NOTHING
RETURNING *;
-- name: UpdateAcquisitionService :one
-- Everything on the settings form except the credentials, which have their
-- own write-only calls.
UPDATE acquisition_services SET
label = sqlc.arg(label),
enabled = sqlc.arg(enabled),
base_url = sqlc.arg(base_url),
username = sqlc.arg(username),
category = sqlc.arg(category),
timeout_ms = sqlc.arg(timeout_ms),
serves_search = sqlc.arg(serves_search),
serves_downloads = sqlc.arg(serves_downloads),
serves_enrichment = sqlc.arg(serves_enrichment),
local_path = sqlc.arg(local_path),
remote_path = sqlc.arg(remote_path),
searches_per_hour = sqlc.arg(searches_per_hour),
updated_at = now()
WHERE code = sqlc.arg(code)
RETURNING *;
-- name: DeleteAcquisitionService :execrows
DELETE FROM acquisition_services WHERE code = sqlc.arg(code);
-- name: SetAcquisitionServiceAPIKey :execrows
UPDATE acquisition_services SET api_key = sqlc.arg(api_key), updated_at = now()
WHERE code = sqlc.arg(code);
-- name: SetAcquisitionServicePassword :execrows
UPDATE acquisition_services SET password = sqlc.arg(password), updated_at = now()
WHERE code = sqlc.arg(code);
+2 -16
View File
@@ -1,5 +1,5 @@
-- The admin Overview's status sections: who is listening, what is
-- downloading, what the library holds. Each is one cheap read.
-- The admin Overview's status sections: who is listening and what the
-- library holds. Each is one cheap read.
-- name: ListListeningNow :many
-- Plays in progress: open play events still inside their track's length,
@@ -29,20 +29,6 @@ SELECT COUNT(*)::bigint AS plays,
FROM play_events
WHERE started_at > now() - INTERVAL '24 hours';
-- name: DownloadsStatus :one
-- Downloads at a glance: running, waiting on a person (and since when), and
-- the last day's imports and give-ups.
SELECT COUNT(*) FILTER (WHERE ended_at IS NULL AND import_state IN ('', 'downloading', 'waiting', 'importing'))::bigint AS running,
COUNT(*) FILTER (WHERE ended_at IS NULL AND import_state IN ('review', 'failed'))::bigint AS needs_you,
MIN(COALESCE(import_updated_at, grabbed_at)) FILTER (WHERE ended_at IS NULL AND import_state IN ('review', 'failed'))::timestamptz AS needs_you_since,
COUNT(*) FILTER (WHERE imported_at > now() - INTERVAL '24 hours')::bigint AS imported_day,
COUNT(*) FILTER (WHERE ended_at > now() - INTERVAL '24 hours'
AND end_reason IN ('stalled', 'failed', 'gone', 'abandoned'))::bigint AS given_up_day
FROM grabs
WHERE ended_at IS NULL
OR ended_at > now() - INTERVAL '24 hours'
OR imported_at > now() - INTERVAL '24 hours';
-- name: LibraryStatus :one
-- What the library holds on disk: tracks whose files are present, the
-- albums and artists they make, and their size.
-2
View File
@@ -5,8 +5,6 @@
-- oldest item still waiting: how long the queue has been left, which is what
-- tells an admin a count of 2 is fine today and wrong after a month.
SELECT
(SELECT count(*) FROM requests WHERE status = 'pending')::bigint AS requests,
(SELECT min(requested_at) FROM requests WHERE status = 'pending')::timestamptz AS requests_since,
-- A broken track is one report however it was found: a listener flagged
-- it, or it failed to play. Counted by track, as the Reports
-- page lists them.
-16
View File
@@ -215,22 +215,6 @@ UPDATE albums
updated_at = now()
WHERE id = $1 AND release_group_mbid IS NULL;
-- name: GetReleaseGroupForReleaseMbid :one
-- The release group the library already knows for a release id, so a
-- request carrying a release id can be repaired without asking MusicBrainz.
SELECT release_group_mbid::text
FROM albums
WHERE mbid = $1 AND release_group_mbid IS NOT NULL
LIMIT 1;
-- name: SetReleaseGroupForReleaseMbidIfNull :exec
-- Cache a MusicBrainz-resolved release group onto the album row that
-- holds the release, filling only a NULL.
UPDATE albums
SET release_group_mbid = sqlc.arg(release_group_mbid)::text,
updated_at = now()
WHERE mbid = sqlc.arg(release_mbid)::text AND release_group_mbid IS NULL;
-- name: ListAlbumsMissingMbidWithTrack :many
-- One-shot MBID backfill: returns each album where mbid IS NULL alongside
-- one of its tracks' file_path so the worker can re-read tags. LIMIT
@@ -1,103 +0,0 @@
-- Folksonomy tags for out-of-library Discover candidates. Parallel to
-- track_tags.sql, but keyed by MBID because the artist has no local row. See
-- 0050_candidate_artist_tags.up.sql for why the bookkeeping is its own table.
-- name: ListCandidateArtistsMissingTags :many
-- Candidates eligible for tag enrichment: never processed (no state row) or
-- settled 'none' under an older provider version.
--
-- artist_similarity_unmatched holds one row per (seed, candidate, source), so
-- this GROUPs to one row per candidate — enriching the same MBID once per seed
-- that pointed at it would multiply the API calls for no gain.
--
-- ORDER BY summed similarity DESC is the load-bearing part. The candidate pool
-- is O(library artists x neighbours per artist) — thousands — and MusicBrainz
-- allows ~1 req/s, so it can NEVER be fully enriched in one pass. Draining in
-- strength order means the candidates most likely to actually reach a user's
-- deck get tags first, and the long tail fills in over subsequent ticks
-- instead of starving behind it.
--
-- Already-in-library candidates are skipped: they have an artists row, so
-- their tags belong in track_tags, and the suggestion query filters them out
-- anyway. $1 = current tag_sources_version, $2 = limit.
-- candidate_name is coalesced to the empty string so it lands non-nullable in
-- Go: the name is only a Last.fm lookup key, and empty simply means "MBID-keyed
-- providers only", which the provider chain already handles. max() is an
-- arbitrary-but-deterministic pick when several seeds spell one MBID
-- differently.
SELECT u.candidate_mbid,
coalesce(max(u.candidate_name), '')::text AS candidate_name,
sum(u.score)::float8 AS total_score
FROM artist_similarity_unmatched u
LEFT JOIN candidate_artist_tag_state s ON s.candidate_mbid = u.candidate_mbid
WHERE NOT EXISTS (SELECT 1 FROM artists a WHERE a.mbid = u.candidate_mbid)
AND (
s.candidate_mbid IS NULL
OR (s.tag_source = 'none' AND s.tag_sources_version < $1)
)
GROUP BY u.candidate_mbid
ORDER BY total_score DESC, u.candidate_mbid
LIMIT $2;
-- name: DeleteCandidateArtistTags :exec
-- Clear a candidate's cached tags before rewriting (atomic replace by the
-- caller, same shape as DeleteTrackTags).
DELETE FROM candidate_artist_tags WHERE candidate_mbid = $1;
-- name: InsertCandidateArtistTag :exec
-- Upsert one (candidate, tag); keep the stronger weight when two providers
-- agree on a tag with different folksonomy strengths.
INSERT INTO candidate_artist_tags (candidate_mbid, tag, weight)
VALUES ($1, $2, $3)
ON CONFLICT (candidate_mbid, tag)
DO UPDATE SET weight = GREATEST(candidate_artist_tags.weight, EXCLUDED.weight);
-- name: SetCandidateArtistTagState :exec
-- Stamp the enrichment outcome so the drainer skips settled candidates.
-- $2 = 'musicbrainz' | 'lastfm' | 'mixed' | 'none', $3 = current version.
INSERT INTO candidate_artist_tag_state (candidate_mbid, tag_source, tag_sources_version)
VALUES ($1, $2, $3)
ON CONFLICT (candidate_mbid) DO UPDATE
SET tag_source = EXCLUDED.tag_source,
tag_sources_version = EXCLUDED.tag_sources_version,
updated_at = now();
-- name: ListCandidateArtistTagsForMbids :many
-- Cached tags for a set of candidates, for the taste-overlap ranking.
-- One row per (candidate, tag).
SELECT candidate_mbid, tag, weight
FROM candidate_artist_tags
WHERE candidate_mbid = ANY($1::text[]);
-- name: CountCandidateArtistTagCoverage :one
-- Operator-facing coverage: how many distinct candidates have been processed,
-- and how many of those actually yielded tags. The gap is the honest ceiling
-- from the task — obscure artists with no MBID presence or no upstream tags
-- stay thin no matter how long the worker runs, and that is worth being able
-- to see rather than inferring from a silent surface.
SELECT count(*)::bigint AS processed,
count(*) FILTER (WHERE tag_source <> 'none')::bigint AS with_tags
FROM candidate_artist_tag_state;
-- name: GcDeleteOrphanedCandidateArtistTags :execrows
-- Drops cached tags for candidates that no longer appear in the similarity
-- feed, or that have since been added to the library (their tags now live in
-- track_tags). The feed is refetched periodically and churns, so without this
-- the cache only ever grows.
DELETE FROM candidate_artist_tags t
WHERE NOT EXISTS (
SELECT 1 FROM artist_similarity_unmatched u
WHERE u.candidate_mbid = t.candidate_mbid
)
OR EXISTS (SELECT 1 FROM artists a WHERE a.mbid = t.candidate_mbid);
-- name: GcDeleteOrphanedCandidateArtistTagState :execrows
-- Same sweep for the bookkeeping rows. Kept as a separate statement rather
-- than a cascade: the two tables are independent by design (a 'none' outcome
-- has state but no tags), so neither can be the parent of the other.
DELETE FROM candidate_artist_tag_state s
WHERE NOT EXISTS (
SELECT 1 FROM artist_similarity_unmatched u
WHERE u.candidate_mbid = s.candidate_mbid
)
OR EXISTS (SELECT 1 FROM artists a WHERE a.mbid = s.candidate_mbid);
+2 -49
View File
@@ -1,34 +1,8 @@
-- The release catalogue read from MusicBrainz.
-- name: UpsertMBArtist :exec
INSERT INTO mb_artists (mbid, name, sort_name, disambiguation, artist_type, country)
VALUES (sqlc.arg(mbid), sqlc.arg(name), sqlc.arg(sort_name), sqlc.arg(disambiguation),
sqlc.arg(artist_type), sqlc.arg(country))
ON CONFLICT (mbid) DO UPDATE SET
name = EXCLUDED.name,
sort_name = EXCLUDED.sort_name,
disambiguation = EXCLUDED.disambiguation,
artist_type = EXCLUDED.artist_type,
country = EXCLUDED.country,
updated_at = now();
-- name: GetMBArtist :one
SELECT * FROM mb_artists WHERE mbid = sqlc.arg(mbid);
-- name: MarkMBArtistDiscographyRefreshed :exec
UPDATE mb_artists SET discography_refreshed_at = now() WHERE mbid = sqlc.arg(mbid);
-- name: ListMBArtistsDueForRefresh :many
-- Artists whose discography has never been read, or was read before the
-- cutoff: never-read first, then oldest first.
SELECT mbid FROM mb_artists
WHERE discography_refreshed_at IS NULL OR discography_refreshed_at < sqlc.arg(before)
ORDER BY discography_refreshed_at NULLS FIRST, mbid
LIMIT sqlc.arg(max_rows);
-- name: UpsertMBReleaseGroup :exec
-- Leaves releases_refreshed_at alone: re-reading a discography says nothing
-- about whether the group's track lists are current.
-- Leaves releases_refreshed_at alone: only a full read of the group's
-- releases and track lists marks it.
INSERT INTO mb_release_groups (mbid, title, primary_type, secondary_types, first_release_date, artist_credit)
VALUES (sqlc.arg(mbid), sqlc.arg(title), sqlc.arg(primary_type), sqlc.arg(secondary_types)::text[],
sqlc.arg(first_release_date), sqlc.arg(artist_credit))
@@ -43,21 +17,6 @@ ON CONFLICT (mbid) DO UPDATE SET
-- name: GetMBReleaseGroup :one
SELECT * FROM mb_release_groups WHERE mbid = sqlc.arg(mbid);
-- name: DeleteMBArtistReleaseGroups :exec
DELETE FROM mb_artist_release_groups WHERE artist_mbid = sqlc.arg(artist_mbid);
-- name: AddMBArtistReleaseGroup :exec
INSERT INTO mb_artist_release_groups (artist_mbid, release_group_mbid)
VALUES (sqlc.arg(artist_mbid), sqlc.arg(release_group_mbid))
ON CONFLICT DO NOTHING;
-- name: ListMBArtistReleaseGroups :many
-- An artist's discography, oldest first; undated groups last.
SELECT g.* FROM mb_release_groups g
JOIN mb_artist_release_groups ag ON ag.release_group_mbid = g.mbid
WHERE ag.artist_mbid = sqlc.arg(artist_mbid)
ORDER BY g.first_release_date = '', g.first_release_date, g.title, g.mbid;
-- name: MarkMBReleaseGroupReleasesRefreshed :exec
UPDATE mb_release_groups SET releases_refreshed_at = now() WHERE mbid = sqlc.arg(mbid);
@@ -88,12 +47,6 @@ VALUES (sqlc.arg(release_mbid), sqlc.arg(medium_position), sqlc.arg(position),
sqlc.arg(medium_format), sqlc.arg(number), sqlc.arg(title), sqlc.arg(length_ms),
sqlc.arg(recording_mbid));
-- name: ListMBReleasesForGroup :many
-- Official editions first, then by date; undated last.
SELECT * FROM mb_releases
WHERE release_group_mbid = sqlc.arg(release_group_mbid)
ORDER BY status <> 'Official', release_date = '', release_date, title, mbid;
-- name: ListMBReleaseTracksForGroup :many
SELECT t.* FROM mb_release_tracks t
JOIN mb_releases r ON r.mbid = t.release_mbid
-176
View File
@@ -1,176 +0,0 @@
-- Import with verification.
-- name: ListLiveGrabs :many
-- Every download still running or waiting on its import, oldest first.
SELECT g.*, w.release_group_mbid, w.artist_mbid, w.artist_name, w.title AS want_title,
w.reason AS want_reason, w.album_id AS want_album_id
FROM grabs g
LEFT JOIN wants w ON w.id = g.want_id
WHERE g.ended_at IS NULL
ORDER BY g.grabbed_at, g.id;
-- name: GetGrab :one
SELECT * FROM grabs WHERE id = $1;
-- name: SetGrabProgress :exec
-- What the download client says of a download not yet finished, or finished
-- and waiting for what the import needs. Leaves a claimed, decided or failed
-- import alone.
UPDATE grabs
SET progress = sqlc.arg(progress),
import_state = sqlc.arg(import_state),
import_reason = sqlc.arg(import_reason),
import_updated_at = now()
WHERE id = sqlc.arg(id) AND ended_at IS NULL
AND import_state IN ('', 'downloading', 'waiting');
-- name: ClaimGrabImport :one
-- Takes a finished download for import. The update is the lock: a second
-- pass finds nothing to claim.
UPDATE grabs
SET import_state = 'importing', import_attempts = import_attempts + 1,
import_updated_at = now(), progress = 1
WHERE id = $1 AND ended_at IS NULL AND import_state IN ('', 'downloading', 'waiting')
RETURNING *;
-- name: ReclaimStaleImports :execrows
-- An import cut short (a restart mid-copy) is tried again.
UPDATE grabs
SET import_state = 'waiting',
import_reason = 'The last import was cut short, so it is being tried again.'
WHERE ended_at IS NULL AND import_state = 'importing'
AND import_updated_at < now() - make_interval(mins => sqlc.arg(minutes)::int);
-- name: RecordImport :one
-- The import's outcome. An end reason ends the grab: imported, refused or
-- held. Without one the grab stays live: review and failed wait for a person.
UPDATE grabs
SET import_state = sqlc.arg(import_state),
import_reason = sqlc.arg(import_reason),
import_detail = sqlc.arg(import_detail),
import_updated_at = now(),
import_forced = false,
imported_at = CASE WHEN sqlc.arg(import_state)::text = 'imported' THEN now() ELSE imported_at END,
-- The files were copied, so an imported download goes on sharing
-- until the lifecycle removes it.
seeding = sqlc.arg(import_state)::text = 'imported',
imported_files = sqlc.arg(imported_files),
destination = sqlc.arg(destination),
ended_at = CASE WHEN sqlc.arg(end_reason)::text <> '' THEN now() ELSE ended_at END,
end_reason = sqlc.arg(end_reason),
end_detail = CASE WHEN sqlc.arg(end_reason)::text <> '' THEN sqlc.arg(import_reason)::text ELSE end_detail END
WHERE id = sqlc.arg(id)
RETURNING *;
-- name: ForceGrabImport :one
-- "Import anyway": the next pass imports the download whatever the check said.
UPDATE grabs
SET import_state = '', import_forced = true, import_reason = '', import_updated_at = now()
WHERE id = $1 AND ended_at IS NULL AND import_state IN ('review', 'failed')
RETURNING *;
-- name: RetryGrabImport :one
UPDATE grabs
SET import_state = '', import_reason = '', import_updated_at = now()
WHERE id = $1 AND ended_at IS NULL AND import_state = 'failed'
RETURNING *;
-- name: RehomeGrab :one
-- A download that turned out to be another wanted album is that want's grab,
-- when the want has none running.
UPDATE grabs g
SET want_id = sqlc.arg(want_id)
WHERE g.id = sqlc.arg(id) AND g.ended_at IS NULL
AND NOT EXISTS (SELECT 1 FROM grabs o WHERE o.want_id = sqlc.arg(want_id) AND o.ended_at IS NULL)
RETURNING *;
-- name: OpenWantsForGroups :many
-- Open wants for these release groups with no download running.
SELECT w.* FROM wants w
WHERE w.fulfilled_at IS NULL AND w.release_group_mbid = ANY(sqlc.arg(groups)::text[])
AND NOT EXISTS (SELECT 1 FROM grabs g WHERE g.want_id = w.id AND g.ended_at IS NULL);
-- name: MarkWantSearched :exec
-- Imported: nothing to search for until the wanted pass settles the want.
UPDATE wants SET searched_at = now() WHERE id = $1;
-- name: SearchWantAgain :exec
-- Refused: search again on the next pass.
UPDATE wants SET searched_at = NULL, search_outcome = '' WHERE id = $1;
-- name: HeldRecordings :many
-- Which of these recordings the library holds a file for.
SELECT DISTINCT mbid::text AS mbid FROM tracks
WHERE mbid = ANY(sqlc.arg(recordings)::text[]) AND missing_since IS NULL;
-- name: HeldPositions :many
-- The disc and track positions the library holds for an album of this group.
SELECT DISTINCT COALESCE(t.disc_number, 1)::int AS disc, t.track_number::int AS track
FROM tracks t
JOIN albums a ON a.id = t.album_id
WHERE a.release_group_mbid = sqlc.arg(release_group_mbid)
AND t.missing_since IS NULL AND t.track_number IS NOT NULL;
-- name: ListGrabs :many
-- The Wanted view's Activity tab: downloads waiting on a person first, then
-- the rest still running, then the ended ones newest first. Each carries
-- what is wrong with it now, if anything.
SELECT g.id, g.want_id, g.name, g.indexer, g.client, g.quality, g.origin, g.forced,
g.grabbed_at, g.ended_at, g.end_reason, g.end_detail, g.progress, g.import_state,
g.import_reason, g.import_detail, g.import_updated_at, g.imported_at, g.imported_files,
g.destination, g.seeding, g.ratio, g.seed_released_at,
w.title AS want_title, w.artist_name AS want_artist,
di.trigger AS condition, di.detail AS condition_detail, di.remediations AS condition_retries,
di.first_seen AS condition_since
FROM grabs g
LEFT JOIN wants w ON w.id = g.want_id
LEFT JOIN download_interventions di
ON di.id = (SELECT x.id FROM download_interventions x
WHERE x.grab_id = g.id AND x.outcome = 'open'
ORDER BY x.last_seen DESC LIMIT 1)
ORDER BY (g.ended_at IS NULL AND g.import_state IN ('review', 'failed')) DESC,
g.ended_at IS NULL DESC, g.grabbed_at DESC, g.id
LIMIT sqlc.arg(max_rows) OFFSET sqlc.arg(skip);
-- name: ListReleaseBlocksPage :many
SELECT b.info_hash, b.name, b.reason, b.detail, b.blocked_at, b.expires_at, w.title AS want_title,
w.artist_name AS want_artist
FROM release_blocks b
LEFT JOIN wants w ON w.id = b.want_id
WHERE b.expires_at IS NULL OR b.expires_at > now()
ORDER BY b.blocked_at DESC, b.info_hash
LIMIT sqlc.arg(max_rows) OFFSET sqlc.arg(skip);
-- name: ListUpgradeTracks :many
-- The tracks an upgrade may replace: the present tracks of the album
-- the want holds.
SELECT id, file_path, mbid, COALESCE(disc_number, 1)::int AS disc, track_number,
file_format, file_size, duration_ms, source_verdict
FROM tracks
WHERE album_id = $1 AND missing_since IS NULL;
-- name: FinishUpgrade :exec
-- An upgrade was imported: the want holds the album the new tracks are in,
-- and is judged again by the next upgrade check.
UPDATE wants
SET album_id = COALESCE(sqlc.narg(album_id), album_id),
upgrade = '',
upgrade_checked_at = NULL,
updated_at = now()
WHERE id = sqlc.arg(id);
-- name: ClearTrackSourceVerdict :exec
-- A track whose file was replaced in place is no longer the rip it was.
UPDATE tracks SET source_verdict = NULL, source_verdict_at = NULL
WHERE id = $1 AND source_verdict = 'suspect';
-- name: ListImportedGrabsHolding :many
-- The downloads Minstrel imported into the folder holding path: the releases a
-- file there came from, to block when the files proved not to be the album
--. A file imported before Minstrel downloaded anything has none.
SELECT id, info_hash, name, want_id
FROM grabs
WHERE import_state = 'imported'
AND info_hash <> ''
AND destination <> ''
AND starts_with(sqlc.arg(path)::text, rtrim(destination, '/') || '/');
-11
View File
@@ -23,17 +23,6 @@ UPDATE albums
SET cover_art_path = sqlc.arg(new_root)::text || substr(cover_art_path, length(sqlc.arg(old_root)::text) + 1)
WHERE starts_with(cover_art_path, sqlc.arg(old_root)::text || '/');
-- name: RerootGrabDestinations :execrows
UPDATE grabs
SET destination = sqlc.arg(new_root)::text || substr(destination, length(sqlc.arg(old_root)::text) + 1)
WHERE destination = sqlc.arg(old_root)::text
OR starts_with(destination, sqlc.arg(old_root)::text || '/');
-- name: RerootImportRoot :execrows
UPDATE acquisition_settings
SET import_root = sqlc.arg(new_root)::text
WHERE import_root = sqlc.arg(old_root)::text;
-- name: SampleTrackPathsUnder :many
-- A few present tracks under a folder, to check a move before rewriting it.
SELECT file_path FROM tracks
-102
View File
@@ -1,102 +0,0 @@
-- The download lifecycle: the intervention ledger, giving a
-- download up, and seeding after import.
-- name: RaiseIntervention :one
-- A condition seen, or seen again: a second pass lands on the open row, so
-- a download stuck for a week is one row that says how long. The name only
-- improves, and the detail is the latest word.
INSERT INTO download_interventions (client, info_hash, grab_id, name, trigger, detail)
VALUES (sqlc.arg(client), sqlc.arg(info_hash), sqlc.narg(grab_id), sqlc.arg(name),
sqlc.arg(trigger), sqlc.arg(detail))
ON CONFLICT (client, info_hash, trigger) WHERE outcome = 'open' DO UPDATE SET
observations = download_interventions.observations + 1,
last_seen = now(),
detail = EXCLUDED.detail,
grab_id = COALESCE(EXCLUDED.grab_id, download_interventions.grab_id),
name = CASE WHEN EXCLUDED.name <> '' THEN EXCLUDED.name ELSE download_interventions.name END
RETURNING *;
-- name: SettleIntervention :execrows
-- The condition cleared.
UPDATE download_interventions
SET outcome = 'resolved', settled_at = now(), last_seen = now()
WHERE client = sqlc.arg(client) AND info_hash = sqlc.arg(info_hash)
AND trigger = sqlc.arg(trigger) AND outcome = 'open';
-- name: SettleInterventionsFor :execrows
-- Everything open about one download is over: it is gone from the client.
UPDATE download_interventions
SET outcome = 'resolved', settled_at = now(), last_seen = now()
WHERE client = sqlc.arg(client) AND info_hash = sqlc.arg(info_hash) AND outcome = 'open';
-- name: SettleEndedGrabs :execrows
-- What was wrong with a download whose grab has ended (imported, refused,
-- given up) is over, except a download left in the client to remove.
UPDATE download_interventions di
SET outcome = 'resolved', settled_at = now(), last_seen = now()
FROM grabs g
WHERE g.id = di.grab_id AND g.ended_at IS NOT NULL
AND di.outcome = 'open' AND di.trigger <> 'not_removed';
-- name: CountRemediation :one
-- Minstrel acted on an open condition. A retry climbs the ladder; giving up
-- or removing ends the condition, since what it was about is gone. Written
-- before the act, so a retry that happened is never uncounted.
UPDATE download_interventions
SET remediations = remediations + 1,
action = sqlc.arg(action),
last_seen = now(),
outcome = CASE WHEN sqlc.arg(action)::text IN ('given_up', 'removed') THEN 'resolved' ELSE outcome END,
settled_at = CASE WHEN sqlc.arg(action)::text IN ('given_up', 'removed') THEN now() ELSE settled_at END
WHERE client = sqlc.arg(client) AND info_hash = sqlc.arg(info_hash)
AND trigger = sqlc.arg(trigger) AND outcome = 'open'
RETURNING remediations;
-- name: ListLeftovers :many
-- Downloads given up on that the client would not remove; each pass asks
-- again.
SELECT * FROM download_interventions
WHERE trigger = 'not_removed' AND outcome = 'open' AND info_hash <> ''
ORDER BY first_seen;
-- name: ListClientsDown :many
-- Download clients not answering now, for the Activity tab.
SELECT client, detail, first_seen FROM download_interventions
WHERE trigger = 'client_unreachable' AND outcome = 'open'
ORDER BY client;
-- name: LiveGrabOnHash :one
SELECT EXISTS (SELECT 1 FROM grabs WHERE info_hash = sqlc.arg(info_hash) AND ended_at IS NULL)::bool;
-- name: GiveUpGrab :one
-- Ends a running grab, given up on. Only from the states named: the update
-- is the lock against an import that claimed the download meanwhile.
UPDATE grabs
SET ended_at = now(), end_reason = sqlc.arg(end_reason), end_detail = sqlc.arg(end_detail),
import_updated_at = now()
WHERE id = sqlc.arg(id) AND ended_at IS NULL AND import_state = ANY(sqlc.arg(states)::text[])
RETURNING *;
-- name: CountGoneGrabs :one
-- How many of a release's grabs ended gone since a time: a release that keeps
-- leaving its client is held back rather than taken on every pass.
SELECT count(*) FROM grabs
WHERE info_hash = sqlc.arg(info_hash) AND end_reason = 'gone' AND ended_at > sqlc.arg(since);
-- name: ListSeedingGrabs :many
-- Imported downloads still sharing, oldest import first.
SELECT id, client, info_hash, name, imported_at FROM grabs
WHERE seeding AND seed_released_at IS NULL
ORDER BY imported_at, id;
-- name: SetGrabRatio :exec
UPDATE grabs SET ratio = sqlc.arg(ratio) WHERE id = sqlc.arg(id);
-- name: ReleaseSeed :exec
-- The seeding download was removed, or found gone.
UPDATE grabs SET seed_released_at = now() WHERE id = $1 AND seed_released_at IS NULL;
-- name: DeleteExpiredBlocks :execrows
-- A release given up on is held back for a while; after that the search may
-- take it again.
DELETE FROM release_blocks WHERE expires_at IS NOT NULL AND expires_at <= now();
-4
View File
@@ -33,10 +33,6 @@ UPDATE contextual_likes SET track_id = sqlc.arg(survivor_id)::uuid WHERE track_i
-- name: MergeRepointPlaybackErrors :execrows
UPDATE playback_errors SET track_id = sqlc.arg(survivor_id)::uuid WHERE track_id = sqlc.arg(loser_id)::uuid;
-- name: MergeRepointRequests :execrows
UPDATE requests SET matched_track_id = sqlc.arg(survivor_id)::uuid
WHERE matched_track_id = sqlc.arg(loser_id)::uuid;
-- name: MergeRepointPlaylistTracks :many
-- playlist_tracks is keyed by (playlist_id, position), so repointing keeps every
-- entry exactly where it was. A playlist that held both copies simply holds the
-93
View File
@@ -1,93 +0,0 @@
-- Re-acquisition of missing files.
-- The unit is the album: an album whose files went missing becomes a want for
-- its release group, and the search schedule looks for it.
-- name: GetReacquisitionSettings :one
SELECT * FROM reacquisition_settings WHERE id = true;
-- name: UpdateReacquisitionSettings :one
-- Whole-row write from the admin card; the CHECKs in migration 0056 are the
-- validation, so a bad value fails loudly rather than being clamped silently.
UPDATE reacquisition_settings
SET enabled = sqlc.arg(enabled),
grace_hours = sqlc.arg(grace_hours),
max_per_pass = sqlc.arg(max_per_pass)
WHERE id = true
RETURNING *;
-- name: ListAlbumsDueReacquisition :many
-- The sweeper's selection. An album qualifies when:
--
-- * it still has at least one track whose file has been missing longer than
-- the grace window. Measured on missing_since, which reconcile never
-- re-stamps, so it is a genuine "gone since" clock;
-- * MusicBrainz can name it: the artist has an MBID, and the album a
-- release MBID or a release group. The release MBID is only ever used to
-- find the group, and an album matched to a group may carry no release
-- id. Albums failing this are counted separately
-- (CountAlbumsMissingWithoutMbid);
-- * it is on the wanted list: an album taken off the list, or
-- dropped into the library by hand, is not fetched back;
-- * no want already covers it: none open, and none held since its files
-- went missing. A want held after the loss means the library has a full
-- copy despite the missing rows, so asking again would loop.
SELECT albums.id AS album_id,
albums.title AS album_title,
albums.mbid AS album_mbid,
albums.release_group_mbid AS album_release_group_mbid,
artists.name AS artist_name,
artists.mbid AS artist_mbid,
COUNT(tracks.id)::bigint AS missing_track_count,
MAX(tracks.missing_since)::timestamptz AS last_missing_since
FROM albums
JOIN artists ON artists.id = albums.artist_id
JOIN tracks ON tracks.album_id = albums.id
WHERE tracks.missing_since IS NOT NULL
AND tracks.missing_since <= now() - make_interval(hours => sqlc.arg(grace_hours)::int)
AND (albums.mbid IS NOT NULL OR albums.release_group_mbid IS NOT NULL)
AND artists.mbid IS NOT NULL
GROUP BY albums.id, albums.title, albums.mbid, albums.release_group_mbid,
artists.name, artists.mbid
HAVING EXISTS (
SELECT 1 FROM wants w
WHERE w.album_id = albums.id OR w.release_group_mbid = albums.release_group_mbid)
AND NOT EXISTS (
SELECT 1 FROM wants w
WHERE (w.album_id = albums.id OR w.release_group_mbid = albums.release_group_mbid)
AND (w.fulfilled_at IS NULL OR w.fulfilled_at > MAX(tracks.missing_since)))
ORDER BY MIN(tracks.missing_since), albums.sort_title
LIMIT sqlc.arg(page_limit);
-- name: CountAlbumsMissingWithoutMbid :one
-- Albums with missing files that can never be wanted automatically because
-- nothing identifies them to MusicBrainz. Surfaced on the admin card so the
-- gap is visible: silently doing nothing for these would read as broken.
SELECT COUNT(DISTINCT albums.id)::bigint
FROM albums
JOIN artists ON artists.id = albums.artist_id
JOIN tracks ON tracks.album_id = albums.id
WHERE tracks.missing_since IS NOT NULL
AND ((albums.mbid IS NULL AND albums.release_group_mbid IS NULL) OR artists.mbid IS NULL);
-- name: WantMissingAlbum :one
-- Records an album with missing files as wanted. A held want for the same
-- group is reopened; an open one is left as it is, and no row comes back.
INSERT INTO wants (release_group_mbid, artist_mbid, artist_name, title, reason, album_id)
VALUES (sqlc.arg(release_group_mbid), sqlc.arg(artist_mbid), sqlc.arg(artist_name),
sqlc.arg(title), 'missing', sqlc.arg(album_id))
ON CONFLICT (release_group_mbid) DO UPDATE
SET reason = 'missing',
album_id = EXCLUDED.album_id,
wanted_at = now(),
fulfilled_at = NULL,
updated_at = now()
WHERE wants.fulfilled_at IS NOT NULL
RETURNING *;
-- name: GetWantsForAlbums :many
-- What the wanted list says about each album on the missing-files page.
SELECT a.id AS album_id, w.id AS want_id, w.wanted_at, w.fulfilled_at
FROM albums a
JOIN wants w ON w.album_id = a.id OR w.release_group_mbid = a.release_group_mbid
WHERE a.id = ANY(sqlc.arg(album_ids)::uuid[])
ORDER BY w.wanted_at DESC;
-103
View File
@@ -310,109 +310,6 @@ WHERE user_id = $1
ORDER BY started_at DESC
LIMIT 1;
-- name: SuggestArtistsForUser :many
-- Per-user artist suggestions ranked by taste signal x similarity, projected
-- through artist_similarity_unmatched (out-of-library candidates only).
--
-- Seeds are TIERED so the surface never empties:
-- tier 1 - taste_profile_artists.weight: engagement-graded, time-decayed and
-- SIGNED by internal/taste, so an artist the user has drifted away
-- from stops contributing instead of accumulating forever.
-- tier 2 - likes + completed plays, used ONLY when the profile has no rows
-- (new account, or before the first daily recompute).
--
-- The signal is log-damped: contribution is signal x similarity, and an
-- undamped sum would let one heavily-played artist's neighbours take every
-- slot -- entrenching harder the MORE the user listened.
--
-- Candidates already in the library, already requested and not terminal, or
-- snoozed by this user, are excluded. $1=user_id, $2=half_life_days (tier 2
-- decay), $3=limit.
WITH artist_plays AS (
-- Completed plays only: counting every play_event would make skipping an
-- artist repeatedly INCREASE its signal and push more of its neighbours
-- at the user.
SELECT t.artist_id, count(*)::bigint AS play_count
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1 AND pe.was_skipped = false
GROUP BY t.artist_id
),
profile_seeds AS (
SELECT tpa.artist_id, tpa.weight AS raw_signal
FROM taste_profile_artists tpa
WHERE tpa.user_id = $1 AND tpa.weight > 0
),
fallback_seeds AS (
SELECT a.id AS artist_id,
5.0 * (CASE WHEN gla.artist_id IS NOT NULL THEN 1 ELSE 0 END)
+ COALESCE(SUM(EXP(- EXTRACT(epoch FROM now() - pe.started_at) / ($2::float8 * 86400.0))), 0)
AS raw_signal
FROM artists a
LEFT JOIN general_likes_artists gla ON gla.artist_id = a.id AND gla.user_id = $1
LEFT JOIN tracks t ON t.artist_id = a.id
LEFT JOIN play_events pe
ON pe.track_id = t.id AND pe.user_id = $1 AND pe.was_skipped = false
WHERE (gla.artist_id IS NOT NULL OR pe.id IS NOT NULL)
AND NOT EXISTS (SELECT 1 FROM profile_seeds)
GROUP BY a.id, gla.artist_id
),
seeds AS (
SELECT s.artist_id,
ln(1.0 + s.raw_signal) AS signal,
(gla.artist_id IS NOT NULL) AS is_liked,
COALESCE(ap.play_count, 0)::bigint AS play_count
FROM (
SELECT artist_id, raw_signal FROM profile_seeds
UNION ALL
SELECT artist_id, raw_signal FROM fallback_seeds
) s
LEFT JOIN general_likes_artists gla
ON gla.artist_id = s.artist_id AND gla.user_id = $1
LEFT JOIN artist_plays ap ON ap.artist_id = s.artist_id
WHERE s.raw_signal > 0
),
contributions AS (
SELECT u.candidate_mbid,
u.candidate_name,
seeds.artist_id AS seed_id,
seeds.is_liked,
seeds.play_count,
seeds.signal * u.score AS contribution
FROM artist_similarity_unmatched u
JOIN seeds ON seeds.artist_id = u.seed_artist_id
WHERE NOT EXISTS (SELECT 1 FROM artists WHERE mbid = u.candidate_mbid)
AND NOT EXISTS (
SELECT 1 FROM requests r
WHERE r.user_id = $1
AND r.artist_mbid = u.candidate_mbid
AND r.status NOT IN ('rejected', 'failed')
)
-- Snoozed by this user and not yet expired. Time-boxed and
-- per-user: the candidate returns on its own once snoozed_until
-- passes, and stays visible to everyone else meanwhile. Deliberately
-- filtered at the candidate stage, NOT folded into the score — a
-- snooze carries no opinion about the music, so it must not become a
-- ranking signal.
AND NOT EXISTS (
SELECT 1 FROM suggestion_snoozes s
WHERE s.user_id = $1
AND s.candidate_mbid = u.candidate_mbid
AND s.snoozed_until > now()
)
)
SELECT candidate_mbid,
candidate_name,
SUM(contribution)::float8 AS total_score,
((array_agg(seed_id ORDER BY contribution DESC))[1:3])::uuid[] AS top_seed_ids,
((array_agg(contribution ORDER BY contribution DESC))[1:3])::float8[] AS top_contributions,
((array_agg(is_liked ORDER BY contribution DESC))[1:3])::boolean[] AS top_is_liked,
((array_agg(play_count ORDER BY contribution DESC))[1:3])::bigint[] AS top_play_counts
FROM contributions
GROUP BY candidate_mbid, candidate_name
ORDER BY total_score DESC
LIMIT $3;
-- name: ListMostPlayedTracksForUser :many
-- Top-N tracks by completed-play count for the user. was_skipped
-- excludes skips so a user spamming next doesn't fabricate a top track.
@@ -53,24 +53,6 @@ UPDATE taste_tuning
WHERE singleton = true
RETURNING *;
-- name: UpsertDiscoverTuningDefaults :exec
-- Boot reconcile for the Discover scope. Never overwrites
-- admin-tuned values, same contract as the other two.
INSERT INTO discover_tuning (singleton, tag_overlap_weight, snooze_days)
VALUES (true, $1, $2)
ON CONFLICT (singleton) DO NOTHING;
-- name: GetDiscoverTuning :one
SELECT * FROM discover_tuning WHERE singleton = true;
-- name: UpdateDiscoverTuning :one
UPDATE discover_tuning
SET tag_overlap_weight = $1,
snooze_days = $2,
updated_at = now()
WHERE singleton = true
RETURNING *;
-- name: InsertTuningAudit :exec
-- changes is a jsonb array of {field, old, new} objects.
INSERT INTO recommendation_tuning_audit (scope, action, changes)
-119
View File
@@ -1,119 +0,0 @@
-- name: CreateRequest :one
INSERT INTO requests (
user_id, kind,
artist_mbid, album_mbid, track_mbid,
artist_name, album_title, track_title
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
RETURNING *;
-- name: GetRequestByID :one
SELECT * FROM requests WHERE id = $1;
-- name: ListRequestsForUser :many
SELECT * FROM requests
WHERE user_id = $1
ORDER BY requested_at DESC
LIMIT $2;
-- name: ListRequestsByStatus :many
SELECT * FROM requests
WHERE status = $1
ORDER BY requested_at DESC
LIMIT $2;
-- name: ListApprovedRequestsForReconcile :many
SELECT * FROM requests
WHERE status = 'approved'
ORDER BY decided_at ASC
LIMIT $1;
-- name: ApproveRequest :one
UPDATE requests
SET status = 'approved',
decided_at = now(),
decided_by = $2,
updated_at = now()
WHERE id = $1 AND status = 'pending'
RETURNING *;
-- name: RejectRequest :one
UPDATE requests
SET status = 'rejected',
notes = $2,
decided_at = now(),
decided_by = $3,
updated_at = now()
WHERE id = $1 AND status = 'pending'
RETURNING *;
-- name: CancelRequest :one
-- $2 serves as both the ownership guard (user_id = $2) and the
-- decided_by audit field. The generated parameter struct names it
-- DecidedBy — callers must pass the cancelling user's UUID, and
-- a wrong UUID will silently no-op (zero rows updated) rather than
-- erroring at the SQL layer.
UPDATE requests
SET status = 'rejected',
notes = 'cancelled by user',
decided_at = now(),
decided_by = $2,
updated_at = now()
WHERE id = $1 AND user_id = $2 AND status = 'pending'
RETURNING *;
-- name: CompleteRequest :one
-- Reconciler transitions an approved request to completed when its
-- target track/album/artist has appeared in the library. Callers
-- should set ONLY the matched_*_id corresponding to the request's
-- kind and pass pgtype.UUID{} (Valid: false) for the others —
-- per-kind one-of-three is the contract, NULL for the others
-- preserves auditability if the matched row is later evicted.
UPDATE requests
SET status = 'completed',
matched_track_id = $2,
matched_album_id = $3,
matched_artist_id = $4,
completed_at = now(),
updated_at = now()
WHERE id = $1 AND status = 'approved'
RETURNING *;
-- name: HasNonTerminalRequestForMBID :one
-- Whether something is already on its way: a pending, approved or completed
-- request at the given level, an open want for the release group, or a
-- follow of the artist. Sets the `requested` flag on catalogue search.
-- Terminal-status (rejected, failed) requests do not count. MBIDs are unique
-- per entity type, so checking one id against every column cannot
-- false-match.
SELECT (EXISTS (
SELECT 1 FROM requests r
WHERE r.status IN ('pending', 'approved', 'completed')
AND ((r.kind = 'artist' AND r.artist_mbid = @mbid)
OR (r.kind = 'album' AND r.album_mbid = @mbid)
OR (r.kind = 'track' AND r.track_mbid = @mbid))
) OR EXISTS (
SELECT 1 FROM wants w WHERE w.release_group_mbid = @mbid AND w.fulfilled_at IS NULL
) OR EXISTS (
SELECT 1 FROM follows f WHERE f.artist_mbid = @mbid
))::bool AS requested;
-- name: GetNonTerminalRequestForMBID :one
-- Returns the oldest non-terminal (pending/approved/completed) request
-- whose own-kind MBID matches @mbid, or pgx.ErrNoRows when none. Create()
-- uses this to dedupe: a re-request for something already in flight
-- returns the existing row instead of inserting a duplicate.
SELECT * FROM requests
WHERE status IN ('pending', 'approved', 'completed')
AND ((kind = 'artist' AND artist_mbid = @mbid)
OR (kind = 'album' AND album_mbid = @mbid)
OR (kind = 'track' AND track_mbid = @mbid))
ORDER BY requested_at ASC
LIMIT 1;
-- name: RepointRequestsAlbumMbid :exec
-- A want moved from a release id to its group takes the requests
-- that name it along, so the reconciler finds it.
UPDATE requests
SET album_mbid = sqlc.arg(new_mbid),
updated_at = now()
WHERE album_mbid = sqlc.arg(old_mbid);
-141
View File
@@ -1,141 +0,0 @@
-- Searching for wants and grabbing releases.
-- name: GetAcquisitionSettings :one
SELECT * FROM acquisition_settings WHERE id = true;
-- name: UpdateAcquisitionSettings :one
UPDATE acquisition_settings
SET auto_grab = sqlc.arg(auto_grab),
search_interval_hours = sqlc.arg(search_interval_hours),
allowed_qualities = sqlc.arg(allowed_qualities),
import_root = sqlc.arg(import_root),
seed_ratio = sqlc.arg(seed_ratio),
seed_days = sqlc.arg(seed_days),
stall_hours = sqlc.arg(stall_hours),
upgrade_until = sqlc.arg(upgrade_until),
updated_at = now()
WHERE id = true
RETURNING *;
-- name: RecordSearchPass :exec
UPDATE acquisition_settings SET last_pass_at = now(), last_pass = sqlc.arg(report) WHERE id = true;
-- name: ListWantsDueForSearch :many
-- Wants with no download running that were never searched or not since
-- @hours ago: open wants first (never-searched, then requests, then missing
-- files, then followed artists' releases), then held albums wanted for an
-- upgrade, which are searched at most weekly.
SELECT w.id, w.release_group_mbid, w.artist_name, w.title, w.reason, w.upgrade, w.held_tier, w.held_quality,
COALESCE(NULLIF(w.primary_type, ''), g.primary_type, '')::text AS primary_type,
COALESCE(g.secondary_types, '{}')::text[] AS secondary_types,
COALESCE(NULLIF(w.first_release_date, ''), g.first_release_date, '')::text AS first_release_date
FROM wants w
LEFT JOIN mb_release_groups g ON g.mbid = w.release_group_mbid
WHERE (w.fulfilled_at IS NULL OR w.upgrade <> '')
AND NOT EXISTS (SELECT 1 FROM grabs gr WHERE gr.want_id = w.id AND gr.ended_at IS NULL)
AND (w.searched_at IS NULL
OR w.searched_at < now() - make_interval(hours => CASE WHEN w.fulfilled_at IS NULL
THEN sqlc.arg(hours)::int
ELSE GREATEST(sqlc.arg(hours)::int, 168) END))
ORDER BY w.fulfilled_at IS NOT NULL,
w.searched_at IS NOT NULL,
CASE w.reason WHEN 'request' THEN 0 WHEN 'missing' THEN 1 ELSE 2 END,
w.searched_at NULLS FIRST, w.wanted_at, w.id
LIMIT sqlc.arg(max_rows);
-- name: GetWantForSearch :one
SELECT w.id, w.release_group_mbid, w.artist_mbid, w.artist_name, w.title, w.reason, w.fulfilled_at,
w.upgrade, w.held_tier, w.held_quality, w.album_id,
COALESCE(NULLIF(w.primary_type, ''), g.primary_type, '')::text AS primary_type,
COALESCE(g.secondary_types, '{}')::text[] AS secondary_types,
COALESCE(NULLIF(w.first_release_date, ''), g.first_release_date, '')::text AS first_release_date
FROM wants w
LEFT JOIN mb_release_groups g ON g.mbid = w.release_group_mbid
WHERE w.id = $1;
-- name: EditionTrackSpan :one
-- The fewest tracks an official edition of the group has (any edition when
-- none is official), and the most any has. Zero when none is catalogued.
SELECT COALESCE(MIN(track_count) FILTER (WHERE status = 'Official'), MIN(track_count), 0)::int AS min_tracks,
COALESCE(MAX(track_count), 0)::int AS max_tracks
FROM mb_releases
WHERE release_group_mbid = $1 AND track_count > 0;
-- name: RecordWantSearch :exec
UPDATE wants
SET searched_at = now(),
search_outcome = sqlc.arg(outcome),
search_considered = sqlc.arg(considered),
best_release = sqlc.arg(best_release),
best_verdict = sqlc.arg(best_verdict),
search_error = sqlc.arg(search_error),
updated_at = now()
WHERE id = sqlc.arg(id);
-- name: DeleteWantCandidates :exec
DELETE FROM want_candidates WHERE want_id = $1;
-- name: InsertWantCandidate :exec
INSERT INTO want_candidates (want_id, key, name, info_hash, indexer, source, size_bytes, seeders,
files_count, format, quality, year, verdict, reason, reasons, folded_from, rank)
VALUES (sqlc.arg(want_id), sqlc.arg(key), sqlc.arg(name), sqlc.arg(info_hash), sqlc.arg(indexer),
sqlc.arg(source), sqlc.arg(size_bytes), sqlc.arg(seeders), sqlc.arg(files_count), sqlc.arg(format),
sqlc.arg(quality), sqlc.arg(year), sqlc.arg(verdict), sqlc.arg(reason), sqlc.arg(reasons),
sqlc.arg(folded_from), sqlc.arg(rank))
ON CONFLICT (want_id, key) DO NOTHING;
-- name: ListWantCandidates :many
SELECT * FROM want_candidates WHERE want_id = $1 ORDER BY rank, key;
-- name: GetWantCandidate :one
SELECT * FROM want_candidates WHERE want_id = $1 AND key = $2;
-- name: MarkCandidatesBlocked :exec
UPDATE want_candidates
SET verdict = 'blocked', reason = sqlc.arg(reason),
reasons = jsonb_build_array(jsonb_build_object('kind', 'hard', 'code', 'blocked', 'text', sqlc.arg(reason)::text))
WHERE info_hash = sqlc.arg(info_hash);
-- name: MarkCandidatesUnblocked :exec
UPDATE want_candidates
SET verdict = 'review', reason = sqlc.arg(reason),
reasons = jsonb_build_array(jsonb_build_object('kind', 'soft', 'code', 'unblocked', 'text', sqlc.arg(reason)::text))
WHERE info_hash = sqlc.arg(info_hash) AND verdict = 'blocked';
-- name: GetLiveGrab :one
SELECT * FROM grabs WHERE want_id = $1 AND ended_at IS NULL;
-- name: InsertGrab :one
INSERT INTO grabs (want_id, info_hash, name, key, indexer, source, client, save_path, quality,
origin, forced, grabbed_by)
VALUES (sqlc.narg(want_id), sqlc.arg(info_hash), sqlc.arg(name), sqlc.arg(key), sqlc.arg(indexer),
sqlc.arg(source), sqlc.arg(client), sqlc.arg(save_path), sqlc.arg(quality),
sqlc.arg(origin), sqlc.arg(forced), sqlc.narg(grabbed_by))
RETURNING *;
-- name: EndGrab :exec
UPDATE grabs
SET ended_at = now(), end_reason = sqlc.arg(end_reason), end_detail = sqlc.arg(end_detail)
WHERE id = sqlc.arg(id) AND ended_at IS NULL;
-- name: ListReleaseBlocks :many
-- Every release set aside and still held back: a stalled one lifts by itself
-- at expires_at.
SELECT * FROM release_blocks
WHERE expires_at IS NULL OR expires_at > now()
ORDER BY blocked_at DESC;
-- name: BlockRelease :exec
-- A release set aside. expires_at NULL is for good. A block for good is never
-- shortened by a timed one: only a timed block, or another for good, is
-- replaced.
INSERT INTO release_blocks (info_hash, name, reason, detail, want_id, blocked_by, expires_at)
VALUES (sqlc.arg(info_hash), sqlc.arg(name), sqlc.arg(reason), sqlc.arg(detail),
sqlc.narg(want_id), sqlc.narg(blocked_by), sqlc.narg(expires_at))
ON CONFLICT (info_hash) DO UPDATE
SET reason = EXCLUDED.reason, detail = EXCLUDED.detail, blocked_at = now(),
expires_at = EXCLUDED.expires_at, blocked_by = EXCLUDED.blocked_by
WHERE release_blocks.expires_at IS NOT NULL OR EXCLUDED.expires_at IS NULL;
-- name: UnblockRelease :execrows
DELETE FROM release_blocks WHERE info_hash = $1;
-12
View File
@@ -121,18 +121,6 @@ VALUES ($1, $2, $3, 'listenbrainz', now())
ON CONFLICT (artist_a_id, artist_b_id, source)
DO UPDATE SET score = EXCLUDED.score, fetched_at = EXCLUDED.fetched_at;
-- name: UpsertArtistSimilarityUnmatched :exec
-- Persists an out-of-library similar-artist MBID. Idempotent on
-- (seed_artist_id, candidate_mbid, source) — re-fetches refresh the
-- name/score and bump fetched_at.
INSERT INTO artist_similarity_unmatched (
seed_artist_id, candidate_mbid, candidate_name, score, source
) VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (seed_artist_id, candidate_mbid, source) DO UPDATE SET
candidate_name = EXCLUDED.candidate_name,
score = EXCLUDED.score,
fetched_at = now();
-- name: ListSimilarArtistsForArtist :many
-- In-library artists similar to a seed artist, ranked by best similarity
-- score across sources (deduped per candidate). cover_album_id + album_count
@@ -1,39 +0,0 @@
-- Time-boxed "not right now" on a Discover artist suggestion.
--
-- See 0049_suggestion_snoozes.up.sql for why this is a snooze and not a
-- dismissal: it records no verdict on the music, expires on its own, and
-- must never reach the taste profile. Nothing in internal/taste may read
-- this table.
-- name: SnoozeSuggestion :exec
-- Upsert, so re-snoozing an already-snoozed candidate EXTENDS it instead of
-- erroring on the PK. The name is refreshed too — a later suggestion may
-- carry a corrected spelling from the similarity feed.
-- $1=user_id, $2=candidate_mbid, $3=candidate_name, $4=duration in days.
INSERT INTO suggestion_snoozes (user_id, candidate_mbid, candidate_name, snoozed_until)
VALUES ($1, $2, $3, now() + ($4::float8 * INTERVAL '1 day'))
ON CONFLICT (user_id, candidate_mbid) DO UPDATE
SET snoozed_until = EXCLUDED.snoozed_until,
candidate_name = EXCLUDED.candidate_name;
-- name: UnsnoozeSuggestion :execrows
-- Row count is returned so the handler can 404 an MBID that was never
-- snoozed rather than reporting success for a no-op.
DELETE FROM suggestion_snoozes
WHERE user_id = $1 AND candidate_mbid = $2;
-- name: ListActiveSuggestionSnoozes :many
-- Backs the manage / un-snooze surface. Expired rows are filtered HERE
-- rather than left to the sweeper: gc runs on an hourly tick, so a row can
-- outlive its expiry by up to a tick and must not read as still-snoozed in
-- the meantime.
SELECT candidate_mbid, candidate_name, snoozed_until, created_at
FROM suggestion_snoozes
WHERE user_id = $1 AND snoozed_until > now()
ORDER BY snoozed_until, candidate_mbid;
-- name: GcDeleteExpiredSuggestionSnoozes :execrows
-- Keeps the table from growing without bound. Every read already filters on
-- snoozed_until > now(), so deleting an expired row changes no behaviour —
-- this is purely reclamation.
DELETE FROM suggestion_snoozes WHERE snoozed_until < now();
+1 -9
View File
@@ -305,18 +305,10 @@ SELECT t.id,
artists.id AS artist_id,
artists.name AS artist_name,
t.source_verdict,
t.source_verdict_at,
-- The album's place on the wanted list: wanted for a clean copy, and
-- when that was last searched for.
COALESCE(w.upgrade, '')::text AS want_upgrade,
w.searched_at AS searched_at
t.source_verdict_at
FROM tracks t
JOIN albums ON albums.id = t.album_id
JOIN artists ON artists.id = t.artist_id
LEFT JOIN LATERAL (
SELECT x.upgrade, x.searched_at FROM wants x
WHERE x.album_id = albums.id AND x.fulfilled_at IS NOT NULL LIMIT 1
) w ON true
WHERE t.missing_since IS NULL
AND regexp_replace(t.file_path, '^.*/', '') ~* sqlc.arg(pattern)::text
ORDER BY directory, t.disc_number NULLS FIRST, t.track_number NULLS FIRST, t.title
+1 -9
View File
@@ -44,8 +44,7 @@ UPDATE users SET subsonic_password = $2 WHERE id = $1;
SELECT * FROM users WHERE id = $1;
-- name: ListUsernamesByIDs :many
-- Names a list's people in one round trip, e.g. who made each request on
-- /admin/requests.
-- Names a list's people in one round trip.
SELECT id, username FROM users WHERE id = ANY($1::uuid[]);
-- name: SetListenBrainzToken :exec
@@ -115,13 +114,6 @@ UPDATE users
SET password_hash = $2
WHERE id = $1;
-- name: UpdateUserAutoApprove :one
-- Toggle the per-user auto_approve_requests flag.
UPDATE users
SET auto_approve_requests = $2
WHERE id = $1
RETURNING *;
-- name: SetDebugMode :one
-- Toggle the per-account diagnostics/debug-reporting opt-in. Admin-driven
-- from /admin (flip remotely while a bug is live) or self-driven OFF from
-501
View File
@@ -1,501 +0,0 @@
-- The wanted list: release-type profiles, follows and wants.
-- name: ListReleaseTypeProfiles :many
SELECT p.*,
(SELECT COUNT(*) FROM follows f WHERE f.profile_id = p.id)::bigint AS follow_count
FROM release_type_profiles p
ORDER BY p.is_default DESC, p.name;
-- name: GetReleaseTypeProfile :one
SELECT * FROM release_type_profiles WHERE id = $1;
-- name: GetDefaultReleaseTypeProfile :one
SELECT * FROM release_type_profiles WHERE is_default;
-- name: CreateReleaseTypeProfile :one
INSERT INTO release_type_profiles (name, primary_types, secondary_types)
VALUES ($1, $2, $3)
RETURNING *;
-- name: UpdateReleaseTypeProfile :one
UPDATE release_type_profiles
SET name = $2,
primary_types = $3,
secondary_types = $4,
updated_at = now()
WHERE id = $1
RETURNING *;
-- name: ClearDefaultReleaseTypeProfile :exec
UPDATE release_type_profiles SET is_default = false, updated_at = now() WHERE is_default;
-- name: SetDefaultReleaseTypeProfile :execrows
UPDATE release_type_profiles SET is_default = true, updated_at = now() WHERE id = $1;
-- name: DeleteReleaseTypeProfile :execrows
-- The default stays; one in use is refused by the follows foreign key.
DELETE FROM release_type_profiles WHERE id = $1 AND NOT is_default;
-- name: UpsertFollow :one
-- Following an artist already followed changes its profile. want_existing
-- only ever turns on here; the worker turns it off once applied, and turning
-- it on again re-reads the baseline so the existing releases are judged.
INSERT INTO follows (artist_mbid, artist_name, profile_id, followed_by, want_existing)
VALUES (sqlc.arg(artist_mbid), sqlc.arg(artist_name), sqlc.arg(profile_id),
sqlc.narg(followed_by), sqlc.arg(want_existing))
ON CONFLICT (artist_mbid) DO UPDATE
SET artist_name = EXCLUDED.artist_name,
profile_id = EXCLUDED.profile_id,
want_existing = follows.want_existing OR EXCLUDED.want_existing,
baseline_at = CASE WHEN EXCLUDED.want_existing THEN NULL ELSE follows.baseline_at END,
updated_at = now()
RETURNING *;
-- name: GetFollow :one
SELECT f.*, p.name AS profile_name
FROM follows f
JOIN release_type_profiles p ON p.id = f.profile_id
WHERE f.artist_mbid = $1;
-- name: DeleteFollow :execrows
DELETE FROM follows WHERE artist_mbid = $1;
-- name: ListFollows :many
SELECT f.artist_mbid, f.artist_name, f.profile_id, f.followed_at, f.baseline_at, f.checked_at,
p.name AS profile_name,
(SELECT a.id FROM artists a WHERE a.mbid = f.artist_mbid) AS artist_id,
(SELECT COUNT(*) FROM wants w
WHERE w.artist_mbid = f.artist_mbid AND w.fulfilled_at IS NULL)::bigint AS open_wants
FROM follows f
JOIN release_type_profiles p ON p.id = f.profile_id
ORDER BY lower(f.artist_name), f.artist_mbid;
-- name: ListFollowsDue :many
-- Follows whose discography has not been applied since @before, never-read
-- first, so a new follow's baseline is not stuck behind daily re-checks.
SELECT f.*
FROM follows f
WHERE f.baseline_at IS NULL OR f.checked_at IS NULL OR f.checked_at < sqlc.arg(before)
ORDER BY f.baseline_at NULLS FIRST, f.checked_at NULLS FIRST, f.artist_mbid
LIMIT sqlc.arg(max_rows);
-- name: MarkFollowChecked :exec
UPDATE follows
SET checked_at = now(),
baseline_at = COALESCE(baseline_at, now()),
want_existing = false,
updated_at = now()
WHERE artist_mbid = $1;
-- name: ListFollowGroups :many
-- An artist's release groups from the catalogue, each with whether the
-- follow has judged it before.
SELECT g.mbid, g.title, g.primary_type, g.secondary_types, g.first_release_date,
EXISTS (SELECT 1 FROM follow_seen_groups s
WHERE s.artist_mbid = sqlc.arg(artist_mbid) AND s.release_group_mbid = g.mbid) AS seen
FROM mb_artist_release_groups ag
JOIN mb_release_groups g ON g.mbid = ag.release_group_mbid
WHERE ag.artist_mbid = sqlc.arg(artist_mbid)
ORDER BY g.first_release_date, g.title;
-- name: MarkGroupSeen :exec
INSERT INTO follow_seen_groups (artist_mbid, release_group_mbid)
VALUES ($1, $2)
ON CONFLICT DO NOTHING;
-- name: LibraryHoldsReleaseGroup :one
-- Whether the library already has the group: an album tagged with it or
-- with one of its releases, or failing that an album of the artist's with
-- the same title. Only albums with a track on disk count.
SELECT EXISTS (
SELECT 1 FROM albums a
WHERE (a.release_group_mbid = sqlc.arg(group_mbid)::text
OR a.mbid = sqlc.arg(group_mbid)
OR a.mbid IN (SELECT r.mbid FROM mb_releases r WHERE r.release_group_mbid = sqlc.arg(group_mbid))
OR (lower(a.title) = lower(sqlc.arg(title)::text)
AND a.artist_id IN (SELECT ar.id FROM artists ar WHERE ar.mbid = sqlc.arg(artist_mbid)::text)))
AND EXISTS (SELECT 1 FROM tracks t WHERE t.album_id = a.id AND t.missing_since IS NULL)
);
-- name: WantReleaseGroup :one
-- Wants a release group, or returns the want already recorded for it. A
-- held want is not reopened here: the library has it.
INSERT INTO wants (release_group_mbid, artist_mbid, artist_name, title, primary_type,
first_release_date, reason, created_by)
VALUES (sqlc.arg(release_group_mbid), sqlc.arg(artist_mbid), sqlc.arg(artist_name), sqlc.arg(title),
sqlc.arg(primary_type), sqlc.arg(first_release_date), sqlc.arg(reason), sqlc.narg(created_by))
ON CONFLICT (release_group_mbid) DO UPDATE SET updated_at = wants.updated_at
RETURNING *;
-- name: GetWantByReleaseGroup :one
SELECT * FROM wants WHERE release_group_mbid = $1;
-- name: ListOpenWants :many
SELECT * FROM wants
WHERE fulfilled_at IS NULL
ORDER BY wanted_at, id
LIMIT $1;
-- name: WantHoldings :many
-- Each library album that may answer a want, with what it has on disk:
-- tracks present, whether one arrived since the want, and whether a track
-- missing before the want is still missing.
SELECT a.id AS album_id,
COUNT(*) FILTER (WHERE t.missing_since IS NULL)::int AS present,
COALESCE(bool_or(t.missing_since IS NULL AND t.added_at > sqlc.arg(since)::timestamptz), false)::bool AS arrived_since,
COALESCE(bool_or(t.missing_since IS NOT NULL AND t.missing_since < sqlc.arg(since)::timestamptz), false)::bool AS missing_before,
-- Missing tracks no track on disk stands in for, by position or
-- title. A missing row whose song is back at a new path is
-- covered; a lost bonus track is not. A track with no disc number is
-- on disc 1, so a missing track 6 with no disc beside a present 1/6
-- does not keep the album wanted.
COUNT(*) FILTER (WHERE t.missing_since IS NOT NULL AND NOT EXISTS (
SELECT 1 FROM tracks p
WHERE p.album_id = a.id AND p.missing_since IS NULL
AND ((p.track_number = t.track_number AND COALESCE(p.disc_number, 1) = COALESCE(t.disc_number, 1))
OR lower(p.title) = lower(t.title))))::int AS uncovered_missing
FROM albums a
JOIN tracks t ON t.album_id = a.id
WHERE a.release_group_mbid = sqlc.arg(group_mbid)::text
OR a.mbid = sqlc.arg(group_mbid)
OR a.id = sqlc.narg(album_id)
OR a.mbid IN (SELECT r.mbid FROM mb_releases r WHERE r.release_group_mbid = sqlc.arg(group_mbid))
GROUP BY a.id
ORDER BY present DESC, a.id;
-- name: FulfillWant :exec
UPDATE wants
SET fulfilled_at = now(),
album_id = sqlc.arg(album_id),
updated_at = now()
WHERE id = sqlc.arg(id) AND fulfilled_at IS NULL;
-- name: RepointWant :exec
-- A want recorded under a release id is moved to its group.
UPDATE wants SET release_group_mbid = $2, updated_at = now() WHERE id = $1;
-- name: DeleteWant :execrows
DELETE FROM wants WHERE id = $1;
-- name: ListWants :many
-- The Wanted page: open wants (all but missing ones, or only missing ones),
-- the held ones, or all of them; artist_mbid narrows to one artist's page,
-- want_ids to given wants. Ordered like the library: artists A–Z by the
-- library's sort name when it holds the artist, each artist's albums by
-- release date, so the page groups them as they arrive. q narrows to an
-- artist or album title; total counts every match, past the page.
SELECT w.id, w.release_group_mbid, w.artist_mbid, w.artist_name, w.title, w.reason,
w.album_id, w.wanted_at, w.fulfilled_at,
-- Why a held album is wanted again, and the copy it has.
w.upgrade, w.held_quality,
-- A request names an album before the catalogue has read it: the
-- catalogue's type and date fill in once it has.
COALESCE(NULLIF(w.primary_type, ''), g.primary_type, '')::text AS primary_type,
COALESCE(g.secondary_types, '{}')::text[] AS secondary_types,
COALESCE(NULLIF(w.first_release_date, ''), g.first_release_date, '')::text AS first_release_date,
u.username AS requested_by,
la.id AS library_artist_id,
COUNT(*) OVER () AS total,
-- The search schedule's last look, and the download running.
w.searched_at, w.search_outcome, w.search_considered, w.best_release, w.best_verdict, w.search_error,
-- Its latest download, running or ended, and what the import made of
-- it.
gr.id AS grab_id, gr.name AS grab_name, gr.grabbed_at, gr.origin AS grab_origin,
gr.quality AS grab_quality, gr.ended_at AS grab_ended_at, gr.end_reason AS grab_end_reason,
gr.progress AS grab_progress, gr.import_state AS grab_import_state,
gr.import_reason AS grab_import_reason, gr.end_detail AS grab_end_detail, gr.imported_at AS grab_imported_at,
gr.imported_files AS grab_imported_files
FROM wants w
LEFT JOIN users u ON u.id = w.created_by
LEFT JOIN mb_release_groups g ON g.mbid = w.release_group_mbid
LEFT JOIN LATERAL (SELECT ar.id, ar.sort_name FROM artists ar WHERE ar.mbid = w.artist_mbid LIMIT 1) la ON true
LEFT JOIN grabs gr ON gr.id = (SELECT x.id FROM grabs x WHERE x.want_id = w.id
ORDER BY x.ended_at IS NULL DESC, x.grabbed_at DESC LIMIT 1)
WHERE CASE sqlc.arg(state)::text
WHEN 'all' THEN true
WHEN 'held' THEN w.fulfilled_at IS NOT NULL
WHEN 'upgrade' THEN w.fulfilled_at IS NOT NULL AND w.upgrade <> ''
WHEN 'missing' THEN w.fulfilled_at IS NULL AND w.reason = 'missing'
ELSE w.fulfilled_at IS NULL AND w.reason <> 'missing'
END
AND (sqlc.arg(artist_mbid)::text = '' OR w.artist_mbid = sqlc.arg(artist_mbid)::text)
-- want_ids narrows to given wants: the Missing page's folders.
AND (COALESCE(cardinality(sqlc.arg(want_ids)::uuid[]), 0) = 0 OR w.id = ANY(sqlc.arg(want_ids)::uuid[]))
AND (sqlc.arg(q)::text = ''
OR strpos(lower(w.artist_name), lower(sqlc.arg(q)::text)) > 0
OR strpos(lower(w.title), lower(sqlc.arg(q)::text)) > 0)
ORDER BY lower(COALESCE(NULLIF(la.sort_name, ''), w.artist_name)), w.artist_mbid,
NULLIF(COALESCE(NULLIF(w.first_release_date, ''), g.first_release_date, ''), '') NULLS LAST,
lower(w.title), w.id
LIMIT sqlc.arg(max_rows) OFFSET sqlc.arg(skip);
-- name: ListWantArtists :many
-- The Wanted view's main page: every artist on the list, A–Z like
-- the library, with how many of their albums are held of how many are on
-- the list. q matches the artist's name or any of their albums' titles;
-- artist_mbid narrows to one artist, for their page's header.
WITH a AS (
SELECT w.artist_mbid,
(array_agg(w.artist_name ORDER BY w.wanted_at DESC))[1] AS artist_name,
COUNT(*) FILTER (WHERE w.fulfilled_at IS NOT NULL)::int AS held,
COUNT(*) FILTER (WHERE w.fulfilled_at IS NULL AND w.reason <> 'missing')::int AS wanted,
COUNT(*) FILTER (WHERE w.fulfilled_at IS NULL AND w.reason = 'missing')::int AS missing,
COUNT(*)::int AS albums,
bool_or(sqlc.arg(q)::text <> '' AND strpos(lower(w.title), lower(sqlc.arg(q)::text)) > 0) AS title_match
FROM wants w
WHERE sqlc.arg(artist_mbid)::text = '' OR w.artist_mbid = sqlc.arg(artist_mbid)::text
GROUP BY w.artist_mbid
)
SELECT a.artist_mbid, a.artist_name::text AS artist_name, a.held, a.wanted, a.missing, a.albums,
COALESCE(NULLIF(la.sort_name, ''), a.artist_name)::text AS sort_name,
la.id AS library_artist_id,
(f.artist_mbid IS NOT NULL)::bool AS following,
(SELECT COUNT(*) FROM grabs g JOIN wants x ON x.id = g.want_id
WHERE x.artist_mbid = a.artist_mbid AND g.ended_at IS NULL)::int AS downloading,
COUNT(*) OVER () AS total
FROM a
LEFT JOIN LATERAL (SELECT ar.id, ar.sort_name FROM artists ar WHERE ar.mbid = a.artist_mbid LIMIT 1) la ON true
LEFT JOIN follows f ON f.artist_mbid = a.artist_mbid
WHERE sqlc.arg(q)::text = ''
OR strpos(lower(a.artist_name), lower(sqlc.arg(q)::text)) > 0
OR a.title_match
ORDER BY lower(COALESCE(NULLIF(la.sort_name, ''), a.artist_name)), a.artist_mbid
LIMIT sqlc.arg(max_rows) OFFSET sqlc.arg(skip);
-- name: GetWantAlbum :one
-- What the Wanted view's track list needs of a want.
SELECT id, release_group_mbid, album_id FROM wants WHERE id = $1;
-- name: ListLibraryAlbumTracks :many
-- A held or missing album's own tracks, for the Wanted view's track list;
-- missing says its file is gone.
SELECT title, disc_number, track_number, duration_ms, (missing_since IS NOT NULL)::bool AS missing
FROM tracks
WHERE album_id = $1
ORDER BY disc_number NULLS FIRST, track_number NULLS LAST, title;
-- name: CountWants :one
SELECT COUNT(*) FILTER (WHERE fulfilled_at IS NULL AND reason <> 'missing')::bigint AS wanted,
COUNT(*) FILTER (WHERE fulfilled_at IS NULL AND reason = 'missing')::bigint AS missing,
COUNT(*) FILTER (WHERE fulfilled_at IS NOT NULL)::bigint AS held,
-- Held albums wanted for a better copy.
COUNT(*) FILTER (WHERE fulfilled_at IS NOT NULL AND upgrade <> '')::bigint AS upgrades,
COUNT(DISTINCT artist_mbid)::bigint AS artists,
-- Library albums waiting to be matched by hand.
(SELECT COUNT(*) FROM library_album_matches m
WHERE m.outcome = 'unmatched'
AND EXISTS (SELECT 1 FROM tracks t WHERE t.album_id = m.album_id))::bigint AS unmatched,
(SELECT COUNT(*) FROM follows)::bigint AS following,
-- The Wanted view's other tabs: downloads running, those
-- waiting on a person, and releases set aside.
(SELECT COUNT(*) FROM grabs WHERE ended_at IS NULL)::bigint AS activity,
(SELECT COUNT(*) FROM grabs
WHERE ended_at IS NULL AND import_state IN ('review', 'failed'))::bigint AS attention,
(SELECT COUNT(*) FROM release_blocks b
WHERE b.expires_at IS NULL OR b.expires_at > now())::bigint AS blocked
FROM wants;
-- The one-time listing of the library. ---------------------------
-- name: GetLibraryListing :one
SELECT library_listing_started_at, library_listed_at, library_unlisted
FROM acquisition_settings WHERE id = true;
-- name: StartLibraryListing :one
-- Marks the listing begun, once; its time is the cutoff for which albums
-- count as the library's.
UPDATE acquisition_settings
SET library_listing_started_at = COALESCE(library_listing_started_at, now())
WHERE id = true
RETURNING library_listing_started_at;
-- name: ListLibraryAlbums :execrows
-- Puts library albums on the list as held: each release group once, by the
-- album holding the most tracks. Only albums the library had at the cutoff,
-- and with album_id given only that album. A group already on the list is
-- left as it is. An album with tracks already missing is held from just
-- before the first loss, so re-acquisition still fetches it back.
INSERT INTO wants (release_group_mbid, artist_mbid, artist_name, title, reason, album_id, wanted_at, fulfilled_at)
SELECT DISTINCT ON (a.release_group_mbid)
a.release_group_mbid, ar.mbid, ar.name, a.title, 'library', a.id, now(),
COALESCE((SELECT MIN(t.missing_since) FROM tracks t WHERE t.album_id = a.id) - interval '1 second', now())
FROM albums a
JOIN artists ar ON ar.id = a.artist_id
WHERE a.release_group_mbid IS NOT NULL AND a.release_group_mbid <> ''
AND ar.mbid IS NOT NULL AND ar.mbid <> ''
AND a.created_at <= sqlc.arg(cutoff)::timestamptz
AND (sqlc.narg(album_id)::uuid IS NULL OR a.id = sqlc.narg(album_id)::uuid)
AND EXISTS (SELECT 1 FROM tracks t WHERE t.album_id = a.id)
ORDER BY a.release_group_mbid,
(SELECT COUNT(*) FROM tracks t WHERE t.album_id = a.id AND t.missing_since IS NULL) DESC, a.id
ON CONFLICT (release_group_mbid) DO NOTHING;
-- name: RecordListedAlbums :exec
-- The albums ListLibraryAlbums takes are decided as listed by their tags,
-- so the matching that follows leaves them alone.
INSERT INTO library_album_matches (album_id, outcome, method, release_group_mbid)
SELECT a.id, 'listed', 'tags', a.release_group_mbid
FROM albums a
JOIN artists ar ON ar.id = a.artist_id
WHERE a.release_group_mbid IS NOT NULL AND a.release_group_mbid <> ''
AND ar.mbid IS NOT NULL AND ar.mbid <> ''
AND a.created_at <= sqlc.arg(cutoff)::timestamptz
AND EXISTS (SELECT 1 FROM tracks t WHERE t.album_id = a.id)
ON CONFLICT (album_id) DO NOTHING;
-- name: ListAlbumsToMatch :many
-- Library albums the listing has not decided, or that an older matcher
-- refused, or whose tracks changed since they were refused: each is matched
-- to MusicBrainz. A library album is one from before the cutoff, or
-- one whose tracks were: a re-tag files a track under a new album row but
-- keeps the track. An album row with no tracks is not an album in
-- the library; one a person left off stays off.
SELECT a.id, a.title, a.mbid, a.release_group_mbid, ar.name AS artist_name, ar.mbid AS artist_mbid
FROM albums a
JOIN artists ar ON ar.id = a.artist_id
WHERE (a.created_at <= sqlc.arg(cutoff)::timestamptz
OR EXISTS (SELECT 1 FROM tracks t WHERE t.album_id = a.id AND t.added_at <= sqlc.arg(cutoff)::timestamptz))
AND EXISTS (SELECT 1 FROM tracks t WHERE t.album_id = a.id)
AND NOT EXISTS (SELECT 1 FROM library_album_matches m
WHERE m.album_id = a.id
AND (m.outcome IN ('listed', 'dismissed')
OR (m.matcher >= sqlc.arg(matcher)::int
AND NOT EXISTS (SELECT 1 FROM tracks t
WHERE t.album_id = a.id AND t.updated_at > m.decided_at))))
ORDER BY a.created_at, a.id
LIMIT sqlc.arg(max_rows);
-- name: CountAlbumsToMatch :one
-- Whether a finished listing has albums to go back to: those
-- ListAlbumsToMatch would hand it.
SELECT COUNT(*)
FROM albums a
WHERE (a.created_at <= sqlc.arg(cutoff)::timestamptz
OR EXISTS (SELECT 1 FROM tracks t WHERE t.album_id = a.id AND t.added_at <= sqlc.arg(cutoff)::timestamptz))
AND EXISTS (SELECT 1 FROM tracks t WHERE t.album_id = a.id)
AND NOT EXISTS (SELECT 1 FROM library_album_matches m
WHERE m.album_id = a.id
AND (m.outcome IN ('listed', 'dismissed')
OR (m.matcher >= sqlc.arg(matcher)::int
AND NOT EXISTS (SELECT 1 FROM tracks t
WHERE t.album_id = a.id AND t.updated_at > m.decided_at))));
-- name: ListAlbumTracksToMatch :many
-- What an album's tracks say about themselves, to check a match against.
SELECT title, disc_number, track_number, duration_ms, mbid
FROM tracks
WHERE album_id = $1
ORDER BY disc_number NULLS FIRST, track_number NULLS LAST, title;
-- name: SetAlbumReleaseGroup :exec
-- Names a matched album's release group, filling only what nothing named.
UPDATE albums SET release_group_mbid = sqlc.arg(release_group_mbid), updated_at = now()
WHERE id = sqlc.arg(id) AND (release_group_mbid IS NULL OR release_group_mbid = '');
-- name: InsertLibraryWant :execrows
-- Puts one matched album on the list as held, from just before its first
-- lost track if it has one. A group already on the list is left as it is.
INSERT INTO wants (release_group_mbid, artist_mbid, artist_name, title, primary_type, first_release_date,
reason, album_id, wanted_at, fulfilled_at)
VALUES (sqlc.arg(release_group_mbid), sqlc.arg(artist_mbid), sqlc.arg(artist_name), sqlc.arg(title),
sqlc.arg(primary_type), sqlc.arg(first_release_date), 'library', sqlc.arg(album_id)::uuid, now(),
COALESCE((SELECT MIN(t.missing_since) FROM tracks t WHERE t.album_id = sqlc.arg(album_id)::uuid)
- interval '1 second', now()))
ON CONFLICT (release_group_mbid) DO NOTHING;
-- name: RecordAlbumMatch :exec
INSERT INTO library_album_matches (album_id, outcome, method, release_group_mbid, detail, matcher)
VALUES (sqlc.arg(album_id), sqlc.arg(outcome), sqlc.arg(method), sqlc.arg(release_group_mbid), sqlc.arg(detail),
sqlc.arg(matcher))
ON CONFLICT (album_id) DO UPDATE
SET outcome = EXCLUDED.outcome, method = EXCLUDED.method, release_group_mbid = EXCLUDED.release_group_mbid,
detail = EXCLUDED.detail, matcher = EXCLUDED.matcher, decided_at = now();
-- name: FinishLibraryListing :one
-- The listing is done, or done again after a newer matcher's retries:
-- count the albums no match was found for. The first finish is kept.
UPDATE acquisition_settings
SET library_listed_at = COALESCE(library_listed_at, now()),
library_unlisted = (SELECT COUNT(*) FROM library_album_matches m
WHERE m.outcome = 'unmatched'
AND EXISTS (SELECT 1 FROM tracks t WHERE t.album_id = m.album_id))
WHERE id = true
RETURNING library_listed_at, library_unlisted;
-- name: ListUnmatchedLibraryAlbums :many
-- The Collection's Unmatched section: the albums the listing could
-- not match, and below them the ones a person left off. An album whose
-- tracks have all moved to another row is gone, not unmatched.
SELECT m.album_id, a.title, ar.name AS artist_name, m.detail, m.outcome, m.decided_at,
(SELECT COUNT(*) FROM tracks t WHERE t.album_id = a.id AND t.missing_since IS NULL)::int AS tracks
FROM library_album_matches m
JOIN albums a ON a.id = m.album_id
JOIN artists ar ON ar.id = a.artist_id
WHERE m.outcome IN ('unmatched', 'dismissed')
AND EXISTS (SELECT 1 FROM tracks t WHERE t.album_id = m.album_id)
ORDER BY m.outcome DESC, lower(ar.name), lower(a.title)
LIMIT sqlc.arg(max_rows);
-- name: RecountLibraryUnlisted :exec
-- After a person matches or leaves off an album, the count the catalogue
-- shows.
UPDATE acquisition_settings
SET library_unlisted = (SELECT COUNT(*) FROM library_album_matches m
WHERE m.outcome = 'unmatched'
AND EXISTS (SELECT 1 FROM tracks t WHERE t.album_id = m.album_id))
WHERE id = true;
-- name: GetAlbumToMatch :one
-- One album a person is matching by hand, as the matcher reads albums.
SELECT a.id, a.title, a.mbid, a.release_group_mbid, ar.name AS artist_name, ar.mbid AS artist_mbid
FROM albums a
JOIN artists ar ON ar.id = a.artist_id
JOIN library_album_matches m ON m.album_id = a.id AND m.outcome IN ('unmatched', 'dismissed')
WHERE a.id = $1;
-- name: SetAlbumMatchOutcome :execrows
-- Leave an unmatched album off the list, or put a left-off one back.
UPDATE library_album_matches SET outcome = sqlc.arg(outcome), decided_at = now()
WHERE album_id = sqlc.arg(album_id) AND outcome = sqlc.arg(was);
-- Upgrades. -------------------------------------------------------
-- name: ListHeldWantFormats :many
-- What each held album's copy is made of, for judging whether it is worth
-- upgrading: its present tracks grouped by format, with their bytes and
-- length summed, and how many are held back as video rips. A held want whose
-- album has no track with a length yet has no rows.
SELECT w.id AS want_id, w.upgrade, w.held_tier, w.held_quality,
lower(t.file_format)::text AS file_format,
SUM(t.file_size)::bigint AS bytes,
SUM(t.duration_ms)::bigint AS duration_ms,
COUNT(*)::int AS tracks,
COUNT(*) FILTER (WHERE t.source_verdict = 'suspect')::int AS rips
FROM wants w
JOIN tracks t ON t.album_id = w.album_id
WHERE w.fulfilled_at IS NOT NULL
AND t.missing_since IS NULL AND t.duration_ms > 0
GROUP BY w.id, lower(t.file_format)
ORDER BY w.id;
-- name: SetWantUpgrades :exec
-- Writes the upgrade check's verdicts, one row per want.
UPDATE wants w
SET upgrade = v.upgrade,
held_tier = v.held_tier,
held_quality = v.held_quality,
upgrade_checked_at = now(),
updated_at = CASE WHEN w.upgrade <> v.upgrade THEN now() ELSE w.updated_at END
FROM (SELECT unnest(sqlc.arg(ids)::uuid[]) AS id,
unnest(sqlc.arg(upgrades)::text[]) AS upgrade,
unnest(sqlc.arg(held_tiers)::text[]) AS held_tier,
unnest(sqlc.arg(held_qualities)::text[]) AS held_quality) v
WHERE w.id = v.id;
-- name: ClearHeldUpgrades :execrows
-- A held want whose album has no track left to judge is not wanted again.
UPDATE wants SET upgrade = '', held_tier = '', held_quality = '', upgrade_checked_at = now()
WHERE fulfilled_at IS NOT NULL AND upgrade <> ''
AND NOT EXISTS (SELECT 1 FROM tracks t
WHERE t.album_id = wants.album_id AND t.missing_since IS NULL AND t.duration_ms > 0);
-156
View File
@@ -1,156 +0,0 @@
package downloader
import (
"context"
"errors"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
)
func TestPathMap(t *testing.T) {
m := PathMap{Local: "/downloads", Remote: "/data/torrents/"}
for in, want := range map[string]string{
"/downloads": "/data/torrents",
"/downloads/abc/x.mp3": "/data/torrents/abc/x.mp3",
"/downloads2/abc": "/downloads2/abc",
"/music/a": "/music/a",
"": "",
} {
if got := m.ToRemote(in); got != want {
t.Errorf("ToRemote(%q) = %q, want %q", in, got, want)
}
}
if got := m.ToLocal("/data/torrents/abc"); got != "/downloads/abc" {
t.Errorf("ToLocal = %q", got)
}
if (PathMap{Local: "/downloads", Remote: "/downloads/"}).Active() {
t.Error("the same folder under one name is not a mapping")
}
}
// fakeQbit is a qBittorrent WebUI that wants a login, and that expires the
// session once to make the client log in again.
func fakeQbit(t *testing.T, expireOnce bool) (*httptest.Server, *atomic.Int32) {
t.Helper()
var logins atomic.Int32
var expired atomic.Bool
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Referer") == "" {
w.WriteHeader(http.StatusForbidden)
return
}
switch r.URL.Path {
case "/api/v2/auth/login":
_ = r.ParseForm()
if r.Form.Get("password") != "pw" {
_, _ = w.Write([]byte("Fails."))
return
}
logins.Add(1)
http.SetCookie(w, &http.Cookie{Name: "SID", Value: "s", Path: "/"})
_, _ = w.Write([]byte("Ok."))
case "/api/v2/torrents/info":
if _, err := r.Cookie("SID"); err != nil {
w.WriteHeader(http.StatusForbidden)
return
}
if expireOnce && !expired.Swap(true) {
w.WriteHeader(http.StatusForbidden)
return
}
if r.URL.Query().Get("category") != "minstrel" {
t.Errorf("category = %q", r.URL.Query().Get("category"))
}
_, _ = w.Write([]byte(`[{"hash": "ABC", "name": "x", "save_path": "/data/torrents/abc", "progress": 1}]`))
case "/api/v2/torrents/add":
body, _ := io.ReadAll(r.Body)
if !strings.Contains(string(body), "/data/torrents/new") {
t.Errorf("add body has no translated save path: %s", body)
}
_, _ = w.Write([]byte("Ok."))
default:
w.WriteHeader(http.StatusNotFound)
}
}))
t.Cleanup(srv.Close)
return srv, &logins
}
func TestQBittorrent_LogsInAndLogsInAgainWhenTheSessionExpires(t *testing.T) {
srv, logins := fakeQbit(t, true)
d := Mapped(&Client{Config: Config{BaseURL: srv.URL, Username: "u", Password: "pw", Category: "minstrel"}},
PathMap{Local: "/downloads", Remote: "/data/torrents"})
ts, err := d.Torrents(context.Background())
if err != nil {
t.Fatalf("torrents: %v", err)
}
if logins.Load() != 2 {
t.Errorf("logins = %d, want 2 (the second after the 403)", logins.Load())
}
if len(ts) != 1 || ts[0].Hash != "abc" || ts[0].SavePath != "/downloads/abc" || !ts[0].Done() {
t.Errorf("torrents = %+v, want one, hash lowercased, path in Minstrel's terms", ts)
}
if err := d.AddTorrent(context.Background(), Add{Magnet: "magnet:?xt=urn:btih:abc", SavePath: "/downloads/new"}); err != nil {
t.Errorf("add: %v", err)
}
}
func TestQBittorrent_AWrongPasswordIsAuth(t *testing.T) {
srv, _ := fakeQbit(t, false)
c := &Client{Config: Config{BaseURL: srv.URL, Password: "nope"}}
if _, err := c.Torrents(context.Background()); !errors.Is(err, ErrAuth) {
t.Fatalf("err = %v, want ErrAuth", err)
}
}
func TestExchange_MapsDownloadsAndRefusesWhileRestoring(t *testing.T) {
var restoring, sentKey atomic.Bool
restoring.Store(true)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "" {
sentKey.Store(true)
}
pending := "0"
if restoring.Load() {
pending = "3"
}
_, _ = io.WriteString(w, `{"enabled": true, "running": false, "held_reason": "disk full", "restore_pending": `+pending+`, "downloads": [
{"infohash": "ABC", "name": "Album", "save_path": "/dl", "state": "seeding", "progress": 1, "length": 10, "bytes": 10, "category": "minstrel"},
{"infohash": "DEF", "name": "Other", "save_path": "/dl", "state": "fast", "category": "someone-else"},
{"infohash": "123", "name": "Broken", "save_path": "/dl", "failed": true, "fail_reason": "tracker gone", "category": "minstrel"}
]}`)
}))
defer srv.Close()
c := &Exchange{Config: Config{BaseURL: srv.URL, Category: "minstrel"}}
if _, err := c.Torrents(context.Background()); !errors.Is(err, ErrRestoring) {
t.Fatalf("while restoring: err = %v, want ErrRestoring", err)
}
if sentKey.Load() {
t.Error("an Authorization header was sent with no key set")
}
restoring.Store(false)
ts, err := c.Torrents(context.Background())
if err != nil {
t.Fatalf("torrents: %v", err)
}
if len(ts) != 2 {
t.Fatalf("torrents = %+v, want the two in Minstrel's category", ts)
}
if ts[0].Hash != "abc" || ts[0].State != "uploading" || ts[0].ContentPath != "/dl/Album" || !strings.Contains(ts[0].EngineDown, "disk full") {
t.Errorf("first = %+v", ts[0])
}
if !ts[1].Failed() || ts[1].FailReason != "tracker gone" {
t.Errorf("second = %+v, want failed with Exchange's reason", ts[1])
}
}
func TestFor_RefusesAnUnknownProtocol(t *testing.T) {
if _, err := For("transmission", Config{}); err == nil {
t.Fatal("want an error for a client Minstrel cannot drive")
}
}
-197
View File
@@ -1,197 +0,0 @@
// Package downloader drives the download clients Minstrel hands a grab to:
// qBittorrent's WebUI v2, and FabledExchange's native /api/downloads.
//
// Both answer through one Driver, and Torrent is qBittorrent's shape: the
// Exchange driver maps its own downloads onto it, so the grab, the importer
// and the stall watchdog read one kind of record whichever client holds the
// torrent.
package downloader
import (
"context"
"errors"
"fmt"
"time"
)
// Protocols a download client row may name.
const (
KindQBittorrent = "qbittorrent"
KindExchange = "exchange"
)
// ErrUnconfigured is a client with no address.
var ErrUnconfigured = errors.New("download client is not configured")
// ErrUnreachable is a call that could not be made, or a refusal that is not
// about credentials.
var ErrUnreachable = errors.New("download client unreachable")
// ErrAuth is credentials the client refused.
var ErrAuth = errors.New("download client refused the credentials")
// ErrRestoring is an Exchange still bringing its downloads back after a
// restart: its list is incomplete, so nothing may be concluded from it.
var ErrRestoring = errors.New("download client is still restoring its downloads after a restart")
// ErrDuplicate is a torrent the client already holds.
var ErrDuplicate = errors.New("download client already holds this torrent")
// ErrUnsupported is a call this client has no equivalent for.
var ErrUnsupported = errors.New("download client cannot do that")
// UserAgent names Minstrel in the client's logs.
const UserAgent = "Minstrel (self-hosted music server; download client)"
// DefaultTimeout is what an unset Timeout means: a download client answers
// from its own state.
const DefaultTimeout = 15 * time.Second
// Config is one download client.
type Config struct {
BaseURL string
// Username and Password are qBittorrent's. Both may be empty for a WebUI
// that bypasses auth for local addresses.
Username string
Password string
// APIKey is Exchange's, sent as a bearer only when set.
APIKey string
// Category files everything Minstrel adds, so its torrents can be told
// from anything else in the client.
Category string
Timeout time.Duration
}
// Driver is what Minstrel needs from a download client.
type Driver interface {
AddTorrent(ctx context.Context, a Add) error
// Torrents lists this client's torrents in Config.Category, or only the
// given hashes.
Torrents(ctx context.Context, hashes ...string) ([]Torrent, error)
// Connected says whether the client can reach any peer at all.
Connected(ctx context.Context) (bool, error)
Stop(ctx context.Context, hashes ...string) error
Start(ctx context.Context, hashes ...string) error
Reannounce(ctx context.Context, hashes ...string) error
Recheck(ctx context.Context, hashes ...string) error
Delete(ctx context.Context, deleteFiles bool, hashes ...string) error
}
// FileSkipper sets which files of a torrent are downloaded.
type FileSkipper interface {
SkipFiles(ctx context.Context, hash string, indices []int) error
KeepFiles(ctx context.Context, hash string, indices []int) error
}
// File is one file inside a torrent.
type File struct {
Index int
Path string
Size int64
}
// FileLister lists a torrent's files.
type FileLister interface {
Files(ctx context.Context, hash string) ([]File, error)
}
// FileState is one file's progress.
type FileState struct {
Path string
Size int64
Done int64
Skipped bool
}
// Unfinished says whether the file is wanted and not yet whole.
func (f FileState) Unfinished() bool { return !f.Skipped && f.Done < f.Size }
// FileProgress reports per-file progress.
type FileProgress interface {
FileStates(ctx context.Context, hash string) ([]FileState, error)
}
var (
_ Driver = (*Client)(nil)
_ Driver = (*Exchange)(nil)
_ FileSkipper = (*Client)(nil)
_ FileSkipper = (*Exchange)(nil)
_ FileSkipper = (*mapped)(nil)
_ FileLister = (*Client)(nil)
_ FileLister = (*Exchange)(nil)
_ FileLister = (*mapped)(nil)
_ FileProgress = (*Client)(nil)
_ FileProgress = (*Exchange)(nil)
_ FileProgress = (*mapped)(nil)
)
// For builds the driver a protocol names.
func For(protocol string, cfg Config) (Driver, error) {
switch protocol {
case KindQBittorrent:
return &Client{Config: cfg}, nil
case KindExchange:
return &Exchange{Config: cfg}, nil
}
return nil, fmt.Errorf("download client %q is not one Minstrel can drive", protocol)
}
// Torrent is one torrent in qBittorrent's JSON shape.
type Torrent struct {
Hash string `json:"hash"`
Name string `json:"name"`
Size int64 `json:"size"`
Progress float64 `json:"progress"`
State string `json:"state"`
Category string `json:"category"`
Tags string `json:"tags"`
SavePath string `json:"save_path"`
ContentPath string `json:"content_path"`
AddedOn int64 `json:"added_on"`
NumSeeds int `json:"num_seeds"`
NumLeechs int `json:"num_leechs"`
AmountLeft int64 `json:"amount_left"`
Ratio float64 `json:"ratio"`
DLSpeed int64 `json:"dlspeed"`
ETA int64 `json:"eta"`
// EngineDown is why the client's whole engine is not moving anything, in
// words; empty when it is running. Exchange says so; qBittorrent does not.
EngineDown string `json:"-"`
// FailReason is the client's own reason for a failed torrent.
FailReason string `json:"-"`
}
// Done says the torrent is whole.
func (t Torrent) Done() bool { return t.Progress >= 1 }
// Failed says the client gave up on it.
func (t Torrent) Failed() bool { return t.State == "error" || t.State == "missingFiles" }
// Fetching says it is still fetching metadata.
func (t Torrent) Fetching() bool { return t.State == "metaDL" || t.State == "forcedMetaDL" }
// Alone says nobody is sharing an unfinished torrent.
func (t Torrent) Alone() bool { return !t.Done() && t.NumSeeds == 0 && t.NumLeechs == 0 }
// Added is when the client took it.
func (t Torrent) Added() time.Time {
if t.AddedOn <= 0 {
return time.Time{}
}
return time.Unix(t.AddedOn, 0)
}
// Add is one torrent to hand a client: a magnet or the .torrent bytes.
type Add struct {
Magnet string
Torrent []byte
// SavePath is where it lands, as Minstrel sees it; Mapped translates it.
// Required: a download with no save path lands outside the folder
// Minstrel imports from.
SavePath string
Tags []string
Paused bool
// StopAtMetadata asks qBittorrent to stop once it has the file list, so
// unwanted files can be skipped before any data moves.
StopAtMetadata bool
}
-340
View File
@@ -1,340 +0,0 @@
package downloader
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"path"
"strings"
"time"
)
// Exchange drives FabledExchange's native /api/downloads: not its qBittorrent
// facade, so its own states, failure reasons and engine status come through.
type Exchange struct {
Config Config
// HTTP is injectable for tests. Nil takes a plain client.
HTTP *http.Client
}
type exchangeItem struct {
InfoHash string `json:"infohash"`
Name string `json:"name"`
Bytes int64 `json:"bytes"`
Length int64 `json:"length"`
Progress float64 `json:"progress"`
State string `json:"state"`
Peers int `json:"peers"`
Category string `json:"category"`
Tags []string `json:"tags"`
SavePath string `json:"save_path"`
AddedAt time.Time `json:"added_at"`
Failed bool `json:"failed"`
FailReason string `json:"fail_reason"`
Ratio float64 `json:"ratio"`
}
type exchangeSnapshot struct {
Enabled bool `json:"enabled"`
Running bool `json:"running"`
HeldReason string `json:"held_reason"`
InboundConns int64 `json:"inbound_conns"`
RestorePending int `json:"restore_pending"`
Downloads []exchangeItem `json:"downloads"`
}
func (c *Exchange) AddTorrent(ctx context.Context, a Add) error {
if a.Magnet == "" && len(a.Torrent) == 0 {
return fmt.Errorf("add: neither a magnet nor torrent bytes")
}
if a.SavePath == "" {
return fmt.Errorf("add: no save path, which would land the download outside the folder Minstrel imports from")
}
if len(a.Torrent) > 0 {
q := url.Values{"save_path": {a.SavePath}}
if c.Config.Category != "" {
q.Set("category", c.Config.Category)
}
for _, t := range a.Tags {
q.Add("tag", t)
}
if a.Paused {
q.Set("paused", "true")
}
_, err := c.call(ctx, http.MethodPost, "/api/downloads/torrent?"+q.Encode(), "application/x-bittorrent", a.Torrent)
return err
}
body, err := json.Marshal(map[string]any{
"magnet": a.Magnet, "category": c.Config.Category, "save_path": a.SavePath,
"tags": a.Tags, "paused": a.Paused,
})
if err != nil {
return fmt.Errorf("add: %w", err)
}
_, err = c.call(ctx, http.MethodPost, "/api/downloads", "application/json", body)
return err
}
// Torrents maps Exchange's downloads onto qBittorrent's shape. While Exchange
// is still restoring after a restart its list is incomplete, so it answers
// ErrRestoring rather than a list that looks like downloads vanished.
func (c *Exchange) Torrents(ctx context.Context, hashes ...string) ([]Torrent, error) {
snap, err := c.snapshot(ctx)
if err != nil {
return nil, err
}
if snap.RestorePending > 0 {
return nil, fmt.Errorf("%w (%d still to come back)", ErrRestoring, snap.RestorePending)
}
want := map[string]bool{}
for _, h := range hashes {
want[strings.ToLower(strings.TrimSpace(h))] = true
}
down := ""
switch {
case snap.Running:
case !snap.Enabled:
down = "the download engine is switched off in FabledExchange"
case snap.HeldReason != "":
down = "the download engine is held (" + snap.HeldReason + ") and resumes by itself"
default:
down = "the download engine is enabled but not running; FabledExchange is retrying it"
}
out := make([]Torrent, 0, len(snap.Downloads))
for _, it := range snap.Downloads {
if c.Config.Category != "" && it.Category != c.Config.Category {
continue
}
t := asTorrent(it)
t.EngineDown = down
if len(want) > 0 && !want[t.Hash] {
continue
}
out = append(out, t)
}
return out, nil
}
func asTorrent(it exchangeItem) Torrent {
content := it.SavePath
if it.Name != "" {
content = path.Join(it.SavePath, it.Name)
}
left := max(it.Length-it.Bytes, 0)
state := exchangeState(it.State, it.Progress)
if it.Failed {
state = "error"
}
var added int64
if !it.AddedAt.IsZero() {
added = it.AddedAt.Unix()
}
return Torrent{
Hash: strings.ToLower(it.InfoHash), Name: it.Name, Size: it.Length,
Progress: it.Progress, State: state, Category: it.Category,
Tags: strings.Join(it.Tags, ","), SavePath: it.SavePath, ContentPath: content,
AddedOn: added, NumLeechs: it.Peers, AmountLeft: left,
FailReason: it.FailReason, Ratio: it.Ratio,
}
}
// exchangeState names Exchange's state in qBittorrent's vocabulary.
func exchangeState(tier string, progress float64) string {
switch tier {
case "fast", "slow":
return "downloading"
case "metadata":
return "metaDL"
case "stalled":
return "stalledDL"
case "queued":
return "queuedDL"
case "seeding":
return "uploading"
case "relocating":
return "moving"
case "paused":
if progress >= 1 {
return "pausedUP"
}
return "pausedDL"
}
return "unknown"
}
// Connected is a running engine that has had an inbound connection: the
// nearest Exchange comes to qBittorrent's "connected".
func (c *Exchange) Connected(ctx context.Context) (bool, error) {
snap, err := c.snapshot(ctx)
if err != nil {
return false, err
}
return snap.Running && snap.InboundConns > 0, nil
}
func (c *Exchange) Stop(ctx context.Context, hashes ...string) error {
return c.each(ctx, hashes, func(h string) error {
_, err := c.call(ctx, http.MethodPost, "/api/downloads/"+url.PathEscape(h)+"/pause", "", nil)
return err
})
}
func (c *Exchange) Start(ctx context.Context, hashes ...string) error {
return c.each(ctx, hashes, func(h string) error {
_, err := c.call(ctx, http.MethodPost, "/api/downloads/"+url.PathEscape(h)+"/resume", "", nil)
return err
})
}
func (c *Exchange) SkipFiles(ctx context.Context, hash string, indices []int) error {
return c.filePriority(ctx, hash, indices, "skip")
}
func (c *Exchange) KeepFiles(ctx context.Context, hash string, indices []int) error {
return c.filePriority(ctx, hash, indices, "normal")
}
func (c *Exchange) filePriority(ctx context.Context, hash string, indices []int, priority string) error {
if len(indices) == 0 {
return nil
}
body, err := json.Marshal(map[string]any{"indices": indices, "priority": priority})
if err != nil {
return err
}
_, err = c.call(ctx, http.MethodPatch, "/api/downloads/"+url.PathEscape(hash)+"/files", "application/json", body)
return err
}
func (c *Exchange) Files(ctx context.Context, hash string) ([]File, error) {
raw, err := c.files(ctx, hash)
if err != nil {
return nil, err
}
out := make([]File, len(raw))
for i, f := range raw {
out[i] = File{Index: f.Index, Path: f.Path, Size: f.Length}
}
return out, nil
}
func (c *Exchange) FileStates(ctx context.Context, hash string) ([]FileState, error) {
raw, err := c.files(ctx, hash)
if err != nil {
return nil, err
}
out := make([]FileState, len(raw))
for i, f := range raw {
if f.Bytes == nil {
return nil, fmt.Errorf("%w: FabledExchange does not say how much of each file it has", ErrUnsupported)
}
out[i] = FileState{Path: f.Path, Size: f.Length, Done: *f.Bytes, Skipped: f.Priority == "skip"}
}
return out, nil
}
type exchangeFile struct {
Index int `json:"index"`
Path string `json:"path"`
Length int64 `json:"length"`
Bytes *int64 `json:"bytes"`
Priority string `json:"priority"`
}
func (c *Exchange) files(ctx context.Context, hash string) ([]exchangeFile, error) {
body, err := c.call(ctx, http.MethodGet, "/api/downloads/"+url.PathEscape(hash), "", nil)
if err != nil {
return nil, err
}
var detail struct {
Files []exchangeFile `json:"files"`
}
if err := json.Unmarshal(body, &detail); err != nil {
return nil, fmt.Errorf("%w: parse download detail: %v", ErrUnreachable, err)
}
return detail.Files, nil
}
func (c *Exchange) Reannounce(context.Context, ...string) error {
return fmt.Errorf("%w: FabledExchange schedules its own announces", ErrUnsupported)
}
func (c *Exchange) Recheck(context.Context, ...string) error {
return fmt.Errorf("%w: FabledExchange has no recheck", ErrUnsupported)
}
func (c *Exchange) Delete(ctx context.Context, deleteFiles bool, hashes ...string) error {
return c.each(ctx, hashes, func(h string) error {
p := "/api/downloads/" + url.PathEscape(h)
if deleteFiles {
p += "?delete_files=true"
}
_, err := c.call(ctx, http.MethodDelete, p, "", nil)
return err
})
}
func (c *Exchange) each(ctx context.Context, hashes []string, do func(string) error) error {
for _, h := range hashes {
if ctx.Err() != nil {
return ctx.Err()
}
if err := do(strings.ToLower(strings.TrimSpace(h))); err != nil {
return err
}
}
return nil
}
func (c *Exchange) snapshot(ctx context.Context) (exchangeSnapshot, error) {
var snap exchangeSnapshot
body, err := c.call(ctx, http.MethodGet, "/api/downloads", "", nil)
if err != nil {
return snap, err
}
if err := json.Unmarshal(body, &snap); err != nil {
return snap, fmt.Errorf("%w: parse download list: %v", ErrUnreachable, err)
}
return snap, nil
}
func (c *Exchange) call(ctx context.Context, method, p, contentType string, body []byte) ([]byte, error) {
if c.Config.BaseURL == "" {
return nil, ErrUnconfigured
}
httpc := c.HTTP
if httpc == nil {
httpc = &http.Client{}
}
status, out, err := send(ctx, httpc, c.Config.Timeout, method,
strings.TrimRight(c.Config.BaseURL, "/")+p, contentType, body, func(h http.Header) {
h.Set("Accept", "application/json")
if c.Config.APIKey != "" {
h.Set("Authorization", "Bearer "+c.Config.APIKey)
}
})
if err != nil {
return nil, err
}
if status >= 200 && status < 300 {
return out, nil
}
var e struct {
Error string `json:"error"`
Reason string `json:"reason"`
}
_ = json.Unmarshal(out, &e)
said := strings.TrimSpace(e.Error)
if said == "" {
said = fmt.Sprintf("HTTP %d", status)
}
switch {
case status == http.StatusUnauthorized || status == http.StatusForbidden:
return nil, fmt.Errorf("%w: FabledExchange refused the API key (%s)", ErrAuth, said)
case status == http.StatusConflict && e.Reason == "duplicate":
return nil, fmt.Errorf("%w: %s", ErrDuplicate, said)
}
return nil, fmt.Errorf("%w: %s", ErrUnreachable, said)
}

Some files were not shown because too many files have changed in this diff Show More